From patchwork Thu May 26 21:19:52 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolai Stange X-Patchwork-Id: 9137427 X-Patchwork-Delegate: herbert@gondor.apana.org.au Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id E63F7602A7 for ; Thu, 26 May 2016 21:21:27 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id D83B1282F6 for ; Thu, 26 May 2016 21:21:27 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id CD44A282FA; Thu, 26 May 2016 21:21:27 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.8 required=2.0 tests=BAYES_00, DKIM_ADSP_CUSTOM_MED, DKIM_SIGNED, FREEMAIL_FROM, RCVD_IN_DNSWL_HI, T_DKIM_INVALID autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 81352282F6 for ; Thu, 26 May 2016 21:21:27 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1755383AbcEZVVH (ORCPT ); Thu, 26 May 2016 17:21:07 -0400 Received: from mail-wm0-f68.google.com ([74.125.82.68]:33162 "EHLO mail-wm0-f68.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1755426AbcEZVUL (ORCPT ); Thu, 26 May 2016 17:20:11 -0400 Received: by mail-wm0-f68.google.com with SMTP id a136so9053853wme.0; Thu, 26 May 2016 14:20:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=AqKtRCrASu/pzowpG29GdVhPDYbZLzkBo6DE1AoBbTo=; b=hDCkKT0rounKoVV1nj3ZbHVg+BbXkdaKF6p7NYjmg/rlyf2yRloGRnBKHRvK6J7FTa 3oKVc2XsA8DaYxVR5HMW4hNZJwMRFU7oPrOJVxQX+hbgDUQgj08zvkSG/uwxTS3gbSxi /PrspP0DwJVjlkf5SuI4fyTRgwHym5wBS+byD0S+j2YePVEvP/vcCcyKA3Ucboy5aeB5 8xoIM0DK4ux82x+TcqBnJPGLa86+V3YyeJqBrGJvb0UOXB5ssdfqv1Y588/+dJiX5xAe /QUR16ocL42lJ+7eK6kO4eHZP406Kn9SMB8NBz9tXVR9oiRZvrZ6SqyR9X8RccjyNJiZ CHRA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=AqKtRCrASu/pzowpG29GdVhPDYbZLzkBo6DE1AoBbTo=; b=FpGgbAelR5vROaILHzb+9KTG4zWzEhPtju613xAwA+A2KY1TK2Nzs+HlqvCvlJbls0 QRZAFpom9ESzv4PhTKhLD+GabGu+2KahNofUCWgqFF0PCyy5lgXEcq8uZApgyv82pg5G X2x+yzCVymqfJVgMBba6vAcoQp2b4R5TojDj2Ytr4I/LPUiwjI5H9yztCW45CESJIJc7 +JfWBiMkF6RpyoCIAAy6fqqJLWfaRPAileDvVRF8LlVUUI0HtyxFlfugrxmjjg4jzKaU xIz1G184m/uozepXUUD/jlpk5kgJafeAz2QHN7OtmWPcsoNo8HYVgZuDjZF3ZOD+ACSJ ORQA== X-Gm-Message-State: ALyK8tLJNXzMzQYrbmQqasaDS0+xfWOhJi2XmiWL+FJkRBOxenGuEAqy3Xa+/ZXkwMgNxw== X-Received: by 10.28.105.17 with SMTP id e17mr5850465wmc.75.1464297610025; Thu, 26 May 2016 14:20:10 -0700 (PDT) Received: from localhost (x55b166e2.dyn.telefonica.de. [85.177.102.226]) by smtp.gmail.com with ESMTPSA id az2sm16160206wjc.6.2016.05.26.14.20.09 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 26 May 2016 14:20:09 -0700 (PDT) From: Nicolai Stange To: Herbert Xu Cc: David Howells , Tadeusz Struk , Michal Marek , linux-crypto@vger.kernel.org, linux-kernel@vger.kernel.org, Nicolai Stange Subject: [PATCH 2/5] lib/digsig: digsig_verify_rsa(): return -EINVAL if modulo length is zero Date: Thu, 26 May 2016 23:19:52 +0200 Message-Id: <1464297595-24032-3-git-send-email-nicstange@gmail.com> X-Mailer: git-send-email 2.8.2 In-Reply-To: <1464297595-24032-1-git-send-email-nicstange@gmail.com> References: <1464297595-24032-1-git-send-email-nicstange@gmail.com> Sender: linux-crypto-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-crypto@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Currently, if digsig_verify_rsa() detects that the modulo's length is zero, i.e. mlen == 0, it returns -ENOMEM which doesn't really fit here. Make digsig_verify_rsa() return -EINVAL upon mlen == 0. Signed-off-by: Nicolai Stange --- lib/digsig.c | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/lib/digsig.c b/lib/digsig.c index a121cbc..55b8b2f 100644 --- a/lib/digsig.c +++ b/lib/digsig.c @@ -114,13 +114,15 @@ static int digsig_verify_rsa(struct key *key, datap += remaining; } - err = -ENOMEM; - mblen = mpi_get_nbits(pkey[0]); mlen = DIV_ROUND_UP(mblen, 8); - if (mlen == 0) + if (mlen == 0) { + err = -EINVAL; goto err; + } + + err = -ENOMEM; out1 = kzalloc(mlen, GFP_KERNEL); if (!out1)