From patchwork Wed Nov 21 08:05:00 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Selvin Xavier X-Patchwork-Id: 10692087 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id E585314BD for ; Wed, 21 Nov 2018 08:05:41 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id DABFD2B85C for ; Wed, 21 Nov 2018 08:05:41 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id CF4DF2B86D; Wed, 21 Nov 2018 08:05:41 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id BC0412B85C for ; Wed, 21 Nov 2018 08:05:40 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728262AbeKUSjJ (ORCPT ); Wed, 21 Nov 2018 13:39:09 -0500 Received: from mail-wr1-f65.google.com ([209.85.221.65]:42329 "EHLO mail-wr1-f65.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725999AbeKUSjJ (ORCPT ); Wed, 21 Nov 2018 13:39:09 -0500 Received: by mail-wr1-f65.google.com with SMTP id q18so4587024wrx.9 for ; Wed, 21 Nov 2018 00:05:38 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=broadcom.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=rlKDWcl+ti0+8+LsCFka1b5gj3TN2ICH5VyTFluxAS8=; b=ePd+YN46t4zMHKvwiMwQFn/KoMz5I4Lih+05AZ/OgsDc381zjhXvbTHxmTbsXpv8vO cwrHnz4eIeDllE9UV3g1jTJIrwZBT9t1QXdeINC69qSP7RJr2LaWuniujIqP29RZXvjo 7knZ/zAXY20gVokbEma0rnCJKg9omckogiI9c= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=rlKDWcl+ti0+8+LsCFka1b5gj3TN2ICH5VyTFluxAS8=; b=g/Xutns9diA4E4KJXDVvTM58jtjpUMkdCXdPrCePD54U8aGv6ayZJzrHYh7ZgURbS4 WmNims98YcLcrbqeseR+7Vx4Qwc/zWTM8B1pr21Qj/5aneejklHP3XmcGv1Wf0PlNB+h CwpketpvBSGChkH2GEozozZBwdIZt4AzAL8mZoJyhkrBOvlP4VpKYfuYs/m49iR6YEJY K+hHe0+NDrUSIX7RTLE+X0WkZnS/3x43Nk0CEVDv6WuYrDHJjJnvoqea9NhjgN9xh2uA fCtU+HPgjOzVN4sntHPPOXtukejSf4mUWEKgE/uGjknVo44PYyUufIuOEOtciv9QtkiG FJ7A== X-Gm-Message-State: AA+aEWZcNTlO1GaozN6iZp+PictSoI8xvtWwLy/gud+9kwRbGtaLqpco QaRwxJIMrawaCk4Sz3KWanXLcQ== X-Google-Smtp-Source: AFSGD/VUAtbzSYUjMh3Rd0hIMPW79P6XvINbP52L5kjLGyhLy1czV/wgelu5mxG6aqLm9FGhlKHSTA== X-Received: by 2002:a5d:454f:: with SMTP id p15mr5116913wrr.39.1542787538074; Wed, 21 Nov 2018 00:05:38 -0800 (PST) Received: from dhcp-10-192-206-197.dhcp.broadcom.net ([192.19.234.250]) by smtp.gmail.com with ESMTPSA id x79sm451746wmd.42.2018.11.21.00.05.34 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 21 Nov 2018 00:05:37 -0800 (PST) From: Selvin Xavier To: dledford@redhat.com, jgg@mellanox.com Cc: linux-rdma@vger.kernel.org, michael.chan@broadcom.com, Selvin Xavier Subject: [PATCH for-rc 1/2] RDMA/bnxt_re: Fix system hang when registration with L2 driver fails Date: Wed, 21 Nov 2018 00:05:00 -0800 Message-Id: <1542787501-1501-2-git-send-email-selvin.xavier@broadcom.com> X-Mailer: git-send-email 2.5.5 In-Reply-To: <1542787501-1501-1-git-send-email-selvin.xavier@broadcom.com> References: <1542787501-1501-1-git-send-email-selvin.xavier@broadcom.com> Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Driver doesn't release rtnl lock if registration with L2 driver (bnxt_re_register_netdev) fais and this causes hang while requesting for the next lock. [ 371.635416] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [ 371.635417] kworker/u48:1 D 0 634 2 0x80000000 [ 371.635423] Workqueue: bnxt_re bnxt_re_task [bnxt_re] [ 371.635424] Call Trace: [ 371.635426] ? __schedule+0x36b/0xbd0 [ 371.635429] schedule+0x39/0x90 [ 371.635430] schedule_preempt_disabled+0x11/0x20 [ 371.635431] __mutex_lock+0x45b/0x9c0 [ 371.635433] ? __mutex_lock+0x16d/0x9c0 [ 371.635435] ? bnxt_re_ib_reg+0x2b/0xb30 [bnxt_re] [ 371.635438] ? wake_up_klogd+0x37/0x40 [ 371.635442] bnxt_re_ib_reg+0x2b/0xb30 [bnxt_re] [ 371.635447] bnxt_re_task+0xfd/0x180 [bnxt_re] [ 371.635449] process_one_work+0x216/0x5b0 [ 371.635450] ? process_one_work+0x189/0x5b0 [ 371.635453] worker_thread+0x4e/0x3d0 [ 371.635455] kthread+0x10e/0x140 [ 371.635456] ? process_one_work+0x5b0/0x5b0 [ 371.635458] ? kthread_stop+0x220/0x220 [ 371.635460] ret_from_fork+0x3a/0x50 [ 371.635477] INFO: task NetworkManager:1228 blocked for more than 120 seconds. [ 371.635478] Tainted: G B OE 4.20.0-rc1+ #42 [ 371.635479] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. Release the rtnl_lock correctly in the failure path. Fixes: de5c95d0f518 ("RDMA/bnxt_re: Fix system crash during RDMA resource initialization") Signed-off-by: Selvin Xavier --- drivers/infiniband/hw/bnxt_re/main.c | 1 + 1 file changed, 1 insertion(+) diff --git a/drivers/infiniband/hw/bnxt_re/main.c b/drivers/infiniband/hw/bnxt_re/main.c index cf22826..bd5ded5 100644 --- a/drivers/infiniband/hw/bnxt_re/main.c +++ b/drivers/infiniband/hw/bnxt_re/main.c @@ -1268,6 +1268,7 @@ static int bnxt_re_ib_reg(struct bnxt_re_dev *rdev) /* Registered a new RoCE device instance to netdev */ rc = bnxt_re_register_netdev(rdev); if (rc) { + rtnl_unlock(); pr_err("Failed to register with netedev: %#x\n", rc); return -EINVAL; } From patchwork Wed Nov 21 08:05:01 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Selvin Xavier X-Patchwork-Id: 10692089 X-Patchwork-Delegate: jgg@ziepe.ca Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 31A5814BD for ; Wed, 21 Nov 2018 08:05:45 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 251AB2B870 for ; Wed, 21 Nov 2018 08:05:45 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 193DA2B87B; Wed, 21 Nov 2018 08:05:45 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id ADDF62B870 for ; Wed, 21 Nov 2018 08:05:44 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728260AbeKUSjN (ORCPT ); Wed, 21 Nov 2018 13:39:13 -0500 Received: from mail-wr1-f67.google.com ([209.85.221.67]:39240 "EHLO mail-wr1-f67.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725999AbeKUSjN (ORCPT ); Wed, 21 Nov 2018 13:39:13 -0500 Received: by mail-wr1-f67.google.com with SMTP id b13so4611530wrx.6 for ; Wed, 21 Nov 2018 00:05:42 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=broadcom.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=BPBXRke4JDfw3zCR4yVZOfgFjNETfauJYd+hDytklEQ=; b=A206uz/tbEBvDIeVdTaSktZzquzb4oTMjNnJG1tT3P3SX59W8dTf+atBRDIlQipWVp v+eZM5x32w15lVUbr2ma/Djibgzb9OvUvlBkl4pyqq71BIh1MXcIwAZ2DG5Lby01aedD SuwLgipvr2TIIn0QIbOizVbCffufdzKUdfFfw= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=BPBXRke4JDfw3zCR4yVZOfgFjNETfauJYd+hDytklEQ=; b=XAGeFMdP+9BRa5UsvB60qi9WYUzkMLDcIspzCrKTeeAu6wAua8OlacUWwvafx3dhor 7HzYIjWe8k2zOLTBbMd6Pa8HmayBLrm/1mqIkNyIqGDZg7Vl3G9jvfFPSrMeyVgTpcZa tgYJYMBfbveV7m+XXZocqQCLG3/C44JO4kEhe6w740/U5OSsAm1csLCaoUshLRouFCIY 9eYAQsgNAEkSC/1C5KIqaX+5lvsF3c7fSEg4RKSAciq7OHClqfOF1aiq3xlJ1+tbEiL1 JjIXzAM98TcwpaINoOpGP/3G0hoCd/1Qke1szTxCYbQKFMkrxXVwfrrRKzG3kHpkzvFr TFDA== X-Gm-Message-State: AA+aEWZ2s7Aace+h5C0/Vd1OA1NgqZvqcdiEm5z/5S7aAt0bxqZPJpHX mfVeOYoaUMQ8ylXOuZ7NRl5GcA== X-Google-Smtp-Source: AFSGD/V+4BuYtqMKMsS4/YhZn7vNziIr1eDR4AMG0dc2MVeREmBaNWtgsopC8kycB64brnRAkJXaeQ== X-Received: by 2002:adf:9f10:: with SMTP id l16mr4861684wrf.206.1542787541754; Wed, 21 Nov 2018 00:05:41 -0800 (PST) Received: from dhcp-10-192-206-197.dhcp.broadcom.net ([192.19.234.250]) by smtp.gmail.com with ESMTPSA id x79sm451746wmd.42.2018.11.21.00.05.38 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 21 Nov 2018 00:05:41 -0800 (PST) From: Selvin Xavier To: dledford@redhat.com, jgg@mellanox.com Cc: linux-rdma@vger.kernel.org, michael.chan@broadcom.com, Selvin Xavier Subject: [PATCH for-rc 2/2] RDMA/bnxt_re: Avoid accessing the device structure after it is freed Date: Wed, 21 Nov 2018 00:05:01 -0800 Message-Id: <1542787501-1501-3-git-send-email-selvin.xavier@broadcom.com> X-Mailer: git-send-email 2.5.5 In-Reply-To: <1542787501-1501-1-git-send-email-selvin.xavier@broadcom.com> References: <1542787501-1501-1-git-send-email-selvin.xavier@broadcom.com> Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP When bnxt_re_ib_reg returns failure, the device structure gets freed. Driver tries to access the device pointer after it is freed. [ 4871.034744] Failed to register with netedev: 0xffffffa1 [ 4871.034765] infiniband (null): Failed to register with IB: 0xffffffea [ 4871.046430] ================================================================== [ 4871.046437] BUG: KASAN: use-after-free in bnxt_re_task+0x63/0x180 [bnxt_re] [ 4871.046439] Write of size 4 at addr ffff880fa8406f48 by task kworker/u48:2/17813 [ 4871.046443] CPU: 20 PID: 17813 Comm: kworker/u48:2 Kdump: loaded Tainted: G B OE 4.20.0-rc1+ #42 [ 4871.046444] Hardware name: Dell Inc. PowerEdge R730/0599V5, BIOS 1.0.4 08/28/2014 [ 4871.046447] Workqueue: bnxt_re bnxt_re_task [bnxt_re] [ 4871.046449] Call Trace: [ 4871.046454] dump_stack+0x91/0xeb [ 4871.046458] print_address_description+0x6a/0x2a0 [ 4871.046461] kasan_report+0x176/0x2d0 [ 4871.046463] ? bnxt_re_task+0x63/0x180 [bnxt_re] [ 4871.046466] bnxt_re_task+0x63/0x180 [bnxt_re] [ 4871.046470] process_one_work+0x216/0x5b0 [ 4871.046471] ? process_one_work+0x189/0x5b0 [ 4871.046475] worker_thread+0x4e/0x3d0 [ 4871.046479] kthread+0x10e/0x140 [ 4871.046480] ? process_one_work+0x5b0/0x5b0 [ 4871.046482] ? kthread_stop+0x220/0x220 [ 4871.046486] ret_from_fork+0x3a/0x50 [ 4871.046492] The buggy address belongs to the page: [ 4871.046494] page:ffffea003ea10180 count:0 mapcount:0 mapping:0000000000000000 index:0x0 [ 4871.046495] flags: 0x57ffffc0000000() [ 4871.046498] raw: 0057ffffc0000000 0000000000000000 ffffea003ea10188 0000000000000000 [ 4871.046500] raw: 0000000000000000 0000000000000000 00000000ffffffff 0000000000000000 [ 4871.046501] page dumped because: kasan: bad access detected Avoid accessing the device structure once it is freed. Fixes: 497158aa5f52 ("RDMA/bnxt_re: Fix the ib_reg failure cleanup") Signed-off-by: Selvin Xavier --- drivers/infiniband/hw/bnxt_re/main.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/drivers/infiniband/hw/bnxt_re/main.c b/drivers/infiniband/hw/bnxt_re/main.c index bd5ded5..77f095e 100644 --- a/drivers/infiniband/hw/bnxt_re/main.c +++ b/drivers/infiniband/hw/bnxt_re/main.c @@ -1467,6 +1467,7 @@ static void bnxt_re_task(struct work_struct *work) "Failed to register with IB: %#x", rc); bnxt_re_remove_one(rdev); bnxt_re_dev_unreg(rdev); + goto exit; } break; case NETDEV_UP: @@ -1490,6 +1491,7 @@ static void bnxt_re_task(struct work_struct *work) } smp_mb__before_atomic(); atomic_dec(&rdev->sched_count); +exit: kfree(re_work); }