From patchwork Mon Mar 7 05:31:32 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Devesh Sharma X-Patchwork-Id: 8515661 Return-Path: X-Original-To: patchwork-linux-rdma@patchwork.kernel.org Delivered-To: patchwork-parsemail@patchwork1.web.kernel.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.136]) by patchwork1.web.kernel.org (Postfix) with ESMTP id 8F9D19F46A for ; Mon, 7 Mar 2016 05:31:46 +0000 (UTC) Received: from mail.kernel.org (localhost [127.0.0.1]) by mail.kernel.org (Postfix) with ESMTP id 80DFC20120 for ; Mon, 7 Mar 2016 05:31:45 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 717F32010E for ; Mon, 7 Mar 2016 05:31:44 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751958AbcCGFbn (ORCPT ); Mon, 7 Mar 2016 00:31:43 -0500 Received: from cmrelayp1.emulex.com ([138.239.112.140]:55602 "EHLO CMRELAYP1.ad.emulex.com" rhost-flags-OK-FAIL-OK-FAIL) by vger.kernel.org with ESMTP id S1751938AbcCGFbm (ORCPT ); Mon, 7 Mar 2016 00:31:42 -0500 Received: from neo01-el71.iig.avagotech.net ([10.192.204.61]) by CMRELAYP1.ad.emulex.com with Microsoft SMTPSVC(7.5.7601.17514); Sun, 6 Mar 2016 21:31:50 -0800 From: Devesh Sharma To: linux-rdma@vger.kernel.org Cc: Devesh Sharma , Yishai Hadas Subject: [PATCH] RDMA/uverbs: Fix race between uverbs_close and remove_one Date: Mon, 7 Mar 2016 00:31:32 -0500 Message-Id: <1457328692-884-1-git-send-email-devesh.sharma@broadcom.com> X-Mailer: git-send-email 1.8.3.1 X-OriginalArrivalTime: 07 Mar 2016 05:31:50.0906 (UTC) FILETIME=[A696E5A0:01D17832] Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org X-Spam-Status: No, score=-6.9 required=5.0 tests=BAYES_00, RCVD_IN_DNSWL_HI, RP_MATCHES_RCVD, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on mail.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Fixes: 35d4a0b63dc0c6d1177d4f532a9deae958f0662c ("IB/uverbs: Fix race between ib_uverbs_open and remove_one") While testing ocrdma for disassociate_ucontext support following kernel panic was seen: BUG: unable to handle kernel paging request at ffffffffa07ccd7a [67139.981020] IP: [] 0xffffffffa07ccd7a [67139.987185] PGD 19c5067 PUD 19c6063 PMD 469d08067 PTE 0 [67139.993370] Oops: 0010 [#1] SMP [67140.257286] Call Trace: [67140.260665] [] ? prepare_to_wait_event+0xf0/0xf0 [67140.268337] [] ? ib_dereg_mr+0x23/0x30 [ib_core] [67140.276009] [] ? ib_uverbs_cleanup_ucontext+0x320/0x440 [ib_uverbs] [67140.285550] [] ? ib_uverbs_close+0x59/0xb0 [ib_uverbs] [67140.293807] [] ? __fput+0xe4/0x210 [67140.300132] [] ? ____fput+0xe/0x10 [67140.306457] [] ? task_work_run+0x77/0x90 [67140.313388] [] ? do_exit+0x2d2/0xab0 [67140.319910] [] ? do_group_exit+0x3f/0xa0 [67140.326821] [] ? get_signal+0x1cc/0x5e0 [67140.333635] [] ? do_signal+0x37/0x660 [67140.340257] [] ? ucma_write+0x7a/0xc0 [rdma_ucm] [67140.347949] [] ? exit_to_usermode_loop+0x59/0xa2 [67140.355651] [] ? syscall_return_slowpath+0x8d/0xa0 [67140.363554] [] ? int_ret_from_sys_call+0x25/0x8f [67140.371259] Code: Bad RIP value. [67140.375678] RIP [] 0xffffffffa07ccd7a [67140.382314] RSP [67140.386894] CR2: ffffffffa07ccd7a [67140.393737] ---[ end trace 807b4472c30412d0 ]--- [67141.682413] Kernel panic - not syncing: Fatal exception [67141.682431] Kernel Offset: disabled [67141.733934] ---[ end Kernel panic - not syncing: Fatal exception Root Cause: During rmmod "ib_uverbs_close()" context is still running, while "ib_uverbs_remove_one()" context completes and ends up freeing ib_dev pointer, thus causing a Kernel Panic. This patch fixes the race. ib_uverbs_close validates dev->ib_dev against NULL inside an srcu lock. If it is NULL, it waits for a completion and drops the srcu else continues with the normal flow. CC: Yishai Hadas Signed-off-by: Devesh Sharma --- drivers/infiniband/core/uverbs.h | 1 + drivers/infiniband/core/uverbs_main.c | 19 +++++++++++++++++-- 2 files changed, 18 insertions(+), 2 deletions(-) diff --git a/drivers/infiniband/core/uverbs.h b/drivers/infiniband/core/uverbs.h index 612ccfd..94a7339 100644 --- a/drivers/infiniband/core/uverbs.h +++ b/drivers/infiniband/core/uverbs.h @@ -121,6 +121,7 @@ struct ib_uverbs_file { struct ib_event_handler event_handler; struct ib_uverbs_event_file *async_file; struct list_head list; + struct completion fcomp; int is_closed; }; diff --git a/drivers/infiniband/core/uverbs_main.c b/drivers/infiniband/core/uverbs_main.c index 39680ae..f1948bc 100644 --- a/drivers/infiniband/core/uverbs_main.c +++ b/drivers/infiniband/core/uverbs_main.c @@ -928,6 +928,7 @@ static int ib_uverbs_open(struct inode *inode, struct file *filp) file->async_file = NULL; kref_init(&file->ref); mutex_init(&file->mutex); + init_completion(&file->fcomp); filp->private_data = file; kobject_get(&dev->kobj); @@ -954,21 +955,33 @@ static int ib_uverbs_close(struct inode *inode, struct file *filp) struct ib_uverbs_file *file = filp->private_data; struct ib_uverbs_device *dev = file->device; struct ib_ucontext *ucontext = NULL; + struct ib_device *ib_dev; + int srcu_key; + + srcu_key = srcu_read_lock(&dev->disassociate_srcu); + ib_dev = srcu_dereference(dev->ib_dev, + &dev->disassociate_srcu); + if (!ib_dev) + goto out; - mutex_lock(&file->device->lists_mutex); + mutex_lock(&dev->lists_mutex); ucontext = file->ucontext; file->ucontext = NULL; if (!file->is_closed) { list_del(&file->list); file->is_closed = 1; } - mutex_unlock(&file->device->lists_mutex); + mutex_unlock(&dev->lists_mutex); if (ucontext) ib_uverbs_cleanup_ucontext(file, ucontext); if (file->async_file) kref_put(&file->async_file->ref, ib_uverbs_release_event_file); + complete(&file->fcomp); +out: + wait_for_completion(&file->fcomp); + srcu_read_unlock(&dev->disassociate_srcu, srcu_key); kref_put(&file->ref, ib_uverbs_release_file); kobject_put(&dev->kobj); @@ -1199,6 +1212,8 @@ static void ib_uverbs_free_hw_resources(struct ib_uverbs_device *uverbs_dev, } mutex_lock(&uverbs_dev->lists_mutex); + + complete(&file->fcomp); kref_put(&file->ref, ib_uverbs_release_file); }