From patchwork Mon Nov 28 14:37:24 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Boyer X-Patchwork-Id: 9449665 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id D4C8C600CB for ; Mon, 28 Nov 2016 14:38:09 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id CD1DB25D99 for ; Mon, 28 Nov 2016 14:38:09 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id C1FBE27C0C; Mon, 28 Nov 2016 14:38:09 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.8 required=2.0 tests=BAYES_00,DKIM_SIGNED, RCVD_IN_DNSWL_HI,T_DKIM_INVALID autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 63D1D25D99 for ; Mon, 28 Nov 2016 14:38:09 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S933051AbcK1OiI (ORCPT ); Mon, 28 Nov 2016 09:38:08 -0500 Received: from esa4.dell-outbound.iphmx.com ([68.232.149.214]:29199 "EHLO esa4.dell-outbound.iphmx.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S932283AbcK1OiH (ORCPT ); Mon, 28 Nov 2016 09:38:07 -0500 DomainKey-Signature: s=smtpout; d=dell.com; c=simple; q=dns; h=Received:Received:Received:X-DKIM:Received:Received:From: To:Cc:Subject:Date:Message-Id:X-Mailer:In-Reply-To: References:X-RSA-Classifications:X-Sentrion-Hostname; b=oslMYj14DxNAIlCClSYzeH3ZhYoQ677Dw6Wkdxre9OeZcyxbUpCc2u2E 9RRJgKsIuDXZg94HjYMdHWz8TgpnCNBpyQgLwxUKhnXlG44Lr+pqx2fGi Ir+1lW8Klk3LkJBKzSYqRhLuk9ZqzBC3uNag+gPGmRXG0nGDXp+CUQEB2 g=; DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=dell.com; i=@dell.com; q=dns/txt; s=smtpout; t=1480343887; x=1511879887; h=from:to:cc:subject:date:message-id:in-reply-to: references; bh=QMaHDie2a73gNG+DwGFaO7jmwBn0aANuIP1L9++xrlw=; b=0xTt5B1xtXvcTOqObR4CGyRRZyRcKz73nPjEq7A2GEk52qKjPfKD2Dfv p5E/VDid3lxQoHTZewTKUjHa1zqnGtDfvcEnKkwCPzQUJXencE4kqmKij qp4tImBYO4yZUV1Ro7LuUONmeEZjGUcKvMMJYTMrISKNkDwyz6HOX6+KN g=; Received: from esa4.dell-outbound2.iphmx.com ([68.232.154.98]) by esa4.dell-outbound.iphmx.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 28 Nov 2016 08:38:06 -0600 Received: from mailuogwhop.emc.com ([168.159.213.141]) by esa4.dell-outbound2.iphmx.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 28 Nov 2016 20:38:04 +0600 Received: from maildlpprd01.lss.emc.com (maildlpprd01.lss.emc.com [10.253.24.33]) by mailuogwprd02.lss.emc.com (Sentrion-MTA-4.3.1/Sentrion-MTA-4.3.0) with ESMTP id uASEc1UY022409 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Mon, 28 Nov 2016 09:38:02 -0500 X-DKIM: OpenDKIM Filter v2.4.3 mailuogwprd02.lss.emc.com uASEc1UY022409 Received: from mailsyshubprd56.lss.emc.com (mailhub.lss.emc.com [10.106.48.138]) by maildlpprd01.lss.emc.com (RSA Interceptor); Mon, 28 Nov 2016 09:37:34 -0500 Received: from hopcyc-boyera-1.corp.emc.com (hopcyc-boyera-1.cec.lab.emc.com [10.244.91.191]) by mailsyshubprd56.lss.emc.com (Sentrion-MTA-4.3.1/Sentrion-MTA-4.3.0) with ESMTP id uASEbSY6004582; Mon, 28 Nov 2016 09:37:49 -0500 From: Andrew Boyer To: monis@mellanox.com, yonatanc@mellanox.com, linux-rdma@vger.kernel.org Cc: Andrew Boyer Subject: [PATCH 2/2] IB/rxe: Hold refs when running tasklets Date: Mon, 28 Nov 2016 09:37:24 -0500 Message-Id: <1480343844-8381-3-git-send-email-andrew.boyer@dell.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1480343844-8381-1-git-send-email-andrew.boyer@dell.com> References: <1480343844-8381-1-git-send-email-andrew.boyer@dell.com> X-RSA-Classifications: public X-Sentrion-Hostname: mailuogwprd02.lss.emc.com Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP It might be possible for all of a QP's references to be dropped while one of that QP's tasklets is running. For example, the completer might run during QP destroy. If qp->valid is false, it will drop all of the packets on the resp_pkts list, potentially removing the last reference. Then it tries to advance the SQ consumer pointer. If the SQ's buffer has already been destroyed, the system will panic. To be safe, hold a reference on the QP for the duration of each tasklet. Signed-off-by: Andrew Boyer --- drivers/infiniband/sw/rxe/rxe_comp.c | 4 ++++ drivers/infiniband/sw/rxe/rxe_req.c | 4 ++++ drivers/infiniband/sw/rxe/rxe_resp.c | 3 +++ 3 files changed, 11 insertions(+) diff --git a/drivers/infiniband/sw/rxe/rxe_comp.c b/drivers/infiniband/sw/rxe/rxe_comp.c index 6c5e29d..3687fcd 100644 --- a/drivers/infiniband/sw/rxe/rxe_comp.c +++ b/drivers/infiniband/sw/rxe/rxe_comp.c @@ -510,6 +510,8 @@ int rxe_completer(void *arg) struct rxe_pkt_info *pkt = NULL; enum comp_state state; + rxe_add_ref(qp); + if (!qp->valid) { while ((skb = skb_dequeue(&qp->resp_pkts))) { rxe_drop_ref(qp); @@ -739,11 +741,13 @@ int rxe_completer(void *arg) /* we come here if we are done with processing and want the task to * exit from the loop calling us */ + rxe_drop_ref(qp); return -EAGAIN; done: /* we come here if we have processed a packet we want the task to call * us again to see if there is anything else to do */ + rxe_drop_ref(qp); return 0; } diff --git a/drivers/infiniband/sw/rxe/rxe_req.c b/drivers/infiniband/sw/rxe/rxe_req.c index 22bd963..035cb90 100644 --- a/drivers/infiniband/sw/rxe/rxe_req.c +++ b/drivers/infiniband/sw/rxe/rxe_req.c @@ -596,6 +596,8 @@ int rxe_requester(void *arg) struct rxe_qp rollback_qp; struct rxe_send_wqe rollback_wqe; + rxe_add_ref(qp); + next_wqe: if (unlikely(!qp->valid || qp->req.state == QP_STATE_ERROR)) goto exit; @@ -756,8 +758,10 @@ int rxe_requester(void *arg) */ wqe->wr.send_flags |= IB_SEND_SIGNALED; __rxe_do_task(&qp->comp.task); + rxe_drop_ref(qp); return -EAGAIN; exit: + rxe_drop_ref(qp); return -EAGAIN; } diff --git a/drivers/infiniband/sw/rxe/rxe_resp.c b/drivers/infiniband/sw/rxe/rxe_resp.c index dd3d88a..337a1cb 100644 --- a/drivers/infiniband/sw/rxe/rxe_resp.c +++ b/drivers/infiniband/sw/rxe/rxe_resp.c @@ -1198,6 +1198,8 @@ int rxe_responder(void *arg) struct rxe_pkt_info *pkt = NULL; int ret = 0; + rxe_add_ref(qp); + qp->resp.aeth_syndrome = AETH_ACK_UNLIMITED; if (!qp->valid) { @@ -1386,5 +1388,6 @@ int rxe_responder(void *arg) exit: ret = -EAGAIN; done: + rxe_drop_ref(qp); return ret; }