From patchwork Mon Dec 5 13:43:21 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Boyer X-Patchwork-Id: 9460997 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 3E00F60236 for ; Mon, 5 Dec 2016 13:44:08 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 2EDD622ADC for ; Mon, 5 Dec 2016 13:44:08 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 22FA527B81; Mon, 5 Dec 2016 13:44:08 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.8 required=2.0 tests=BAYES_00,DKIM_SIGNED, RCVD_IN_DNSWL_HI,T_DKIM_INVALID autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id BF43322ADC for ; Mon, 5 Dec 2016 13:44:07 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751244AbcLENoG (ORCPT ); Mon, 5 Dec 2016 08:44:06 -0500 Received: from esa8.dell-outbound.iphmx.com ([68.232.149.218]:17230 "EHLO esa8.dell-outbound.iphmx.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1750998AbcLENoF (ORCPT ); Mon, 5 Dec 2016 08:44:05 -0500 DomainKey-Signature: s=smtpout; d=dell.com; c=simple; q=dns; h=Received:Received:Received:X-DKIM:Received:Received:From: To:Cc:Subject:Date:Message-Id:X-Mailer:In-Reply-To: References:X-RSA-Classifications:X-Sentrion-Hostname; b=WTuuHKk/tvQW0BNfrS4FZ/8Ws9tcSt0nFdC3nUyh/LuEsUd+kHXhGYB5 JozILiRAEpLIpXOr0Rdri3rGcZSchD0U0axRnOIlxPDTIC1rM5x+4ijdm /p01gXUFdrBLSKvljzAe2hfRUQu5raOIuZJHq5HV1MuW/Qqa660ec7/00 8=; DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=dell.com; i=@dell.com; q=dns/txt; s=smtpout; t=1480945445; x=1512481445; h=from:to:cc:subject:date:message-id:in-reply-to: references; bh=QMaHDie2a73gNG+DwGFaO7jmwBn0aANuIP1L9++xrlw=; b=PPjj2OVbsbmHy/ctQpKsGSt2mn2RDg4lYhIMxvqiE+E6BS7qUSnlrPRF 9AmxBRbr4X3Q8iLdqu+UOkwJcrRdrdbnS4o/U73ybE63Y9uZSCpuE/En5 gXFbs54fYJHBO5uaFyeUIbnEEdOL2xkBkMLVUQupiykLB4ar9UH6x7J2E I=; Received: from esa6.dell-outbound2.iphmx.com ([68.232.154.99]) by esa8.dell-outbound.iphmx.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 05 Dec 2016 07:43:44 -0600 Received: from mailuogwhop.emc.com ([168.159.213.141]) by esa6.dell-outbound2.iphmx.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 05 Dec 2016 19:43:44 +0600 Received: from maildlpprd03.lss.emc.com (maildlpprd03.lss.emc.com [10.253.24.35]) by mailuogwprd04.lss.emc.com (Sentrion-MTA-4.3.1/Sentrion-MTA-4.3.0) with ESMTP id uB5DhfNk010592 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Mon, 5 Dec 2016 08:43:42 -0500 X-DKIM: OpenDKIM Filter v2.4.3 mailuogwprd04.lss.emc.com uB5DhfNk010592 Received: from mailapphubprd03.lss.emc.com (emcmail.lss.emc.com [10.253.24.70]) by maildlpprd03.lss.emc.com (RSA Interceptor); Mon, 5 Dec 2016 08:43:14 -0500 Received: from hopcyc-boyera-1.corp.emc.com (hopcyc-boyera-1.cec.lab.emc.com [10.244.91.191]) by mailapphubprd03.lss.emc.com (Sentrion-MTA-4.3.1/Sentrion-MTA-4.3.0) with ESMTP id uB5DhRfN016371; Mon, 5 Dec 2016 08:43:31 -0500 From: Andrew Boyer To: monis@mellanox.com, yonatanc@mellanox.com, linux-rdma@vger.kernel.org Cc: Andrew Boyer Subject: [PATCH v2 2/2] IB/rxe: Hold refs when running tasklets Date: Mon, 5 Dec 2016 08:43:21 -0500 Message-Id: <1480945401-3025-3-git-send-email-andrew.boyer@dell.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1480945401-3025-2-git-send-email-andrew.boyer@dell.com> References: <1480945401-3025-2-git-send-email-andrew.boyer@dell.com> X-RSA-Classifications: public X-Sentrion-Hostname: mailuogwprd04.lss.emc.com Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP It might be possible for all of a QP's references to be dropped while one of that QP's tasklets is running. For example, the completer might run during QP destroy. If qp->valid is false, it will drop all of the packets on the resp_pkts list, potentially removing the last reference. Then it tries to advance the SQ consumer pointer. If the SQ's buffer has already been destroyed, the system will panic. To be safe, hold a reference on the QP for the duration of each tasklet. Signed-off-by: Andrew Boyer --- drivers/infiniband/sw/rxe/rxe_comp.c | 4 ++++ drivers/infiniband/sw/rxe/rxe_req.c | 4 ++++ drivers/infiniband/sw/rxe/rxe_resp.c | 3 +++ 3 files changed, 11 insertions(+) diff --git a/drivers/infiniband/sw/rxe/rxe_comp.c b/drivers/infiniband/sw/rxe/rxe_comp.c index 6c5e29d..3687fcd 100644 --- a/drivers/infiniband/sw/rxe/rxe_comp.c +++ b/drivers/infiniband/sw/rxe/rxe_comp.c @@ -510,6 +510,8 @@ int rxe_completer(void *arg) struct rxe_pkt_info *pkt = NULL; enum comp_state state; + rxe_add_ref(qp); + if (!qp->valid) { while ((skb = skb_dequeue(&qp->resp_pkts))) { rxe_drop_ref(qp); @@ -739,11 +741,13 @@ int rxe_completer(void *arg) /* we come here if we are done with processing and want the task to * exit from the loop calling us */ + rxe_drop_ref(qp); return -EAGAIN; done: /* we come here if we have processed a packet we want the task to call * us again to see if there is anything else to do */ + rxe_drop_ref(qp); return 0; } diff --git a/drivers/infiniband/sw/rxe/rxe_req.c b/drivers/infiniband/sw/rxe/rxe_req.c index 22bd963..035cb90 100644 --- a/drivers/infiniband/sw/rxe/rxe_req.c +++ b/drivers/infiniband/sw/rxe/rxe_req.c @@ -596,6 +596,8 @@ int rxe_requester(void *arg) struct rxe_qp rollback_qp; struct rxe_send_wqe rollback_wqe; + rxe_add_ref(qp); + next_wqe: if (unlikely(!qp->valid || qp->req.state == QP_STATE_ERROR)) goto exit; @@ -756,8 +758,10 @@ int rxe_requester(void *arg) */ wqe->wr.send_flags |= IB_SEND_SIGNALED; __rxe_do_task(&qp->comp.task); + rxe_drop_ref(qp); return -EAGAIN; exit: + rxe_drop_ref(qp); return -EAGAIN; } diff --git a/drivers/infiniband/sw/rxe/rxe_resp.c b/drivers/infiniband/sw/rxe/rxe_resp.c index dd3d88a..337a1cb 100644 --- a/drivers/infiniband/sw/rxe/rxe_resp.c +++ b/drivers/infiniband/sw/rxe/rxe_resp.c @@ -1198,6 +1198,8 @@ int rxe_responder(void *arg) struct rxe_pkt_info *pkt = NULL; int ret = 0; + rxe_add_ref(qp); + qp->resp.aeth_syndrome = AETH_ACK_UNLIMITED; if (!qp->valid) { @@ -1386,5 +1388,6 @@ int rxe_responder(void *arg) exit: ret = -EAGAIN; done: + rxe_drop_ref(qp); return ret; }