From patchwork Fri Nov 18 14:36:44 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Boyer X-Patchwork-Id: 9436665 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 6ABB360237 for ; Fri, 18 Nov 2016 14:47:19 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 65FE929908 for ; Fri, 18 Nov 2016 14:47:19 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 5AE8229910; Fri, 18 Nov 2016 14:47:19 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.3 required=2.0 tests=BAYES_00,DKIM_SIGNED, RCVD_IN_DNSWL_HI, RCVD_IN_SORBS_SPAM, T_DKIM_INVALID autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 5014329908 for ; Fri, 18 Nov 2016 14:47:18 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752338AbcKROrQ (ORCPT ); Fri, 18 Nov 2016 09:47:16 -0500 Received: from esa7.dell-outbound.iphmx.com ([68.232.153.96]:59630 "EHLO esa7.dell-outbound.iphmx.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753280AbcKROrN (ORCPT ); Fri, 18 Nov 2016 09:47:13 -0500 DomainKey-Signature: s=smtpout; d=dell.com; c=simple; q=dns; h=Received:Received:Received:X-DKIM:Received:Received:From: To:Cc:Subject:Date:Message-Id:X-Mailer:In-Reply-To: References:X-RSA-Classifications:X-Sentrion-Hostname; b=dGi+XrQgrRblf5yUnrUCeBqb5FpCT5AbIpX1+/p5UjPmALAfLN+yqZx2 LFRfoCO119DO8tLmRVyom0yYoJa4NuwRfe0otH6U5cr7s6WCOklM8IvqG X3PR38VDTNRk3MAAdRaM1AvrlHVfMTmnJeZXEGnPwu0tyLBp0DC1j7dXI g=; DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=dell.com; i=@dell.com; q=dns/txt; s=smtpout; t=1479480433; x=1511016433; h=from:to:cc:subject:date:message-id:in-reply-to: references; bh=XjJ0O2rFaHzSK4iqeQIBAJr0I3oVFbiS64lCmz2wJcE=; b=gH6QJIGbl4/D+wz+DWj6fExZdf8179A9bA8jZ8PV3X7AtUl7mMT0x/Ra 4C+hrm4lAY+ideZRxOVQ7tDostf33t4ES54x+jwYNZh6VwTi3ZhAx1c5q EPR91qhByfZ1J0zMcdER5dTwcbPj9WrnONWmHt+9K//0NARxsgpHCIiMr Y=; Received: from esa1.dell-outbound2.iphmx.com ([68.232.153.201]) by esa7.dell-outbound.iphmx.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 18 Nov 2016 08:37:41 -0600 Received: from mailuogwdur.emc.com ([128.221.224.79]) by esa1.dell-outbound2.iphmx.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 18 Nov 2016 20:37:41 +0600 Received: from maildlpprd53.lss.emc.com (maildlpprd53.lss.emc.com [10.106.48.157]) by mailuogwprd53.lss.emc.com (Sentrion-MTA-4.3.1/Sentrion-MTA-4.3.0) with ESMTP id uAIEbc5M029607 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Fri, 18 Nov 2016 09:37:39 -0500 X-DKIM: OpenDKIM Filter v2.4.3 mailuogwprd53.lss.emc.com uAIEbc5M029607 Received: from mailapphubprd02.lss.emc.com (emcmail.lss.emc.com [10.253.24.52]) by maildlpprd53.lss.emc.com (RSA Interceptor); Fri, 18 Nov 2016 09:37:09 -0500 Received: from hopcyc-boyera-1.corp.emc.com (hopcyc-boyera-1.cec.lab.emc.com [10.244.91.191]) by mailapphubprd02.lss.emc.com (Sentrion-MTA-4.3.1/Sentrion-MTA-4.3.0) with ESMTP id uAIEbBSr021870; Fri, 18 Nov 2016 09:37:24 -0500 From: Andrew Boyer To: monis@mellanox.com, linux-rdma@vger.kernel.org Cc: Andrew Boyer Subject: [PATCH 2/7] IB/rxe: Advance the consumer pointer before posting the CQE Date: Fri, 18 Nov 2016 09:36:44 -0500 Message-Id: <1479479809-10798-2-git-send-email-andrew.boyer@dell.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1479479809-10798-1-git-send-email-andrew.boyer@dell.com> References: <1479479809-10798-1-git-send-email-andrew.boyer@dell.com> X-RSA-Classifications: public X-Sentrion-Hostname: mailuogwprd53.lss.emc.com Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP A simple userspace application might poll the CQ, find a completion, and then attempt to post a new WQE to the SQ. A spurious error can occur if the userspace application detects a full SQ in the instant before the kernel is able to advance the SQ consumer pointer. This is noticeable when using single-entry SQs with ibv_rc_pingpong() if lots of kernel and userspace library debugging is enabled. Signed-off-by: Andrew Boyer Reviewed-by: Yonatan Cohen --- drivers/infiniband/sw/rxe/rxe_comp.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/drivers/infiniband/sw/rxe/rxe_comp.c b/drivers/infiniband/sw/rxe/rxe_comp.c index 6c5e29d..d46c49b 100644 --- a/drivers/infiniband/sw/rxe/rxe_comp.c +++ b/drivers/infiniband/sw/rxe/rxe_comp.c @@ -420,11 +420,12 @@ static void do_complete(struct rxe_qp *qp, struct rxe_send_wqe *wqe) (wqe->wr.send_flags & IB_SEND_SIGNALED) || (qp->req.state == QP_STATE_ERROR)) { make_send_cqe(qp, wqe, &cqe); + advance_consumer(qp->sq.queue); rxe_cq_post(qp->scq, &cqe, 0); + } else { + advance_consumer(qp->sq.queue); } - advance_consumer(qp->sq.queue); - /* * we completed something so let req run again * if it is trying to fence