From patchwork Tue Nov 10 14:13:55 2015 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Marciniszyn, Mike" X-Patchwork-Id: 7591231 Return-Path: X-Original-To: patchwork-linux-rdma@patchwork.kernel.org Delivered-To: patchwork-parsemail@patchwork2.web.kernel.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.136]) by patchwork2.web.kernel.org (Postfix) with ESMTP id 72E73C05C6 for ; Tue, 10 Nov 2015 14:14:44 +0000 (UTC) Received: from mail.kernel.org (localhost [127.0.0.1]) by mail.kernel.org (Postfix) with ESMTP id 85AB820741 for ; Tue, 10 Nov 2015 14:14:43 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 86F42206E6 for ; Tue, 10 Nov 2015 14:14:42 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753113AbbKJOOk (ORCPT ); Tue, 10 Nov 2015 09:14:40 -0500 Received: from mga14.intel.com ([192.55.52.115]:41351 "EHLO mga14.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752976AbbKJOOk (ORCPT ); Tue, 10 Nov 2015 09:14:40 -0500 Received: from fmsmga003.fm.intel.com ([10.253.24.29]) by fmsmga103.fm.intel.com with ESMTP; 10 Nov 2015 06:13:57 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.20,270,1444719600"; d="scan'208";a="597567591" Received: from sedona.ch.intel.com ([143.182.228.65]) by FMSMGA003.fm.intel.com with ESMTP; 10 Nov 2015 06:13:57 -0800 Received: from phlsvsles11.ph.intel.com (phlsvsles11.ph.intel.com [10.228.195.43]) by sedona.ch.intel.com (8.13.6/8.14.3/Standard MailSET/Hub) with ESMTP id tAAEDuK1015399; Tue, 10 Nov 2015 07:13:56 -0700 Received: from phlsvslse11.ph.intel.com (localhost [127.0.0.1]) by phlsvsles11.ph.intel.com with ESMTP id tAAEDtGu030192; Tue, 10 Nov 2015 09:13:55 -0500 Subject: [PATCH 1/2] staging/rdma/hfi1: add common routine for queuing acks To: devel@driverdev.osuosl.org From: Mike Marciniszyn Cc: linux-rdma@vger.kernel.org, dledford@redhat.com, linux-next@vger.kernel.org Date: Tue, 10 Nov 2015 09:13:55 -0500 Message-ID: <20151110141355.30166.93359.stgit@phlsvslse11.ph.intel.com> User-Agent: StGit/0.16 MIME-Version: 1.0 Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org X-Spam-Status: No, score=-7.2 required=5.0 tests=BAYES_00, RCVD_IN_DNSWL_HI, RP_MATCHES_RCVD, UNPARSEABLE_RELAY autolearn=unavailable version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on mail.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP This patch is a prelimary patch required to coalesce acks. The routine to "schedule" a QP for sending a NAK is now centralized in rc_defer_ack(). The flag is changed for clarity since the all acks will potentially use the deferral mechanism. Reviewed-by: Dennis Dalessandro Signed-off-by: Mike Marciniszyn --- drivers/staging/rdma/hfi1/driver.c | 4 ++- drivers/staging/rdma/hfi1/rc.c | 42 +++++++++++++----------------------- drivers/staging/rdma/hfi1/verbs.h | 12 ++++++---- 3 files changed, 24 insertions(+), 34 deletions(-) -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html diff --git a/drivers/staging/rdma/hfi1/driver.c b/drivers/staging/rdma/hfi1/driver.c index ce69141..e8e0f55 100644 --- a/drivers/staging/rdma/hfi1/driver.c +++ b/drivers/staging/rdma/hfi1/driver.c @@ -721,8 +721,8 @@ static inline void process_rcv_qp_work(struct hfi1_packet *packet) */ list_for_each_entry_safe(qp, nqp, &rcd->qp_wait_list, rspwait) { list_del_init(&qp->rspwait); - if (qp->r_flags & HFI1_R_RSP_NAK) { - qp->r_flags &= ~HFI1_R_RSP_NAK; + if (qp->r_flags & HFI1_R_RSP_DEFERED_ACK) { + qp->r_flags &= ~HFI1_R_RSP_DEFERED_ACK; hfi1_send_rc_ack(rcd, qp, 0); } if (qp->r_flags & HFI1_R_RSP_SEND) { diff --git a/drivers/staging/rdma/hfi1/rc.c b/drivers/staging/rdma/hfi1/rc.c index 0b19206..6fe0104 100644 --- a/drivers/staging/rdma/hfi1/rc.c +++ b/drivers/staging/rdma/hfi1/rc.c @@ -1608,6 +1608,16 @@ bail: return; } +static inline void rc_defered_ack(struct hfi1_ctxtdata *rcd, + struct hfi1_qp *qp) +{ + if (list_empty(&qp->rspwait)) { + qp->r_flags |= HFI1_R_RSP_DEFERED_ACK; + atomic_inc(&qp->refcount); + list_add_tail(&qp->rspwait, &rcd->qp_wait_list); + } +} + /** * rc_rcv_error - process an incoming duplicate or error RC packet * @ohdr: the other headers for this packet @@ -1650,11 +1660,7 @@ static noinline int rc_rcv_error(struct hfi1_other_headers *ohdr, void *data, * in the receive queue have been processed. * Otherwise, we end up propagating congestion. */ - if (list_empty(&qp->rspwait)) { - qp->r_flags |= HFI1_R_RSP_NAK; - atomic_inc(&qp->refcount); - list_add_tail(&qp->rspwait, &rcd->qp_wait_list); - } + rc_defered_ack(rcd, qp); } goto done; } @@ -2337,11 +2343,7 @@ rnr_nak: qp->r_nak_state = IB_RNR_NAK | qp->r_min_rnr_timer; qp->r_ack_psn = qp->r_psn; /* Queue RNR NAK for later */ - if (list_empty(&qp->rspwait)) { - qp->r_flags |= HFI1_R_RSP_NAK; - atomic_inc(&qp->refcount); - list_add_tail(&qp->rspwait, &rcd->qp_wait_list); - } + rc_defered_ack(rcd, qp); return; nack_op_err: @@ -2349,11 +2351,7 @@ nack_op_err: qp->r_nak_state = IB_NAK_REMOTE_OPERATIONAL_ERROR; qp->r_ack_psn = qp->r_psn; /* Queue NAK for later */ - if (list_empty(&qp->rspwait)) { - qp->r_flags |= HFI1_R_RSP_NAK; - atomic_inc(&qp->refcount); - list_add_tail(&qp->rspwait, &rcd->qp_wait_list); - } + rc_defered_ack(rcd, qp); return; nack_inv_unlck: @@ -2363,11 +2361,7 @@ nack_inv: qp->r_nak_state = IB_NAK_INVALID_REQUEST; qp->r_ack_psn = qp->r_psn; /* Queue NAK for later */ - if (list_empty(&qp->rspwait)) { - qp->r_flags |= HFI1_R_RSP_NAK; - atomic_inc(&qp->refcount); - list_add_tail(&qp->rspwait, &rcd->qp_wait_list); - } + rc_defered_ack(rcd, qp); return; nack_acc_unlck: @@ -2421,13 +2415,7 @@ void hfi1_rc_hdrerr( * Otherwise, we end up * propagating congestion. */ - if (list_empty(&qp->rspwait)) { - qp->r_flags |= HFI1_R_RSP_NAK; - atomic_inc(&qp->refcount); - list_add_tail( - &qp->rspwait, - &rcd->qp_wait_list); - } + rc_defered_ack(rcd, qp); } /* Out of sequence NAK */ } /* QP Request NAKs */ } diff --git a/drivers/staging/rdma/hfi1/verbs.h b/drivers/staging/rdma/hfi1/verbs.h index e4a8a0d..c5e6f47 100644 --- a/drivers/staging/rdma/hfi1/verbs.h +++ b/drivers/staging/rdma/hfi1/verbs.h @@ -547,11 +547,13 @@ struct hfi1_qp { /* * Bit definitions for r_flags. */ -#define HFI1_R_REUSE_SGE 0x01 -#define HFI1_R_RDMAR_SEQ 0x02 -#define HFI1_R_RSP_NAK 0x04 -#define HFI1_R_RSP_SEND 0x08 -#define HFI1_R_COMM_EST 0x10 +#define HFI1_R_REUSE_SGE 0x01 +#define HFI1_R_RDMAR_SEQ 0x02 +/* defer ack until end of interrupt session */ +#define HFI1_R_RSP_DEFERED_ACK 0x04 +/* relay ack to send engine */ +#define HFI1_R_RSP_SEND 0x08 +#define HFI1_R_COMM_EST 0x10 /* * Bit definitions for s_flags.