From patchwork Mon Sep 8 12:51:38 2014 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sreedhar Kodali X-Patchwork-Id: 4861761 Return-Path: X-Original-To: patchwork-linux-rdma@patchwork.kernel.org Delivered-To: patchwork-parsemail@patchwork1.web.kernel.org Received: from mail.kernel.org (mail.kernel.org [198.145.19.201]) by patchwork1.web.kernel.org (Postfix) with ESMTP id A205C9F32F for ; Mon, 8 Sep 2014 12:51:54 +0000 (UTC) Received: from mail.kernel.org (localhost [127.0.0.1]) by mail.kernel.org (Postfix) with ESMTP id 7099720108 for ; Mon, 8 Sep 2014 12:51:53 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 3814E2010E for ; Mon, 8 Sep 2014 12:51:52 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753750AbaIHMvp (ORCPT ); Mon, 8 Sep 2014 08:51:45 -0400 Received: from e39.co.us.ibm.com ([32.97.110.160]:60536 "EHLO e39.co.us.ibm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753018AbaIHMvn (ORCPT ); Mon, 8 Sep 2014 08:51:43 -0400 Received: from /spool/local by e39.co.us.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Mon, 8 Sep 2014 06:51:42 -0600 Received: from d01dlp03.pok.ibm.com (9.56.250.168) by e39.co.us.ibm.com (192.168.1.139) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; Mon, 8 Sep 2014 06:51:39 -0600 Received: from b01cxnp22036.gho.pok.ibm.com (b01cxnp22036.gho.pok.ibm.com [9.57.198.26]) by d01dlp03.pok.ibm.com (Postfix) with ESMTP id DED9DC9003E for ; Mon, 8 Sep 2014 08:51:30 -0400 (EDT) Received: from d01av02.pok.ibm.com (d01av02.pok.ibm.com [9.56.224.216]) by b01cxnp22036.gho.pok.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id s88Cpd0a5898634 for ; Mon, 8 Sep 2014 12:51:39 GMT Received: from d01av02.pok.ibm.com (localhost [127.0.0.1]) by d01av02.pok.ibm.com (8.14.4/8.14.4/NCO v10.0 AVout) with ESMTP id s88CpcEb006091 for ; Mon, 8 Sep 2014 08:51:39 -0400 Received: from ltcweb.rtp.raleigh.ibm.com (ltcweb.rtp.raleigh.ibm.com [9.37.210.204]) by d01av02.pok.ibm.com (8.14.4/8.14.4/NCO v10.0 AVin) with ESMTP id s88CpcQY006070; Mon, 8 Sep 2014 08:51:38 -0400 Received: from ltc.linux.ibm.com (localhost.localdomain [127.0.0.1]) by ltcweb.rtp.raleigh.ibm.com (Postfix) with ESMTP id A245AC0103; Mon, 8 Sep 2014 08:51:38 -0400 (EDT) MIME-Version: 1.0 Date: Mon, 08 Sep 2014 18:21:38 +0530 From: Sreedhar Kodali To: linux-rdma@vger.kernel.org Cc: sean.hefty@intel.com, pradeeps@linux.vnet.ibm.com, sithirug@linux.vnet.ibm.com Subject: [PATCH v3 4/4] rsockets: acknowledge completion queue events in batch Message-ID: X-Sender: srkodali@linux.vnet.ibm.com User-Agent: Roundcube Webmail/1.0.1 X-TM-AS-MML: disable X-Content-Scanned: Fidelis XPS MAILER x-cbid: 14090812-9332-0000-0000-000001EF889D Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org X-Spam-Status: No, score=-9.4 required=5.0 tests=BAYES_00, RCVD_IN_DNSWL_HI, RP_MATCHES_RCVD, UNPARSEABLE_RELAY autolearn=unavailable version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on mail.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Author: Sreedhar Kodali Change note from v2 to v3: eliminated setting through option as suggested by Sean Date: Mon Sep 8 15:38:54 2014 +0530 Perform completion event acknowledgments in batches instead of individually to minimze locking overheads. Size of the completion queue decides the size of a batch. Signed-off-by: Sreedhar Kodali --- fastlock_init(&rs->cq_lock); @@ -1044,8 +1046,11 @@ static void rs_free(struct rsocket *rs) if (rs->cm_id) { rs_free_iomappings(rs); - if (rs->cm_id->qp) + if (rs->cm_id->qp) { + if (rs->unack_cqe > 0) + ibv_ack_cq_events(rs->cm_id->recv_cq, rs->unack_cqe); rdma_destroy_qp(rs->cm_id); + } rdma_destroy_id(rs->cm_id); } @@ -2026,7 +2031,11 @@ static int rs_get_cq_event(struct rsocket *rs) resume_get_cq_event: ret = ibv_get_cq_event(rs->cm_id->recv_cq_channel, &cq, &context); if (!ret) { - ibv_ack_cq_events(rs->cm_id->recv_cq, 1); + rs->unack_cqe += 1; + if (rs->unack_cqe == rs->sq_size + rs->rq_size) { + ibv_ack_cq_events(rs->cm_id->recv_cq, rs->unack_cqe); + rs->unack_cqe = 0; + } rs->cq_armed = 0; } else if (restart_onintr == 1 && errno == EINTR) { errno = 0; --- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html diff --git a/src/rsocket.c b/src/rsocket.c index ffea0ca..d201e0a 100644 --- a/src/rsocket.c +++ b/src/rsocket.c @@ -384,6 +384,7 @@ struct rsocket { dlist_entry iomap_list; dlist_entry iomap_queue; int iomap_pending; + int unack_cqe; }; #define DS_UDP_TAG 0x55555555 @@ -638,6 +639,7 @@ static struct rsocket *rs_alloc(struct rsocket *inherited_rs, int type) rs->target_iomap_size = def_iomap_size; } } + rs->unack_cqe = 0; fastlock_init(&rs->slock); fastlock_init(&rs->rlock);