From patchwork Tue Oct 20 02:11:30 2015 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ira Weiny X-Patchwork-Id: 7441571 Return-Path: X-Original-To: patchwork-linux-rdma@patchwork.kernel.org Delivered-To: patchwork-parsemail@patchwork2.web.kernel.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.136]) by patchwork2.web.kernel.org (Postfix) with ESMTP id BBA2EBEEA4 for ; Tue, 20 Oct 2015 02:12:23 +0000 (UTC) Received: from mail.kernel.org (localhost [127.0.0.1]) by mail.kernel.org (Postfix) with ESMTP id EB9A2207AE for ; Tue, 20 Oct 2015 02:12:22 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id A4DBE20797 for ; Tue, 20 Oct 2015 02:12:21 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1750997AbbJTCMP (ORCPT ); Mon, 19 Oct 2015 22:12:15 -0400 Received: from mga02.intel.com ([134.134.136.20]:55880 "EHLO mga02.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752037AbbJTCMN (ORCPT ); Mon, 19 Oct 2015 22:12:13 -0400 Received: from orsmga003.jf.intel.com ([10.7.209.27]) by orsmga101.jf.intel.com with ESMTP; 19 Oct 2015 19:12:12 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.17,705,1437462000"; d="scan'208";a="667687974" Received: from phlsvsds.ph.intel.com ([10.228.195.38]) by orsmga003.jf.intel.com with ESMTP; 19 Oct 2015 19:12:11 -0700 Received: from phlsvsds.ph.intel.com (localhost.localdomain [127.0.0.1]) by phlsvsds.ph.intel.com (8.13.8/8.13.8) with ESMTP id t9K2CBir008752; Mon, 19 Oct 2015 22:12:11 -0400 Received: (from iweiny@localhost) by phlsvsds.ph.intel.com (8.13.8/8.13.8/Submit) id t9K2CAVQ008747; Mon, 19 Oct 2015 22:12:10 -0400 X-Authentication-Warning: phlsvsds.ph.intel.com: iweiny set sender to ira.weiny@intel.com using -f From: ira.weiny@intel.com To: gregkh@linuxfoundation.org, devel@driverdev.osuosl.org Cc: dledford@redhat.com, linux-rdma@vger.kernel.org, dennis.dalessandro@intel.com, mike.marciniszyn@intel.com, Mitko Haralanov , Ira Weiny Subject: [PATCH v2 15/22] staging/rdma/hfi1: Allow tuning of SDMA interrupt rate Date: Mon, 19 Oct 2015 22:11:30 -0400 Message-Id: <1445307097-8244-16-git-send-email-ira.weiny@intel.com> X-Mailer: git-send-email 1.7.1 In-Reply-To: <1445307097-8244-1-git-send-email-ira.weiny@intel.com> References: <1445307097-8244-1-git-send-email-ira.weiny@intel.com> Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org X-Spam-Status: No, score=-6.9 required=5.0 tests=BAYES_00, RCVD_IN_DNSWL_HI, RP_MATCHES_RCVD, UNPARSEABLE_RELAY autolearn=unavailable version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on mail.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Mitko Haralanov The SDMA engines were configured to generate progress interrupts every time they processed N/2 descriptors (where N is the size of the descriptor queue). This interval was too infrequent, leading to degraded performance. This commit adds a module parameter, which allows for the tuning of the interrupt frequency until an optimal frequency is found for both PSM and Verbs. At that time, the parameter could be pulled out, if desired. Reviewed-by: Mike Marciniszyn Signed-off-by: Mitko Haralanov Signed-off-by: Ira Weiny --- drivers/staging/rdma/hfi1/sdma.c | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/drivers/staging/rdma/hfi1/sdma.c b/drivers/staging/rdma/hfi1/sdma.c index 53b3e4d9518b..452e7edcee7a 100644 --- a/drivers/staging/rdma/hfi1/sdma.c +++ b/drivers/staging/rdma/hfi1/sdma.c @@ -80,6 +80,10 @@ uint mod_num_sdma; module_param_named(num_sdma, mod_num_sdma, uint, S_IRUGO); MODULE_PARM_DESC(num_sdma, "Set max number SDMA engines to use"); +static uint sdma_desct_intr; +module_param_named(desct_intr, sdma_desct_intr, uint, S_IRUGO | S_IWUSR); +MODULE_PARM_DESC(desct_intr, "Number of SDMA descriptor before interrupt"); + #define SDMA_WAIT_BATCH_SIZE 20 /* max wait time for a SDMA engine to indicate it has halted */ #define SDMA_ERR_HALT_TIMEOUT 10 /* ms */ @@ -1046,6 +1050,8 @@ int sdma_init(struct hfi1_devdata *dd, u8 port) return -ENOMEM; idle_cnt = ns_to_cclock(dd, idle_cnt); + if (!sdma_desct_intr) + sdma_desct_intr = descq_cnt / 2; /* Allocate memory for SendDMA descriptor FIFOs */ for (this_idx = 0; this_idx < num_engines; ++this_idx) { sde = &dd->per_sdma[this_idx]; @@ -1548,7 +1554,7 @@ void sdma_engine_interrupt(struct sdma_engine *sde, u64 status) { trace_hfi1_sdma_engine_interrupt(sde, status); write_seqlock(&sde->head_lock); - sdma_set_desc_cnt(sde, sde->descq_cnt / 2); + sdma_set_desc_cnt(sde, sdma_desct_intr); sdma_make_progress(sde, status); write_sequnlock(&sde->head_lock); }