From patchwork Mon Oct 19 16:43:40 2015 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ira Weiny X-Patchwork-Id: 7438601 Return-Path: X-Original-To: patchwork-linux-rdma@patchwork.kernel.org Delivered-To: patchwork-parsemail@patchwork1.web.kernel.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.136]) by patchwork1.web.kernel.org (Postfix) with ESMTP id E99359F30B for ; Mon, 19 Oct 2015 16:44:58 +0000 (UTC) Received: from mail.kernel.org (localhost [127.0.0.1]) by mail.kernel.org (Postfix) with ESMTP id 1723A207A9 for ; Mon, 19 Oct 2015 16:44:58 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 21AA4207A2 for ; Mon, 19 Oct 2015 16:44:57 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753771AbbJSQoq (ORCPT ); Mon, 19 Oct 2015 12:44:46 -0400 Received: from mga14.intel.com ([192.55.52.115]:41450 "EHLO mga14.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753625AbbJSQol (ORCPT ); Mon, 19 Oct 2015 12:44:41 -0400 Received: from orsmga003.jf.intel.com ([10.7.209.27]) by fmsmga103.fm.intel.com with ESMTP; 19 Oct 2015 09:44:42 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.17,702,1437462000"; d="scan'208";a="667378705" Received: from phlsvsds.ph.intel.com ([10.228.195.38]) by orsmga003.jf.intel.com with ESMTP; 19 Oct 2015 09:44:41 -0700 Received: from phlsvsds.ph.intel.com (localhost.localdomain [127.0.0.1]) by phlsvsds.ph.intel.com (8.13.8/8.13.8) with ESMTP id t9JGid2f002020; Mon, 19 Oct 2015 12:44:39 -0400 Received: (from iweiny@localhost) by phlsvsds.ph.intel.com (8.13.8/8.13.8/Submit) id t9JGidVu002017; Mon, 19 Oct 2015 12:44:39 -0400 X-Authentication-Warning: phlsvsds.ph.intel.com: iweiny set sender to ira.weiny@intel.com using -f From: ira.weiny@intel.com To: gregkh@linuxfoundation.org, devel@driverdev.osuosl.org Cc: dledford@redhat.com, linux-rdma@vger.kernel.org, dennis.dalessandro@intel.com, mike.marciniszyn@intel.com, Mitko Haralanov , Ira Weiny Subject: [PATCH 16/23] staging/rdma/hfi1: Allow tuning of SDMA interrupt rate Date: Mon, 19 Oct 2015 12:43:40 -0400 Message-Id: <1445273027-29634-17-git-send-email-ira.weiny@intel.com> X-Mailer: git-send-email 1.7.1 In-Reply-To: <1445273027-29634-1-git-send-email-ira.weiny@intel.com> References: <1445273027-29634-1-git-send-email-ira.weiny@intel.com> Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org X-Spam-Status: No, score=-6.9 required=5.0 tests=BAYES_00, RCVD_IN_DNSWL_HI, RP_MATCHES_RCVD, UNPARSEABLE_RELAY autolearn=unavailable version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on mail.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Mitko Haralanov The SDMA engines were configured to generate progrss interrupts every time they processed N/2 descriptors (where N is the size of the descriptor queue). This interval was too infrequent, leading to degraded performance. This commit adds a module parameter, which allows for the tuning of the interrupt frequency until an optimal frequency is found for both PSM and Verbs. At that time, the parameter could be pulled out, if desired. Reviewed-by: Mike Marciniszyn Signed-off-by: Mitko Haralanov Signed-off-by: Ira Weiny --- drivers/staging/rdma/hfi1/sdma.c | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/drivers/staging/rdma/hfi1/sdma.c b/drivers/staging/rdma/hfi1/sdma.c index 53b3e4d9518b..452e7edcee7a 100644 --- a/drivers/staging/rdma/hfi1/sdma.c +++ b/drivers/staging/rdma/hfi1/sdma.c @@ -80,6 +80,10 @@ uint mod_num_sdma; module_param_named(num_sdma, mod_num_sdma, uint, S_IRUGO); MODULE_PARM_DESC(num_sdma, "Set max number SDMA engines to use"); +static uint sdma_desct_intr; +module_param_named(desct_intr, sdma_desct_intr, uint, S_IRUGO | S_IWUSR); +MODULE_PARM_DESC(desct_intr, "Number of SDMA descriptor before interrupt"); + #define SDMA_WAIT_BATCH_SIZE 20 /* max wait time for a SDMA engine to indicate it has halted */ #define SDMA_ERR_HALT_TIMEOUT 10 /* ms */ @@ -1046,6 +1050,8 @@ int sdma_init(struct hfi1_devdata *dd, u8 port) return -ENOMEM; idle_cnt = ns_to_cclock(dd, idle_cnt); + if (!sdma_desct_intr) + sdma_desct_intr = descq_cnt / 2; /* Allocate memory for SendDMA descriptor FIFOs */ for (this_idx = 0; this_idx < num_engines; ++this_idx) { sde = &dd->per_sdma[this_idx]; @@ -1548,7 +1554,7 @@ void sdma_engine_interrupt(struct sdma_engine *sde, u64 status) { trace_hfi1_sdma_engine_interrupt(sde, status); write_seqlock(&sde->head_lock); - sdma_set_desc_cnt(sde, sde->descq_cnt / 2); + sdma_set_desc_cnt(sde, sdma_desct_intr); sdma_make_progress(sde, status); write_sequnlock(&sde->head_lock); }