[v2,15/22] staging/rdma/hfi1: Allow tuning of SDMA interrupt rate

Commit Message

Ira Weiny Oct. 20, 2015, 2:11 a.m. UTC

From: Mitko Haralanov <[e-mail]>

The SDMA engines were configured to generate progress interrupts every time they
processed N/2 descriptors (where N is the size of the descriptor queue). This
interval was too infrequent, leading to degraded performance.

This commit adds a module parameter, which allows for the tuning of the
interrupt frequency until an optimal frequency is found for both PSM and Verbs.
At that time, the parameter could be pulled out, if desired.

Reviewed-by: Mike Marciniszyn <[e-mail]>
Signed-off-by: Mitko Haralanov <[e-mail]>
Signed-off-by: Ira Weiny <[e-mail]>
---
 drivers/staging/rdma/hfi1/sdma.c | 8 +++++++-
 1 file changed, 7 insertions(+), 1 deletion(-)

Comments

Dan Carpenter Oct. 22, 2015, 10:54 a.m. UTC | #1

What values work well for this parameter?

regards,
dan carpenter

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to [e-mail]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Ira Weiny Oct. 22, 2015, 10:27 p.m. UTC | #2

On Thu, Oct 22, 2015 at 01:54:21PM +0300, Dan Carpenter wrote:
> What values work well for this parameter?

64.

For v3, I've squashed a change which was farther down my list of changes
which set this after some testing.

Thanks,
Ira

> 
> regards,
> dan carpenter
> 
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to [e-mail]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Patch

diff --git a/drivers/staging/rdma/hfi1/sdma.c b/drivers/staging/rdma/hfi1/sdma.c
index 53b3e4d9518b..452e7edcee7a 100644
--- a/drivers/staging/rdma/hfi1/sdma.c
+++ b/drivers/staging/rdma/hfi1/sdma.c
@@ -80,6 +80,10 @@  uint mod_num_sdma;
 module_param_named(num_sdma, mod_num_sdma, uint, S_IRUGO);
 MODULE_PARM_DESC(num_sdma, "Set max number SDMA engines to use");
 
+static uint sdma_desct_intr;
+module_param_named(desct_intr, sdma_desct_intr, uint, S_IRUGO | S_IWUSR);
+MODULE_PARM_DESC(desct_intr, "Number of SDMA descriptor before interrupt");
+
 #define SDMA_WAIT_BATCH_SIZE 20
 /* max wait time for a SDMA engine to indicate it has halted */
 #define SDMA_ERR_HALT_TIMEOUT 10 /* ms */
@@ -1046,6 +1050,8 @@  int sdma_init(struct hfi1_devdata *dd, u8 port)
 		return -ENOMEM;
 
 	idle_cnt = ns_to_cclock(dd, idle_cnt);
+	if (!sdma_desct_intr)
+		sdma_desct_intr = descq_cnt / 2;
 	/* Allocate memory for SendDMA descriptor FIFOs */
 	for (this_idx = 0; this_idx < num_engines; ++this_idx) {
 		sde = &dd->per_sdma[this_idx];
@@ -1548,7 +1554,7 @@  void sdma_engine_interrupt(struct sdma_engine *sde, u64 status)
 {
 	trace_hfi1_sdma_engine_interrupt(sde, status);
 	write_seqlock(&sde->head_lock);
-	sdma_set_desc_cnt(sde, sde->descq_cnt / 2);
+	sdma_set_desc_cnt(sde, sdma_desct_intr);
 	sdma_make_progress(sde, status);
 	write_sequnlock(&sde->head_lock);
 }