From patchwork Wed Oct 14 12:26:29 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kashyap Desai X-Patchwork-Id: 11837591 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 3B1411592 for ; Wed, 14 Oct 2020 12:26:57 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 05EC020848 for ; Wed, 14 Oct 2020 12:26:56 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=broadcom.com header.i=@broadcom.com header.b="gAbDVWS2" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1730170AbgJNM04 (ORCPT ); Wed, 14 Oct 2020 08:26:56 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40630 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1730097AbgJNM0z (ORCPT ); Wed, 14 Oct 2020 08:26:55 -0400 Received: from mail-ej1-x644.google.com (mail-ej1-x644.google.com [IPv6:2a00:1450:4864:20::644]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6E51CC0613D2 for ; Wed, 14 Oct 2020 05:26:55 -0700 (PDT) Received: by mail-ej1-x644.google.com with SMTP id h24so4521848ejg.9 for ; Wed, 14 Oct 2020 05:26:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=broadcom.com; s=google; h=from:to:cc:subject:date:message-id; bh=IjKh0FaFgmuuZc9CkOcgT87lCaWqpTRvHPDaIWdNegc=; b=gAbDVWS2l0Sxtn+QAq5lOe/4lvJI5u0wy5YQezRHu4ExJjyTKDbN1mgSihNVAcgy/E 7oi5n6R3ocDYw+DYlldoo1zwdh+ow//MWTYvR1/LloqUAUzoJMdfOb9Gurmk8Vcy12Ky LPRTKizbrRIITCZajLCytz4iRyCk/rLS2YROY= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id; bh=IjKh0FaFgmuuZc9CkOcgT87lCaWqpTRvHPDaIWdNegc=; b=kyEWaezcL45SquNNwQzze5nHA3HnzwhHyH7FydaA57CjLefYbRPQ/rgD9kcgdYnFyv Bm2HLQgg0Il66SPBXL6PufDESIfB4Xe8AX0mjtJzcttFf4lQ4k8UGUU5vsu1XWW/wJzY LLpN0F0pebK4LKU7Ijm4mQDZ6UfVJUGY3qJV5cM2EKdhAVvt8FqjCi/nmpzGAAlo3xtO Mw1xsrq82XWfv4pUsc6LYyZz7ElqAa3t4oMu8DzJqf39Hs/jCPYSv3/KmO79v+FQRiRH l5+dE+d6MG9w7xXCs27PnoAtkLaNVVD+NLf5uE4S+6l92A9k4QsGs3q0F13v5PvuslUJ RbJg== X-Gm-Message-State: AOAM5337JB4p5NuLQ5Pyqur4t37287VpSkB8OyUbkqOCZa7a7UIZFC6P I2Wzz2Z8TV8uKSfZRSas0Ztvac6mWMAAVYzFsHoxVzMjQwOvNypailZWOof5pU605amnODe6aTg vPjZfwgDOomSrdA/C3IYpjitXYwBF3fSR1trxeBGT1fwE8wemKd0J+dvg9t2QOm/D9e7tZ6Y2VB 3kksrLSyrgmFI= X-Google-Smtp-Source: ABdhPJzwT9b98N+pkQsDgYgVYPfebGg3/RX/c+RsgBVURQByt57iPpzOdgzTnLRExjGbAH47I/Vpjg== X-Received: by 2002:a17:906:4a97:: with SMTP id x23mr4858751eju.471.1602678413580; Wed, 14 Oct 2020 05:26:53 -0700 (PDT) Received: from drv-bst-rhel8.static.broadcom.net ([192.19.234.250]) by smtp.gmail.com with ESMTPSA id w4sm1532505eds.83.2020.10.14.05.26.50 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 14 Oct 2020 05:26:52 -0700 (PDT) From: Kashyap Desai To: linux-scsi@vger.kernel.org Cc: Kashyap Desai , sumit.saxena@broadcom.com, chandrakanth.patil@broadcom.com, linux-block@vger.kernel.org Subject: [PATCH 1/3] add io_uring with IOPOLL support in scsi layer Date: Wed, 14 Oct 2020 17:56:29 +0530 Message-Id: <20201014122629.201223-1-kashyap.desai@broadcom.com> X-Mailer: git-send-email 2.18.1 Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org io_uring with IOPOLL is not currently supported in scsi mid layer. Outside of that everything else should work and no extra support in the driver is needed. Currently io_uring with IOPOLL support is only available in block layer. This patch is to extend support of mq_poll in scsi layer. Signed-off-by: Kashyap Desai Cc: sumit.saxena@broadcom.com Cc: chandrakanth.patil@broadcom.com Cc: linux-block@vger.kernel.org --- drivers/scsi/scsi_lib.c | 16 ++++++++++++++++ include/scsi/scsi_cmnd.h | 1 + include/scsi/scsi_host.h | 11 +++++++++++ 3 files changed, 28 insertions(+) diff --git a/drivers/scsi/scsi_lib.c b/drivers/scsi/scsi_lib.c index a89478a0c588..0140d9aac64b 100644 --- a/drivers/scsi/scsi_lib.c +++ b/drivers/scsi/scsi_lib.c @@ -1757,6 +1757,19 @@ static void scsi_mq_exit_request(struct blk_mq_tag_set *set, struct request *rq, cmd->sense_buffer); } + +static int scsi_mq_poll(struct blk_mq_hw_ctx *hctx) +{ + struct request_queue *q = hctx->queue; + struct scsi_device *sdev = q->queuedata; + struct Scsi_Host *shost = sdev->host; + + if (shost->hostt->mq_poll) + return shost->hostt->mq_poll(shost, hctx->queue_num); + + return 0; +} + static int scsi_map_queues(struct blk_mq_tag_set *set) { struct Scsi_Host *shost = container_of(set, struct Scsi_Host, tag_set); @@ -1824,6 +1837,7 @@ static const struct blk_mq_ops scsi_mq_ops_no_commit = { .cleanup_rq = scsi_cleanup_rq, .busy = scsi_mq_lld_busy, .map_queues = scsi_map_queues, + .poll = scsi_mq_poll, }; @@ -1852,6 +1866,7 @@ static const struct blk_mq_ops scsi_mq_ops = { .cleanup_rq = scsi_cleanup_rq, .busy = scsi_mq_lld_busy, .map_queues = scsi_map_queues, + .poll = scsi_mq_poll, }; struct request_queue *scsi_mq_alloc_queue(struct scsi_device *sdev) @@ -1884,6 +1899,7 @@ int scsi_mq_setup_tags(struct Scsi_Host *shost) else tag_set->ops = &scsi_mq_ops_no_commit; tag_set->nr_hw_queues = shost->nr_hw_queues ? : 1; + tag_set->nr_maps = shost->nr_maps ? : 1; tag_set->queue_depth = shost->can_queue; tag_set->cmd_size = cmd_size; tag_set->numa_node = NUMA_NO_NODE; diff --git a/include/scsi/scsi_cmnd.h b/include/scsi/scsi_cmnd.h index e76bac4d14c5..5844374a85b1 100644 --- a/include/scsi/scsi_cmnd.h +++ b/include/scsi/scsi_cmnd.h @@ -9,6 +9,7 @@ #include #include #include +#include #include #include diff --git a/include/scsi/scsi_host.h b/include/scsi/scsi_host.h index 701f178b20ae..905ee6b00c55 100644 --- a/include/scsi/scsi_host.h +++ b/include/scsi/scsi_host.h @@ -270,6 +270,16 @@ struct scsi_host_template { */ int (* map_queues)(struct Scsi_Host *shost); + /* + * SCSI interface of blk_poll - poll for IO completions. + * Possible interface only if scsi LLD expose multiple h/w queues. + * + * Return values: Number of completed entries found. + * + * Status: OPTIONAL + */ + int (* mq_poll)(struct Scsi_Host *shost, unsigned int queue_num); + /* * Check if scatterlists need to be padded for DMA draining. * @@ -610,6 +620,7 @@ struct Scsi_Host { * the total queue depth is can_queue. */ unsigned nr_hw_queues; + unsigned nr_maps; unsigned active_mode:2; unsigned unchecked_isa_dma:1; From patchwork Wed Oct 14 12:26:55 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kashyap Desai X-Patchwork-Id: 11837595 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id A65B71592 for ; Wed, 14 Oct 2020 12:27:23 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 7154D21527 for ; Wed, 14 Oct 2020 12:27:23 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=broadcom.com header.i=@broadcom.com header.b="Q/44sR4z" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1730173AbgJNM1W (ORCPT ); Wed, 14 Oct 2020 08:27:22 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40700 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1730076AbgJNM1W (ORCPT ); Wed, 14 Oct 2020 08:27:22 -0400 Received: from mail-ej1-x641.google.com (mail-ej1-x641.google.com [IPv6:2a00:1450:4864:20::641]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 17080C0613D2 for ; Wed, 14 Oct 2020 05:27:22 -0700 (PDT) Received: by mail-ej1-x641.google.com with SMTP id h24so4523652ejg.9 for ; Wed, 14 Oct 2020 05:27:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=broadcom.com; s=google; h=from:to:cc:subject:date:message-id; bh=kx4waEv5Goe3pVSDLkZu50zz+Znyr7IOGOJfV9Jf+4M=; b=Q/44sR4zc5fFzVBa6/WglvzIMztur/C8Dh64sAP96700ptRI2czPZ91kH6v69qObyV ajBUvkIW3i6NLjoGyjECQSW2fJPm32XtBREnrdqVEhRJi9UFMIH2R+o1oZVXAAG1R56d 86u3TtaBeAFfwm/B5qEGpzm1LDZLmWFga9pAg= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id; bh=kx4waEv5Goe3pVSDLkZu50zz+Znyr7IOGOJfV9Jf+4M=; b=HmXIp22ZsW4owUx9N74MJhrVti52IZ6IGx0Ajcc6Jbg3MzVXm9RvMjdg9YtiDA0ZUh 1eyPPTcL4CqXshOsR8/a/n7cDh4QfcKK2Sx5FH75F87Q9Py63aVK4rFzwZtSzuSxsrTL lGcCPjmmnDRSndjI1x2kgxKGd2WbQHWlMnBMAxnivu+fYRernJ/cQvo0TAnSJCxFr4j5 T2g+DRnSSPZBoPSuiUpWB06LYVlOjZ1f1wBJRY4TA14YSDaRaZOnsv2EmJttO2bBrPPx qkcfBDS87KcgW+M5yBrNSXkSDqGhvcCuB9tn6bZ3jwt3nszHxCTwA1T+N0J/U7i2cUUf t/bA== X-Gm-Message-State: AOAM530FRPrU3OEaMpSFcysqsruK9Rk7t5b28OY3w9+6p74w0etDF1Lo Z0JgZdrCCVRWeJnUw1Xv1evJBOXb03uwMVQHK4GJ7b4eJlf2BPkmnui0qOepivNFoTahbndhOuo 0yLdDXuUaKicKJxgM8o1TczBy3i4Jtcd5Cw8H7MBM7CJqHouqEXQxUHxjFuZ5qzV/PPnj+UhiGN Xas1esBhA+yaU= X-Google-Smtp-Source: ABdhPJxNLJKaMwJd1yL3KpGxW4/Ct33sHCHdjl8tiSSKUY0eRNcPjo5OpG7lUt5H27O2m7jx5aPGAg== X-Received: by 2002:a17:906:4941:: with SMTP id f1mr5193299ejt.417.1602678440182; Wed, 14 Oct 2020 05:27:20 -0700 (PDT) Received: from drv-bst-rhel8.static.broadcom.net ([192.19.234.250]) by smtp.gmail.com with ESMTPSA id pw17sm1619676ejb.116.2020.10.14.05.27.16 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 14 Oct 2020 05:27:18 -0700 (PDT) From: Kashyap Desai To: linux-scsi@vger.kernel.org Cc: Kashyap Desai , sumit.saxena@broadcom.com, chandrakanth.patil@broadcom.com, linux-block@vger.kernel.org Subject: [PATCH 2/3] megaraid_sas: iouring iopoll support Date: Wed, 14 Oct 2020 17:56:55 +0530 Message-Id: <20201014122655.201272-1-kashyap.desai@broadcom.com> X-Mailer: git-send-email 2.18.1 Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org Add support of iouring iopoll interface. This feature requires shared hosttag support in kernel and driver. Driver will work in non-IRQ mode = There will not be any msix vector associated for poll_queues and h/w can still work in this mode. MegaRaid h/w is single submission queue and multiple reply queue, but using shared host tagset support it will enable simulated multiple hw queue. Driver allocates some extra reply queues and it will be marked as poll_queue. These poll_queues will not have associated msix vectors. All the IO completion on this queue will be done from IOPOLL interface. megaraid_sas driver having 8 poll_queues and using io_uring hiprio=1 settings, It can reach 3.2M IOPs and there is zero interrupt generated by h/w. This feature can be enabled using module parameter poll_queues. Signed-off-by: Kashyap Desai Cc: sumit.saxena@broadcom.com Cc: chandrakanth.patil@broadcom.com Cc: linux-block@vger.kernel.org Reported-by: kernel test robot Reported-by: kernel test robot --- drivers/scsi/megaraid/megaraid_sas.h | 2 + drivers/scsi/megaraid/megaraid_sas_base.c | 90 ++++++++++++++++++--- drivers/scsi/megaraid/megaraid_sas_fusion.c | 43 +++++++++- drivers/scsi/megaraid/megaraid_sas_fusion.h | 3 + 4 files changed, 127 insertions(+), 11 deletions(-) diff --git a/drivers/scsi/megaraid/megaraid_sas.h b/drivers/scsi/megaraid/megaraid_sas.h index 5e4137f10e0e..4bf6fde49565 100644 --- a/drivers/scsi/megaraid/megaraid_sas.h +++ b/drivers/scsi/megaraid/megaraid_sas.h @@ -2212,6 +2212,7 @@ struct megasas_irq_context { struct irq_poll irqpoll; bool irq_poll_scheduled; bool irq_line_enable; + atomic_t in_used; }; struct MR_DRV_SYSTEM_INFO { @@ -2446,6 +2447,7 @@ struct megasas_instance { bool support_pci_lane_margining; u8 low_latency_index_start; int perf_mode; + int iopoll_q_count; }; struct MR_LD_VF_MAP { diff --git a/drivers/scsi/megaraid/megaraid_sas_base.c b/drivers/scsi/megaraid/megaraid_sas_base.c index 020270ce790b..e2c307523a8a 100644 --- a/drivers/scsi/megaraid/megaraid_sas_base.c +++ b/drivers/scsi/megaraid/megaraid_sas_base.c @@ -114,6 +114,15 @@ unsigned int enable_sdev_max_qd; module_param(enable_sdev_max_qd, int, 0444); MODULE_PARM_DESC(enable_sdev_max_qd, "Enable sdev max qd as can_queue. Default: 0"); +int poll_queues; +module_param(poll_queues, int, 0444); +MODULE_PARM_DESC(poll_queues, "Number of queues to be use for io_uring poll mode.\n\t\t" + "This parameter is effective only if host_tagset_enable=1 &\n\t\t" + "It is not applicable for MFI_SERIES. &\n\t\t" + "Driver will work in latency mode. &\n\t\t" + "High iops queues are not allocated &\n\t\t" + ); + int host_tagset_enable = 1; module_param(host_tagset_enable, int, 0444); MODULE_PARM_DESC(host_tagset_enable, "Shared host tagset enable/disable Default: enable(1)"); @@ -207,6 +216,7 @@ static bool support_pci_lane_margining; static spinlock_t poll_aen_lock; extern struct dentry *megasas_debugfs_root; +extern int megasas_blk_mq_poll(struct Scsi_Host *shost, unsigned int queue_num); void megasas_complete_cmd(struct megasas_instance *instance, struct megasas_cmd *cmd, @@ -3127,14 +3137,45 @@ megasas_bios_param(struct scsi_device *sdev, struct block_device *bdev, static int megasas_map_queues(struct Scsi_Host *shost) { struct megasas_instance *instance; + int i, qoff, offset; instance = (struct megasas_instance *)shost->hostdata; if (shost->nr_hw_queues == 1) return 0; - return blk_mq_pci_map_queues(&shost->tag_set.map[HCTX_TYPE_DEFAULT], - instance->pdev, instance->low_latency_index_start); + offset = instance->low_latency_index_start; + + for (i = 0, qoff = 0; i < HCTX_MAX_TYPES; i++) { + struct blk_mq_queue_map *map = &shost->tag_set.map[i]; + + map->nr_queues = 0; + + if (i == HCTX_TYPE_DEFAULT) + map->nr_queues = instance->msix_vectors - offset; + else if (i == HCTX_TYPE_POLL) + map->nr_queues = instance->iopoll_q_count; + + if (!map->nr_queues) { + BUG_ON(i == HCTX_TYPE_DEFAULT); + continue; + } + + /* + * The poll queue(s) doesn't have an IRQ (and hence IRQ + * affinity), so use the regular blk-mq cpu mapping + */ + map->queue_offset = qoff; + if (i != HCTX_TYPE_POLL) + blk_mq_pci_map_queues(map, instance->pdev, offset); + else + blk_mq_map_queues(map); + + qoff += map->nr_queues; + offset += map->nr_queues; + } + + return 0; } static void megasas_aen_polling(struct work_struct *work); @@ -3446,6 +3487,7 @@ static struct scsi_host_template megasas_template = { .shost_attrs = megaraid_host_attrs, .bios_param = megasas_bios_param, .map_queues = megasas_map_queues, + .mq_poll = megasas_blk_mq_poll, .change_queue_depth = scsi_change_queue_depth, .max_segment_size = 0xffffffff, }; @@ -5834,13 +5876,16 @@ __megasas_alloc_irq_vectors(struct megasas_instance *instance) irq_flags = PCI_IRQ_MSIX; if (instance->smp_affinity_enable) - irq_flags |= PCI_IRQ_AFFINITY; + irq_flags |= PCI_IRQ_AFFINITY | PCI_IRQ_ALL_TYPES; else descp = NULL; + /* Do not allocate msix vectors for poll_queues. + * msix_vectors is always within a range of FW supported reply queue. + */ i = pci_alloc_irq_vectors_affinity(instance->pdev, instance->low_latency_index_start, - instance->msix_vectors, irq_flags, descp); + instance->msix_vectors - instance->iopoll_q_count, irq_flags, descp); return i; } @@ -5856,10 +5901,25 @@ megasas_alloc_irq_vectors(struct megasas_instance *instance) int i; unsigned int num_msix_req; + instance->iopoll_q_count = 0; + if ((instance->adapter_type != MFI_SERIES) && + poll_queues) { + + instance->perf_mode = MR_LATENCY_PERF_MODE; + instance->low_latency_index_start = 1; + + /* reserve for default and non-mananged pre-vector. */ + if (instance->msix_vectors > (instance->iopoll_q_count + 2)) + instance->iopoll_q_count = poll_queues; + else + instance->iopoll_q_count = 0; + } + i = __megasas_alloc_irq_vectors(instance); - if ((instance->perf_mode == MR_BALANCED_PERF_MODE) && - (i != instance->msix_vectors)) { + if (((instance->perf_mode == MR_BALANCED_PERF_MODE) + || instance->iopoll_q_count) && + (i != (instance->msix_vectors - instance->iopoll_q_count))) { if (instance->msix_vectors) pci_free_irq_vectors(instance->pdev); /* Disable Balanced IOPS mode and try realloc vectors */ @@ -5870,12 +5930,15 @@ megasas_alloc_irq_vectors(struct megasas_instance *instance) instance->msix_vectors = min(num_msix_req, instance->msix_vectors); + instance->iopoll_q_count = 0; i = __megasas_alloc_irq_vectors(instance); } dev_info(&instance->pdev->dev, - "requested/available msix %d/%d\n", instance->msix_vectors, i); + "requested/available msix %d/%d poll_queue %d\n", + instance->msix_vectors - instance->iopoll_q_count, + i, instance->iopoll_q_count); if (i > 0) instance->msix_vectors = i; @@ -6841,12 +6904,18 @@ static int megasas_io_attach(struct megasas_instance *instance) instance->smp_affinity_enable) { host->host_tagset = 1; host->nr_hw_queues = instance->msix_vectors - - instance->low_latency_index_start; + instance->low_latency_index_start + instance->iopoll_q_count; + if (instance->iopoll_q_count) + host->nr_maps = 3; + } else { + instance->iopoll_q_count = 0; } dev_info(&instance->pdev->dev, - "Max firmware commands: %d shared with nr_hw_queues = %d\n", - instance->max_fw_cmds, host->nr_hw_queues); + "Max firmware commands: %d shared with default " + "hw_queues = %d poll_queues %d\n", instance->max_fw_cmds, + host->nr_hw_queues - instance->iopoll_q_count, + instance->iopoll_q_count); /* * Notify the mid-layer about the new controller */ @@ -8907,6 +8976,7 @@ static int __init megasas_init(void) msix_vectors = 1; rdpq_enable = 0; dual_qdepth_disable = 1; + poll_queues = 0; } /* diff --git a/drivers/scsi/megaraid/megaraid_sas_fusion.c b/drivers/scsi/megaraid/megaraid_sas_fusion.c index fd607287608e..49c8cc3507e0 100644 --- a/drivers/scsi/megaraid/megaraid_sas_fusion.c +++ b/drivers/scsi/megaraid/megaraid_sas_fusion.c @@ -685,6 +685,8 @@ megasas_alloc_reply_fusion(struct megasas_instance *instance) fusion = instance->ctrl_context; count = instance->msix_vectors > 0 ? instance->msix_vectors : 1; + count += instance->iopoll_q_count; + fusion->reply_frames_desc_pool = dma_pool_create("mr_reply", &instance->pdev->dev, fusion->reply_alloc_sz * count, 16, 0); @@ -779,6 +781,7 @@ megasas_alloc_rdpq_fusion(struct megasas_instance *instance) } msix_count = instance->msix_vectors > 0 ? instance->msix_vectors : 1; + msix_count += instance->iopoll_q_count; fusion->reply_frames_desc_pool = dma_pool_create("mr_rdpq", &instance->pdev->dev, @@ -1129,7 +1132,7 @@ megasas_ioc_init_fusion(struct megasas_instance *instance) MPI2_IOCINIT_MSGFLAG_RDPQ_ARRAY_MODE : 0; IOCInitMessage->SystemRequestFrameBaseAddress = cpu_to_le64(fusion->io_request_frames_phys); IOCInitMessage->SenseBufferAddressHigh = cpu_to_le32(upper_32_bits(fusion->sense_phys_addr)); - IOCInitMessage->HostMSIxVectors = instance->msix_vectors; + IOCInitMessage->HostMSIxVectors = instance->msix_vectors + instance->iopoll_q_count; IOCInitMessage->HostPageSize = MR_DEFAULT_NVME_PAGE_SHIFT; time = ktime_get_real(); @@ -1823,6 +1826,8 @@ megasas_init_adapter_fusion(struct megasas_instance *instance) sizeof(union MPI2_SGE_IO_UNION))/16; count = instance->msix_vectors > 0 ? instance->msix_vectors : 1; + count += instance->iopoll_q_count; + for (i = 0 ; i < count; i++) fusion->last_reply_idx[i] = 0; @@ -1835,6 +1840,9 @@ megasas_init_adapter_fusion(struct megasas_instance *instance) MEGASAS_FUSION_IOCTL_CMDS); sema_init(&instance->ioctl_sem, MEGASAS_FUSION_IOCTL_CMDS); + for (i = 0; i < MAX_MSIX_QUEUES_FUSION; i++) + atomic_set(&fusion->busy_mq_poll[i], 0); + if (megasas_alloc_ioc_init_frame(instance)) return 1; @@ -3500,6 +3508,9 @@ complete_cmd_fusion(struct megasas_instance *instance, u32 MSIxIndex, if (reply_descript_type == MPI2_RPY_DESCRIPT_FLAGS_UNUSED) return IRQ_NONE; + if (irq_context && !atomic_add_unless(&irq_context->in_used, 1, 1)) + return 0; + num_completed = 0; while (d_val.u.low != cpu_to_le32(UINT_MAX) && @@ -3613,6 +3624,7 @@ complete_cmd_fusion(struct megasas_instance *instance, u32 MSIxIndex, irq_context->irq_line_enable = true; irq_poll_sched(&irq_context->irqpoll); } + atomic_dec(&irq_context->in_used); return num_completed; } } @@ -3630,9 +3642,36 @@ complete_cmd_fusion(struct megasas_instance *instance, u32 MSIxIndex, instance->reply_post_host_index_addr[0]); megasas_check_and_restore_queue_depth(instance); } + + if (irq_context) + atomic_dec(&irq_context->in_used); + return num_completed; } +int megasas_blk_mq_poll(struct Scsi_Host *shost, unsigned int queue_num) +{ + + struct megasas_instance *instance; + int num_entries = 0; + struct fusion_context *fusion; + + instance = (struct megasas_instance *)shost->hostdata; + + fusion = instance->ctrl_context; + + queue_num = queue_num + instance->low_latency_index_start; + + if (!atomic_add_unless(&fusion->busy_mq_poll[queue_num], 1, 1)) + return 0; + + num_entries = complete_cmd_fusion(instance, queue_num, NULL); + atomic_dec(&fusion->busy_mq_poll[queue_num]); + + return num_entries; +} + + /** * megasas_enable_irq_poll() - enable irqpoll * @instance: Adapter soft state @@ -4164,6 +4203,8 @@ void megasas_reset_reply_desc(struct megasas_instance *instance) fusion = instance->ctrl_context; count = instance->msix_vectors > 0 ? instance->msix_vectors : 1; + count += instance->iopoll_q_count; + for (i = 0 ; i < count ; i++) { fusion->last_reply_idx[i] = 0; reply_desc = fusion->reply_frames_desc[i]; diff --git a/drivers/scsi/megaraid/megaraid_sas_fusion.h b/drivers/scsi/megaraid/megaraid_sas_fusion.h index 30de4b01f703..242ff58a3404 100644 --- a/drivers/scsi/megaraid/megaraid_sas_fusion.h +++ b/drivers/scsi/megaraid/megaraid_sas_fusion.h @@ -1303,6 +1303,9 @@ struct fusion_context { u8 *sense; dma_addr_t sense_phys_addr; + atomic_t busy_mq_poll[MAX_MSIX_QUEUES_FUSION]; + + dma_addr_t reply_frames_desc_phys[MAX_MSIX_QUEUES_FUSION]; union MPI2_REPLY_DESCRIPTORS_UNION *reply_frames_desc[MAX_MSIX_QUEUES_FUSION]; struct rdpq_alloc_detail rdpq_tracker[RDPQ_MAX_CHUNK_COUNT]; From patchwork Wed Oct 14 12:27:34 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kashyap Desai X-Patchwork-Id: 11837599 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 2532914B2 for ; Wed, 14 Oct 2020 12:28:03 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id ECAA920848 for ; Wed, 14 Oct 2020 12:28:02 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=broadcom.com header.i=@broadcom.com header.b="JFX5nxCo" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1730205AbgJNM2A (ORCPT ); Wed, 14 Oct 2020 08:28:00 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40798 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1730076AbgJNM2A (ORCPT ); Wed, 14 Oct 2020 08:28:00 -0400 Received: from mail-ej1-x643.google.com (mail-ej1-x643.google.com [IPv6:2a00:1450:4864:20::643]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B893CC0613D2 for ; Wed, 14 Oct 2020 05:27:59 -0700 (PDT) Received: by mail-ej1-x643.google.com with SMTP id md26so4523373ejb.10 for ; Wed, 14 Oct 2020 05:27:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=broadcom.com; s=google; h=from:to:cc:subject:date:message-id; bh=qpkr1vSvoZSuVFSKn3fAQOkUnJKGcmV6cJlH0K5bk80=; b=JFX5nxCotCZSaZ4MBPYdS0OofAG0ltouu3DddnvZjuT1nnnsK0stOT/Q8gMOzx570J R0x3LdBjMGTxdvUVKGWLa2AR+z9CdWGch3AUOIJlUFpRR2USTQcAZJUjtq5DbK7A/I0h D2+5ZZraakJKiMQuK2JZ5y1NpgjGYsJOtRBU8= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id; bh=qpkr1vSvoZSuVFSKn3fAQOkUnJKGcmV6cJlH0K5bk80=; b=E0INr0jf8i7HVoAYWH3PNXtbyZXPstBE7UVrwC/496b0tbzbnixcHZEkVW4G5MPqVe akr+IMwPwFlMIhYkZWykYMaqm2La6bhxvo+Wgp9vpnhENFo55hTpXRA62jPZeAN71z6w AdcTH09CfuDmq4YesaZj1P6qItOEpTcu85CEouhp2sqYVsTwzPc1B6rUYY4j9H3oKzxN PahjtyQRPrjqLC5FRhoq1BWWBnNCkfZbspeJfH/eJrverTKr4H73DDdOg4RH2DziAXGJ 9VcAkttnv/KlTnu46hov/ful8Dsavr6wobYvM6vR3JSETNBARYDSV90n1ucTiIphs0KO cYTg== X-Gm-Message-State: AOAM532SQwAob6UA+Bj7YuLn+LpNgVLrixjRc1UOPMGl+AsM93DSPc6F frMTuGYpdminUvK4M4vMEJzEyD5TPwo2KhkGupQ6fPnjEtrCh+zqYxKP1vmzPDpkaOAMqV3KYAn 6pQJr+0z+7quaS2v679Or9FQZvN23UzpQyQSVWCelbNQiW0ckpjlCf8loj8zOa8JqTCthJ1fwY3 IViQTPtZpkJpo= X-Google-Smtp-Source: ABdhPJxa8YXtCHj8mJYiSVYJdNLmyfslgoeV30FJ03rrdJPJO//OJUSKcMKWQm+slRhgUkK/yHK/gQ== X-Received: by 2002:a17:906:6bce:: with SMTP id t14mr5194238ejs.118.1602678477888; Wed, 14 Oct 2020 05:27:57 -0700 (PDT) Received: from drv-bst-rhel8.static.broadcom.net ([192.19.234.250]) by smtp.gmail.com with ESMTPSA id k21sm1505779edv.31.2020.10.14.05.27.55 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 14 Oct 2020 05:27:57 -0700 (PDT) From: Kashyap Desai To: linux-scsi@vger.kernel.org Cc: Kashyap Desai , dgilbert@interlog.com, linux-block@vger.kernel.org Subject: [PATCH 3/3] scsi_debug: iouring iopoll support Date: Wed, 14 Oct 2020 17:57:34 +0530 Message-Id: <20201014122734.201426-1-kashyap.desai@broadcom.com> X-Mailer: git-send-email 2.18.1 Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org Add support of iouring iopoll interface in scsi_debug. This feature requires shared hosttag support in kernel and driver. Signed-off-by: Kashyap Desai Cc: dgilbert@interlog.com Cc: linux-block@vger.kernel.org Reported-by: kernel test robot Reported-by: kernel test robot --- drivers/scsi/scsi_debug.c | 123 ++++++++++++++++++++++++++++++++++++++ 1 file changed, 123 insertions(+) diff --git a/drivers/scsi/scsi_debug.c b/drivers/scsi/scsi_debug.c index a87e40aec11f..4e441aaa60db 100644 --- a/drivers/scsi/scsi_debug.c +++ b/drivers/scsi/scsi_debug.c @@ -826,6 +826,7 @@ static int sdeb_zbc_max_open = DEF_ZBC_MAX_OPEN_ZONES; static int sdeb_zbc_nr_conv = DEF_ZBC_NR_CONV_ZONES; static int submit_queues = DEF_SUBMIT_QUEUES; /* > 1 for multi-queue (mq) */ +static int poll_queues; /* iouring iopoll interface.*/ static struct sdebug_queue *sdebug_q_arr; /* ptr to array of submit queues */ static DEFINE_RWLOCK(atomic_rw); @@ -5422,6 +5423,14 @@ static int schedule_resp(struct scsi_cmnd *cmnd, struct sdebug_dev_info *devip, cmnd->host_scribble = (unsigned char *)sqcp; sd_dp = sqcp->sd_dp; spin_unlock_irqrestore(&sqp->qc_lock, iflags); + + /* Do not complete IO from default completion path. + * Let it to be on queue. + * Completion should happen from mq_poll interface. + */ + if ((sqp - sdebug_q_arr) >= (submit_queues - poll_queues)) + return; + if (!sd_dp) { sd_dp = kzalloc(sizeof(*sd_dp), GFP_ATOMIC); if (!sd_dp) { @@ -5604,6 +5613,7 @@ module_param_named(sector_size, sdebug_sector_size, int, S_IRUGO); module_param_named(statistics, sdebug_statistics, bool, S_IRUGO | S_IWUSR); module_param_named(strict, sdebug_strict, bool, S_IRUGO | S_IWUSR); module_param_named(submit_queues, submit_queues, int, S_IRUGO); +module_param_named(poll_queues, poll_queues, int, S_IRUGO); module_param_named(tur_ms_to_ready, sdeb_tur_ms_to_ready, int, S_IRUGO); module_param_named(unmap_alignment, sdebug_unmap_alignment, int, S_IRUGO); module_param_named(unmap_granularity, sdebug_unmap_granularity, int, S_IRUGO); @@ -5673,6 +5683,7 @@ MODULE_PARM_DESC(sector_size, "logical block size in bytes (def=512)"); MODULE_PARM_DESC(statistics, "collect statistics on commands, queues (def=0)"); MODULE_PARM_DESC(strict, "stricter checks: reserved field in cdb (def=0)"); MODULE_PARM_DESC(submit_queues, "support for block multi-queue (def=1)"); +MODULE_PARM_DESC(poll_queues, "support for iouring iopoll queues"); MODULE_PARM_DESC(tur_ms_to_ready, "TEST UNIT READY millisecs before initial good status (def=0)"); MODULE_PARM_DESC(unmap_alignment, "lowest aligned thin provisioning lba (def=0)"); MODULE_PARM_DESC(unmap_granularity, "thin provisioning granularity in blocks (def=1)"); @@ -7140,6 +7151,104 @@ static int resp_not_ready(struct scsi_cmnd *scp, struct sdebug_dev_info *devip) return check_condition_result; } +static int sdebug_map_queues(struct Scsi_Host *shost) +{ + int i, qoff; + + if (shost->nr_hw_queues == 1) + return 0; + + for (i = 0, qoff = 0; i < HCTX_MAX_TYPES; i++) { + struct blk_mq_queue_map *map = &shost->tag_set.map[i]; + + map->nr_queues = 0; + + if (i == HCTX_TYPE_DEFAULT) + map->nr_queues = submit_queues - poll_queues; + else if (i == HCTX_TYPE_POLL) + map->nr_queues = poll_queues; + + if (!map->nr_queues) { + BUG_ON(i == HCTX_TYPE_DEFAULT); + continue; + } + + map->queue_offset = qoff; + blk_mq_map_queues(map); + + qoff += map->nr_queues; + } + + return 0; + +} + +int sdebug_blk_mq_poll(struct Scsi_Host *shost, unsigned int queue_num) +{ + int qc_idx; + int retiring = 0; + unsigned long iflags; + struct sdebug_queue *sqp; + struct sdebug_queued_cmd *sqcp; + struct scsi_cmnd *scp; + struct sdebug_dev_info *devip; + int num_entries = 0; + + sqp = sdebug_q_arr + queue_num; + + do { + spin_lock_irqsave(&sqp->qc_lock, iflags); + qc_idx = find_first_bit(sqp->in_use_bm, sdebug_max_queue); + if (unlikely((qc_idx < 0) || (qc_idx >= SDEBUG_CANQUEUE))) + goto out; + + sqcp = &sqp->qc_arr[qc_idx]; + scp = sqcp->a_cmnd; + if (unlikely(scp == NULL)) { + pr_err("scp is NULL, queue_num=%d, qc_idx=%d from %s\n", + queue_num, qc_idx, __func__); + goto out; + } + devip = (struct sdebug_dev_info *)scp->device->hostdata; + if (likely(devip)) + atomic_dec(&devip->num_in_q); + else + pr_err("devip=NULL from %s\n", __func__); + if (unlikely(atomic_read(&retired_max_queue) > 0)) + retiring = 1; + + sqcp->a_cmnd = NULL; + if (unlikely(!test_and_clear_bit(qc_idx, sqp->in_use_bm))) { + pr_err("Unexpected completion sqp %p queue_num=%d qc_idx=%d from %s\n", + sqp, queue_num, qc_idx, __func__); + goto out; + } + + if (unlikely(retiring)) { /* user has reduced max_queue */ + int k, retval; + + retval = atomic_read(&retired_max_queue); + if (qc_idx >= retval) { + pr_err("index %d too large\n", retval); + goto out; + } + k = find_last_bit(sqp->in_use_bm, retval); + if ((k < sdebug_max_queue) || (k == retval)) + atomic_set(&retired_max_queue, 0); + else + atomic_set(&retired_max_queue, k + 1); + } + spin_unlock_irqrestore(&sqp->qc_lock, iflags); + scp->scsi_done(scp); /* callback to mid level */ + num_entries++; + } while (1); + +out: + spin_unlock_irqrestore(&sqp->qc_lock, iflags); + return num_entries; +} + + static int scsi_debug_queuecommand(struct Scsi_Host *shost, struct scsi_cmnd *scp) { @@ -7318,6 +7427,8 @@ static struct scsi_host_template sdebug_driver_template = { .ioctl = scsi_debug_ioctl, .queuecommand = scsi_debug_queuecommand, .change_queue_depth = sdebug_change_qdepth, + .map_queues = sdebug_map_queues, + .mq_poll = sdebug_blk_mq_poll, .eh_abort_handler = scsi_debug_abort, .eh_device_reset_handler = scsi_debug_device_reset, .eh_target_reset_handler = scsi_debug_target_reset, @@ -7365,6 +7476,18 @@ static int sdebug_driver_probe(struct device *dev) if (sdebug_host_max_queue) hpnt->host_tagset = 1; + /* poll queues are possible for nr_hw_queues > 1 */ + if (hpnt->nr_hw_queues == 1) + poll_queues = 0; + + /* poll queues */ + if (poll_queues >= submit_queues) { + pr_warn("%s: trim poll_queues to 1\n", my_name); + poll_queues = 1; + } + if (poll_queues) + hpnt->nr_maps = 3; + sdbg_host->shost = hpnt; *((struct sdebug_host_info **)hpnt->hostdata) = sdbg_host; if ((hpnt->this_id >= 0) && (sdebug_num_tgts > hpnt->this_id))