From patchwork Wed Jul 12 11:59:52 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ranjan Kumar X-Patchwork-Id: 13310261 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6EAE2EB64D9 for ; Wed, 12 Jul 2023 12:01:29 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231932AbjGLMB2 (ORCPT ); Wed, 12 Jul 2023 08:01:28 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:52352 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233317AbjGLMBI (ORCPT ); Wed, 12 Jul 2023 08:01:08 -0400 Received: from mail-pf1-x42f.google.com (mail-pf1-x42f.google.com [IPv6:2607:f8b0:4864:20::42f]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id BBE481FEC for ; Wed, 12 Jul 2023 05:01:02 -0700 (PDT) Received: by mail-pf1-x42f.google.com with SMTP id d2e1a72fcca58-6686a05bc66so4082244b3a.1 for ; Wed, 12 Jul 2023 05:01:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=broadcom.com; s=google; t=1689163261; x=1691755261; h=mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:from:to:cc:subject:date:message-id:reply-to; bh=vvpM38+QvYjMBuq/IukFPvZoGADStcCeYzevXxdeN80=; b=eB6Vw679pF9DnECwc5qur+L6/Rs080e0IRKPFYHP0qPSgCmQEqLA6rdGH28FVD9FI6 sgrlDyHuqCssoasqrDd5GQglhu3NvYghHDxYDFQD3NYyS19fVRJclcQdEKSAvWSYfOEy yBGdBGYP0CVB9fChVfrtSMylN6rCAQHRBbTls= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1689163261; x=1691755261; h=mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=vvpM38+QvYjMBuq/IukFPvZoGADStcCeYzevXxdeN80=; b=jkD4eoZUHBmATna8jpKlTTo/I3J9L5f4f8DQijCpwx/5RQXOeaq/2A9OTbV2E0vWok 9PP3bwgSwqbYxVUICkuunnCMF+kH08vD+Opb61DV5fwOIqrODvZTxOUGxmW8TxN0j9ju Y9Jw2BpoRtZ+JO4aVSEwqFKa9rD3LCcgGo/xGEq1Kbf429Q5/nW1UyCFR5AykVDM9XuS 3076FCgyWBmQZvqn0lRg5vt8bglcyDBonfIBZv7gRRxjW1p39B6ccU6Ld5QA0Mt6xSJt UVjHwHAlvIMNanXlzIlk6N9r95V9PZ65rF7WFYbIcaozXhek+7aBAJMf+oqX0yRtvbU/ k/hQ== X-Gm-Message-State: ABy/qLYul5Lo5XNGVsqis6OGoVJlcAaV5sWs8/lIQxzpnQTEU4gOes3F cAm/lIaVCwSvFd7/mK98SgH76aiKcVjtpNAra6xAPWnwG9S/odY5HzwXpImV8l3l0wVXvkCdJk3 vym42nt6r+vlJIOi/nzJixJ/U5yrbL2pTzayNmbcS8LxMVRVBhsTjtmyGCJQoGfx7KIYB1yQFLE 5ixQpOC/KZaQ== X-Google-Smtp-Source: APBJJlHEdEk/kqFzXDzbDewR5+8GLo9f1wEH6BMZ0EUGX7bR5NfdrvUbRrPOFaxBdFPoQDuWqJaEbQ== X-Received: by 2002:a05:6a21:6d87:b0:121:bda6:2f85 with SMTP id wl7-20020a056a216d8700b00121bda62f85mr18746684pzb.30.1689163261182; Wed, 12 Jul 2023 05:01:01 -0700 (PDT) Received: from localhost.localdomain ([192.19.234.250]) by smtp.gmail.com with ESMTPSA id x7-20020a1709029a4700b001b531e8a000sm3799627plv.157.2023.07.12.05.00.58 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 12 Jul 2023 05:01:00 -0700 (PDT) From: Ranjan Kumar To: linux-scsi@vger.kernel.org, martin.petersen@oracle.com Cc: rajsekhar.chundru@broadcom.com, sathya.prakash@broadcom.com, sumit.saxena@broadcom.com, sreekanth.reddy@broadcom.com, chandrakanth.patil@broadcom.com, Ranjan Kumar Subject: [PATCH 3/6] mpi3mr: Add support for more than 1MB I/O Date: Wed, 12 Jul 2023 17:29:52 +0530 Message-Id: <20230712115955.6312-4-ranjan.kumar@broadcom.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20230712115955.6312-1-ranjan.kumar@broadcom.com> References: <20230712115955.6312-1-ranjan.kumar@broadcom.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org The driver is enhanced to get the maximum data length per I/O request from IOC Facts data and report that to the upper layers. If the IOC facts data is not reported then the default I/O size of 1MB is reported to the OS. Signed-off-by: Ranjan Kumar --- drivers/scsi/mpi3mr/mpi3mr.h | 8 ++++++-- drivers/scsi/mpi3mr/mpi3mr_fw.c | 29 ++++++++++++++++++++++++----- drivers/scsi/mpi3mr/mpi3mr_os.c | 24 ++++++++++++++++++++---- 3 files changed, 50 insertions(+), 11 deletions(-) diff --git a/drivers/scsi/mpi3mr/mpi3mr.h b/drivers/scsi/mpi3mr/mpi3mr.h index 0afb687402e1..fd3619775739 100644 --- a/drivers/scsi/mpi3mr/mpi3mr.h +++ b/drivers/scsi/mpi3mr/mpi3mr.h @@ -66,11 +66,12 @@ extern atomic64_t event_counter; #define MPI3MR_NAME_LENGTH 32 #define IOCNAME "%s: " -#define MPI3MR_MAX_SECTORS 2048 +#define MPI3MR_DEFAULT_MAX_IO_SIZE (1 * 1024 * 1024) /* Definitions for internal SGL and Chain SGL buffers */ #define MPI3MR_PAGE_SIZE_4K 4096 -#define MPI3MR_SG_DEPTH (MPI3MR_PAGE_SIZE_4K / sizeof(struct mpi3_sge_common)) +#define MPI3MR_DEFAULT_SGL_ENTRIES 256 +#define MPI3MR_MAX_SGL_ENTRIES 2048 /* Definitions for MAX values for shost */ #define MPI3MR_MAX_CMDS_LUN 128 @@ -323,6 +324,7 @@ struct mpi3mr_ioc_facts { u16 max_perids; u16 max_pds; u16 max_sasexpanders; + u32 max_data_length; u16 max_sasinitiators; u16 max_enclosures; u16 max_pcie_switches; @@ -959,6 +961,7 @@ struct scmd_priv { * @stop_drv_processing: Stop all command processing * @device_refresh_on: Don't process the events until devices are refreshed * @max_host_ios: Maximum host I/O count + * @max_sgl_entries: Max SGL entries per I/O * @chain_buf_count: Chain buffer count * @chain_buf_pool: Chain buffer pool * @chain_sgl_list: Chain SGL list @@ -1129,6 +1132,7 @@ struct mpi3mr_ioc { u16 max_host_ios; spinlock_t tgtdev_lock; struct list_head tgtdev_list; + u16 max_sgl_entries; u32 chain_buf_count; struct dma_pool *chain_buf_pool; diff --git a/drivers/scsi/mpi3mr/mpi3mr_fw.c b/drivers/scsi/mpi3mr/mpi3mr_fw.c index 11b78d4a87a0..f039f1d98647 100644 --- a/drivers/scsi/mpi3mr/mpi3mr_fw.c +++ b/drivers/scsi/mpi3mr/mpi3mr_fw.c @@ -1163,6 +1163,12 @@ mpi3mr_revalidate_factsdata(struct mpi3mr_ioc *mrioc) return -EPERM; } + if (mrioc->shost->max_sectors != (mrioc->facts.max_data_length / 512)) + ioc_err(mrioc, "Warning: The maximum data transfer length\n" + "\tchanged after reset: previous(%d), new(%d),\n" + "the driver cannot change this at run time\n", + mrioc->shost->max_sectors * 512, mrioc->facts.max_data_length); + if ((mrioc->sas_transport_enabled) && (mrioc->facts.ioc_capabilities & MPI3_IOCFACTS_CAPABILITY_MULTIPATH_ENABLED)) ioc_err(mrioc, @@ -2856,6 +2862,7 @@ static void mpi3mr_process_factsdata(struct mpi3mr_ioc *mrioc, le16_to_cpu(facts_data->max_pcie_switches); mrioc->facts.max_sasexpanders = le16_to_cpu(facts_data->max_sas_expanders); + mrioc->facts.max_data_length = le16_to_cpu(facts_data->max_data_length); mrioc->facts.max_sasinitiators = le16_to_cpu(facts_data->max_sas_initiators); mrioc->facts.max_enclosures = le16_to_cpu(facts_data->max_enclosures); @@ -2893,13 +2900,18 @@ static void mpi3mr_process_factsdata(struct mpi3mr_ioc *mrioc, mrioc->facts.io_throttle_high = le16_to_cpu(facts_data->io_throttle_high); + if (mrioc->facts.max_data_length == + MPI3_IOCFACTS_MAX_DATA_LENGTH_NOT_REPORTED) + mrioc->facts.max_data_length = MPI3MR_DEFAULT_MAX_IO_SIZE; + else + mrioc->facts.max_data_length *= MPI3MR_PAGE_SIZE_4K; /* Store in 512b block count */ if (mrioc->facts.io_throttle_data_length) mrioc->io_throttle_data_length = (mrioc->facts.io_throttle_data_length * 2 * 4); else /* set the length to 1MB + 1K to disable throttle */ - mrioc->io_throttle_data_length = MPI3MR_MAX_SECTORS + 2; + mrioc->io_throttle_data_length = (mrioc->facts.max_data_length / 512) + 2; mrioc->io_throttle_high = (mrioc->facts.io_throttle_high * 2 * 1024); mrioc->io_throttle_low = (mrioc->facts.io_throttle_low * 2 * 1024); @@ -2914,9 +2926,9 @@ static void mpi3mr_process_factsdata(struct mpi3mr_ioc *mrioc, ioc_info(mrioc, "SGEModMask 0x%x SGEModVal 0x%x SGEModShift 0x%x ", mrioc->facts.sge_mod_mask, mrioc->facts.sge_mod_value, mrioc->facts.sge_mod_shift); - ioc_info(mrioc, "DMA mask %d InitialPE status 0x%x\n", + ioc_info(mrioc, "DMA mask %d InitialPE status 0x%x max_data_len (%d)\n", mrioc->facts.dma_mask, (facts_flags & - MPI3_IOCFACTS_FLAGS_INITIAL_PORT_ENABLE_MASK)); + MPI3_IOCFACTS_FLAGS_INITIAL_PORT_ENABLE_MASK), mrioc->facts.max_data_length); ioc_info(mrioc, "max_dev_per_throttle_group(%d), max_throttle_groups(%d)\n", mrioc->facts.max_dev_per_tg, mrioc->facts.max_io_throttle_group); @@ -3414,7 +3426,14 @@ static int mpi3mr_alloc_chain_bufs(struct mpi3mr_ioc *mrioc) if (!mrioc->chain_sgl_list) goto out_failed; - sz = MPI3MR_PAGE_SIZE_4K; + if (mrioc->max_sgl_entries > (mrioc->facts.max_data_length / + MPI3MR_PAGE_SIZE_4K)) + mrioc->max_sgl_entries = mrioc->facts.max_data_length / + MPI3MR_PAGE_SIZE_4K; + sz = mrioc->max_sgl_entries * sizeof(struct mpi3_sge_common); + ioc_info(mrioc, "number of sgl entries=%d chain buffer size=%dKB\n", + mrioc->max_sgl_entries, sz/1024); + mrioc->chain_buf_pool = dma_pool_create("chain_buf pool", &mrioc->pdev->dev, sz, 16, 0); if (!mrioc->chain_buf_pool) { @@ -3813,7 +3832,7 @@ int mpi3mr_init_ioc(struct mpi3mr_ioc *mrioc) } mrioc->max_host_ios = mrioc->facts.max_reqs - MPI3MR_INTERNAL_CMDS_RESVD; - + mrioc->shost->max_sectors = mrioc->facts.max_data_length / 512; mrioc->num_io_throttle_group = mrioc->facts.max_io_throttle_group; atomic_set(&mrioc->pend_large_data_sz, 0); diff --git a/drivers/scsi/mpi3mr/mpi3mr_os.c b/drivers/scsi/mpi3mr/mpi3mr_os.c index d627355303d7..b19b624d0e97 100644 --- a/drivers/scsi/mpi3mr/mpi3mr_os.c +++ b/drivers/scsi/mpi3mr/mpi3mr_os.c @@ -33,6 +33,12 @@ static int logging_level; module_param(logging_level, int, 0); MODULE_PARM_DESC(logging_level, " bits for enabling additional logging info (default=0)"); +static int max_sgl_entries = MPI3MR_DEFAULT_SGL_ENTRIES; +module_param(max_sgl_entries, int, 0444); +MODULE_PARM_DESC(max_sgl_entries, + "Preferred max number of SG entries to be used for a single I/O\n" + "The actual value will be determined by the driver\n" + "(Minimum=256, Maximum=2048, default=256)"); /* Forward declarations*/ static void mpi3mr_send_event_ack(struct mpi3mr_ioc *mrioc, u8 event, @@ -3413,7 +3419,7 @@ static int mpi3mr_prepare_sg_scmd(struct mpi3mr_ioc *mrioc, scsi_bufflen(scmd)); return -ENOMEM; } - if (sges_left > MPI3MR_SG_DEPTH) { + if (sges_left > mrioc->max_sgl_entries) { sdev_printk(KERN_ERR, scmd->device, "scsi_dma_map returned unsupported sge count %d!\n", sges_left); @@ -4818,10 +4824,10 @@ static const struct scsi_host_template mpi3mr_driver_template = { .no_write_same = 1, .can_queue = 1, .this_id = -1, - .sg_tablesize = MPI3MR_SG_DEPTH, + .sg_tablesize = MPI3MR_DEFAULT_SGL_ENTRIES, /* max xfer supported is 1M (2K in 512 byte sized sectors) */ - .max_sectors = 2048, + .max_sectors = (MPI3MR_DEFAULT_MAX_IO_SIZE / 512), .cmd_per_lun = MPI3MR_MAX_CMDS_LUN, .max_segment_size = 0xffffffff, .track_queue_depth = 1, @@ -5004,6 +5010,16 @@ mpi3mr_probe(struct pci_dev *pdev, const struct pci_device_id *id) mrioc->pdev = pdev; mrioc->stop_bsgs = 1; + mrioc->max_sgl_entries = max_sgl_entries; + if (max_sgl_entries > MPI3MR_MAX_SGL_ENTRIES) + mrioc->max_sgl_entries = MPI3MR_MAX_SGL_ENTRIES; + else if (max_sgl_entries < MPI3MR_DEFAULT_SGL_ENTRIES) + mrioc->max_sgl_entries = MPI3MR_DEFAULT_SGL_ENTRIES; + else { + mrioc->max_sgl_entries /= MPI3MR_DEFAULT_SGL_ENTRIES; + mrioc->max_sgl_entries *= MPI3MR_DEFAULT_SGL_ENTRIES; + } + /* init shost parameters */ shost->max_cmd_len = MPI3MR_MAX_CDB_LENGTH; shost->max_lun = -1; @@ -5068,7 +5084,7 @@ mpi3mr_probe(struct pci_dev *pdev, const struct pci_device_id *id) shost->nr_maps = 3; shost->can_queue = mrioc->max_host_ios; - shost->sg_tablesize = MPI3MR_SG_DEPTH; + shost->sg_tablesize = mrioc->max_sgl_entries; shost->max_id = mrioc->facts.max_perids + 1; retval = scsi_add_host(shost, &pdev->dev);