From patchwork Wed Jan 19 04:27:19 2011 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Dillow X-Patchwork-Id: 488151 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by demeter1.kernel.org (8.14.4/8.14.3) with ESMTP id p0J50ubS026308 for ; Wed, 19 Jan 2011 05:01:14 GMT Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1750898Ab1ASFBN (ORCPT ); Wed, 19 Jan 2011 00:01:13 -0500 Received: from emroute4.ornl.gov ([160.91.86.27]:64328 "EHLO emroute4.ornl.gov" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1750872Ab1ASFBM (ORCPT ); Wed, 19 Jan 2011 00:01:12 -0500 Received: from emroute4.ornl.gov ([127.0.0.1]) by emroute4.ornl.gov (PMDF V6.5-x5 #31823) with ESMTP id <0LF9009HO5PQPU@emroute4.ornl.gov> for linux-rdma@vger.kernel.org; Tue, 18 Jan 2011 23:27:26 -0500 (EST) Received: from CONVERSION-DAEMON.emroute4.ornl.gov by emroute4.ornl.gov (PMDF V6.5-x5 #31823) id <0LF900C015PQXA@emroute4.ornl.gov> for linux-rdma@vger.kernel.org; Tue, 18 Jan 2011 23:27:26 -0500 (EST) Received: from lap75545.ornl.gov (lap75545.ornl.gov [160.91.210.54]) by emroute4.ornl.gov (PMDF V6.5-x5 #31823) with ESMTPS id <0LF900B7Z5PQBL@emroute4.ornl.gov> for linux-rdma@vger.kernel.org; Tue, 18 Jan 2011 23:27:26 -0500 (EST) Received: from lap75545.ornl.gov (localhost.localdomain [127.0.0.1]) by lap75545.ornl.gov (8.14.4/8.14.3) with ESMTP id p0J4RPqE026214 for ; Tue, 18 Jan 2011 23:27:25 -0500 Received: (from dad@localhost) by lap75545.ornl.gov (8.14.4/8.14.4/Submit) id p0J4RPrb026211 for linux-rdma@vger.kernel.org; Tue, 18 Jan 2011 23:27:25 -0500 Date: Tue, 18 Jan 2011 23:27:19 -0500 From: David Dillow Subject: [RFC 5/8] IB/srp: add safety valve for large SG tables without HW support In-reply-to: <1295411242-26148-1-git-send-email-dillowda@ornl.gov> To: linux-rdma@vger.kernel.org Message-id: <1295411242-26148-6-git-send-email-dillowda@ornl.gov> X-Mailer: git-send-email 1.7.3.4 References: <1295411242-26148-1-git-send-email-dillowda@ornl.gov> Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org X-Greylist: IP, sender and recipient auto-whitelisted, not delayed by milter-greylist-4.2.6 (demeter1.kernel.org [140.211.167.41]); Wed, 19 Jan 2011 05:01:15 +0000 (UTC) diff --git a/drivers/infiniband/ulp/srp/ib_srp.c b/drivers/infiniband/ulp/srp/ib_srp.c index 9ce129a..df4c3b9 100644 --- a/drivers/infiniband/ulp/srp/ib_srp.c +++ b/drivers/infiniband/ulp/srp/ib_srp.c @@ -61,6 +61,7 @@ MODULE_LICENSE("Dual BSD/GPL"); static unsigned int srp_sg_tablesize; static unsigned int cmd_sg_entries; +static bool allow_ext_sg; static int topspin_workarounds = 1; module_param(srp_sg_tablesize, uint, 0444); @@ -70,6 +71,10 @@ module_param(cmd_sg_entries, uint, 0444); MODULE_PARM_DESC(cmd_sg_entries, "Default number of gather/scatter entries in the SRP command (default is 12, max 255)"); +module_param(allow_ext_sg, bool, 0444); +MODULE_PARM_DESC(allow_ext_sg, + "Default behavior when there are more than cmd_sg_entries S/G entries after mapping; fails the request when false (default false)"); + module_param(topspin_workarounds, int, 0444); MODULE_PARM_DESC(topspin_workarounds, "Enable workarounds for Topspin/Cisco SRP target bugs if != 0"); @@ -885,6 +890,13 @@ backtrack: goto map_complete; } + if (unlikely(target->cmd_sg_cnt < state.ndesc && + !target->allow_ext_sg)) { + shost_printk(KERN_ERR, target->scsi_host, + "Could not fit S/G list into SRP_CMD\n"); + return -EIO; + } + table_len = state.ndesc * sizeof (struct srp_direct_buf); fmt = SRP_DATA_DESC_INDIRECT; @@ -1759,6 +1771,14 @@ static ssize_t show_cmd_sg_entries(struct device *dev, return sprintf(buf, "%u\n", target->cmd_sg_cnt); } +static ssize_t show_allow_ext_sg(struct device *dev, + struct device_attribute *attr, char *buf) +{ + struct srp_target_port *target = host_to_target(class_to_shost(dev)); + + return sprintf(buf, "%s\n", target->allow_ext_sg ? "true" : "false"); +} + static DEVICE_ATTR(id_ext, S_IRUGO, show_id_ext, NULL); static DEVICE_ATTR(ioc_guid, S_IRUGO, show_ioc_guid, NULL); static DEVICE_ATTR(service_id, S_IRUGO, show_service_id, NULL); @@ -1770,6 +1790,7 @@ static DEVICE_ATTR(zero_req_lim, S_IRUGO, show_zero_req_lim, NULL); static DEVICE_ATTR(local_ib_port, S_IRUGO, show_local_ib_port, NULL); static DEVICE_ATTR(local_ib_device, S_IRUGO, show_local_ib_device, NULL); static DEVICE_ATTR(cmd_sg_entries, S_IRUGO, show_cmd_sg_entries, NULL); +static DEVICE_ATTR(allow_ext_sg, S_IRUGO, show_allow_ext_sg, NULL); static struct device_attribute *srp_host_attrs[] = { &dev_attr_id_ext, @@ -1783,6 +1804,7 @@ static struct device_attribute *srp_host_attrs[] = { &dev_attr_local_ib_port, &dev_attr_local_ib_device, &dev_attr_cmd_sg_entries, + &dev_attr_allow_ext_sg, NULL }; @@ -1868,6 +1890,7 @@ enum { SRP_OPT_IO_CLASS = 1 << 7, SRP_OPT_INITIATOR_EXT = 1 << 8, SRP_OPT_CMD_SG_ENTRIES = 1 << 9, + SRP_OPT_ALLOW_EXT_SG = 1 << 10, SRP_OPT_ALL = (SRP_OPT_ID_EXT | SRP_OPT_IOC_GUID | SRP_OPT_DGID | @@ -1886,6 +1909,7 @@ static const match_table_t srp_opt_tokens = { { SRP_OPT_IO_CLASS, "io_class=%x" }, { SRP_OPT_INITIATOR_EXT, "initiator_ext=%s" }, { SRP_OPT_CMD_SG_ENTRIES, "cmd_sg_entries=%u" }, + { SRP_OPT_ALLOW_EXT_SG, "allow_ext_sg=%u" }, { SRP_OPT_ERR, NULL } }; @@ -2021,6 +2045,14 @@ static int srp_parse_options(const char *buf, struct srp_target_port *target) target->cmd_sg_cnt = token; break; + case SRP_OPT_ALLOW_EXT_SG: + if (match_int(args, &token)) { + printk(KERN_WARNING PFX "bad allow_ext_sg parameter '%s'\n", p); + goto out; + } + target->allow_ext_sg = !!token; + break; + default: printk(KERN_WARNING PFX "unknown parameter or missing value " "'%s' in target creation request\n", p); @@ -2070,6 +2102,7 @@ static ssize_t srp_create_target(struct device *dev, target->lkey = host->srp_dev->mr->lkey; target->rkey = host->srp_dev->mr->rkey; target->cmd_sg_cnt = cmd_sg_entries; + target->allow_ext_sg = allow_ext_sg; ret = srp_parse_options(buf, target); if (ret) diff --git a/drivers/infiniband/ulp/srp/ib_srp.h b/drivers/infiniband/ulp/srp/ib_srp.h index b43b5e7..dd7c9fe 100644 --- a/drivers/infiniband/ulp/srp/ib_srp.h +++ b/drivers/infiniband/ulp/srp/ib_srp.h @@ -137,6 +137,7 @@ struct srp_target_port { enum srp_target_state state; unsigned int max_iu_len; unsigned int cmd_sg_cnt; + bool allow_ext_sg; /* Everything above this point is used in the hot path of * command processing. Try to keep them packed into cachelines.