From patchwork Mon Dec 28 21:19:05 2015 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dennis Dalessandro X-Patchwork-Id: 7927441 Return-Path: X-Original-To: patchwork-linux-rdma@patchwork.kernel.org Delivered-To: patchwork-parsemail@patchwork1.web.kernel.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.136]) by patchwork1.web.kernel.org (Postfix) with ESMTP id E23AE9F32E for ; Mon, 28 Dec 2015 21:19:10 +0000 (UTC) Received: from mail.kernel.org (localhost [127.0.0.1]) by mail.kernel.org (Postfix) with ESMTP id EE817201DD for ; Mon, 28 Dec 2015 21:19:09 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id CB8D5201CE for ; Mon, 28 Dec 2015 21:19:08 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752805AbbL1VTH (ORCPT ); Mon, 28 Dec 2015 16:19:07 -0500 Received: from mga11.intel.com ([192.55.52.93]:65518 "EHLO mga11.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752695AbbL1VTG (ORCPT ); Mon, 28 Dec 2015 16:19:06 -0500 Received: from orsmga001.jf.intel.com ([10.7.209.18]) by fmsmga102.fm.intel.com with ESMTP; 28 Dec 2015 13:19:06 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.20,492,1444719600"; d="scan'208";a="849732298" Received: from scymds02.sc.intel.com ([10.82.195.37]) by orsmga001.jf.intel.com with ESMTP; 28 Dec 2015 13:19:05 -0800 Received: from scvm10.sc.intel.com (scvm10.sc.intel.com [10.82.195.27]) by scymds02.sc.intel.com with ESMTP id tBSLJ5FW023691; Mon, 28 Dec 2015 13:19:05 -0800 Received: from scvm10.sc.intel.com (localhost [127.0.0.1]) by scvm10.sc.intel.com with ESMTP id tBSLJ5Y3023521; Mon, 28 Dec 2015 13:19:05 -0800 Subject: [PATCH v2 36/36] IB/rdmavt: Add pkey support To: dledford@redhat.com From: Dennis Dalessandro Cc: linux-rdma@vger.kernel.org, Mike Marciniszyn , Ira Weiny Date: Mon, 28 Dec 2015 13:19:05 -0800 Message-ID: <20151228211901.29850.898.stgit@scvm10.sc.intel.com> In-Reply-To: <20151228205612.29850.7328.stgit@scvm10.sc.intel.com> References: <20151228205612.29850.7328.stgit@scvm10.sc.intel.com> User-Agent: StGit/0.16 MIME-Version: 1.0 Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org X-Spam-Status: No, score=-6.9 required=5.0 tests=BAYES_00, RCVD_IN_DNSWL_HI, RP_MATCHES_RCVD, UNPARSEABLE_RELAY autolearn=unavailable version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on mail.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Add pkey table in rdi per port data structure. Also bring in related pkey functions. Drivers will still be responsible for allocating and maintaining the pkey table. However they need to tell rdmavt where to find the pkey table. We can not move the pkey table up into rdmavt because drivers need to manipulate this long before registering with it. Reviewed-by: Ira Weiny Reviewed-by: Mike Marciniszyn Signed-off-by: Dennis Dalessandro --- drivers/infiniband/sw/rdmavt/vt.c | 46 ++++++++++++++++++++++++------------- include/rdma/rdma_vt.h | 38 +++++++++++++++++++++++++++---- 2 files changed, 64 insertions(+), 20 deletions(-) -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html diff --git a/drivers/infiniband/sw/rdmavt/vt.c b/drivers/infiniband/sw/rdmavt/vt.c index ab4105a..18b5f43 100644 --- a/drivers/infiniband/sw/rdmavt/vt.c +++ b/drivers/infiniband/sw/rdmavt/vt.c @@ -154,6 +154,17 @@ static int rvt_query_pkey(struct ib_device *ibdev, u8 port, u16 index, * lock, if a stale value is read and sent to the user so be it there is * no way to protect against that anyway. */ + struct rvt_dev_info *rdi = ib_to_rvt(ibdev); + int port_index; + + if (index >= rvt_get_npkeys(rdi)) + return -EINVAL; + + port_index = port - 1; /* IB ports start at 1 our array at 0 */ + if ((port_index < 0) || (port_index >= rdi->dparms.nports)) + return -EINVAL; + + *pkey = rvt_get_pkey(rdi, port_index, index); return 0; } @@ -227,19 +238,6 @@ int rvt_register_device(struct rvt_dev_info *rdi) return -EINVAL; } - if (!rdi->dparms.nports) { - rvt_pr_err(rdi, "Driver says it has no ports.\n"); - return -EINVAL; - } - - rdi->ports = kcalloc(rdi->dparms.nports, - sizeof(struct rvt_ibport **), - GFP_KERNEL); - if (!rdi->ports) { - rvt_pr_err(rdi, "Could not allocate port mem.\n"); - return -ENOMEM; - } - /* Once we get past here we can use the rvt_pr macros */ rvt_mmap_init(rdi); @@ -355,9 +353,25 @@ EXPORT_SYMBOL(rvt_unregister_device); * Keep track of a list of ports. No need to have a detach port. * They persist until the driver goes away. */ -void rvt_attach_port(struct rvt_dev_info *rdi, struct rvt_ibport *port, - int portnum) +int rvt_init_port(struct rvt_dev_info *rdi, struct rvt_ibport *port, + int portnum, u16 *pkey_table) { + if (!rdi->dparms.nports) { + rvt_pr_err(rdi, "Driver says it has no ports.\n"); + return -EINVAL; + } + + rdi->ports = kcalloc(rdi->dparms.nports, + sizeof(struct rvt_ibport **), + GFP_KERNEL); + if (!rdi->ports) { + rvt_pr_err(rdi, "Could not allocate port mem.\n"); + return -ENOMEM; + } + rdi->ports[portnum] = port; + rdi->ports[portnum]->pkey_table = pkey_table; + + return 0; } -EXPORT_SYMBOL(rvt_attach_port); +EXPORT_SYMBOL(rvt_init_port); diff --git a/include/rdma/rdma_vt.h b/include/rdma/rdma_vt.h index fd25d23..3a78f20 100644 --- a/include/rdma/rdma_vt.h +++ b/include/rdma/rdma_vt.h @@ -69,6 +69,8 @@ #define RVT_FLAG_QP_INIT_DRIVER BIT(2) #define RVT_FLAG_CQ_INIT_DRIVER BIT(3) +#define RVT_MAX_PKEY_VALUES 16 + struct rvt_ibport { struct rvt_qp __rcu *qp[2]; struct ib_mad_agent *send_agent; /* agent for SMI (traps) */ @@ -125,6 +127,14 @@ struct rvt_ibport { void *priv; /* driver private data */ + /* + * The pkey table is allocated and maintained by the driver. Drivers + * need to have access to this before registering with rdmav. However + * rdmavt will need access to it so drivers need to proviee this during + * the attach port API call. + */ + u16 *pkey_table; + /* TODO: Move sm_ah and smi_ah into here as well*/ }; @@ -178,6 +188,7 @@ struct rvt_driver_params { int qpn_res_start; int qpn_res_end; int nports; + int npkeys; u8 qos_shift; }; @@ -238,8 +249,6 @@ struct rvt_dev_info { struct rvt_mregion __rcu *dma_mr; struct rvt_lkey_table lkey_table; - /* PKey Table goes here */ - /* Driver specific helper functions */ struct rvt_driver_provided driver_f; @@ -282,11 +291,32 @@ static inline struct rvt_srq *ibsrq_to_rvtsrq(struct ib_srq *ibsrq) return container_of(ibsrq, struct rvt_srq, ibsrq); } +static inline unsigned rvt_get_npkeys(struct rvt_dev_info *rdi) +{ + /* + * All ports have same number of pkeys. + */ + return rdi->dparms.npkeys; +} + +/* + * Return the indexed PKEY from the port PKEY table. + */ +static inline u16 rvt_get_pkey(struct rvt_dev_info *rdi, + int port_index, + unsigned index) +{ + if (index >= rvt_get_npkeys(rdi)) + return 0; + else + return rdi->ports[port_index]->pkey_table[index]; +} + int rvt_register_device(struct rvt_dev_info *rvd); void rvt_unregister_device(struct rvt_dev_info *rvd); int rvt_check_ah(struct ib_device *ibdev, struct ib_ah_attr *ah_attr); -void rvt_attach_port(struct rvt_dev_info *rdi, struct rvt_ibport *port, - int portnum); +int rvt_init_port(struct rvt_dev_info *rdi, struct rvt_ibport *port, + int portnum, u16 *pkey_table); int rvt_rkey_ok(struct rvt_qp *qp, struct rvt_sge *sge, u32 len, u64 vaddr, u32 rkey, int acc); int rvt_lkey_ok(struct rvt_lkey_table *rkt, struct rvt_pd *pd,