From patchwork Mon Dec 14 17:28:19 2015 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dennis Dalessandro X-Patchwork-Id: 7846061 Return-Path: X-Original-To: patchwork-linux-rdma@patchwork.kernel.org Delivered-To: patchwork-parsemail@patchwork1.web.kernel.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.136]) by patchwork1.web.kernel.org (Postfix) with ESMTP id 8C2409F32E for ; Mon, 14 Dec 2015 17:28:25 +0000 (UTC) Received: from mail.kernel.org (localhost [127.0.0.1]) by mail.kernel.org (Postfix) with ESMTP id 7DC6C203A9 for ; Mon, 14 Dec 2015 17:28:24 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id C76A4203C3 for ; Mon, 14 Dec 2015 17:28:22 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752571AbbLNR2W (ORCPT ); Mon, 14 Dec 2015 12:28:22 -0500 Received: from mga03.intel.com ([134.134.136.65]:42434 "EHLO mga03.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752406AbbLNR2V (ORCPT ); Mon, 14 Dec 2015 12:28:21 -0500 Received: from orsmga003.jf.intel.com ([10.7.209.27]) by orsmga103.jf.intel.com with ESMTP; 14 Dec 2015 09:28:21 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.20,428,1444719600"; d="scan'208";a="707156261" Received: from sedona.ch.intel.com ([143.182.228.65]) by orsmga003.jf.intel.com with ESMTP; 14 Dec 2015 09:28:20 -0800 Received: from phlsvsles11.ph.intel.com (phlsvsles11.ph.intel.com [10.228.195.43]) by sedona.ch.intel.com (8.13.6/8.14.3/Standard MailSET/Hub) with ESMTP id tBEHSJlG029749; Mon, 14 Dec 2015 10:28:20 -0700 Received: from phlsvslse11.ph.intel.com (localhost [127.0.0.1]) by phlsvsles11.ph.intel.com with ESMTP id tBEHSJki012528; Mon, 14 Dec 2015 12:28:19 -0500 Subject: [RFC PATCH 04/15] IB/hfi1: Use rdmavt protection domain To: devel@driverdev.osuosl.org, linux-rdma@vger.kernel.org From: Dennis Dalessandro Cc: gregkh@linuxfoundation.org, dledford@redhat.com Date: Mon, 14 Dec 2015 12:28:19 -0500 Message-ID: <20151214172817.11576.45609.stgit@phlsvslse11.ph.intel.com> In-Reply-To: <20151214172509.11576.19722.stgit@phlsvslse11.ph.intel.com> References: <20151214172509.11576.19722.stgit@phlsvslse11.ph.intel.com> User-Agent: StGit/0.16 MIME-Version: 1.0 Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org X-Spam-Status: No, score=-6.9 required=5.0 tests=BAYES_00, RCVD_IN_DNSWL_HI, T_RP_MATCHES_RCVD, UNPARSEABLE_RELAY autolearn=unavailable version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on mail.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Remove protection domain from hfi1 and use rdmavt's version. Reviewed-by: Ira Weiny Reviewed-by: Mike Marciniszyn Signed-off-by: Dennis Dalessandro --- drivers/staging/rdma/hfi1/keys.c | 4 +- drivers/staging/rdma/hfi1/mr.c | 2 + drivers/staging/rdma/hfi1/ruc.c | 4 +- drivers/staging/rdma/hfi1/verbs.c | 67 +++---------------------------------- drivers/staging/rdma/hfi1/verbs.h | 15 +------- 5 files changed, 12 insertions(+), 80 deletions(-) -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html diff --git a/drivers/staging/rdma/hfi1/keys.c b/drivers/staging/rdma/hfi1/keys.c index cb4e608..57a266f 100644 --- a/drivers/staging/rdma/hfi1/keys.c +++ b/drivers/staging/rdma/hfi1/keys.c @@ -176,7 +176,7 @@ out: * Check the IB SGE for validity and initialize our internal version * of it. */ -int hfi1_lkey_ok(struct hfi1_lkey_table *rkt, struct hfi1_pd *pd, +int hfi1_lkey_ok(struct hfi1_lkey_table *rkt, struct rvt_pd *pd, struct hfi1_sge *isge, struct ib_sge *sge, int acc) { struct hfi1_mregion *mr; @@ -285,7 +285,7 @@ int hfi1_rkey_ok(struct hfi1_qp *qp, struct hfi1_sge *sge, */ rcu_read_lock(); if (rkey == 0) { - struct hfi1_pd *pd = to_ipd(qp->ibqp.pd); + struct rvt_pd *pd = ibpd_to_rvtpd(qp->ibqp.pd); struct hfi1_ibdev *dev = to_idev(pd->ibpd.device); if (pd->user) diff --git a/drivers/staging/rdma/hfi1/mr.c b/drivers/staging/rdma/hfi1/mr.c index 568f185..02589b2 100644 --- a/drivers/staging/rdma/hfi1/mr.c +++ b/drivers/staging/rdma/hfi1/mr.c @@ -116,7 +116,7 @@ struct ib_mr *hfi1_get_dma_mr(struct ib_pd *pd, int acc) struct ib_mr *ret; int rval; - if (to_ipd(pd)->user) { + if (ibpd_to_rvtpd(pd)->user) { ret = ERR_PTR(-EPERM); goto bail; } diff --git a/drivers/staging/rdma/hfi1/ruc.c b/drivers/staging/rdma/hfi1/ruc.c index 317bf6f..eb7aea9 100644 --- a/drivers/staging/rdma/hfi1/ruc.c +++ b/drivers/staging/rdma/hfi1/ruc.c @@ -102,11 +102,11 @@ static int init_sge(struct hfi1_qp *qp, struct hfi1_rwqe *wqe) int i, j, ret; struct ib_wc wc; struct hfi1_lkey_table *rkt; - struct hfi1_pd *pd; + struct rvt_pd *pd; struct hfi1_sge_state *ss; rkt = &to_idev(qp->ibqp.device)->lk_table; - pd = to_ipd(qp->ibqp.srq ? qp->ibqp.srq->pd : qp->ibqp.pd); + pd = ibpd_to_rvtpd(qp->ibqp.srq ? qp->ibqp.srq->pd : qp->ibqp.pd); ss = &qp->r_sge; ss->sg_list = qp->r_sg_list; qp->r_len = 0; diff --git a/drivers/staging/rdma/hfi1/verbs.c b/drivers/staging/rdma/hfi1/verbs.c index 22e2742..1390755 100644 --- a/drivers/staging/rdma/hfi1/verbs.c +++ b/drivers/staging/rdma/hfi1/verbs.c @@ -368,7 +368,7 @@ static int post_one_send(struct hfi1_qp *qp, struct ib_send_wr *wr) int j; int acc; struct hfi1_lkey_table *rkt; - struct hfi1_pd *pd; + struct rvt_pd *pd; struct hfi1_devdata *dd = dd_from_ibdev(qp->ibqp.device); struct hfi1_pportdata *ppd; struct hfi1_ibport *ibp; @@ -413,7 +413,7 @@ static int post_one_send(struct hfi1_qp *qp, struct ib_send_wr *wr) return -ENOMEM; rkt = &to_idev(qp->ibqp.device)->lk_table; - pd = to_ipd(qp->ibqp.pd); + pd = ibpd_to_rvtpd(qp->ibqp.pd); wqe = get_swqe_ptr(qp, qp->s_head); @@ -1394,7 +1394,7 @@ static int query_device(struct ib_device *ibdev, props->max_mr = dev->lk_table.max; props->max_fmr = dev->lk_table.max; props->max_map_per_fmr = 32767; - props->max_pd = hfi1_max_pds; + props->max_pd = dev->rdi.dparms.props.max_pd; props->max_qp_rd_atom = HFI1_MAX_RDMA_ATOMIC; props->max_qp_init_rd_atom = 255; /* props->max_res_rd_atom */ @@ -1592,61 +1592,6 @@ static int query_gid(struct ib_device *ibdev, u8 port, return ret; } -static struct ib_pd *alloc_pd(struct ib_device *ibdev, - struct ib_ucontext *context, - struct ib_udata *udata) -{ - struct hfi1_ibdev *dev = to_idev(ibdev); - struct hfi1_pd *pd; - struct ib_pd *ret; - - /* - * This is actually totally arbitrary. Some correctness tests - * assume there's a maximum number of PDs that can be allocated. - * We don't actually have this limit, but we fail the test if - * we allow allocations of more than we report for this value. - */ - - pd = kmalloc(sizeof(*pd), GFP_KERNEL); - if (!pd) { - ret = ERR_PTR(-ENOMEM); - goto bail; - } - - spin_lock(&dev->n_pds_lock); - if (dev->n_pds_allocated == hfi1_max_pds) { - spin_unlock(&dev->n_pds_lock); - kfree(pd); - ret = ERR_PTR(-ENOMEM); - goto bail; - } - - dev->n_pds_allocated++; - spin_unlock(&dev->n_pds_lock); - - /* ib_alloc_pd() will initialize pd->ibpd. */ - pd->user = udata != NULL; - - ret = &pd->ibpd; - -bail: - return ret; -} - -static int dealloc_pd(struct ib_pd *ibpd) -{ - struct hfi1_pd *pd = to_ipd(ibpd); - struct hfi1_ibdev *dev = to_idev(ibpd->device); - - spin_lock(&dev->n_pds_lock); - dev->n_pds_allocated--; - spin_unlock(&dev->n_pds_lock); - - kfree(pd); - - return 0; -} - /* * convert ah port,sl to sc */ @@ -1920,7 +1865,7 @@ int hfi1_register_ib_device(struct hfi1_devdata *dd) init_ibport(ppd + i); /* Only need to initialize non-zero fields. */ - spin_lock_init(&dev->n_pds_lock); + spin_lock_init(&dev->n_ahs_lock); spin_lock_init(&dev->n_cqs_lock); spin_lock_init(&dev->n_qps_lock); @@ -2029,8 +1974,8 @@ int hfi1_register_ib_device(struct hfi1_devdata *dd) ibdev->query_gid = query_gid; ibdev->alloc_ucontext = alloc_ucontext; ibdev->dealloc_ucontext = dealloc_ucontext; - ibdev->alloc_pd = alloc_pd; - ibdev->dealloc_pd = dealloc_pd; + ibdev->alloc_pd = NULL; + ibdev->dealloc_pd = NULL; ibdev->create_ah = create_ah; ibdev->destroy_ah = destroy_ah; ibdev->modify_ah = modify_ah; diff --git a/drivers/staging/rdma/hfi1/verbs.h b/drivers/staging/rdma/hfi1/verbs.h index 4f23e00..9043e5d 100644 --- a/drivers/staging/rdma/hfi1/verbs.h +++ b/drivers/staging/rdma/hfi1/verbs.h @@ -234,12 +234,6 @@ struct hfi1_mcast { int n_attached; }; -/* Protection domain */ -struct hfi1_pd { - struct ib_pd ibpd; - int user; /* non-zero if created from user space */ -}; - /* Address Handle */ struct hfi1_ah { struct ib_ah ibah; @@ -776,8 +770,6 @@ struct hfi1_ibdev { u64 n_kmem_wait; u64 n_send_schedule; - u32 n_pds_allocated; /* number of PDs allocated for device */ - spinlock_t n_pds_lock; u32 n_ahs_allocated; /* number of AHs allocated for device */ spinlock_t n_ahs_lock; u32 n_cqs_allocated; /* number of CQs allocated for device */ @@ -817,11 +809,6 @@ static inline struct hfi1_mr *to_imr(struct ib_mr *ibmr) return container_of(ibmr, struct hfi1_mr, ibmr); } -static inline struct hfi1_pd *to_ipd(struct ib_pd *ibpd) -{ - return container_of(ibpd, struct hfi1_pd, ibpd); -} - static inline struct hfi1_ah *to_iah(struct ib_ah *ibah) { return container_of(ibah, struct hfi1_ah, ibah); @@ -983,7 +970,7 @@ int hfi1_alloc_lkey(struct hfi1_mregion *mr, int dma_region); void hfi1_free_lkey(struct hfi1_mregion *mr); -int hfi1_lkey_ok(struct hfi1_lkey_table *rkt, struct hfi1_pd *pd, +int hfi1_lkey_ok(struct hfi1_lkey_table *rkt, struct rvt_pd *pd, struct hfi1_sge *isge, struct ib_sge *sge, int acc); int hfi1_rkey_ok(struct hfi1_qp *qp, struct hfi1_sge *sge,