From patchwork Thu Nov 10 03:30:28 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Henry Orosco X-Patchwork-Id: 9420561 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id E5199601C2 for ; Thu, 10 Nov 2016 03:30:39 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id D7A7229467 for ; Thu, 10 Nov 2016 03:30:39 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id CB3682948C; Thu, 10 Nov 2016 03:30:39 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.9 required=2.0 tests=BAYES_00,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 63A9F29467 for ; Thu, 10 Nov 2016 03:30:38 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1755026AbcKJDag (ORCPT ); Wed, 9 Nov 2016 22:30:36 -0500 Received: from mga03.intel.com ([134.134.136.65]:15909 "EHLO mga03.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1754571AbcKJDae (ORCPT ); Wed, 9 Nov 2016 22:30:34 -0500 Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by orsmga103.jf.intel.com with ESMTP; 09 Nov 2016 19:30:34 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.31,616,1473145200"; d="scan'208";a="1066471485" Received: from horosco-mobl2.amr.corp.intel.com ([10.254.112.45]) by fmsmga001.fm.intel.com with ESMTP; 09 Nov 2016 19:30:33 -0800 From: Henry Orosco To: dledford@redhat.com Cc: linux-rdma@vger.kernel.org, e1000-rdma@lists.sourceforge.net, Henry Orosco Subject: [PATCH] i40iw: Query device accounts for internal rsrc Date: Wed, 9 Nov 2016 21:30:28 -0600 Message-Id: <20161110033028.16356-1-henry.orosco@intel.com> X-Mailer: git-send-email 2.8.3 Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Some resources are consumed internally and not available to the user. After hw is initialized, figure out how many resources are consumed and subtract those numbers from the initial max device capability in i40iw_query_device(). Signed-off-by: Henry Orosco --- drivers/infiniband/hw/i40iw/i40iw.h | 4 ++++ drivers/infiniband/hw/i40iw/i40iw_hw.c | 2 +- drivers/infiniband/hw/i40iw/i40iw_main.c | 15 +++++++++++++++ drivers/infiniband/hw/i40iw/i40iw_user.h | 3 ++- drivers/infiniband/hw/i40iw/i40iw_verbs.c | 8 ++++---- 5 files changed, 26 insertions(+), 6 deletions(-) diff --git a/drivers/infiniband/hw/i40iw/i40iw.h b/drivers/infiniband/hw/i40iw/i40iw.h index 8ec09e4..22af66a 100644 --- a/drivers/infiniband/hw/i40iw/i40iw.h +++ b/drivers/infiniband/hw/i40iw/i40iw.h @@ -297,6 +297,10 @@ struct i40iw_device { u32 mr_stagmask; u32 mpa_version; bool dcb; + u32 used_pds; + u32 used_cqs; + u32 used_mrs; + u32 used_qps; }; struct i40iw_ib_device { diff --git a/drivers/infiniband/hw/i40iw/i40iw_hw.c b/drivers/infiniband/hw/i40iw/i40iw_hw.c index 0c92a40..886b8c8 100644 --- a/drivers/infiniband/hw/i40iw/i40iw_hw.c +++ b/drivers/infiniband/hw/i40iw/i40iw_hw.c @@ -62,7 +62,7 @@ u32 i40iw_initialize_hw_resources(struct i40iw_device *iwdev) max_mr = iwdev->sc_dev.hmc_info->hmc_obj[I40IW_HMC_IW_MR].cnt; arp_table_size = iwdev->sc_dev.hmc_info->hmc_obj[I40IW_HMC_IW_ARP].cnt; iwdev->max_cqe = 0xFFFFF; - num_pds = max_qp * 4; + num_pds = I40IW_MAX_PDS; resources_size = sizeof(struct i40iw_arp_entry) * arp_table_size; resources_size += sizeof(unsigned long) * BITS_TO_LONGS(max_qp); resources_size += sizeof(unsigned long) * BITS_TO_LONGS(max_mr); diff --git a/drivers/infiniband/hw/i40iw/i40iw_main.c b/drivers/infiniband/hw/i40iw/i40iw_main.c index 078d784..ba45b16 100644 --- a/drivers/infiniband/hw/i40iw/i40iw_main.c +++ b/drivers/infiniband/hw/i40iw/i40iw_main.c @@ -1540,6 +1540,20 @@ static enum i40iw_status_code i40iw_setup_init_state(struct i40iw_handler *hdl, } /** + * i40iw_get_used_rsrc - determine resources used internally + * @iwdev: iwarp device + * + * Called after internal allocations + */ +static void i40iw_get_used_rsrc(struct i40iw_device *iwdev) +{ + iwdev->used_pds = find_next_zero_bit(iwdev->allocated_pds, iwdev->max_pd, 0); + iwdev->used_qps = find_next_zero_bit(iwdev->allocated_qps, iwdev->max_qp, 0); + iwdev->used_cqs = find_next_zero_bit(iwdev->allocated_cqs, iwdev->max_cq, 0); + iwdev->used_mrs = find_next_zero_bit(iwdev->allocated_mrs, iwdev->max_mr, 0); +} + +/** * i40iw_open - client interface operation open for iwarp/uda device * @ldev: lan device information * @client: iwarp client information, provided during registration @@ -1611,6 +1625,7 @@ static int i40iw_open(struct i40e_info *ldev, struct i40e_client *client) status = i40iw_initialize_hw_resources(iwdev); if (status) break; + i40iw_get_used_rsrc(iwdev); dev->ccq_ops->ccq_arm(dev->ccq); status = i40iw_hmc_init_pble(&iwdev->sc_dev, iwdev->pble_rsrc); if (status) diff --git a/drivers/infiniband/hw/i40iw/i40iw_user.h b/drivers/infiniband/hw/i40iw/i40iw_user.h index e65c2baa..66263fc 100644 --- a/drivers/infiniband/hw/i40iw/i40iw_user.h +++ b/drivers/infiniband/hw/i40iw/i40iw_user.h @@ -76,7 +76,8 @@ enum i40iw_device_capabilities_const { I40IW_MAX_ORD_SIZE = 127, I40IW_MAX_WQ_ENTRIES = 2048, I40IW_Q2_BUFFER_SIZE = (248 + 100), - I40IW_QP_CTX_SIZE = 248 + I40IW_QP_CTX_SIZE = 248, + I40IW_MAX_PDS = 32768 }; #define i40iw_handle void * diff --git a/drivers/infiniband/hw/i40iw/i40iw_verbs.c b/drivers/infiniband/hw/i40iw/i40iw_verbs.c index 6329c97..bfea715 100644 --- a/drivers/infiniband/hw/i40iw/i40iw_verbs.c +++ b/drivers/infiniband/hw/i40iw/i40iw_verbs.c @@ -67,13 +67,13 @@ static int i40iw_query_device(struct ib_device *ibdev, props->vendor_part_id = iwdev->ldev->pcidev->device; props->hw_ver = (u32)iwdev->sc_dev.hw_rev; props->max_mr_size = I40IW_MAX_OUTBOUND_MESSAGE_SIZE; - props->max_qp = iwdev->max_qp; + props->max_qp = iwdev->max_qp - iwdev->used_qps; props->max_qp_wr = (I40IW_MAX_WQ_ENTRIES >> 2) - 1; props->max_sge = I40IW_MAX_WQ_FRAGMENT_COUNT; - props->max_cq = iwdev->max_cq; + props->max_cq = iwdev->max_cq - iwdev->used_cqs; props->max_cqe = iwdev->max_cqe; - props->max_mr = iwdev->max_mr; - props->max_pd = iwdev->max_pd; + props->max_mr = iwdev->max_mr - iwdev->used_mrs; + props->max_pd = iwdev->max_pd - iwdev->used_pds; props->max_sge_rd = I40IW_MAX_SGE_RD; props->max_qp_rd_atom = I40IW_MAX_IRD_SIZE; props->max_qp_init_rd_atom = props->max_qp_rd_atom;