From patchwork Thu Dec 10 12:56:22 2015 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Wan, Kaike" X-Patchwork-Id: 7818801 Return-Path: X-Original-To: patchwork-linux-rdma@patchwork.kernel.org Delivered-To: patchwork-parsemail@patchwork2.web.kernel.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.136]) by patchwork2.web.kernel.org (Postfix) with ESMTP id 2C41ABEEE1 for ; Thu, 10 Dec 2015 12:57:29 +0000 (UTC) Received: from mail.kernel.org (localhost [127.0.0.1]) by mail.kernel.org (Postfix) with ESMTP id 4675D205BE for ; Thu, 10 Dec 2015 12:57:28 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 422E6205B9 for ; Thu, 10 Dec 2015 12:57:27 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751318AbbLJM50 (ORCPT ); Thu, 10 Dec 2015 07:57:26 -0500 Received: from mga01.intel.com ([192.55.52.88]:11585 "EHLO mga01.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751276AbbLJM5Z (ORCPT ); Thu, 10 Dec 2015 07:57:25 -0500 Received: from orsmga003.jf.intel.com ([10.7.209.27]) by fmsmga101.fm.intel.com with ESMTP; 10 Dec 2015 04:56:31 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.20,408,1444719600"; d="scan'208";a="704627842" Received: from phlsvsds.ph.intel.com ([10.228.195.38]) by orsmga003.jf.intel.com with ESMTP; 10 Dec 2015 04:56:31 -0800 Received: from phlsvsds.ph.intel.com (localhost.localdomain [127.0.0.1]) by phlsvsds.ph.intel.com (8.13.8/8.13.8) with ESMTP id tBACuUEE023278; Thu, 10 Dec 2015 07:56:30 -0500 Received: (from kaikewan@localhost) by phlsvsds.ph.intel.com (8.13.8/8.13.8/Submit) id tBACuUY5023275; Thu, 10 Dec 2015 07:56:30 -0500 X-Authentication-Warning: phlsvsds.ph.intel.com: kaikewan set sender to kaike.wan@intel.com using -f From: kaike.wan@intel.com To: sean.hefty@intel.com Cc: linux-rdma@vger.kernel.org, Kaike Wan Subject: [PATCH V2 1/1] [PATCH 1/1] Ibacm: default pkey for partitioned fabrics Date: Thu, 10 Dec 2015 07:56:22 -0500 Message-Id: <1449752182-23245-1-git-send-email-kaike.wan@intel.com> X-Mailer: git-send-email 1.7.1 Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org X-Spam-Status: No, score=-6.9 required=5.0 tests=BAYES_00, RCVD_IN_DNSWL_HI, T_RP_MATCHES_RCVD, UNPARSEABLE_RELAY autolearn=unavailable version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on mail.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Kaike Wan In an insecure IB fabric, the default pkey in a port is 0xffff, where each node is allowed to talk to any other node in the fabric, including the SA node. However, in a secure fabric, to limit member access, not all nodes can have the full-member default pkey 0xffff. A typical configuration is to let SA node have pkey 0xffff while all other nodes have pkey 0x7fff; in addition, each node can be assigned some other full-member pkeys, such as 0x8001 and 0x8002, so that it can be assigned to different partitions. In this case, each node can access SA, and yet limits its other access to only those nodes in its assigned partitions. In such a secure fabric, however, ibacm will not work by interpreting "default" in its default address file as 0xffff. To solve the problem, we will use pkey 0 as the default pkey, in line with ipoib convention. Signed-off-by: Kaike Wan --- Change since v1: - Use pkey 0 as the default pkey for parsing address file. src/acm.c | 20 +++++++++++++------- 1 files changed, 13 insertions(+), 7 deletions(-) diff --git a/src/acm.c b/src/acm.c index ada0bfb..8b6d762 100644 --- a/src/acm.c +++ b/src/acm.c @@ -114,7 +114,8 @@ struct acmc_port { union ibv_gid *gid_tbl; uint16_t lid; uint16_t lid_mask; - int default_pkey_index; + int sa_pkey_index; + uint16_t def_acm_pkey; }; struct acmc_device { @@ -2009,7 +2010,7 @@ static int acm_assign_ep_names(struct acmc_ep *ep) continue; } } else { - pkey = 0xFFFF; + pkey = ep->port->def_acm_pkey; } if (!stricmp(dev_name, dev) && @@ -2203,6 +2204,7 @@ static void acm_port_up(struct acmc_port *port) int i, ret; struct acmc_prov_context *dev_ctx; int index = -1; + uint16_t first_pkey = 0; acm_log(1, "%s %d\n", port->dev->device.verbs->device->name, port->port.port_num); @@ -2248,15 +2250,18 @@ static void acm_port_up(struct acmc_port *port) goto err1; } - /* Determine the default pkey first. - Order of preference: 0xffff, 0x7fff, first pkey - */ + /* Determine the default pkey for SA access first. + * Order of preference: 0xffff, 0x7fff + * Use the first pkey as the default pkey for parsing address file. + */ for (i = 0; i < attr.pkey_tbl_len; i++) { ret = ibv_query_pkey(port->dev->device.verbs, port->port.port_num, i, &pkey); if (ret) continue; pkey = ntohs(pkey); + if (i == 0) + first_pkey = pkey; if (pkey == 0xffff) { index = i; break; @@ -2265,7 +2270,8 @@ static void acm_port_up(struct acmc_port *port) index = i; } } - port->default_pkey_index = index < 0 ? 0: index; + port->sa_pkey_index = index < 0 ? 0 : index; + port->def_acm_pkey = first_pkey; for (i = 0; i < attr.pkey_tbl_len; i++) { ret = ibv_query_pkey(port->dev->device.verbs, @@ -2775,7 +2781,7 @@ int acm_send_sa_mad(struct acm_sa_mad *mad) mad->umad.addr.qkey = port->sa_addr.qkey; mad->umad.addr.lid = htons(port->sa_addr.lid); mad->umad.addr.sl = port->sa_addr.sl; - mad->umad.addr.pkey_index = req->ep->port->default_pkey_index; + mad->umad.addr.pkey_index = req->ep->port->sa_pkey_index; lock_acquire(&port->lock); if (port->sa_credits && DListEmpty(&port->sa_wait)) {