From patchwork Tue Jun 1 14:35:27 2010 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Heinz X-Patchwork-Id: 103550 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by demeter.kernel.org (8.14.3/8.14.3) with ESMTP id o51EaFFe016775 for ; Tue, 1 Jun 2010 14:36:15 GMT Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1756918Ab0FAOfd (ORCPT ); Tue, 1 Jun 2010 10:35:33 -0400 Received: from avexcashub1.qlogic.com ([198.70.193.61]:36119 "EHLO avexcashub1.qlogic.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1756510Ab0FAOfb convert rfc822-to-8bit (ORCPT ); Tue, 1 Jun 2010 10:35:31 -0400 Received: from avexcashub2.qlogic.org (10.1.4.116) by avexcashub1.qlogic.org (10.1.4.161) with Microsoft SMTP Server (TLS) id 8.1.375.2; Tue, 1 Jun 2010 07:35:30 -0700 Received: from MNEXCASHUB2.qlogic.org (10.33.2.104) by avexcashub2.qlogic.org (10.1.4.162) with Microsoft SMTP Server (TLS) id 8.1.375.2; Tue, 1 Jun 2010 07:35:30 -0700 Received: from MNEXMB1.qlogic.org ([fe80::c6b:fda:afec:79a1]) by MNEXCASHUB2.qlogic.org ([::1]) with mapi; Tue, 1 Jun 2010 09:35:29 -0500 From: Mike Heinz To: "linux-rdma@vger.kernel.org" Date: Tue, 1 Jun 2010 09:35:27 -0500 Subject: [PATCH] ipoib pkey race condition Thread-Topic: [PATCH] ipoib pkey race condition Thread-Index: AcsBl6237M45lmObTMC0l4Mu1JmF/w== Message-ID: <4C2744E8AD2982428C5BFE523DF8CDCB49A488D743@MNEXMB1.qlogic.org> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US MIME-Version: 1.0 Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org X-Greylist: IP, sender and recipient auto-whitelisted, not delayed by milter-greylist-4.2.3 (demeter.kernel.org [140.211.167.41]); Tue, 01 Jun 2010 14:36:15 +0000 (UTC) differently. However at this point IPoIB has already started using the incorrect pkey. It appears that the initially formatted 'broadcast' mgid is never updated to supply actual pkey value if ipoib comes up before hca port. Proposed patch targets two issues: 1. Suppress activation of interface and join multicast group queries (it will fail anyway) until hca port is initialized. When port becomes active - update pkey value and move on. 2. Update broadcast mgid based on actual pkey, then issue join broadcast group request. Signed-Off-By: Michael Heinz ------- diff --git a/drivers/infiniband/ulp/ipoib/ipoib_ib.c b/drivers/infiniband/ulp/ipoib/ipoib_ib.c index ec6b4fb..496d96c 100644 --- a/drivers/infiniband/ulp/ipoib/ipoib_ib.c +++ b/drivers/infiniband/ulp/ipoib/ipoib_ib.c @@ -51,6 +51,7 @@ MODULE_PARM_DESC(data_debug_level, #endif static DEFINE_MUTEX(pkey_mutex); +static void ipoib_pkey_dev_check_presence(struct net_device *dev); struct ipoib_ah *ipoib_create_ah(struct net_device *dev, struct ib_pd *pd, struct ib_ah_attr *attr) @@ -654,12 +655,13 @@ int ipoib_ib_dev_open(struct net_device *dev) struct ipoib_dev_priv *priv = netdev_priv(dev); int ret; - if (ib_find_pkey(priv->ca, priv->port, priv->pkey, &priv->pkey_index)) { + ipoib_pkey_dev_check_presence(dev); + + if (!test_bit(IPOIB_PKEY_ASSIGNED, &priv->flags)) { ipoib_warn(priv, "P_Key 0x%04x not found\n", priv->pkey); clear_bit(IPOIB_PKEY_ASSIGNED, &priv->flags); return -1; } - set_bit(IPOIB_PKEY_ASSIGNED, &priv->flags); ret = ipoib_init_qp(dev); if (ret) { @@ -694,9 +696,26 @@ int ipoib_ib_dev_open(struct net_device *dev) static void ipoib_pkey_dev_check_presence(struct net_device *dev) { struct ipoib_dev_priv *priv = netdev_priv(dev); - u16 pkey_index = 0; + struct ib_port_attr port_attr; + + if (!test_bit(IPOIB_FLAG_SUBINTERFACE, &priv->flags)) { + clear_bit(IPOIB_PKEY_ASSIGNED, &priv->flags); + if (ib_query_port(priv->ca, priv->port, &port_attr)) { + ipoib_warn(priv, "Query port attrs failed\n"); + return; + } + + if (port_attr.state != IB_PORT_ACTIVE) + return; + + if (ib_query_pkey(priv->ca, priv->port, 0, &priv->pkey)) { + ipoib_warn(priv, "Query P_Key table entry 0 failed\n"); + return; + } + set_bit(IPOIB_PKEY_ASSIGNED, &priv->flags); + } - if (ib_find_pkey(priv->ca, priv->port, priv->pkey, &pkey_index)) + if (ib_find_pkey(priv->ca, priv->port, priv->pkey, &priv->pkey_index)) clear_bit(IPOIB_PKEY_ASSIGNED, &priv->flags); else set_bit(IPOIB_PKEY_ASSIGNED, &priv->flags); @@ -955,7 +974,8 @@ static void __ipoib_ib_dev_flush(struct ipoib_dev_priv *priv, } /* restart QP only if P_Key index is changed */ - if (test_and_set_bit(IPOIB_PKEY_ASSIGNED, &priv->flags) && + if (test_bit(IPOIB_FLAG_SUBINTERFACE, &priv->flags) && + test_and_set_bit(IPOIB_PKEY_ASSIGNED, &priv->flags) && new_index == priv->pkey_index) { ipoib_dbg(priv, "Not flushing - P_Key index not changed.\n"); return; diff --git a/drivers/infiniband/ulp/ipoib/ipoib_multicast.c b/drivers/infiniband/ulp/ipoib/ipoib_multicast.c index 3871ac6..6fe6527 100644 --- a/drivers/infiniband/ulp/ipoib/ipoib_multicast.c +++ b/drivers/infiniband/ulp/ipoib/ipoib_multicast.c @@ -552,6 +552,13 @@ void ipoib_mcast_join_task(struct work_struct *work) } spin_lock_irq(&priv->lock); + + if (!test_bit(IPOIB_FLAG_SUBINTERFACE, &priv->flags)) { + /* fix broadcast gid in case if pkey was changed */ + priv->pkey |= 0x8000; + priv->dev->broadcast[8] = priv->pkey >> 8; + priv->dev->broadcast[9] = priv->pkey & 0xff; + } memcpy(broadcast->mcmember.mgid.raw, priv->dev->broadcast + 4, sizeof (union ib_gid)); priv->broadcast = broadcast;