From patchwork Thu Feb 12 01:43:43 2015 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Doug Ledford X-Patchwork-Id: 5815161 Return-Path: X-Original-To: patchwork-linux-rdma@patchwork.kernel.org Delivered-To: patchwork-parsemail@patchwork2.web.kernel.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.136]) by patchwork2.web.kernel.org (Postfix) with ESMTP id 5909BBF440 for ; Thu, 12 Feb 2015 01:44:32 +0000 (UTC) Received: from mail.kernel.org (localhost [127.0.0.1]) by mail.kernel.org (Postfix) with ESMTP id 7309E200E0 for ; Thu, 12 Feb 2015 01:44:31 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 6E475201F4 for ; Thu, 12 Feb 2015 01:44:30 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1754705AbbBLBoU (ORCPT ); Wed, 11 Feb 2015 20:44:20 -0500 Received: from mx1.redhat.com ([209.132.183.28]:54249 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1754500AbbBLBoS (ORCPT ); Wed, 11 Feb 2015 20:44:18 -0500 Received: from int-mx14.intmail.prod.int.phx2.redhat.com (int-mx14.intmail.prod.int.phx2.redhat.com [10.5.11.27]) by mx1.redhat.com (8.14.4/8.14.4) with ESMTP id t1C1iD11031875 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=FAIL); Wed, 11 Feb 2015 20:44:13 -0500 Received: from linux-ws.xsintricity.com ([10.10.116.22]) by int-mx14.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id t1C1i1Ev029536; Wed, 11 Feb 2015 20:44:13 -0500 From: Doug Ledford To: linux-rdma@vger.kernel.org, roland@kernel.org Cc: Or Gerlitz , Erez Shitrit , Doug Ledford Subject: [PATCH 20/22] IB/ipoib: don't queue a work struct up twice Date: Wed, 11 Feb 2015 20:43:43 -0500 Message-Id: <30a5bd6461381448c52af0d7408dbc14da9ac4d0.1423703861.git.dledford@redhat.com> In-Reply-To: References: In-Reply-To: References: X-Scanned-By: MIMEDefang 2.68 on 10.5.11.27 Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org X-Spam-Status: No, score=-6.9 required=5.0 tests=BAYES_00, RCVD_IN_DNSWL_HI, T_RP_MATCHES_RCVD, UNPARSEABLE_RELAY autolearn=unavailable version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on mail.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP In b2d408f78b3 (IB/ipoib: make delayed tasks not hold up everything) I added the ability to have some joins processed around delayed joins. Because the join task processes one join at a time, we needed to queue the task to run again immediately and then again at the appropriate delay time. I didn't think about the fact that our work struct can only be on the workqueue once at a given time and instead tried to queue the same struct up twice. Instead, kick the queue again immediately on a delayed restart attempt, when the task sees that it has nothing to do but delayed work, it will pick the shortest work item and queue a delay equal to its expiration. Finally, in case we have delayed work, in the send_mcast routine we need to cancel any delayed work before kicking the thread immediately. Signed-off-by: Doug Ledford --- drivers/infiniband/ulp/ipoib/ipoib_multicast.c | 49 ++++++++++++++++---------- 1 file changed, 31 insertions(+), 18 deletions(-) diff --git a/drivers/infiniband/ulp/ipoib/ipoib_multicast.c b/drivers/infiniband/ulp/ipoib/ipoib_multicast.c index ad2bcd105e4..68b47bed33a 100644 --- a/drivers/infiniband/ulp/ipoib/ipoib_multicast.c +++ b/drivers/infiniband/ulp/ipoib/ipoib_multicast.c @@ -71,29 +71,28 @@ static void __ipoib_mcast_continue_join_thread(struct ipoib_dev_priv *priv, int delay) { if (test_bit(IPOIB_MCAST_RUN, &priv->flags)) { - /* - * Mark this mcast for its delay and set a timer to kick the - * thread when the delay completes - */ if (mcast && delay) { mcast->backoff *= 2; if (mcast->backoff > IPOIB_MAX_BACKOFF_SECONDS) mcast->backoff = IPOIB_MAX_BACKOFF_SECONDS; mcast->delay_until = jiffies + (mcast->backoff * HZ); - queue_delayed_work(priv->wq, &priv->mcast_task, - mcast->backoff * HZ); + /* + * Mark this mcast for its delay, but restart the + * task immediately. It will make sure to clear + * out all entries without delays, and then + * schedule itself for run again when the delay + * expires + */ + queue_delayed_work(priv->wq, &priv->mcast_task, 0); } else if (delay) { - /* Special case of retrying after a failure to + /* + * Special case of retrying after a failure to * allocate the broadcast multicast group, wait * 1 second and try again */ queue_delayed_work(priv->wq, &priv->mcast_task, HZ); - } - /* - * But also rekick the thread immediately for any other - * tasks in queue behind this one - */ - queue_delayed_work(priv->wq, &priv->mcast_task, delay); + } else + queue_delayed_work(priv->wq, &priv->mcast_task, 0); } } @@ -568,6 +567,7 @@ void ipoib_mcast_join_task(struct work_struct *work) struct ib_port_attr port_attr; struct ipoib_mcast *mcast = NULL; int create = 1; + unsigned long delay_until = 0; if (!test_bit(IPOIB_MCAST_RUN, &priv->flags)) return; @@ -636,11 +636,14 @@ void ipoib_mcast_join_task(struct work_struct *work) list_for_each_entry(mcast, &priv->multicast_list, list) { if (IS_ERR_OR_NULL(mcast->mc) && !test_bit(IPOIB_MCAST_FLAG_BUSY, &mcast->flags) && - !test_bit(IPOIB_MCAST_FLAG_ATTACHED, &mcast->flags) && - (mcast->backoff == 1 || - time_after_eq(jiffies, mcast->delay_until))) { - /* Found the next unjoined group */ - break; + !test_bit(IPOIB_MCAST_FLAG_ATTACHED, &mcast->flags)) { + if (mcast->backoff == 1 || + time_after_eq(jiffies, mcast->delay_until)) + /* Found the next unjoined group */ + break; + else if (!delay_until || + time_before(mcast->delay_until, delay_until)) + delay_until = mcast->delay_until; } } @@ -653,6 +656,9 @@ void ipoib_mcast_join_task(struct work_struct *work) mcast = NULL; ipoib_dbg_mcast(priv, "successfully joined all " "multicast groups\n"); + if (delay_until) + queue_delayed_work(priv->wq, &priv->mcast_task, + delay_until - jiffies); } out: @@ -766,6 +772,13 @@ void ipoib_mcast_send(struct net_device *dev, u8 *daddr, struct sk_buff *skb) memcpy(mcast->mcmember.mgid.raw, mgid, sizeof (union ib_gid)); __ipoib_mcast_add(dev, mcast); list_add_tail(&mcast->list, &priv->multicast_list); + /* + * Make sure that if we are currently waiting for a backoff + * delay to expire, that we cancel that, run immediately, + * and then requeue our task to fire when the backoff + * timer is over. + */ + cancel_delayed_work(&priv->mcast_task); __ipoib_mcast_continue_join_thread(priv, NULL, 0); }