From patchwork Tue Aug 14 16:24:38 2012 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sasha Levin X-Patchwork-Id: 1322981 Return-Path: X-Original-To: patchwork-dm-devel@patchwork.kernel.org Delivered-To: patchwork-process-083081@patchwork1.kernel.org Received: from mx3-phx2.redhat.com (mx3-phx2.redhat.com [209.132.183.24]) by patchwork1.kernel.org (Postfix) with ESMTP id 4953B3FC81 for ; Tue, 14 Aug 2012 18:40:37 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by mx3-phx2.redhat.com (8.13.8/8.13.8) with ESMTP id q7EIc2nI006346; Tue, 14 Aug 2012 14:38:02 -0400 Received: from int-mx12.intmail.prod.int.phx2.redhat.com (int-mx12.intmail.prod.int.phx2.redhat.com [10.5.11.25]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id q7EGPcej013005 for ; Tue, 14 Aug 2012 12:25:38 -0400 Received: from mx1.redhat.com (ext-mx11.extmail.prod.ext.phx2.redhat.com [10.5.110.16]) by int-mx12.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id q7EGPWXf004659; Tue, 14 Aug 2012 12:25:32 -0400 Received: from mail-bk0-f46.google.com (mail-bk0-f46.google.com [209.85.214.46]) by mx1.redhat.com (8.14.4/8.14.4) with ESMTP id q7EGOqgn002535; Tue, 14 Aug 2012 12:25:22 -0400 Received: by mail-bk0-f46.google.com with SMTP id j10so238412bkw.33 for ; Tue, 14 Aug 2012 09:25:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=from:to:cc:subject:date:message-id:x-mailer:in-reply-to:references; bh=2ijmJGzlVKHgCatHn+WIw+XEGSra6QyFjQd5kSgKuNc=; b=MRE3MAuYDNBRCofA68oAZSr7fWaxgNXV0a7QwC2mqoX6N8VQ/xg8gBgxLUsAxNUaa0 JbJROS/qJ38pCBve+MsDb5l3aUZRvFiScGGC7sO/BuMylSTe4x3uifbbx1pgQXYlgPef QblZ8+0oZGBnPm7rtgwNmB9IZF+jAOxIxgeXIGKsKgaAzINSrb91BpkitPgmshvLsanm MEAEFBY736hJ7W+6MAr5ewuI8FZa4l1O/kenZ3cebUEP91VGy72xIQIGsMb8PHC9pGyQ zdis0QuAfq+ylUcL28e2cYKUNKjQ+PSe0HOGe1EB9NIoU8vWlZWLkmO23GZbK5crpZKz 1kzg== Received: by 10.204.152.27 with SMTP id e27mr6495685bkw.56.1344961521911; Tue, 14 Aug 2012 09:25:21 -0700 (PDT) Received: from localhost.localdomain (95-89-78-76-dynip.superkabel.de. [95.89.78.76]) by mx.google.com with ESMTPS id t23sm1470660bks.4.2012.08.14.09.25.07 (version=TLSv1/SSLv3 cipher=OTHER); Tue, 14 Aug 2012 09:25:20 -0700 (PDT) From: Sasha Levin To: torvalds@linux-foundation.org Date: Tue, 14 Aug 2012 18:24:38 +0200 Message-Id: <1344961490-4068-5-git-send-email-levinsasha928@gmail.com> In-Reply-To: <1344961490-4068-1-git-send-email-levinsasha928@gmail.com> References: <1344961490-4068-1-git-send-email-levinsasha928@gmail.com> X-RedHat-Spam-Score: -2.45 (BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, FREEMAIL_ENVFROM_END_DIGIT, FREEMAIL_FROM, RCVD_IN_DNSWL_LOW, SPF_PASS) X-Scanned-By: MIMEDefang 2.68 on 10.5.11.25 X-Scanned-By: MIMEDefang 2.68 on 10.5.110.16 X-loop: dm-devel@redhat.com X-Mailman-Approved-At: Tue, 14 Aug 2012 14:21:01 -0400 Cc: snitzer@redhat.com, fweisbec@gmail.com, Trond.Myklebust@netapp.com, bfields@fieldses.org, paul.gortmaker@windriver.com, dm-devel@redhat.com, agk@redhat.com, aarcange@redhat.com, rds-devel@oss.oracle.com, eric.dumazet@gmail.com, venkat.x.venkatsubra@oracle.com, ccaulfie@redhat.com, mingo@elte.hu, dev@openvswitch.org, jesse@nicira.com, josh@joshtriplett.org, rostedt@goodmis.org, lw@cn.fujitsu.com, mathieu.desnoyers@efficios.com, Sasha Levin , axboe@kernel.dk, linux-nfs@vger.kernel.org, edumazet@google.com, linux-mm@kvack.org, netdev@vger.kernel.org, linux-kernel@vger.kernel.org, ejt@redhat.com, ebiederm@xmission.com, tj@kernel.org, teigland@redhat.com, akpm@linux-foundation.org, davem@davemloft.net Subject: [dm-devel] [PATCH 04/16] workqueue: use new hashtable implementation X-BeenThere: dm-devel@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk Reply-To: device-mapper development List-Id: device-mapper development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Sender: dm-devel-bounces@redhat.com Errors-To: dm-devel-bounces@redhat.com Switch workqueues to use the new hashtable implementation. This reduces the amount of generic unrelated code in the workqueues. Signed-off-by: Sasha Levin --- kernel/workqueue.c | 86 +++++++++------------------------------------------- 1 files changed, 15 insertions(+), 71 deletions(-) diff --git a/kernel/workqueue.c b/kernel/workqueue.c index 692d976..1db6328 100644 --- a/kernel/workqueue.c +++ b/kernel/workqueue.c @@ -41,6 +41,7 @@ #include #include #include +#include #include "workqueue_sched.h" @@ -82,8 +83,6 @@ enum { NR_WORKER_POOLS = 2, /* # worker pools per gcwq */ BUSY_WORKER_HASH_ORDER = 6, /* 64 pointers */ - BUSY_WORKER_HASH_SIZE = 1 << BUSY_WORKER_HASH_ORDER, - BUSY_WORKER_HASH_MASK = BUSY_WORKER_HASH_SIZE - 1, MAX_IDLE_WORKERS_RATIO = 4, /* 1/4 of busy can be idle */ IDLE_WORKER_TIMEOUT = 300 * HZ, /* keep idle ones for 5 mins */ @@ -180,7 +179,7 @@ struct global_cwq { unsigned int flags; /* L: GCWQ_* flags */ /* workers are chained either in busy_hash or pool idle_list */ - struct hlist_head busy_hash[BUSY_WORKER_HASH_SIZE]; + DEFINE_HASHTABLE(busy_hash, BUSY_WORKER_HASH_ORDER); /* L: hash of busy workers */ struct worker_pool pools[2]; /* normal and highpri pools */ @@ -288,8 +287,7 @@ EXPORT_SYMBOL_GPL(system_nrt_freezable_wq); (pool) < &(gcwq)->pools[NR_WORKER_POOLS]; (pool)++) #define for_each_busy_worker(worker, i, pos, gcwq) \ - for (i = 0; i < BUSY_WORKER_HASH_SIZE; i++) \ - hlist_for_each_entry(worker, pos, &gcwq->busy_hash[i], hentry) + hash_for_each(gcwq->busy_hash, i, pos, worker, hentry) static inline int __next_gcwq_cpu(int cpu, const struct cpumask *mask, unsigned int sw) @@ -822,63 +820,6 @@ static inline void worker_clr_flags(struct worker *worker, unsigned int flags) } /** - * busy_worker_head - return the busy hash head for a work - * @gcwq: gcwq of interest - * @work: work to be hashed - * - * Return hash head of @gcwq for @work. - * - * CONTEXT: - * spin_lock_irq(gcwq->lock). - * - * RETURNS: - * Pointer to the hash head. - */ -static struct hlist_head *busy_worker_head(struct global_cwq *gcwq, - struct work_struct *work) -{ - const int base_shift = ilog2(sizeof(struct work_struct)); - unsigned long v = (unsigned long)work; - - /* simple shift and fold hash, do we need something better? */ - v >>= base_shift; - v += v >> BUSY_WORKER_HASH_ORDER; - v &= BUSY_WORKER_HASH_MASK; - - return &gcwq->busy_hash[v]; -} - -/** - * __find_worker_executing_work - find worker which is executing a work - * @gcwq: gcwq of interest - * @bwh: hash head as returned by busy_worker_head() - * @work: work to find worker for - * - * Find a worker which is executing @work on @gcwq. @bwh should be - * the hash head obtained by calling busy_worker_head() with the same - * work. - * - * CONTEXT: - * spin_lock_irq(gcwq->lock). - * - * RETURNS: - * Pointer to worker which is executing @work if found, NULL - * otherwise. - */ -static struct worker *__find_worker_executing_work(struct global_cwq *gcwq, - struct hlist_head *bwh, - struct work_struct *work) -{ - struct worker *worker; - struct hlist_node *tmp; - - hlist_for_each_entry(worker, tmp, bwh, hentry) - if (worker->current_work == work) - return worker; - return NULL; -} - -/** * find_worker_executing_work - find worker which is executing a work * @gcwq: gcwq of interest * @work: work to find worker for @@ -897,8 +838,14 @@ static struct worker *__find_worker_executing_work(struct global_cwq *gcwq, static struct worker *find_worker_executing_work(struct global_cwq *gcwq, struct work_struct *work) { - return __find_worker_executing_work(gcwq, busy_worker_head(gcwq, work), - work); + struct worker *worker; + struct hlist_node *tmp; + + hash_for_each_possible(gcwq->busy_hash, worker, tmp, hentry, (unsigned long)work) + if (worker->current_work == work) + return worker; + + return NULL; } /** @@ -1932,7 +1879,6 @@ __acquires(&gcwq->lock) struct cpu_workqueue_struct *cwq = get_work_cwq(work); struct worker_pool *pool = worker->pool; struct global_cwq *gcwq = pool->gcwq; - struct hlist_head *bwh = busy_worker_head(gcwq, work); bool cpu_intensive = cwq->wq->flags & WQ_CPU_INTENSIVE; work_func_t f = work->func; int work_color; @@ -1964,7 +1910,7 @@ __acquires(&gcwq->lock) * already processing the work. If so, defer the work to the * currently executing one. */ - collision = __find_worker_executing_work(gcwq, bwh, work); + collision = find_worker_executing_work(gcwq, work); if (unlikely(collision)) { move_linked_works(work, &collision->scheduled, NULL); return; @@ -1972,7 +1918,7 @@ __acquires(&gcwq->lock) /* claim and process */ debug_work_deactivate(work); - hlist_add_head(&worker->hentry, bwh); + hash_add(gcwq->busy_hash, &worker->hentry, (unsigned long)worker); worker->current_work = work; worker->current_cwq = cwq; work_color = get_work_color(work); @@ -2027,7 +1973,7 @@ __acquires(&gcwq->lock) worker_clr_flags(worker, WORKER_CPU_INTENSIVE); /* we're done with it, release */ - hlist_del_init(&worker->hentry); + hash_del(&worker->hentry); worker->current_work = NULL; worker->current_cwq = NULL; cwq_dec_nr_in_flight(cwq, work_color, false); @@ -3690,7 +3636,6 @@ out_unlock: static int __init init_workqueues(void) { unsigned int cpu; - int i; cpu_notifier(workqueue_cpu_up_callback, CPU_PRI_WORKQUEUE_UP); cpu_notifier(workqueue_cpu_down_callback, CPU_PRI_WORKQUEUE_DOWN); @@ -3704,8 +3649,7 @@ static int __init init_workqueues(void) gcwq->cpu = cpu; gcwq->flags |= GCWQ_DISASSOCIATED; - for (i = 0; i < BUSY_WORKER_HASH_SIZE; i++) - INIT_HLIST_HEAD(&gcwq->busy_hash[i]); + hash_init(gcwq->busy_hash); for_each_worker_pool(pool, gcwq) { pool->gcwq = gcwq;