From patchwork Wed Aug 28 19:55:17 2013 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kent Overstreet X-Patchwork-Id: 2850993 Return-Path: X-Original-To: patchwork-kvm@patchwork.kernel.org Delivered-To: patchwork-parsemail@patchwork1.web.kernel.org Received: from mail.kernel.org (mail.kernel.org [198.145.19.201]) by patchwork1.web.kernel.org (Postfix) with ESMTP id 6594A9F271 for ; Wed, 28 Aug 2013 19:56:58 +0000 (UTC) Received: from mail.kernel.org (localhost [127.0.0.1]) by mail.kernel.org (Postfix) with ESMTP id CD418205AB for ; Wed, 28 Aug 2013 19:56:52 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id D37DE20598 for ; Wed, 28 Aug 2013 19:56:51 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1755663Ab3H1T4s (ORCPT ); Wed, 28 Aug 2013 15:56:48 -0400 Received: from mail-pb0-f47.google.com ([209.85.160.47]:60250 "EHLO mail-pb0-f47.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1755517Ab3H1TzN (ORCPT ); Wed, 28 Aug 2013 15:55:13 -0400 Received: by mail-pb0-f47.google.com with SMTP id rr4so6622196pbb.20 for ; Wed, 28 Aug 2013 12:55:12 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-type:content-disposition:in-reply-to :user-agent; bh=CZIan7lEtd+SBC9fvcs7lA/ugz6A0yOHAF9xfSx/f98=; b=IUwsoMKsznzVR7X2F1GHIcSOJ3L7vtv7tjiJpkzoWUewuNpxxrBHaQ1geX/yUzkeeA BhqSirOyxFDxd4975yZFdbeLFSh7X1yl9xdqnegFHNpbHj/zF/s6MKO3IhHu/QSbmRT6 SQRLkXzpwxVWkz/JwlHHvRIL+wTj+/cZQ97PARcgfZTQcCQs0snTbevdhFcXZa2lrCvn vI8kUL7F5s1rPgqksqu+sewa/oE7eyyOp/3G6fL7OUo7Vh2+LtincyKDSDk4hplf7coF toR48fcpZ2ZJ8vqmwysAu2Xzcgq1TlFTh2rcoCGyTBhkmv6vi7O7TpuVtV9JCq5KDKrN JUYw== X-Gm-Message-State: ALoCoQml1G6YlqeDnXp6J5/Rm0r0q1vKv6s4LciJ1Hg10Ns8kLpwS9wJ3/vdonmUutcnS8jrbPlU X-Received: by 10.66.196.110 with SMTP id il14mr241246pac.130.1377719712418; Wed, 28 Aug 2013 12:55:12 -0700 (PDT) Received: from kmo-pixel (c-107-3-149-94.hsd1.ca.comcast.net. [107.3.149.94]) by mx.google.com with ESMTPSA id mz5sm33038046pbc.18.1969.12.31.16.00.00 (version=TLSv1.2 cipher=RC4-SHA bits=128/128); Wed, 28 Aug 2013 12:55:11 -0700 (PDT) Date: Wed, 28 Aug 2013 12:55:17 -0700 From: Kent Overstreet To: Andrew Morton Cc: "Nicholas A. Bellinger" , target-devel , lf-virt , lkml , kvm-devel , "Michael S. Tsirkin" , Asias He , Jens Axboe , Tejun Heo , Ingo Molnar , Andi Kleen , Christoph Lameter , Oleg Nesterov , Christoph Lameter Subject: [PATCH] percpu ida: Switch to cpumask_t, add some comments Message-ID: <20130828195517.GF8032@kmo-pixel> References: <1376694549-20609-1-git-send-email-nab@linux-iscsi.org> <1376694549-20609-2-git-send-email-nab@linux-iscsi.org> <20130820143157.f91bf59d16352989b54e431e@linux-foundation.org> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <20130820143157.f91bf59d16352989b54e431e@linux-foundation.org> User-Agent: Mutt/1.5.21 (2010-09-15) Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org X-Spam-Status: No, score=-9.4 required=5.0 tests=BAYES_00, RCVD_IN_DNSWL_HI, RP_MATCHES_RCVD, UNPARSEABLE_RELAY autolearn=unavailable version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on mail.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Fixup patch, addressing Andrew's review feedback: Signed-off-by: Kent Overstreet --- include/linux/idr.h | 2 +- lib/idr.c | 38 +++++++++++++++++++++----------------- 2 files changed, 22 insertions(+), 18 deletions(-) diff --git a/include/linux/idr.h b/include/linux/idr.h index f0db12b..cdf39be 100644 --- a/include/linux/idr.h +++ b/include/linux/idr.h @@ -267,7 +267,7 @@ struct percpu_ida { * will just keep looking - but the bitmap _must_ be set whenever a * percpu freelist does have tags. */ - unsigned long *cpus_have_tags; + cpumask_t cpus_have_tags; struct { spinlock_t lock; diff --git a/lib/idr.c b/lib/idr.c index 26495e1..15c021c 100644 --- a/lib/idr.c +++ b/lib/idr.c @@ -1178,7 +1178,13 @@ EXPORT_SYMBOL(ida_init); #define IDA_PCPU_SIZE ((IDA_PCPU_BATCH_MOVE * 3) / 2) struct percpu_ida_cpu { + /* + * Even though this is percpu, we need a lock for tag stealing by remote + * CPUs: + */ spinlock_t lock; + + /* nr_free/freelist form a stack of free IDs */ unsigned nr_free; unsigned freelist[]; }; @@ -1209,21 +1215,21 @@ static inline void steal_tags(struct percpu_ida *pool, unsigned cpus_have_tags, cpu = pool->cpu_last_stolen; struct percpu_ida_cpu *remote; - for (cpus_have_tags = bitmap_weight(pool->cpus_have_tags, nr_cpu_ids); + for (cpus_have_tags = cpumask_weight(&pool->cpus_have_tags); cpus_have_tags * IDA_PCPU_SIZE > pool->nr_tags / 2; cpus_have_tags--) { - cpu = find_next_bit(pool->cpus_have_tags, nr_cpu_ids, cpu); + cpu = cpumask_next(cpu, &pool->cpus_have_tags); - if (cpu == nr_cpu_ids) - cpu = find_first_bit(pool->cpus_have_tags, nr_cpu_ids); + if (cpu >= nr_cpu_ids) + cpu = cpumask_first(&pool->cpus_have_tags); - if (cpu == nr_cpu_ids) + if (cpu >= nr_cpu_ids) BUG(); pool->cpu_last_stolen = cpu; remote = per_cpu_ptr(pool->tag_cpu, cpu); - clear_bit(cpu, pool->cpus_have_tags); + cpumask_clear_cpu(cpu, &pool->cpus_have_tags); if (remote == tags) continue; @@ -1246,6 +1252,10 @@ static inline void steal_tags(struct percpu_ida *pool, } } +/* + * Pop up to IDA_PCPU_BATCH_MOVE IDs off the global freelist, and push them onto + * our percpu freelist: + */ static inline void alloc_global_tags(struct percpu_ida *pool, struct percpu_ida_cpu *tags) { @@ -1317,8 +1327,8 @@ int percpu_ida_alloc(struct percpu_ida *pool, gfp_t gfp) if (tags->nr_free) { tag = tags->freelist[--tags->nr_free]; if (tags->nr_free) - set_bit(smp_processor_id(), - pool->cpus_have_tags); + cpumask_set_cpu(smp_processor_id(), + &pool->cpus_have_tags); } spin_unlock(&pool->lock); @@ -1363,8 +1373,8 @@ void percpu_ida_free(struct percpu_ida *pool, unsigned tag) spin_unlock(&tags->lock); if (nr_free == 1) { - set_bit(smp_processor_id(), - pool->cpus_have_tags); + cpumask_set_cpu(smp_processor_id(), + &pool->cpus_have_tags); wake_up(&pool->wait); } @@ -1398,7 +1408,6 @@ EXPORT_SYMBOL_GPL(percpu_ida_free); void percpu_ida_destroy(struct percpu_ida *pool) { free_percpu(pool->tag_cpu); - kfree(pool->cpus_have_tags); free_pages((unsigned long) pool->freelist, get_order(pool->nr_tags * sizeof(unsigned))); } @@ -1428,7 +1437,7 @@ int percpu_ida_init(struct percpu_ida *pool, unsigned long nr_tags) /* Guard against overflow */ if (nr_tags > (unsigned) INT_MAX + 1) { - pr_err("tags.c: nr_tags too large\n"); + pr_err("percpu_ida_init(): nr_tags too large\n"); return -EINVAL; } @@ -1442,11 +1451,6 @@ int percpu_ida_init(struct percpu_ida *pool, unsigned long nr_tags) pool->nr_free = nr_tags; - pool->cpus_have_tags = kzalloc(BITS_TO_LONGS(nr_cpu_ids) * - sizeof(unsigned long), GFP_KERNEL); - if (!pool->cpus_have_tags) - goto err; - pool->tag_cpu = __alloc_percpu(sizeof(struct percpu_ida_cpu) + IDA_PCPU_SIZE * sizeof(unsigned), sizeof(unsigned));