From patchwork Thu Jun 12 03:21:42 2014 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joonsoo Kim X-Patchwork-Id: 4339431 Return-Path: X-Original-To: patchwork-linux-arm@patchwork.kernel.org Delivered-To: patchwork-parsemail@patchwork1.web.kernel.org Received: from mail.kernel.org (mail.kernel.org [198.145.19.201]) by patchwork1.web.kernel.org (Postfix) with ESMTP id 5CF6E9F314 for ; Thu, 12 Jun 2014 03:21:53 +0000 (UTC) Received: from mail.kernel.org (localhost [127.0.0.1]) by mail.kernel.org (Postfix) with ESMTP id 7310F2022A for ; Thu, 12 Jun 2014 03:21:52 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.9]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 86522200DF for ; Thu, 12 Jun 2014 03:21:51 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.80.1 #2 (Red Hat Linux)) id 1WuvY6-0001uQ-7R; Thu, 12 Jun 2014 03:19:34 +0000 Received: from lgeamrelo01.lge.com ([156.147.1.125]) by bombadil.infradead.org with esmtp (Exim 4.80.1 #2 (Red Hat Linux)) id 1WuvWo-00018t-1m for linux-arm-kernel@lists.infradead.org; Thu, 12 Jun 2014 03:18:18 +0000 Received: from unknown (HELO js1304-P5Q-DELUXE.LGE.NET) (10.177.220.145) by 156.147.1.125 with ESMTP; 12 Jun 2014 12:17:50 +0900 X-Original-SENDERIP: 10.177.220.145 X-Original-MAILFROM: iamjoonsoo.kim@lge.com From: Joonsoo Kim To: Andrew Morton , "Aneesh Kumar K.V" , Marek Szyprowski , Michal Nazarewicz Subject: [PATCH v2 05/10] DMA, CMA: support arbitrary bitmap granularity Date: Thu, 12 Jun 2014 12:21:42 +0900 Message-Id: <1402543307-29800-6-git-send-email-iamjoonsoo.kim@lge.com> X-Mailer: git-send-email 1.7.9.5 In-Reply-To: <1402543307-29800-1-git-send-email-iamjoonsoo.kim@lge.com> References: <1402543307-29800-1-git-send-email-iamjoonsoo.kim@lge.com> X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20140611_201814_446433_77DE91FA X-CRM114-Status: GOOD ( 16.08 ) X-Spam-Score: -0.7 (/) Cc: Russell King - ARM Linux , kvm@vger.kernel.org, linux-mm@kvack.org, Gleb Natapov , Greg Kroah-Hartman , Alexander Graf , kvm-ppc@vger.kernel.org, linux-kernel@vger.kernel.org, Minchan Kim , Paul Mackerras , Benjamin Herrenschmidt , Paolo Bonzini , Joonsoo Kim , linuxppc-dev@lists.ozlabs.org, linux-arm-kernel@lists.infradead.org X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org X-Spam-Status: No, score=-2.5 required=5.0 tests=BAYES_00,RP_MATCHES_RCVD, UNPARSEABLE_RELAY autolearn=unavailable version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on mail.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP ppc kvm's cma region management requires arbitrary bitmap granularity, since they want to reserve very large memory and manage this region with bitmap that one bit for several pages to reduce management overheads. So support arbitrary bitmap granularity for following generalization. Signed-off-by: Joonsoo Kim Acked-by: Minchan Kim Acked-by: Zhang Yanfei Acked-by: Michal Nazarewicz Reviewed-by: Aneesh Kumar K.V diff --git a/drivers/base/dma-contiguous.c b/drivers/base/dma-contiguous.c index bc4c171..9bc9340 100644 --- a/drivers/base/dma-contiguous.c +++ b/drivers/base/dma-contiguous.c @@ -38,6 +38,7 @@ struct cma { unsigned long base_pfn; unsigned long count; unsigned long *bitmap; + int order_per_bit; /* Order of pages represented by one bit */ struct mutex lock; }; @@ -157,9 +158,38 @@ void __init dma_contiguous_reserve(phys_addr_t limit) static DEFINE_MUTEX(cma_mutex); +static unsigned long cma_bitmap_aligned_mask(struct cma *cma, int align_order) +{ + return (1 << (align_order >> cma->order_per_bit)) - 1; +} + +static unsigned long cma_bitmap_maxno(struct cma *cma) +{ + return cma->count >> cma->order_per_bit; +} + +static unsigned long cma_bitmap_pages_to_bits(struct cma *cma, + unsigned long pages) +{ + return ALIGN(pages, 1 << cma->order_per_bit) >> cma->order_per_bit; +} + +static void clear_cma_bitmap(struct cma *cma, unsigned long pfn, int count) +{ + unsigned long bitmapno, nr_bits; + + bitmapno = (pfn - cma->base_pfn) >> cma->order_per_bit; + nr_bits = cma_bitmap_pages_to_bits(cma, count); + + mutex_lock(&cma->lock); + bitmap_clear(cma->bitmap, bitmapno, nr_bits); + mutex_unlock(&cma->lock); +} + static int __init cma_activate_area(struct cma *cma) { - int bitmap_size = BITS_TO_LONGS(cma->count) * sizeof(long); + int bitmap_maxno = cma_bitmap_maxno(cma); + int bitmap_size = BITS_TO_LONGS(bitmap_maxno) * sizeof(long); unsigned long base_pfn = cma->base_pfn, pfn = base_pfn; unsigned i = cma->count >> pageblock_order; struct zone *zone; @@ -221,6 +251,7 @@ core_initcall(cma_init_reserved_areas); * @base: Base address of the reserved area optional, use 0 for any * @limit: End address of the reserved memory (optional, 0 for any). * @alignment: Alignment for the contiguous memory area, should be power of 2 + * @order_per_bit: Order of pages represented by one bit on bitmap. * @res_cma: Pointer to store the created cma region. * @fixed: hint about where to place the reserved area * @@ -235,7 +266,7 @@ core_initcall(cma_init_reserved_areas); */ static int __init __dma_contiguous_reserve_area(phys_addr_t size, phys_addr_t base, phys_addr_t limit, - phys_addr_t alignment, + phys_addr_t alignment, int order_per_bit, struct cma **res_cma, bool fixed) { struct cma *cma = &cma_areas[cma_area_count]; @@ -269,6 +300,8 @@ static int __init __dma_contiguous_reserve_area(phys_addr_t size, base = ALIGN(base, alignment); size = ALIGN(size, alignment); limit &= ~(alignment - 1); + /* size should be aligned with order_per_bit */ + BUG_ON(!IS_ALIGNED(size >> PAGE_SHIFT, 1 << order_per_bit)); /* Reserve memory */ if (base && fixed) { @@ -294,6 +327,7 @@ static int __init __dma_contiguous_reserve_area(phys_addr_t size, */ cma->base_pfn = PFN_DOWN(base); cma->count = size >> PAGE_SHIFT; + cma->order_per_bit = order_per_bit; *res_cma = cma; cma_area_count++; @@ -313,7 +347,7 @@ int __init dma_contiguous_reserve_area(phys_addr_t size, phys_addr_t base, { int ret; - ret = __dma_contiguous_reserve_area(size, base, limit, 0, + ret = __dma_contiguous_reserve_area(size, base, limit, 0, 0, res_cma, fixed); if (ret) return ret; @@ -324,13 +358,6 @@ int __init dma_contiguous_reserve_area(phys_addr_t size, phys_addr_t base, return 0; } -static void clear_cma_bitmap(struct cma *cma, unsigned long pfn, int count) -{ - mutex_lock(&cma->lock); - bitmap_clear(cma->bitmap, pfn - cma->base_pfn, count); - mutex_unlock(&cma->lock); -} - /** * dma_alloc_from_contiguous() - allocate pages from contiguous area * @dev: Pointer to device for which the allocation is performed. @@ -345,7 +372,8 @@ static void clear_cma_bitmap(struct cma *cma, unsigned long pfn, int count) static struct page *__dma_alloc_from_contiguous(struct cma *cma, int count, unsigned int align) { - unsigned long mask, pfn, pageno, start = 0; + unsigned long mask, pfn, start = 0; + unsigned long bitmap_maxno, bitmapno, nr_bits; struct page *page = NULL; int ret; @@ -358,18 +386,19 @@ static struct page *__dma_alloc_from_contiguous(struct cma *cma, int count, if (!count) return NULL; - mask = (1 << align) - 1; - + mask = cma_bitmap_aligned_mask(cma, align); + bitmap_maxno = cma_bitmap_maxno(cma); + nr_bits = cma_bitmap_pages_to_bits(cma, count); for (;;) { mutex_lock(&cma->lock); - pageno = bitmap_find_next_zero_area(cma->bitmap, cma->count, - start, count, mask); - if (pageno >= cma->count) { + bitmapno = bitmap_find_next_zero_area(cma->bitmap, + bitmap_maxno, start, nr_bits, mask); + if (bitmapno >= bitmap_maxno) { mutex_unlock(&cma->lock); break; } - bitmap_set(cma->bitmap, pageno, count); + bitmap_set(cma->bitmap, bitmapno, nr_bits); /* * It's safe to drop the lock here. We've marked this region for * our exclusive use. If the migration fails we will take the @@ -377,7 +406,7 @@ static struct page *__dma_alloc_from_contiguous(struct cma *cma, int count, */ mutex_unlock(&cma->lock); - pfn = cma->base_pfn + pageno; + pfn = cma->base_pfn + (bitmapno << cma->order_per_bit); mutex_lock(&cma_mutex); ret = alloc_contig_range(pfn, pfn + count, MIGRATE_CMA); mutex_unlock(&cma_mutex); @@ -392,7 +421,7 @@ static struct page *__dma_alloc_from_contiguous(struct cma *cma, int count, pr_debug("%s(): memory range at %p is busy, retrying\n", __func__, pfn_to_page(pfn)); /* try again with a bit different memory target */ - start = pageno + mask + 1; + start = bitmapno + mask + 1; } pr_debug("%s(): returned %p\n", __func__, page);