From patchwork Thu Jan 19 10:02:20 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christoph Hellwig X-Patchwork-Id: 13107651 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id C8F4EC38142 for ; Thu, 19 Jan 2023 10:02:59 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 63A4B6B007D; Thu, 19 Jan 2023 05:02:59 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 59C136B007E; Thu, 19 Jan 2023 05:02:59 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 43CCD6B0080; Thu, 19 Jan 2023 05:02:59 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 2C9126B007D for ; Thu, 19 Jan 2023 05:02:59 -0500 (EST) Received: from smtpin01.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id EF95D80948 for ; Thu, 19 Jan 2023 10:02:58 +0000 (UTC) X-FDA: 80371110036.01.CBB7A8B Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) by imf24.hostedemail.com (Postfix) with ESMTP id E3681180018 for ; Thu, 19 Jan 2023 10:02:56 +0000 (UTC) Authentication-Results: imf24.hostedemail.com; dkim=pass header.d=infradead.org header.s=bombadil.20210309 header.b=Q3Mzg9Ex; spf=none (imf24.hostedemail.com: domain of BATV+d6c99cb0ec1a2cfa7d20+7088+infradead.org+hch@bombadil.srs.infradead.org has no SPF policy when checking 198.137.202.133) smtp.mailfrom=BATV+d6c99cb0ec1a2cfa7d20+7088+infradead.org+hch@bombadil.srs.infradead.org; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1674122577; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=FD2JG+d6yA1abtYvFdChjy0cPOnrDbMf2ONV7gibGRw=; b=hTDU5vz/t+ltWZeWgz+Seoy0j1dm8TLDEe4+veMBhvG16CpkUom7vIF79lwXr68XQ1lJ2A WFxndijBN+1L2qJcP7SAip2FxdBHmXuHSQfEta1HPnIfbyLTzgs5vxSJvCDCHscIko3WzI Xq9nSCVr/k52A7X1krLTZUbMAiCqoO0= ARC-Authentication-Results: i=1; imf24.hostedemail.com; dkim=pass header.d=infradead.org header.s=bombadil.20210309 header.b=Q3Mzg9Ex; spf=none (imf24.hostedemail.com: domain of BATV+d6c99cb0ec1a2cfa7d20+7088+infradead.org+hch@bombadil.srs.infradead.org has no SPF policy when checking 198.137.202.133) smtp.mailfrom=BATV+d6c99cb0ec1a2cfa7d20+7088+infradead.org+hch@bombadil.srs.infradead.org; dmarc=none ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1674122577; a=rsa-sha256; cv=none; b=F1B42rMkwoyh0qdQ0p3ua6RiLsbwwJx2IXqsgNTHqHbHu+IaBCFhAG56ziRKvvk2HgvTSE n0ykt+sNdHma+0C+izirlUhCQ9ms7FFb5EETfYtnNz1u0/ab1fWz4N7k+8IgseM/Sv7rTg crrjVQf9ocCrpk6ZcbpsmKIkPysPHP0= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender :Reply-To:Content-Type:Content-ID:Content-Description; bh=FD2JG+d6yA1abtYvFdChjy0cPOnrDbMf2ONV7gibGRw=; b=Q3Mzg9ExOfQ9QIFTYVyn+Td4Mx b9PuPdiTdsjSkZt2rCoUKE2lEOxr1sf23v+d984Fpyeg62PW18/wLtwZ4bBsRvz2kxlOFkZXOvBlQ CDwRrLk2w1Dla3vHsHPWYwBsBkVmbozdHf7Ae+qbGq1tfXzySCrmr2TAKhw6YHkmfxDGrsKZrREKM ZfXlaGUW849DpkLqY1d850ORmqx3fzMya4S5nHSyy1g6mriRi40whoGbKmEVhLTCnegA6kT8UobKQ t9QQ2ZHvpyKKNoEXqOhw18/h4nKpho8m9GVKdcwOabodZeZ33HuAjjIE4kS/VjSaPdeWkssUAe8tq JpUK0TSw==; Received: from 089144197172.atnat0006.highway.a1.net ([89.144.197.172] helo=localhost) by bombadil.infradead.org with esmtpsa (Exim 4.94.2 #2 (Red Hat Linux)) id 1pIRl0-004Lc0-6v; Thu, 19 Jan 2023 10:02:54 +0000 From: Christoph Hellwig To: Andrew Morton , Uladzislau Rezki Cc: linux-mm@kvack.org Subject: [PATCH 04/10] mm: move vmalloc_init and free_work down in vmalloc.c Date: Thu, 19 Jan 2023 11:02:20 +0100 Message-Id: <20230119100226.789506-5-hch@lst.de> X-Mailer: git-send-email 2.39.0 In-Reply-To: <20230119100226.789506-1-hch@lst.de> References: <20230119100226.789506-1-hch@lst.de> MIME-Version: 1.0 X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org. See http://www.infradead.org/rpr.html X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: E3681180018 X-Rspam-User: X-Stat-Signature: cnx6ysgrfm5qob9rgby4mdmd76tat9hu X-HE-Tag: 1674122576-468820 X-HE-Meta: U2FsdGVkX19qDI/dnVSQRkDwjT1w5H43XQ+87QvVINFSBUCtOG32MhleaJr6RXS9dPGuOFk9K2QQiLDlVulA13W/JvVM0nTr920E1GpnUT4ldSB0rppnfhI3P7cc/pqbYz2+iXaJHn3nEM94gaBtAHV1QTLN9PiZZL9coNazu17c090kghYwi+tC+dE0GUCdtBhQdcnxjZ+T3GlPAQejKm9TTEdlrUZatuIM1b/mgTwZNbp6UvEBKPQS4Slwj4oktd7lGkz8dsi3zzxDIr+cncm2++yh2zL9fENn1qRLysYaxRh7m/z/YqpByIPHVwqjONTjXek3Zs4vHDfQmSJV8h9EtyxJGm62si7NBWwbm6pxzL0+EJR/dITIrKtlryhJCxJgvlBOUrZK19SHFqSVb5jUURLeSWlCeTs3aMG7JE4YDeOtdP5uiPZPL0HFu9GfJ4RWPkd5huG1ViQaSELjzGVFQL7OUtyZJMcHwRL8phKyg67fDB1u6ZdstvYS+FQMFZ3Duy3CEdBnLmWVbl1SRpg2Haw7ZoTWwujzwzUWFkdsTXMjIgI+/38KHauYBRYyXBl6wtL+SEtj6oK8PfViFidqlYAk2O80pWyEfl8rt/zs1rBWAmM3j/yIP5fYQuZ3elT8Gem+0u2HRsK/YSP9bWzxfynbX30kTFLW7ctYtPmoL8nL6nKg83f06JOSkOM+KK2Ko0W35yfiAtW7LZW4KeV6/NkzJwzoAtjOUNvc7+vG6grWqP2WAv+Cjo/EQG1mYBwvYTrPgI74myhiRJ1QmSWFL6VI3CpC4iJwkXYgjX7r3b8P/TfXvd+DUcqX6P/0Ky7bm4nfHQH2xeFdWNvs1sfQCTHOWCmk2W8Gk4TtuIL2TtVHzGJuJgV/KUh5IqMdMOEWRUp0VnOOd7aBoHEwnJz6XlfRVIb5l2Q7t+AMlLFAwmBI0YW9y7rZCjLATmA+jP5RiAW6eFLmYGnQftz JXuRy5qQ NhNzEBNzmO2Y5BqHQCa3WJbikyj1h3ktOJc/6QSV842PJ1g812VggIXsGMSRXrLBqa898HfD22rDx4O4I5fvbBMz5zAzxxtv7I9cuCHek2XMnmOis6mMB0DvFszQrFTr3Iqth+H5K/24GAqYoTwjWapHOnFtELr1cQxZwk0iw0jloYMdfYJwZJDGgHqQC7uIgWKPbCWH76GwX+KQ/xaOhYXOdYE6yLhXuh7brdh4d63ASFK8= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Move these two functions around a bit to avoid forward declarations. Signed-off-by: Christoph Hellwig Reviewed-by: Uladzislau Rezki (Sony) --- mm/vmalloc.c | 105 +++++++++++++++++++++++++-------------------------- 1 file changed, 52 insertions(+), 53 deletions(-) diff --git a/mm/vmalloc.c b/mm/vmalloc.c index fafb6227f4428f..daeb28b54663d5 100644 --- a/mm/vmalloc.c +++ b/mm/vmalloc.c @@ -89,17 +89,6 @@ struct vfree_deferred { }; static DEFINE_PER_CPU(struct vfree_deferred, vfree_deferred); -static void __vunmap(const void *, int); - -static void free_work(struct work_struct *w) -{ - struct vfree_deferred *p = container_of(w, struct vfree_deferred, wq); - struct llist_node *t, *llnode; - - llist_for_each_safe(llnode, t, llist_del_all(&p->list)) - __vunmap((void *)llnode, 1); -} - /*** Page table manipulation functions ***/ static int vmap_pte_range(pmd_t *pmd, unsigned long addr, unsigned long end, phys_addr_t phys_addr, pgprot_t prot, @@ -2449,48 +2438,6 @@ static void vmap_init_free_space(void) } } -void __init vmalloc_init(void) -{ - struct vmap_area *va; - struct vm_struct *tmp; - int i; - - /* - * Create the cache for vmap_area objects. - */ - vmap_area_cachep = KMEM_CACHE(vmap_area, SLAB_PANIC); - - for_each_possible_cpu(i) { - struct vmap_block_queue *vbq; - struct vfree_deferred *p; - - vbq = &per_cpu(vmap_block_queue, i); - spin_lock_init(&vbq->lock); - INIT_LIST_HEAD(&vbq->free); - p = &per_cpu(vfree_deferred, i); - init_llist_head(&p->list); - INIT_WORK(&p->wq, free_work); - } - - /* Import existing vmlist entries. */ - for (tmp = vmlist; tmp; tmp = tmp->next) { - va = kmem_cache_zalloc(vmap_area_cachep, GFP_NOWAIT); - if (WARN_ON_ONCE(!va)) - continue; - - va->va_start = (unsigned long)tmp->addr; - va->va_end = va->va_start + tmp->size; - va->vm = tmp; - insert_vmap_area(va, &vmap_area_root, &vmap_area_list); - } - - /* - * Now we can initialize a free vmap space. - */ - vmap_init_free_space(); - vmap_initialized = true; -} - static inline void setup_vmalloc_vm_locked(struct vm_struct *vm, struct vmap_area *va, unsigned long flags, const void *caller) { @@ -2769,6 +2716,15 @@ static void __vunmap(const void *addr, int deallocate_pages) kfree(area); } +static void delayed_vfree_work(struct work_struct *w) +{ + struct vfree_deferred *p = container_of(w, struct vfree_deferred, wq); + struct llist_node *t, *llnode; + + llist_for_each_safe(llnode, t, llist_del_all(&p->list)) + __vunmap((void *)llnode, 1); +} + /** * vfree_atomic - release memory allocated by vmalloc() * @addr: memory base address @@ -4315,3 +4271,46 @@ static int __init proc_vmalloc_init(void) module_init(proc_vmalloc_init); #endif + +void __init vmalloc_init(void) +{ + struct vmap_area *va; + struct vm_struct *tmp; + int i; + + /* + * Create the cache for vmap_area objects. + */ + vmap_area_cachep = KMEM_CACHE(vmap_area, SLAB_PANIC); + + for_each_possible_cpu(i) { + struct vmap_block_queue *vbq; + struct vfree_deferred *p; + + vbq = &per_cpu(vmap_block_queue, i); + spin_lock_init(&vbq->lock); + INIT_LIST_HEAD(&vbq->free); + p = &per_cpu(vfree_deferred, i); + init_llist_head(&p->list); + INIT_WORK(&p->wq, delayed_vfree_work); + } + + /* Import existing vmlist entries. */ + for (tmp = vmlist; tmp; tmp = tmp->next) { + va = kmem_cache_zalloc(vmap_area_cachep, GFP_NOWAIT); + if (WARN_ON_ONCE(!va)) + continue; + + va->va_start = (unsigned long)tmp->addr; + va->va_end = va->va_start + tmp->size; + va->vm = tmp; + insert_vmap_area(va, &vmap_area_root, &vmap_area_list); + } + + /* + * Now we can initialize a free vmap space. + */ + vmap_init_free_space(); + vmap_initialized = true; +} +