From patchwork Tue Nov 27 10:00:18 2012 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Wen Congyang X-Patchwork-Id: 1810021 Return-Path: X-Original-To: patchwork-linux-sh@patchwork.kernel.org Delivered-To: patchwork-process-083081@patchwork1.kernel.org Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by patchwork1.kernel.org (Postfix) with ESMTP id A43663FC54 for ; Tue, 27 Nov 2012 10:01:24 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1758722Ab2K0J6Z (ORCPT ); Tue, 27 Nov 2012 04:58:25 -0500 Received: from cn.fujitsu.com ([222.73.24.84]:39448 "EHLO song.cn.fujitsu.com" rhost-flags-OK-FAIL-OK-OK) by vger.kernel.org with ESMTP id S1758659Ab2K0J6V (ORCPT ); Tue, 27 Nov 2012 04:58:21 -0500 X-IronPort-AV: E=Sophos;i="4.83,327,1352044800"; d="scan'208";a="6284834" Received: from unknown (HELO tang.cn.fujitsu.com) ([10.167.250.3]) by song.cn.fujitsu.com with ESMTP; 27 Nov 2012 17:56:28 +0800 Received: from fnstmail02.fnst.cn.fujitsu.com (tang.cn.fujitsu.com [127.0.0.1]) by tang.cn.fujitsu.com (8.14.3/8.13.1) with ESMTP id qAR9w5Hl002920; Tue, 27 Nov 2012 17:58:09 +0800 Received: from ghost.fnst.cn.fujitsu.com ([10.167.225.226]) by fnstmail02.fnst.cn.fujitsu.com (Lotus Domino Release 8.5.3) with ESMTP id 2012112717532986-97636 ; Tue, 27 Nov 2012 17:53:29 +0800 From: Wen Congyang To: x86@kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-acpi@vger.kernel.org, linux-s390@vger.kernel.org, linux-sh@vger.kernel.org, linux-ia64@vger.kernel.org, cmetcalf@tilera.com, sparclinux@vger.kernel.org Cc: David Rientjes , Jiang Liu , Len Brown , benh@kernel.crashing.org, paulus@samba.org, Christoph Lameter , Minchan Kim , Andrew Morton , KOSAKI Motohiro , Yasuaki Ishimatsu , Jianguo Wu , Wen Congyang Subject: [Patch v4 08/12] memory-hotplug: remove memmap of sparse-vmemmap Date: Tue, 27 Nov 2012 18:00:18 +0800 Message-Id: <1354010422-19648-9-git-send-email-wency@cn.fujitsu.com> X-Mailer: git-send-email 1.8.0 In-Reply-To: <1354010422-19648-1-git-send-email-wency@cn.fujitsu.com> References: <1354010422-19648-1-git-send-email-wency@cn.fujitsu.com> X-MIMETrack: Itemize by SMTP Server on mailserver/fnst(Release 8.5.3|September 15, 2011) at 2012/11/27 17:53:29, Serialize by Router on mailserver/fnst(Release 8.5.3|September 15, 2011) at 2012/11/27 17:57:46, Serialize complete at 2012/11/27 17:57:46 Sender: linux-sh-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-sh@vger.kernel.org From: Yasuaki Ishimatsu All pages of virtual mapping in removed memory cannot be freed, since some pages used as PGD/PUD includes not only removed memory but also other memory. So the patch checks whether page can be freed or not. How to check whether page can be freed or not? 1. When removing memory, the page structs of the revmoved memory are filled with 0FD. 2. All page structs are filled with 0xFD on PT/PMD, PT/PMD can be cleared. In this case, the page used as PT/PMD can be freed. Applying patch, __remove_section() of CONFIG_SPARSEMEM_VMEMMAP is integrated into one. So __remove_section() of CONFIG_SPARSEMEM_VMEMMAP is deleted. Note: vmemmap_kfree() and vmemmap_free_bootmem() are not implemented for ia64, ppc, s390, and sparc. CC: David Rientjes CC: Jiang Liu CC: Len Brown CC: Christoph Lameter Cc: Minchan Kim CC: Andrew Morton CC: KOSAKI Motohiro Signed-off-by: Yasuaki Ishimatsu Signed-off-by: Jianguo Wu Signed-off-by: Wen Congyang Reviewed-by: Tang Chen --- arch/ia64/mm/discontig.c | 8 ++++ arch/powerpc/mm/init_64.c | 8 ++++ arch/s390/mm/vmem.c | 8 ++++ arch/sparc/mm/init_64.c | 8 ++++ arch/x86/mm/init_64.c | 119 ++++++++++++++++++++++++++++++++++++++++++++++ include/linux/mm.h | 2 + mm/memory_hotplug.c | 17 +------ mm/sparse.c | 19 ++++---- 8 files changed, 165 insertions(+), 24 deletions(-) diff --git a/arch/ia64/mm/discontig.c b/arch/ia64/mm/discontig.c index 33943db..0d23b69 100644 --- a/arch/ia64/mm/discontig.c +++ b/arch/ia64/mm/discontig.c @@ -823,6 +823,14 @@ int __meminit vmemmap_populate(struct page *start_page, return vmemmap_populate_basepages(start_page, size, node); } +void vmemmap_kfree(struct page *memmap, unsigned long nr_pages) +{ +} + +void vmemmap_free_bootmem(struct page *memmap, unsigned long nr_pages) +{ +} + void register_page_bootmem_memmap(unsigned long section_nr, struct page *start_page, unsigned long size) { diff --git a/arch/powerpc/mm/init_64.c b/arch/powerpc/mm/init_64.c index 6466440..df7d155 100644 --- a/arch/powerpc/mm/init_64.c +++ b/arch/powerpc/mm/init_64.c @@ -298,6 +298,14 @@ int __meminit vmemmap_populate(struct page *start_page, return 0; } +void vmemmap_kfree(struct page *memmap, unsigned long nr_pages) +{ +} + +void vmemmap_free_bootmem(struct page *memmap, unsigned long nr_pages) +{ +} + void register_page_bootmem_memmap(unsigned long section_nr, struct page *start_page, unsigned long size) { diff --git a/arch/s390/mm/vmem.c b/arch/s390/mm/vmem.c index 4f4803a..ab69c34 100644 --- a/arch/s390/mm/vmem.c +++ b/arch/s390/mm/vmem.c @@ -236,6 +236,14 @@ out: return ret; } +void vmemmap_kfree(struct page *memmap, unsigned long nr_pages) +{ +} + +void vmemmap_free_bootmem(struct page *memmap, unsigned long nr_pages) +{ +} + void register_page_bootmem_memmap(unsigned long section_nr, struct page *start_page, unsigned long size) { diff --git a/arch/sparc/mm/init_64.c b/arch/sparc/mm/init_64.c index 75a984b..546855d 100644 --- a/arch/sparc/mm/init_64.c +++ b/arch/sparc/mm/init_64.c @@ -2232,6 +2232,14 @@ void __meminit vmemmap_populate_print_last(void) } } +void vmemmap_kfree(struct page *memmap, unsigned long nr_pages) +{ +} + +void vmemmap_free_bootmem(struct page *memmap, unsigned long nr_pages) +{ +} + void register_page_bootmem_memmap(unsigned long section_nr, struct page *start_page, unsigned long size) { diff --git a/arch/x86/mm/init_64.c b/arch/x86/mm/init_64.c index 795dae3..e85626d 100644 --- a/arch/x86/mm/init_64.c +++ b/arch/x86/mm/init_64.c @@ -998,6 +998,125 @@ vmemmap_populate(struct page *start_page, unsigned long size, int node) return 0; } +#define PAGE_INUSE 0xFD + +unsigned long find_and_clear_pte_page(unsigned long addr, unsigned long end, + struct page **pp, int *page_size) +{ + pgd_t *pgd; + pud_t *pud; + pmd_t *pmd; + pte_t *pte = NULL; + void *page_addr; + unsigned long next; + + *pp = NULL; + + pgd = pgd_offset_k(addr); + if (pgd_none(*pgd)) + return pgd_addr_end(addr, end); + + pud = pud_offset(pgd, addr); + if (pud_none(*pud)) + return pud_addr_end(addr, end); + + if (!cpu_has_pse) { + next = (addr + PAGE_SIZE) & PAGE_MASK; + pmd = pmd_offset(pud, addr); + if (pmd_none(*pmd)) + return next; + + pte = pte_offset_kernel(pmd, addr); + if (pte_none(*pte)) + return next; + + *page_size = PAGE_SIZE; + *pp = pte_page(*pte); + } else { + next = pmd_addr_end(addr, end); + + pmd = pmd_offset(pud, addr); + if (pmd_none(*pmd)) + return next; + + *page_size = PMD_SIZE; + *pp = pmd_page(*pmd); + } + + /* + * Removed page structs are filled with 0xFD. + */ + memset((void *)addr, PAGE_INUSE, next - addr); + + page_addr = page_address(*pp); + + /* + * Check the page is filled with 0xFD or not. + * memchr_inv() returns the address. In this case, we cannot + * clear PTE/PUD entry, since the page is used by other. + * So we cannot also free the page. + * + * memchr_inv() returns NULL. In this case, we can clear + * PTE/PUD entry, since the page is not used by other. + * So we can also free the page. + */ + if (memchr_inv(page_addr, PAGE_INUSE, *page_size)) { + *pp = NULL; + return next; + } + + if (!cpu_has_pse) + pte_clear(&init_mm, addr, pte); + else + pmd_clear(pmd); + + return next; +} + +void vmemmap_kfree(struct page *memmap, unsigned long nr_pages) +{ + unsigned long addr = (unsigned long)memmap; + unsigned long end = (unsigned long)(memmap + nr_pages); + unsigned long next; + struct page *page; + int page_size; + + for (; addr < end; addr = next) { + page = NULL; + page_size = 0; + next = find_and_clear_pte_page(addr, end, &page, &page_size); + if (!page) + continue; + + free_pages((unsigned long)page_address(page), + get_order(page_size)); + __flush_tlb_one(addr); + } +} + +void vmemmap_free_bootmem(struct page *memmap, unsigned long nr_pages) +{ + unsigned long addr = (unsigned long)memmap; + unsigned long end = (unsigned long)(memmap + nr_pages); + unsigned long next; + struct page *page; + int page_size; + unsigned long magic; + + for (; addr < end; addr = next) { + page = NULL; + page_size = 0; + next = find_and_clear_pte_page(addr, end, &page, &page_size); + if (!page) + continue; + + magic = (unsigned long) page->lru.next; + if (magic == SECTION_INFO) + put_page_bootmem(page); + flush_tlb_kernel_range(addr, end); + } +} + void register_page_bootmem_memmap(unsigned long section_nr, struct page *start_page, unsigned long size) { diff --git a/include/linux/mm.h b/include/linux/mm.h index 5657670..94d5ccd 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -1642,6 +1642,8 @@ int vmemmap_populate(struct page *start_page, unsigned long pages, int node); void vmemmap_populate_print_last(void); void register_page_bootmem_memmap(unsigned long section_nr, struct page *map, unsigned long size); +void vmemmap_kfree(struct page *memmpa, unsigned long nr_pages); +void vmemmap_free_bootmem(struct page *memmpa, unsigned long nr_pages); enum mf_flags { MF_COUNT_INCREASED = 1 << 0, diff --git a/mm/memory_hotplug.c b/mm/memory_hotplug.c index ccc11b6..7797e91 100644 --- a/mm/memory_hotplug.c +++ b/mm/memory_hotplug.c @@ -301,19 +301,6 @@ static int __meminit __add_section(int nid, struct zone *zone, return register_new_memory(nid, __pfn_to_section(phys_start_pfn)); } -#ifdef CONFIG_SPARSEMEM_VMEMMAP -static int __remove_section(struct zone *zone, struct mem_section *ms) -{ - int ret = -EINVAL; - - if (!valid_section(ms)) - return ret; - - ret = unregister_memory_section(ms); - - return ret; -} -#else static int __remove_section(struct zone *zone, struct mem_section *ms) { unsigned long flags; @@ -330,9 +317,9 @@ static int __remove_section(struct zone *zone, struct mem_section *ms) pgdat_resize_lock(pgdat, &flags); sparse_remove_one_section(zone, ms); pgdat_resize_unlock(pgdat, &flags); - return 0; + + return ret; } -#endif /* * Reasonably generic function for adding memory. It is diff --git a/mm/sparse.c b/mm/sparse.c index fac95f2..c723bc2 100644 --- a/mm/sparse.c +++ b/mm/sparse.c @@ -613,12 +613,13 @@ static inline struct page *kmalloc_section_memmap(unsigned long pnum, int nid, /* This will make the necessary allocations eventually. */ return sparse_mem_map_populate(pnum, nid); } -static void __kfree_section_memmap(struct page *memmap, unsigned long nr_pages) +static void __kfree_section_memmap(struct page *page, unsigned long nr_pages) { - return; /* XXX: Not implemented yet */ + vmemmap_kfree(page, nr_pages); } -static void free_map_bootmem(struct page *page, unsigned long nr_pages) +static void free_map_bootmem(struct page *page) { + vmemmap_free_bootmem(page, PAGES_PER_SECTION); } #else static struct page *__kmalloc_section_memmap(unsigned long nr_pages) @@ -658,10 +659,14 @@ static void __kfree_section_memmap(struct page *memmap, unsigned long nr_pages) get_order(sizeof(struct page) * nr_pages)); } -static void free_map_bootmem(struct page *page, unsigned long nr_pages) +static void free_map_bootmem(struct page *page) { unsigned long maps_section_nr, removing_section_nr, i; unsigned long magic; + unsigned long nr_pages; + + nr_pages = PAGE_ALIGN(PAGES_PER_SECTION * sizeof(struct page)) + >> PAGE_SHIFT; for (i = 0; i < nr_pages; i++, page++) { magic = (unsigned long) page->lru.next; @@ -688,7 +693,6 @@ static void free_map_bootmem(struct page *page, unsigned long nr_pages) static void free_section_usemap(struct page *memmap, unsigned long *usemap) { struct page *usemap_page; - unsigned long nr_pages; if (!usemap) return; @@ -713,10 +717,7 @@ static void free_section_usemap(struct page *memmap, unsigned long *usemap) struct page *memmap_page; memmap_page = virt_to_page(memmap); - nr_pages = PAGE_ALIGN(PAGES_PER_SECTION * sizeof(struct page)) - >> PAGE_SHIFT; - - free_map_bootmem(memmap_page, nr_pages); + free_map_bootmem(memmap_page); } }