From patchwork Fri Jul 17 11:10:06 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Zijlstra X-Patchwork-Id: 11669903 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 6183D1392 for ; Fri, 17 Jul 2020 11:14:42 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 47D7920768 for ; Fri, 17 Jul 2020 11:14:42 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="nyyWe7Al" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726619AbgGQLOl (ORCPT ); Fri, 17 Jul 2020 07:14:41 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42198 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726528AbgGQLOk (ORCPT ); Fri, 17 Jul 2020 07:14:40 -0400 Received: from merlin.infradead.org (merlin.infradead.org [IPv6:2001:8b0:10b:1231::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id DE45DC08C5DC; Fri, 17 Jul 2020 04:14:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=merlin.20170209; h=Content-Type:MIME-Version:References: Subject:Cc:To:From:Date:Message-ID:Sender:Reply-To:Content-Transfer-Encoding: Content-ID:Content-Description:In-Reply-To; bh=FUm1Ir3tUJJdB8qdXiVgRoT6pMrHP/Au+B6bLpG9yBA=; b=nyyWe7AlZtYdYSw49Sonwdxz2r KoQtTILVvXgJriLvasWk1Jh78FRgv6TwvxgW9OfTZnHOfpDdp0XRGhHkROr9M0rS8EE1ZWERqEZCb B4aLODSSNLN9BuP9VeeCsJRd1lCzDfSXB1o9kJN9xYbsbwUPyglN+NaEeeHEdZoR+6xv56usNBrAg E4M1ALbe2VwlUOwGw0TtgdLJDkOPtGYvMUB7f/nq3raSnihsty8lX4VKZGHH7eb7vSNthUAW4d0m3 ClYXcabVNTdqUwUgL5MxtmMU6aZva1lf7kQ0POZDK3bIv1gmVVtN3QMYW68wBEeJlpSJOR+/z2oJT H3PPqaIA==; Received: from j217100.upc-j.chello.nl ([24.132.217.100] helo=noisy.programming.kicks-ass.net) by merlin.infradead.org with esmtpsa (Exim 4.92.3 #3 (Red Hat Linux)) id 1jwOJZ-0001Zl-Hw; Fri, 17 Jul 2020 11:14:05 +0000 Received: from hirez.programming.kicks-ass.net (hirez.programming.kicks-ass.net [192.168.1.225]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by noisy.programming.kicks-ass.net (Postfix) with ESMTPS id 1525F305E21; Fri, 17 Jul 2020 13:14:02 +0200 (CEST) Received: by hirez.programming.kicks-ass.net (Postfix, from userid 0) id A9E9E203D408E; Fri, 17 Jul 2020 13:14:02 +0200 (CEST) Message-ID: <20200717111349.417688532@infradead.org> User-Agent: quilt/0.66 Date: Fri, 17 Jul 2020 13:10:06 +0200 From: Peter Zijlstra To: Will Deacon , "Aneesh Kumar K.V" , Andrew Morton , Nick Piggin , Peter Zijlstra Cc: linux-arch@vger.kernel.org, linux-sh@vger.kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Yoshinori Sato , Rich Felker , "David S. Miller" , Helge Deller , Geert Uytterhoeven , Paul Burton , Tony Luck , Richard Henderson , Nick Hu , Paul Walmsley , John Paul Adrian Glaubitz , Christoph Hellwig Subject: [PATCH v2 01/11] asm-generic/tlb: Fix MMU_GATHER_TABLE_FREE References: <20200717111005.024867618@infradead.org> MIME-Version: 1.0 Sender: linux-sh-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-sh@vger.kernel.org The first MMU_GATHER_TABLE_FREE user showed a logic error in the tlb_needs_table_invalidate() definition. Make sure any TABLE_FREE has it defined. Fixes: 0d6e24d430ef ("asm-generic/tlb: provide MMU_GATHER_TABLE_FREE") Signed-off-by: Peter Zijlstra (Intel) --- include/asm-generic/tlb.h | 35 +++++++++++++++++++---------------- 1 file changed, 19 insertions(+), 16 deletions(-) --- a/include/asm-generic/tlb.h +++ b/include/asm-generic/tlb.h @@ -172,6 +172,18 @@ * various ptep_get_and_clear() functions. */ +#ifndef CONFIG_MMU_GATHER_RCU_TABLE_FREE + +/* + * Only RCU table free can override this; otherwise the TLBI is needed to + * provide existence guarantees for software walkers. + */ +#ifdef tlb_needs_table_invalidate +#error tlb_needs_table_invalidate() requires MMU_GATHER_RCU_TABLE_FREE +#endif + +#endif /* CONFIG_MMU_GATHER_RCU_TABLE_FREE */ + #ifdef CONFIG_MMU_GATHER_TABLE_FREE struct mmu_table_batch { @@ -187,17 +199,6 @@ struct mmu_table_batch { extern void tlb_remove_table(struct mmu_gather *tlb, void *table); -#else /* !CONFIG_MMU_GATHER_HAVE_TABLE_FREE */ - -/* - * Without MMU_GATHER_TABLE_FREE the architecture is assumed to have page based - * page directories and we can use the normal page batching to free them. - */ -#define tlb_remove_table(tlb, page) tlb_remove_page((tlb), (page)) - -#endif /* CONFIG_MMU_GATHER_TABLE_FREE */ - -#ifdef CONFIG_MMU_GATHER_RCU_TABLE_FREE /* * This allows an architecture that does not use the linux page-tables for * hardware to skip the TLBI when freeing page tables. @@ -206,13 +207,15 @@ extern void tlb_remove_table(struct mmu_ #define tlb_needs_table_invalidate() (true) #endif -#else +#else /* !CONFIG_MMU_GATHER_HAVE_TABLE_FREE */ -#ifdef tlb_needs_table_invalidate -#error tlb_needs_table_invalidate() requires MMU_GATHER_RCU_TABLE_FREE -#endif +/* + * Without MMU_GATHER_TABLE_FREE the architecture is assumed to have page based + * page directories and we can use the normal page batching to free them. + */ +#define tlb_remove_table(tlb, page) tlb_remove_page((tlb), (page)) -#endif /* CONFIG_MMU_GATHER_RCU_TABLE_FREE */ +#endif /* CONFIG_MMU_GATHER_TABLE_FREE */ #ifndef CONFIG_MMU_GATHER_NO_GATHER From patchwork Fri Jul 17 11:10:07 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Zijlstra X-Patchwork-Id: 11669919 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 2CA20138C for ; Fri, 17 Jul 2020 11:15:08 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 14767207E8 for ; Fri, 17 Jul 2020 11:15:08 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="GsN9f7zd" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726870AbgGQLPE (ORCPT ); Fri, 17 Jul 2020 07:15:04 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42266 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726256AbgGQLPC (ORCPT ); Fri, 17 Jul 2020 07:15:02 -0400 Received: from casper.infradead.org (casper.infradead.org [IPv6:2001:8b0:10b:1236::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id ECCFFC061755; Fri, 17 Jul 2020 04:15:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Type:MIME-Version:References: Subject:Cc:To:From:Date:Message-ID:Sender:Reply-To:Content-Transfer-Encoding: Content-ID:Content-Description:In-Reply-To; bh=JpsKOsZ9DAnC7hF8A6UR44iGc8j6T+HCUrwgJT3qna0=; b=GsN9f7zd3xGQm9E7Rs/9GO6Onv tQFOciWVNRa0dSkuBi2M3LCrvRWcBpajEzmYqEbvlBaXEJj5TVu8rwQy1Q7CNM7FZR48rVSTeBEGI rxqAK1jJtMJgtXRWRByIMPriLYyhgef4TVlkdIWBk/HEuGJcrF5ZqBi4hDqMeVize23cLJ0dbQ2Rr ip6Sj9RwHTkJWgq4VEYAlzp2yq6rkCH7ht7KXWI/JDmBlDwMD1hfEqR0twYDRHj0D9qMYMWXXgvJy 1co1QpT2PKFq+vwxgBdv2Cf4ZMf4aetp6NLBpyaXqUnapfRc5si+FOdHmr45RHoiibPH/LNSuP/jA R5Ykw5SA==; Received: from j217100.upc-j.chello.nl ([24.132.217.100] helo=noisy.programming.kicks-ass.net) by casper.infradead.org with esmtpsa (Exim 4.92.3 #3 (Red Hat Linux)) id 1jwOJa-00051V-96; Fri, 17 Jul 2020 11:14:07 +0000 Received: from hirez.programming.kicks-ass.net (hirez.programming.kicks-ass.net [192.168.1.225]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by noisy.programming.kicks-ass.net (Postfix) with ESMTPS id 0D7ED300446; Fri, 17 Jul 2020 13:14:02 +0200 (CEST) Received: by hirez.programming.kicks-ass.net (Postfix, from userid 0) id AC1F7203D4090; Fri, 17 Jul 2020 13:14:02 +0200 (CEST) Message-ID: <20200717111349.475459591@infradead.org> User-Agent: quilt/0.66 Date: Fri, 17 Jul 2020 13:10:07 +0200 From: Peter Zijlstra To: Will Deacon , "Aneesh Kumar K.V" , Andrew Morton , Nick Piggin , Peter Zijlstra Cc: linux-arch@vger.kernel.org, linux-sh@vger.kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Yoshinori Sato , Rich Felker , "David S. Miller" , Helge Deller , Geert Uytterhoeven , Paul Burton , Tony Luck , Richard Henderson , Nick Hu , Paul Walmsley , John Paul Adrian Glaubitz , Christoph Hellwig , Geert Uytterhoeven Subject: [PATCH v2 02/11] sh/tlb: Fix PGTABLE_LEVELS > 2 References: <20200717111005.024867618@infradead.org> MIME-Version: 1.0 Sender: linux-sh-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-sh@vger.kernel.org Geert reported that his SH7722-based Migo-R board failed to boot after commit: c5b27a889da9 ("sh/tlb: Convert SH to generic mmu_gather") That commit fell victim to copying the wrong pattern -- __pmd_free_tlb() used to be implemented with pmd_free(). Fixes: c5b27a889da9 ("sh/tlb: Convert SH to generic mmu_gather") Reported-by: Geert Uytterhoeven Signed-off-by: Peter Zijlstra (Intel) Reviewed-by: Geert Uytterhoeven Tested-by: Geert Uytterhoeven --- a/arch/sh/include/asm/pgalloc.h +++ b/arch/sh/include/asm/pgalloc.h @@ -12,6 +12,7 @@ extern void pgd_free(struct mm_struct *m extern void pud_populate(struct mm_struct *mm, pud_t *pudp, pmd_t *pmd); extern pmd_t *pmd_alloc_one(struct mm_struct *mm, unsigned long address); extern void pmd_free(struct mm_struct *mm, pmd_t *pmd); +#define __pmd_free_tlb(tlb, pmdp, addr) pmd_free((tlb)->mm, (pmdp)) #endif static inline void pmd_populate_kernel(struct mm_struct *mm, pmd_t *pmd, @@ -33,13 +34,4 @@ do { \ tlb_remove_page((tlb), (pte)); \ } while (0) -#if CONFIG_PGTABLE_LEVELS > 2 -#define __pmd_free_tlb(tlb, pmdp, addr) \ -do { \ - struct page *page = virt_to_page(pmdp); \ - pgtable_pmd_page_dtor(page); \ - tlb_remove_page((tlb), page); \ -} while (0); -#endif - #endif /* __ASM_SH_PGALLOC_H */ From patchwork Fri Jul 17 11:10:08 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Zijlstra X-Patchwork-Id: 11669905 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 9E9A614E3 for ; Fri, 17 Jul 2020 11:14:42 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 8665520768 for ; Fri, 17 Jul 2020 11:14:42 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="mW9BkUrd" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726634AbgGQLOl (ORCPT ); Fri, 17 Jul 2020 07:14:41 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42196 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726525AbgGQLOk (ORCPT ); Fri, 17 Jul 2020 07:14:40 -0400 Received: from merlin.infradead.org (merlin.infradead.org [IPv6:2001:8b0:10b:1231::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D871BC08C5DB; Fri, 17 Jul 2020 04:14:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=merlin.20170209; h=Content-Type:MIME-Version:References: Subject:Cc:To:From:Date:Message-ID:Sender:Reply-To:Content-Transfer-Encoding: Content-ID:Content-Description:In-Reply-To; bh=LJ3sw53oU99bJUNlz+ZPKQESK/v2SDXFGx5kri1RxHA=; b=mW9BkUrdORbTIfPggDbuXHG04P vCB4+S08tm1Ocf2AwK2fgRL5Z3xoCetQtur/ZtwHqsPjZyWzsEWXg/Ip+JJ01RKTNXQtvbRs+p4wQ Rtlh3rxfbsLXEdDC4EA+JGYP4+cth5V1pYwo/CUI+uhfMF/FCH+FPxsGHa0Qxv98nUlMB2O5LfeGn 76uku3heKukpjwJsmr+QWhayyffHIapPntJuifnOtS8eftI1cLoiD8fwy79B12vBBVdWVK6nwNu9V c3dXF9QSaUXQEzzdrBQNbf7xX4kUygNN/Os21nFZoo+1Bzj9/2pKz1V+KcRM2zJ+FzRpvfRyHdXc7 T+DnKayA==; Received: from j217100.upc-j.chello.nl ([24.132.217.100] helo=noisy.programming.kicks-ass.net) by merlin.infradead.org with esmtpsa (Exim 4.92.3 #3 (Red Hat Linux)) id 1jwOJZ-0001Zm-Jw; Fri, 17 Jul 2020 11:14:05 +0000 Received: from hirez.programming.kicks-ass.net (hirez.programming.kicks-ass.net [192.168.1.225]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by noisy.programming.kicks-ass.net (Postfix) with ESMTPS id 144D5305DD1; Fri, 17 Jul 2020 13:14:02 +0200 (CEST) Received: by hirez.programming.kicks-ass.net (Postfix, from userid 0) id AF019203D4091; Fri, 17 Jul 2020 13:14:02 +0200 (CEST) Message-ID: <20200717111349.533637732@infradead.org> User-Agent: quilt/0.66 Date: Fri, 17 Jul 2020 13:10:08 +0200 From: Peter Zijlstra To: Will Deacon , "Aneesh Kumar K.V" , Andrew Morton , Nick Piggin , Peter Zijlstra Cc: linux-arch@vger.kernel.org, linux-sh@vger.kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Yoshinori Sato , Rich Felker , "David S. Miller" , Helge Deller , Geert Uytterhoeven , Paul Burton , Tony Luck , Richard Henderson , Nick Hu , Paul Walmsley , John Paul Adrian Glaubitz , Christoph Hellwig Subject: [PATCH v2 03/11] sh/tlb: Fix __pmd_free_tlb() References: <20200717111005.024867618@infradead.org> MIME-Version: 1.0 Sender: linux-sh-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-sh@vger.kernel.org SH violates the freeing order for it's PMD page directories. It's __pmd_free_tlb() does not ensure there is a TLB invalidation between itself and the eventualy freeing of the page. Further complicating the situation is that SH uses non page based allocation for it's PMDs. Use the shiny new HAVE_TABLE_FREE option to enable a custom page table freeer. (SuperH uses IPI based TLB invalidation and therefore doesn't need HAVE_RCU_TABLE_FREE for its HAVE_FAST_GUP). Signed-off-by: Peter Zijlstra (Intel) --- arch/sh/Kconfig | 1 + arch/sh/include/asm/pgalloc.h | 3 ++- arch/sh/mm/pgtable.c | 6 ++++++ 3 files changed, 9 insertions(+), 1 deletion(-) --- a/arch/sh/Kconfig +++ b/arch/sh/Kconfig @@ -14,6 +14,7 @@ config SUPERH select HAVE_PERF_EVENTS select HAVE_DEBUG_BUGVERBOSE select HAVE_FAST_GUP if MMU + select MMU_GATHER_TABLE_FREE if X2TLB select ARCH_HAVE_CUSTOM_GPIO_H select ARCH_HAVE_NMI_SAFE_CMPXCHG if (GUSA_RB || CPU_SH4A) select ARCH_HAS_GCOV_PROFILE_ALL --- a/arch/sh/include/asm/pgalloc.h +++ b/arch/sh/include/asm/pgalloc.h @@ -12,7 +12,8 @@ extern void pgd_free(struct mm_struct *m extern void pud_populate(struct mm_struct *mm, pud_t *pudp, pmd_t *pmd); extern pmd_t *pmd_alloc_one(struct mm_struct *mm, unsigned long address); extern void pmd_free(struct mm_struct *mm, pmd_t *pmd); -#define __pmd_free_tlb(tlb, pmdp, addr) pmd_free((tlb)->mm, (pmdp)) +extern void __tlb_remove_table(void *table); +#define __pmd_free_tlb(tlb, pmdp, addr) tlb_remove_table((tlb), (pmdp)) #endif static inline void pmd_populate_kernel(struct mm_struct *mm, pmd_t *pmd, --- a/arch/sh/mm/pgtable.c +++ b/arch/sh/mm/pgtable.c @@ -1,6 +1,7 @@ // SPDX-License-Identifier: GPL-2.0 #include #include +#include #define PGALLOC_GFP GFP_KERNEL | __GFP_ZERO @@ -55,4 +56,9 @@ void pmd_free(struct mm_struct *mm, pmd_ { kmem_cache_free(pmd_cachep, pmd); } + +void __tlb_remove_table(void *table) +{ + pmd_free(NULL, table); +} #endif /* PAGETABLE_LEVELS > 2 */ From patchwork Fri Jul 17 11:10:09 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Zijlstra X-Patchwork-Id: 11669899 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id EA5D414E3 for ; Fri, 17 Jul 2020 11:14:39 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id D12A220768 for ; Fri, 17 Jul 2020 11:14:39 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="i1mn0vBQ" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726418AbgGQLOj (ORCPT ); Fri, 17 Jul 2020 07:14:39 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42186 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725912AbgGQLOi (ORCPT ); Fri, 17 Jul 2020 07:14:38 -0400 Received: from merlin.infradead.org (merlin.infradead.org [IPv6:2001:8b0:10b:1231::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4DF6EC061755; Fri, 17 Jul 2020 04:14:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=merlin.20170209; h=Content-Type:MIME-Version:References: Subject:Cc:To:From:Date:Message-ID:Sender:Reply-To:Content-Transfer-Encoding: Content-ID:Content-Description:In-Reply-To; bh=fCoot8j0cWA9ciC5yvURdWvuq4otc5g+z9FwRrkMqPI=; b=i1mn0vBQKBaNacLXDfsoK+7QG6 g0aa0ADr+4bhdnmt9IAIw4EPzJLymoj2ob/Pya5sBqVBjtTrLs5vxWkeWzIq7rGnfwzBQO0abZ1Uj BuR8IUQ0Ml0ePeFrjIt+l0gsm+yIU4UoeY5S/I68dX4EeXb+RtLM/VFN4c/WB11lJSyBJ/1GN29F3 yGhK/yyH5IZPFpfHrT6WRUl9WY0rbD0MfUTCczycRrvJeXpzl0OT0EXN7kuPWC6KR61BCLR5PDDTJ 8OmSei+N/7sTQS1KghadHxTFf1CdjVubg7GAgPWtYzeluEq1AZ7yh7NnFu/4QSNEOEyQqJLXEsnwH dBXDZN6w==; Received: from j217100.upc-j.chello.nl ([24.132.217.100] helo=noisy.programming.kicks-ass.net) by merlin.infradead.org with esmtpsa (Exim 4.92.3 #3 (Red Hat Linux)) id 1jwOJb-0001eW-7I; Fri, 17 Jul 2020 11:14:07 +0000 Received: from hirez.programming.kicks-ass.net (hirez.programming.kicks-ass.net [192.168.1.225]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by noisy.programming.kicks-ass.net (Postfix) with ESMTPS id 0E5B3304B90; Fri, 17 Jul 2020 13:14:02 +0200 (CEST) Received: by hirez.programming.kicks-ass.net (Postfix, from userid 0) id B515C203D4093; Fri, 17 Jul 2020 13:14:02 +0200 (CEST) Message-ID: <20200717111349.591934380@infradead.org> User-Agent: quilt/0.66 Date: Fri, 17 Jul 2020 13:10:09 +0200 From: Peter Zijlstra To: Will Deacon , "Aneesh Kumar K.V" , Andrew Morton , Nick Piggin , Peter Zijlstra Cc: linux-arch@vger.kernel.org, linux-sh@vger.kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Yoshinori Sato , Rich Felker , "David S. Miller" , Helge Deller , Geert Uytterhoeven , Paul Burton , Tony Luck , Richard Henderson , Nick Hu , Paul Walmsley , John Paul Adrian Glaubitz , Christoph Hellwig Subject: [PATCH v2 04/11] sparc32/tlb: Fix __p*_free_tlb() References: <20200717111005.024867618@infradead.org> MIME-Version: 1.0 Sender: linux-sh-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-sh@vger.kernel.org Just like regular pages, page directories need to observe the following order: 1) unhook 2) TLB invalidate 3) free to ensure it is safe against concurrent accesses. Because Sparc32 has non-page based page directories, use a custom table freeer. Signed-off-by: Peter Zijlstra (Intel) --- arch/sparc/Kconfig | 1 + arch/sparc/include/asm/pgalloc_32.h | 7 +++++-- arch/sparc/mm/srmmu.c | 28 ++++++++++++++++++++++++++++ 3 files changed, 34 insertions(+), 2 deletions(-) --- a/arch/sparc/Kconfig +++ b/arch/sparc/Kconfig @@ -57,6 +57,7 @@ config SPARC32 select CLZ_TAB select HAVE_UID16 select OLD_SIGACTION + select MMU_GATHER_TABLE_FREE config SPARC64 def_bool 64BIT --- a/arch/sparc/include/asm/pgalloc_32.h +++ b/arch/sparc/include/asm/pgalloc_32.h @@ -12,6 +12,9 @@ struct page; +extern void pgtable_free_tlb(struct mmu_gather *tlb, void *table, int idx); +extern void __tlb_remove_table(void *table); + void *srmmu_get_nocache(int size, int align); void srmmu_free_nocache(void *addr, int size); @@ -48,7 +51,7 @@ static inline void free_pmd_fast(pmd_t * } #define pmd_free(mm, pmd) free_pmd_fast(pmd) -#define __pmd_free_tlb(tlb, pmd, addr) pmd_free((tlb)->mm, pmd) +#define __pmd_free_tlb(tlb, pmd, addr) pgtable_free_tlb((tlb), (pmd), 1) #define pmd_populate(mm, pmd, pte) pmd_set(pmd, pte) #define pmd_pgtable(pmd) (pgtable_t)__pmd_page(pmd) @@ -73,6 +76,6 @@ static inline void free_pte_fast(pte_t * #define pte_free_kernel(mm, pte) free_pte_fast(pte) void pte_free(struct mm_struct * mm, pgtable_t pte); -#define __pte_free_tlb(tlb, pte, addr) pte_free((tlb)->mm, pte) +#define __pte_free_tlb(tlb, pte, addr) pgtable_free_tlb((tlb), (pte), 0) #endif /* _SPARC_PGALLOC_H */ --- a/arch/sparc/mm/srmmu.c +++ b/arch/sparc/mm/srmmu.c @@ -38,6 +38,7 @@ #include #include #include +#include #include /* Now the cpu specific definitions. */ @@ -1831,3 +1832,30 @@ void __init load_mmu(void) sun4m_init_smp(); #endif } + +#define TLB_IDX_MASK 1UL + +void __tlb_remove_table(void *table) +{ + void *dir = (void *)((unsigned long)table & ~TLB_IDX_MASK); + int idx = (unsigned long)table & TLB_IDX_MASK; + + switch (idx) { + case 1: /* PMD */ + pmd_free(NULL, dir); + break; + case 0: /* PTE */ + pte_free(NULL, dir); + break; + } +} + +void pgtable_free_tlb(struct mmu_gather *tlb, void *table, int idx) +{ + unsigned long pgf = (unsigned long)table; + BUG_ON(idx > TLB_IDX_MASK); + BUG_ON(pgf & TLB_IDX_MASK); + pgf |= idx; + tlb_remove_table(tlb, (void *)pgf); +} + From patchwork Fri Jul 17 11:10:10 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Zijlstra X-Patchwork-Id: 11669915 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 27E251392 for ; Fri, 17 Jul 2020 11:15:04 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 08F7821741 for ; Fri, 17 Jul 2020 11:15:03 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="a6jjRyXa" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726848AbgGQLO4 (ORCPT ); Fri, 17 Jul 2020 07:14:56 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42212 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726673AbgGQLOn (ORCPT ); Fri, 17 Jul 2020 07:14:43 -0400 Received: from casper.infradead.org (casper.infradead.org [IPv6:2001:8b0:10b:1236::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id DAFAFC08C5C0; Fri, 17 Jul 2020 04:14:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Type:MIME-Version:References: Subject:Cc:To:From:Date:Message-ID:Sender:Reply-To:Content-Transfer-Encoding: Content-ID:Content-Description:In-Reply-To; bh=S8UknOxkHjh+fFZ6g8rERXF5PzCEDpVIPOf+v6XVy+E=; b=a6jjRyXa8+VsSAkTLZRRukYqZ6 sF/rHVHvFgmcHZ2kQMPhnZVqjNxs1t3wZduvAlgEeZGgwwjdyIUHL1XjV8vRf25apeRy8T0u0LBDL WBzFUF7pYu0sRdQU48yeiTjaXuGdmTVZ+CPQW/fszXLSP1O1C4kwFvK6FW6X+83ZOVVAumaU/qsE2 Z6fLJzH0bMeObAvJKwRGuKb1rn4oZeuKUjav1U2XCxVVdcYjh4wSShDvK7dQlYkNx2IOcY+q30M+P lir3rDJY5+14N+MhQ4r2ZdXZ8OdOFXwxGx/7flyn0th+WbXNKxKO7MjGmpiVAHad8XlBsgfFatTfB PUxtEjHg==; Received: from j217100.upc-j.chello.nl ([24.132.217.100] helo=noisy.programming.kicks-ass.net) by casper.infradead.org with esmtpsa (Exim 4.92.3 #3 (Red Hat Linux)) id 1jwOJZ-00051I-1P; Fri, 17 Jul 2020 11:14:05 +0000 Received: from hirez.programming.kicks-ass.net (hirez.programming.kicks-ass.net [192.168.1.225]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by noisy.programming.kicks-ass.net (Postfix) with ESMTPS id 6BB553060C5; Fri, 17 Jul 2020 13:14:04 +0200 (CEST) Received: by hirez.programming.kicks-ass.net (Postfix, from userid 0) id BC664203D4094; Fri, 17 Jul 2020 13:14:02 +0200 (CEST) Message-ID: <20200717111349.650029395@infradead.org> User-Agent: quilt/0.66 Date: Fri, 17 Jul 2020 13:10:10 +0200 From: Peter Zijlstra To: Will Deacon , "Aneesh Kumar K.V" , Andrew Morton , Nick Piggin , Peter Zijlstra Cc: linux-arch@vger.kernel.org, linux-sh@vger.kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Yoshinori Sato , Rich Felker , "David S. Miller" , Helge Deller , Geert Uytterhoeven , Paul Burton , Tony Luck , Richard Henderson , Nick Hu , Paul Walmsley , John Paul Adrian Glaubitz , Christoph Hellwig Subject: [PATCH v2 05/11] parisc/tlb: Fix __p*_free_tlb() References: <20200717111005.024867618@infradead.org> MIME-Version: 1.0 Sender: linux-sh-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-sh@vger.kernel.org Just like regular pages, page directories need to observe the following order: 1) unhook 2) TLB invalidate 3) free to ensure it is safe against concurrent accesses. Because PARISC has non-page-size PMDs, use a custom table freeer. Signed-off-by: Peter Zijlstra (Intel) --- arch/parisc/Kconfig | 1 + arch/parisc/include/asm/pgalloc.h | 23 ++++++++++++++++++++--- arch/parisc/include/asm/tlb.h | 9 +++++---- 3 files changed, 26 insertions(+), 7 deletions(-) --- a/arch/parisc/Kconfig +++ b/arch/parisc/Kconfig @@ -63,6 +63,7 @@ config PARISC select HAVE_KPROBES_ON_FTRACE select HAVE_DYNAMIC_FTRACE_WITH_REGS select HAVE_COPY_THREAD_TLS + select MMU_GATHER_TABLE_FREE if PGTABLE_LEVELS >= 3 help The PA-RISC microprocessor is designed by Hewlett-Packard and used --- a/arch/parisc/include/asm/pgalloc.h +++ b/arch/parisc/include/asm/pgalloc.h @@ -73,7 +73,7 @@ static inline pmd_t *pmd_alloc_one(struc return pmd; } -static inline void pmd_free(struct mm_struct *mm, pmd_t *pmd) +static inline bool __pmd_free(struct mm_struct *mm, pmd_t *pmd) { if (pmd_flag(*pmd) & PxD_FLAG_ATTACHED) { /* @@ -83,11 +83,28 @@ static inline void pmd_free(struct mm_st * done by generic mm code. */ mm_inc_nr_pmds(mm); - return; + return false; } - free_pages((unsigned long)pmd, PMD_ORDER); + return true; +} + +static inline void pmd_free(struct mm_struct *mm, pmd_t *pmd) +{ + if (__pmd_free(mm, pmd)) + free_pages((unsigned long)pmd, PMD_ORDER); } +static inline void __tlb_remove_table(void *table) +{ + free_pages((unsigned long)table, PMD_ORDER); +} + +#define __pmd_free_tlb(tlb, pmd, addr) \ +do { \ + if (__pmd_free((tlb)->mm, (pmd))) \ + tlb_remove_table((tlb), (pmd)); \ +} while (0) + #endif static inline void --- a/arch/parisc/include/asm/tlb.h +++ b/arch/parisc/include/asm/tlb.h @@ -4,9 +4,10 @@ #include -#if CONFIG_PGTABLE_LEVELS == 3 -#define __pmd_free_tlb(tlb, pmd, addr) pmd_free((tlb)->mm, pmd) -#endif -#define __pte_free_tlb(tlb, pte, addr) pte_free((tlb)->mm, pte) +#define __pte_free_tlb(tlb,pte,addr) \ +do { \ + pgtable_pte_page_dtor(pte); \ + tlb_remove_page((tlb), (pte)); \ +} while (0) #endif From patchwork Fri Jul 17 11:10:11 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Zijlstra X-Patchwork-Id: 11669925 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 162641392 for ; Fri, 17 Jul 2020 11:15:15 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id F037D2070E for ; Fri, 17 Jul 2020 11:15:14 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="Y/MXDI0t" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726256AbgGQLPL (ORCPT ); Fri, 17 Jul 2020 07:15:11 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42200 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726550AbgGQLOk (ORCPT ); Fri, 17 Jul 2020 07:14:40 -0400 Received: from merlin.infradead.org (merlin.infradead.org [IPv6:2001:8b0:10b:1231::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E7945C08C5DD; Fri, 17 Jul 2020 04:14:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=merlin.20170209; h=Content-Type:MIME-Version:References: Subject:Cc:To:From:Date:Message-ID:Sender:Reply-To:Content-Transfer-Encoding: Content-ID:Content-Description:In-Reply-To; bh=gOQYLPqlkXaE5f1BcTQenQfqzoyHPAVuIDnigWhSoRE=; b=Y/MXDI0tbjGV1H6k+/3UUA8Lu5 zcG4hZO3jyz9bLcRi2dIJysN79dO3dnN/J6fbvIK4gxxQiyBlOKQuuCn0E4ddYCbGpPOTo3VVtlki b8indkgQ/FVRbEsFBMBPCZcy3KunV3UanWjKfz3yzA2GvqjF7q6418pyW6NM0aW8XTHtuYN9wNG74 sxLzg+HiCTzpnr/zc2fq3XNvfm31YDJdhEXKopYgPN6tBPKbu/nVWCyNI6khi3A9VUpNMTP2aF7eI 3elFTvXbPIuOkzZ1J7mS4tlF7nVaUtIA8IAJ8rLPZMHlzNolSp/mIGsc5xtO6SOhkwzpM6jWmaDOb UHPdiS5w==; Received: from j217100.upc-j.chello.nl ([24.132.217.100] helo=noisy.programming.kicks-ass.net) by merlin.infradead.org with esmtpsa (Exim 4.92.3 #3 (Red Hat Linux)) id 1jwOJZ-0001Zn-LT; Fri, 17 Jul 2020 11:14:05 +0000 Received: from hirez.programming.kicks-ass.net (hirez.programming.kicks-ass.net [192.168.1.225]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by noisy.programming.kicks-ass.net (Postfix) with ESMTPS id 6F914306CCF; Fri, 17 Jul 2020 13:14:04 +0200 (CEST) Received: by hirez.programming.kicks-ass.net (Postfix, from userid 0) id C0DDF203D4095; Fri, 17 Jul 2020 13:14:02 +0200 (CEST) Message-ID: <20200717111349.708289813@infradead.org> User-Agent: quilt/0.66 Date: Fri, 17 Jul 2020 13:10:11 +0200 From: Peter Zijlstra To: Will Deacon , "Aneesh Kumar K.V" , Andrew Morton , Nick Piggin , Peter Zijlstra Cc: linux-arch@vger.kernel.org, linux-sh@vger.kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Yoshinori Sato , Rich Felker , "David S. Miller" , Helge Deller , Geert Uytterhoeven , Paul Burton , Tony Luck , Richard Henderson , Nick Hu , Paul Walmsley , John Paul Adrian Glaubitz , Christoph Hellwig Subject: [PATCH v2 06/11] mips/tlb: Fix __p*_free_tlb() References: <20200717111005.024867618@infradead.org> MIME-Version: 1.0 Sender: linux-sh-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-sh@vger.kernel.org Just like regular pages, page directories need to observe the following order: 1) unhook 2) TLB invalidate 3) free to ensure it is safe against concurrent accesses. MIPS has (thanks to Paul Burton for setting me straight): - HAVE_FAST_GUP, which means we need to consider software walkers (many MIPS also have software TLB-fill, which is a similar software walker). - non-page, page directories, which requires a custom table free. - Mostly IPI-based TLB invalidate, but it wouldn't be MIPS if it didn't also have broadcast TLB invalidate (I6500, GINVT). This means that in generic MIPS needs HAVE_RCU_TABLE_FREE. Signed-off-by: Peter Zijlstra (Intel) --- arch/mips/Kconfig | 1 + arch/mips/include/asm/pgalloc.h | 13 ++++++------- arch/mips/mm/pgtable.c | 34 ++++++++++++++++++++++++++++++++++ 3 files changed, 41 insertions(+), 7 deletions(-) --- a/arch/mips/Kconfig +++ b/arch/mips/Kconfig @@ -84,6 +84,7 @@ config MIPS select HAVE_VIRT_CPU_ACCOUNTING_GEN if 64BIT || !SMP select IRQ_FORCED_THREADING select ISA if EISA + select MMU_GATHER_RCU_TABLE_FREE select MODULES_USE_ELF_REL if MODULES select MODULES_USE_ELF_RELA if MODULES && 64BIT select PERF_USE_VMALLOC --- a/arch/mips/include/asm/pgalloc.h +++ b/arch/mips/include/asm/pgalloc.h @@ -41,6 +41,9 @@ static inline void pud_populate(struct m } #endif +extern void __tlb_remove_table(void *table); +extern void pgtable_free_tlb(struct mmu_gather *tlb, void *table, int idx); + /* * Initialize a new pgd / pmd table with invalid pointers. */ @@ -52,11 +55,7 @@ static inline void pgd_free(struct mm_st free_pages((unsigned long)pgd, PGD_ORDER); } -#define __pte_free_tlb(tlb,pte,address) \ -do { \ - pgtable_pte_page_dtor(pte); \ - tlb_remove_page((tlb), pte); \ -} while (0) +#define __pte_free_tlb(tlb,pte,address) pgtable_free_tlb((tlb), (pte), 0) #ifndef __PAGETABLE_PMD_FOLDED @@ -75,7 +74,7 @@ static inline void pmd_free(struct mm_st free_pages((unsigned long)pmd, PMD_ORDER); } -#define __pmd_free_tlb(tlb, x, addr) pmd_free((tlb)->mm, x) +#define __pmd_free_tlb(tlb, x, addr) pgtable_free_tlb((tlb), (x), 1) #endif @@ -101,7 +100,7 @@ static inline void p4d_populate(struct m set_p4d(p4d, __p4d((unsigned long)pud)); } -#define __pud_free_tlb(tlb, x, addr) pud_free((tlb)->mm, x) +#define __pud_free_tlb(tlb, x, addr) pgtable_free_tlb((tlb), (x), 2) #endif /* __PAGETABLE_PUD_FOLDED */ --- a/arch/mips/mm/pgtable.c +++ b/arch/mips/mm/pgtable.c @@ -7,6 +7,7 @@ #include #include #include +#include pgd_t *pgd_alloc(struct mm_struct *mm) { @@ -23,3 +24,36 @@ pgd_t *pgd_alloc(struct mm_struct *mm) return ret; } EXPORT_SYMBOL_GPL(pgd_alloc); + +#define TLB_IDX_MASK 0x3 + +void pgtable_free_tlb(struct mmu_gather *tlb, void *table, int idx) +{ + unsigned long pgf = (unsigned long)table; + BUG_ON(idx > TLB_IDX_MASK); + BUG_ON(pgf & TLB_IDX_MASK); + pgf |= idx; + tlb_remove_table(tlb, (void *)pgf); +} + +void __tlb_remove_table(void *table) +{ + void *dir = (void *)((unsigned long)table & ~TLB_IDX_MASK); + int idx = (unsigned long)table & TLB_IDX_MASK; + + switch (idx) { +#ifndef __PAGETABLE_PUD_FOLDED + case 2: /* PUD */ + pud_free(NULL, dir); + break; +#endif +#ifndef __PAGETABLE_PMD_FOLDED + case 1: /* PMD */ + pmd_free(NULL, dir); + break; +#endif + case 0: /* PTE */ + pte_free(NULL, dir); + break; + } +} From patchwork Fri Jul 17 11:10:12 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Zijlstra X-Patchwork-Id: 11669921 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id D8F63138C for ; Fri, 17 Jul 2020 11:15:10 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id BF4C1207E8 for ; Fri, 17 Jul 2020 11:15:10 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="Ka83rRYo" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726593AbgGQLOl (ORCPT ); Fri, 17 Jul 2020 07:14:41 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42192 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726491AbgGQLOk (ORCPT ); Fri, 17 Jul 2020 07:14:40 -0400 Received: from merlin.infradead.org (merlin.infradead.org [IPv6:2001:8b0:10b:1231::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A1796C08C5C0; Fri, 17 Jul 2020 04:14:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=merlin.20170209; h=Content-Type:MIME-Version:References: Subject:Cc:To:From:Date:Message-ID:Sender:Reply-To:Content-Transfer-Encoding: Content-ID:Content-Description:In-Reply-To; bh=G2PV3JD1hofFY1qgnR3BY+EQPfEM9HsXume4Dr7DRnk=; b=Ka83rRYoEtlhZlelqXZ2reoVNG c3psqE8LlKgY5/dxe8pHVAsn5xsC/K10d0dms9/MK3//sK20Xjwt/qTRwMPKptZyaoM81YRhrOOyP aPK2g/aIBYbfai8KbUpc7keOXMTTHznTBmSyA3RzslV20ge7d4zbgDncUMgNPU1/0KeXMsTiUzwFP GM4AAkK6f4JzqywkFCYyQ3YtScmPM34DM2DMpmTJPuElir8eUzZt6X5kKc2kReIUxv5+Rx+nDvvuU DhZTZbgf87SagTOmVsHLiZ5AGaCSE9R5k6ITPOprV8aaSl4WiMdbTHimgTsh/lAoEISmsPFyKW9Cz 4ck418bQ==; Received: from j217100.upc-j.chello.nl ([24.132.217.100] helo=noisy.programming.kicks-ass.net) by merlin.infradead.org with esmtpsa (Exim 4.92.3 #3 (Red Hat Linux)) id 1jwOJb-0001da-1P; Fri, 17 Jul 2020 11:14:07 +0000 Received: from hirez.programming.kicks-ass.net (hirez.programming.kicks-ass.net [192.168.1.225]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by noisy.programming.kicks-ass.net (Postfix) with ESMTPS id 7A3E2306E07; Fri, 17 Jul 2020 13:14:04 +0200 (CEST) Received: by hirez.programming.kicks-ass.net (Postfix, from userid 0) id C4D5B203D4096; Fri, 17 Jul 2020 13:14:02 +0200 (CEST) Message-ID: <20200717111349.766472738@infradead.org> User-Agent: quilt/0.66 Date: Fri, 17 Jul 2020 13:10:12 +0200 From: Peter Zijlstra To: Will Deacon , "Aneesh Kumar K.V" , Andrew Morton , Nick Piggin , Peter Zijlstra Cc: linux-arch@vger.kernel.org, linux-sh@vger.kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Yoshinori Sato , Rich Felker , "David S. Miller" , Helge Deller , Geert Uytterhoeven , Paul Burton , Tony Luck , Richard Henderson , Nick Hu , Paul Walmsley , John Paul Adrian Glaubitz , Christoph Hellwig Subject: [PATCH v2 07/11] ia64/tlb: Fix __p*_free_tlb() References: <20200717111005.024867618@infradead.org> MIME-Version: 1.0 Sender: linux-sh-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-sh@vger.kernel.org Just like regular pages, page directories need to observe the following order: 1) unhook 2) TLB invalidate 3) free to ensure it is safe against concurrent accesses. Since IA64 has page based P[UM]Ds, no software walkers and IPI based TLB invalidation, it can use the simple tlb_remove_page() based freeer. Signed-off-by: Peter Zijlstra (Intel) --- arch/ia64/include/asm/pgalloc.h | 13 ++++++++++--- 1 file changed, 10 insertions(+), 3 deletions(-) --- a/arch/ia64/include/asm/pgalloc.h +++ b/arch/ia64/include/asm/pgalloc.h @@ -50,7 +50,9 @@ static inline void pud_free(struct mm_st { free_page((unsigned long)pud); } -#define __pud_free_tlb(tlb, pud, address) pud_free((tlb)->mm, pud) + +#define __pud_free_tlb(tlb, pud, address) \ + tlb_remove_table((tlb), virt_to_page(pud)) #endif /* CONFIG_PGTABLE_LEVELS == 4 */ static inline void @@ -69,7 +71,8 @@ static inline void pmd_free(struct mm_st free_page((unsigned long)pmd); } -#define __pmd_free_tlb(tlb, pmd, address) pmd_free((tlb)->mm, pmd) +#define __pmd_free_tlb(tlb, pmd, address) \ + tlb_remove_table((tlb), virt_to_page(pmd) static inline void pmd_populate(struct mm_struct *mm, pmd_t * pmd_entry, pgtable_t pte) @@ -84,6 +87,10 @@ pmd_populate_kernel(struct mm_struct *mm pmd_val(*pmd_entry) = __pa(pte); } -#define __pte_free_tlb(tlb, pte, address) pte_free((tlb)->mm, pte) +#define __pte_free_tlb(tlb, pte, address) \ +do { \ + pgtable_pte_page_dtor(pte); \ + tlb_remove_table((tlb), (pte)); \ +} while (0) #endif /* _ASM_IA64_PGALLOC_H */ From patchwork Fri Jul 17 11:10:13 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Zijlstra X-Patchwork-Id: 11669927 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 603E6138C for ; Fri, 17 Jul 2020 11:15:16 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 468BA2070E for ; Fri, 17 Jul 2020 11:15:16 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="DJYHEvab" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726550AbgGQLPP (ORCPT ); Fri, 17 Jul 2020 07:15:15 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42194 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726512AbgGQLOk (ORCPT ); Fri, 17 Jul 2020 07:14:40 -0400 Received: from merlin.infradead.org (merlin.infradead.org [IPv6:2001:8b0:10b:1231::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id AA4A1C08C5CE; Fri, 17 Jul 2020 04:14:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=merlin.20170209; h=Content-Type:MIME-Version:References: Subject:Cc:To:From:Date:Message-ID:Sender:Reply-To:Content-Transfer-Encoding: Content-ID:Content-Description:In-Reply-To; bh=lqUXsp1YoP1pMIfyAAamvA0e1hiYKdF5cDUHBRasd7E=; b=DJYHEvab2ej+if7sHNrHhPkAHQ dtX/b3zsSgVYJU7V5Dn3E9vuGr7oZGGGUJyrkLESLeIdfken09z/+Xmeoodj8j7J9Uk2bFnWw7QqD ED2Gj3xnfSJLH0AuDO2Dwk+yAsN1WzjUR+wGs7/dI39ZwLfCcluJFWHLyvtNIu04nk9BpMHOUh4qn Jmhq0K3zEMjJPwDa8SJxeMCF2hez8A5+YgdfCwXMhG+Dzv3FYYDbde19J4LCbtVi2XDdYJJi5rnr0 0QwwAPp770wRZZV9tISe/AAnO/4Sa2F6Q7Lm1yCiPKJzjQAPXpuMXMa+hL4mcj2yPlg+0sc+kKuKj 88FCPyug==; Received: from j217100.upc-j.chello.nl ([24.132.217.100] helo=noisy.programming.kicks-ass.net) by merlin.infradead.org with esmtpsa (Exim 4.92.3 #3 (Red Hat Linux)) id 1jwOJa-0001b0-H7; Fri, 17 Jul 2020 11:14:06 +0000 Received: from hirez.programming.kicks-ass.net (hirez.programming.kicks-ass.net [192.168.1.225]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by noisy.programming.kicks-ass.net (Postfix) with ESMTPS id 82F35307488; Fri, 17 Jul 2020 13:14:04 +0200 (CEST) Received: by hirez.programming.kicks-ass.net (Postfix, from userid 0) id C89A9203D4092; Fri, 17 Jul 2020 13:14:02 +0200 (CEST) Message-ID: <20200717111349.825172312@infradead.org> User-Agent: quilt/0.66 Date: Fri, 17 Jul 2020 13:10:13 +0200 From: Peter Zijlstra To: Will Deacon , "Aneesh Kumar K.V" , Andrew Morton , Nick Piggin , Peter Zijlstra Cc: linux-arch@vger.kernel.org, linux-sh@vger.kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Yoshinori Sato , Rich Felker , "David S. Miller" , Helge Deller , Geert Uytterhoeven , Paul Burton , Tony Luck , Richard Henderson , Nick Hu , Paul Walmsley , John Paul Adrian Glaubitz , Christoph Hellwig Subject: [PATCH v2 08/11] alpha/tlb: Fix __p*_free_tlb() References: <20200717111005.024867618@infradead.org> MIME-Version: 1.0 Sender: linux-sh-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-sh@vger.kernel.org Just like regular pages, page directories need to observe the following order: 1) unhook 2) TLB invalidate 3) free to ensure it is safe against concurrent accesses. Since Alpha has page based PMDs, no software walkers and IPI based TLB invalidation, it can use the simple tlb_remove_page() based freeer. Signed-off-by: Peter Zijlstra (Intel) --- arch/alpha/include/asm/tlb.h | 13 +++++++++++-- 1 file changed, 11 insertions(+), 2 deletions(-) --- a/arch/alpha/include/asm/tlb.h +++ b/arch/alpha/include/asm/tlb.h @@ -4,7 +4,16 @@ #include -#define __pte_free_tlb(tlb, pte, address) pte_free((tlb)->mm, pte) -#define __pmd_free_tlb(tlb, pmd, address) pmd_free((tlb)->mm, pmd) +extern void ___pte_free_tlb(struct mmu_gather *tlb, pte_t *pte); +extern void ___pmd_free_tlb(struct mmu_gather *tlb, pmd_t *pmd); + +#define __pte_free_tlb(tlb, pte, address) \ +do { \ + pgtable_pte_page_dtor(pte); \ + tlb_remove_table((tlb), (pte)); \ +} while (0) + +#define __pmd_free_tlb(tlb, pmd, address) \ + tlb_remove_table((tlb), virt_to_page(pmd)) #endif From patchwork Fri Jul 17 11:10:14 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Zijlstra X-Patchwork-Id: 11669923 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 8D9BD1392 for ; Fri, 17 Jul 2020 11:15:11 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 730762070E for ; Fri, 17 Jul 2020 11:15:11 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="yjSqa+5f" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726040AbgGQLOk (ORCPT ); Fri, 17 Jul 2020 07:14:40 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42190 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726446AbgGQLOj (ORCPT ); Fri, 17 Jul 2020 07:14:39 -0400 Received: from merlin.infradead.org (merlin.infradead.org [IPv6:2001:8b0:10b:1231::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 81221C061755; Fri, 17 Jul 2020 04:14:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=merlin.20170209; h=Content-Type:MIME-Version:References: Subject:Cc:To:From:Date:Message-ID:Sender:Reply-To:Content-Transfer-Encoding: Content-ID:Content-Description:In-Reply-To; bh=mXdH0EdEU6Cy2G+amp6Zc6DclrxoI0Xq6oO6KLBpmqE=; b=yjSqa+5fwh6Wy8j+pNJsjC/qU1 B2W55tXrcly0UqYSpaENQEh4QFIfSAwqFfE+vAJgyC6OLtPX9la8lpviT1MifASoGfPTlDnAlsFvJ 1c7lO2spg8rO5c925FNqxh1Z9SaTqtPcBMJ2TFskFB3abEcjcuuaM3ZvTLwzF6SNJJP0jWPMprbG/ wDNzdmdXu8XExJClvzvqC7btCKV0ixVINJH7odcI9gFbt8hDwfQyEzNMiQmv+yNKd/h1tTAO/7mmI DdVsystUlYxe9pki8bthn8jpHCIaw8jgSwxbdgw4wr0T2AMqTmkLQ82Lfkon8Q/Fl65+OKpC0AK1a mraN58Yg==; Received: from j217100.upc-j.chello.nl ([24.132.217.100] helo=noisy.programming.kicks-ass.net) by merlin.infradead.org with esmtpsa (Exim 4.92.3 #3 (Red Hat Linux)) id 1jwOJZ-0001Zo-Nd; Fri, 17 Jul 2020 11:14:05 +0000 Received: from hirez.programming.kicks-ass.net (hirez.programming.kicks-ass.net [192.168.1.225]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by noisy.programming.kicks-ass.net (Postfix) with ESMTPS id 82068306FFE; Fri, 17 Jul 2020 13:14:04 +0200 (CEST) Received: by hirez.programming.kicks-ass.net (Postfix, from userid 0) id CBE5F203D4097; Fri, 17 Jul 2020 13:14:02 +0200 (CEST) Message-ID: <20200717111349.883694582@infradead.org> User-Agent: quilt/0.66 Date: Fri, 17 Jul 2020 13:10:14 +0200 From: Peter Zijlstra To: Will Deacon , "Aneesh Kumar K.V" , Andrew Morton , Nick Piggin , Peter Zijlstra Cc: linux-arch@vger.kernel.org, linux-sh@vger.kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Yoshinori Sato , Rich Felker , "David S. Miller" , Helge Deller , Geert Uytterhoeven , Paul Burton , Tony Luck , Richard Henderson , Nick Hu , Paul Walmsley , John Paul Adrian Glaubitz , Christoph Hellwig Subject: [PATCH v2 09/11] nds32/tlb: Fix __p*_free_tlb() References: <20200717111005.024867618@infradead.org> MIME-Version: 1.0 Sender: linux-sh-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-sh@vger.kernel.org Just like regular pages, page directories need to observe the following order: 1) unhook 2) TLB invalidate 3) free to ensure it is safe against concurrent accesses. Even though NDS32 is UP only, we still need to observe this order because mmu_gather is preemptible. NDS32 does not in fact have PMDs. Signed-off-by: Peter Zijlstra (Intel) --- arch/nds32/include/asm/tlb.h | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) --- a/arch/nds32/include/asm/tlb.h +++ b/arch/nds32/include/asm/tlb.h @@ -6,6 +6,11 @@ #include -#define __pte_free_tlb(tlb, pte, addr) pte_free((tlb)->mm, pte) +#define __pte_free_tlb(tlb, pte, address) \ +do { \ + pgtable_pte_page_dtor(pte); \ + tlb_remove_table((tlb), (pte)); \ +} while (0) + #endif From patchwork Fri Jul 17 11:10:15 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Zijlstra X-Patchwork-Id: 11669917 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 71C1B17C7 for ; Fri, 17 Jul 2020 11:15:04 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 593EB20768 for ; Fri, 17 Jul 2020 11:15:04 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="FAUumSd9" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726843AbgGQLO4 (ORCPT ); Fri, 17 Jul 2020 07:14:56 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42214 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726686AbgGQLOn (ORCPT ); Fri, 17 Jul 2020 07:14:43 -0400 Received: from casper.infradead.org (casper.infradead.org [IPv6:2001:8b0:10b:1236::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E0A46C08C5CE; Fri, 17 Jul 2020 04:14:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Type:MIME-Version:References: Subject:Cc:To:From:Date:Message-ID:Sender:Reply-To:Content-Transfer-Encoding: Content-ID:Content-Description:In-Reply-To; bh=3R2DK7HvOJXWBjGxheTnCh1+oyh0A3xwV1bV88LUreM=; b=FAUumSd9I3/VNHUgNXx4k0r+oK XPsBvhXBs2S9D8KdP1DdzCn+DjvPQ/N53rubfaZdx/Zq28lfMnE4FWFrXxchpt7Fbf1H73vWibr4j kfeW2q7hWNzjn/xQ6xAH8dq5lajcLheqYNu/je7J2zCrVzuEsYRRAFG8On6JhRJVcrbLv0gwFgUug bT0FH7mxmojgXV6jLLYfJmaqGbBGG8nQCUBThUvi3fkTD9RIO3Clg7X9ZI3AaI+BL+SV+CRdZCc+o BsKMmuBNmewZ0MHaEgdFKq/FNkMMPDtazHdq2QEWx7CFn5z/oZZa+NB8HoSkX62V2Q62o6751m0i0 7KMjrc4A==; Received: from j217100.upc-j.chello.nl ([24.132.217.100] helo=noisy.programming.kicks-ass.net) by casper.infradead.org with esmtpsa (Exim 4.92.3 #3 (Red Hat Linux)) id 1jwOJa-00051T-8Y; Fri, 17 Jul 2020 11:14:06 +0000 Received: from hirez.programming.kicks-ass.net (hirez.programming.kicks-ass.net [192.168.1.225]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by noisy.programming.kicks-ass.net (Postfix) with ESMTPS id 81233306E3A; Fri, 17 Jul 2020 13:14:04 +0200 (CEST) Received: by hirez.programming.kicks-ass.net (Postfix, from userid 0) id D05A1203D4099; Fri, 17 Jul 2020 13:14:02 +0200 (CEST) Message-ID: <20200717111349.942447764@infradead.org> User-Agent: quilt/0.66 Date: Fri, 17 Jul 2020 13:10:15 +0200 From: Peter Zijlstra To: Will Deacon , "Aneesh Kumar K.V" , Andrew Morton , Nick Piggin , Peter Zijlstra Cc: linux-arch@vger.kernel.org, linux-sh@vger.kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Yoshinori Sato , Rich Felker , "David S. Miller" , Helge Deller , Geert Uytterhoeven , Paul Burton , Tony Luck , Richard Henderson , Nick Hu , Paul Walmsley , John Paul Adrian Glaubitz , Christoph Hellwig Subject: [PATCH v2 10/11] riscv/tlb: Fix __p*_free_tlb() References: <20200717111005.024867618@infradead.org> MIME-Version: 1.0 Sender: linux-sh-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-sh@vger.kernel.org Just like regular pages, page directories need to observe the following order: 1) unhook 2) TLB invalidate 3) free to ensure it is safe against concurrent accesses. Since RISC-V has page based PMDs, no software walkers and IPI based TLB invalidation, it can use the simple tlb_remove_page() based freeer. Signed-off-by: Peter Zijlstra (Intel) --- arch/riscv/include/asm/pgalloc.h | 11 ++++++----- 1 file changed, 6 insertions(+), 5 deletions(-) --- a/arch/riscv/include/asm/pgalloc.h +++ b/arch/riscv/include/asm/pgalloc.h @@ -73,14 +73,15 @@ static inline void pmd_free(struct mm_st free_page((unsigned long)pmd); } -#define __pmd_free_tlb(tlb, pmd, addr) pmd_free((tlb)->mm, pmd) +#define __pmd_free_tlb(tlb, pmd, addr) \ + tlb_remove_table((tlb), virt_to_page(pmd)) #endif /* __PAGETABLE_PMD_FOLDED */ -#define __pte_free_tlb(tlb, pte, buf) \ -do { \ - pgtable_pte_page_dtor(pte); \ - tlb_remove_page((tlb), pte); \ +#define __pte_free_tlb(tlb, pte, buf) \ +do { \ + pgtable_pte_page_dtor(pte); \ + tlb_remove_table((tlb), (pte)); \ } while (0) #endif /* CONFIG_MMU */ From patchwork Fri Jul 17 11:10:16 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Zijlstra X-Patchwork-Id: 11669911 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 38C7114E3 for ; Fri, 17 Jul 2020 11:14:55 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 1FE332076A for ; Fri, 17 Jul 2020 11:14:55 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="tp/BH55Z" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726696AbgGQLOo (ORCPT ); Fri, 17 Jul 2020 07:14:44 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42210 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726446AbgGQLOn (ORCPT ); Fri, 17 Jul 2020 07:14:43 -0400 Received: from casper.infradead.org (casper.infradead.org [IPv6:2001:8b0:10b:1236::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D39D6C061755; Fri, 17 Jul 2020 04:14:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Type:MIME-Version:References: Subject:Cc:To:From:Date:Message-ID:Sender:Reply-To:Content-Transfer-Encoding: Content-ID:Content-Description:In-Reply-To; bh=nUUfhERBps7kvabI82RVV43naTK71jo6AAVltGUbW/s=; b=tp/BH55Z/QaPl4II/4kpXNcQdv XDl6UXJmW/vuilqUj195qptQ8vI7Xjlkz+6fvZRtbA3aih5li6rf3HtArbX12fyCj3oHtV5B5n4x9 pcc8tGMTIGSttnHzrRfWW0GvzPzKr+OYzu71OrB+gClwoAs8q2snBYoiufgmW5vPb/O0n7oS/VgLp 1NhQGSuISW5KYKcEhffZD+fJUGtlP066i0gESEXqxUnAkm4l8RznaZm0krzFJi3a2czVK2nIuw55x OboZwga/+8b43D0SyvxLOGCPFwZjTAjdlw36TvLSKt/cw886FL9tjZ2hH7WNnF1cGZeJlxfAKWDZY wt8k6hUA==; Received: from j217100.upc-j.chello.nl ([24.132.217.100] helo=noisy.programming.kicks-ass.net) by casper.infradead.org with esmtpsa (Exim 4.92.3 #3 (Red Hat Linux)) id 1jwOJZ-00051O-Og; Fri, 17 Jul 2020 11:14:06 +0000 Received: from hirez.programming.kicks-ass.net (hirez.programming.kicks-ass.net [192.168.1.225]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by noisy.programming.kicks-ass.net (Postfix) with ESMTPS id 86B813074D4; Fri, 17 Jul 2020 13:14:04 +0200 (CEST) Received: by hirez.programming.kicks-ass.net (Postfix, from userid 0) id D4590203D409A; Fri, 17 Jul 2020 13:14:02 +0200 (CEST) Message-ID: <20200717111350.001250805@infradead.org> User-Agent: quilt/0.66 Date: Fri, 17 Jul 2020 13:10:16 +0200 From: Peter Zijlstra To: Will Deacon , "Aneesh Kumar K.V" , Andrew Morton , Nick Piggin , Peter Zijlstra Cc: linux-arch@vger.kernel.org, linux-sh@vger.kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Yoshinori Sato , Rich Felker , "David S. Miller" , Helge Deller , Geert Uytterhoeven , Paul Burton , Tony Luck , Richard Henderson , Nick Hu , Paul Walmsley , John Paul Adrian Glaubitz , Christoph Hellwig Subject: [PATCH v2 11/11] m68k/tlb: Fix __p*_free_tlb() References: <20200717111005.024867618@infradead.org> MIME-Version: 1.0 Sender: linux-sh-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-sh@vger.kernel.org Just like regular pages, page directories need to observe the following order: 1) unhook 2) TLB invalidate 3) free to ensure it is safe against concurrent accesses. The Motorola MMU has funny PMD stuff, so use a custom table freeer. Signed-off-by: Peter Zijlstra (Intel) --- arch/m68k/Kconfig | 1 + arch/m68k/include/asm/mcf_pgalloc.h | 14 ++++++-------- arch/m68k/include/asm/motorola_pgalloc.h | 10 ++++++---- arch/m68k/mm/motorola.c | 17 +++++++++++++++++ 4 files changed, 30 insertions(+), 12 deletions(-) --- a/arch/m68k/Kconfig +++ b/arch/m68k/Kconfig @@ -32,6 +32,7 @@ config M68K select OLD_SIGSUSPEND3 select OLD_SIGACTION select MMU_GATHER_NO_RANGE if MMU + select MMU_GATHER_TABLE_FREE if MMU_MOTOROLA config CPU_BIG_ENDIAN def_bool y --- a/arch/m68k/include/asm/mcf_pgalloc.h +++ b/arch/m68k/include/asm/mcf_pgalloc.h @@ -34,14 +34,12 @@ extern inline pmd_t *pmd_alloc_kernel(pg #define pmd_pgtable(pmd) pfn_to_virt(pmd_val(pmd) >> PAGE_SHIFT) -static inline void __pte_free_tlb(struct mmu_gather *tlb, pgtable_t pgtable, - unsigned long address) -{ - struct page *page = virt_to_page(pgtable); - - pgtable_pte_page_dtor(page); - __free_page(page); -} +#define __pte_free_tlb(tlb, pgtable, addr) \ +do { \ + struct page *page = virt_to_page(pgtable); \ + pgtable_pte_page_dtor(page); \ + tlb_remove_page((tlb), page); \ +} while (0) static inline pgtable_t pte_alloc_one(struct mm_struct *mm) { --- a/arch/m68k/include/asm/motorola_pgalloc.h +++ b/arch/m68k/include/asm/motorola_pgalloc.h @@ -14,6 +14,8 @@ enum m68k_table_types { TABLE_PTE = 1, }; +extern void pgtable_free_tlb(struct mmu_gather *tlb, void *table, int type); +extern void __tlb_remove_table(void *table); extern void init_pointer_table(void *table, int type); extern void *get_pointer_table(int type); extern int free_pointer_table(void *table, int type); @@ -47,7 +49,7 @@ static inline void pte_free(struct mm_st static inline void __pte_free_tlb(struct mmu_gather *tlb, pgtable_t pgtable, unsigned long address) { - free_pointer_table(pgtable, TABLE_PTE); + pgtable_free_tlb(tlb, pgtable, TABLE_PTE); } @@ -61,10 +63,10 @@ static inline int pmd_free(struct mm_str return free_pointer_table(pmd, TABLE_PMD); } -static inline int __pmd_free_tlb(struct mmu_gather *tlb, pmd_t *pmd, - unsigned long address) +static inline void __pmd_free_tlb(struct mmu_gather *tlb, pmd_t *pmd, + unsigned long address) { - return free_pointer_table(pmd, TABLE_PMD); + pgtable_free_tlb(tlb, pmd, TABLE_PMD); } --- a/arch/m68k/mm/motorola.c +++ b/arch/m68k/mm/motorola.c @@ -215,6 +215,23 @@ int free_pointer_table(void *table, int return 0; } +void pgtable_free_tlb(struct mmu_gather *tlb, void *table, int type) +{ + unsigned long ptr = (unsigned long)table; + BUG_ON(type > 1); + BUG_ON(ptr & 1); + ptr |= type; + tlb_remove_table(tlb, (void *)ptr); +} + +void __tlb_remove_table(void *table) +{ + int type = (unsigned long)table & 1; + table = (void *)((unsigned long)table & ~1); + + free_pointer_table(table, type); +} + /* size of memory already mapped in head.S */ extern __initdata unsigned long m68k_init_mapped_size;