From patchwork Wed May 29 22:47:11 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul Burton X-Patchwork-Id: 10967893 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 1A04B1398 for ; Wed, 29 May 2019 22:48:59 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 02C8C286A4 for ; Wed, 29 May 2019 22:48:59 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id EB684289D2; Wed, 29 May 2019 22:48:58 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 6FFEF286A4 for ; Wed, 29 May 2019 22:48:56 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726189AbfE2Ws4 (ORCPT ); Wed, 29 May 2019 18:48:56 -0400 Received: from mail-eopbgr760135.outbound.protection.outlook.com ([40.107.76.135]:54443 "EHLO NAM02-CY1-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1726428AbfE2Wsz (ORCPT ); Wed, 29 May 2019 18:48:55 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=wavesemi.onmicrosoft.com; s=selector1-wavesemi-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=KYNUPDpLbeLqcGmCWUEb2dcdImVyYawjb+FFP/9ZiAU=; b=Oq3e1RSp8O5rPDfNlQgqulyhWGmUp1NXGs26ib+lMPfKL2zBq8bh1HTgF+4JtW/jlpgbcXfH+UtoUkL9DsiKRgntmUJ3keEcylxyTd/leDFllIV1n7zewpxuR58dcbjXQlRF4joIvfHPg9jIBqsLIZZwb7vurh3R8w7WTPdUQ30= Received: from MWHPR2201MB1277.namprd22.prod.outlook.com (10.174.162.17) by MWHPR2201MB1757.namprd22.prod.outlook.com (10.164.133.167) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.1922.18; Wed, 29 May 2019 22:47:11 +0000 Received: from MWHPR2201MB1277.namprd22.prod.outlook.com ([fe80::90ff:8d19:8459:834b]) by MWHPR2201MB1277.namprd22.prod.outlook.com ([fe80::90ff:8d19:8459:834b%7]) with mapi id 15.20.1922.021; Wed, 29 May 2019 22:47:11 +0000 From: Paul Burton To: "linux-mips@vger.kernel.org" , Chris Packham CC: Paul Burton Subject: [PATCH] MIPS: mm: Implement flush_cache_v(un)map using __flush_kernel_vmap_range Thread-Topic: [PATCH] MIPS: mm: Implement flush_cache_v(un)map using __flush_kernel_vmap_range Thread-Index: AQHVFnB0gMlUP9a1GkCwDAB1IXJGDg== Date: Wed, 29 May 2019 22:47:11 +0000 Message-ID: <20190529224659.27614-1-paul.burton@mips.com> References: <20190528221255.22460-1-chris.packham@alliedtelesis.co.nz> In-Reply-To: <20190528221255.22460-1-chris.packham@alliedtelesis.co.nz> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-clientproxiedby: BY5PR04CA0030.namprd04.prod.outlook.com (2603:10b6:a03:1d0::40) To MWHPR2201MB1277.namprd22.prod.outlook.com (2603:10b6:301:24::17) authentication-results: spf=none (sender IP is ) smtp.mailfrom=pburton@wavecomp.com; x-ms-exchange-messagesentrepresentingtype: 1 x-mailer: git-send-email 2.21.0 x-originating-ip: [73.93.155.82] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: f98d5c67-2b2f-44ac-fc3a-08d6e487964a x-microsoft-antispam: BCL:0;PCL:0;RULEID:(2390118)(7020095)(4652040)(8989299)(4534185)(4627221)(201703031133081)(201702281549075)(8990200)(5600148)(711020)(4605104)(1401327)(2017052603328)(7193020);SRVR:MWHPR2201MB1757; x-ms-traffictypediagnostic: MWHPR2201MB1757: x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:9508; x-forefront-prvs: 0052308DC6 x-forefront-antispam-report: SFV:NSPM;SFS:(10019020)(346002)(39850400004)(396003)(376002)(136003)(366004)(189003)(199004)(2906002)(25786009)(50226002)(6116002)(53936002)(3846002)(4326008)(36756003)(107886003)(256004)(14444005)(8676002)(81166006)(81156014)(8936002)(66946007)(64756008)(66446008)(73956011)(66556008)(478600001)(316002)(76176011)(386003)(6506007)(102836004)(66476007)(2501003)(486006)(99286004)(42882007)(186003)(44832011)(476003)(2616005)(11346002)(446003)(26005)(52116002)(110136005)(14454004)(305945005)(68736007)(6436002)(7736002)(71200400001)(71190400001)(6512007)(66066001)(5660300002)(1076003)(6486002);DIR:OUT;SFP:1102;SCL:1;SRVR:MWHPR2201MB1757;H:MWHPR2201MB1277.namprd22.prod.outlook.com;FPR:;SPF:None;LANG:en;PTR:InfoNoRecords;MX:1;A:1; received-spf: None (protection.outlook.com: wavecomp.com does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 x-microsoft-antispam-message-info: SXepklhnrl3iI6n1PkHtMnRGQ/K3xgTVHyd1//6WGoateslX1bX6oGMlhi6it5m2DIHjxI3zCA+X6o4uAy5uHQcghaP3O3qaJ5w4HgfMTTFwsAiIbFy/5rjVQvyBQueYrDPz1oMBbZNUTfxCXGz6epnW+Bb5XFXuWfznglsxESF4KPcd2F5acmgAPUAHGUOXgf6RJpVh1J/WTFIaP5dizaIc/h/r6/Cgv7QYEG7nV/oUIiTeQ+Ak7/wylePIxIu7OcaJWUaAF0AG9hh62bRbh6PgArC6dBY67jfNL/f9nyjtNGXumUplhNNRfYKVHmQPJyWnKL+8/79oGzzxAw4SGVGys86xXGBZk3dPaMXRFGiu2vkl3hbbBRUIDk0M/hZv97MALSrWdwlo24jw57W2RWpqPj8UydViN0J7KgIhpCo= MIME-Version: 1.0 X-OriginatorOrg: mips.com X-MS-Exchange-CrossTenant-Network-Message-Id: f98d5c67-2b2f-44ac-fc3a-08d6e487964a X-MS-Exchange-CrossTenant-originalarrivaltime: 29 May 2019 22:47:11.6861 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 463607d3-1db3-40a0-8a29-970c56230104 X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: pburton@wavecomp.com X-MS-Exchange-Transport-CrossTenantHeadersStamped: MWHPR2201MB1757 Sender: linux-mips-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-mips@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Our flush_cache_vmap() & flush_cache_vunmap() implementations for R4k style systems simply call r4k_blast_dcache() to wipe out the whole L1 dcache if it suffers from aliases. This is unsafe on SMP for 2 reasons: 1) r4k_blast_dcache() relies upon preemption being disabled so that it can use current_cpu_data/smp_processor_id() to discover the properties of the current CPU's dcache & ensure the whole flush operation happens on one CPU. This may not be the case when flush_cache_vmap() or flush_cache_vunmap() are called. 2) It only flushes caches on one CPU, which means the caches for other CPUs may still contain stale data. We already have an implementation of flush_kernel_vmap_range() which does exactly what we need - it invalidates dcache entries on all CPUs safely, and is better optimized to avoid wiping out the entire cache for small flushes. Reimplement flush_cache_vmap() & flush_cache_vunmap() using __flush_kernel_vmap_range() which does what we need already. For tx39 __flush_kernel_vmap_range() will simply BUG(), but so far as I can see tx39 systems don't suffer from dcache aliasing so this should be fine since it should never be called. Signed-off-by: Paul Burton Reported-by: Chris Packham --- Chris, would you mind giving this a try? --- arch/mips/include/asm/cacheflush.h | 28 ++++++++++++---------------- arch/mips/mm/c-r4k.c | 15 --------------- arch/mips/mm/c-tx39.c | 16 ---------------- arch/mips/mm/cache.c | 3 --- 4 files changed, 12 insertions(+), 50 deletions(-) diff --git a/arch/mips/include/asm/cacheflush.h b/arch/mips/include/asm/cacheflush.h index d687b40b9fbb..6285c830c9f2 100644 --- a/arch/mips/include/asm/cacheflush.h +++ b/arch/mips/include/asm/cacheflush.h @@ -85,22 +85,6 @@ extern void (*__flush_icache_user_range)(unsigned long start, extern void (*__local_flush_icache_user_range)(unsigned long start, unsigned long end); -extern void (*__flush_cache_vmap)(void); - -static inline void flush_cache_vmap(unsigned long start, unsigned long end) -{ - if (cpu_has_dc_aliases) - __flush_cache_vmap(); -} - -extern void (*__flush_cache_vunmap)(void); - -static inline void flush_cache_vunmap(unsigned long start, unsigned long end) -{ - if (cpu_has_dc_aliases) - __flush_cache_vunmap(); -} - extern void copy_to_user_page(struct vm_area_struct *vma, struct page *page, unsigned long vaddr, void *dst, const void *src, unsigned long len); @@ -150,4 +134,16 @@ static inline void invalidate_kernel_vmap_range(void *vaddr, int size) __flush_kernel_vmap_range((unsigned long) vaddr, size); } +static inline void flush_cache_vmap(unsigned long start, unsigned long end) +{ + if (cpu_has_dc_aliases) + __flush_kernel_vmap_range(start, end - start); +} + +static inline void flush_cache_vunmap(unsigned long start, unsigned long end) +{ + if (cpu_has_dc_aliases) + __flush_kernel_vmap_range(start, end - start); +} + #endif /* _ASM_CACHEFLUSH_H */ diff --git a/arch/mips/mm/c-r4k.c b/arch/mips/mm/c-r4k.c index 5166e38cd1c6..2b2953d3949d 100644 --- a/arch/mips/mm/c-r4k.c +++ b/arch/mips/mm/c-r4k.c @@ -559,16 +559,6 @@ static inline int has_valid_asid(const struct mm_struct *mm, unsigned int type) return 0; } -static void r4k__flush_cache_vmap(void) -{ - r4k_blast_dcache(); -} - -static void r4k__flush_cache_vunmap(void) -{ - r4k_blast_dcache(); -} - /* * Note: flush_tlb_range() assumes flush_cache_range() sufficiently flushes * whole caches when vma is executable. @@ -1854,9 +1844,6 @@ void r4k_cache_init(void) else shm_align_mask = PAGE_SIZE-1; - __flush_cache_vmap = r4k__flush_cache_vmap; - __flush_cache_vunmap = r4k__flush_cache_vunmap; - flush_cache_all = cache_noop; __flush_cache_all = r4k___flush_cache_all; flush_cache_mm = r4k_flush_cache_mm; @@ -1931,8 +1918,6 @@ void r4k_cache_init(void) case CPU_LOONGSON3: /* Loongson-3 maintains cache coherency by hardware */ __flush_cache_all = cache_noop; - __flush_cache_vmap = cache_noop; - __flush_cache_vunmap = cache_noop; __flush_kernel_vmap_range = (void *)cache_noop; flush_cache_mm = (void *)cache_noop; flush_cache_page = (void *)cache_noop; diff --git a/arch/mips/mm/c-tx39.c b/arch/mips/mm/c-tx39.c index b7c8a9d79c35..6bf13a7db485 100644 --- a/arch/mips/mm/c-tx39.c +++ b/arch/mips/mm/c-tx39.c @@ -121,16 +121,6 @@ static inline void tx39_blast_icache(void) local_irq_restore(flags); } -static void tx39__flush_cache_vmap(void) -{ - tx39_blast_dcache(); -} - -static void tx39__flush_cache_vunmap(void) -{ - tx39_blast_dcache(); -} - static inline void tx39_flush_cache_all(void) { if (!cpu_has_dc_aliases) @@ -339,8 +329,6 @@ void tx39_cache_init(void) switch (current_cpu_type()) { case CPU_TX3912: /* TX39/H core (writethru direct-map cache) */ - __flush_cache_vmap = tx39__flush_cache_vmap; - __flush_cache_vunmap = tx39__flush_cache_vunmap; flush_cache_all = tx39h_flush_icache_all; __flush_cache_all = tx39h_flush_icache_all; flush_cache_mm = (void *) tx39h_flush_icache_all; @@ -363,10 +351,6 @@ void tx39_cache_init(void) default: /* TX39/H2,H3 core (writeback 2way-set-associative cache) */ /* board-dependent init code may set WBON */ - - __flush_cache_vmap = tx39__flush_cache_vmap; - __flush_cache_vunmap = tx39__flush_cache_vunmap; - flush_cache_all = tx39_flush_cache_all; __flush_cache_all = tx39___flush_cache_all; flush_cache_mm = tx39_flush_cache_mm; diff --git a/arch/mips/mm/cache.c b/arch/mips/mm/cache.c index 3da216988672..c6c0ef539d3a 100644 --- a/arch/mips/mm/cache.c +++ b/arch/mips/mm/cache.c @@ -40,9 +40,6 @@ EXPORT_SYMBOL_GPL(__flush_icache_user_range); void (*__local_flush_icache_user_range)(unsigned long start, unsigned long end); EXPORT_SYMBOL_GPL(__local_flush_icache_user_range); -void (*__flush_cache_vmap)(void); -void (*__flush_cache_vunmap)(void); - void (*__flush_kernel_vmap_range)(unsigned long vaddr, int size); EXPORT_SYMBOL_GPL(__flush_kernel_vmap_range);