From patchwork Thu Nov 15 05:50:51 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Huacai Chen X-Patchwork-Id: 10683523 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 24D8614DB for ; Thu, 15 Nov 2018 05:52:49 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 0D1C12C13A for ; Thu, 15 Nov 2018 05:52:49 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 0196A2C13D; Thu, 15 Nov 2018 05:52:48 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.7 required=2.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 600FC2C13A for ; Thu, 15 Nov 2018 05:52:48 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728887AbeKOP7M (ORCPT ); Thu, 15 Nov 2018 10:59:12 -0500 Received: from mail-pl1-f195.google.com ([209.85.214.195]:44396 "EHLO mail-pl1-f195.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727045AbeKOP7L (ORCPT ); Thu, 15 Nov 2018 10:59:11 -0500 Received: by mail-pl1-f195.google.com with SMTP id s5-v6so8937385plq.11; Wed, 14 Nov 2018 21:52:46 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:from:to:cc:subject:date:message-id:in-reply-to:references; bh=P1o/kNpFTynfCCO8e3CTgrAzpq+FNajDEU1dHiCL9BM=; b=k5kdZmSbxr27F+Jv6z4PsZxllpN8sKfaULLAbT5kIoGO7odg4kQj+xnm5ZH5kG4ll6 8AtZAJdnokbeXE9J56gbTD1reXto53LeSI7MPBOJIdVxX6PBiDulKTHQ8vJ2kj8XaETd iZdfgJAskZFQTpY9Edybli8RevvJyyScpwgevFt2ngJSUqey91RyhB7qmsJS9xeXeusX 2+DoK/SSRSuLc6IeGikhdp/V8MWXq9yQLgpk9wWMEJeWwI3bS8iX406Ua4VtSy7D8lOK ZEH0nvxjfokFw63sCaqC1HDZEfR1DjvtJ+6kcdU616Rj5tB4Df9Va7Lt41Y3tat0PDvC HQXA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:from:to:cc:subject:date:message-id :in-reply-to:references; bh=P1o/kNpFTynfCCO8e3CTgrAzpq+FNajDEU1dHiCL9BM=; b=E6dqEZdWteVPfT60B3C0KpenucyLsXmqoxvp124vbkKwCkI8Q03vMMKIXg5W/gEcK4 Q8+dSmyOhLEidvdUVtlubCUUgfOeARVdfr2CCF31yEGo6qs9pXNLq+a0UAqk6bO6Tos/ k95AXgpQUsG1Jck0GgtxYbTnSELGQkx29K4vp+DmWVHtxiMafT+gNFrcyeLZm0v5YSUI wLveQu2H7AvDVw+Bp2yt+T+nMcg6p36WTUBYpNSXwrq4Le8a/efdQHZebXE4/QjRLWJO N4A+AIqREIihNIXaY82IbiuR6J0pcO5cQn6dIXBGRuzAlGToauGnnOqBBBfyhW9mau3e 9Muw== X-Gm-Message-State: AGRZ1gKVs87quMEZDXD0K9QPKr0j8g7TG5MVzaFeGCpJ4IkwaBfgS7i+ G4xpOOhexNDNZrPqXYkg7MU= X-Google-Smtp-Source: AJdET5cOUyFcr/EIAfJ7GUNokwm/jT6h0jTsPyHOpKBryzNvU9zIKcuaUonY3IcYdJiBChufl/Hjxg== X-Received: by 2002:a17:902:166:: with SMTP id 93-v6mr4814763plb.68.1542261166359; Wed, 14 Nov 2018 21:52:46 -0800 (PST) Received: from software.domain.org ([222.92.8.142]) by smtp.gmail.com with ESMTPSA id o1sm34882306pgn.63.2018.11.14.21.52.43 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Wed, 14 Nov 2018 21:52:45 -0800 (PST) From: Huacai Chen To: Jean Delvare Cc: Guenter Roeck , linux-hwmon@vger.kernel.org, Fuxin Zhang , Huacai Chen , "# 3 . 15+" Subject: [PATCH V5 2/8] MIPS: c-r4k: Add r4k_blast_scache_node for Loongson-3 Date: Thu, 15 Nov 2018 13:50:51 +0800 Message-Id: <1542261057-6019-3-git-send-email-chenhc@lemote.com> X-Mailer: git-send-email 2.7.0 In-Reply-To: <1542261057-6019-1-git-send-email-chenhc@lemote.com> References: <1542261057-6019-1-git-send-email-chenhc@lemote.com> Sender: linux-hwmon-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-hwmon@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP For multi-node Loongson-3 (NUMA configuration), r4k_blast_scache() can only flush Node-0's scache. So we add r4k_blast_scache_node() by using (CAC_BASE | (node_id << NODE_ADDRSPACE_SHIFT)) instead of CKSEG0 as the start address. Cc: # 3.15+ Signed-off-by: Huacai Chen --- arch/mips/include/asm/mach-loongson64/mmzone.h | 1 + arch/mips/include/asm/mmzone.h | 8 +++++ arch/mips/include/asm/r4kcache.h | 25 +++++++++++++++ arch/mips/mm/c-r4k.c | 44 ++++++++++++++++++++++---- 4 files changed, 71 insertions(+), 7 deletions(-) diff --git a/arch/mips/include/asm/mach-loongson64/mmzone.h b/arch/mips/include/asm/mach-loongson64/mmzone.h index c9f7e23..59c8b11 100644 --- a/arch/mips/include/asm/mach-loongson64/mmzone.h +++ b/arch/mips/include/asm/mach-loongson64/mmzone.h @@ -21,6 +21,7 @@ #define NODE3_ADDRSPACE_OFFSET 0x300000000000UL #define pa_to_nid(addr) (((addr) & 0xf00000000000) >> NODE_ADDRSPACE_SHIFT) +#define nid_to_addrbase(nid) ((nid) << NODE_ADDRSPACE_SHIFT) #define LEVELS_PER_SLICE 128 diff --git a/arch/mips/include/asm/mmzone.h b/arch/mips/include/asm/mmzone.h index f085fba..2a0fe1d 100644 --- a/arch/mips/include/asm/mmzone.h +++ b/arch/mips/include/asm/mmzone.h @@ -9,6 +9,14 @@ #include #include +#ifndef pa_to_nid +#define pa_to_nid(addr) 0 +#endif + +#ifndef nid_to_addrbase +#define nid_to_addrbase(nid) 0 +#endif + #ifdef CONFIG_DISCONTIGMEM #define pfn_to_nid(pfn) pa_to_nid((pfn) << PAGE_SHIFT) diff --git a/arch/mips/include/asm/r4kcache.h b/arch/mips/include/asm/r4kcache.h index d19b2d6..0815c2a 100644 --- a/arch/mips/include/asm/r4kcache.h +++ b/arch/mips/include/asm/r4kcache.h @@ -674,4 +674,25 @@ __BUILD_BLAST_CACHE_RANGE(s, scache, Hit_Writeback_Inv_SD, , ) __BUILD_BLAST_CACHE_RANGE(inv_d, dcache, Hit_Invalidate_D, , ) __BUILD_BLAST_CACHE_RANGE(inv_s, scache, Hit_Invalidate_SD, , ) +/* Currently, this is very specific to Loongson-3 */ +#define __BUILD_BLAST_CACHE_NODE(pfx, desc, indexop, hitop, lsize) \ +static inline void blast_##pfx##cache##lsize##_node(long node) \ +{ \ + unsigned long start = CAC_BASE | nid_to_addrbase(node); \ + unsigned long end = start + current_cpu_data.desc.waysize; \ + unsigned long ws_inc = 1UL << current_cpu_data.desc.waybit; \ + unsigned long ws_end = current_cpu_data.desc.ways << \ + current_cpu_data.desc.waybit; \ + unsigned long ws, addr; \ + \ + for (ws = 0; ws < ws_end; ws += ws_inc) \ + for (addr = start; addr < end; addr += lsize * 32) \ + cache##lsize##_unroll32(addr|ws, indexop); \ +} + +__BUILD_BLAST_CACHE_NODE(s, scache, Index_Writeback_Inv_SD, Hit_Writeback_Inv_SD, 16) +__BUILD_BLAST_CACHE_NODE(s, scache, Index_Writeback_Inv_SD, Hit_Writeback_Inv_SD, 32) +__BUILD_BLAST_CACHE_NODE(s, scache, Index_Writeback_Inv_SD, Hit_Writeback_Inv_SD, 64) +__BUILD_BLAST_CACHE_NODE(s, scache, Index_Writeback_Inv_SD, Hit_Writeback_Inv_SD, 128) + #endif /* _ASM_R4KCACHE_H */ diff --git a/arch/mips/mm/c-r4k.c b/arch/mips/mm/c-r4k.c index 7e430b4..96d666a 100644 --- a/arch/mips/mm/c-r4k.c +++ b/arch/mips/mm/c-r4k.c @@ -459,11 +459,28 @@ static void r4k_blast_scache_setup(void) r4k_blast_scache = blast_scache128; } +static void (*r4k_blast_scache_node)(long node); + +static void r4k_blast_scache_node_setup(void) +{ + unsigned long sc_lsize = cpu_scache_line_size(); + + if (current_cpu_type() != CPU_LOONGSON3) + r4k_blast_scache_node = (void *)cache_noop; + else if (sc_lsize == 16) + r4k_blast_scache_node = blast_scache16_node; + else if (sc_lsize == 32) + r4k_blast_scache_node = blast_scache32_node; + else if (sc_lsize == 64) + r4k_blast_scache_node = blast_scache64_node; + else if (sc_lsize == 128) + r4k_blast_scache_node = blast_scache128_node; +} + static inline void local_r4k___flush_cache_all(void * args) { switch (current_cpu_type()) { case CPU_LOONGSON2: - case CPU_LOONGSON3: case CPU_R4000SC: case CPU_R4000MC: case CPU_R4400SC: @@ -480,6 +497,11 @@ static inline void local_r4k___flush_cache_all(void * args) r4k_blast_scache(); break; + case CPU_LOONGSON3: + /* Use get_ebase_cpunum() for both NUMA=y/n */ + r4k_blast_scache_node(get_ebase_cpunum() >> 2); + break; + case CPU_BMIPS5000: r4k_blast_scache(); __sync(); @@ -840,10 +862,14 @@ static void r4k_dma_cache_wback_inv(unsigned long addr, unsigned long size) preempt_disable(); if (cpu_has_inclusive_pcaches) { - if (size >= scache_size) - r4k_blast_scache(); - else + if (size >= scache_size) { + if (current_cpu_type() != CPU_LOONGSON3) + r4k_blast_scache(); + else + r4k_blast_scache_node(pa_to_nid(addr)); + } else { blast_scache_range(addr, addr + size); + } preempt_enable(); __sync(); return; @@ -877,9 +903,12 @@ static void r4k_dma_cache_inv(unsigned long addr, unsigned long size) preempt_disable(); if (cpu_has_inclusive_pcaches) { - if (size >= scache_size) - r4k_blast_scache(); - else { + if (size >= scache_size) { + if (current_cpu_type() != CPU_LOONGSON3) + r4k_blast_scache(); + else + r4k_blast_scache_node(pa_to_nid(addr)); + } else { /* * There is no clearly documented alignment requirement * for the cache instruction on MIPS processors and @@ -1918,6 +1947,7 @@ void r4k_cache_init(void) r4k_blast_scache_page_setup(); r4k_blast_scache_page_indexed_setup(); r4k_blast_scache_setup(); + r4k_blast_scache_node_setup(); #ifdef CONFIG_EVA r4k_blast_dcache_user_page_setup(); r4k_blast_icache_user_page_setup();