From patchwork Sat Mar 9 14:52:04 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Gary Guo X-Patchwork-Id: 10845953 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 110674B65 for ; Sat, 9 Mar 2019 14:52:17 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id E92C32BBEB for ; Sat, 9 Mar 2019 14:52:16 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id DD6D92BC2B; Sat, 9 Mar 2019 14:52:16 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-4.2 required=2.0 tests=BAD_ENC_HEADER,BAYES_00, DKIM_SIGNED,DKIM_VALID,MAILING_LIST_MULTI,MISSING_HEADERS,RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 37F352BC0E for ; Sat, 9 Mar 2019 14:52:16 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:Cc:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:In-Reply-To:References: Message-ID:Date:Subject:From:Reply-To:To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=uoYLO/ZOO91r02c2zNMfb7tQyCFpXSREDDLmxAJ+R40=; b=RlZxAlm+dkCEI8 rk3WfVcv+nhO7V934bRaEWQssWuo1AX6uAo+Pk4q8nvdMKkL3Y2zXFv4EdEQLChRyvLZgzba8Bybw rD0MbG12NueeCtRLvIHFmZC7TBKid4FxH9+NRXuG6MqPJi8vGl9yRHfpHqSjSvbRbLg5wLYtnzamu SyoXK9cc6fP6+im7Bzlm4e8MKjV3liNtJh2a+gNRgy5y92RrOO1RmAJzYxB8hZA7FUxpU9QcMgR3y qfnc/BQKd7D8EcHADub8eSRXCbn+zwLMfpjiJaS5BH4GijulqOfgktvMiSDlrC5Cid/dcQpiuLOuA Ws6W490q07H9MDqZBkpw==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.90_1 #2 (Red Hat Linux)) id 1h2dKe-0006A4-Pk; Sat, 09 Mar 2019 14:52:12 +0000 Received: from mail-eopbgr110109.outbound.protection.outlook.com ([40.107.11.109] helo=GBR01-CWL-obe.outbound.protection.outlook.com) by bombadil.infradead.org with esmtps (Exim 4.90_1 #2 (Red Hat Linux)) id 1h2dKc-00068V-CA for linux-riscv@lists.infradead.org; Sat, 09 Mar 2019 14:52:11 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=garyguo.net; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=73GXrscOF7oDNb7BvZksg+VnZfN+o+ZWacdQz5M6dk8=; b=LKzhnYajUUNSF8b07//yCxNFgdix2B3/NbMQOjKbTfRePFqTT5+4NlNMiFZ7fDbjDVHbLn1kvWOVwEN72YuegRKYewckZltDpEzTJ2ypxRjylxZNEgBnZrIDSuKV+UDNUNmMdAp3tNIJSAOiDktnGnMLjjp0nYd5nUYf/9GAlB8= Received: from LO2P265MB0847.GBRP265.PROD.OUTLOOK.COM (20.176.139.20) by LO2P265MB0158.GBRP265.PROD.OUTLOOK.COM (10.166.98.23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.1686.18; Sat, 9 Mar 2019 14:52:04 +0000 Received: from LO2P265MB0847.GBRP265.PROD.OUTLOOK.COM ([fe80::ed34:1290:4306:3157]) by LO2P265MB0847.GBRP265.PROD.OUTLOOK.COM ([fe80::ed34:1290:4306:3157%3]) with mapi id 15.20.1686.021; Sat, 9 Mar 2019 14:52:04 +0000 From: Gary Guo Subject: [PATCH v3 1/4] riscv: move flush_icache_{all,mm} to cacheflush.c Thread-Topic: [PATCH v3 1/4] riscv: move flush_icache_{all, mm} to cacheflush.c Thread-Index: AQHU1oeoVwi1k3J/IEOIYrEWiVDqZw== Date: Sat, 9 Mar 2019 14:52:04 +0000 Message-ID: References: In-Reply-To: Accept-Language: en-GB, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-clientproxiedby: LNXP123CA0015.GBRP123.PROD.OUTLOOK.COM (2603:10a6:600:d2::27) To LO2P265MB0847.GBRP265.PROD.OUTLOOK.COM (2603:10a6:600:8c::20) x-mailer: git-send-email 2.17.1 authentication-results: spf=none (sender IP is ) smtp.mailfrom=gary@garyguo.net; x-ms-exchange-messagesentrepresentingtype: 1 x-originating-ip: [2001:630:212:238:3697:f6ff:fe55:55b1] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 79a80851-4df4-4f34-b34b-08d6a49ecb19 x-microsoft-antispam: BCL:0; PCL:0; RULEID:(2390118)(7020095)(4652040)(7021145)(8989299)(4534185)(7022145)(4603075)(4627221)(201702281549075)(8990200)(7048125)(7024125)(7027125)(7023125)(5600127)(711020)(4605104)(2017052603328)(7153060)(7193020); SRVR:LO2P265MB0158; x-ms-traffictypediagnostic: LO2P265MB0158: x-microsoft-exchange-diagnostics: =?iso-8859-1?Q?1; LO2P265MB0158; 23:UTRbM9qcNUMGuKv3aSe5FKmnvFw0EAAmwaNbNz0?= =?iso-8859-1?q?8XuQGomPAM4MuVQ?= =?iso-8859-1?q?t60xV8T5B65k7QuM7w+ILBhHHo1Q0XdWqiIRtm/Yl767g4pmHNxBt2Ve8mmN?= =?iso-8859-1?q?lJ6FAIhhl4tfTnEEqIK9wq1sjeYbvADh939XNEgMQ1+AfhT7d2YeXHlcQqfZ?= =?iso-8859-1?q?LlyyRkEmM5Rvb+sMEkVEJMbFWA+nnUubqtDxb8b83E4WxCMa767bFOelf2xd?= =?iso-8859-1?q?NMxpa5NlqzUCOHn1QK2y3tSF+v+ayaMPi/TVS2PZ86uO9DqBNhzQCEPOjOfa?= =?iso-8859-1?q?o2tKgBbQroeh2vYBcEduGqAOCyHVlXNqtmiIZxqS39k4gerBZTOxz5I6SeRa?= =?iso-8859-1?q?f/hPfXek/dtuu4OLTBDgCqhspn9iN2R1SSZ+feujbFWDmF+5DLMi+IheS4pf?= =?iso-8859-1?q?VgLA5SA+0FRgDX8MDeaLqeEI5pF6FOPHEwLCW54ypnavghby52Y5Zk4PJp6B?= =?iso-8859-1?q?6B8/BfJ5hNpFDnCxVK32kt39oksWMhW02anhy1E8CMkbjZU0z6eC1K7fLkRm?= =?iso-8859-1?q?Gr/GzqNEZS7ybI1ochk3vZWTjQln21NMQA1KS52NSkr+XRO+5xUsPqV1aj8x?= =?iso-8859-1?q?JeZYViEhZ+mPKwPZvkYRr0VCqDndcHrW4EiXwWxdPMTCt4vPkciNRC7b/V08?= =?iso-8859-1?q?0F0V6cLilISUAnA6fqGHeMcJtw/kv1XpP91tz5g776wy/j/jbd3H4JD8VtAN?= =?iso-8859-1?q?Ax3dpKW+/6ZODCEpk2NBUZyVOVZ4LjKWXqAjzSlV2Vo8sKhqTGzQ+8bNRPAA?= =?iso-8859-1?q?ASm7PKarsgchdy3Hzu+HKU3DePYgt2Wm0IPKz6IQc/wN1f0ug/rf/UpGoEua?= =?iso-8859-1?q?6LGRBs6UdNp55nRHDNRWAMkwJ9+rD8ZmWZsDzPQg/tgLMTiMIyTTsCGvWqfL?= =?iso-8859-1?q?H9ADaZ/xhAwuYqGtDcaSbZlUbQZwGOWtTMot8aADRYg4g5BY5ghRx16pRVOe?= =?iso-8859-1?q?oSoCIZgSSv6qpGGqs/0gcd4XBs09kYtkZh0a3sWBLrJ+DUppDd1RPlct2iAp?= =?iso-8859-1?q?SUUwxrwf3RH9643U52s30XY4nLFw5uyGuiOvvzCs56Xz00Pffeyr4b7B1C1h?= =?iso-8859-1?q?ppyIHdKM6bZ2eITqDjG9ivkNsz04p/kI110UgKLEERgFMsGDKjXF8ViQGJeu?= =?iso-8859-1?q?aQ8loTEGJbszfvVN/7m0GwM7AsMb7SQFK2JBucwAVldEs/RDkoNKc2/ICDUO?= =?iso-8859-1?q?cS4Pl1OE4TKwAOhdAZB7Rh1AewfXREBN/g6fDVidUnrsRM3IttJqrGFxut3q?= =?iso-8859-1?q?b4lALYXt+GeqrtMOwmys64wliO?= x-microsoft-antispam-prvs: x-forefront-prvs: 0971922F40 x-forefront-antispam-report: SFV:NSPM; SFS:(10019020)(376002)(39830400003)(346002)(366004)(136003)(396003)(199004)(189003)(305945005)(53936002)(6116002)(8676002)(1671002)(102836004)(86362001)(508600001)(97736004)(52116002)(36756003)(14454004)(76176011)(2906002)(99286004)(386003)(6506007)(7736002)(25786009)(81156014)(81166006)(4326008)(6436002)(6486002)(6512007)(446003)(11346002)(118296001)(2616005)(476003)(105586002)(68736007)(106356001)(71200400001)(71190400001)(186003)(486006)(8936002)(50226002)(5660300002)(14444005)(256004)(109986005)(107886003)(54906003)(59246006)(316002)(46003)(142933001); DIR:OUT; SFP:1102; SCL:1; SRVR:LO2P265MB0158; H:LO2P265MB0847.GBRP265.PROD.OUTLOOK.COM; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; A:1; MX:1; received-spf: None (protection.outlook.com: garyguo.net does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 x-microsoft-antispam-message-info: nl9Wt2rwE/P9aw2JSNxuU0qt1MqguIvDkqBLXtto8YebR+TP5zfPe+zm4xsMu+5mlfkILGDv+ur9+rRugDU4ZF1wof2lwJb2AARlmQamZks6Lte5IyVEbu8CHjii2SShYEptm0vTFKyVkfoTk4hR+RSLHWPhptiHJhMZg+RiGkOOqCTEkhFAzVO1ZgVUqfBqhcfZEyG/9fLXiNr4n265FVvDUlv99AbSqeB9wSLDKZfDrHjY9qk2VWlFdsVZcDtgCABMa2RL53jn9syHisqqmqzZhlBi/4ntLEfFxIr9ODzylKLo6GtOOdoIjePAsaVMIZ8Ha0wj4UeGOC4qV81OF4tvj2rC8qjT1JN+IVIut5e4ev1bYmcWw0P0ytZIGiQCjbWOHhbjjuJmT7BpmJvwrRPHrmE6X63Itck1Jmocg6M= MIME-Version: 1.0 X-OriginatorOrg: garyguo.net X-MS-Exchange-CrossTenant-Network-Message-Id: 79a80851-4df4-4f34-b34b-08d6a49ecb19 X-MS-Exchange-CrossTenant-originalarrivaltime: 09 Mar 2019 14:52:04.0922 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: bbc898ad-b10f-4e10-8552-d9377b823d45 X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-Transport-CrossTenantHeadersStamped: LO2P265MB0158 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20190309_065210_412161_9E3D26A8 X-CRM114-Status: GOOD ( 15.06 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Palmer Dabbelt , Anup Patel , Christoph Hellwig , Atish Patra , Albert Ou , Gary Guo , "linux-riscv@lists.infradead.org" Sender: "linux-riscv" Errors-To: linux-riscv-bounces+patchwork-linux-riscv=patchwork.kernel.org@lists.infradead.org X-Virus-Scanned: ClamAV using ClamSMTP From: Gary Guo Currently, flush_icache_all is macro-expanded into a SBI call, yet no asm/sbi.h is included in asm/cacheflush.h. This could be moved to mm/cacheflush.c instead (SBI call will dominate performance-wise and there is no worry to not have it inlined. Currently, flush_icache_mm stays in kernel/smp.c, which looks like a hack to prevent it from being compiled when CONFIG_SMP=n. It should also be in mm/cacheflush.c. Signed-off-by: Gary Guo --- arch/riscv/include/asm/cacheflush.h | 2 +- arch/riscv/kernel/smp.c | 49 ----------------------- arch/riscv/mm/cacheflush.c | 61 +++++++++++++++++++++++++++++ 3 files changed, 62 insertions(+), 50 deletions(-) diff --git a/arch/riscv/include/asm/cacheflush.h b/arch/riscv/include/asm/cacheflush.h index 8f13074413a7..1f4ba68ab9aa 100644 --- a/arch/riscv/include/asm/cacheflush.h +++ b/arch/riscv/include/asm/cacheflush.h @@ -47,7 +47,7 @@ static inline void flush_dcache_page(struct page *page) #else /* CONFIG_SMP */ -#define flush_icache_all() sbi_remote_fence_i(NULL) +void flush_icache_all(void); void flush_icache_mm(struct mm_struct *mm, bool local); #endif /* CONFIG_SMP */ diff --git a/arch/riscv/kernel/smp.c b/arch/riscv/kernel/smp.c index 0c41d07ec281..17f491e8ed0a 100644 --- a/arch/riscv/kernel/smp.c +++ b/arch/riscv/kernel/smp.c @@ -199,52 +199,3 @@ void smp_send_reschedule(int cpu) send_ipi_message(cpumask_of(cpu), IPI_RESCHEDULE); } -/* - * Performs an icache flush for the given MM context. RISC-V has no direct - * mechanism for instruction cache shoot downs, so instead we send an IPI that - * informs the remote harts they need to flush their local instruction caches. - * To avoid pathologically slow behavior in a common case (a bunch of - * single-hart processes on a many-hart machine, ie 'make -j') we avoid the - * IPIs for harts that are not currently executing a MM context and instead - * schedule a deferred local instruction cache flush to be performed before - * execution resumes on each hart. - */ -void flush_icache_mm(struct mm_struct *mm, bool local) -{ - unsigned int cpu; - cpumask_t others, hmask, *mask; - - preempt_disable(); - - /* Mark every hart's icache as needing a flush for this MM. */ - mask = &mm->context.icache_stale_mask; - cpumask_setall(mask); - /* Flush this hart's I$ now, and mark it as flushed. */ - cpu = smp_processor_id(); - cpumask_clear_cpu(cpu, mask); - local_flush_icache_all(); - - /* - * Flush the I$ of other harts concurrently executing, and mark them as - * flushed. - */ - cpumask_andnot(&others, mm_cpumask(mm), cpumask_of(cpu)); - local |= cpumask_empty(&others); - if (mm != current->active_mm || !local) { - cpumask_clear(&hmask); - riscv_cpuid_to_hartid_mask(&others, &hmask); - sbi_remote_fence_i(hmask.bits); - } else { - /* - * It's assumed that at least one strongly ordered operation is - * performed on this hart between setting a hart's cpumask bit - * and scheduling this MM context on that hart. Sending an SBI - * remote message will do this, but in the case where no - * messages are sent we still need to order this hart's writes - * with flush_icache_deferred(). - */ - smp_mb(); - } - - preempt_enable(); -} diff --git a/arch/riscv/mm/cacheflush.c b/arch/riscv/mm/cacheflush.c index 498c0a0814fe..497b7d07af0c 100644 --- a/arch/riscv/mm/cacheflush.c +++ b/arch/riscv/mm/cacheflush.c @@ -14,6 +14,67 @@ #include #include +#ifdef CONFIG_SMP + +#include + +void flush_icache_all(void) +{ + sbi_remote_fence_i(NULL); +} + +/* + * Performs an icache flush for the given MM context. RISC-V has no direct + * mechanism for instruction cache shoot downs, so instead we send an IPI that + * informs the remote harts they need to flush their local instruction caches. + * To avoid pathologically slow behavior in a common case (a bunch of + * single-hart processes on a many-hart machine, ie 'make -j') we avoid the + * IPIs for harts that are not currently executing a MM context and instead + * schedule a deferred local instruction cache flush to be performed before + * execution resumes on each hart. + */ +void flush_icache_mm(struct mm_struct *mm, bool local) +{ + unsigned int cpu; + cpumask_t others, hmask, *mask; + + preempt_disable(); + + /* Mark every hart's icache as needing a flush for this MM. */ + mask = &mm->context.icache_stale_mask; + cpumask_setall(mask); + /* Flush this hart's I$ now, and mark it as flushed. */ + cpu = smp_processor_id(); + cpumask_clear_cpu(cpu, mask); + local_flush_icache_all(); + + /* + * Flush the I$ of other harts concurrently executing, and mark them as + * flushed. + */ + cpumask_andnot(&others, mm_cpumask(mm), cpumask_of(cpu)); + local |= cpumask_empty(&others); + if (mm != current->active_mm || !local) { + cpumask_clear(&hmask); + riscv_cpuid_to_hartid_mask(&others, &hmask); + sbi_remote_fence_i(hmask.bits); + } else { + /* + * It's assumed that at least one strongly ordered operation is + * performed on this hart between setting a hart's cpumask bit + * and scheduling this MM context on that hart. Sending an SBI + * remote message will do this, but in the case where no + * messages are sent we still need to order this hart's writes + * with flush_icache_deferred(). + */ + smp_mb(); + } + + preempt_enable(); +} + +#endif /* CONFIG_SMP */ + void flush_icache_pte(pte_t pte) { struct page *page = pte_page(pte);