From patchwork Tue Nov 26 22:44:45 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Marc Gauthier X-Patchwork-Id: 11263123 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id CC16514ED for ; Tue, 26 Nov 2019 22:45:41 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id A8B9D20727 for ; Tue, 26 Nov 2019 22:45:41 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="TatNifhr"; dkim=fail reason="signature verification failed" (1024-bit key) header.d=gstardust.com header.i=@gstardust.com header.b="EJJI6/5I"; dkim=fail reason="signature verification failed" (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b="Z8h069Gb" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org A8B9D20727 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=gstardust.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-riscv-bounces+patchwork-linux-riscv=patchwork.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:MIME-Version:Cc:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id:References: In-Reply-To:Message-Id:Date:Subject:To:From:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=d3gnRMbi9pyXR6T9OZ17YlVt1EpSgJXTYpN+W5a5GnM=; b=TatNifhrjv5FcAX03KTHfGo81G CUZH5maFsvD8KqBBb4DUOlouOQ9bE5xbfzH8jIe1L1XYOQw5rXM3YyBxXJ+46GssOxM5VsKgU/yHP 6mcNR+4kZb8vPsYmOZk4osBIhxmAytOLvRzD3M6wIa50l2dHM2BuGmz5Ony3sjtO4dqlzBY4gytxu W9TbpsHmszpcQ6gWuuyd9VXDl6U1DdOgO6mWgNvNV5PjoJ2OuJGCpzuvALsNaIlbSZjGoirQX9f6Z AlY/Iv8ftvCFwy0LMzI/TuGMtyj5zIRlStFif5x5pyeiNLl7IJY5g/RT8f8LHbCfRW42cJav9Af5o ZQ/bGEOg==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1iZjaX-0005gb-8z; Tue, 26 Nov 2019 22:45:41 +0000 Received: from wout3-smtp.messagingengine.com ([64.147.123.19]) by bombadil.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1iZjaU-0005gB-4E for linux-riscv@lists.infradead.org; Tue, 26 Nov 2019 22:45:39 +0000 Received: from compute7.internal (compute7.nyi.internal [10.202.2.47]) by mailout.west.internal (Postfix) with ESMTP id 4E40185C; Tue, 26 Nov 2019 17:45:37 -0500 (EST) Received: from mailfrontend2 ([10.202.2.163]) by compute7.internal (MEProxy); Tue, 26 Nov 2019 17:45:37 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gstardust.com; h=from:to:cc:subject:date:message-id:in-reply-to:references; s= mesmtp; bh=/67MZLwHQD9tg2Lk/4ZkTp40ytZAb/Bq1guU4IDp2yo=; b=EJJI6 /5IHEvfDKe+GbXBhYndpkMrGyma1nWP34Wjzqbe8HsjbJXBE2qrzTqsYXW3F+rRR wWZpTQdENHjjk7Ykd0H2odCu1gzkkV6L6BFEGhhj/E4TYnqM6mv6ilhAOBfj6Ub4 ARBnWtri91Bhh0ukK7JBejbctu3we5uUuuYJbY= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:date:from:in-reply-to:message-id :references:subject:to:x-me-proxy:x-me-proxy:x-me-sender :x-me-sender:x-sasl-enc; s=fm1; bh=/67MZLwHQD9tg2Lk/4ZkTp40ytZAb /Bq1guU4IDp2yo=; b=Z8h069Gbc0Z7Ej7jm8QyJ5fhkdv5E5AXZb3eCLU9Shwny MRBPrRjyJZaNT1IeMLMnFr36ljWeEASph0U3NahVOMRvJhWIW/gtl57Uj7dYYL7j j+Gr/FlfQZml0d3xQkRQQlHlsEVea5SZvKhUAw4Tgj9dV2B2I6mLTKwCOxQyNFq3 fuplH+zD1FGj+OPTsyEAMEj5DBsRHQkLcv0UQaVKgKPLk3x5yKk6TX0oROsy+ki7 HIPG57UT4WzCseNB+Cb2UUrdLvOF4d3pItcWBxg94Wsj7Y2Qb/nOs5yvE9HEb4pP jM2r6ZVCL9It6MVbA0/vhknmFVAp6zXcz2qLFAYgA== X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedufedrudeigedgtdduucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhephffvufffkffojghfsedttdertdertddtnecuhfhrohhmpeforghrtgcuifgr uhhthhhivghruceotghonhhsuhhlthdqmhhgsehgshhtrghrughushhtrdgtohhmqeenuc fkphepudegvddrheelrdehledrvdelnecurfgrrhgrmhepmhgrihhlfhhrohhmpegtohhn shhulhhtqdhmghesghhsthgrrhguuhhsthdrtghomhenucevlhhushhtvghrufhiiigvpe dt X-ME-Proxy: Received: from SurfBoard.localdomain (unknown [142.59.59.29]) by mail.messagingengine.com (Postfix) with ESMTPA id 769DF3060062; Tue, 26 Nov 2019 17:45:36 -0500 (EST) From: Marc Gauthier To: linux-riscv@lists.infradead.org Subject: [PATCH v2 1/2] riscv: Align shared mappings to SHMLBA Date: Tue, 26 Nov 2019 17:44:45 -0500 Message-Id: <20191126224446.15145-2-consult-mg@gstardust.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20191126224446.15145-1-consult-mg@gstardust.com> References: <20191126224446.15145-1-consult-mg@gstardust.com> X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20191126_144538_210349_30416FCD X-CRM114-Status: GOOD ( 13.11 ) X-Spam-Score: -0.9 (/) X-Spam-Report: SpamAssassin version 3.4.2 on bombadil.infradead.org summary: Content analysis details: (-0.9 points) pts rule name description ---- ---------------------- -------------------------------------------------- -0.7 RCVD_IN_DNSWL_LOW RBL: Sender listed at https://www.dnswl.org/, low trust [64.147.123.19 listed in list.dnswl.org] -0.0 SPF_HELO_PASS SPF: HELO matches SPF record 0.0 SPF_NONE SPF: sender does not publish an SPF Record -0.1 DKIM_VALID_EF Message has a valid DKIM or DK signature from envelope-from domain 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from author's domain X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Marc Gauthier , Palmer Dabbelt , Albert Ou , Paul Walmsley MIME-Version: 1.0 Sender: "linux-riscv" Errors-To: linux-riscv-bounces+patchwork-linux-riscv=patchwork.kernel.org@lists.infradead.org Align shared mappings according to SHMLBA for VIPT cache performance. arch_get_unmapped_area() and arch_get_unmapped_area_topdown() are essentially copies of their default implementations in mm/mmap.c, modified to align the address to SHMLBA for shared mappings, i.e. where MAP_SHARED is specified or a file pointer is provided. Allow MAP_FIXED to request unaligned shared mappings. Although this may potentially reduce performance, very little software does this, as it is not portable across architectures that enforce alignment. Signed-off-by: Marc Gauthier --- arch/riscv/include/asm/pgtable.h | 4 ++ arch/riscv/kernel/sys_riscv.c | 113 +++++++++++++++++++++++++++++++ 2 files changed, 117 insertions(+) diff --git a/arch/riscv/include/asm/pgtable.h b/arch/riscv/include/asm/pgtable.h index d3221017194d..7d1cc47ac5f9 100644 --- a/arch/riscv/include/asm/pgtable.h +++ b/arch/riscv/include/asm/pgtable.h @@ -436,6 +436,10 @@ extern void *dtb_early_va; extern void setup_bootmem(void); extern void paging_init(void); +/* We provide arch_get_unmapped_area to handle VIPT caches efficiently. */ +#define HAVE_ARCH_UNMAPPED_AREA +#define HAVE_ARCH_UNMAPPED_AREA_TOPDOWN + /* * Task size is 0x4000000000 for RV64 or 0x9fc00000 for RV32. * Note that PGDIR_SIZE must evenly divide TASK_SIZE. diff --git a/arch/riscv/kernel/sys_riscv.c b/arch/riscv/kernel/sys_riscv.c index f3619f59d85c..3e739b30b1f7 100644 --- a/arch/riscv/kernel/sys_riscv.c +++ b/arch/riscv/kernel/sys_riscv.c @@ -3,11 +3,15 @@ * Copyright (C) 2012 Regents of the University of California * Copyright (C) 2014 Darius Rad * Copyright (C) 2017 SiFive + * Copyright (C) 2019 Aril Inc */ #include #include #include +#include +#include +#include static long riscv_sys_mmap(unsigned long addr, unsigned long len, unsigned long prot, unsigned long flags, @@ -65,3 +69,112 @@ SYSCALL_DEFINE3(riscv_flush_icache, uintptr_t, start, uintptr_t, end, return 0; } + +/* + * RISC-V requires implementations to function correctly in the presence + * of cache aliasing, regardless of page alignment. It says nothing about + * performance. To ensure healthy performance with commonly implemented + * VIPT caches, the following code avoids most cases of cache aliasing by + * aligning shared mappings such that all mappings of a given physical + * page of an object are at a multiple of SHMLBA bytes from each other. + * + * It does not enforce alignment. Using MAP_FIXED to request unaligned + * shared mappings is not common, and may perform poorly with VIPT caches. + */ +unsigned long +arch_get_unmapped_area(struct file *filp, unsigned long addr, + unsigned long len, unsigned long pgoff, + unsigned long flags) +{ + struct mm_struct *mm = current->mm; + struct vm_area_struct *vma, *prev; + struct vm_unmapped_area_info info; + const unsigned long pgoffset = pgoff << PAGE_SHIFT; + int do_align = (filp || (flags & MAP_SHARED)); + + if (len > TASK_SIZE - mmap_min_addr) + return -ENOMEM; + + if (flags & MAP_FIXED) + return addr; + + if (addr) { + if (do_align) + addr = ALIGN(addr, SHMLBA) + (pgoffset & (SHMLBA - 1)); + else + addr = PAGE_ALIGN(addr); + vma = find_vma_prev(mm, addr, &prev); + if (TASK_SIZE - len >= addr && addr >= mmap_min_addr && + (!vma || addr + len <= vm_start_gap(vma)) && + (!prev || addr >= vm_end_gap(prev))) + return addr; + } + + info.flags = 0; + info.length = len; + info.low_limit = mm->mmap_base; + info.high_limit = TASK_SIZE; + info.align_mask = do_align ? SHMLBA - 1 : 0; + info.align_offset = pgoffset; + return vm_unmapped_area(&info); +} + +/* + * Similar to arch_get_unmapped_area(), but allocating top-down from below the + * stack's low limit (the base). + */ +unsigned long +arch_get_unmapped_area_topdown(struct file *filp, unsigned long addr, + unsigned long len, unsigned long pgoff, + unsigned long flags) +{ + struct vm_area_struct *vma, *prev; + struct mm_struct *mm = current->mm; + struct vm_unmapped_area_info info; + const unsigned long pgoffset = pgoff << PAGE_SHIFT; + int do_align = (filp || (flags & MAP_SHARED)); + + /* requested length too big for entire address space */ + if (len > TASK_SIZE - mmap_min_addr) + return -ENOMEM; + + if (flags & MAP_FIXED) + return addr; + + /* requesting a specific address */ + if (addr) { + if (do_align) + addr = ALIGN(addr, SHMLBA) + (pgoffset & (SHMLBA - 1)); + else + addr = PAGE_ALIGN(addr); + vma = find_vma_prev(mm, addr, &prev); + if (TASK_SIZE - len >= addr && addr >= mmap_min_addr && + (!vma || addr + len <= vm_start_gap(vma)) && + (!prev || addr >= vm_end_gap(prev))) + return addr; + } + + info.flags = VM_UNMAPPED_AREA_TOPDOWN; + info.length = len; + info.low_limit = max(PAGE_SIZE, mmap_min_addr); + info.high_limit = mm->mmap_base; + info.align_mask = do_align ? SHMLBA - 1 : 0; + info.align_offset = pgoffset; + addr = vm_unmapped_area(&info); + + /* + * A failed mmap() very likely causes application failure, + * so fall back to the bottom-up function here. This scenario + * can happen with large stack limits and large mmap() + * allocations. + */ + if (offset_in_page(addr)) { + VM_BUG_ON(addr != -ENOMEM); + info.flags = 0; + info.low_limit = TASK_UNMAPPED_BASE; + info.high_limit = TASK_SIZE; + addr = vm_unmapped_area(&info); + } + + return addr; +} From patchwork Tue Nov 26 22:44:46 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Marc Gauthier X-Patchwork-Id: 11263125 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id BAF05930 for ; Tue, 26 Nov 2019 22:46:21 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 95F0620722 for ; Tue, 26 Nov 2019 22:46:21 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="CL+ZL5qQ"; dkim=fail reason="signature verification failed" (1024-bit key) header.d=gstardust.com header.i=@gstardust.com header.b="fxHivDGa"; dkim=fail reason="signature verification failed" (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b="MB1vGKt+" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 95F0620722 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=gstardust.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-riscv-bounces+patchwork-linux-riscv=patchwork.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:MIME-Version:Cc:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id:References: In-Reply-To:Message-Id:Date:Subject:To:From:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=GY4uv4thifqgEv+JjFBROpmkNi0AEX+GZ0FnSX4u73c=; b=CL+ZL5qQa/XgJSs7lPhYRN2j9b XKB6Nn2RoF2MD2ae3BOwEH9hlQD9mS6aElkB2rGFtCdus0c9dU352GXlEAvOWpp6Xs01WoiXvfNQH PRKbEOI4+I6xeya+9TfOiZNoPIqCnWlx1G1PolOg0Cz7mC7E3b5KfIgrOV/7EENc+Zk9EONTAsms8 VGxKjFAk3G7TOhsZMfLKz79lvADfNg34wh2Jbxj6iBiG6bSobwr+9iz40BRNNVcQvchjGvzRkOnQS efzXBSBAHIeOnHh1nsX02aYxYGrFvxI47hzp44d3cBJbk2Ql4oQ1XAWdyZULcuH0/Ixbk6BZs7PRL pnPqLalQ==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1iZjbA-0005mO-7X; Tue, 26 Nov 2019 22:46:20 +0000 Received: from wout3-smtp.messagingengine.com ([64.147.123.19]) by bombadil.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1iZjb6-0005m0-UF for linux-riscv@lists.infradead.org; Tue, 26 Nov 2019 22:46:18 +0000 Received: from compute7.internal (compute7.nyi.internal [10.202.2.47]) by mailout.west.internal (Postfix) with ESMTP id DA5E5769; Tue, 26 Nov 2019 17:46:15 -0500 (EST) Received: from mailfrontend2 ([10.202.2.163]) by compute7.internal (MEProxy); Tue, 26 Nov 2019 17:46:16 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gstardust.com; h=from:to:cc:subject:date:message-id:in-reply-to:references; s= mesmtp; bh=GR3YREDwUxTesxmdecPn1Ziirn6iQvdiG1EtQR1iGss=; b=fxHiv DGaXRqokvFf60X0gYboLdG7MszxKLt1MbEd2PfSNhg8miAgxcT+1HJ6LEz5qDyLk 1HxJf+vuZtpeRXQYADJKqfDZLuDaqI7XFPYEKLuE1prQ3e2Q3Rhh6YLklb6wtO9o iJYrlhSxo/xo+34npfwg4VDHv2cBkPBQS+Yf0Q= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:date:from:in-reply-to:message-id :references:subject:to:x-me-proxy:x-me-proxy:x-me-sender :x-me-sender:x-sasl-enc; s=fm1; bh=GR3YREDwUxTesxmdecPn1Ziirn6iQ vdiG1EtQR1iGss=; b=MB1vGKt+0BGzQjwOSMHB9rxpNdY9DlF6QmXmoagNkQ0Ae UUnOBoYrpHuc6aQ2Ir8kIK4n3XP7cu8sgTSeWva7dF84CLNC8hvoFkd30ir+lBsD IiBfl8gDKybUDEYGF3frQPhQBSgxfcBiKJTLHNK00saap7kFsn3GieZdvc/ziveE lm+ZgeBKQPtVlQsGnTZ/vP8GGDmOW/C708hbe9mRGLQwAx3Lg+AA6welvaYLnpOW B50RY1xw5K69YDTRCDG8KyBFxlnQUovrx8JLCDxdWMpv2pYt/Etj1+/T9ScRg8b8 V0m2m3QxsuiOpjCz5SAYKUJCB0fj4f9qOlGc9w9YA== X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedufedrudeigedgtdduucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhephffvufffkffojghfsedttdertdertddtnecuhfhrohhmpeforghrtgcuifgr uhhthhhivghruceotghonhhsuhhlthdqmhhgsehgshhtrghrughushhtrdgtohhmqeenuc fkphepudegvddrheelrdehledrvdelnecurfgrrhgrmhepmhgrihhlfhhrohhmpegtohhn shhulhhtqdhmghesghhsthgrrhguuhhsthdrtghomhenucevlhhushhtvghrufhiiigvpe du X-ME-Proxy: Received: from SurfBoard.localdomain (unknown [142.59.59.29]) by mail.messagingengine.com (Postfix) with ESMTPA id 10DE93060060; Tue, 26 Nov 2019 17:46:15 -0500 (EST) From: Marc Gauthier To: linux-riscv@lists.infradead.org Subject: [PATCH v2 2/2] riscv: Set SHMLBA according to cache geometry Date: Tue, 26 Nov 2019 17:44:46 -0500 Message-Id: <20191126224446.15145-3-consult-mg@gstardust.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20191126224446.15145-1-consult-mg@gstardust.com> References: <20191126224446.15145-1-consult-mg@gstardust.com> X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20191126_144617_022074_49880A9C X-CRM114-Status: GOOD ( 15.16 ) X-Spam-Score: -0.9 (/) X-Spam-Report: SpamAssassin version 3.4.2 on bombadil.infradead.org summary: Content analysis details: (-0.9 points) pts rule name description ---- ---------------------- -------------------------------------------------- -0.7 RCVD_IN_DNSWL_LOW RBL: Sender listed at https://www.dnswl.org/, low trust [64.147.123.19 listed in list.dnswl.org] -0.0 SPF_HELO_PASS SPF: HELO matches SPF record 0.0 SPF_NONE SPF: sender does not publish an SPF Record -0.1 DKIM_VALID_EF Message has a valid DKIM or DK signature from envelope-from domain 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from author's domain X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Marc Gauthier , Palmer Dabbelt , Albert Ou , Paul Walmsley MIME-Version: 1.0 Sender: "linux-riscv" Errors-To: linux-riscv-bounces+patchwork-linux-riscv=patchwork.kernel.org@lists.infradead.org Set SHMLBA to the maximum cache "span" (line size * number of sets) of all CPU L1 instruction and data caches (L2 and up are rarely VIPT). This avoids VIPT cache aliasing with minimal alignment constraints. If the device tree does not provide cache parameters, use a conservative default of 16 KB: only large enough to avoid aliasing in most VIPT caches. Signed-off-by: Marc Gauthier --- arch/riscv/include/asm/Kbuild | 1 - arch/riscv/include/asm/shmparam.h | 12 +++++++ arch/riscv/kernel/cacheinfo.c | 52 +++++++++++++++++++++++++++++++ 3 files changed, 64 insertions(+), 1 deletion(-) create mode 100644 arch/riscv/include/asm/shmparam.h diff --git a/arch/riscv/include/asm/Kbuild b/arch/riscv/include/asm/Kbuild index 16970f246860..3905765807af 100644 --- a/arch/riscv/include/asm/Kbuild +++ b/arch/riscv/include/asm/Kbuild @@ -27,7 +27,6 @@ generic-y += percpu.h generic-y += preempt.h generic-y += sections.h generic-y += serial.h -generic-y += shmparam.h generic-y += topology.h generic-y += trace_clock.h generic-y += unaligned.h diff --git a/arch/riscv/include/asm/shmparam.h b/arch/riscv/include/asm/shmparam.h new file mode 100644 index 000000000000..9b6a98153648 --- /dev/null +++ b/arch/riscv/include/asm/shmparam.h @@ -0,0 +1,12 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#ifndef _ASM_RISCV_SHMPARAM_H +#define _ASM_RISCV_SHMPARAM_H + +/* + * Minimum alignment of shared memory segments as a function of cache geometry. + */ +#define SHMLBA arch_shmlba() + +long arch_shmlba(void); + +#endif /* _ASM_RISCV_SHMPARAM_H */ diff --git a/arch/riscv/kernel/cacheinfo.c b/arch/riscv/kernel/cacheinfo.c index 4c90c07d8c39..1bc7df8577d6 100644 --- a/arch/riscv/kernel/cacheinfo.c +++ b/arch/riscv/kernel/cacheinfo.c @@ -1,12 +1,61 @@ // SPDX-License-Identifier: GPL-2.0-only /* * Copyright (C) 2017 SiFive + * Copyright (C) 2019 Aril Inc */ #include #include #include #include +#include + +static long shmlba; + + +/* + * Assuming cache size = line size * #sets * N for N-way associative caches, + * return the max cache "span" == (line size * #sets) == (cache size / N) + * across all L1 caches, or 0 if cache parameters are not available. + * VIPT caches with span > min page size are susceptible to aliasing. + */ +static long get_max_cache_span(void) +{ + struct cpu_cacheinfo *this_cpu_ci; + struct cacheinfo *this_leaf; + long span, max_span = 0; + int cpu, leaf; + + for_each_possible_cpu(cpu) { + this_cpu_ci = get_cpu_cacheinfo(cpu); + this_leaf = this_cpu_ci->info_list; + for (leaf = 0; leaf < this_cpu_ci->num_leaves; leaf++) { + if (this_leaf->level > 1) + break; + span = this_leaf->coherency_line_size * + this_leaf->number_of_sets; + if (span > max_span) + max_span = span; + this_leaf++; + } + } + return max_span; +} + +/* + * Align shared mappings to the maximum cache "span" to avoid aliasing + * in VIPT caches, for performance. + * The returned SHMLBA value is always a power-of-two multiple of PAGE_SIZE. + */ +long arch_shmlba(void) +{ + if (shmlba == 0) { + long max_span = get_max_cache_span(); + + shmlba = max_span ? PAGE_ALIGN(max_span) : 4 * PAGE_SIZE; + } + return shmlba; +} static void ci_leaf_init(struct cacheinfo *this_leaf, struct device_node *node, @@ -93,6 +142,9 @@ static int __populate_cache_leaves(unsigned int cpu) } of_node_put(np); + /* Force recalculating SHMLBA if cache parameters are updated. */ + shmlba = 0; + return 0; }