From patchwork Mon Jul 26 14:11:32 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Arnd Bergmann X-Patchwork-Id: 12399587 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6FBC9C4338F for ; Mon, 26 Jul 2021 14:11:51 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id EC8AB60F57 for ; Mon, 26 Jul 2021 14:11:50 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org EC8AB60F57 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=kvack.org Received: by kanga.kvack.org (Postfix) id 959418D0001; Mon, 26 Jul 2021 10:11:50 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 8B0DF8D0002; Mon, 26 Jul 2021 10:11:50 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 6B3738D0001; Mon, 26 Jul 2021 10:11:50 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0237.hostedemail.com [216.40.44.237]) by kanga.kvack.org (Postfix) with ESMTP id 517C16B005D for ; Mon, 26 Jul 2021 10:11:50 -0400 (EDT) Received: from smtpin12.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id E66C7127A4 for ; Mon, 26 Jul 2021 14:11:49 +0000 (UTC) X-FDA: 78404927538.12.C4CCE7F Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf30.hostedemail.com (Postfix) with ESMTP id 92FD5E001800 for ; Mon, 26 Jul 2021 14:11:49 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 2864660F44; Mon, 26 Jul 2021 14:11:47 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1627308708; bh=5kcXJiHE0PQpWAyz9VUZ4OV94qxkV3Y2oDR5o1D2FOk=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=edl2wetoyjeM475gLpRFpus5Lw3iCGQUSLyjx59TAayU1mfJZVjoegxXh7crKj/Fv DwRwg4HYWIj5I71VkADRX3WqOpuJxMCbJF9wvdQ/3HcoITTfnTUPrfhDgCKIcIijFi 1DcvZTIZa1t1cfhx3LFoLesc4BGq76w5KtdQf/UzLnVL5XuuAr7FlYvDLXvHmx70GZ OfzBk+yr2qlGUc4S8Hla4OaI7+ZIuF5yR6gqAQh6tR7OSl1KkFmAS6evg30YOf4E+T MBpnsrMi+Trz1apkV3orFWPMK/wMVeMuI6J9P8UasfJa9G3HpEtTGBFpj2MT6ehu0I qMfyHcos58hyg== From: Arnd Bergmann To: Russell King Cc: Arnd Bergmann , linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-arch@vger.kernel.org, linux-mm@kvack.org, Alexander Viro , Linus Walleij , Christoph Hellwig Subject: [PATCH v5 01/10] mm/maccess: fix unaligned copy_{from,to}_kernel_nofault Date: Mon, 26 Jul 2021 16:11:32 +0200 Message-Id: <20210726141141.2839385-2-arnd@kernel.org> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20210726141141.2839385-1-arnd@kernel.org> References: <20210726141141.2839385-1-arnd@kernel.org> MIME-Version: 1.0 X-Rspamd-Server: rspam01 X-Rspamd-Queue-Id: 92FD5E001800 Authentication-Results: imf30.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=edl2weto; spf=pass (imf30.hostedemail.com: domain of arnd@kernel.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=arnd@kernel.org; dmarc=pass (policy=none) header.from=kernel.org X-Stat-Signature: s6w87qkig6xxnz8a6o67md74s6nrxhmx X-HE-Tag: 1627308709-540979 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Arnd Bergmann On machines such as ARMv5 that trap unaligned accesses, these two functions can be slow when each access needs to be emulated, or they might not work at all. Change them so that each loop is only used when both the src and dst pointers are naturally aligned. Reviewed-by: Christoph Hellwig Reviewed-by: Linus Walleij Signed-off-by: Arnd Bergmann --- mm/maccess.c | 28 ++++++++++++++++++++++------ 1 file changed, 22 insertions(+), 6 deletions(-) diff --git a/mm/maccess.c b/mm/maccess.c index 3bd70405f2d8..d3f1a1f0b1c1 100644 --- a/mm/maccess.c +++ b/mm/maccess.c @@ -24,13 +24,21 @@ bool __weak copy_from_kernel_nofault_allowed(const void *unsafe_src, long copy_from_kernel_nofault(void *dst, const void *src, size_t size) { + unsigned long align = 0; + + if (!IS_ENABLED(CONFIG_HAVE_EFFICIENT_UNALIGNED_ACCESS)) + align = (unsigned long)dst | (unsigned long)src; + if (!copy_from_kernel_nofault_allowed(src, size)) return -ERANGE; pagefault_disable(); - copy_from_kernel_nofault_loop(dst, src, size, u64, Efault); - copy_from_kernel_nofault_loop(dst, src, size, u32, Efault); - copy_from_kernel_nofault_loop(dst, src, size, u16, Efault); + if (!(align & 7)) + copy_from_kernel_nofault_loop(dst, src, size, u64, Efault); + if (!(align & 3)) + copy_from_kernel_nofault_loop(dst, src, size, u32, Efault); + if (!(align & 1)) + copy_from_kernel_nofault_loop(dst, src, size, u16, Efault); copy_from_kernel_nofault_loop(dst, src, size, u8, Efault); pagefault_enable(); return 0; @@ -50,10 +58,18 @@ EXPORT_SYMBOL_GPL(copy_from_kernel_nofault); long copy_to_kernel_nofault(void *dst, const void *src, size_t size) { + unsigned long align = 0; + + if (!IS_ENABLED(CONFIG_HAVE_EFFICIENT_UNALIGNED_ACCESS)) + align = (unsigned long)dst | (unsigned long)src; + pagefault_disable(); - copy_to_kernel_nofault_loop(dst, src, size, u64, Efault); - copy_to_kernel_nofault_loop(dst, src, size, u32, Efault); - copy_to_kernel_nofault_loop(dst, src, size, u16, Efault); + if (!(align & 7)) + copy_to_kernel_nofault_loop(dst, src, size, u64, Efault); + if (!(align & 3)) + copy_to_kernel_nofault_loop(dst, src, size, u32, Efault); + if (!(align & 1)) + copy_to_kernel_nofault_loop(dst, src, size, u16, Efault); copy_to_kernel_nofault_loop(dst, src, size, u8, Efault); pagefault_enable(); return 0;