From patchwork Fri Aug 30 15:57:05 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christophe Leroy X-Patchwork-Id: 13785297 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3C163CA1008 for ; Fri, 30 Aug 2024 15:57:27 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id CD65C6B0188; Fri, 30 Aug 2024 11:57:26 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id C86926B0189; Fri, 30 Aug 2024 11:57:26 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B4D736B018A; Fri, 30 Aug 2024 11:57:26 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 944C56B0188 for ; Fri, 30 Aug 2024 11:57:26 -0400 (EDT) Received: from smtpin25.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 4ABB2C0636 for ; Fri, 30 Aug 2024 15:57:26 +0000 (UTC) X-FDA: 82509366492.25.2E7FC8E Received: from pegase2.c-s.fr (pegase2.c-s.fr [93.17.235.10]) by imf06.hostedemail.com (Postfix) with ESMTP id 1C037180025 for ; Fri, 30 Aug 2024 15:57:23 +0000 (UTC) Authentication-Results: imf06.hostedemail.com; dkim=none; spf=pass (imf06.hostedemail.com: domain of christophe.leroy@csgroup.eu designates 93.17.235.10 as permitted sender) smtp.mailfrom=christophe.leroy@csgroup.eu; dmarc=pass (policy=quarantine) header.from=csgroup.eu ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1725033399; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ayUK2pr8smF7SVh+g5UUyIWNs282xIZKaVO//GfrcKQ=; b=E9rpvSvZWkmFSrtK+x+YT9i1giPgC7MYtPiam5f83sP3dBe9N2HahMnBlPmxBcJBMWqGLD z+tWe+k1yB2n1iAT1SVWXPkQmzitx9FTfMvOHXM1YF3o+8tCho+d6AweYP/zJ6vSEXUH8Y 2k5dSqwxopkEZMX8gkOpbwrkqyGL3Hc= ARC-Authentication-Results: i=1; imf06.hostedemail.com; dkim=none; spf=pass (imf06.hostedemail.com: domain of christophe.leroy@csgroup.eu designates 93.17.235.10 as permitted sender) smtp.mailfrom=christophe.leroy@csgroup.eu; dmarc=pass (policy=quarantine) header.from=csgroup.eu ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1725033399; a=rsa-sha256; cv=none; b=sPbs0vPrDzNiENGJDmT/7hnq/hNRbTNSl49zO4nhIz8KCZaCiw9t/KGcfcR/BqLO8AD+2U GE9nhExv98ZRU13/heC/qsy0eA40PEeYAqsN+jq3BX50eU90ZP1ZFYxkVNqFtSh49dpTKW IvlAniCJ9HrId2atAQjN3IHoY2QC12I= Received: from localhost (mailhub3.si.c-s.fr [172.26.127.67]) by localhost (Postfix) with ESMTP id 4WwN7y1Dknz9sSC; Fri, 30 Aug 2024 17:57:18 +0200 (CEST) X-Virus-Scanned: amavisd-new at c-s.fr Received: from pegase2.c-s.fr ([172.26.127.65]) by localhost (pegase2.c-s.fr [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id lm1E0y2khmaS; Fri, 30 Aug 2024 17:57:18 +0200 (CEST) Received: from messagerie.si.c-s.fr (messagerie.si.c-s.fr [192.168.25.192]) by pegase2.c-s.fr (Postfix) with ESMTP id 4WwN7x6THCz9sS7; Fri, 30 Aug 2024 17:57:17 +0200 (CEST) Received: from localhost (localhost [127.0.0.1]) by messagerie.si.c-s.fr (Postfix) with ESMTP id CC6B38B799; Fri, 30 Aug 2024 17:57:17 +0200 (CEST) X-Virus-Scanned: amavisd-new at c-s.fr Received: from messagerie.si.c-s.fr ([127.0.0.1]) by localhost (messagerie.si.c-s.fr [127.0.0.1]) (amavisd-new, port 10023) with ESMTP id yyiLWwKmUyql; Fri, 30 Aug 2024 17:57:17 +0200 (CEST) Received: from PO20335.idsi0.si.c-s.fr (unknown [192.168.234.133]) by messagerie.si.c-s.fr (Postfix) with ESMTP id D53208B794; Fri, 30 Aug 2024 17:57:16 +0200 (CEST) From: Christophe Leroy To: Andrew Morton , Steven Rostedt , Masami Hiramatsu , Mathieu Desnoyers , Michael Ellerman , Nicholas Piggin , Naveen N Rao , Nathan Chancellor , Nick Desaulniers , Bill Wendling , Justin Stitt , Shuah Khan , "Jason A . Donenfeld" Cc: Christophe Leroy , linux-kernel@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-kselftest@vger.kernel.org, llvm@lists.linux.dev, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, linux-trace-kernel@vger.kernel.org, Adhemerval Zanella , Xi Ruoyao Subject: [PATCH v3 1/5] mm: Define VM_DROPPABLE for powerpc/32 Date: Fri, 30 Aug 2024 17:57:05 +0200 Message-ID: X-Mailer: git-send-email 2.44.0 In-Reply-To: References: MIME-Version: 1.0 X-Developer-Signature: v=1; a=ed25519-sha256; t=1725033425; l=2812; i=christophe.leroy@csgroup.eu; s=20211009; h=from:subject:message-id; bh=/uJanEimm/Fnr8lBP4NFr4vOoV+ijz2MaIjmXbR52ek=; b=1tf5k5ovFZ5zvOOs0ROZ1zJin8FeyXW/XMwB35JKUkbdy50HnTLPUu5Rthe/j4vD2d3CUhq9h mkc1aRp0CyZAmDFhvAq0sK2FUPIXeJUTM5Pd+7zIiWOaabJTjWYkZRQ X-Developer-Key: i=christophe.leroy@csgroup.eu; a=ed25519; pk=HIzTzUj91asvincQGOFx6+ZF5AoUuP9GdOtQChs7Mm0= X-Rspamd-Server: rspam03 X-Rspam-User: X-Rspamd-Queue-Id: 1C037180025 X-Stat-Signature: ss6dtu9y6pw8tdnh453x7d46tt39uup1 X-HE-Tag: 1725033443-305528 X-HE-Meta: U2FsdGVkX1+kTi165Y1aPY68Tg47mW2a29hbdmkrUpMKbNQrY8FsGc2QwBnVLne0xJx2lHAwO/HhJagjhe04x7Z3ilLzq8szoNJcgfkPe05R/SxovLckQ+NjviGvlVZgybkKcH3NXXZGd1kVd1aEIS+jJ065cLn/6bRcRIgpdsh1WUhVnnl7MN80S9LHwx/7NpX6Cq7PThwjj86saRRHiWd3OrqNpvd+GSMAZZ97LxymH88vUuqRhtNR9Spz9dx/5FRE5hgqlsS/6kQW5bj5HRnLMgE3ye8bkl0hCyCm/d/VLdYsnuw6S+o7R7nb9H0VdCLpYnhVQ2+mX2OEgo1nW/ntooDDnLE1RXklYJMrjVDtwkg/awVLNV5e/+gCYO4Rx2DIuXQyOYltyhBKPZvEJYLcLwTy4YB4H3AVZCBXaGANGXPczZJrGFeGsyEJ6Fc2MRpocLk5BjwCDU2LLQEtrGTMNssqPcy3dGWmzhtjgN5Meu3WBZhh4/oYrbWkRCKRmqeTM6bWDLnTV45tMOIMWF5xyWUcM9FyXdhtfRbtt4SwmfwhGC0w1UUJZ/I/tzwU/F4wyFxZwLIR0vaU6qPabf7uup4dEWqL9FAY9uqEmi88cdpiRf2CRNH1tlokYyjysxKpQ+58n6xYv5EJHzrTBQASPas82uq1oeeAgJUmDE7SVVxMXsgkDWVcTZ3VtYJDt81XiVxvQHNu3QRne6lAW6JaSgWQ8QSi0R1kckDAuHp4kIxM6NK9fX8Py4lROUASsj74ktZn91znCAGRTyTGNKFk/BITXhcuqoF4Zj4kfCLagxHdQwBG35nwS01n0bGSUdo5OXxFWDacejl+RMuWyZLZEmSTsPGqFFoszny6ab1ILzDVEApRT+siPN+Vc28b9Av5EQUybPQgcLwHk09eTWnBLHB37bf84qa2OF+JcBEY19cI0UsiXCyGePi1IQ065nt9pd7djUmJQxBOQqr QCL8Si3A O3vANY/USr7+HSxiKQdUVaN7EkMKjPytZrvjp68BCacmnR1HxdCRADS4VTlPQC6FkVpJdRyQ1JDQ38kCpZcJJ0gTznasK2Rtgf95TgMerklNTMIS09Vg5nzAr9VD9l2fX4ySN8MKAK8ysqgiua7wT5lBsxJXlWx7OK+F2BuWEws1JyzQ= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Commit 9651fcedf7b9 ("mm: add MAP_DROPPABLE for designating always lazily freeable mappings") only adds VM_DROPPABLE for 64 bits architectures. In order to also use the getrandom vDSO implementation on powerpc/32, use VM_ARCH_1 for VM_DROPPABLE on powerpc/32. This is possible because VM_ARCH_1 is used for VM_SAO on powerpc and VM_SAO is only for powerpc/64. Signed-off-by: Christophe Leroy --- v3: Fixed build failure reported by robots. --- fs/proc/task_mmu.c | 4 +++- include/linux/mm.h | 4 +++- include/trace/events/mmflags.h | 4 ++-- 3 files changed, 8 insertions(+), 4 deletions(-) diff --git a/fs/proc/task_mmu.c b/fs/proc/task_mmu.c index 5f171ad7b436..3a07e13e2f81 100644 --- a/fs/proc/task_mmu.c +++ b/fs/proc/task_mmu.c @@ -987,8 +987,10 @@ static void show_smap_vma_flags(struct seq_file *m, struct vm_area_struct *vma) #ifdef CONFIG_X86_USER_SHADOW_STACK [ilog2(VM_SHADOW_STACK)] = "ss", #endif -#ifdef CONFIG_64BIT +#if defined(CONFIG_64BIT) || defined(CONFIG_PPC32) [ilog2(VM_DROPPABLE)] = "dp", +#endif +#ifdef CONFIG_64BIT [ilog2(VM_SEALED)] = "sl", #endif }; diff --git a/include/linux/mm.h b/include/linux/mm.h index 6549d0979b28..028847f39442 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -359,7 +359,7 @@ extern unsigned int kobjsize(const void *objp); #if defined(CONFIG_X86) # define VM_PAT VM_ARCH_1 /* PAT reserves whole VMA at once (x86) */ -#elif defined(CONFIG_PPC) +#elif defined(CONFIG_PPC64) # define VM_SAO VM_ARCH_1 /* Strong Access Ordering (powerpc) */ #elif defined(CONFIG_PARISC) # define VM_GROWSUP VM_ARCH_1 @@ -409,6 +409,8 @@ extern unsigned int kobjsize(const void *objp); #ifdef CONFIG_64BIT #define VM_DROPPABLE_BIT 40 #define VM_DROPPABLE BIT(VM_DROPPABLE_BIT) +#elif defined(CONFIG_PPC32) +#define VM_DROPPABLE VM_ARCH_1 #else #define VM_DROPPABLE VM_NONE #endif diff --git a/include/trace/events/mmflags.h b/include/trace/events/mmflags.h index b63d211bd141..37265977d524 100644 --- a/include/trace/events/mmflags.h +++ b/include/trace/events/mmflags.h @@ -143,7 +143,7 @@ IF_HAVE_PG_ARCH_X(arch_3) #if defined(CONFIG_X86) #define __VM_ARCH_SPECIFIC_1 {VM_PAT, "pat" } -#elif defined(CONFIG_PPC) +#elif defined(CONFIG_PPC64) #define __VM_ARCH_SPECIFIC_1 {VM_SAO, "sao" } #elif defined(CONFIG_PARISC) #define __VM_ARCH_SPECIFIC_1 {VM_GROWSUP, "growsup" } @@ -165,7 +165,7 @@ IF_HAVE_PG_ARCH_X(arch_3) # define IF_HAVE_UFFD_MINOR(flag, name) #endif -#ifdef CONFIG_64BIT +#if defined(CONFIG_64BIT) || defined(CONFIG_PPC32) # define IF_HAVE_VM_DROPPABLE(flag, name) {flag, name}, #else # define IF_HAVE_VM_DROPPABLE(flag, name) From patchwork Fri Aug 30 15:57:06 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christophe Leroy X-Patchwork-Id: 13785298 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id E63B5CA1005 for ; Fri, 30 Aug 2024 15:57:31 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 7F4476B018A; Fri, 30 Aug 2024 11:57:31 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 7A4346B018B; Fri, 30 Aug 2024 11:57:31 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 644D46B018C; Fri, 30 Aug 2024 11:57:31 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 454036B018A for ; Fri, 30 Aug 2024 11:57:31 -0400 (EDT) Received: from smtpin14.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id EBF921A1ADA for ; Fri, 30 Aug 2024 15:57:30 +0000 (UTC) X-FDA: 82509366660.14.1F7183D Received: from pegase2.c-s.fr (pegase2.c-s.fr [93.17.235.10]) by imf11.hostedemail.com (Postfix) with ESMTP id A126C40025 for ; Fri, 30 Aug 2024 15:57:28 +0000 (UTC) Authentication-Results: imf11.hostedemail.com; dkim=none; spf=pass (imf11.hostedemail.com: domain of christophe.leroy@csgroup.eu designates 93.17.235.10 as permitted sender) smtp.mailfrom=christophe.leroy@csgroup.eu; dmarc=pass (policy=quarantine) header.from=csgroup.eu ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1725033404; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=MwCQPr+DkzFiDHzjMK222UDVKoya0EMyZrgWc4ytvxg=; b=bAaAJer/SOxoXeEjksg12mvdPyk2wcVeO7+mxAYZB9RQkoeKfxU4V/4H6lmzaDxCnWQbgL RjMwtkZqjzrxjxEkir2ygO2oaXihcRdVphcBdnv7WrMKiiiuxcbQMBZRXoElDS7rJNfxmL Rk15MicpcDJ69NwlRb9ZXGANhlHWghI= ARC-Authentication-Results: i=1; imf11.hostedemail.com; dkim=none; spf=pass (imf11.hostedemail.com: domain of christophe.leroy@csgroup.eu designates 93.17.235.10 as permitted sender) smtp.mailfrom=christophe.leroy@csgroup.eu; dmarc=pass (policy=quarantine) header.from=csgroup.eu ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1725033404; a=rsa-sha256; cv=none; b=PKMU74nSpzt344L0ZTGc7kNQu1B6Q2Lw6JrMHXrJg/zldWRuDJKcvuaZV6I6Tgm+F/FUrc lrYhKk53VCV16NPAxOkzMZyZbgzlqjkCHXZXdQYDD2S1yzppytdGxJ6LMBAh4EEnMatYMD HNbEiX2vawbm3uozPIGjOdAG/uTRHFc= Received: from localhost (mailhub3.si.c-s.fr [172.26.127.67]) by localhost (Postfix) with ESMTP id 4WwN7z3h8Qz9sSH; Fri, 30 Aug 2024 17:57:19 +0200 (CEST) X-Virus-Scanned: amavisd-new at c-s.fr Received: from pegase2.c-s.fr ([172.26.127.65]) by localhost (pegase2.c-s.fr [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id HMgbftnZ8nbN; Fri, 30 Aug 2024 17:57:19 +0200 (CEST) Received: from messagerie.si.c-s.fr (messagerie.si.c-s.fr [192.168.25.192]) by pegase2.c-s.fr (Postfix) with ESMTP id 4WwN7y61GNz9sS7; Fri, 30 Aug 2024 17:57:18 +0200 (CEST) Received: from localhost (localhost [127.0.0.1]) by messagerie.si.c-s.fr (Postfix) with ESMTP id BA8D48B799; Fri, 30 Aug 2024 17:57:18 +0200 (CEST) X-Virus-Scanned: amavisd-new at c-s.fr Received: from messagerie.si.c-s.fr ([127.0.0.1]) by localhost (messagerie.si.c-s.fr [127.0.0.1]) (amavisd-new, port 10023) with ESMTP id Nr-gvXNmG0Cw; Fri, 30 Aug 2024 17:57:18 +0200 (CEST) Received: from PO20335.idsi0.si.c-s.fr (unknown [192.168.234.133]) by messagerie.si.c-s.fr (Postfix) with ESMTP id C44E88B764; Fri, 30 Aug 2024 17:57:17 +0200 (CEST) From: Christophe Leroy To: Andrew Morton , Steven Rostedt , Masami Hiramatsu , Mathieu Desnoyers , Michael Ellerman , Nicholas Piggin , Naveen N Rao , Nathan Chancellor , Nick Desaulniers , Bill Wendling , Justin Stitt , Shuah Khan , "Jason A . Donenfeld" Cc: Christophe Leroy , linux-kernel@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-kselftest@vger.kernel.org, llvm@lists.linux.dev, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, linux-trace-kernel@vger.kernel.org, Adhemerval Zanella , Xi Ruoyao Subject: [PATCH v3 2/5] powerpc/vdso32: Add crtsavres Date: Fri, 30 Aug 2024 17:57:06 +0200 Message-ID: <8ee8ae8dd7dcd565a46926f164d549b21fcb419c.1725031952.git.christophe.leroy@csgroup.eu> X-Mailer: git-send-email 2.44.0 In-Reply-To: References: MIME-Version: 1.0 X-Developer-Signature: v=1; a=ed25519-sha256; t=1725033425; l=2796; i=christophe.leroy@csgroup.eu; s=20211009; h=from:subject:message-id; bh=utdv6V/SvZxVA50hOk7t/lkGWDvSE1q/kJYchx6i5SU=; b=42jpAPflsOn3qVlMzH5/Gqfu25AjaEU/oJWoPfoeM5YZD4jpO4qqtwcz3emXwiSz6I53fTUGL itD2EsCfXO+CDXfDWiCxQ1kXwwjwpt5ONIwLyuzPLdu4/6bUeFIyJ3Z X-Developer-Key: i=christophe.leroy@csgroup.eu; a=ed25519; pk=HIzTzUj91asvincQGOFx6+ZF5AoUuP9GdOtQChs7Mm0= X-Rspamd-Server: rspam03 X-Rspam-User: X-Rspamd-Queue-Id: A126C40025 X-Stat-Signature: wybts7cwbwippi7e9b6d48trsj5mamgh X-HE-Tag: 1725033448-744870 X-HE-Meta: U2FsdGVkX18bKRYoi3ihHsgkMAI7EP00A0kowfnods/fmgMMiBEjIyWgxHRZg1d5CoValky6FppRiel5MZoIhd45irk5hbknQ0latK9hsA/8V9yfHuLZT9okILK8x5eBkFxH+Nqt1E1DV24PaKO2t7FlnSF2hWDdzU/nKPFCx4S+EdPnJzXiAoB5WNjQ+szHJ9lLiBbeZt/h9S4hS0I6OSE+dx/yEqVtmxCleQEi+CAnYXVTzFe2qoDjEW3H6zm8uRMt7Ulyog6iCTOcld0DfszTwfhpq4n/iXdfpS8yAun0xfZSmQquAVVdlKJZ3ONej3bAmRuSAdqC3ReUDeTFJH2Gsv0SuTFnQ18mKaJ21FirfnC4RdPuY09TCWoXoqLuAXfB1/J35tWhm7aaSG3xNhRlgyLqQ5bVmQkqFuxPWDqxzqXZiN0VRcnX1eBtCfa4oUx9Geueb4cNP/tCbH9TAhLK+jUvlsX9OOolJTYYToCLTxil7cvA3eNlUFuqh86anm1xzB9HcErxpv0XfDDsiVbWeIFXJFvX0DRCR8P548k36pYF5jF5bwrkl0CGwe3jooUM+mTNTjZ1eYfX00RU+3tCt4z6/DDVbmXf9oYKm/KonYGWNDt8H5HAOAOSeXyD0K7FSV/HJ54CTGJ5gxCwGpsU9A5Gt9t7TpHbK0s9PyrhBnLos9cNuRIWuSx0TRUFgFzfG5U+8a1PonH4SRTTsp3fnY6QjjUYctXfil8hfaWpNCmAatj9S5BkcaDz8AT2V1Rs/XBIdzwe9K7YQ2zbRB3G0UHkrw5qfJ48lscfB+amw+HR60zGERF1EG7tyUl72wj5CW94w8WoxFj9fu4Bz8iEu+RH5u9ZLBvbHOMaQT0GRI8xZLyZP5yK443fHWi7XGQZGzhNhxzRS/14erJ72o0dkZpbmxaZuXxhpfIunEiLH5aVSiG8beBvXCzRu5iOU8Yrc5ZMHsTrtxuvEZq cLcUA977 8onyrNfaPnxHCtQyyqtgtvK8Jl6ZArVMKEsEijL1HJlKf++Rb/UCeeJj7tMh0tVl0bESQXxAqknubOS16JNjHmT4o0R0xCeiIv3A/22b8XUwCuUYnpUafq0IqpVAF/gBYheIK X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Commit 08c18b63d965 ("powerpc/vdso32: Add missing _restgpr_31_x to fix build failure") added _restgpr_31_x to the vdso for gettimeofday, but the work on getrandom shows that we will need more of those functions. Remove _restgpr_31_x and link in crtsavres.o so that we get all save/restore functions when optimising the kernel for size. Signed-off-by: Christophe Leroy --- arch/powerpc/kernel/vdso/Makefile | 5 ++++- arch/powerpc/kernel/vdso/gettimeofday.S | 13 ------------- 2 files changed, 4 insertions(+), 14 deletions(-) diff --git a/arch/powerpc/kernel/vdso/Makefile b/arch/powerpc/kernel/vdso/Makefile index 1425b6edc66b..c07a425b8f78 100644 --- a/arch/powerpc/kernel/vdso/Makefile +++ b/arch/powerpc/kernel/vdso/Makefile @@ -43,6 +43,7 @@ else endif targets := $(obj-vdso32) vdso32.so.dbg vgettimeofday-32.o +targets += crtsavres-32.o obj-vdso32 := $(addprefix $(obj)/, $(obj-vdso32)) targets += $(obj-vdso64) vdso64.so.dbg vgettimeofday-64.o obj-vdso64 := $(addprefix $(obj)/, $(obj-vdso64)) @@ -68,7 +69,7 @@ targets += vdso64.lds CPPFLAGS_vdso64.lds += -P -C # link rule for the .so file, .lds has to be first -$(obj)/vdso32.so.dbg: $(obj)/vdso32.lds $(obj-vdso32) $(obj)/vgettimeofday-32.o FORCE +$(obj)/vdso32.so.dbg: $(obj)/vdso32.lds $(obj-vdso32) $(obj)/vgettimeofday-32.o $(obj)/crtsavres-32.o FORCE $(call if_changed,vdso32ld_and_check) $(obj)/vdso64.so.dbg: $(obj)/vdso64.lds $(obj-vdso64) $(obj)/vgettimeofday-64.o FORCE $(call if_changed,vdso64ld_and_check) @@ -76,6 +77,8 @@ $(obj)/vdso64.so.dbg: $(obj)/vdso64.lds $(obj-vdso64) $(obj)/vgettimeofday-64.o # assembly rules for the .S files $(obj-vdso32): %-32.o: %.S FORCE $(call if_changed_dep,vdso32as) +$(obj)/crtsavres-32.o: %-32.o: $(srctree)/arch/powerpc/lib/crtsavres.S FORCE + $(call if_changed_dep,vdso32as) $(obj)/vgettimeofday-32.o: %-32.o: %.c FORCE $(call if_changed_dep,vdso32cc) $(obj-vdso64): %-64.o: %.S FORCE diff --git a/arch/powerpc/kernel/vdso/gettimeofday.S b/arch/powerpc/kernel/vdso/gettimeofday.S index 48fc6658053a..67254ac9c8bb 100644 --- a/arch/powerpc/kernel/vdso/gettimeofday.S +++ b/arch/powerpc/kernel/vdso/gettimeofday.S @@ -118,16 +118,3 @@ V_FUNCTION_END(__kernel_clock_getres) V_FUNCTION_BEGIN(__kernel_time) cvdso_call __c_kernel_time call_time=1 V_FUNCTION_END(__kernel_time) - -/* Routines for restoring integer registers, called by the compiler. */ -/* Called with r11 pointing to the stack header word of the caller of the */ -/* function, just beyond the end of the integer restore area. */ -#ifndef __powerpc64__ -_GLOBAL(_restgpr_31_x) -_GLOBAL(_rest32gpr_31_x) - lwz r0,4(r11) - lwz r31,-4(r11) - mtlr r0 - mr r1,r11 - blr -#endif From patchwork Fri Aug 30 15:57:07 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christophe Leroy X-Patchwork-Id: 13785299 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 94137CA1007 for ; Fri, 30 Aug 2024 15:57:36 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 116176B018C; Fri, 30 Aug 2024 11:57:36 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 0C68E6B018D; Fri, 30 Aug 2024 11:57:36 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id ECFCA6B018E; Fri, 30 Aug 2024 11:57:35 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id D07516B018C for ; Fri, 30 Aug 2024 11:57:35 -0400 (EDT) Received: from smtpin18.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 8D8161A1696 for ; Fri, 30 Aug 2024 15:57:35 +0000 (UTC) X-FDA: 82509366870.18.5206ECC Received: from pegase2.c-s.fr (pegase2.c-s.fr [93.17.235.10]) by imf05.hostedemail.com (Postfix) with ESMTP id 637C510000A for ; Fri, 30 Aug 2024 15:57:33 +0000 (UTC) Authentication-Results: imf05.hostedemail.com; dkim=none; spf=pass (imf05.hostedemail.com: domain of christophe.leroy@csgroup.eu designates 93.17.235.10 as permitted sender) smtp.mailfrom=christophe.leroy@csgroup.eu; dmarc=pass (policy=quarantine) header.from=csgroup.eu ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1725033408; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=DytUxfbmplaMrRHhl3qFQ+Z19NOu/EOZY8HpIR/7NxA=; b=1kWz8iTuN9n68rBmkC9Da3mVJPnJYUoiJUZR6N5WAXDjqEoCcQHEnO9csfLZqkc7L2ziVk GaHR2y3gf8sWDu88L24MA8XThQTQ6iMWBSoE6DoOuSHBAMX7msSSz2A/8RfchY2m4phN55 beSOHiSH0W6RFMAzQpbwOGIjlVOgie4= ARC-Authentication-Results: i=1; imf05.hostedemail.com; dkim=none; spf=pass (imf05.hostedemail.com: domain of christophe.leroy@csgroup.eu designates 93.17.235.10 as permitted sender) smtp.mailfrom=christophe.leroy@csgroup.eu; dmarc=pass (policy=quarantine) header.from=csgroup.eu ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1725033408; a=rsa-sha256; cv=none; b=0O+pzprsnIK1x8gOTgdJAKqxSQnyVKn6eMF4l3BPv1TuF7gfqeuLh26J2aiOn4CR//J3HG Ux6s1U3psr3tnkIf9roRtjBsz3KLC04VqxI6ZWHH00Q4jiNrVFEbW7lj+N9pqNTI6DJt1u BrKgmpntfLh5cOxJYX0rZwHifAdkFt8= Received: from localhost (mailhub3.si.c-s.fr [172.26.127.67]) by localhost (Postfix) with ESMTP id 4WwN802ZX5z9sSK; Fri, 30 Aug 2024 17:57:20 +0200 (CEST) X-Virus-Scanned: amavisd-new at c-s.fr Received: from pegase2.c-s.fr ([172.26.127.65]) by localhost (pegase2.c-s.fr [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Q0mB1JG6GN8s; Fri, 30 Aug 2024 17:57:20 +0200 (CEST) Received: from messagerie.si.c-s.fr (messagerie.si.c-s.fr [192.168.25.192]) by pegase2.c-s.fr (Postfix) with ESMTP id 4WwN7z5nMHz9sS7; Fri, 30 Aug 2024 17:57:19 +0200 (CEST) Received: from localhost (localhost [127.0.0.1]) by messagerie.si.c-s.fr (Postfix) with ESMTP id B41978B799; Fri, 30 Aug 2024 17:57:19 +0200 (CEST) X-Virus-Scanned: amavisd-new at c-s.fr Received: from messagerie.si.c-s.fr ([127.0.0.1]) by localhost (messagerie.si.c-s.fr [127.0.0.1]) (amavisd-new, port 10023) with ESMTP id uyx2pzQ4Hlnz; Fri, 30 Aug 2024 17:57:19 +0200 (CEST) Received: from PO20335.idsi0.si.c-s.fr (unknown [192.168.234.133]) by messagerie.si.c-s.fr (Postfix) with ESMTP id B7E7C8B794; Fri, 30 Aug 2024 17:57:18 +0200 (CEST) From: Christophe Leroy To: Andrew Morton , Steven Rostedt , Masami Hiramatsu , Mathieu Desnoyers , Michael Ellerman , Nicholas Piggin , Naveen N Rao , Nathan Chancellor , Nick Desaulniers , Bill Wendling , Justin Stitt , Shuah Khan , "Jason A . Donenfeld" Cc: Christophe Leroy , linux-kernel@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-kselftest@vger.kernel.org, llvm@lists.linux.dev, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, linux-trace-kernel@vger.kernel.org, Adhemerval Zanella , Xi Ruoyao Subject: [PATCH v3 3/5] powerpc/vdso: Refactor CFLAGS for CVDSO build Date: Fri, 30 Aug 2024 17:57:07 +0200 Message-ID: <7c1887fa51ececc322ce1bd66a775a41e87dbc15.1725031952.git.christophe.leroy@csgroup.eu> X-Mailer: git-send-email 2.44.0 In-Reply-To: References: MIME-Version: 1.0 X-Developer-Signature: v=1; a=ed25519-sha256; t=1725033425; l=4208; i=christophe.leroy@csgroup.eu; s=20211009; h=from:subject:message-id; bh=8uyRGNGCCgVJfGMmDTrx0v5vR/iTZQuTWlNxnOkFWq8=; b=Ou6uC5Ey7EcHKFQsu9WPadpyZaZcA98uuLQGZhz2EkTjguuygu/aCnPKJxOYTFzPTTW2nhxai 7S/HmjN2ZjaDWg/Pij9CV+qCtd7P+Ofbb+keJz6knCTIYIjC7IXYElU X-Developer-Key: i=christophe.leroy@csgroup.eu; a=ed25519; pk=HIzTzUj91asvincQGOFx6+ZF5AoUuP9GdOtQChs7Mm0= X-Rspamd-Server: rspam03 X-Rspam-User: X-Rspamd-Queue-Id: 637C510000A X-Stat-Signature: 6otiu1nh7ueq1mm1uehjozhojaahrgq7 X-HE-Tag: 1725033453-223411 X-HE-Meta: U2FsdGVkX19qj4r7Ww5rGkKIzYaDDdP/wZGxgE/hH6fyyrl14kUvN7e2zEmlfjvhHD+rc4+NdjEIMA8GO38e7Zh5OyfrmINKUmsNoabntaVdjR849XJHN4A+XsQ53Ze+4HNavY4yt2846NqxvL07BgyjPMtEg1NkvjwRfGpzWMsHINDPJ2NuYr4nPUm32H5oqU6TNQ9jNTxbIQgo0Wkdi5bjACK6ZRovEMKMOsB8fjRqzP6zlY8NxaVB2fXVOGVQgA65m7rLhvYHy71pajTHYZO38KYPOaaAO8gVROdmUBT2uBeXAwCCmlnFGyLC8FR/7oeHec5zQN07QvlvYvruIzmGc3FC18eZPj3rJq5X4p22/HswdWCp8aeKxhDOhu6tOyIf6VqYxw5HbyHuTX4HUAsRbk5HID2MNercW9VEcsrpXpMBWlXPGu/A32TE8b1Dtf6wfO6KkS4P5h92wrCH3x+jESt89yiNmsCn+OaAyT0Q1ZzDYROsrmPyLeLN9EYbHvvrU5EcgJRm2qBpFUu3cFB2dj/BzmEFjsrljsE83/BrSoUAw2lx++5oebQHlg4qJ/e3FtpZWQXCSZwG2t15kgQ7HSTzuzpOH0MxREC3Q4qpa2W4kMOlJu8QZcyv21AwcVf1sXSkgBm4Z/lJv0Vb28d2FNtLSPGWb8y91ysI7RsRIeF+OobMpZSCM0tMwSl5jxS9UcW+vWCBE+vw8uJFC/9GLHPYJzeT0imYXQ+HL6aAyPhet2tCPBuniOdkg1MtH29dbPcm7WAVYAxlCDkGGGdp81dHk7vIIT/kY89CCQrN8bHzVkIbix5FY8mWLli8GqXtC5dzJoNhL8ZNSUS9ygXTbI1Xq6xarP2uHnQDvPYjhOnyPrNxBdmwq8LNXdEFy4rJ2VOI8oFXXsCsdw820CZ/ShhT7M53BQNf0TfEU0FT+FCl4ESbrQV9U2bcEe46NyE4BEKKw+BmVGx5dQT QUmQy2bu Nj0qDYuRxB+ypMaWtR2/rzHq0wJTeTyeDAmuz9EyomIM3iiIlzhO4C1IjLu/27NFSuv/dSYY51VC4wfP19dmhma21bZAcEYoivtJWWXD/vaawygGf2LLeEWRj87OsM4tV752E X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: In order to avoid two much duplication when we add new VDSO functionnalities in C like getrandom, refactor common CFLAGS. Signed-off-by: Christophe Leroy --- v3: Also refactor removed flags --- arch/powerpc/kernel/vdso/Makefile | 32 +++++++++++++------------------ 1 file changed, 13 insertions(+), 19 deletions(-) diff --git a/arch/powerpc/kernel/vdso/Makefile b/arch/powerpc/kernel/vdso/Makefile index c07a425b8f78..67fe79d26fae 100644 --- a/arch/powerpc/kernel/vdso/Makefile +++ b/arch/powerpc/kernel/vdso/Makefile @@ -10,28 +10,11 @@ obj-vdso64 = sigtramp64-64.o gettimeofday-64.o datapage-64.o cacheflush-64.o not ifneq ($(c-gettimeofday-y),) CFLAGS_vgettimeofday-32.o += -include $(c-gettimeofday-y) - CFLAGS_vgettimeofday-32.o += $(DISABLE_LATENT_ENTROPY_PLUGIN) - CFLAGS_vgettimeofday-32.o += $(call cc-option, -fno-stack-protector) - CFLAGS_vgettimeofday-32.o += -DDISABLE_BRANCH_PROFILING - CFLAGS_vgettimeofday-32.o += -ffreestanding -fasynchronous-unwind-tables - CFLAGS_REMOVE_vgettimeofday-32.o = $(CC_FLAGS_FTRACE) - CFLAGS_REMOVE_vgettimeofday-32.o += -mcmodel=medium -mabi=elfv1 -mabi=elfv2 -mcall-aixdesc - # This flag is supported by clang for 64-bit but not 32-bit so it will cause - # an unused command line flag warning for this file. - ifdef CONFIG_CC_IS_CLANG - CFLAGS_REMOVE_vgettimeofday-32.o += -fno-stack-clash-protection - endif - CFLAGS_vgettimeofday-64.o += -include $(c-gettimeofday-y) - CFLAGS_vgettimeofday-64.o += $(DISABLE_LATENT_ENTROPY_PLUGIN) - CFLAGS_vgettimeofday-64.o += $(call cc-option, -fno-stack-protector) - CFLAGS_vgettimeofday-64.o += -DDISABLE_BRANCH_PROFILING - CFLAGS_vgettimeofday-64.o += -ffreestanding -fasynchronous-unwind-tables - CFLAGS_REMOVE_vgettimeofday-64.o = $(CC_FLAGS_FTRACE) # Go prior to 1.16.x assumes r30 is not clobbered by any VDSO code. That used to be true # by accident when the VDSO was hand-written asm code, but may not be now that the VDSO is # compiler generated. To avoid breaking Go tell GCC not to use r30. Impact on code # generation is minimal, it will just use r29 instead. - CFLAGS_vgettimeofday-64.o += $(call cc-option, -ffixed-r30) + CFLAGS_vgettimeofday-64.o += -include $(c-gettimeofday-y) $(call cc-option, -ffixed-r30) endif # Build rules @@ -49,6 +32,11 @@ targets += $(obj-vdso64) vdso64.so.dbg vgettimeofday-64.o obj-vdso64 := $(addprefix $(obj)/, $(obj-vdso64)) ccflags-y := -fno-common -fno-builtin +ccflags-y += $(DISABLE_LATENT_ENTROPY_PLUGIN) +ccflags-y += $(call cc-option, -fno-stack-protector) +ccflags-y += -DDISABLE_BRANCH_PROFILING +ccflags-y += -ffreestanding -fasynchronous-unwind-tables +ccflags-remove-y := $(CC_FLAGS_FTRACE) ldflags-y := -Wl,--hash-style=both -nostdlib -shared -z noexecstack $(CLANG_FLAGS) ldflags-$(CONFIG_LD_IS_LLD) += $(call cc-option,--ld-path=$(LD),-fuse-ld=lld) ldflags-$(CONFIG_LD_ORPHAN_WARN) += -Wl,--orphan-handling=$(CONFIG_LD_ORPHAN_WARN_LEVEL) @@ -57,6 +45,12 @@ ldflags-$(CONFIG_LD_ORPHAN_WARN) += -Wl,--orphan-handling=$(CONFIG_LD_ORPHAN_WAR ldflags-y += $(filter-out $(CC_AUTO_VAR_INIT_ZERO_ENABLER) $(CC_FLAGS_FTRACE) -Wa$(comma)%, $(KBUILD_CFLAGS)) CC32FLAGS := -m32 +CC32FLAGSREMOVE := -mcmodel=medium -mabi=elfv1 -mabi=elfv2 -mcall-aixdesc + # This flag is supported by clang for 64-bit but not 32-bit so it will cause + # an unused command line flag warning for this file. +ifdef CONFIG_CC_IS_CLANG +CC32FLAGSREMOVE += -fno-stack-clash-protection +endif LD32FLAGS := -Wl,-soname=linux-vdso32.so.1 AS32FLAGS := -D__VDSO32__ @@ -105,7 +99,7 @@ quiet_cmd_vdso32ld_and_check = VDSO32L $@ quiet_cmd_vdso32as = VDSO32A $@ cmd_vdso32as = $(VDSOCC) $(a_flags) $(CC32FLAGS) $(AS32FLAGS) -c -o $@ $< quiet_cmd_vdso32cc = VDSO32C $@ - cmd_vdso32cc = $(VDSOCC) $(c_flags) $(CC32FLAGS) -c -o $@ $< + cmd_vdso32cc = $(VDSOCC) $(filter-out $(CC32FLAGSREMOVE), $(c_flags)) $(CC32FLAGS) -c -o $@ $< quiet_cmd_vdso64ld_and_check = VDSO64L $@ cmd_vdso64ld_and_check = $(VDSOCC) $(ldflags-y) $(LD64FLAGS) -o $@ -Wl,-T$(filter %.lds,$^) $(filter %.o,$^); $(cmd_vdso_check) From patchwork Fri Aug 30 15:57:08 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christophe Leroy X-Patchwork-Id: 13785300 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 93D51CA1005 for ; Fri, 30 Aug 2024 15:57:41 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 339216B018E; Fri, 30 Aug 2024 11:57:41 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 30FA96B018F; Fri, 30 Aug 2024 11:57:41 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 1D77C6B0190; Fri, 30 Aug 2024 11:57:41 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id EC9B66B018E for ; Fri, 30 Aug 2024 11:57:40 -0400 (EDT) Received: from smtpin25.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id AD54D160FB0 for ; Fri, 30 Aug 2024 15:57:40 +0000 (UTC) X-FDA: 82509367080.25.4F10EF1 Received: from pegase2.c-s.fr (pegase2.c-s.fr [93.17.235.10]) by imf23.hostedemail.com (Postfix) with ESMTP id 685E5140011 for ; Fri, 30 Aug 2024 15:57:38 +0000 (UTC) Authentication-Results: imf23.hostedemail.com; dkim=none; spf=pass (imf23.hostedemail.com: domain of christophe.leroy@csgroup.eu designates 93.17.235.10 as permitted sender) smtp.mailfrom=christophe.leroy@csgroup.eu; dmarc=pass (policy=quarantine) header.from=csgroup.eu ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1725033413; a=rsa-sha256; cv=none; b=ivRJ7SjEHeTPzD/AUTAq3+IUv/TG005BxNeeOokPP+iieVkczb0v79G9tqWOHnqpiJcmQZ Pmqg9cn0yAFb95iPhUPuJQub0KrJ2EKfaTedgb0Esr13GLR9IYhngPt/VFTa2FWwvGf5E5 erh7UjUstCyCriQy8Efg8RFWxNLjyX4= ARC-Authentication-Results: i=1; imf23.hostedemail.com; dkim=none; spf=pass (imf23.hostedemail.com: domain of christophe.leroy@csgroup.eu designates 93.17.235.10 as permitted sender) smtp.mailfrom=christophe.leroy@csgroup.eu; dmarc=pass (policy=quarantine) header.from=csgroup.eu ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1725033413; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Nf6+8fazIsvHJNgYQrS4IE85cmWQna7PUbhjoWnv9wU=; b=peLuoPrdm4Fk/PA46FbrBYTvxjX49XomqTv9WzDo/uy0xenBAinzhAqcPvQFXbpI018b56 OIqqbuoVFOSi5a/bEuHzRCF3bkg0qRqXRr+kVEB1IxeD4VUkzwKxnazqU4dllMCukcovyu AXJaOlILOQlBEQgT8cz2V0XS5lzJ0JM= Received: from localhost (mailhub3.si.c-s.fr [172.26.127.67]) by localhost (Postfix) with ESMTP id 4WwN811cC2z9sSN; Fri, 30 Aug 2024 17:57:21 +0200 (CEST) X-Virus-Scanned: amavisd-new at c-s.fr Received: from pegase2.c-s.fr ([172.26.127.65]) by localhost (pegase2.c-s.fr [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id yYeX1GWSo-jT; Fri, 30 Aug 2024 17:57:21 +0200 (CEST) Received: from messagerie.si.c-s.fr (messagerie.si.c-s.fr [192.168.25.192]) by pegase2.c-s.fr (Postfix) with ESMTP id 4WwN806Jpqz9sS7; Fri, 30 Aug 2024 17:57:20 +0200 (CEST) Received: from localhost (localhost [127.0.0.1]) by messagerie.si.c-s.fr (Postfix) with ESMTP id C71B58B799; Fri, 30 Aug 2024 17:57:20 +0200 (CEST) X-Virus-Scanned: amavisd-new at c-s.fr Received: from messagerie.si.c-s.fr ([127.0.0.1]) by localhost (messagerie.si.c-s.fr [127.0.0.1]) (amavisd-new, port 10023) with ESMTP id lxFQRnFVCPTN; Fri, 30 Aug 2024 17:57:20 +0200 (CEST) Received: from PO20335.idsi0.si.c-s.fr (unknown [192.168.234.133]) by messagerie.si.c-s.fr (Postfix) with ESMTP id AC4EB8B764; Fri, 30 Aug 2024 17:57:19 +0200 (CEST) From: Christophe Leroy To: Andrew Morton , Steven Rostedt , Masami Hiramatsu , Mathieu Desnoyers , Michael Ellerman , Nicholas Piggin , Naveen N Rao , Nathan Chancellor , Nick Desaulniers , Bill Wendling , Justin Stitt , Shuah Khan , "Jason A . Donenfeld" Cc: Christophe Leroy , linux-kernel@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-kselftest@vger.kernel.org, llvm@lists.linux.dev, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, linux-trace-kernel@vger.kernel.org, Adhemerval Zanella , Xi Ruoyao Subject: [PATCH v3 4/5] powerpc/vdso: Wire up getrandom() vDSO implementation on PPC32 Date: Fri, 30 Aug 2024 17:57:08 +0200 Message-ID: X-Mailer: git-send-email 2.44.0 In-Reply-To: References: MIME-Version: 1.0 X-Developer-Signature: v=1; a=ed25519-sha256; t=1725033425; l=17568; i=christophe.leroy@csgroup.eu; s=20211009; h=from:subject:message-id; bh=0w4ulQ4gcMEsE3ThfrWB+nL+4Z0DOw2p7OgH3/0aDS8=; b=+ceKvNzluBV2XXdigcJSSvbEPCI9JXmj4O5C6yvL0o9Orct4EXswhhFAS0MNcMX53AK5sAH2t 94n4l5gdkI0AwMHGqWE0SOonwovMQ9+pG43JOUhc17SsUzns00vAMrD X-Developer-Key: i=christophe.leroy@csgroup.eu; a=ed25519; pk=HIzTzUj91asvincQGOFx6+ZF5AoUuP9GdOtQChs7Mm0= X-Stat-Signature: 9qze63b1sttr8wnnkow8guffb85yfa6b X-Rspamd-Queue-Id: 685E5140011 X-Rspam-User: X-Rspamd-Server: rspam10 X-HE-Tag: 1725033458-111050 X-HE-Meta: U2FsdGVkX19e6f+FNvz+BkQL9Lj+Xu+56u81ih5tN4hn2vK2p2WFjd00dReJ5GOUtsmRB0Xhy3NQMYM35ew3kltG+nUIWc3MlNU756YyJOW9CQVHyLZYKH17qQr/PHpvQ7StmqrcU8M6gEQmTo+HXp2lEq9IAja7sLFNAIPrtBP+xyG9G8gUBFtCaylxeRgQwuJz77YAOcQuS7WlG6VLlc5ka4IiIGa5AbNwSjszuckDzjggzjdDA3SFMrfsL/Sf087K3da3hKpmnTWgipBaKF1KGMwNdfD/kCJXpMCfgflm3CC7oo1P2xYgnseM/qBlR7gc8vspHwcq2YP8iFy1x9Ltt8e9L1yC2smcFdCg2y98flXj0c7wTmbudy/aD4gWLgKkHJWTlkVomI8dEa/m9RFSDdGW1Gvnt5B/+oiEzr074cN+AQTYMHwMqqTFLQRgXhRFECFtLUtKMwuk2iD3Smva6Ir69ZnsXgPKGa/dfRW35oUvqP5nK+X27kUOEU4iKU3wJO+RuzDMe7V7D9QDmQguQIO8kfKECsBaKySsNMCpf0EDxUlCpdkNCT7tA/oTNsXWGTecsyz91kr0LNRFq+bgjkBW07Xz/8jc9wiIt4EDNSmJk47A6mDC0cP2ERW9ETyM0necP/kqBwcovpNuw5cwF+YIhPfDfrb01eGrTKqRREDnyYPfREMVRQcar2t0rdPPHqYpQLZgAeIaTJ4RGkb0cDCr1j6/VwtPEt6x0uyUVbEfhi7mqNwjKfWXMenfnDsCh39KFrEcOYlxBQHhx4mJtpU8mT+8MmRrmdoatD7B1NU1PxPFB9Ca9WRRGyYRl9YRkyww+bZygm7Tu5IP2WXbcDURHq3+bkcmMD1l2CxeLATKCK+qMSLuKeXHcK+KT+MuJmXVqf2jwdNnTirKqbb8WnjnH+NZvQ1F1BLbmgCKjoeN8jTdS8oIjeMXQ6OvXbG8KwWqN4lH0/ldQYu P2kUQI5m sRRzLJudmZehuf+DQWABtXvk4OUnZF8dei14dsulcIvN+5NuYuKFPUy8IMtsj93XtOqnoxzbK/RZLsJUk/BJ0CdGUNRd7ORluAx5GMfADv75txVi0WjRpS/B65Eo7YwbUhlLz/dAdQH9CwjY+CC22nhCRu9JuCMXMyOU9 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: To be consistent with other VDSO functions, the function is called __kernel_getrandom() __arch_chacha20_blocks_nostack() fonction is implemented basically with 32 bits operations. It performs 4 QUARTERROUND operations in parallele. There are enough registers to avoid using the stack: On input: r3: output bytes r4: 32-byte key input r5: 8-byte counter input/output r6: number of 64-byte blocks to write to output During operation: stack: pointer to counter (r5) and non-volatile registers (r14-131) r0: counter of blocks (initialised with r6) r4: Value '4' after key has been read, used for indexing r5-r12: key r14-r15: block counter r16-r31: chacha state At the end: r0, r6-r12: Zeroised r5, r14-r31: Restored Performance on powerpc 885 (using kernel selftest): ~# ./vdso_test_getrandom bench-single vdso: 25000000 times in 62.938002291 seconds libc: 25000000 times in 535.581916866 seconds syscall: 25000000 times in 531.525042806 seconds Performance on powerpc 8321 (using kernel selftest): ~# ./vdso_test_getrandom bench-single vdso: 25000000 times in 16.899318858 seconds libc: 25000000 times in 131.050596522 seconds syscall: 25000000 times in 129.794790389 seconds Signed-off-by: Christophe Leroy --- v3: - Preserve r13, implies saving r5 on stack - Split PPC64 implementation out. --- arch/powerpc/Kconfig | 1 + arch/powerpc/include/asm/asm-compat.h | 8 + arch/powerpc/include/asm/vdso/getrandom.h | 54 +++++ arch/powerpc/include/asm/vdso/vsyscall.h | 6 + arch/powerpc/include/asm/vdso_datapage.h | 2 + arch/powerpc/kernel/asm-offsets.c | 1 + arch/powerpc/kernel/vdso/Makefile | 12 +- arch/powerpc/kernel/vdso/getrandom.S | 50 +++++ arch/powerpc/kernel/vdso/vdso32.lds.S | 1 + arch/powerpc/kernel/vdso/vgetrandom-chacha.S | 201 +++++++++++++++++++ arch/powerpc/kernel/vdso/vgetrandom.c | 14 ++ tools/arch/powerpc/vdso | 1 + tools/testing/selftests/vDSO/Makefile | 4 + 13 files changed, 353 insertions(+), 2 deletions(-) create mode 100644 arch/powerpc/include/asm/vdso/getrandom.h create mode 100644 arch/powerpc/kernel/vdso/getrandom.S create mode 100644 arch/powerpc/kernel/vdso/vgetrandom-chacha.S create mode 100644 arch/powerpc/kernel/vdso/vgetrandom.c create mode 120000 tools/arch/powerpc/vdso diff --git a/arch/powerpc/Kconfig b/arch/powerpc/Kconfig index d7b09b064a8a..54b270ef18b1 100644 --- a/arch/powerpc/Kconfig +++ b/arch/powerpc/Kconfig @@ -311,6 +311,7 @@ config PPC select SYSCTL_EXCEPTION_TRACE select THREAD_INFO_IN_TASK select TRACE_IRQFLAGS_SUPPORT + select VDSO_GETRANDOM if PPC32 # # Please keep this list sorted alphabetically. # diff --git a/arch/powerpc/include/asm/asm-compat.h b/arch/powerpc/include/asm/asm-compat.h index b0b209c1df50..cce8c31b1b33 100644 --- a/arch/powerpc/include/asm/asm-compat.h +++ b/arch/powerpc/include/asm/asm-compat.h @@ -59,4 +59,12 @@ #endif +#ifdef __BIG_ENDIAN__ +#define LWZX_LE stringify_in_c(lwbrx) +#define STWX_LE stringify_in_c(stwbrx) +#else +#define LWZX_LE stringify_in_c(lwzx) +#define STWX_LE stringify_in_c(stwx) +#endif + #endif /* _ASM_POWERPC_ASM_COMPAT_H */ diff --git a/arch/powerpc/include/asm/vdso/getrandom.h b/arch/powerpc/include/asm/vdso/getrandom.h new file mode 100644 index 000000000000..501d6bb14e8a --- /dev/null +++ b/arch/powerpc/include/asm/vdso/getrandom.h @@ -0,0 +1,54 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* + * Copyright (C) 2024 Christophe Leroy , CS GROUP France + */ +#ifndef _ASM_POWERPC_VDSO_GETRANDOM_H +#define _ASM_POWERPC_VDSO_GETRANDOM_H + +#ifndef __ASSEMBLY__ + +static __always_inline int do_syscall_3(const unsigned long _r0, const unsigned long _r3, + const unsigned long _r4, const unsigned long _r5) +{ + register long r0 asm("r0") = _r0; + register unsigned long r3 asm("r3") = _r3; + register unsigned long r4 asm("r4") = _r4; + register unsigned long r5 asm("r5") = _r5; + register int ret asm ("r3"); + + asm volatile( + " sc\n" + " bns+ 1f\n" + " neg %0, %0\n" + "1:\n" + : "=r" (ret), "+r" (r4), "+r" (r5), "+r" (r0) + : "r" (r3) + : "memory", "r6", "r7", "r8", "r9", "r10", "r11", "r12", "cr0", "ctr"); + + return ret; +} + +/** + * getrandom_syscall - Invoke the getrandom() syscall. + * @buffer: Destination buffer to fill with random bytes. + * @len: Size of @buffer in bytes. + * @flags: Zero or more GRND_* flags. + * Returns: The number of bytes written to @buffer, or a negative value indicating an error. + */ +static __always_inline ssize_t getrandom_syscall(void *buffer, size_t len, unsigned int flags) +{ + return do_syscall_3(__NR_getrandom, (unsigned long)buffer, + (unsigned long)len, (unsigned long)flags); +} + +static __always_inline struct vdso_rng_data *__arch_get_vdso_rng_data(void) +{ + return NULL; +} + +ssize_t __c_kernel_getrandom(void *buffer, size_t len, unsigned int flags, void *opaque_state, + size_t opaque_len, const struct vdso_rng_data *vd); + +#endif /* !__ASSEMBLY__ */ + +#endif /* _ASM_POWERPC_VDSO_GETRANDOM_H */ diff --git a/arch/powerpc/include/asm/vdso/vsyscall.h b/arch/powerpc/include/asm/vdso/vsyscall.h index 48cf23f1e273..92f480d8cc6d 100644 --- a/arch/powerpc/include/asm/vdso/vsyscall.h +++ b/arch/powerpc/include/asm/vdso/vsyscall.h @@ -17,6 +17,12 @@ struct vdso_data *__arch_get_k_vdso_data(void) } #define __arch_get_k_vdso_data __arch_get_k_vdso_data +static __always_inline +struct vdso_rng_data *__arch_get_k_vdso_rng_data(void) +{ + return &vdso_data->rng_data; +} + /* The asm-generic header needs to be included after the definitions above */ #include diff --git a/arch/powerpc/include/asm/vdso_datapage.h b/arch/powerpc/include/asm/vdso_datapage.h index a585c8e538ff..e17500c5237e 100644 --- a/arch/powerpc/include/asm/vdso_datapage.h +++ b/arch/powerpc/include/asm/vdso_datapage.h @@ -83,6 +83,7 @@ struct vdso_arch_data { __u32 compat_syscall_map[SYSCALL_MAP_SIZE]; /* Map of compat syscalls */ struct vdso_data data[CS_BASES]; + struct vdso_rng_data rng_data; }; #else /* CONFIG_PPC64 */ @@ -95,6 +96,7 @@ struct vdso_arch_data { __u32 syscall_map[SYSCALL_MAP_SIZE]; /* Map of syscalls */ __u32 compat_syscall_map[0]; /* No compat syscalls on PPC32 */ struct vdso_data data[CS_BASES]; + struct vdso_rng_data rng_data; }; #endif /* CONFIG_PPC64 */ diff --git a/arch/powerpc/kernel/asm-offsets.c b/arch/powerpc/kernel/asm-offsets.c index 23733282de4d..eedb2e04c785 100644 --- a/arch/powerpc/kernel/asm-offsets.c +++ b/arch/powerpc/kernel/asm-offsets.c @@ -335,6 +335,7 @@ int main(void) /* datapage offsets for use by vdso */ OFFSET(VDSO_DATA_OFFSET, vdso_arch_data, data); + OFFSET(VDSO_RNG_DATA_OFFSET, vdso_arch_data, rng_data); OFFSET(CFG_TB_TICKS_PER_SEC, vdso_arch_data, tb_ticks_per_sec); #ifdef CONFIG_PPC64 OFFSET(CFG_ICACHE_BLOCKSZ, vdso_arch_data, icache_block_size); diff --git a/arch/powerpc/kernel/vdso/Makefile b/arch/powerpc/kernel/vdso/Makefile index 67fe79d26fae..fa0b9b3b51af 100644 --- a/arch/powerpc/kernel/vdso/Makefile +++ b/arch/powerpc/kernel/vdso/Makefile @@ -8,6 +8,8 @@ include $(srctree)/lib/vdso/Makefile obj-vdso32 = sigtramp32-32.o gettimeofday-32.o datapage-32.o cacheflush-32.o note-32.o getcpu-32.o obj-vdso64 = sigtramp64-64.o gettimeofday-64.o datapage-64.o cacheflush-64.o note-64.o getcpu-64.o +obj-vdso32 += getrandom-32.o vgetrandom-chacha-32.o + ifneq ($(c-gettimeofday-y),) CFLAGS_vgettimeofday-32.o += -include $(c-gettimeofday-y) # Go prior to 1.16.x assumes r30 is not clobbered by any VDSO code. That used to be true @@ -17,6 +19,10 @@ ifneq ($(c-gettimeofday-y),) CFLAGS_vgettimeofday-64.o += -include $(c-gettimeofday-y) $(call cc-option, -ffixed-r30) endif +ifneq ($(c-getrandom-y),) + CFLAGS_vgetrandom-32.o += -include $(c-getrandom-y) +endif + # Build rules ifdef CROSS32_COMPILE @@ -25,7 +31,7 @@ else VDSOCC := $(CC) endif -targets := $(obj-vdso32) vdso32.so.dbg vgettimeofday-32.o +targets := $(obj-vdso32) vdso32.so.dbg vgettimeofday-32.o vgetrandom-32.o targets += crtsavres-32.o obj-vdso32 := $(addprefix $(obj)/, $(obj-vdso32)) targets += $(obj-vdso64) vdso64.so.dbg vgettimeofday-64.o @@ -63,7 +69,7 @@ targets += vdso64.lds CPPFLAGS_vdso64.lds += -P -C # link rule for the .so file, .lds has to be first -$(obj)/vdso32.so.dbg: $(obj)/vdso32.lds $(obj-vdso32) $(obj)/vgettimeofday-32.o $(obj)/crtsavres-32.o FORCE +$(obj)/vdso32.so.dbg: $(obj)/vdso32.lds $(obj-vdso32) $(obj)/vgettimeofday-32.o $(obj)/vgetrandom-32.o $(obj)/crtsavres-32.o FORCE $(call if_changed,vdso32ld_and_check) $(obj)/vdso64.so.dbg: $(obj)/vdso64.lds $(obj-vdso64) $(obj)/vgettimeofday-64.o FORCE $(call if_changed,vdso64ld_and_check) @@ -75,6 +81,8 @@ $(obj)/crtsavres-32.o: %-32.o: $(srctree)/arch/powerpc/lib/crtsavres.S FORCE $(call if_changed_dep,vdso32as) $(obj)/vgettimeofday-32.o: %-32.o: %.c FORCE $(call if_changed_dep,vdso32cc) +$(obj)/vgetrandom-32.o: %-32.o: %.c FORCE + $(call if_changed_dep,vdso32cc) $(obj-vdso64): %-64.o: %.S FORCE $(call if_changed_dep,vdso64as) $(obj)/vgettimeofday-64.o: %-64.o: %.c FORCE diff --git a/arch/powerpc/kernel/vdso/getrandom.S b/arch/powerpc/kernel/vdso/getrandom.S new file mode 100644 index 000000000000..21773ef3fc1d --- /dev/null +++ b/arch/powerpc/kernel/vdso/getrandom.S @@ -0,0 +1,50 @@ +/* SPDX-License-Identifier: GPL-2.0-or-later */ +/* + * Userland implementation of getrandom() for processes + * for use in the vDSO + * + * Copyright (C) 2024 Christophe Leroy , CS GROUP France + */ +#include +#include +#include +#include +#include +#include + +/* + * The macro sets two stack frames, one for the caller and one for the callee + * because there are no requirement for the caller to set a stack frame when + * calling VDSO so it may have omitted to set one, especially on PPC64 + */ + +.macro cvdso_call funct + .cfi_startproc + PPC_STLU r1, -PPC_MIN_STKFRM(r1) + .cfi_adjust_cfa_offset PPC_MIN_STKFRM + mflr r0 + PPC_STLU r1, -PPC_MIN_STKFRM(r1) + .cfi_adjust_cfa_offset PPC_MIN_STKFRM + PPC_STL r0, PPC_MIN_STKFRM + PPC_LR_STKOFF(r1) + .cfi_rel_offset lr, PPC_MIN_STKFRM + PPC_LR_STKOFF + get_datapage r8 + addi r8, r8, VDSO_RNG_DATA_OFFSET + bl CFUNC(DOTSYM(\funct)) + PPC_LL r0, PPC_MIN_STKFRM + PPC_LR_STKOFF(r1) + cmpwi r3, 0 + mtlr r0 + addi r1, r1, 2 * PPC_MIN_STKFRM + .cfi_restore lr + .cfi_def_cfa_offset 0 + crclr so + bgelr+ + crset so + neg r3, r3 + blr + .cfi_endproc +.endm + + .text +V_FUNCTION_BEGIN(__kernel_getrandom) + cvdso_call __c_kernel_getrandom +V_FUNCTION_END(__kernel_getrandom) diff --git a/arch/powerpc/kernel/vdso/vdso32.lds.S b/arch/powerpc/kernel/vdso/vdso32.lds.S index 8f57107000a2..7b41d5d256e8 100644 --- a/arch/powerpc/kernel/vdso/vdso32.lds.S +++ b/arch/powerpc/kernel/vdso/vdso32.lds.S @@ -130,6 +130,7 @@ VERSION #if defined(CONFIG_PPC64) || !defined(CONFIG_SMP) __kernel_getcpu; #endif + __kernel_getrandom; local: *; }; diff --git a/arch/powerpc/kernel/vdso/vgetrandom-chacha.S b/arch/powerpc/kernel/vdso/vgetrandom-chacha.S new file mode 100644 index 000000000000..a3475b9e8c10 --- /dev/null +++ b/arch/powerpc/kernel/vdso/vgetrandom-chacha.S @@ -0,0 +1,201 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* + * Copyright (C) 2024 Christophe Leroy , CS GROUP France + */ + +#include + +#include + +.macro quarterround4 a1 b1 c1 d1 a2 b2 c2 d2 a3 b3 c3 d3 a4 b4 c4 d4 + add \a1, \a1, \b1 + add \a2, \a2, \b2 + add \a3, \a3, \b3 + add \a4, \a4, \b4 + xor \d1, \d1, \a1 + xor \d2, \d2, \a2 + xor \d3, \d3, \a3 + xor \d4, \d4, \a4 + rotlwi \d1, \d1, 16 + rotlwi \d2, \d2, 16 + rotlwi \d3, \d3, 16 + rotlwi \d4, \d4, 16 + add \c1, \c1, \d1 + add \c2, \c2, \d2 + add \c3, \c3, \d3 + add \c4, \c4, \d4 + xor \b1, \b1, \c1 + xor \b2, \b2, \c2 + xor \b3, \b3, \c3 + xor \b4, \b4, \c4 + rotlwi \b1, \b1, 12 + rotlwi \b2, \b2, 12 + rotlwi \b3, \b3, 12 + rotlwi \b4, \b4, 12 + add \a1, \a1, \b1 + add \a2, \a2, \b2 + add \a3, \a3, \b3 + add \a4, \a4, \b4 + xor \d1, \d1, \a1 + xor \d2, \d2, \a2 + xor \d3, \d3, \a3 + xor \d4, \d4, \a4 + rotlwi \d1, \d1, 8 + rotlwi \d2, \d2, 8 + rotlwi \d3, \d3, 8 + rotlwi \d4, \d4, 8 + add \c1, \c1, \d1 + add \c2, \c2, \d2 + add \c3, \c3, \d3 + add \c4, \c4, \d4 + xor \b1, \b1, \c1 + xor \b2, \b2, \c2 + xor \b3, \b3, \c3 + xor \b4, \b4, \c4 + rotlwi \b1, \b1, 7 + rotlwi \b2, \b2, 7 + rotlwi \b3, \b3, 7 + rotlwi \b4, \b4, 7 +.endm + +#define QUARTERROUND4(a1,b1,c1,d1,a2,b2,c2,d2,a3,b3,c3,d3,a4,b4,c4,d4) \ + quarterround4 16+a1 16+b1 16+c1 16+d1 16+a2 16+b2 16+c2 16+d2 \ + 16+a3 16+b3 16+c3 16+d3 16+a4 16+b4 16+c4 16+d4 + +/* + * Very basic 32 bits implementation of ChaCha20. Produces a given positive number + * of blocks of output with a nonce of 0, taking an input key and 8-byte + * counter. Importantly does not spill to the stack. Its arguments are: + * + * r3: output bytes + * r4: 32-byte key input + * r5: 8-byte counter input/output (saved on stack) + * r6: number of 64-byte blocks to write to output + * + * r0: counter of blocks (initialised with r6) + * r4: Value '4' after key has been read. + * r5-r12: key + * r14-r15: counter + * r16-r31: state + */ +SYM_FUNC_START(__arch_chacha20_blocks_nostack) + stwu r1, -96(r1) + stw r5, 20(r1) + stmw r14, 24(r1) + li r31, 4 + LWZX_LE r14, 0, r5 + LWZX_LE r15, r31, r5 + mr r0, r6 + + lwz r5, 0(r4) + lwz r6, 4(r4) + lwz r7, 8(r4) + lwz r8, 12(r4) + lwz r9, 16(r4) + lwz r10, 20(r4) + lwz r11, 24(r4) + lwz r12, 28(r4) + + li r4, 4 +.Lblock: + li r31, 10 + + lis r16, 0x6170 + lis r17, 0x3320 + lis r18, 0x7962 + lis r19, 0x6b20 + addi r16, r16, 0x7865 + addi r17, r17, 0x646e + addi r18, r18, 0x2d32 + addi r19, r19, 0x6574 + + mtctr r31 + + mr r20, r5 + mr r21, r6 + mr r22, r7 + mr r23, r8 + mr r24, r9 + mr r25, r10 + mr r26, r11 + mr r27, r12 + + mr r28, r14 + mr r29, r15 + li r30, 0 + li r31, 0 + +.Lpermute: + QUARTERROUND4( 0, 4, 8,12, 1, 5, 9,13, 2, 6,10,14, 3, 7,11,15) + QUARTERROUND4( 0, 5,10,15, 1, 6,11,12, 2, 7, 8,13, 3, 4, 9,14) + + bdnz .Lpermute + + addis r16, r16, 0x6170 + addis r17, r17, 0x3320 + addis r18, r18, 0x7962 + addis r19, r19, 0x6b20 + addi r16, r16, 0x7865 + addi r17, r17, 0x646e + addi r18, r18, 0x2d32 + addi r19, r19, 0x6574 + + add r20, r20, r5 + add r21, r21, r6 + add r22, r22, r7 + add r23, r23, r8 + add r24, r24, r9 + add r25, r25, r10 + add r26, r26, r11 + add r27, r27, r12 + + add r28, r28, r14 + add r29, r29, r15 + + STWX_LE r16, 0, r3 + STWX_LE r17, r4, r3 + addi r3, r3, 8 + STWX_LE r18, 0, r3 + STWX_LE r19, r4, r3 + addi r3, r3, 8 + STWX_LE r20, 0, r3 + STWX_LE r21, r4, r3 + addi r3, r3, 8 + STWX_LE r22, 0, r3 + STWX_LE r23, r4, r3 + addi r3, r3, 8 + STWX_LE r24, 0, r3 + STWX_LE r25, r4, r3 + addi r3, r3, 8 + STWX_LE r26, 0, r3 + STWX_LE r27, r4, r3 + addi r3, r3, 8 + STWX_LE r28, 0, r3 + STWX_LE r29, r4, r3 + addi r3, r3, 8 + STWX_LE r30, 0, r3 + STWX_LE r31, r4, r3 + addi r3, r3, 8 + + addic r14, r14, 1 + addze r15, r15 + + subic. r0, r0, 1 /* subi. can't use r0 as source */ + bne .Lblock + + lwz r5, 20(r1) + STWX_LE r14, 0, r5 + STWX_LE r15, r4, r5 + + li r6, 0 + li r7, 0 + li r8, 0 + li r9, 0 + li r10, 0 + li r11, 0 + li r12, 0 + + lmw r14, 24(r1) + addi r1, r1, 96 + blr +SYM_FUNC_END(__arch_chacha20_blocks_nostack) diff --git a/arch/powerpc/kernel/vdso/vgetrandom.c b/arch/powerpc/kernel/vdso/vgetrandom.c new file mode 100644 index 000000000000..5f855d45fb7b --- /dev/null +++ b/arch/powerpc/kernel/vdso/vgetrandom.c @@ -0,0 +1,14 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Powerpc userspace implementation of getrandom() + * + * Copyright (C) 2024 Christophe Leroy , CS GROUP France + */ +#include +#include + +ssize_t __c_kernel_getrandom(void *buffer, size_t len, unsigned int flags, void *opaque_state, + size_t opaque_len, const struct vdso_rng_data *vd) +{ + return __cvdso_getrandom_data(vd, buffer, len, flags, opaque_state, opaque_len); +} diff --git a/tools/arch/powerpc/vdso b/tools/arch/powerpc/vdso new file mode 120000 index 000000000000..d31004bf8f55 --- /dev/null +++ b/tools/arch/powerpc/vdso @@ -0,0 +1 @@ +../../../arch/powerpc/kernel/vdso/ \ No newline at end of file diff --git a/tools/testing/selftests/vDSO/Makefile b/tools/testing/selftests/vDSO/Makefile index e21e78aae24d..e57dcefa1988 100644 --- a/tools/testing/selftests/vDSO/Makefile +++ b/tools/testing/selftests/vDSO/Makefile @@ -14,6 +14,10 @@ ifeq ($(uname_M),x86_64) TEST_GEN_PROGS += vdso_test_getrandom TEST_GEN_PROGS += vdso_test_chacha endif +ifeq ($(ARCH),powerpc) +TEST_GEN_PROGS += vdso_test_getrandom +TEST_GEN_PROGS += vdso_test_chacha +endif CFLAGS := -std=gnu99 From patchwork Fri Aug 30 15:57:09 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christophe Leroy X-Patchwork-Id: 13785301 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4D482CA1005 for ; Fri, 30 Aug 2024 15:57:46 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D23BE6B0190; Fri, 30 Aug 2024 11:57:45 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id CD3E26B0191; Fri, 30 Aug 2024 11:57:45 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B4D6C6B0192; Fri, 30 Aug 2024 11:57:45 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 93DF36B0190 for ; Fri, 30 Aug 2024 11:57:45 -0400 (EDT) Received: from smtpin30.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 30344806C3 for ; Fri, 30 Aug 2024 15:57:45 +0000 (UTC) X-FDA: 82509367290.30.D146FA5 Received: from pegase2.c-s.fr (pegase2.c-s.fr [93.17.235.10]) by imf05.hostedemail.com (Postfix) with ESMTP id F2814100018 for ; Fri, 30 Aug 2024 15:57:42 +0000 (UTC) Authentication-Results: imf05.hostedemail.com; dkim=none; spf=pass (imf05.hostedemail.com: domain of christophe.leroy@csgroup.eu designates 93.17.235.10 as permitted sender) smtp.mailfrom=christophe.leroy@csgroup.eu; dmarc=pass (policy=quarantine) header.from=csgroup.eu ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1725033391; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=exIUXvCbe1yE/siYQUvjG/FTEHXUWA679+0k3qDVN6U=; b=CO/t6g/5+orbm30y87YHv5hKygsLN8HY61HGTGGee/Iv5/ZJAQKO9gbURVBhpolu7VQEtl GfhZw0fo93dr/3Qf9sCPeSbvKcwY/NUzs5MrBjcKVgfVy89Bd0rnflrzHNI66WVOJMWuWF 2LiLzuB0x/ReDUGM3qAB1il4srsHn8w= ARC-Authentication-Results: i=1; imf05.hostedemail.com; dkim=none; spf=pass (imf05.hostedemail.com: domain of christophe.leroy@csgroup.eu designates 93.17.235.10 as permitted sender) smtp.mailfrom=christophe.leroy@csgroup.eu; dmarc=pass (policy=quarantine) header.from=csgroup.eu ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1725033391; a=rsa-sha256; cv=none; b=xKYa+9wD1zqJHUGcli3coduR8CFMVCdSKUA0ARbzoZTegJgP87PqCn0DAyGwlAWlclseRO //jjrUBsDEjuHqGv7m8jDS3YvnP/Zr26cIEdJlodDrRIwZzESCK3ySPnklbNw5qNsVavnd Wh05HTEyCfXRhcORdSM2hN+BvOm4s2g= Received: from localhost (mailhub3.si.c-s.fr [172.26.127.67]) by localhost (Postfix) with ESMTP id 4WwN820xV8z9sSS; Fri, 30 Aug 2024 17:57:22 +0200 (CEST) X-Virus-Scanned: amavisd-new at c-s.fr Received: from pegase2.c-s.fr ([172.26.127.65]) by localhost (pegase2.c-s.fr [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 9BHKF7MYlO9x; Fri, 30 Aug 2024 17:57:22 +0200 (CEST) Received: from messagerie.si.c-s.fr (messagerie.si.c-s.fr [192.168.25.192]) by pegase2.c-s.fr (Postfix) with ESMTP id 4WwN816PBbz9sS7; Fri, 30 Aug 2024 17:57:21 +0200 (CEST) Received: from localhost (localhost [127.0.0.1]) by messagerie.si.c-s.fr (Postfix) with ESMTP id C6DEA8B764; Fri, 30 Aug 2024 17:57:21 +0200 (CEST) X-Virus-Scanned: amavisd-new at c-s.fr Received: from messagerie.si.c-s.fr ([127.0.0.1]) by localhost (messagerie.si.c-s.fr [127.0.0.1]) (amavisd-new, port 10023) with ESMTP id uAS6-PSNHXJa; Fri, 30 Aug 2024 17:57:21 +0200 (CEST) Received: from PO20335.idsi0.si.c-s.fr (unknown [192.168.234.133]) by messagerie.si.c-s.fr (Postfix) with ESMTP id C196F8B794; Fri, 30 Aug 2024 17:57:20 +0200 (CEST) From: Christophe Leroy To: Andrew Morton , Steven Rostedt , Masami Hiramatsu , Mathieu Desnoyers , Michael Ellerman , Nicholas Piggin , Naveen N Rao , Nathan Chancellor , Nick Desaulniers , Bill Wendling , Justin Stitt , Shuah Khan , "Jason A . Donenfeld" Cc: Christophe Leroy , linux-kernel@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-kselftest@vger.kernel.org, llvm@lists.linux.dev, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, linux-trace-kernel@vger.kernel.org, Adhemerval Zanella , Xi Ruoyao Subject: [PATCH v3 5/5] powerpc/vdso: Wire up getrandom() vDSO implementation on PPC64 Date: Fri, 30 Aug 2024 17:57:09 +0200 Message-ID: <8dacc4e43ca17ff5394474a38faf3d1ed04f1fe3.1725031952.git.christophe.leroy@csgroup.eu> X-Mailer: git-send-email 2.44.0 In-Reply-To: References: MIME-Version: 1.0 X-Developer-Signature: v=1; a=ed25519-sha256; t=1725033425; l=8227; i=christophe.leroy@csgroup.eu; s=20211009; h=from:subject:message-id; bh=RZqSySqYFylmyuA5iRHusRb2CMpC6wGsXem6AUNbLuM=; b=Q9SndYRLYnz+rNPgHPdAY1Q+AbqC22AyVLkdSzp8bpb0Yac03csS5glWQQ8l4Gs2G4Bzqnsah ijB2x73ukGrB9RVsjFHt49/QhhNSST/8oUlPNWjD5tn1AQUF4B7gX6v X-Developer-Key: i=christophe.leroy@csgroup.eu; a=ed25519; pk=HIzTzUj91asvincQGOFx6+ZF5AoUuP9GdOtQChs7Mm0= X-Stat-Signature: bnwkkbyc1snoudsyjeaqqic1genmtjzj X-Rspam-User: X-Rspamd-Queue-Id: F2814100018 X-Rspamd-Server: rspam02 X-HE-Tag: 1725033462-599203 X-HE-Meta: U2FsdGVkX1+g/ST4nuRigZvMLQbMxSPw6B0ZC7ph7GqRsq00UjddYl5fakwPNGOHM9QvrZLse2j9TulJ0HAIews0avsEUQ/Cjj2KHOqIbXr7Vaw7IqOuuQKLrSe4gflBOaBNAnfnApfvoi0WPJV8w0Im/fggb9jF6rjqywnlDjYGJDbPQmtQKHN8Egup8y9xmpglRp3IYhLrHigKqGl3JYmGhsjJZRKEVBNuOKdbWdvT50TYUXaMMAt1WbLj1aMyAzLjTaFx8kCmU3bs26w31j47/a2obxRO8mJtJiLtnfcfNRA7+EwYsxe1wWhKQq/r9Hf1jJcMIZ88VRyyNLZkJ5ApDZ9rvxrHuZNcKLpnv8RSKRgO2ElVwS9kG1BQ1pAtEC5s0Wg1vkZjsuqzLSjXh6XbBgeHHHc+/Vv3j4bXHQYl8CbsuOHm85jxdTWah3OSZlP+u1tPfZYOKWZ6f8TykDVQfkw8YP1WW8nx5iO/8nOBOln8UlBFxiiejldDeccIb/kv1qLV7aPrL3pgxfNramzqBbmYsDrWyf41c6zc6R0+skC/0sYbD7gvE5+HnodFXYfAfdguujg3mzL/b9L2XIBba65J1BnvBOukFAi8LgQc9IglpirlcpFl4Q8EYxwlfGsQL9J5s0DT1y20c/c/cjr4RH8Xu0YNE6C6Eu5ysKAP5Jyo21o1ogJYqQONv3aGEWwu5K2pHMztVf8qStAxIQi/WLg80ihJci3d+thoVqAmVrI4zE2RxRRgWOS9fwGhUdKmWq283gJeZ5MwxE4cuJ3GhISZR9IzSfN/IKnSZbIRSsmlWax365tubDL0e03NSSPRr5oVHX5+07WO7wDuAhN9ozl3LrTgG2v7U049yKSb/cn7l9OdsHpoUG2JqHZAYvYCSQD4rADrBUw6+8g2WJ0/5YAtcdQa0Xlzfl6Hw5Gg9itNf47YxwuAmwK3kMVTpEEyh8o3+LT53IEXLBz TnAth178 /mrGL19BGGX4cx/vx7PGXzGEYEnszSkgSvJ6k94NV0fTeipkd9Tqwm+rwamlyDKy+xAK34+jB7XqKZ2RS2jqE08aq5QHBpVlQr1tm6tEVNkZJi79e2uwcL/zGzg+BqUQ4Icd8 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Extend getrandom() vDSO implementation to powerpc64. Tested on QEMU on both ppc64_defconfig and ppc64le_defconfig. The results are not precise as it is QEMU on an x86 laptop, but no need to be precise to see the benefit. ~ # ./vdso_test_getrandom bench-single vdso: 25000000 times in 4.977777162 seconds libc: 25000000 times in 75.516749981 seconds syscall: 25000000 times in 86.842242014 seconds ~ # ./vdso_test_getrandom bench-single vdso: 25000000 times in 6.473814156 seconds libc: 25000000 times in 73.875109463 seconds syscall: 25000000 times in 71.805066229 seconds Signed-off-by: Christophe Leroy --- v3: New (split out of previous patch) --- arch/powerpc/Kconfig | 2 +- arch/powerpc/include/asm/mman.h | 2 +- arch/powerpc/kernel/vdso/Makefile | 10 +- arch/powerpc/kernel/vdso/getrandom.S | 8 ++ arch/powerpc/kernel/vdso/vdso64.lds.S | 1 + arch/powerpc/kernel/vdso/vgetrandom-chacha.S | 98 ++++++++++++++++++++ 6 files changed, 116 insertions(+), 5 deletions(-) diff --git a/arch/powerpc/Kconfig b/arch/powerpc/Kconfig index 54b270ef18b1..b45452ac4a73 100644 --- a/arch/powerpc/Kconfig +++ b/arch/powerpc/Kconfig @@ -311,7 +311,7 @@ config PPC select SYSCTL_EXCEPTION_TRACE select THREAD_INFO_IN_TASK select TRACE_IRQFLAGS_SUPPORT - select VDSO_GETRANDOM if PPC32 + select VDSO_GETRANDOM # # Please keep this list sorted alphabetically. # diff --git a/arch/powerpc/include/asm/mman.h b/arch/powerpc/include/asm/mman.h index 17a77d47ed6d..42a51a993d94 100644 --- a/arch/powerpc/include/asm/mman.h +++ b/arch/powerpc/include/asm/mman.h @@ -6,7 +6,7 @@ #include -#ifdef CONFIG_PPC64 +#if defined(CONFIG_PPC64) && !defined(BUILD_VDSO) #include #include diff --git a/arch/powerpc/kernel/vdso/Makefile b/arch/powerpc/kernel/vdso/Makefile index fa0b9b3b51af..56fb1633529a 100644 --- a/arch/powerpc/kernel/vdso/Makefile +++ b/arch/powerpc/kernel/vdso/Makefile @@ -9,6 +9,7 @@ obj-vdso32 = sigtramp32-32.o gettimeofday-32.o datapage-32.o cacheflush-32.o not obj-vdso64 = sigtramp64-64.o gettimeofday-64.o datapage-64.o cacheflush-64.o note-64.o getcpu-64.o obj-vdso32 += getrandom-32.o vgetrandom-chacha-32.o +obj-vdso64 += getrandom-64.o vgetrandom-chacha-64.o ifneq ($(c-gettimeofday-y),) CFLAGS_vgettimeofday-32.o += -include $(c-gettimeofday-y) @@ -21,6 +22,7 @@ endif ifneq ($(c-getrandom-y),) CFLAGS_vgetrandom-32.o += -include $(c-getrandom-y) + CFLAGS_vgetrandom-64.o += -include $(c-getrandom-y) $(call cc-option, -ffixed-r30) endif # Build rules @@ -34,10 +36,10 @@ endif targets := $(obj-vdso32) vdso32.so.dbg vgettimeofday-32.o vgetrandom-32.o targets += crtsavres-32.o obj-vdso32 := $(addprefix $(obj)/, $(obj-vdso32)) -targets += $(obj-vdso64) vdso64.so.dbg vgettimeofday-64.o +targets += $(obj-vdso64) vdso64.so.dbg vgettimeofday-64.o vgetrandom-64.o obj-vdso64 := $(addprefix $(obj)/, $(obj-vdso64)) -ccflags-y := -fno-common -fno-builtin +ccflags-y := -fno-common -fno-builtin -DBUILD_VDSO ccflags-y += $(DISABLE_LATENT_ENTROPY_PLUGIN) ccflags-y += $(call cc-option, -fno-stack-protector) ccflags-y += -DDISABLE_BRANCH_PROFILING @@ -71,7 +73,7 @@ CPPFLAGS_vdso64.lds += -P -C # link rule for the .so file, .lds has to be first $(obj)/vdso32.so.dbg: $(obj)/vdso32.lds $(obj-vdso32) $(obj)/vgettimeofday-32.o $(obj)/vgetrandom-32.o $(obj)/crtsavres-32.o FORCE $(call if_changed,vdso32ld_and_check) -$(obj)/vdso64.so.dbg: $(obj)/vdso64.lds $(obj-vdso64) $(obj)/vgettimeofday-64.o FORCE +$(obj)/vdso64.so.dbg: $(obj)/vdso64.lds $(obj-vdso64) $(obj)/vgettimeofday-64.o $(obj)/vgetrandom-64.o FORCE $(call if_changed,vdso64ld_and_check) # assembly rules for the .S files @@ -87,6 +89,8 @@ $(obj-vdso64): %-64.o: %.S FORCE $(call if_changed_dep,vdso64as) $(obj)/vgettimeofday-64.o: %-64.o: %.c FORCE $(call if_changed_dep,cc_o_c) +$(obj)/vgetrandom-64.o: %-64.o: %.c FORCE + $(call if_changed_dep,cc_o_c) # Generate VDSO offsets using helper script gen-vdso32sym := $(src)/gen_vdso32_offsets.sh diff --git a/arch/powerpc/kernel/vdso/getrandom.S b/arch/powerpc/kernel/vdso/getrandom.S index 21773ef3fc1d..a957cd2b2b03 100644 --- a/arch/powerpc/kernel/vdso/getrandom.S +++ b/arch/powerpc/kernel/vdso/getrandom.S @@ -27,10 +27,18 @@ .cfi_adjust_cfa_offset PPC_MIN_STKFRM PPC_STL r0, PPC_MIN_STKFRM + PPC_LR_STKOFF(r1) .cfi_rel_offset lr, PPC_MIN_STKFRM + PPC_LR_STKOFF +#ifdef __powerpc64__ + PPC_STL r2, PPC_MIN_STKFRM + STK_GOT(r1) + .cfi_rel_offset r2, PPC_MIN_STKFRM + STK_GOT +#endif get_datapage r8 addi r8, r8, VDSO_RNG_DATA_OFFSET bl CFUNC(DOTSYM(\funct)) PPC_LL r0, PPC_MIN_STKFRM + PPC_LR_STKOFF(r1) +#ifdef __powerpc64__ + PPC_LL r2, PPC_MIN_STKFRM + STK_GOT(r1) + .cfi_restore r2 +#endif cmpwi r3, 0 mtlr r0 addi r1, r1, 2 * PPC_MIN_STKFRM diff --git a/arch/powerpc/kernel/vdso/vdso64.lds.S b/arch/powerpc/kernel/vdso/vdso64.lds.S index 400819258c06..9481e4b892ed 100644 --- a/arch/powerpc/kernel/vdso/vdso64.lds.S +++ b/arch/powerpc/kernel/vdso/vdso64.lds.S @@ -123,6 +123,7 @@ VERSION __kernel_sigtramp_rt64; __kernel_getcpu; __kernel_time; + __kernel_getrandom; local: *; }; diff --git a/arch/powerpc/kernel/vdso/vgetrandom-chacha.S b/arch/powerpc/kernel/vdso/vgetrandom-chacha.S index a3475b9e8c10..c9d4c5b38242 100644 --- a/arch/powerpc/kernel/vdso/vgetrandom-chacha.S +++ b/arch/powerpc/kernel/vdso/vgetrandom-chacha.S @@ -79,12 +79,59 @@ * r16-r31: state */ SYM_FUNC_START(__arch_chacha20_blocks_nostack) +#ifdef __powerpc64__ + std r5, -216(r1) + + std r14, -144(r1) + std r15, -136(r1) + std r16, -128(r1) + std r17, -120(r1) + std r18, -112(r1) + std r19, -104(r1) + std r20, -96(r1) + std r21, -88(r1) + std r22, -80(r1) + std r23, -72(r1) + std r24, -64(r1) + std r25, -56(r1) + std r26, -48(r1) + std r27, -40(r1) + std r28, -32(r1) + std r29, -24(r1) + std r30, -16(r1) + std r31, -8(r1) +#else stwu r1, -96(r1) stw r5, 20(r1) +#if defined(CONFIG_CPU_BIG_ENDIAN) stmw r14, 24(r1) +#else + stw r14, 24(r1) + stw r15, 28(r1) + stw r16, 32(r1) + stw r17, 36(r1) + stw r18, 40(r1) + stw r19, 44(r1) + stw r20, 48(r1) + stw r21, 52(r1) + stw r22, 56(r1) + stw r23, 60(r1) + stw r24, 64(r1) + stw r25, 68(r1) + stw r26, 72(r1) + stw r27, 76(r1) + stw r28, 80(r1) + stw r29, 84(r1) + stw r30, 88(r1) + stw r31, 92(r1) +#endif +#endif li r31, 4 LWZX_LE r14, 0, r5 LWZX_LE r15, r31, r5 +#ifdef __powerpc64__ + rldimi r14, r15, 32, 0 +#endif mr r0, r6 lwz r5, 0(r4) @@ -177,13 +224,22 @@ SYM_FUNC_START(__arch_chacha20_blocks_nostack) STWX_LE r31, r4, r3 addi r3, r3, 8 +#ifdef __powerpc64__ + addi r14, r14, 1 + srdi r15, r14, 32 +#else addic r14, r14, 1 addze r15, r15 +#endif subic. r0, r0, 1 /* subi. can't use r0 as source */ bne .Lblock +#ifdef __powerpc64__ + ld r5, -216(r1) +#else lwz r5, 20(r1) +#endif STWX_LE r14, 0, r5 STWX_LE r15, r4, r5 @@ -195,7 +251,49 @@ SYM_FUNC_START(__arch_chacha20_blocks_nostack) li r11, 0 li r12, 0 +#ifdef __powerpc64__ + ld r14, -144(r1) + ld r15, -136(r1) + ld r16, -128(r1) + ld r17, -120(r1) + ld r18, -112(r1) + ld r19, -104(r1) + ld r20, -96(r1) + ld r21, -88(r1) + ld r22, -80(r1) + ld r23, -72(r1) + ld r24, -64(r1) + ld r25, -56(r1) + ld r26, -48(r1) + ld r27, -40(r1) + ld r28, -32(r1) + ld r29, -24(r1) + ld r30, -16(r1) + ld r31, -8(r1) +#else +#if defined(CONFIG_CPU_BIG_ENDIAN) lmw r14, 24(r1) +#else + lwz r14, 24(r1) + lwz r15, 28(r1) + lwz r16, 32(r1) + lwz r17, 36(r1) + lwz r18, 40(r1) + lwz r19, 44(r1) + lwz r20, 48(r1) + lwz r21, 52(r1) + lwz r22, 56(r1) + lwz r23, 60(r1) + lwz r24, 64(r1) + lwz r25, 68(r1) + lwz r26, 72(r1) + lwz r27, 76(r1) + lwz r28, 80(r1) + lwz r29, 84(r1) + lwz r30, 88(r1) + lwz r31, 92(r1) +#endif addi r1, r1, 96 +#endif blr SYM_FUNC_END(__arch_chacha20_blocks_nostack)