From patchwork Tue Jul 25 06:14:48 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Enze Li X-Patchwork-Id: 13325911 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id E9219EB64DD for ; Tue, 25 Jul 2023 06:15:31 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id EB29B8E0003; Tue, 25 Jul 2023 02:15:29 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id E63F8900003; Tue, 25 Jul 2023 02:15:29 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id CDE8D8E0005; Tue, 25 Jul 2023 02:15:29 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id BA72D8E0003 for ; Tue, 25 Jul 2023 02:15:29 -0400 (EDT) Received: from smtpin30.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 80B1040CEA for ; Tue, 25 Jul 2023 06:15:29 +0000 (UTC) X-FDA: 81049122378.30.9FB14B0 Received: from mailgw.kylinos.cn (mailgw.kylinos.cn [124.126.103.232]) by imf07.hostedemail.com (Postfix) with ESMTP id 4613340003 for ; Tue, 25 Jul 2023 06:15:22 +0000 (UTC) Authentication-Results: imf07.hostedemail.com; dkim=none; spf=pass (imf07.hostedemail.com: domain of lienze@kylinos.cn designates 124.126.103.232 as permitted sender) smtp.mailfrom=lienze@kylinos.cn; dmarc=none ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1690265727; a=rsa-sha256; cv=none; b=bBji6qJAjZAPsYwoD9giO3cvKi/DZ1gcNF0hH/wPvSVr1gZAVwlRZlwe3cpmsdhb6yNqGN H0u+M8vXVEL0P/fl8vzMvYT/RGzeftCnZKOarkBZk7iZ8KSnHcWzbr/zmPzEYUGaGdfC3b yfdYPWYWcWY5IZLty57plldK2yqyn9w= ARC-Authentication-Results: i=1; imf07.hostedemail.com; dkim=none; spf=pass (imf07.hostedemail.com: domain of lienze@kylinos.cn designates 124.126.103.232 as permitted sender) smtp.mailfrom=lienze@kylinos.cn; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1690265727; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=IhGnxe2P5+ULV3kw8CSrg/6QSRV8cYfR7e12dyv3j8E=; b=6oq+5OuV33/gOnOUvmOWPA5m9cyUeXoRfuwZ8BueBDOBQNfnKiwC+0NgX/P9yEbqYi15d8 pdhmlxCn+7xpZqmbFIj+SzwrBLOCKHTVhwKeygoLjCQCHeDdA4tXU9JoMFJbWkoTPVB9Gr LmLdTaoPhTkpscfRJhy/U4VTS21bAJA= X-UUID: 06f546de48c44ad099634202a5a5558d-20230725 X-CID-P-RULE: Release_Ham X-CID-O-INFO: VERSION:1.1.28,REQID:5ca60565-7f43-4364-9457-75cadd98427e,IP:15, URL:0,TC:0,Content:0,EDM:0,RT:0,SF:-15,FILE:0,BULK:0,RULE:Release_Ham,ACTI ON:release,TS:0 X-CID-INFO: VERSION:1.1.28,REQID:5ca60565-7f43-4364-9457-75cadd98427e,IP:15,UR L:0,TC:0,Content:0,EDM:0,RT:0,SF:-15,FILE:0,BULK:0,RULE:Release_Ham,ACTION :release,TS:0 X-CID-META: VersionHash:176cd25,CLOUDID:cebc4ad2-cd77-4e67-bbfd-aa4eaace762f,B ulkID:230725141514N0X3POC0,BulkQuantity:0,Recheck:0,SF:44|38|24|17|19|102, TC:nil,Content:0,EDM:-3,IP:-2,URL:1,File:nil,Bulk:nil,QS:nil,BEC:nil,COL:0 ,OSI:0,OSA:0,AV:0,LES:1,SPR:NO,DKR:0,DKP:0 X-CID-BVR: 0 X-CID-BAS: 0,_,0,_ X-CID-FACTOR: TF_CID_SPAM_SNR,TF_CID_SPAM_FAS,TF_CID_SPAM_FSD,TF_CID_SPAM_FSI, TF_CID_SPAM_ULS X-UUID: 06f546de48c44ad099634202a5a5558d-20230725 X-User: lienze@kylinos.cn Received: from ubuntu.. [(39.156.73.12)] by mailgw (envelope-from ) (Generic MTA with TLSv1.2 ECDHE-RSA-AES256-GCM-SHA384 256/256) with ESMTP id 1695958714; Tue, 25 Jul 2023 14:15:11 +0800 From: Enze Li To: chenhuacai@kernel.org, kernel@xen0n.name, loongarch@lists.linux.dev, glider@google.com, elver@google.com, akpm@linux-foundation.org, kasan-dev@googlegroups.com, linux-mm@kvack.org Cc: zhangqing@loongson.cn, yangtiezhu@loongson.cn, dvyukov@google.com, Enze Li Subject: [PATCH 1/4 v2] LoongArch: mm: Add page table mapped mode support Date: Tue, 25 Jul 2023 14:14:48 +0800 Message-Id: <20230725061451.1231480-2-lienze@kylinos.cn> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230725061451.1231480-1-lienze@kylinos.cn> References: <20230725061451.1231480-1-lienze@kylinos.cn> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: 4613340003 X-Stat-Signature: r81emse4ax65j6jwh4ixfofsfzpkuu34 X-HE-Tag: 1690265722-681277 X-HE-Meta: U2FsdGVkX1+s8179J8OTt5LQAAFY1iKLwVVyQezHyQrQ6mqBK413V76nTp/S9QBt5PjmKWcMzhbIFu+myp6bUV4Sej44OK9uQCUkgYYhYpGRMmECi24bi26Z2HJBv1ATyMK8kuM1A8JEJDtO0jfgl22Th0GSp81F+E9+EeQ34ZVf+RFl+ER3zW7TViDmOJ93gCUyOk22y0EYIBJeZKxmX8UaF7daSd6zxCo3QZsEFPcYzRyxIgxJvwrWthGxdyG7zs03GGtx5pgNeNHae1Nnh7RJJjI86BLvZthzaUv2bKo+Iq1WDPno+K9uQKoxhiwV5gTIwxlkqPXB6vCVf1Tbqsy8D9pgou8C25nrJlIL4MEsg06adKs2Rc/7o2UXXqtD9PgSqABMETiTvSHvs8YY0SdxEZytesEI9KMO/mO870ozrYF6Ej7VjeNjLlyuQX1kaJ9cpn3wEvZ1Q1M/usKBAbM9A2ZUpsVXHutFrL0x0XotC6NgUNQ2nn8HhmFLJ6JqiTLAZhbmJedlm/yBBe1yj0xvqmL808obyKx5XfqUzfD1/9pAgvsyUFtlbnzoBN42VWhRLPkypea6f3qoNDmfhEmoNlR9XGkEVyiihrdSN9coCGSkCAs6rdzpqAjwyNY3Nkj9CCJUPjG2NV952qWznWZMx5WZgzqng/aE8+QAuZIH9mgGBZRZ8k5QG8fwQ506t2ULkHsYvIqVZub64z0Por37Nf46Pn2/iJ6rycrLr2DP3eXeeEvobCEZtEo2wz4ZdA2JjkYQZIzIoNcrXp5KLv8zypbOXbQ7rSM8/Dlrf3WFFG9I68JCAKjmLGi8NSZI95+/5gITfXXZ6oXaQA/ltwR2Wz96EElOhbpOo6ezU52fYeV0q8VSNTDiHPdGQEEQwZMrK/PvxRjkH4LnUo3X38TYZmBDWBX5B9QSYJjdNgC+8l5MgzW8+3WuvTBdKuUsXWbW+A79RJwE8U5HFse +sQajrBI L+aR/lNnh9bE+yP7tCXmQYEHnB0LKZG9i7q0rDvcHUeKYYO6X1o1K9Y0wo0laSNp8c+cEzNUjRCwRLdkVOJprBdiN8ZOqeD+H6Y5D1pxzk03ODsfm7AIEaNI0YPhrI4+t0FK4 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: According to LoongArch documentation online, there are two types of address translation modes: direct mapped address translation mode (direct mapped mode) and page table mapped address translation mode (page table mapped mode). Currently, the upstream kernel only supports direct mapped mode. This patch adds a function that determines whether page table mapped mode should be used, and also adds the corresponding handler functions for both modes. For more details on the two modes, see [1]. [1] https://loongson.github.io/LoongArch-Documentation/LoongArch-Vol1-EN.html#virtual-address-space-and-address-translation-mode Signed-off-by: Enze Li --- arch/loongarch/include/asm/page.h | 19 ++++++++++++++++++- arch/loongarch/include/asm/pgtable.h | 2 ++ arch/loongarch/mm/pgtable.c | 6 ++++++ 3 files changed, 26 insertions(+), 1 deletion(-) diff --git a/arch/loongarch/include/asm/page.h b/arch/loongarch/include/asm/page.h index 26e8dccb6619..e43a2385b2cd 100644 --- a/arch/loongarch/include/asm/page.h +++ b/arch/loongarch/include/asm/page.h @@ -32,6 +32,7 @@ #include #include +#include /* * It's normally defined only for FLATMEM config but it's @@ -84,7 +85,23 @@ typedef struct { unsigned long pgprot; } pgprot_t; #define sym_to_pfn(x) __phys_to_pfn(__pa_symbol(x)) #define virt_to_pfn(kaddr) PFN_DOWN(PHYSADDR(kaddr)) -#define virt_to_page(kaddr) pfn_to_page(virt_to_pfn(kaddr)) + +static inline bool is_tlb_addr(unsigned long kaddr) +{ + if (unlikely((kaddr & GENMASK(BITS_PER_LONG - 1, cpu_vabits)) == + GENMASK(BITS_PER_LONG - 1, cpu_vabits))) + return true; + return false; +} + +#define dwm_virt_to_page(kaddr) pfn_to_page(virt_to_pfn(kaddr)) + +#define virt_to_page(kaddr) \ +({ \ + is_tlb_addr((unsigned long)kaddr) ? \ + tlb_virt_to_page((unsigned long)kaddr) : \ + dwm_virt_to_page((unsigned long)kaddr); \ +}) extern int __virt_addr_valid(volatile void *kaddr); #define virt_addr_valid(kaddr) __virt_addr_valid((volatile void *)(kaddr)) diff --git a/arch/loongarch/include/asm/pgtable.h b/arch/loongarch/include/asm/pgtable.h index 38afeb7dd58b..98a0c98de9d1 100644 --- a/arch/loongarch/include/asm/pgtable.h +++ b/arch/loongarch/include/asm/pgtable.h @@ -353,6 +353,8 @@ static inline void pte_clear(struct mm_struct *mm, unsigned long addr, pte_t *pt #define PMD_T_LOG2 (__builtin_ffs(sizeof(pmd_t)) - 1) #define PTE_T_LOG2 (__builtin_ffs(sizeof(pte_t)) - 1) +inline struct page *tlb_virt_to_page(unsigned long kaddr); + extern pgd_t swapper_pg_dir[]; extern pgd_t invalid_pg_dir[]; diff --git a/arch/loongarch/mm/pgtable.c b/arch/loongarch/mm/pgtable.c index 36a6dc0148ae..20e7425d235d 100644 --- a/arch/loongarch/mm/pgtable.c +++ b/arch/loongarch/mm/pgtable.c @@ -9,6 +9,12 @@ #include #include +inline struct page *tlb_virt_to_page(unsigned long kaddr) +{ + return pte_page(*virt_to_kpte(kaddr)); +} +EXPORT_SYMBOL_GPL(tlb_virt_to_page); + pgd_t *pgd_alloc(struct mm_struct *mm) { pgd_t *ret, *init; From patchwork Tue Jul 25 06:14:49 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Enze Li X-Patchwork-Id: 13325907 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id F3AD3EB64DD for ; Tue, 25 Jul 2023 06:15:24 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 62D066B007B; Tue, 25 Jul 2023 02:15:24 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 5D7458E0002; Tue, 25 Jul 2023 02:15:24 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 4299B6B007E; Tue, 25 Jul 2023 02:15:24 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 345B76B007B for ; Tue, 25 Jul 2023 02:15:24 -0400 (EDT) Received: from smtpin30.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id F3E891C9C33 for ; Tue, 25 Jul 2023 06:15:23 +0000 (UTC) X-FDA: 81049122126.30.19415D4 Received: from mailgw.kylinos.cn (mailgw.kylinos.cn [124.126.103.232]) by imf18.hostedemail.com (Postfix) with ESMTP id CCD8F1C000F for ; Tue, 25 Jul 2023 06:15:20 +0000 (UTC) Authentication-Results: imf18.hostedemail.com; dkim=none; spf=pass (imf18.hostedemail.com: domain of lienze@kylinos.cn designates 124.126.103.232 as permitted sender) smtp.mailfrom=lienze@kylinos.cn; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1690265722; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Zt1iNttuBXY1BSNtMsh+aChBU07iLpi+OG55pqc1lzQ=; b=fta5nXgS5V1tF5hLNnSvBVlnXSF/LuC1N/PunKUBR+TIQZCZGlhzGAY3npkt0zEueoCbCS nzNa3NAOetIPnMEQ7pjYQkeVptc2XsS3+e/aClg268o1Lw1Y5dbYslAAMcdRf9YYd6QeuD wPU2S5apWwpOKuzCZAEHmrunWN9ZwCE= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1690265722; a=rsa-sha256; cv=none; b=TgG28o3nde+f4jEVwBtK+QJsng968BygWaGUxq1vaNGbKdB+Vs+mV3UloADc4tPpFA+Meg sOgM//flLr3yr0m9FCi27U4M6znwjephzsNdeDFTbKEePw2EgUpWyxQ9f+9xOcTQTxGSDz U4+/+xjJWaIQSdmoBQANgzJMDIZ+FWY= ARC-Authentication-Results: i=1; imf18.hostedemail.com; dkim=none; spf=pass (imf18.hostedemail.com: domain of lienze@kylinos.cn designates 124.126.103.232 as permitted sender) smtp.mailfrom=lienze@kylinos.cn; dmarc=none X-UUID: 900e072e26d147c683372e81b498ccb9-20230725 X-CID-P-RULE: Release_Ham X-CID-O-INFO: VERSION:1.1.28,REQID:100611cf-fe0a-44aa-acb6-86579ba34183,IP:15, URL:0,TC:0,Content:0,EDM:0,RT:0,SF:-15,FILE:0,BULK:0,RULE:Release_Ham,ACTI ON:release,TS:0 X-CID-INFO: VERSION:1.1.28,REQID:100611cf-fe0a-44aa-acb6-86579ba34183,IP:15,UR L:0,TC:0,Content:0,EDM:0,RT:0,SF:-15,FILE:0,BULK:0,RULE:Release_Ham,ACTION :release,TS:0 X-CID-META: VersionHash:176cd25,CLOUDID:1d3b7fa0-0933-4333-8d4f-6c3c53ebd55b,B ulkID:230725141514IM3WVZYC,BulkQuantity:0,Recheck:0,SF:44|38|24|17|19|102, TC:nil,Content:0,EDM:-3,IP:-2,URL:0,File:nil,Bulk:nil,QS:nil,BEC:nil,COL:0 ,OSI:0,OSA:0,AV:0,LES:1,SPR:NO,DKR:0,DKP:0 X-CID-BVR: 0 X-CID-BAS: 0,_,0,_ X-CID-FACTOR: TF_CID_SPAM_SNR,TF_CID_SPAM_FAS,TF_CID_SPAM_FSD,TF_CID_SPAM_FSI X-UUID: 900e072e26d147c683372e81b498ccb9-20230725 X-User: lienze@kylinos.cn Received: from ubuntu.. [(39.156.73.12)] by mailgw (envelope-from ) (Generic MTA with TLSv1.2 ECDHE-RSA-AES256-GCM-SHA384 256/256) with ESMTP id 1616501915; Tue, 25 Jul 2023 14:15:11 +0800 From: Enze Li To: chenhuacai@kernel.org, kernel@xen0n.name, loongarch@lists.linux.dev, glider@google.com, elver@google.com, akpm@linux-foundation.org, kasan-dev@googlegroups.com, linux-mm@kvack.org Cc: zhangqing@loongson.cn, yangtiezhu@loongson.cn, dvyukov@google.com, Enze Li Subject: [PATCH 2/4 v2] LoongArch: Get stack without NMI when providing regs parameter Date: Tue, 25 Jul 2023 14:14:49 +0800 Message-Id: <20230725061451.1231480-3-lienze@kylinos.cn> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230725061451.1231480-1-lienze@kylinos.cn> References: <20230725061451.1231480-1-lienze@kylinos.cn> MIME-Version: 1.0 X-Stat-Signature: ioxhrfc9iw6eecaeztemjw4wyemt9154 X-Rspamd-Server: rspam10 X-Rspamd-Queue-Id: CCD8F1C000F X-Rspam-User: X-HE-Tag: 1690265720-750866 X-HE-Meta: U2FsdGVkX18mxe430Eo3f+/dYm3Bww9Xg3ylEu37DiBTKO+AJ71OWNv8HffAPmFRMy4UDGhAhXj/1R+ufyI+IB7O/PXA8szVpFpbIY6zbkl3+F8HqAJTxMnH0aRH4LaulLB+vBlOKhlf+SSRTxaNrw6NZFsAKjFxMyujk2lVaC01kzH+DFyw7qBHw42vT6pdjEV63R40yrCHNpD7U/8B3aNryeloR3NH+xUl5vrp2fhaTIhKfKCuLLTNA/ZFj+JHsnho75CKi0/D0vF4ejfcK4XjK28dhmBK9pUtn+HRip945Plf/hJ3wmEiNoa3KHwioby0T95dkAvyaNofj/REfLAVkSfxhrUx2kBjyqs3epKGG5DgRFayBJkwUtQ4tYeExdwP7ejiDc0R/KgPYlXleHCrXTg8ugxrhzXAtIuZsVWz46MUQuBFoOe3kKhD6s5If91H+AVveI+o3MF3UVpVEXSnKCAWsOZFmO/bcqwyUWmGyr77rUCMSSkZsmEISFqEsBRKIVWFc/H8pWqnD4npf7PgzwSb6ZnYynwoWhRNUBja0HiMRpMpD6uVIrq30Rckh6OLbcLO13y25lfiDVzkkJZOzzkIKf1/+1Hl3jutzHkG9Oh6s2Fssrq6IRGSQyvY3PcrZhgQ9lHP6xEq/0L2l4KDPF0vwpzeL5vPdURajlLcQg7SOmzKCVoSKeZsHMn/7xtiHIbD4vv74RMpg5ebxZDwUBYm0CYMLQzj5g9kZWG8Pcf+3OC7SnV24UvPHwtPTFurC5DoIH0BphOhpSTwrGI5z1zLOde4Y8mMQnmyvW6i6er/jYElERh65zT5Sj+l9Xtz/bNheJzrT2jEWNp5XIch48v18RG8Mt4Bg4WJNRFzTZVsYMsDaK0LsmTaveXj9nDMpwnALsX0/j+fxAspEa2Utjkxyx/LHVBpwdCEWzFcLkGwwA115Cf1wDAt7dSofDUPpC49fjyVLLSrKKg WTzu5Clo hz7da0RmKkDihlZgD0/u4ca5QMKqrPeTSlvpR X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Currently, arch_stack_walk() can only get the full stack information including NMI. This is because the implementation of arch_stack_walk() is forced to ignore the information passed by the regs parameter and use the current stack information instead. For some detection systems like KFENCE, only partial stack information is needed. In particular, the stack frame where the interrupt occurred. To support KFENCE, this patch modifies the implementation of the arch_stack_walk() function so that if this function is called with the regs argument passed, it retains all the stack information in regs and uses it to provide accurate information. Before the patch applied, I get, [ 1.531195 ] ================================================================== [ 1.531442 ] BUG: KFENCE: out-of-bounds read in stack_trace_save_regs+0x48/0x6c [ 1.531442 ] [ 1.531900 ] Out-of-bounds read at 0xffff800012267fff (1B left of kfence-#12): [ 1.532046 ] stack_trace_save_regs+0x48/0x6c [ 1.532169 ] kfence_report_error+0xa4/0x528 [ 1.532276 ] kfence_handle_page_fault+0x124/0x270 [ 1.532388 ] no_context+0x50/0x94 [ 1.532453 ] do_page_fault+0x1a8/0x36c [ 1.532524 ] tlb_do_page_fault_0+0x118/0x1b4 [ 1.532623 ] test_out_of_bounds_read+0xa0/0x1d8 [ 1.532745 ] kunit_generic_run_threadfn_adapter+0x1c/0x28 [ 1.532854 ] kthread+0x124/0x130 [ 1.532922 ] ret_from_kernel_thread+0xc/0xa4 With this patch applied, I get the correct stack information. [ 1.320220 ] ================================================================== [ 1.320401 ] BUG: KFENCE: out-of-bounds read in test_out_of_bounds_read+0xa8/0x1d8 [ 1.320401 ] [ 1.320898 ] Out-of-bounds read at 0xffff800012257fff (1B left of kfence-#10): [ 1.321134 ] test_out_of_bounds_read+0xa8/0x1d8 [ 1.321264 ] kunit_generic_run_threadfn_adapter+0x1c/0x28 [ 1.321392 ] kthread+0x124/0x130 [ 1.321459 ] ret_from_kernel_thread+0xc/0xa4 Signed-off-by: Enze Li --- arch/loongarch/kernel/stacktrace.c | 20 ++++++++++++++------ 1 file changed, 14 insertions(+), 6 deletions(-) diff --git a/arch/loongarch/kernel/stacktrace.c b/arch/loongarch/kernel/stacktrace.c index 2463d2fea21f..9dab30ae68ec 100644 --- a/arch/loongarch/kernel/stacktrace.c +++ b/arch/loongarch/kernel/stacktrace.c @@ -18,16 +18,24 @@ void arch_stack_walk(stack_trace_consume_fn consume_entry, void *cookie, struct pt_regs dummyregs; struct unwind_state state; - regs = &dummyregs; - if (task == current) { - regs->regs[3] = (unsigned long)__builtin_frame_address(0); - regs->csr_era = (unsigned long)__builtin_return_address(0); + if (regs) + memcpy(&dummyregs, regs, sizeof(*regs)); + else { + dummyregs.regs[3] = (unsigned long)__builtin_frame_address(0); + dummyregs.csr_era = (unsigned long)__builtin_return_address(0); + } } else { - regs->regs[3] = thread_saved_fp(task); - regs->csr_era = thread_saved_ra(task); + if (regs) + memcpy(&dummyregs, regs, sizeof(*regs)); + else { + dummyregs.regs[3] = thread_saved_fp(task); + dummyregs.csr_era = thread_saved_ra(task); + } } + regs = &dummyregs; + regs->regs[1] = 0; for (unwind_start(&state, task, regs); !unwind_done(&state) && !unwind_error(&state); unwind_next_frame(&state)) { From patchwork Tue Jul 25 06:14:50 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Enze Li X-Patchwork-Id: 13325908 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id E7713C41513 for ; Tue, 25 Jul 2023 06:15:25 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 895E28E0002; Tue, 25 Jul 2023 02:15:24 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 7856D6B007E; Tue, 25 Jul 2023 02:15:24 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 6258C900002; Tue, 25 Jul 2023 02:15:24 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 4D9976B007B for ; Tue, 25 Jul 2023 02:15:24 -0400 (EDT) Received: from smtpin29.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 1A40380C61 for ; Tue, 25 Jul 2023 06:15:24 +0000 (UTC) X-FDA: 81049122168.29.502F061 Received: from mailgw.kylinos.cn (mailgw.kylinos.cn [124.126.103.232]) by imf24.hostedemail.com (Postfix) with ESMTP id 30AAF180004 for ; Tue, 25 Jul 2023 06:15:20 +0000 (UTC) Authentication-Results: imf24.hostedemail.com; dkim=none; dmarc=none; spf=pass (imf24.hostedemail.com: domain of lienze@kylinos.cn designates 124.126.103.232 as permitted sender) smtp.mailfrom=lienze@kylinos.cn ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1690265722; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Ax87EcoYzEMFko+UngAuu4BzrMqvrK/QnibTDD/L0Eo=; b=tAOUKkWwbGMGz4g3kqHoVuPthgRw7uffh2yP5xojSpj3ODbOeLYTlxSo7TBQljyggzM24M AP0i25sTZDQ2leRpgd4dPcnGM1MBVwj3goQJQ6gahPtObWe1qZSisoo4qOAQOSgi2gs0Pu FPV2FypHt7HlCYJVhKjTrZ77x7m4V4c= ARC-Authentication-Results: i=1; imf24.hostedemail.com; dkim=none; dmarc=none; spf=pass (imf24.hostedemail.com: domain of lienze@kylinos.cn designates 124.126.103.232 as permitted sender) smtp.mailfrom=lienze@kylinos.cn ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1690265722; a=rsa-sha256; cv=none; b=EmT6B0m96n40RUZb19xrG9q60RTpohsofH67OS2Z04qZ5/XRsAio9T6QGg1rbY8tKwUzKn jo06ke5UbsPtmHq4eTwyr/tTjWqVRxyGt4KSlM9H8M3LiDTF3JV23+jdk9sn8CJmOdRrMQ yhvcLq9Nq8DTtzwxfhlVoJQw1ZmkvKU= X-UUID: 8e5b3e4a0453435d8408d3e21c20e6c4-20230725 X-CID-P-RULE: Release_Ham X-CID-O-INFO: VERSION:1.1.28,REQID:68b8ec71-7352-453b-87a7-33a33507e392,IP:15, URL:0,TC:0,Content:-5,EDM:0,RT:0,SF:-15,FILE:0,BULK:0,RULE:Release_Ham,ACT ION:release,TS:-5 X-CID-INFO: VERSION:1.1.28,REQID:68b8ec71-7352-453b-87a7-33a33507e392,IP:15,UR L:0,TC:0,Content:-5,EDM:0,RT:0,SF:-15,FILE:0,BULK:0,RULE:Release_Ham,ACTIO N:release,TS:-5 X-CID-META: VersionHash:176cd25,CLOUDID:1b3b7fa0-0933-4333-8d4f-6c3c53ebd55b,B ulkID:2307251415132XWQSSO0,BulkQuantity:0,Recheck:0,SF:17|19|44|38|24|102, TC:nil,Content:0,EDM:-3,IP:-2,URL:0,File:nil,Bulk:nil,QS:nil,BEC:nil,COL:0 ,OSI:0,OSA:0,AV:0,LES:1,SPR:NO,DKR:0,DKP:0 X-CID-BVR: 0 X-CID-BAS: 0,_,0,_ X-CID-FACTOR: TF_CID_SPAM_SNR,TF_CID_SPAM_FAS,TF_CID_SPAM_FSD,TF_CID_SPAM_FSI X-UUID: 8e5b3e4a0453435d8408d3e21c20e6c4-20230725 X-User: lienze@kylinos.cn Received: from ubuntu.. [(39.156.73.12)] by mailgw (envelope-from ) (Generic MTA with TLSv1.2 ECDHE-RSA-AES256-GCM-SHA384 256/256) with ESMTP id 715763335; Tue, 25 Jul 2023 14:15:11 +0800 From: Enze Li To: chenhuacai@kernel.org, kernel@xen0n.name, loongarch@lists.linux.dev, glider@google.com, elver@google.com, akpm@linux-foundation.org, kasan-dev@googlegroups.com, linux-mm@kvack.org Cc: zhangqing@loongson.cn, yangtiezhu@loongson.cn, dvyukov@google.com, Enze Li Subject: [PATCH 3/4 v2] KFENCE: Defer the assignment of the local variable addr Date: Tue, 25 Jul 2023 14:14:50 +0800 Message-Id: <20230725061451.1231480-4-lienze@kylinos.cn> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230725061451.1231480-1-lienze@kylinos.cn> References: <20230725061451.1231480-1-lienze@kylinos.cn> MIME-Version: 1.0 X-Rspamd-Queue-Id: 30AAF180004 X-Rspam-User: X-Rspamd-Server: rspam02 X-Stat-Signature: a7z5n3pnn5b3zojdz5qa9t5d89conxyp X-HE-Tag: 1690265720-203854 X-HE-Meta: U2FsdGVkX1/D0QiFfWDzs/8Bi3mcef3hYzPBHHPPorXBbHWkBTEDXoLR21MT8+SvVZbCYtZN4MsYPdj/YDAdRlmgA7d1OVkEc8txBTrPqLH57teL6cmgyG9bqgqOtx8q8jxfZAs2AKJwW0Naht49+Ihwu815vBovb/KpQFRBK9Q/gCF3YY5YuHxm0pAkmdRFtZ2/TLpfTWmVRoiwrmIWcH0fOyYV7D3NcpX7C6nLyD2Bc8LZL1ewUu7zfV/9OfdMwohYagbonXpsmpAGYbT98B9JVGTrEwIWhAXRHgXpNWTCmK9p3uoX3QA7LR16VqL1nha29ILJmcxBOwkgdpUyLwJZjh8cIH/Pxh7cHBxDZdtI78lL4theoPEnNSMbzRnJJ2J2M+4SB5yC0SBk9r9m+0ulG48SlCqTof2AZFRP2RL1LUiuU9Zo/jEpil6D7/Uimw3H6TI6/UBluALWCtKQ5SjDJNVm2IMsXhbvgSKkM4jh5BhzGiCVKx0u+PM0+4OsPmJ0kO+p1lXRy+7cNh1L79PH3nnn4U13RHUWMeJzq/FjYTANShiwk9UHvulC8CWVt/3n+cB74BCofqvxElKopKfqKfyV23p+ACxuiBeuNVUchFRgEpwQgMw0NG3HK9R3kSxZPn+D+L+nmij2QsLUHK4Y4XHRdikPn2B4U0qNPmkENu7eO/kwhEZN4CLCeCopMuDOvZK8etBypCGBT5cjWcTevnNeHDU4b2yZYnSRfozkibf8FPG+Ti6XC2MVYLaKqTtlZO15xFEdDQBYfRthUUM2JpNWXvm8JGWloVq11sM395X4eXXosZnQZY57zo6V/r4HmPti+aPAAoGuvUzmk1JZCCSRmeGZHKkJm5dxlH5FFl1UPLoME2TCEQFxSI67tPwADXKJh94CO6IsTdw7BvIGzANCHxCtpRbsCD6TuSO4sES3Gwy8VmbwGB2BreC+k7daVWjGcCddHx+ExvJ fUCOiQ9U Xl0vDB1+FzTmnjUWGbh//vt42piiIlqB/bqj02f5ImWUlepUYBz9cNl2+cQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: The LoongArch architecture is different from other architectures. It needs to update __kfence_pool during arch_kfence_init_pool(). This patch modifies the assignment location of the local variable addr in the kfence_init_pool function to support the case of updating __kfence_pool in arch_kfence_init_pool(). Signed-off-by: Enze Li Acked-by: Marco Elver --- mm/kfence/core.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/mm/kfence/core.c b/mm/kfence/core.c index dad3c0eb70a0..e124ffff489f 100644 --- a/mm/kfence/core.c +++ b/mm/kfence/core.c @@ -566,13 +566,14 @@ static void rcu_guarded_free(struct rcu_head *h) */ static unsigned long kfence_init_pool(void) { - unsigned long addr = (unsigned long)__kfence_pool; + unsigned long addr; struct page *pages; int i; if (!arch_kfence_init_pool()) - return addr; + return (unsigned long)__kfence_pool; + addr = (unsigned long)__kfence_pool; pages = virt_to_page(__kfence_pool); /* From patchwork Tue Jul 25 06:14:51 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Enze Li X-Patchwork-Id: 13325910 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8D814C00528 for ; Tue, 25 Jul 2023 06:15:29 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 25795900002; Tue, 25 Jul 2023 02:15:29 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 16BD08E0003; Tue, 25 Jul 2023 02:15:29 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id EFF80900002; Tue, 25 Jul 2023 02:15:28 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id DF6EA8E0003 for ; Tue, 25 Jul 2023 02:15:28 -0400 (EDT) Received: from smtpin04.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id B0E2280C5A for ; Tue, 25 Jul 2023 06:15:28 +0000 (UTC) X-FDA: 81049122336.04.4B4DEF9 Received: from mailgw.kylinos.cn (mailgw.kylinos.cn [124.126.103.232]) by imf17.hostedemail.com (Postfix) with ESMTP id D7D244000C for ; Tue, 25 Jul 2023 06:15:25 +0000 (UTC) Authentication-Results: imf17.hostedemail.com; dkim=none; dmarc=none; spf=pass (imf17.hostedemail.com: domain of lienze@kylinos.cn designates 124.126.103.232 as permitted sender) smtp.mailfrom=lienze@kylinos.cn ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1690265726; a=rsa-sha256; cv=none; b=cTcMdmO8EBvCYS8+7bLMy3/NWidKzMlJuH/gqB7+y1p6oBg9y9eG1A9w+EBjbkoJxa7pn8 xvlJFGPIbNtpuAw32c7FCNiMYoAuo3/SkXRoQXd05rn/IcOjpR0XZebMZiPKfZEvxuSC/K 4jXccBjl15Lmj39kn/TXdEi/olAXgTU= ARC-Authentication-Results: i=1; imf17.hostedemail.com; dkim=none; dmarc=none; spf=pass (imf17.hostedemail.com: domain of lienze@kylinos.cn designates 124.126.103.232 as permitted sender) smtp.mailfrom=lienze@kylinos.cn ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1690265726; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=EqW6kJ87jfc/UoorJHCNkGGBtCR16wyXTZDbfFoAbyY=; b=ZPNfL3Nfhf0r2XvfJsGgNLXKvyWuv2N2k5Gg57stmr5QwVyCCnj0Ec+OUB9C1x2foM9s6s BTm4Rn3fS/EUdYGSadvUXT6MEM14S5G0O/QaYwcP4ajs30ADt8wHpPE6C/Oe2WQLexHihH ZupZ23Qqsmls/QmtdHiKemdsk6KFtC0= X-UUID: 3f0d37b293b84437aff8eecdace72e24-20230725 X-CID-P-RULE: Release_Ham X-CID-O-INFO: VERSION:1.1.28,REQID:d8a88afa-7b8d-4f33-a7ea-af79e1cee8d2,IP:15, URL:0,TC:0,Content:0,EDM:0,RT:0,SF:-15,FILE:0,BULK:0,RULE:Release_Ham,ACTI ON:release,TS:0 X-CID-INFO: VERSION:1.1.28,REQID:d8a88afa-7b8d-4f33-a7ea-af79e1cee8d2,IP:15,UR L:0,TC:0,Content:0,EDM:0,RT:0,SF:-15,FILE:0,BULK:0,RULE:Release_Ham,ACTION :release,TS:0 X-CID-META: VersionHash:176cd25,CLOUDID:1d6ec1b3-a467-4aa9-9e04-f584452e3794,B ulkID:230725141514S69XBC26,BulkQuantity:0,Recheck:0,SF:38|24|17|19|44|102, TC:nil,Content:0,EDM:-3,IP:-2,URL:1,File:nil,Bulk:nil,QS:nil,BEC:nil,COL:0 ,OSI:0,OSA:0,AV:0,LES:1,SPR:NO,DKR:0,DKP:0 X-CID-BVR: 0 X-CID-BAS: 0,_,0,_ X-CID-FACTOR: TF_CID_SPAM_ULS,TF_CID_SPAM_SNR,TF_CID_SPAM_FAS,TF_CID_SPAM_FSD, TF_CID_SPAM_FSI X-UUID: 3f0d37b293b84437aff8eecdace72e24-20230725 X-User: lienze@kylinos.cn Received: from ubuntu.. [(39.156.73.12)] by mailgw (envelope-from ) (Generic MTA with TLSv1.2 ECDHE-RSA-AES256-GCM-SHA384 256/256) with ESMTP id 755063753; Tue, 25 Jul 2023 14:15:12 +0800 From: Enze Li To: chenhuacai@kernel.org, kernel@xen0n.name, loongarch@lists.linux.dev, glider@google.com, elver@google.com, akpm@linux-foundation.org, kasan-dev@googlegroups.com, linux-mm@kvack.org Cc: zhangqing@loongson.cn, yangtiezhu@loongson.cn, dvyukov@google.com, Enze Li Subject: [PATCH 4/4 v2] LoongArch: Add KFENCE support Date: Tue, 25 Jul 2023 14:14:51 +0800 Message-Id: <20230725061451.1231480-5-lienze@kylinos.cn> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230725061451.1231480-1-lienze@kylinos.cn> References: <20230725061451.1231480-1-lienze@kylinos.cn> MIME-Version: 1.0 X-Rspamd-Server: rspam08 X-Rspamd-Queue-Id: D7D244000C X-Stat-Signature: 8nihupp913w19ukm5oi1npifaimcyndm X-Rspam-User: X-HE-Tag: 1690265725-949355 X-HE-Meta: U2FsdGVkX1/c6LH569kkpDNBLgQz2W5g/gkP/v4THOKjRqzeWI6QuRvgOmHDD1t5HIRrWzfDSLxNNbi5upeuaZYPsRc1x/vhIN14YequabO61f+Uj+s8H3QoT5YTBfCPClYhwyx1EQOgb5O24ajwV7eEkIl6vPRUYRTtAZsWZirYp/ka921J/pva5VL1rXkOeObof8YXgapN8f7+PX6HcBYKN6Wj45kGOVAweceGsPZAOc9RyxqNxMVYrw8+FNl/3z2dFW7LoMsBFWjb7P8nyUJyZKqm8+J+DbTHffyFsfMz6OhSqVuS2ntukNvcL14tQ4doY3xQJ3oM6bADy+M4tOfrQtHpZN4cx4GxVpn775WkJ0vFt9InhtbG9wG7tYkE9fzhmwcYTvTXwffw88u9nOYJR4V1q1vaZU2b5po2GGya6tr/tMZBjhwvjNRDpHhk+nFSuSjTs/TxkzxNq8ddD5fLKAttXWtORW7TMXEV36lGLnS+r8c1gwzshMFnEkCQmJlgQk6GZP2J4cCf78mmcd2ZnX8S29vFE5xHxsBh1il/2u27seBO99+FwKqQtXeGM0Q/GUel/J8c8/Bmz6+nW6epbIpxydaYiUV0BbfYK+nQn72J+ufl0+PbOQxafJ5DUeCN5Vmo0aWwS9aPfP+J43h+xjfDJv93/rqsrYcL2XrvG0+6ehr4sO+0A9PDcIHSmaMSng2toIDCLE2euYZgii65SAKGS0BgZL9lB1z2bB65JGcFTlknLAUc865uW6C5KP0iqLxFXTVl2ZNdDdoYFCw1cn6i1E4c3Vth6Vvrp2qkX/oxAISBHp8SB4+7kDGMTNUUFVWKESQFdZa9GFZ5ZhhK8G9GpK9osoreW5s3GRmYD8VTueW/GNDx/PDOSu6rF6mbhQWaR/QqPVVP9giURXrM34JSgfzozNkVnACli0p+GYS5xlT5DVHlg0fcJrfC/W4i6ztdFRP3JMMsjsA YJ3kYzpe r4j3eKFNXEJTVVCq3HYRnAuL9/c+l24OjMvfHKppwRr66nCHNVjNBvCzdG60+VUxqRDp323ALWMecn5GChiSSaPtJbmLpwdRHc4TMVBtM0e317Z2B9opFsHi0cw== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: The LoongArch architecture is quite different from other architectures. When the allocating of KFENCE itself is done, it is mapped to the direct mapping configuration window [1] by default on LoongArch. It means that it is not possible to use the page table mapped mode which required by the KFENCE system and therefore it should be remapped to the appropriate region. This patch adds architecture specific implementation details for KFENCE. In particular, this implements the required interface in . Tested this patch by running the testcases and all passed. [1] https://loongson.github.io/LoongArch-Documentation/LoongArch-Vol1-EN.html#virtual-address-space-and-address-translation-mode Signed-off-by: Enze Li --- arch/loongarch/Kconfig | 1 + arch/loongarch/include/asm/kfence.h | 62 ++++++++++++++++++++++++++++ arch/loongarch/include/asm/pgtable.h | 14 ++++++- arch/loongarch/mm/fault.c | 22 ++++++---- 4 files changed, 90 insertions(+), 9 deletions(-) create mode 100644 arch/loongarch/include/asm/kfence.h diff --git a/arch/loongarch/Kconfig b/arch/loongarch/Kconfig index 70635ea3d1e4..5b63b16be49e 100644 --- a/arch/loongarch/Kconfig +++ b/arch/loongarch/Kconfig @@ -91,6 +91,7 @@ config LOONGARCH select HAVE_ARCH_AUDITSYSCALL select HAVE_ARCH_JUMP_LABEL select HAVE_ARCH_JUMP_LABEL_RELATIVE + select HAVE_ARCH_KFENCE select HAVE_ARCH_MMAP_RND_BITS if MMU select HAVE_ARCH_SECCOMP_FILTER select HAVE_ARCH_TRACEHOOK diff --git a/arch/loongarch/include/asm/kfence.h b/arch/loongarch/include/asm/kfence.h new file mode 100644 index 000000000000..fb39076fe4d7 --- /dev/null +++ b/arch/loongarch/include/asm/kfence.h @@ -0,0 +1,62 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* + * KFENCE support for LoongArch. + * + * Author: Enze Li + * Copyright (C) 2022-2023 KylinSoft Corporation. + */ + +#ifndef _ASM_LOONGARCH_KFENCE_H +#define _ASM_LOONGARCH_KFENCE_H + +#include +#include +#include + +static inline bool arch_kfence_init_pool(void) +{ + char *kfence_pool = __kfence_pool; + struct vm_struct *area; + int err; + + area = __get_vm_area_caller(KFENCE_POOL_SIZE, VM_IOREMAP, + KFENCE_AREA_START, KFENCE_AREA_END, + __builtin_return_address(0)); + if (!area) + return false; + + __kfence_pool = (char *)area->addr; + err = ioremap_page_range((unsigned long)__kfence_pool, + (unsigned long)__kfence_pool + KFENCE_POOL_SIZE, + virt_to_phys((void *)kfence_pool), + PAGE_KERNEL); + if (err) { + free_vm_area(area); + return false; + } + + return true; +} + +/* Protect the given page and flush TLB. */ +static inline bool kfence_protect_page(unsigned long addr, bool protect) +{ + pte_t *pte = virt_to_kpte(addr); + + if (WARN_ON(!pte) || pte_none(*pte)) + return false; + + if (protect) + set_pte(pte, __pte(pte_val(*pte) & ~(_PAGE_VALID | _PAGE_PRESENT))); + else + set_pte(pte, __pte(pte_val(*pte) | (_PAGE_VALID | _PAGE_PRESENT))); + + /* Flush this CPU's TLB. */ + preempt_disable(); + local_flush_tlb_one(addr); + preempt_enable(); + + return true; +} + +#endif /* _ASM_LOONGARCH_KFENCE_H */ diff --git a/arch/loongarch/include/asm/pgtable.h b/arch/loongarch/include/asm/pgtable.h index 98a0c98de9d1..2702a6ba7122 100644 --- a/arch/loongarch/include/asm/pgtable.h +++ b/arch/loongarch/include/asm/pgtable.h @@ -77,6 +77,13 @@ extern unsigned long zero_page_mask; (virt_to_page((void *)(empty_zero_page + (((unsigned long)(vaddr)) & zero_page_mask)))) #define __HAVE_COLOR_ZERO_PAGE +#ifdef CONFIG_KFENCE +#define KFENCE_AREA_SIZE \ + (((CONFIG_KFENCE_NUM_OBJECTS + 1) * 2 + 2) * PAGE_SIZE) +#else +#define KFENCE_AREA_SIZE 0 +#endif + /* * TLB refill handlers may also map the vmalloc area into xkvrange. * Avoid the first couple of pages so NULL pointer dereferences will @@ -88,11 +95,16 @@ extern unsigned long zero_page_mask; #define VMALLOC_START MODULES_END #define VMALLOC_END \ (vm_map_base + \ - min(PTRS_PER_PGD * PTRS_PER_PUD * PTRS_PER_PMD * PTRS_PER_PTE * PAGE_SIZE, (1UL << cpu_vabits)) - PMD_SIZE - VMEMMAP_SIZE) + min(PTRS_PER_PGD * PTRS_PER_PUD * PTRS_PER_PMD * PTRS_PER_PTE * PAGE_SIZE, (1UL << cpu_vabits)) - PMD_SIZE - VMEMMAP_SIZE - KFENCE_AREA_SIZE) #define vmemmap ((struct page *)((VMALLOC_END + PMD_SIZE) & PMD_MASK)) #define VMEMMAP_END ((unsigned long)vmemmap + VMEMMAP_SIZE - 1) +#ifdef CONFIG_KFENCE +#define KFENCE_AREA_START VMEMMAP_END +#define KFENCE_AREA_END (KFENCE_AREA_START + KFENCE_AREA_SIZE) +#endif + #define pte_ERROR(e) \ pr_err("%s:%d: bad pte %016lx.\n", __FILE__, __LINE__, pte_val(e)) #ifndef __PAGETABLE_PMD_FOLDED diff --git a/arch/loongarch/mm/fault.c b/arch/loongarch/mm/fault.c index da5b6d518cdb..c0319128b221 100644 --- a/arch/loongarch/mm/fault.c +++ b/arch/loongarch/mm/fault.c @@ -23,6 +23,7 @@ #include #include #include +#include #include #include @@ -30,7 +31,8 @@ int show_unhandled_signals = 1; -static void __kprobes no_context(struct pt_regs *regs, unsigned long address) +static void __kprobes no_context(struct pt_regs *regs, unsigned long address, + unsigned long write) { const int field = sizeof(unsigned long) * 2; @@ -38,6 +40,9 @@ static void __kprobes no_context(struct pt_regs *regs, unsigned long address) if (fixup_exception(regs)) return; + if (kfence_handle_page_fault(address, write, regs)) + return; + /* * Oops. The kernel tried to access some bad page. We'll have to * terminate things with extreme prejudice. @@ -51,14 +56,15 @@ static void __kprobes no_context(struct pt_regs *regs, unsigned long address) die("Oops", regs); } -static void __kprobes do_out_of_memory(struct pt_regs *regs, unsigned long address) +static void __kprobes do_out_of_memory(struct pt_regs *regs, unsigned long address, + unsigned long write) { /* * We ran out of memory, call the OOM killer, and return the userspace * (which will retry the fault, or kill us if we got oom-killed). */ if (!user_mode(regs)) { - no_context(regs, address); + no_context(regs, address, write); return; } pagefault_out_of_memory(); @@ -69,7 +75,7 @@ static void __kprobes do_sigbus(struct pt_regs *regs, { /* Kernel mode? Handle exceptions or die */ if (!user_mode(regs)) { - no_context(regs, address); + no_context(regs, address, write); return; } @@ -90,7 +96,7 @@ static void __kprobes do_sigsegv(struct pt_regs *regs, /* Kernel mode? Handle exceptions or die */ if (!user_mode(regs)) { - no_context(regs, address); + no_context(regs, address, write); return; } @@ -149,7 +155,7 @@ static void __kprobes __do_page_fault(struct pt_regs *regs, */ if (address & __UA_LIMIT) { if (!user_mode(regs)) - no_context(regs, address); + no_context(regs, address, write); else do_sigsegv(regs, write, address, si_code); return; @@ -211,7 +217,7 @@ static void __kprobes __do_page_fault(struct pt_regs *regs, if (fault_signal_pending(fault, regs)) { if (!user_mode(regs)) - no_context(regs, address); + no_context(regs, address, write); return; } @@ -232,7 +238,7 @@ static void __kprobes __do_page_fault(struct pt_regs *regs, if (unlikely(fault & VM_FAULT_ERROR)) { mmap_read_unlock(mm); if (fault & VM_FAULT_OOM) { - do_out_of_memory(regs, address); + do_out_of_memory(regs, address, write); return; } else if (fault & VM_FAULT_SIGSEGV) { do_sigsegv(regs, write, address, si_code);