From patchwork Mon Apr 7 15:11:27 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexander Gordeev X-Patchwork-Id: 14041049 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id CA79EC36010 for ; Mon, 7 Apr 2025 15:12:06 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.940329.1340134 (Exim 4.92) (envelope-from ) id 1u1o8c-0005VH-Ur; Mon, 07 Apr 2025 15:11:50 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 940329.1340134; Mon, 07 Apr 2025 15:11:50 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1u1o8c-0005VA-SA; Mon, 07 Apr 2025 15:11:50 +0000 Received: by outflank-mailman (input) for mailman id 940329; Mon, 07 Apr 2025 15:11:50 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1u1o8c-0005V3-5e for xen-devel@lists.xenproject.org; Mon, 07 Apr 2025 15:11:50 +0000 Received: from mx0b-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id 9eed0acb-13c2-11f0-9eaa-5ba50f476ded; Mon, 07 Apr 2025 17:11:47 +0200 (CEST) Received: from pps.filterd (m0360072.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 537En0ZD029644; Mon, 7 Apr 2025 15:11:34 GMT Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 45uwswvt33-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 07 Apr 2025 15:11:34 +0000 (GMT) Received: from m0360072.ppops.net (m0360072.ppops.net [127.0.0.1]) by pps.reinject (8.18.0.8/8.18.0.8) with ESMTP id 537FBXTb013844; Mon, 7 Apr 2025 15:11:33 GMT Received: from ppma13.dal12v.mail.ibm.com (dd.9e.1632.ip4.static.sl-reverse.com [50.22.158.221]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 45uwswvt2v-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 07 Apr 2025 15:11:33 +0000 (GMT) Received: from pps.filterd (ppma13.dal12v.mail.ibm.com [127.0.0.1]) by ppma13.dal12v.mail.ibm.com (8.18.1.2/8.18.1.2) with ESMTP id 537F2iM6018432; Mon, 7 Apr 2025 15:11:32 GMT Received: from smtprelay04.fra02v.mail.ibm.com ([9.218.2.228]) by ppma13.dal12v.mail.ibm.com (PPS) with ESMTPS id 45uh2ke5ug-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 07 Apr 2025 15:11:32 +0000 Received: from smtpav06.fra02v.mail.ibm.com (smtpav06.fra02v.mail.ibm.com [10.20.54.105]) by smtprelay04.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 537FBVQW33751652 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 7 Apr 2025 15:11:31 GMT Received: from smtpav06.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id E82452004B; Mon, 7 Apr 2025 15:11:30 +0000 (GMT) Received: from smtpav06.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id D3F6E20049; Mon, 7 Apr 2025 15:11:30 +0000 (GMT) Received: from tuxmaker.boeblingen.de.ibm.com (unknown [9.152.85.9]) by smtpav06.fra02v.mail.ibm.com (Postfix) with ESMTPS; Mon, 7 Apr 2025 15:11:30 +0000 (GMT) Received: by tuxmaker.boeblingen.de.ibm.com (Postfix, from userid 55669) id 7F795E10FC; Mon, 07 Apr 2025 17:11:30 +0200 (CEST) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 9eed0acb-13c2-11f0-9eaa-5ba50f476ded DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=pp1; bh=JAdRtQpYpJ83Wan/q ay54BErF461aFj9Lbn233V6/Vo=; b=IKe6NKPK0RQuaHMhk18ftPiOYYmCBCWNo B9AMFWR/OSyAEWP+S886dEpJeibdEmZl36pABv5K+wup8TiyL8NG8ViK9od6oBRM 3hfWr3Nzyj+cpTRWraAtGGyrEizF/IwgeyvpxnuqLvo0iHgyxm7f3x2y4rrF4BuZ 8DVbuOci3sgrMA9QHMuCtrLrrnIMC0eF9Lq4xmJACPC8mwWkrruApDLDO3y+TF6K EKoPhfmZsym89+7c65PcFL6w5iov3VKT/uGxaF9PJzGWmz4HwXohtM7m83O2AHTx G4AzXPBlc8+L56yqeI1A9XrdpDtpRX+WW/G2StWfINFf09T/NbHTA== From: Alexander Gordeev To: Andrew Morton , Andrey Ryabinin Cc: Hugh Dickins , Nicholas Piggin , Guenter Roeck , Juergen Gross , Jeremy Fitzhardinge , linux-kernel@vger.kernel.org, linux-mm@kvack.org, kasan-dev@googlegroups.com, sparclinux@vger.kernel.org, xen-devel@lists.xenproject.org, linuxppc-dev@lists.ozlabs.org, linux-s390@vger.kernel.org Subject: [PATCH v1 1/4] kasan: Avoid sleepable page allocation from atomic context Date: Mon, 7 Apr 2025 17:11:27 +0200 Message-ID: X-Mailer: git-send-email 2.45.2 In-Reply-To: References: MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-GUID: noh5DKX9Jwz7KQQbR0xgtrf0Cnrok7JA X-Proofpoint-ORIG-GUID: B9e39pOuvk43sXxp1B5Vd8FFaIlzYizr X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1095,Hydra:6.0.680,FMLib:17.12.68.34 definitions=2025-04-07_04,2025-04-03_03,2024-11-22_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 mlxscore=0 bulkscore=0 mlxlogscore=999 spamscore=0 adultscore=0 clxscore=1011 phishscore=0 lowpriorityscore=0 impostorscore=0 suspectscore=0 malwarescore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.19.0-2502280000 definitions=main-2504070104 apply_to_page_range() enters lazy MMU mode and then invokes kasan_populate_vmalloc_pte() callback on each page table walk iteration. The lazy MMU mode may only be entered only under protection of the page table lock. However, the callback can go into sleep when trying to allocate a single page. Change __get_free_page() allocation mode from GFP_KERNEL to GFP_ATOMIC to avoid scheduling out while in atomic context. Signed-off-by: Alexander Gordeev Reviewed-by: Nicholas Piggin --- mm/kasan/shadow.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/mm/kasan/shadow.c b/mm/kasan/shadow.c index 88d1c9dcb507..edfa77959474 100644 --- a/mm/kasan/shadow.c +++ b/mm/kasan/shadow.c @@ -301,7 +301,7 @@ static int kasan_populate_vmalloc_pte(pte_t *ptep, unsigned long addr, if (likely(!pte_none(ptep_get(ptep)))) return 0; - page = __get_free_page(GFP_KERNEL); + page = __get_free_page(GFP_ATOMIC); if (!page) return -ENOMEM; From patchwork Mon Apr 7 15:11:28 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexander Gordeev X-Patchwork-Id: 14041051 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 4778EC36010 for ; Mon, 7 Apr 2025 15:12:13 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.940337.1340174 (Exim 4.92) (envelope-from ) id 1u1o8r-0006es-Va; Mon, 07 Apr 2025 15:12:05 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 940337.1340174; Mon, 07 Apr 2025 15:12:05 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1u1o8r-0006ef-Qt; Mon, 07 Apr 2025 15:12:05 +0000 Received: by outflank-mailman (input) for mailman id 940337; Mon, 07 Apr 2025 15:12:04 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1u1o8p-0005V9-VG for xen-devel@lists.xenproject.org; Mon, 07 Apr 2025 15:12:03 +0000 Received: from mx0b-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id 9ff6c0a7-13c2-11f0-9ffb-bf95429c2676; Mon, 07 Apr 2025 17:11:48 +0200 (CEST) Received: from pps.filterd (m0360072.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 5378IrTc029651; Mon, 7 Apr 2025 15:11:34 GMT Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 45uwswvt35-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 07 Apr 2025 15:11:34 +0000 (GMT) Received: from m0360072.ppops.net (m0360072.ppops.net [127.0.0.1]) by pps.reinject (8.18.0.8/8.18.0.8) with ESMTP id 537F41Wv022508; Mon, 7 Apr 2025 15:11:33 GMT Received: from ppma22.wdc07v.mail.ibm.com (5c.69.3da9.ip4.static.sl-reverse.com [169.61.105.92]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 45uwswvt30-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 07 Apr 2025 15:11:33 +0000 (GMT) Received: from pps.filterd (ppma22.wdc07v.mail.ibm.com [127.0.0.1]) by ppma22.wdc07v.mail.ibm.com (8.18.1.2/8.18.1.2) with ESMTP id 537DrCNJ011062; Mon, 7 Apr 2025 15:11:33 GMT Received: from smtprelay04.fra02v.mail.ibm.com ([9.218.2.228]) by ppma22.wdc07v.mail.ibm.com (PPS) with ESMTPS id 45uf7yege2-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 07 Apr 2025 15:11:33 +0000 Received: from smtpav04.fra02v.mail.ibm.com (smtpav04.fra02v.mail.ibm.com [10.20.54.103]) by smtprelay04.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 537FBVtw16122324 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 7 Apr 2025 15:11:31 GMT Received: from smtpav04.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 10E992004E; Mon, 7 Apr 2025 15:11:31 +0000 (GMT) Received: from smtpav04.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id EDA652004B; Mon, 7 Apr 2025 15:11:30 +0000 (GMT) Received: from tuxmaker.boeblingen.de.ibm.com (unknown [9.152.85.9]) by smtpav04.fra02v.mail.ibm.com (Postfix) with ESMTPS; Mon, 7 Apr 2025 15:11:30 +0000 (GMT) Received: by tuxmaker.boeblingen.de.ibm.com (Postfix, from userid 55669) id 823D8E15AF; Mon, 07 Apr 2025 17:11:30 +0200 (CEST) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 9ff6c0a7-13c2-11f0-9ffb-bf95429c2676 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=pp1; bh=OcNsdUXLajTS6Up4y b87KBBXCRng/iRPfXkDho493IU=; b=WOoAcnFvpuNxNEiTrGC3QZIq5o+jYBCOb LyLvBIz0pxghOpRjonhLcZeU5GpFEhaJUVrjW4lEjOfClydMsoxIlT1tdJfSYTrD w4ni3gxfKYkXD7bD8soBquTEt6UrKrs12uA2MgZo7ShPARvXziKp6H/vG/OR0j4l xZ01w6SQLYfrZApqIj7cCKLpACusgLuv8O/JsHOSLcLihXpk7kQfCZv5I8mWLAkI jB3Jm3YdqkahCDJoZ3Dg5tjezoL+c8gFXxALeoX8dMZALeFXSQU3kyz0vIaTpzdk 0hozKrK7Ar8LywsJy5kov8c0Zzsq4KVBjV6jp2caOQi+tFMaWRfCQ== From: Alexander Gordeev To: Andrew Morton , Andrey Ryabinin Cc: Hugh Dickins , Nicholas Piggin , Guenter Roeck , Juergen Gross , Jeremy Fitzhardinge , linux-kernel@vger.kernel.org, linux-mm@kvack.org, kasan-dev@googlegroups.com, sparclinux@vger.kernel.org, xen-devel@lists.xenproject.org, linuxppc-dev@lists.ozlabs.org, linux-s390@vger.kernel.org Subject: [PATCH v1 2/4] mm: Cleanup apply_to_pte_range() routine Date: Mon, 7 Apr 2025 17:11:28 +0200 Message-ID: <93102722541b1daf541fce9fb316a1a2614d8c86.1744037648.git.agordeev@linux.ibm.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: References: MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-GUID: ZjXrqfzVuI4O6PuB71DztWSnTeaDAz2F X-Proofpoint-ORIG-GUID: B8AxCnBSacob3HedwAwYgmrsIk7eP_GG X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1095,Hydra:6.0.680,FMLib:17.12.68.34 definitions=2025-04-07_04,2025-04-03_03,2024-11-22_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 mlxscore=0 bulkscore=0 mlxlogscore=912 spamscore=0 adultscore=0 clxscore=1011 phishscore=0 lowpriorityscore=0 impostorscore=0 suspectscore=0 malwarescore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.19.0-2502280000 definitions=main-2504070104 Reverse 'create' vs 'mm == &init_mm' conditions and move page table mask modification out of the atomic context. Signed-off-by: Alexander Gordeev Reviewed-by: Nicholas Piggin --- mm/memory.c | 28 +++++++++++++++++----------- 1 file changed, 17 insertions(+), 11 deletions(-) diff --git a/mm/memory.c b/mm/memory.c index 2d8c265fc7d6..f0201c8ec1ce 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -2915,24 +2915,28 @@ static int apply_to_pte_range(struct mm_struct *mm, pmd_t *pmd, pte_fn_t fn, void *data, bool create, pgtbl_mod_mask *mask) { + int err = create ? -ENOMEM : -EINVAL; pte_t *pte, *mapped_pte; - int err = 0; spinlock_t *ptl; - if (create) { - mapped_pte = pte = (mm == &init_mm) ? - pte_alloc_kernel_track(pmd, addr, mask) : - pte_alloc_map_lock(mm, pmd, addr, &ptl); + if (mm == &init_mm) { + if (create) + pte = pte_alloc_kernel_track(pmd, addr, mask); + else + pte = pte_offset_kernel(pmd, addr); if (!pte) - return -ENOMEM; + return err; } else { - mapped_pte = pte = (mm == &init_mm) ? - pte_offset_kernel(pmd, addr) : - pte_offset_map_lock(mm, pmd, addr, &ptl); + if (create) + pte = pte_alloc_map_lock(mm, pmd, addr, &ptl); + else + pte = pte_offset_map_lock(mm, pmd, addr, &ptl); if (!pte) - return -EINVAL; + return err; + mapped_pte = pte; } + err = 0; arch_enter_lazy_mmu_mode(); if (fn) { @@ -2944,12 +2948,14 @@ static int apply_to_pte_range(struct mm_struct *mm, pmd_t *pmd, } } while (addr += PAGE_SIZE, addr != end); } - *mask |= PGTBL_PTE_MODIFIED; arch_leave_lazy_mmu_mode(); if (mm != &init_mm) pte_unmap_unlock(mapped_pte, ptl); + + *mask |= PGTBL_PTE_MODIFIED; + return err; } From patchwork Mon Apr 7 15:11:29 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexander Gordeev X-Patchwork-Id: 14041047 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 324C0C369A4 for ; Mon, 7 Apr 2025 15:12:06 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.940330.1340143 (Exim 4.92) (envelope-from ) id 1u1o8f-0005jn-88; Mon, 07 Apr 2025 15:11:53 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 940330.1340143; Mon, 07 Apr 2025 15:11:53 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1u1o8f-0005jc-5M; Mon, 07 Apr 2025 15:11:53 +0000 Received: by outflank-mailman (input) for mailman id 940330; Mon, 07 Apr 2025 15:11:52 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1u1o8e-0005V9-2g for xen-devel@lists.xenproject.org; Mon, 07 Apr 2025 15:11:52 +0000 Received: from mx0b-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id 9fd8cb33-13c2-11f0-9ffb-bf95429c2676; Mon, 07 Apr 2025 17:11:48 +0200 (CEST) Received: from pps.filterd (m0356516.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 5378eCst023081; Mon, 7 Apr 2025 15:11:36 GMT Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 45v0spm8qy-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 07 Apr 2025 15:11:36 +0000 (GMT) Received: from m0356516.ppops.net (m0356516.ppops.net [127.0.0.1]) by pps.reinject (8.18.0.8/8.18.0.8) with ESMTP id 537FBZII019687; Mon, 7 Apr 2025 15:11:35 GMT Received: from ppma13.dal12v.mail.ibm.com (dd.9e.1632.ip4.static.sl-reverse.com [50.22.158.221]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 45v0spm8qw-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 07 Apr 2025 15:11:35 +0000 (GMT) Received: from pps.filterd (ppma13.dal12v.mail.ibm.com [127.0.0.1]) by ppma13.dal12v.mail.ibm.com (8.18.1.2/8.18.1.2) with ESMTP id 537Ehxeh017825; Mon, 7 Apr 2025 15:11:34 GMT Received: from smtprelay04.fra02v.mail.ibm.com ([9.218.2.228]) by ppma13.dal12v.mail.ibm.com (PPS) with ESMTPS id 45uh2ke5uh-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 07 Apr 2025 15:11:32 +0000 Received: from smtpav06.fra02v.mail.ibm.com (smtpav06.fra02v.mail.ibm.com [10.20.54.105]) by smtprelay04.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 537FBVTb14156090 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 7 Apr 2025 15:11:31 GMT Received: from smtpav06.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 0B32320049; Mon, 7 Apr 2025 15:11:31 +0000 (GMT) Received: from smtpav06.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id E98012004D; Mon, 7 Apr 2025 15:11:30 +0000 (GMT) Received: from tuxmaker.boeblingen.de.ibm.com (unknown [9.152.85.9]) by smtpav06.fra02v.mail.ibm.com (Postfix) with ESMTPS; Mon, 7 Apr 2025 15:11:30 +0000 (GMT) Received: by tuxmaker.boeblingen.de.ibm.com (Postfix, from userid 55669) id 85284E1613; Mon, 07 Apr 2025 17:11:30 +0200 (CEST) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 9fd8cb33-13c2-11f0-9ffb-bf95429c2676 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=pp1; bh=ngcqzYVpe7xkxjw37 TRZ0RrIQbWfDyVW1WAk2hgrGY8=; b=WCNS4jwcRyfhgA/oGV/C7ufKUl5usySJY JBkrPpB0Kj0xSv0oPCKUqsc+vBFB8dZYewnwf47Zz4FOayaQxY3jaFKQUDjdWPJt ugxuydo8+0yZUmA+ohR403/w5zwKv2CqsgJbjNB2KabEV+JDdelbBoH8nco6FKOT R5XE4tzipG+fY4VoyLYwQ+3uN9XzGnan0j6xdebHv5sDZ2LfDdbtnEUN2Il+RJjB cmqGM+BCXcv4bJsoArQ7K3cEPEa9kiS4jW2g+IRmJx8IwWMG+l59zFFBzIXRlwbd 208F8LABLveUmxIFtMSBQJf8LY6ozCLBM9y7Hx9SlNJ6+MvGB6fBA== From: Alexander Gordeev To: Andrew Morton , Andrey Ryabinin Cc: Hugh Dickins , Nicholas Piggin , Guenter Roeck , Juergen Gross , Jeremy Fitzhardinge , linux-kernel@vger.kernel.org, linux-mm@kvack.org, kasan-dev@googlegroups.com, sparclinux@vger.kernel.org, xen-devel@lists.xenproject.org, linuxppc-dev@lists.ozlabs.org, linux-s390@vger.kernel.org Subject: [PATCH v1 3/4] mm: Protect kernel pgtables in apply_to_pte_range() Date: Mon, 7 Apr 2025 17:11:29 +0200 Message-ID: <11dbe3ac88130dbd2b8554f9369cd93fe138c655.1744037648.git.agordeev@linux.ibm.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: References: MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-GUID: Gsml-lqe4-oFygMCgc_o4kWGTfpeXFVg X-Proofpoint-ORIG-GUID: 9HS87Mcesg4PbHflITRoaij8qwSTG9RL X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1095,Hydra:6.0.680,FMLib:17.12.68.34 definitions=2025-04-07_04,2025-04-03_03,2024-11-22_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 malwarescore=0 clxscore=1011 bulkscore=0 impostorscore=0 suspectscore=0 lowpriorityscore=0 mlxscore=0 adultscore=0 phishscore=0 priorityscore=1501 spamscore=0 mlxlogscore=828 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.19.0-2502280000 definitions=main-2504070104 The lazy MMU mode can only be entered and left under the protection of the page table locks for all page tables which may be modified. Yet, when it comes to kernel mappings apply_to_pte_range() does not take any locks. That does not conform arch_enter|leave_lazy_mmu_mode() semantics and could potentially lead to re-schedulling a process while in lazy MMU mode or racing on a kernel page table updates. Signed-off-by: Alexander Gordeev Reviewed-by: Nicholas Piggin --- mm/kasan/shadow.c | 7 ++----- mm/memory.c | 5 ++++- 2 files changed, 6 insertions(+), 6 deletions(-) diff --git a/mm/kasan/shadow.c b/mm/kasan/shadow.c index edfa77959474..6531a7aa8562 100644 --- a/mm/kasan/shadow.c +++ b/mm/kasan/shadow.c @@ -308,14 +308,14 @@ static int kasan_populate_vmalloc_pte(pte_t *ptep, unsigned long addr, __memset((void *)page, KASAN_VMALLOC_INVALID, PAGE_SIZE); pte = pfn_pte(PFN_DOWN(__pa(page)), PAGE_KERNEL); - spin_lock(&init_mm.page_table_lock); if (likely(pte_none(ptep_get(ptep)))) { set_pte_at(&init_mm, addr, ptep, pte); page = 0; } - spin_unlock(&init_mm.page_table_lock); + if (page) free_page(page); + return 0; } @@ -401,13 +401,10 @@ static int kasan_depopulate_vmalloc_pte(pte_t *ptep, unsigned long addr, page = (unsigned long)__va(pte_pfn(ptep_get(ptep)) << PAGE_SHIFT); - spin_lock(&init_mm.page_table_lock); - if (likely(!pte_none(ptep_get(ptep)))) { pte_clear(&init_mm, addr, ptep); free_page(page); } - spin_unlock(&init_mm.page_table_lock); return 0; } diff --git a/mm/memory.c b/mm/memory.c index f0201c8ec1ce..1f3727104e99 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -2926,6 +2926,7 @@ static int apply_to_pte_range(struct mm_struct *mm, pmd_t *pmd, pte = pte_offset_kernel(pmd, addr); if (!pte) return err; + spin_lock(&init_mm.page_table_lock); } else { if (create) pte = pte_alloc_map_lock(mm, pmd, addr, &ptl); @@ -2951,7 +2952,9 @@ static int apply_to_pte_range(struct mm_struct *mm, pmd_t *pmd, arch_leave_lazy_mmu_mode(); - if (mm != &init_mm) + if (mm == &init_mm) + spin_unlock(&init_mm.page_table_lock); + else pte_unmap_unlock(mapped_pte, ptl); *mask |= PGTBL_PTE_MODIFIED; From patchwork Mon Apr 7 15:11:30 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexander Gordeev X-Patchwork-Id: 14041048 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 8AD2BC369A2 for ; Mon, 7 Apr 2025 15:12:04 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.940331.1340149 (Exim 4.92) (envelope-from ) id 1u1o8f-0005le-HD; Mon, 07 Apr 2025 15:11:53 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 940331.1340149; Mon, 07 Apr 2025 15:11:53 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1u1o8f-0005lE-Bs; Mon, 07 Apr 2025 15:11:53 +0000 Received: by outflank-mailman (input) for mailman id 940331; Mon, 07 Apr 2025 15:11:52 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1u1o8e-0005V9-9Y for xen-devel@lists.xenproject.org; Mon, 07 Apr 2025 15:11:52 +0000 Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id a01b51c7-13c2-11f0-9ffb-bf95429c2676; Mon, 07 Apr 2025 17:11:49 +0200 (CEST) Received: from pps.filterd (m0356517.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 537E3FIb025560; Mon, 7 Apr 2025 15:11:35 GMT Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 45vg4q8cmm-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 07 Apr 2025 15:11:34 +0000 (GMT) Received: from m0356517.ppops.net (m0356517.ppops.net [127.0.0.1]) by pps.reinject (8.18.0.8/8.18.0.8) with ESMTP id 537EoQ3W019216; Mon, 7 Apr 2025 15:11:34 GMT Received: from ppma21.wdc07v.mail.ibm.com (5b.69.3da9.ip4.static.sl-reverse.com [169.61.105.91]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 45vg4q8cmf-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 07 Apr 2025 15:11:34 +0000 (GMT) Received: from pps.filterd (ppma21.wdc07v.mail.ibm.com [127.0.0.1]) by ppma21.wdc07v.mail.ibm.com (8.18.1.2/8.18.1.2) with ESMTP id 537EKIMr013925; Mon, 7 Apr 2025 15:11:33 GMT Received: from smtprelay04.fra02v.mail.ibm.com ([9.218.2.228]) by ppma21.wdc07v.mail.ibm.com (PPS) with ESMTPS id 45ufunecfm-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 07 Apr 2025 15:11:32 +0000 Received: from smtpav04.fra02v.mail.ibm.com (smtpav04.fra02v.mail.ibm.com [10.20.54.103]) by smtprelay04.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 537FBVTI19595766 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 7 Apr 2025 15:11:31 GMT Received: from smtpav04.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 03DF82004D; Mon, 7 Apr 2025 15:11:31 +0000 (GMT) Received: from smtpav04.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id E2DFB20043; Mon, 7 Apr 2025 15:11:30 +0000 (GMT) Received: from tuxmaker.boeblingen.de.ibm.com (unknown [9.152.85.9]) by smtpav04.fra02v.mail.ibm.com (Postfix) with ESMTPS; Mon, 7 Apr 2025 15:11:30 +0000 (GMT) Received: by tuxmaker.boeblingen.de.ibm.com (Postfix, from userid 55669) id 88272E175F; Mon, 07 Apr 2025 17:11:30 +0200 (CEST) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: a01b51c7-13c2-11f0-9ffb-bf95429c2676 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=pp1; bh=pLqNtHwxdz+KjzF0o 4al3IuUdgKeg437/opuwF8rOgY=; b=GG9/qRMTr3c234BdNbl8qcd6JICn1dr6T h+1CwgnJosxDoTGQHDJLnBC+r8sdfje311yGnSrC48KNz6kVG3ETEbXxkIldtS4P DeWcQ/wSyDFqcrEo/l5mNQ2K0Lp2iKaRYkSmtxXTFAGqSvnaN8aWIz5IKTOTOKFq +JDtlQ3sQxmiVDXWN/y5otPtf5J6aL25Ojv2dz/9vWnkrf3ALjS2QQAfJwUkVTgb dIt52+/rxwX4BWgeopjfh5rB3DIp5dxMR6YbMdiedI2eDNzrYRJVARzz4+Y3g9I/ CxasV3YuMtzXVUxrwLXF7t0esmIiJJgRgZro+hqrnkdilacrPiCpg== From: Alexander Gordeev To: Andrew Morton , Andrey Ryabinin Cc: Hugh Dickins , Nicholas Piggin , Guenter Roeck , Juergen Gross , Jeremy Fitzhardinge , linux-kernel@vger.kernel.org, linux-mm@kvack.org, kasan-dev@googlegroups.com, sparclinux@vger.kernel.org, xen-devel@lists.xenproject.org, linuxppc-dev@lists.ozlabs.org, linux-s390@vger.kernel.org Subject: [PATCH v1 4/4] mm: Allow detection of wrong arch_enter_lazy_mmu_mode() context Date: Mon, 7 Apr 2025 17:11:30 +0200 Message-ID: <5204eaec309f454efcb5a799c9e0ed9da1dff971.1744037648.git.agordeev@linux.ibm.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: References: MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-ORIG-GUID: wG_AoPwbLBLy8-Y75yYVC-cpQvofPbGp X-Proofpoint-GUID: Yijp_AWT6tzwOtSLybGJbVPGYVge3z23 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1095,Hydra:6.0.680,FMLib:17.12.68.34 definitions=2025-04-07_04,2025-04-03_03,2024-11-22_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 adultscore=0 impostorscore=0 mlxscore=0 priorityscore=1501 phishscore=0 mlxlogscore=935 malwarescore=0 clxscore=1015 suspectscore=0 spamscore=0 bulkscore=0 lowpriorityscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.19.0-2502280000 definitions=main-2504070104 The lazy MMU batching may be only be entered and left under the protection of the page table locks for all page tables which may be modified. Yet, there were cases arch_enter_lazy_mmu_mode() was called without the locks taken, e.g. commit b9ef323ea168 ("powerpc/64s: Disable preemption in hash lazy mmu mode"). Make default arch_enter|leave|flush_lazy_mmu_mode() callbacks complain at least in case the preemption is enabled to detect wrong contexts. Most platforms do not implement the callbacks, so to aovid a performance impact allow the complaint when CONFIG_DEBUG_VM option is enabled only. Signed-off-by: Alexander Gordeev Reviewed-by: Nicholas Piggin --- include/linux/pgtable.h | 15 ++++++++++++--- 1 file changed, 12 insertions(+), 3 deletions(-) diff --git a/include/linux/pgtable.h b/include/linux/pgtable.h index e2b705c14945..959590bb66da 100644 --- a/include/linux/pgtable.h +++ b/include/linux/pgtable.h @@ -232,9 +232,18 @@ static inline int pmd_dirty(pmd_t pmd) * and the mode cannot be used in interrupt context. */ #ifndef __HAVE_ARCH_ENTER_LAZY_MMU_MODE -#define arch_enter_lazy_mmu_mode() do {} while (0) -#define arch_leave_lazy_mmu_mode() do {} while (0) -#define arch_flush_lazy_mmu_mode() do {} while (0) +static inline void arch_enter_lazy_mmu_mode(void) +{ + VM_WARN_ON(preemptible()); +} +static inline void arch_leave_lazy_mmu_mode(void) +{ + VM_WARN_ON(preemptible()); +} +static inline void arch_flush_lazy_mmu_mode(void) +{ + VM_WARN_ON(preemptible()); +} #endif #ifndef pte_batch_hint