From patchwork Fri Mar 21 13:06:27 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexandre Ghiti X-Patchwork-Id: 14025406 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 815C5C36000 for ; Fri, 21 Mar 2025 13:07:44 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id B65B0280004; Fri, 21 Mar 2025 09:07:42 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id B1465280001; Fri, 21 Mar 2025 09:07:42 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A047B280004; Fri, 21 Mar 2025 09:07:42 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 85E26280001 for ; Fri, 21 Mar 2025 09:07:42 -0400 (EDT) Received: from smtpin25.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 7D713140664 for ; Fri, 21 Mar 2025 13:07:43 +0000 (UTC) X-FDA: 83245585206.25.76A3D99 Received: from mail-wr1-f48.google.com (mail-wr1-f48.google.com [209.85.221.48]) by imf29.hostedemail.com (Postfix) with ESMTP id 6FD27120015 for ; Fri, 21 Mar 2025 13:07:41 +0000 (UTC) Authentication-Results: imf29.hostedemail.com; dkim=pass header.d=rivosinc-com.20230601.gappssmtp.com header.s=20230601 header.b=GF3kNioC; spf=pass (imf29.hostedemail.com: domain of alexghiti@rivosinc.com designates 209.85.221.48 as permitted sender) smtp.mailfrom=alexghiti@rivosinc.com; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1742562461; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=+81In4ilDZ/dsxug/tXtQcTDxsQusJyemdqH5+0jxZc=; b=Zewa5lZ4euhe4ly1IIrHNR2BgHGpPs3cyXOdvDUGH35BE8y7chg8r79g5+73oErpJ0rHR1 Y+sPtq/I/PsYKq4SmspYNVtH6Jt2kHcnL/Avp/0SVK4gFb30I+EnmBzfakY6cD9uJuHJM3 p/ZEvN+UjnkayDIicsDUKCjdOGC/ebU= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1742562461; a=rsa-sha256; cv=none; b=dPiCd2+QCoAsNQOC6tC16+etbdSUKaW4uUm2490odpSA3jLoM2oOqNFKpzos6JTRPone2A aOAfguoOFOVGHWIAmj0WAary2lWg2d1DdBrFiOJVQHWPCpyUIF4YaolO4JAz4wMuUSvWJw y4b5RofgozdCvb8be1vHgEm0DGEKIks= ARC-Authentication-Results: i=1; imf29.hostedemail.com; dkim=pass header.d=rivosinc-com.20230601.gappssmtp.com header.s=20230601 header.b=GF3kNioC; spf=pass (imf29.hostedemail.com: domain of alexghiti@rivosinc.com designates 209.85.221.48 as permitted sender) smtp.mailfrom=alexghiti@rivosinc.com; dmarc=none Received: by mail-wr1-f48.google.com with SMTP id ffacd0b85a97d-39130ee05b0so1850763f8f.3 for ; Fri, 21 Mar 2025 06:07:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc-com.20230601.gappssmtp.com; s=20230601; t=1742562460; x=1743167260; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=+81In4ilDZ/dsxug/tXtQcTDxsQusJyemdqH5+0jxZc=; b=GF3kNioCOPC2yonsupczSD2LnqY/tlUSFqeaAOS6AlwgeYkyHryrcuecB4x5o1tYAd ejr1aw0o5C3GMmK2gNEhV6SEOPNRYBnJGrxqvhQWIi7bEfarFZY08dBKR+ln00POeBun yNsSfSzeEDX6SejVqFFjIaDmbEF/YojwscRT/SL8/mz92B32sMnJ3i35bCQ5tk/OwQX6 +HJ/nA67q7eRgBH0Bb8XvTA5fVRXD4/E1nRdLmK7zmLYuoW4YbLLSnGCS5oCVDlVkxdR ziSOMoOJlsrwsH5RNzQMsEcCD04muR1izVFJc7J+bEWGhrBeeNfb/o1p79QTwBlwL8eK BjAg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1742562460; x=1743167260; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=+81In4ilDZ/dsxug/tXtQcTDxsQusJyemdqH5+0jxZc=; b=PTZ3qidgoaTSIKLZEw4I9oyv5DMOxVGVfC6U25m314YxQN/SpweuLi0EsUA/lCfCxG U7c20/JjlZWRgcUKb2gzns2+90hJ2XBN2QaB0peAGWKkRwWfj/fJAxD+HiJEw7tbr6Lh 9uUoLJOM8xn1JYQiM5UIgXKFiDeQgy47wLwdgqjafc9Z+zIHEJibvokmYLeUF914PVAi hDNgzEFsVj3UpXqpX1ifA8FtkT8LNXIElItxf7D5TWTBs/KrickH3XMmFMLPtxTLCFVa 4TILc7qTOQ+WPRym++H8AGYVaKtanfD2SnVvo1JhqeM9Y3ULJGU78pM3fJOHLD+ffLen i5Dw== X-Forwarded-Encrypted: i=1; AJvYcCUZjUKBWD055TLeCU9SCE4ixcwvDMawEiI4bJloXhIKn1MPjqSIKnkKJc35h6a0iZu9sxDYT5yg3A==@kvack.org X-Gm-Message-State: AOJu0YzWX0XmWX5ARKooTZXOSKBpFR929doSMewizHW/StvReV4iRlaf LNojxsVAMC2WBN0hdLSu2J9KKzruen8H9iNYApXcjuVbsPUS016zvFZZfU/7FwQ= X-Gm-Gg: ASbGnctLouvgXOKbcEw/4sh9ieY4lpjehm2BZX0iaT1wl+pbZQmLumODl1ACJwmIvMu otXp/9d/w3J7FReiGpdo5wN2Zcb2vC+d1a6fvEgVza1KMuIGZx+euyzPlUP70jwPgzEcT9maGbj gmcj/0PdxlMhebF8ExlXw/hr3D/Dt8DdOzEkC8EoW85AMbBu02icjG5OFXAam/YmCYaiQi5tGQ0 CJSSwaIZvOjp66nAdwXHS2NnECjSK8tT7UFF5MulmgLat7VZRNsIuiHFYPJkKm+ah4Q5GpPSjU0 BR8xnxs2KrYPkmCIN/QYKjP9Xg1klobP0h1i3uObzSPSZJ5qTZ8F9X/cOkSZZ07r1oLn8A== X-Google-Smtp-Source: AGHT+IFGcNIQFtkSrVrMaEJyHkzdoC3QT/2GtmdRgmqzsJ6C1xedRKK8otT6Nr1+/A9BoBAqiRGLXw== X-Received: by 2002:a05:6000:1f86:b0:391:4743:6de4 with SMTP id ffacd0b85a97d-3997f940e28mr2542677f8f.50.1742562459890; Fri, 21 Mar 2025 06:07:39 -0700 (PDT) Received: from alex-rivos.ba.rivosinc.com ([2001:861:3382:ef90:3d12:52fe:c1cc:c94]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-3997f9efd6csm2364857f8f.90.2025.03.21.06.07.38 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 21 Mar 2025 06:07:39 -0700 (PDT) From: Alexandre Ghiti To: Catalin Marinas , Will Deacon , Ryan Roberts , Mark Rutland , Matthew Wilcox , Paul Walmsley , Palmer Dabbelt , Alexandre Ghiti , Andrew Morton , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org, linux-mm@kvack.org Cc: Alexandre Ghiti Subject: [PATCH v5 1/9] riscv: Safely remove huge_pte_offset() when manipulating NAPOT ptes Date: Fri, 21 Mar 2025 14:06:27 +0100 Message-Id: <20250321130635.227011-2-alexghiti@rivosinc.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20250321130635.227011-1-alexghiti@rivosinc.com> References: <20250321130635.227011-1-alexghiti@rivosinc.com> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam10 X-Rspamd-Queue-Id: 6FD27120015 X-Stat-Signature: ynf663dfbhz6unmn43tr9quictbpj4na X-HE-Tag: 1742562461-893210 X-HE-Meta: U2FsdGVkX18Qz6LsEvim7S6gxCIVHEsB26idjlPbMD9kD4jdWVwC0k7R8kznxoUp3aBCXXU+2LOMgZY7kGLJd6lM3TxCiLIoNLZyqCefcLDI79nYeXCimwL3ZwjdtPxVIpA6qLOL36o6CazAysRZ92rnImA4GTxjsc4m3kGEQ4GSrEqCQOxhBAHBJMd1LaszwS5Ku9WAhrF7sljci3kXRlcwAGlP/7fC3bWa/BZ7+vp4o51WCSdLhEptPmUPmhH0ab//m2Cy9YS38v23l4uYrzbp1/tATJbe2avNFxG2Yq2/YMUeTZ9DJEt1RKsP3xpKr1u30Dct/PdM7/zidV2iqcKTLTj/4TKwQggvY7Z19N6qmCnnRvyugzEDA7nJRgB2K2WgHocPvcvt8Bz2UrIyeD425RQ7OXGKTewKBnkaG2Fzc+fqPmSiEX1KWgdtCYT+nOd48v9DFsbv0MeLEk5gNy44Q+LucqGPTIa6+9bLYLFmTpfJOYA3hv4LzLfQl5VqO/kvBQIOLAwh5X7cKQpTU4UJkH7/Nmd2kAreYWQcKyxNJCTYgXHfuQkYNdq28TqrxkLlnQA6vKP5R+GnTCUKIiP6KEqfrKANlz4AKs2gPj/5NO5PShqQoC0kMA0BoIJZl0K7oRRhE/5olKd4pdv9clz8bQGtwplqyRZCIa1WGsciIM9cKq5tzolsrjZJXL2g/sAyHfWD6Iz1Wiu2JlHyP1PClqR20l0mZHps0ar1VifJ4yTQN3XeUBKK14cpv2J0ylcKA8q+pBqZoEMGwmAH5gIL312UzYEgcFDJyL7jMsIp6O3GB7cvJswV5cc+V6wlYuEbntJxSHwpklb9opZYnEaI1d5UVlJ9Yo13dJjRk7NCQa+8u6RM4A2R1P0HNQXImh5uPCHKC3/T7e1XlckaGIEJuwjF6cmTX4wyXntvnR9HxLVN8Ea2xHy+twQEVAfuqJ215l7YyZwJrpotrBU xPxZaidG hjp55sumjN9lR7JlRwMrijpESoZN2Ulydb+QWeJDeFzX/4DH1/rgrenXCbyVMwvcLw+V75djwxMFqnJF04MYDGUXfnMf0YMRkRssKrW7av2yFHLRDenxhqGRZ+J/5i/D+04sR/AgLUFz+nTVWO+YPvjeLU94yNscl/gsXUpDDh3wsjhAazXqRLNc+nbPl6jIUv5vI7B4yCdAbOmycCle2CYj3b0dqk+ZK05vZU/Q+HB+xT348ocxIdO1t2/AZF1CGdDnWpfiei0AP0ZC/1obHUXxkFYlQrJ+KA9FZZ9pL3k3oousLyf1ZFRoU42U9y7emYkSALoi/IUgcitVV8Q85LKPZ7uwgRiu9ArxJx5I+qTNh6+g+x5S/dQJv40MDsvM9b7o0X8zb6aUIsduH6UJ1+ePAyl0AQGH+PMDO8ORapLyVvxuudCSdWkwdxior6EeyPEIXy8I26us7f6kMThAv020DsA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: The pte_t pointer is expected to point to the first entry of the NAPOT mapping so no need to use huge_pte_offset(), similarly to what is done in arm64. Signed-off-by: Alexandre Ghiti --- arch/riscv/mm/hugetlbpage.c | 2 -- 1 file changed, 2 deletions(-) diff --git a/arch/riscv/mm/hugetlbpage.c b/arch/riscv/mm/hugetlbpage.c index 375dd96bb4a0..3192ad804279 100644 --- a/arch/riscv/mm/hugetlbpage.c +++ b/arch/riscv/mm/hugetlbpage.c @@ -287,7 +287,6 @@ int huge_ptep_set_access_flags(struct vm_area_struct *vma, order = napot_cont_order(pte); pte_num = napot_pte_num(order); - ptep = huge_pte_offset(mm, addr, napot_cont_size(order)); orig_pte = get_clear_contig_flush(mm, addr, ptep, pte_num); if (pte_dirty(orig_pte)) @@ -334,7 +333,6 @@ void huge_ptep_set_wrprotect(struct mm_struct *mm, order = napot_cont_order(pte); pte_num = napot_pte_num(order); - ptep = huge_pte_offset(mm, addr, napot_cont_size(order)); orig_pte = get_clear_contig_flush(mm, addr, ptep, pte_num); orig_pte = pte_wrprotect(orig_pte); From patchwork Fri Mar 21 13:06:28 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexandre Ghiti X-Patchwork-Id: 14025407 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id C8750C36000 for ; Fri, 21 Mar 2025 13:08:46 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D3A5B280005; Fri, 21 Mar 2025 09:08:44 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id CEA6A280001; Fri, 21 Mar 2025 09:08:44 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B8B63280005; Fri, 21 Mar 2025 09:08:44 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 9B3CD280001 for ; Fri, 21 Mar 2025 09:08:44 -0400 (EDT) Received: from smtpin26.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 7F0D682114 for ; Fri, 21 Mar 2025 13:08:45 +0000 (UTC) X-FDA: 83245587810.26.C537ACD Received: from mail-wm1-f52.google.com (mail-wm1-f52.google.com [209.85.128.52]) by imf20.hostedemail.com (Postfix) with ESMTP id 925791C0036 for ; Fri, 21 Mar 2025 13:08:43 +0000 (UTC) Authentication-Results: imf20.hostedemail.com; dkim=pass header.d=rivosinc-com.20230601.gappssmtp.com header.s=20230601 header.b=QUyKaCTO; dmarc=none; spf=pass (imf20.hostedemail.com: domain of alexghiti@rivosinc.com designates 209.85.128.52 as permitted sender) smtp.mailfrom=alexghiti@rivosinc.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1742562523; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=MHcTeoTN4G5JoOrCU+WuSXOyCDmNaouJRLSJHK0ciYw=; b=UvJn3z1S6mLloT+iWQcv34/uCqMG8X02GInSbzkyCBDxuWNFDl9uzvVvIVpf+GILL6P+nP SbiPaZtuuRcvudetIKgAoulBzQuSJbkVYLemq9adnL/DJvW18Okei4vvqBY8QtnzewhBpz pOtxp0FKd/CvAL9TA8e0Wzp9HE+bC98= ARC-Authentication-Results: i=1; imf20.hostedemail.com; dkim=pass header.d=rivosinc-com.20230601.gappssmtp.com header.s=20230601 header.b=QUyKaCTO; dmarc=none; spf=pass (imf20.hostedemail.com: domain of alexghiti@rivosinc.com designates 209.85.128.52 as permitted sender) smtp.mailfrom=alexghiti@rivosinc.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1742562523; a=rsa-sha256; cv=none; b=uDl8qJ25JAE9lfxSpUXVLACMOYyJI6JXlol9e8XklQ6e4p81OVRhjrBwRJpU7UwPF38VjM NNjzv+rL6nZ03DHYgfvBRDjzMyBz1BBNGpVO+HFwRoSXr7FQibOnZcmtVEshW/vALF5fDd Gi1nSXfrZVnQRdC162O9ioV03XdSuoA= Received: by mail-wm1-f52.google.com with SMTP id 5b1f17b1804b1-43cf58eea0fso11034515e9.0 for ; Fri, 21 Mar 2025 06:08:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc-com.20230601.gappssmtp.com; s=20230601; t=1742562522; x=1743167322; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=MHcTeoTN4G5JoOrCU+WuSXOyCDmNaouJRLSJHK0ciYw=; b=QUyKaCTOLb1f+YV9yLx/9NQuPSnbMAHsSRy3xusAKqVM0xIBX+8kQOmr3BnaGX80qx W5QFe4uvWqjD8plzHU3vecbSEGMCq7jMQt5aaOyBoZYCfwhD7gphpPe8mKDq5/F9NOSm +6MgTEDdxLImZmAbN5S4oNMG7zmREF6Yscuwa3qPPe9kwVxmHUpbfJBEGMF9mhgFH+TJ fs0gB/vI35/VesektThi+LDnCy9BY9HDTzQzdHjAMEbZN1YHA7qoMW1tUCkMxe+Ct+MT 2TRq+KLCTVoeIeCwS2gYHhtgx30Wn+6wd3zUEWlXOj6Mwz0W2ESCM6brvl+ApSG0Hb7o fAUw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1742562522; x=1743167322; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=MHcTeoTN4G5JoOrCU+WuSXOyCDmNaouJRLSJHK0ciYw=; b=txDiPw2KZtz4ZDrcab3BQUXc5w6oUAH9FoF7oIzuwkPBUdip9dRkSLwZ9N+7x1Ht42 F5y8xDDvbayjUxIiqm1d4t07sLEATJqRQEjTin7qM6jrCEH196yq3QnQZ8uOIu8dMJWM aZ/aaB/TRzq5IARYy8kEGdwrMA9J/ycQTmRZjkyZAKr6BT+lbOBRNqH0vkMbwKJiuNCZ Kt5gL3NkcqXXsh5vkmn+fAleYV/WKLMVvnzSVASfrJ9fqR2sCQUb7ns0vWOoW2SmFW3Q HV/9YyGa4+QlZ+OLc5A/9pdQDqG6GjSC4K0Pbw6xeUNWsoO8fB2fIiWHp0HI9Frmw1J9 5ofA== X-Forwarded-Encrypted: i=1; AJvYcCXUFfGVjSaq98ipWRAiTHchdP9hIq+GR6ZHbYjQFcXkl7sPeKZJqTDNKuJBfpIG1N7DP+OqZGZ2kA==@kvack.org X-Gm-Message-State: AOJu0YyEy1qHYPgsD8e91smlKl2DOonfhgYLnWvhXGgaHqqawotPvyEj VAX8cNPruQY7ZIGI7z05Yvg2gGPMz+CjvWRw4kQ1ihwHeI+y9L0vssGF5h10rTk= X-Gm-Gg: ASbGncsbAlJtfMkZuLSwE3ZdF6ToQyJOAATHHvwOUf19ol7soC+KzAUQU7iSaNjv2dY STLgn0hOLfLBm+/NR47kcirW/Us6qXw2EC4xecyFjwmKdHnQt+29l9us5bje3h53LBwNBKpR/s3 qp/aDHxCdG0uQU6Z9qEmL+9JJNnCQhFdUI5MT2DGvYMlC3M8j0+4yvItSrZbYa+K4JIp32j6+0i 8VIWppUfmnmrCJDODLbGaI0prCzvEr7Uj422EgDIaWFQtqABhCTIrPGrawvrsML2UEA8ouJBKS+ thNnnqbqQmwaSJuN02jIibI3e+3e5XL6JeJ+q9BcVxHFPj6gHSj6ta5MtoYtnyFThV95TqcSKKI wlXyx X-Google-Smtp-Source: AGHT+IEMJBrD5VHSiFntLl7eq4OXY9R2l0VvwmSKG3LcIwl+yD5imfUnEeI+G2u9jHEtl52hrfshfg== X-Received: by 2002:a05:600c:46d4:b0:43c:e6d1:efe7 with SMTP id 5b1f17b1804b1-43d50a3b62emr18553825e9.26.1742562521606; Fri, 21 Mar 2025 06:08:41 -0700 (PDT) Received: from alex-rivos.ba.rivosinc.com ([2001:861:3382:ef90:3d12:52fe:c1cc:c94]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-43d43fdec51sm78104015e9.27.2025.03.21.06.08.40 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 21 Mar 2025 06:08:41 -0700 (PDT) From: Alexandre Ghiti To: Catalin Marinas , Will Deacon , Ryan Roberts , Mark Rutland , Matthew Wilcox , Paul Walmsley , Palmer Dabbelt , Alexandre Ghiti , Andrew Morton , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org, linux-mm@kvack.org Cc: Alexandre Ghiti Subject: [PATCH v5 2/9] riscv: Restore the pfn in a NAPOT pte when manipulated by core mm code Date: Fri, 21 Mar 2025 14:06:28 +0100 Message-Id: <20250321130635.227011-3-alexghiti@rivosinc.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20250321130635.227011-1-alexghiti@rivosinc.com> References: <20250321130635.227011-1-alexghiti@rivosinc.com> MIME-Version: 1.0 X-Rspamd-Server: rspam11 X-Rspamd-Queue-Id: 925791C0036 X-Stat-Signature: cd1etkgpapfkcdt3nkqa8z5gazerbc1f X-Rspam-User: X-HE-Tag: 1742562523-347164 X-HE-Meta: U2FsdGVkX19O0DDfUQUoYe8ZhJmJ32R7/aZbhTjlgUWPhR1iCpOlY7p3QRo9A4euZlsS6MKCxjqWE5cU48E7vslb2JJdt0bBO7ZGnVcOBj/VplsbyGUEWBvZNXmteiVknnz9EzdhGYJnXBARoX2eabpy1QOZYhfSnQLgPAryjOKZSGZx98DUZO8m4dx3JzItK6Nm1gGtCJUcIuiGqJ/n5hmxqqBEy37FomHcZ/bhyRo8yVewaBFDfi1df4SMERHTlWxJolvA4SSR9maIL4jATjXw7Dz1bHYheuqvarvmjtQytmwgPXIVyOESHwBf3s+W58lEh2VFkgQlqIpP/IM2OjWRlyeAdkOzt4xF1iWmagRuwGp79iT/JCrGApQ2q/H29acJfS+JrtEpmxoyqhIbadIp5BfOKXUH8aWneUfXi4pcpt9U17sgHMghoBeuzQUGVp8pdO0DgzQ8yZqTvYqzHTV9jEnRkE7bLt+/ZnREICTBmlzPGE9k5ssY8TBDgX6CQJRmi6hQqBeTygpk6a5mkL4qYsKxM0Je74x6c6OF9PUEYxConEaYTFxD7CAsGZK2nXVYlRB/qjKdsaFR3Ye3AHexMjGd4cEKeYvpYBJsf/W6rfn5kNUexSCMXZlxiWUJyHJ245YpREtVZIjvg4LO/oKR6ZzcNsow1gMLwZMD2Bb874blgE1Y3tW6DNKAP0MLNLzo/YljOEaXyM9y2Rmsd3MmNaDxMnCwHgPIyoI89JIcx9wAA7ZLlLBaQbEE9zt9CgN5gsXk4+hXJir4BMxKHGeAX6GWlQCfAhWlu9HRdAlYdp4278cCcZpq0zsm2zVJUuiLFiPLNiF4HwUq4q2e80MX7M+MbJySnXNmHSUehvFzvnIg1vzXD7fxEX6svqQ+AyFfp4oPVC6qISixyB04GZRBXBi4zppNAQsJbaA65nuajLU721lORjCPWzitm6IGuAj2AR3S0nHrosKPTg5 EE8/gssy TEgTMjgEGesNv1jVswltuXmmNkWa8NkBQZkrobGgXaF7FtQoqwhMLzxdmJoZZVrcJ5M3YTKttZisXmdAM7fqAxynOnMn863dyjPPbqRM51ZrauQPdNMbtYnUKpq36Li7dbZRYQ1qBMNnnLNBCcRdqzTG9RrFx4zd2HGaKCDAtBbV8PPhnVE02ObC25I/2mbFUMNLCPwBtvZSGCgZR64jS58GhzE2tg6oZSVxXyoAUANb+2zLv6gL6dmnhkI54iOAxu5976ao2e1Q1/fu0lvzfcDzovN5gCQIjvtw8vfD94aDAQ0T8OqdW+NmF2Fjas7nanSKjDqzK6Y1LR6bgRn4BU7W+CGC/lBMRuUlz+SJ2ci8dA2yrreUrNxVyaJuIzRBm5x3NpKNSpsskcjMjAN/1wkUkoH1tdLZhvsvPDRMiT26egkw= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: The core mm code expects to be able to extract the pfn from a pte. NAPOT mappings work differently since its ptes actually point to the first pfn of the mapping, the other bits being used to encode the size of the mapping. So modify ptep_get() so that it returns a pte value that contains the *real* pfn (which is then different from what the HW expects) and right before storing the ptes to the page table, reset the pfn LSBs to the size of the mapping. And make sure that all NAPOT mappings are set using set_ptes(). Signed-off-by: Alexandre Ghiti --- arch/riscv/include/asm/pgtable-64.h | 11 ++ arch/riscv/include/asm/pgtable.h | 155 ++++++++++++++++++++++++---- arch/riscv/mm/hugetlbpage.c | 15 ++- 3 files changed, 152 insertions(+), 29 deletions(-) diff --git a/arch/riscv/include/asm/pgtable-64.h b/arch/riscv/include/asm/pgtable-64.h index 0897dd99ab8d..cddbe426f618 100644 --- a/arch/riscv/include/asm/pgtable-64.h +++ b/arch/riscv/include/asm/pgtable-64.h @@ -104,6 +104,17 @@ enum napot_cont_order { #define napot_cont_mask(order) (~(napot_cont_size(order) - 1UL)) #define napot_pte_num(order) BIT(order) +static inline bool is_napot_order(unsigned int order) +{ + unsigned int napot_order; + + for_each_napot_order(napot_order) + if (order == napot_order) + return true; + + return false; +} + #ifdef CONFIG_RISCV_ISA_SVNAPOT #define HUGE_MAX_HSTATE (2 + (NAPOT_ORDER_MAX - NAPOT_CONT_ORDER_BASE)) #else diff --git a/arch/riscv/include/asm/pgtable.h b/arch/riscv/include/asm/pgtable.h index 050fdc49b5ad..2e62d7e607db 100644 --- a/arch/riscv/include/asm/pgtable.h +++ b/arch/riscv/include/asm/pgtable.h @@ -296,6 +296,17 @@ static inline unsigned long pte_napot(pte_t pte) return pte_val(pte) & _PAGE_NAPOT; } +#define pte_valid_napot(pte) (pte_present(pte) && pte_napot(pte)) + +/* + * contpte is what we expose to the core mm code, this is not exactly a napot + * mapping since the size is not encoded in the pfn yet. + */ +static inline pte_t pte_mkcont(pte_t pte) +{ + return __pte(pte_val(pte) | _PAGE_NAPOT); +} + static inline pte_t pte_mknapot(pte_t pte, unsigned int order) { int pos = order - 1 + _PAGE_PFN_SHIFT; @@ -305,6 +316,12 @@ static inline pte_t pte_mknapot(pte_t pte, unsigned int order) return __pte((pte_val(pte) & napot_mask) | napot_bit | _PAGE_NAPOT); } +/* pte at entry must *not* encode the mapping size in the pfn LSBs. */ +static inline pte_t pte_clear_napot(pte_t pte) +{ + return __pte(pte_val(pte) & ~_PAGE_NAPOT); +} + #else static __always_inline bool has_svnapot(void) { return false; } @@ -314,17 +331,24 @@ static inline unsigned long pte_napot(pte_t pte) return 0; } +static inline pte_t pte_clear_napot(pte_t pte) +{ + return pte; +} + +static inline pte_t pte_mknapot(pte_t pte, unsigned int order) +{ + return pte; +} + +#define pte_valid_napot(pte) false + #endif /* CONFIG_RISCV_ISA_SVNAPOT */ /* Yields the page frame number (PFN) of a page table entry */ static inline unsigned long pte_pfn(pte_t pte) { - unsigned long res = __page_val_to_pfn(pte_val(pte)); - - if (has_svnapot() && pte_napot(pte)) - res = res & (res - 1UL); - - return res; + return __page_val_to_pfn(pte_val(pte)); } #define pte_page(x) pfn_to_page(pte_pfn(x)) @@ -559,8 +583,13 @@ static inline void __set_pte_at(struct mm_struct *mm, pte_t *ptep, pte_t pteval) #define PFN_PTE_SHIFT _PAGE_PFN_SHIFT -static inline void set_ptes(struct mm_struct *mm, unsigned long addr, - pte_t *ptep, pte_t pteval, unsigned int nr) +static inline pte_t ___ptep_get(pte_t *ptep) +{ + return READ_ONCE(*ptep); +} + +static inline void ___set_ptes(struct mm_struct *mm, unsigned long addr, + pte_t *ptep, pte_t pteval, unsigned int nr) { page_table_check_ptes_set(mm, ptep, pteval, nr); @@ -569,10 +598,13 @@ static inline void set_ptes(struct mm_struct *mm, unsigned long addr, if (--nr == 0) break; ptep++; + + if (unlikely(pte_valid_napot(pteval))) + continue; + pte_val(pteval) += 1 << _PAGE_PFN_SHIFT; } } -#define set_ptes set_ptes static inline void pte_clear(struct mm_struct *mm, unsigned long addr, pte_t *ptep) @@ -587,17 +619,6 @@ extern int ptep_set_access_flags(struct vm_area_struct *vma, unsigned long addre extern int ptep_test_and_clear_young(struct vm_area_struct *vma, unsigned long address, pte_t *ptep); -#define __HAVE_ARCH_PTEP_GET_AND_CLEAR -static inline pte_t ptep_get_and_clear(struct mm_struct *mm, - unsigned long address, pte_t *ptep) -{ - pte_t pte = __pte(atomic_long_xchg((atomic_long_t *)ptep, 0)); - - page_table_check_pte_clear(mm, pte); - - return pte; -} - #define __HAVE_ARCH_PTEP_SET_WRPROTECT static inline void ptep_set_wrprotect(struct mm_struct *mm, unsigned long address, pte_t *ptep) @@ -627,6 +648,100 @@ static inline int ptep_clear_flush_young(struct vm_area_struct *vma, return ptep_test_and_clear_young(vma, address, ptep); } +#ifdef CONFIG_RISCV_ISA_SVNAPOT +static inline pte_t pte_napot_clear_pfn(pte_t *ptep, pte_t pte) +{ + /* + * The pte we load has the N bit set and the size of the mapping in + * the pfn LSBs: keep the N bit and replace the mapping size with + * the *real* pfn since the core mm code expects to find it there. + * The mapping size will be reset just before being written to the + * page table in set_ptes(). + */ + if (unlikely(pte_valid_napot(pte))) { + unsigned int order = napot_cont_order(pte); + int pos = order - 1 + _PAGE_PFN_SHIFT; + unsigned long napot_mask = ~GENMASK(pos, _PAGE_PFN_SHIFT); + pte_t *orig_ptep = PTR_ALIGN_DOWN(ptep, sizeof(*ptep) * napot_pte_num(order)); + + pte = __pte((pte_val(pte) & napot_mask) + ((ptep - orig_ptep) << _PAGE_PFN_SHIFT)); + } + + return pte; +} + +#define __HAVE_ARCH_PTEP_GET_AND_CLEAR +static inline pte_t ptep_get_and_clear(struct mm_struct *mm, + unsigned long address, pte_t *ptep) +{ + pte_t pte = __pte(atomic_long_xchg((atomic_long_t *)ptep, 0)); + + pte = pte_napot_clear_pfn(ptep, pte); + + page_table_check_pte_clear(mm, pte); + + return pte; +} + +static inline pte_t __ptep_get(pte_t *ptep) +{ + pte_t pte = ___ptep_get(ptep); + + return pte_napot_clear_pfn(ptep, pte); +} + +static inline void __set_ptes(struct mm_struct *mm, unsigned long addr, + pte_t *ptep, pte_t pteval, unsigned int nr) +{ + if (unlikely(pte_valid_napot(pteval))) { + unsigned int order = ilog2(nr); + + if (!is_napot_order(order)) { + /* + * Something's weird, we are given a NAPOT pte but the + * size of the mapping is not a known NAPOT mapping + * size, so clear the NAPOT bit and map this without + * NAPOT support: core mm only manipulates pte with the + * real pfn so we know the pte is valid without the N + * bit. + */ + pr_err("Incorrect NAPOT mapping, resetting.\n"); + pteval = pte_clear_napot(pteval); + } else { + /* + * NAPOT ptes that arrive here only have the N bit set + * and their pfn does not contain the mapping size, so + * set that here. + */ + pteval = pte_mknapot(pteval, order); + } + } + + ___set_ptes(mm, addr, ptep, pteval, nr); +} + +#define ptep_get __ptep_get +#define set_ptes __set_ptes +#define set_contptes(mm, addr, ptep, pte, nr, pgsize) \ + set_ptes(mm, addr, ptep, pte, nr) +#else +#define ptep_get ___ptep_get +#define __ptep_get ___ptep_get +#define set_ptes ___set_ptes +#define __set_ptes ___set_ptes + +#define __HAVE_ARCH_PTEP_GET_AND_CLEAR +static inline pte_t ptep_get_and_clear(struct mm_struct *mm, + unsigned long address, pte_t *ptep) +{ + pte_t pte = __pte(atomic_long_xchg((atomic_long_t *)ptep, 0)); + + page_table_check_pte_clear(mm, pte); + + return pte; +} +#endif /* CONFIG_RISCV_ISA_SVNAPOT */ + #define pgprot_nx pgprot_nx static inline pgprot_t pgprot_nx(pgprot_t _prot) { diff --git a/arch/riscv/mm/hugetlbpage.c b/arch/riscv/mm/hugetlbpage.c index 3192ad804279..60b7e738b31a 100644 --- a/arch/riscv/mm/hugetlbpage.c +++ b/arch/riscv/mm/hugetlbpage.c @@ -190,7 +190,7 @@ pte_t arch_make_huge_pte(pte_t entry, unsigned int shift, vm_flags_t flags) for_each_napot_order(order) { if (shift == napot_cont_shift(order)) { - entry = pte_mknapot(entry, order); + entry = pte_mkcont(entry); break; } } @@ -267,8 +267,7 @@ void set_huge_pte_at(struct mm_struct *mm, clear_flush(mm, addr, ptep, pgsize, pte_num); - for (i = 0; i < pte_num; i++, ptep++, addr += pgsize) - set_pte_at(mm, addr, ptep, pte); + set_ptes(mm, addr, ptep, pte, pte_num); } int huge_ptep_set_access_flags(struct vm_area_struct *vma, @@ -280,7 +279,7 @@ int huge_ptep_set_access_flags(struct vm_area_struct *vma, struct mm_struct *mm = vma->vm_mm; unsigned long order; pte_t orig_pte; - int i, pte_num; + int pte_num; if (!pte_napot(pte)) return ptep_set_access_flags(vma, addr, ptep, pte, dirty); @@ -295,8 +294,7 @@ int huge_ptep_set_access_flags(struct vm_area_struct *vma, if (pte_young(orig_pte)) pte = pte_mkyoung(pte); - for (i = 0; i < pte_num; i++, addr += PAGE_SIZE, ptep++) - set_pte_at(mm, addr, ptep, pte); + set_ptes(mm, addr, ptep, pte, pte_num); return true; } @@ -324,7 +322,7 @@ void huge_ptep_set_wrprotect(struct mm_struct *mm, pte_t pte = ptep_get(ptep); unsigned long order; pte_t orig_pte; - int i, pte_num; + int pte_num; if (!pte_napot(pte)) { ptep_set_wrprotect(mm, addr, ptep); @@ -337,8 +335,7 @@ void huge_ptep_set_wrprotect(struct mm_struct *mm, orig_pte = pte_wrprotect(orig_pte); - for (i = 0; i < pte_num; i++, addr += PAGE_SIZE, ptep++) - set_pte_at(mm, addr, ptep, orig_pte); + set_ptes(mm, addr, ptep, orig_pte, pte_num); } pte_t huge_ptep_clear_flush(struct vm_area_struct *vma, From patchwork Fri Mar 21 13:06:29 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexandre Ghiti X-Patchwork-Id: 14025408 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id E30CEC36000 for ; Fri, 21 Mar 2025 13:09:47 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id EEFB1280006; Fri, 21 Mar 2025 09:09:45 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id E9E63280001; Fri, 21 Mar 2025 09:09:45 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D1CA3280006; Fri, 21 Mar 2025 09:09:45 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id AF377280001 for ; Fri, 21 Mar 2025 09:09:45 -0400 (EDT) Received: from smtpin05.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id AB39D5A8D8 for ; Fri, 21 Mar 2025 13:09:46 +0000 (UTC) X-FDA: 83245590372.05.38CD1AB Received: from mail-wm1-f45.google.com (mail-wm1-f45.google.com [209.85.128.45]) by imf29.hostedemail.com (Postfix) with ESMTP id 986BB12000A for ; Fri, 21 Mar 2025 13:09:44 +0000 (UTC) Authentication-Results: imf29.hostedemail.com; dkim=pass header.d=rivosinc-com.20230601.gappssmtp.com header.s=20230601 header.b=MKed6hgv; spf=pass (imf29.hostedemail.com: domain of alexghiti@rivosinc.com designates 209.85.128.45 as permitted sender) smtp.mailfrom=alexghiti@rivosinc.com; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1742562584; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=e8EtFYpkeeZSqWY8m05eSUUviRVNOc3GBKbkMNHHDeA=; b=rcffQpIPCtLQuFRha3xV6sSKVGA3cNYGRHa4xgzm0tZN3DrRGwfgU/XY21Ho4cgbMwnUub Lz7ZbGzoeWLCYyLtn4fY2ohzmCREwP1UIvGoMRFz7WPsQN2zuDR0XNqHrr+VZm8xiCG9Bu LMigA33wjWmFPK1eKrOgMpIEuo3DRyo= ARC-Authentication-Results: i=1; imf29.hostedemail.com; dkim=pass header.d=rivosinc-com.20230601.gappssmtp.com header.s=20230601 header.b=MKed6hgv; spf=pass (imf29.hostedemail.com: domain of alexghiti@rivosinc.com designates 209.85.128.45 as permitted sender) smtp.mailfrom=alexghiti@rivosinc.com; dmarc=none ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1742562584; a=rsa-sha256; cv=none; b=Hk27f8OfbLMWPtMamJfYs6tpEIVvL8NPCEY8ocqZjOv3gRHJsXBF48AerTG+jpa9MUJFD9 Ncd01nBJTBcGvJN1Q/+no5AP/UrrOWFGrZELCzO7UlPXW1j2B+dy1iTa+nchhCKBZLQCfC QCOdMXJIbwB9vuREtHJ/XCB0FZrLRI0= Received: by mail-wm1-f45.google.com with SMTP id 5b1f17b1804b1-43d0359b1fcso12730345e9.0 for ; Fri, 21 Mar 2025 06:09:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc-com.20230601.gappssmtp.com; s=20230601; t=1742562583; x=1743167383; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=e8EtFYpkeeZSqWY8m05eSUUviRVNOc3GBKbkMNHHDeA=; b=MKed6hgvRLknDj7hcelZIQQhXsg/iv+bwLVgQ+CxcOyi5kfhMS3AANNuszTc4WrFY4 aRUmuadj64nsc3n74uXwnmG0jI/m/+/hTU2iUCj99+uXRYSEKyJWphjHKgkz6uQA2SXV 51aj2oLPaTE9hSsfzH6IHGfgpyRegMkS9RYlukJpA3nZjccze9aHOKclNWQJyV0zdce5 P5PZ7iedAZdHHuJyy1KacnJQpTnaBh4UM0HaKp9KG0B5INPZINmqVVyb2hNk/PzdxvKQ 6xid+qHhWQLjZ28MO/ZgOTVr7xM+I0rC9WQhx19q4JDGHHTTf/XLIsci4hFoe+ERz7R7 xA6g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1742562583; x=1743167383; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=e8EtFYpkeeZSqWY8m05eSUUviRVNOc3GBKbkMNHHDeA=; b=ArroIK7d7YtG3ioRujAokD6TeomNV6GNYvZNupz8bq91tggonGOQk0j8jYonKlr4X3 hCeMescUdYDzFYDVPdlA+hDYi3Od9e1qyYpfjqPlSAJ2KB9nm15nkPIF77pxEo6tZYD0 wZOenPGvaL0pL93M9m5Hqnfq4YALErZZm9cR16MvSxao2hR6+Immf2RQ8OxrENaFSpdA 6Mpog9fu3Cc8JKJZFcP787QoEeZUk4T30t4LbPPUPkZj0crFQe1Bgr3+gRrd81jgDaog Y80zbJRbfa61MV2YQJRzr7Pm1U9zaeBRJb8DSGX/fbjEmUfB0vMrFtt7n1FEElUfpPtk phMg== X-Forwarded-Encrypted: i=1; AJvYcCXjteqfIss67ZXEwZ6n4dCdN3GqfhFi5YSmJHE9Fa1A9cKnFIcH5vYJwKbs4rYz/j4juCpgipArGQ==@kvack.org X-Gm-Message-State: AOJu0Ywes4cOd/1ydKK3C/duSMNmwCLMB4EYUZ926/2L6RfybhHq9zS3 Pr9HLf3WJvsOC7Okf5Bl/ZI6gxwMEx3rS1vUD9Odod+svbTGBPrdz8vF8s7fOfDzVZwWdDCfkCG 3 X-Gm-Gg: ASbGncvdKMkmcfmI+Qac8/TAoMx3sFhYPYHLnJDi9RumQYsCXwgTGnT7ZDpTSlL4lz2 O93mGpwCK3cMghsQB3IrZUkO/sZzEKR9z7LgN/GKZBa5gOIyQkO9z+gdkLU4lbKr6keioNNwVPw ifW+nrLdtO6Sm7v/BhkNn9jaEw09D9ZzMq+Hs7hJ1QfbNxelh7RKL6C3hOIHphRLbV27kQ2s77b s6mowm2KzkuDKFHFqhMOxge1aiGy/LG5jj/Tc1FQOKWC/5QQrtc4WIQy9IRSZuJV3oSWNFY02L3 j5+pcS9/ZsIE6BWlZAK8fCZIoY3c13/c5WmE9/o647sTtWWykd/Idx+WK+3NOgI83r7tQg== X-Google-Smtp-Source: AGHT+IFrlDybl2QKCVYqfxTyu64dXX62tEzPiiOD5kkpTcHrUIRBIEyVsyFQKXhsbfpJSLh/l0FWog== X-Received: by 2002:a05:600c:4e4b:b0:439:9737:675b with SMTP id 5b1f17b1804b1-43d502f8536mr33623225e9.7.1742562582938; Fri, 21 Mar 2025 06:09:42 -0700 (PDT) Received: from alex-rivos.ba.rivosinc.com ([2001:861:3382:ef90:3d12:52fe:c1cc:c94]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-3997f9a3c7fsm2377861f8f.34.2025.03.21.06.09.42 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 21 Mar 2025 06:09:42 -0700 (PDT) From: Alexandre Ghiti To: Catalin Marinas , Will Deacon , Ryan Roberts , Mark Rutland , Matthew Wilcox , Paul Walmsley , Palmer Dabbelt , Alexandre Ghiti , Andrew Morton , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org, linux-mm@kvack.org Cc: Alexandre Ghiti Subject: [PATCH v5 3/9] mm: Use common huge_ptep_get() function for riscv/arm64 Date: Fri, 21 Mar 2025 14:06:29 +0100 Message-Id: <20250321130635.227011-4-alexghiti@rivosinc.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20250321130635.227011-1-alexghiti@rivosinc.com> References: <20250321130635.227011-1-alexghiti@rivosinc.com> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam09 X-Rspamd-Queue-Id: 986BB12000A X-Stat-Signature: 5ahmkrjpbj5iig1gc5obr8dt9zdc368c X-HE-Tag: 1742562584-581775 X-HE-Meta: U2FsdGVkX1+4o+6tr8LCCiv3fhvq24kSo4KLBXtckX1kcYgXqKLheqv2KSMw8oIRN8RymwT/PIV0LyN8f07jZ8KzMD1gVcoqlWQYPaTmkHhJqfWK56V8hULKlr1SQlAPqHMhLdR1SuRZ3eDkHrr8+whKoOhG8s2shuNkSqEzTYAS9ioSjVYHUu1XD8VlVtSxVlVjRfk9efLN+p8sDphAczBp9B+UA0V8kWA5QxdkzBFBR0rurw7g7YtxX1p71u6Fzl/VyBkefH2XUswx9RXrv+XYFGtQ/cmyrHzcKX7SnK+HicpW9514T0Cd/79Nbgl2AaDpg2fGKA5SN5Kw8yn9Ot7BDmK1skdsskUEtSwZbc6Lxk/CJ80JKwxeKFVCAIK/pp6/HoenJx26B6r8PxnoDqdSdmQ5LEOqpPzuYpLY+7BtP/KE8YP0lql4EJF0LnKYhXGtbmBaMpzTKSMq0iGlGMjWkTEzOeDsTt2FZAGSzhbSkIujnpfgmZM6FpuJokKlkmYw+ZJs7Czzwjhs5NjFcQgocnH18NvSwzYW0HenZYJz/zG0NLLEM9vRjhFfWnYRZTX/cYSRyF8FUBxG9t8of7J9ubTqw8N5BPlGExQolBro8f+QouL4spwc+56dsDfgL0Z28XyvotwZ3j19FlXsONdKdLTeJCQo13RgPhK4flGP/6bfblL+cxHedcyBZ0SrQPBp8ctwK+8OW2gVxVxn2XE2J1oUfvSKaR3FbxlgZNaPqaB4x5KjX3GjGe7KntsQDNagndNWyVuSntQPvlqntFzvEaue48zFqMPsj+eUycE3NrOKthglsl+OXixJW1U1p3OnGYNmn1u/guIZADW5SmKKjmqJikP97VWYo+tWSXVWkB1/+ItP03hY/4r8i9bPdEaB0qOxXj1M44rtOpVuhqKBr55hycdD3pfWq2NvZIOGavJ+YyG+D4kal+rx0W8gbn3MD3h9qHxQb6dtifU sOD82v3r BbKrTRwEutwc4c62SNNlss/4VwMzaiSmpBfLYcXT3Ar+Xyi6ptANncJXHqyhHHXDXrm239I923K5/hTNTtuCYJn1tdFeghVBYzeCO6ShZ2xuAOgR6ntG4JhQj/D1L5mMHvTeKGjvg/D3zzOrxmh56P3mxMlEPOO7Fq19X+LZ+7VBGrpAc/1J9FNITX4RTXlckzlhCOkVEOjwGqH0n0rDtik4UQguwKuS3axsUiWVPvispzbbxHgI3X7cbWWUwczGBNqFh1Nc9K8APOlkJ9yRavxUvq7bekrHaqoXqB08Qn3fF4/BM1r3kYgdl07Obpp79526wXZ8kOJm0D7LobDQn+FQYo7PC6COosTxktm+rBmIe3Pib/2OSsOCYEk+YgJosUFyYOBHmGC/0gjNl3jg97yJbc6nZPlDiyd6AHhjoMMmcNsSee+upLSOPCBP1m2b3FZoD7f4848Gf9pmV1lS+cJhNew== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: After some adjustments, both architectures have the same implementation so move it to the generic code. Signed-off-by: Alexandre Ghiti --- arch/arm64/Kconfig | 1 + arch/arm64/include/asm/hugetlb.h | 3 +- arch/arm64/include/asm/pgtable.h | 57 +++++++++++++++++++++++++-- arch/arm64/mm/hugetlbpage.c | 66 ++------------------------------ arch/riscv/Kconfig | 1 + arch/riscv/include/asm/hugetlb.h | 6 +-- arch/riscv/include/asm/pgtable.h | 45 ++++++++++++++++++++++ arch/riscv/mm/hugetlbpage.c | 62 +++--------------------------- include/linux/hugetlb_contpte.h | 12 ++++++ mm/Kconfig | 3 ++ mm/Makefile | 1 + mm/hugetlb_contpte.c | 32 ++++++++++++++++ 12 files changed, 161 insertions(+), 128 deletions(-) create mode 100644 include/linux/hugetlb_contpte.h create mode 100644 mm/hugetlb_contpte.c diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig index 940343beb3d4..5a1e1bc73c15 100644 --- a/arch/arm64/Kconfig +++ b/arch/arm64/Kconfig @@ -113,6 +113,7 @@ config ARM64 select ARCH_WANT_DEFAULT_BPF_JIT select ARCH_WANT_DEFAULT_TOPDOWN_MMAP_LAYOUT select ARCH_WANT_FRAME_POINTERS + select ARCH_WANT_GENERAL_HUGETLB_CONTPTE select ARCH_WANT_HUGE_PMD_SHARE if ARM64_4K_PAGES || (ARM64_16K_PAGES && !ARM64_VA_BITS_36) select ARCH_WANT_LD_ORPHAN_WARN select ARCH_WANTS_EXECMEM_LATE diff --git a/arch/arm64/include/asm/hugetlb.h b/arch/arm64/include/asm/hugetlb.h index 07fbf5bf85a7..0604e01dca97 100644 --- a/arch/arm64/include/asm/hugetlb.h +++ b/arch/arm64/include/asm/hugetlb.h @@ -13,6 +13,7 @@ #include #include #include +#include #ifdef CONFIG_ARCH_ENABLE_HUGEPAGE_MIGRATION #define arch_hugetlb_migration_supported arch_hugetlb_migration_supported @@ -53,8 +54,6 @@ extern pte_t huge_ptep_clear_flush(struct vm_area_struct *vma, #define __HAVE_ARCH_HUGE_PTE_CLEAR extern void huge_pte_clear(struct mm_struct *mm, unsigned long addr, pte_t *ptep, unsigned long sz); -#define __HAVE_ARCH_HUGE_PTEP_GET -extern pte_t huge_ptep_get(struct mm_struct *mm, unsigned long addr, pte_t *ptep); void __init arm64_hugetlb_cma_reserve(void); diff --git a/arch/arm64/include/asm/pgtable.h b/arch/arm64/include/asm/pgtable.h index 0b2a2ad1b9e8..af8156929c1d 100644 --- a/arch/arm64/include/asm/pgtable.h +++ b/arch/arm64/include/asm/pgtable.h @@ -420,9 +420,10 @@ static inline pte_t pte_advance_pfn(pte_t pte, unsigned long nr) return pfn_pte(pte_pfn(pte) + nr, pte_pgprot(pte)); } -static inline void __set_ptes(struct mm_struct *mm, - unsigned long __always_unused addr, - pte_t *ptep, pte_t pte, unsigned int nr) +static inline void ___set_ptes(struct mm_struct *mm, + unsigned long __always_unused addr, + pte_t *ptep, pte_t pte, unsigned int nr, + size_t pgsize) { page_table_check_ptes_set(mm, ptep, pte, nr); __sync_cache_and_tags(pte, nr); @@ -433,10 +434,15 @@ static inline void __set_ptes(struct mm_struct *mm, if (--nr == 0) break; ptep++; - pte = pte_advance_pfn(pte, 1); + pte = pte_advance_pfn(pte, pgsize >> PAGE_SHIFT); } } +#define __set_ptes(mm, addr, ptep, pte, nr) \ + ___set_ptes(mm, addr, ptep, pte, nr, PAGE_SIZE) + +#define set_contptes ___set_ptes + /* * Hugetlb definitions. */ @@ -1825,6 +1831,49 @@ static inline void clear_young_dirty_ptes(struct vm_area_struct *vma, #endif /* CONFIG_ARM64_CONTPTE */ +static inline bool __hugetlb_valid_size(unsigned long size) +{ + switch (size) { +#ifndef __PAGETABLE_PMD_FOLDED + case PUD_SIZE: + return pud_sect_supported(); +#endif + case CONT_PMD_SIZE: + case PMD_SIZE: + case CONT_PTE_SIZE: + return true; + } + + return false; +} + +static inline int arch_contpte_get_num_contig(pte_t *ptep, + unsigned long size, + size_t *pgsize) +{ + int contig_ptes = 1; + + if (pgsize) + *pgsize = size; + + switch (size) { + case CONT_PMD_SIZE: + if (pgsize) + *pgsize = PMD_SIZE; + contig_ptes = CONT_PMDS; + break; + case CONT_PTE_SIZE: + if (pgsize) + *pgsize = PAGE_SIZE; + contig_ptes = CONT_PTES; + break; + default: + WARN_ON(!__hugetlb_valid_size(size)); + } + + return contig_ptes; +} + #endif /* !__ASSEMBLY__ */ #endif /* __ASM_PGTABLE_H */ diff --git a/arch/arm64/mm/hugetlbpage.c b/arch/arm64/mm/hugetlbpage.c index b3a7fafe8892..60a2bb7575c1 100644 --- a/arch/arm64/mm/hugetlbpage.c +++ b/arch/arm64/mm/hugetlbpage.c @@ -49,22 +49,6 @@ void __init arm64_hugetlb_cma_reserve(void) } #endif /* CONFIG_CMA */ -static bool __hugetlb_valid_size(unsigned long size) -{ - switch (size) { -#ifndef __PAGETABLE_PMD_FOLDED - case PUD_SIZE: - return pud_sect_supported(); -#endif - case CONT_PMD_SIZE: - case PMD_SIZE: - case CONT_PTE_SIZE: - return true; - } - - return false; -} - #ifdef CONFIG_ARCH_ENABLE_HUGEPAGE_MIGRATION bool arch_hugetlb_migration_supported(struct hstate *h) { @@ -98,50 +82,6 @@ static int find_num_contig(struct mm_struct *mm, unsigned long addr, return CONT_PTES; } -static inline int num_contig_ptes(unsigned long size, size_t *pgsize) -{ - int contig_ptes = 1; - - *pgsize = size; - - switch (size) { - case CONT_PMD_SIZE: - *pgsize = PMD_SIZE; - contig_ptes = CONT_PMDS; - break; - case CONT_PTE_SIZE: - *pgsize = PAGE_SIZE; - contig_ptes = CONT_PTES; - break; - default: - WARN_ON(!__hugetlb_valid_size(size)); - } - - return contig_ptes; -} - -pte_t huge_ptep_get(struct mm_struct *mm, unsigned long addr, pte_t *ptep) -{ - int ncontig, i; - size_t pgsize; - pte_t orig_pte = __ptep_get(ptep); - - if (!pte_present(orig_pte) || !pte_cont(orig_pte)) - return orig_pte; - - ncontig = num_contig_ptes(page_size(pte_page(orig_pte)), &pgsize); - for (i = 0; i < ncontig; i++, ptep++) { - pte_t pte = __ptep_get(ptep); - - if (pte_dirty(pte)) - orig_pte = pte_mkdirty(orig_pte); - - if (pte_young(pte)) - orig_pte = pte_mkyoung(orig_pte); - } - return orig_pte; -} - /* * Changing some bits of contiguous entries requires us to follow a * Break-Before-Make approach, breaking the whole contiguous set @@ -221,7 +161,7 @@ void set_huge_pte_at(struct mm_struct *mm, unsigned long addr, unsigned long pfn, dpfn; pgprot_t hugeprot; - ncontig = num_contig_ptes(sz, &pgsize); + ncontig = arch_contpte_get_num_contig(ptep, sz, &pgsize); if (!pte_present(pte)) { for (i = 0; i < ncontig; i++, ptep++, addr += pgsize) @@ -382,7 +322,7 @@ void huge_pte_clear(struct mm_struct *mm, unsigned long addr, int i, ncontig; size_t pgsize; - ncontig = num_contig_ptes(sz, &pgsize); + ncontig = arch_contpte_get_num_contig(ptep, sz, &pgsize); for (i = 0; i < ncontig; i++, addr += pgsize, ptep++) __pte_clear(mm, addr, ptep); @@ -394,7 +334,7 @@ pte_t huge_ptep_get_and_clear(struct mm_struct *mm, unsigned long addr, int ncontig; size_t pgsize; - ncontig = num_contig_ptes(sz, &pgsize); + ncontig = arch_contpte_get_num_contig(ptep, sz, &pgsize); return get_clear_contig(mm, addr, ptep, pgsize, ncontig); } diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig index 7612c52e9b1e..2a5b2a9f2816 100644 --- a/arch/riscv/Kconfig +++ b/arch/riscv/Kconfig @@ -79,6 +79,7 @@ config RISCV select ARCH_WANT_DEFAULT_TOPDOWN_MMAP_LAYOUT if MMU select ARCH_WANT_FRAME_POINTERS select ARCH_WANT_GENERAL_HUGETLB if !RISCV_ISA_SVNAPOT + select ARCH_WANT_GENERAL_HUGETLB_CONTPTE if RISCV_ISA_SVNAPOT select ARCH_WANT_HUGE_PMD_SHARE if 64BIT select ARCH_WANT_LD_ORPHAN_WARN if !XIP_KERNEL select ARCH_WANT_OPTIMIZE_DAX_VMEMMAP diff --git a/arch/riscv/include/asm/hugetlb.h b/arch/riscv/include/asm/hugetlb.h index 446126497768..69393346ade0 100644 --- a/arch/riscv/include/asm/hugetlb.h +++ b/arch/riscv/include/asm/hugetlb.h @@ -4,6 +4,9 @@ #include #include +#ifdef CONFIG_ARCH_WANT_GENERAL_HUGETLB_CONTPTE +#include +#endif static inline void arch_clear_hugetlb_flags(struct folio *folio) { @@ -44,9 +47,6 @@ int huge_ptep_set_access_flags(struct vm_area_struct *vma, unsigned long addr, pte_t *ptep, pte_t pte, int dirty); -#define __HAVE_ARCH_HUGE_PTEP_GET -pte_t huge_ptep_get(struct mm_struct *mm, unsigned long addr, pte_t *ptep); - pte_t arch_make_huge_pte(pte_t entry, unsigned int shift, vm_flags_t flags); #define arch_make_huge_pte arch_make_huge_pte diff --git a/arch/riscv/include/asm/pgtable.h b/arch/riscv/include/asm/pgtable.h index 2e62d7e607db..286fe1a32ded 100644 --- a/arch/riscv/include/asm/pgtable.h +++ b/arch/riscv/include/asm/pgtable.h @@ -296,6 +296,8 @@ static inline unsigned long pte_napot(pte_t pte) return pte_val(pte) & _PAGE_NAPOT; } +#define pte_cont pte_napot + #define pte_valid_napot(pte) (pte_present(pte) && pte_napot(pte)) /* @@ -606,6 +608,49 @@ static inline void ___set_ptes(struct mm_struct *mm, unsigned long addr, } } +#ifdef CONFIG_RISCV_ISA_SVNAPOT +/* + * Some hugetlb functions can be called on !present ptes, so we must use the + * size parameter when it is passed. + */ +static inline int arch_contpte_get_num_contig(pte_t *ptep, unsigned long size, + size_t *pgsize) +{ + unsigned long hugepage_shift; + pte_t __pte; + + if (size) { + if (size >= PGDIR_SIZE) + hugepage_shift = PGDIR_SHIFT; + else if (size >= P4D_SIZE) + hugepage_shift = P4D_SHIFT; + else if (size >= PUD_SIZE) + hugepage_shift = PUD_SHIFT; + else if (size >= PMD_SIZE) + hugepage_shift = PMD_SHIFT; + else + hugepage_shift = PAGE_SHIFT; + } else { + /* + * We must read the raw value of the pte to get the size of + * the mapping + */ + __pte = ___ptep_get(ptep); + + /* Make sure __pte is not a swap entry */ + BUG_ON(!pte_valid_napot(__pte)); + + hugepage_shift = PAGE_SHIFT; + size = napot_cont_size(napot_cont_order(__pte)); + } + + if (pgsize) + *pgsize = BIT(hugepage_shift); + + return size >> hugepage_shift; +} +#endif + static inline void pte_clear(struct mm_struct *mm, unsigned long addr, pte_t *ptep) { diff --git a/arch/riscv/mm/hugetlbpage.c b/arch/riscv/mm/hugetlbpage.c index 60b7e738b31a..b9eb6b7b214d 100644 --- a/arch/riscv/mm/hugetlbpage.c +++ b/arch/riscv/mm/hugetlbpage.c @@ -3,30 +3,6 @@ #include #ifdef CONFIG_RISCV_ISA_SVNAPOT -pte_t huge_ptep_get(struct mm_struct *mm, unsigned long addr, pte_t *ptep) -{ - unsigned long pte_num; - int i; - pte_t orig_pte = ptep_get(ptep); - - if (!pte_present(orig_pte) || !pte_napot(orig_pte)) - return orig_pte; - - pte_num = napot_pte_num(napot_cont_order(orig_pte)); - - for (i = 0; i < pte_num; i++, ptep++) { - pte_t pte = ptep_get(ptep); - - if (pte_dirty(pte)) - orig_pte = pte_mkdirty(orig_pte); - - if (pte_young(pte)) - orig_pte = pte_mkyoung(orig_pte); - } - - return orig_pte; -} - pte_t *huge_pte_alloc(struct mm_struct *mm, struct vm_area_struct *vma, unsigned long addr, @@ -215,26 +191,6 @@ static void clear_flush(struct mm_struct *mm, flush_tlb_range(&vma, saddr, addr); } -static int num_contig_ptes_from_size(unsigned long sz, size_t *pgsize) -{ - unsigned long hugepage_shift; - - if (sz >= PGDIR_SIZE) - hugepage_shift = PGDIR_SHIFT; - else if (sz >= P4D_SIZE) - hugepage_shift = P4D_SHIFT; - else if (sz >= PUD_SIZE) - hugepage_shift = PUD_SHIFT; - else if (sz >= PMD_SIZE) - hugepage_shift = PMD_SHIFT; - else - hugepage_shift = PAGE_SHIFT; - - *pgsize = 1 << hugepage_shift; - - return sz >> hugepage_shift; -} - /* * When dealing with NAPOT mappings, the privileged specification indicates that * "if an update needs to be made, the OS generally should first mark all of the @@ -252,7 +208,7 @@ void set_huge_pte_at(struct mm_struct *mm, size_t pgsize; int i, pte_num; - pte_num = num_contig_ptes_from_size(sz, &pgsize); + pte_num = arch_contpte_get_num_contig(ptep, sz, &pgsize); if (!pte_present(pte)) { for (i = 0; i < pte_num; i++, ptep++, addr += pgsize) @@ -277,15 +233,13 @@ int huge_ptep_set_access_flags(struct vm_area_struct *vma, int dirty) { struct mm_struct *mm = vma->vm_mm; - unsigned long order; pte_t orig_pte; int pte_num; if (!pte_napot(pte)) return ptep_set_access_flags(vma, addr, ptep, pte, dirty); - order = napot_cont_order(pte); - pte_num = napot_pte_num(order); + pte_num = arch_contpte_get_num_contig(ptep, 0, NULL); orig_pte = get_clear_contig_flush(mm, addr, ptep, pte_num); if (pte_dirty(orig_pte)) @@ -303,14 +257,13 @@ pte_t huge_ptep_get_and_clear(struct mm_struct *mm, unsigned long addr, pte_t *ptep, unsigned long sz) { - size_t pgsize; pte_t orig_pte = ptep_get(ptep); int pte_num; if (!pte_napot(orig_pte)) return ptep_get_and_clear(mm, addr, ptep); - pte_num = num_contig_ptes_from_size(sz, &pgsize); + pte_num = arch_contpte_get_num_contig(ptep, sz, NULL); return get_clear_contig(mm, addr, ptep, pte_num); } @@ -320,7 +273,6 @@ void huge_ptep_set_wrprotect(struct mm_struct *mm, pte_t *ptep) { pte_t pte = ptep_get(ptep); - unsigned long order; pte_t orig_pte; int pte_num; @@ -329,8 +281,7 @@ void huge_ptep_set_wrprotect(struct mm_struct *mm, return; } - order = napot_cont_order(pte); - pte_num = napot_pte_num(order); + pte_num = arch_contpte_get_num_contig(ptep, 0, NULL); orig_pte = get_clear_contig_flush(mm, addr, ptep, pte_num); orig_pte = pte_wrprotect(orig_pte); @@ -348,7 +299,7 @@ pte_t huge_ptep_clear_flush(struct vm_area_struct *vma, if (!pte_napot(pte)) return ptep_clear_flush(vma, addr, ptep); - pte_num = napot_pte_num(napot_cont_order(pte)); + pte_num = arch_contpte_get_num_contig(ptep, 0, NULL); return get_clear_contig_flush(vma->vm_mm, addr, ptep, pte_num); } @@ -367,8 +318,7 @@ void huge_pte_clear(struct mm_struct *mm, return; } - pte_num = num_contig_ptes_from_size(sz, &pgsize); - + pte_num = arch_contpte_get_num_contig(ptep, sz, &pgsize); for (i = 0; i < pte_num; i++, addr += pgsize, ptep++) pte_clear(mm, addr, ptep); } diff --git a/include/linux/hugetlb_contpte.h b/include/linux/hugetlb_contpte.h new file mode 100644 index 000000000000..2ea17e4fe36b --- /dev/null +++ b/include/linux/hugetlb_contpte.h @@ -0,0 +1,12 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (C) 2025 Rivos Inc. + */ + +#ifndef _LINUX_HUGETLB_CONTPTE_H +#define _LINUX_HUGETLB_CONTPTE_H + +#define __HAVE_ARCH_HUGE_PTEP_GET +extern pte_t huge_ptep_get(struct mm_struct *mm, unsigned long addr, pte_t *ptep); + +#endif /* _LINUX_HUGETLB_CONTPTE_H */ diff --git a/mm/Kconfig b/mm/Kconfig index 1b501db06417..f9d3f3d49f3e 100644 --- a/mm/Kconfig +++ b/mm/Kconfig @@ -853,6 +853,9 @@ config NOMMU_INITIAL_TRIM_EXCESS config ARCH_WANT_GENERAL_HUGETLB bool +config ARCH_WANT_GENERAL_HUGETLB_CONTPTE + bool + config ARCH_WANTS_THP_SWAP def_bool n diff --git a/mm/Makefile b/mm/Makefile index 850386a67b3e..76e8b995f551 100644 --- a/mm/Makefile +++ b/mm/Makefile @@ -96,6 +96,7 @@ obj-$(CONFIG_MIGRATION) += migrate.o obj-$(CONFIG_NUMA) += memory-tiers.o obj-$(CONFIG_DEVICE_MIGRATION) += migrate_device.o obj-$(CONFIG_TRANSPARENT_HUGEPAGE) += huge_memory.o khugepaged.o +obj-$(CONFIG_ARCH_WANT_GENERAL_HUGETLB_CONTPTE) += hugetlb_contpte.o obj-$(CONFIG_PAGE_COUNTER) += page_counter.o obj-$(CONFIG_MEMCG_V1) += memcontrol-v1.o obj-$(CONFIG_MEMCG) += memcontrol.o vmpressure.o diff --git a/mm/hugetlb_contpte.c b/mm/hugetlb_contpte.c new file mode 100644 index 000000000000..500d0b96a680 --- /dev/null +++ b/mm/hugetlb_contpte.c @@ -0,0 +1,32 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright 2025 Rivos Inc. + */ + +#include +#include +#include + +pte_t huge_ptep_get(struct mm_struct *mm, unsigned long addr, pte_t *ptep) +{ + int ncontig, i; + pte_t orig_pte = __ptep_get(ptep); + + if (!pte_present(orig_pte) || !pte_cont(orig_pte)) + return orig_pte; + + ncontig = arch_contpte_get_num_contig(ptep, + page_size(pte_page(orig_pte)), + NULL); + + for (i = 0; i < ncontig; i++, ptep++) { + pte_t pte = __ptep_get(ptep); + + if (pte_dirty(pte)) + orig_pte = pte_mkdirty(orig_pte); + + if (pte_young(pte)) + orig_pte = pte_mkyoung(orig_pte); + } + return orig_pte; +} From patchwork Fri Mar 21 13:06:30 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexandre Ghiti X-Patchwork-Id: 14025409 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6AE92C36000 for ; Fri, 21 Mar 2025 13:10:49 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 644E6280005; Fri, 21 Mar 2025 09:10:47 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 5F29C280001; Fri, 21 Mar 2025 09:10:47 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 495AA280005; Fri, 21 Mar 2025 09:10:47 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 2CCC6280001 for ; Fri, 21 Mar 2025 09:10:47 -0400 (EDT) Received: from smtpin06.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 72A76C1DFF for ; Fri, 21 Mar 2025 13:10:48 +0000 (UTC) X-FDA: 83245592976.06.AD6E3FD Received: from mail-wm1-f42.google.com (mail-wm1-f42.google.com [209.85.128.42]) by imf03.hostedemail.com (Postfix) with ESMTP id 607AF20022 for ; Fri, 21 Mar 2025 13:10:46 +0000 (UTC) Authentication-Results: imf03.hostedemail.com; dkim=pass header.d=rivosinc-com.20230601.gappssmtp.com header.s=20230601 header.b=bgndQfWj; dmarc=none; spf=pass (imf03.hostedemail.com: domain of alexghiti@rivosinc.com designates 209.85.128.42 as permitted sender) smtp.mailfrom=alexghiti@rivosinc.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1742562646; a=rsa-sha256; cv=none; b=enkfWSFh4Mwzbteo/6tKzzbAkZNU6tOLNEk6wIgAqB8MrDIeqeWgAq19R7uOjr/QQY8vSz lJ63Hq06kySoTnjrs/Y6fizqQi6hgsZMK9F8yj/S4nseUgI2k14hObZPtxs4m4H2nSBJGM FszZPfQO1AGThvtl6Ih5d0n8jbhd6Qk= ARC-Authentication-Results: i=1; imf03.hostedemail.com; dkim=pass header.d=rivosinc-com.20230601.gappssmtp.com header.s=20230601 header.b=bgndQfWj; dmarc=none; spf=pass (imf03.hostedemail.com: domain of alexghiti@rivosinc.com designates 209.85.128.42 as permitted sender) smtp.mailfrom=alexghiti@rivosinc.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1742562646; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=eAv+4atVMvRknGVbTPj58gq6ySa46Mekau0XoGNQASg=; b=x1gjs3xnvoutlM9uRS9pqEhzZQ9mI8NI2kc1rBASetfBcRrywLVghDPKbZYlcOsXLiwi0O XmFhrgQTJtzmAJ5z9WPYSUJuPKPWOVDzs/egZfAuCzGNrRSVJxgG47vRDjzO6QG3x6u4Pi oE5jHGEAIt+3ShfLhE54tKF4bZWYC+k= Received: by mail-wm1-f42.google.com with SMTP id 5b1f17b1804b1-43690d4605dso13286525e9.0 for ; Fri, 21 Mar 2025 06:10:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc-com.20230601.gappssmtp.com; s=20230601; t=1742562645; x=1743167445; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=eAv+4atVMvRknGVbTPj58gq6ySa46Mekau0XoGNQASg=; b=bgndQfWjcuLVd9r1P3Q+PVQlUTvZLkbIvECHYknmztWNDtWAi472Ke9OfvmjiYO/vw cC4OTxG0/tqlrgsZDh8jTFppSN4b0fA2VspZRdJnATg7zcR48v6Vfz41K7G26HW8lGZL oa6svHOOFHekc6Seyf8yiwhqOqoJS1sw9FqpZe9KBeyn5P/qZaWqGGitLs12qXuAP+VM jyzHqSqXLxGurg/nB+/sFxTANH0pLh5+zKhAqjkVIXg+bJFKXkQo5zpFNCIXdrYBlTfs iZ7fZjMc1mCQL/qfQIoJsMAld8J09XQanBieHP2hqyFro3Gw4MlrD5JiRUxxFbDarKmp NkAA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1742562645; x=1743167445; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=eAv+4atVMvRknGVbTPj58gq6ySa46Mekau0XoGNQASg=; b=E5tcN/j8YSUJ1oA+2jGLOAIpo3D77qLfaWR6MH8jnhxyovlMhTvUR/87nx0b6Ybbdt xlqDsmWCR217l5edOL5a9j0qDkNqXUzYK6Q/Tgwr2nZp8ZBnyZ8nzT9Cpias2KhKRkL9 djhlAlNPzdtDJ9Eh//3eEYJK8M2Y5pCaumACoTNDExlhtBzWN7rpKAq78rWuEqkAA8sB dhO4BJcH7lz+B8YL4sLKu02djbv1I59s6RczQNd24HFAeg2Xzuou3U8ekhAfKk7EZ2xx kdbmgJEorI0JIFLqph1ihIamTE3uCHDgPmrrX5O+cG/d1uQLTwMT9DxEzxcQHNh8gfYL rNgw== X-Forwarded-Encrypted: i=1; AJvYcCUHPwCFSGStvehePTZvf7VaJHDWOoec9VLVgfwy989Ub+kGzlce1/Z0UjzMACqGBXjV+qPezy49Ew==@kvack.org X-Gm-Message-State: AOJu0YzwvhWiz2dFbMw5dG7J2TUBTOLDPcDLOeHl70mRsSmrev85/3DZ QL0RDbam3SC1Ju4MMmK17tWT9i+dMqzhgnKKZ0ifzNV+rjCmk4fagpRTNcFkLwM= X-Gm-Gg: ASbGnctwFP9ZoaQAHifN8ObATsVi8rCU6a+CqZ0hFNE1fQbhjog93dtIqhOlPUI0YIW EIrE1GT2yV+vWo4KyCfdrnHuk98Ym6kUnKFBTcMyQhmndgBhOJX5aidr3dUwynAUTGmPO1d3VR5 3OErj19SDeWnHq/G1AVxu77TiPXCr3d7SPkltzBno3lrsKjGvWriB9kIghB5ugnX5VE3/cgrVTs GQcunsIF4JTRz4xlg+98TBmjMQrUn/koKnyc+HbzY+ygxwSSWiOwnubpIMc74HebP3OI104ISTH cYLpDAn41Y2+PXYfQyAKVIlQeqUJZiQEl44JQHVyus1NJ/1fSE4v6D4TVyUFj7c9mptoLg== X-Google-Smtp-Source: AGHT+IFFzxLTl7dkoWPQa6CIZER6lpnHzrVNqhIgyBgY3Y3nEtgERu0W1NS2+ffacirYXfcMWjF6NQ== X-Received: by 2002:a05:600c:83cf:b0:439:91dd:cf9c with SMTP id 5b1f17b1804b1-43d509ec70cmr37202275e9.10.1742562644586; Fri, 21 Mar 2025 06:10:44 -0700 (PDT) Received: from alex-rivos.ba.rivosinc.com ([2001:861:3382:ef90:3d12:52fe:c1cc:c94]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-43d42b8fcd4sm47982695e9.1.2025.03.21.06.10.43 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 21 Mar 2025 06:10:44 -0700 (PDT) From: Alexandre Ghiti To: Catalin Marinas , Will Deacon , Ryan Roberts , Mark Rutland , Matthew Wilcox , Paul Walmsley , Palmer Dabbelt , Alexandre Ghiti , Andrew Morton , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org, linux-mm@kvack.org Cc: Alexandre Ghiti Subject: [PATCH v5 4/9] mm: Use common set_huge_pte_at() function for riscv/arm64 Date: Fri, 21 Mar 2025 14:06:30 +0100 Message-Id: <20250321130635.227011-5-alexghiti@rivosinc.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20250321130635.227011-1-alexghiti@rivosinc.com> References: <20250321130635.227011-1-alexghiti@rivosinc.com> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam03 X-Rspamd-Queue-Id: 607AF20022 X-Stat-Signature: t7q1dyp4jrinmpnsk3rqi1i5cxxc9ynw X-HE-Tag: 1742562646-355016 X-HE-Meta: U2FsdGVkX18dFX4SnjWjKjUB+bqNga2e2sSEAEalttju/gQ2KCUiBPMhecbdADWyzee8zcOuTeGUdU/bfwXOC19Pw8L85fSuUj2sKpF/eZNZ7jf7eaE+afCDeXCbeVPTBCMAL6LNIPVd9AoeqyZWecQ2Hc0HKQZ5om8uKvh3MqtbFpqPdOxeEBCZh+eaPxVlCDvYJ3++TF2EOifvDk3TZ5N7e9OgD9ZRChweY31w2lX9X8yQCgMNK0WV6zmG4GL7LODMqQ1of2IlLbkiwNNKv/xCeg4g8J/VMBgEBFJDrc9nivy/+FFbQS6ZcvzP4OnlqLfCvVX8ibGC5vhSoBZ3L9Uyf/p1KcrM9d1CRUXB3eq7z9q6sSxXXW1KbhB3mSH2XbP5vdKak0EipTxVipGEIFfK2hGnEV0VLskbrCCreW3+OY+8oNmRWRTKgF9rwX+EGczlrdfixMfwD8iL5OAtQs3/ryXBuSeAPsWelbg7Ee0s8kq+uHNarsGtbWJQ/aYxm/0EC/mzdKMXh2cRt/haE6ytonV55B57U7bLE8twPiskxZs95HNrKmCRwMBL54Ugb0TUmA75WwIdycWZDghB4wA1klhMiDG9+VY+yMstF8SEutoICaIoZ2B92d3FJENxnZDXqK+DmZIi2bqly0B9A29jGa2+zcqG1Vezn4Bk6p+htcmCyDuIShvQpIWFLbBD9vEPTUetUgjSB7pG/3Fmb2MqFhQJyPGv128fK213iiunkIV9nYmseR5eaM/agMFuRMWsrBbxfVpuHnNcY9gptrzy6DxQ4WGlk0H7SICNap9nWVSgKlQMANfs3kCeOzCWW4crE4Sqzxaz8E5SL4e/+ZgYC4/VPbbsRm1r+3Sah813ZxrNPzYGVtNf8JyD35nzwAByJZBlaGf0ss9lXUTOHE7sVbnRdGyOWb/WgzrQUavStUYyRaE/QX3CtyMyrvmGNglKRfffP+X9i6hN3jA QLhIf5gR tWWYJPCuGaC2ROj0HJam/af5ag2r7n8RmdReOo4u2VQ9na2tiB2mq6fsH9cq6uNsslBnbkwKvI5PpGrTEcTby1IqJKW0mYH1QV+Ej14Q3W6nzFlu6KPbL14Gj6QagNzO28/GNKXSX8Dgen1BVKgtDkesFCzyYr7HmKOdJLMqkh5JhEJSo4XExZ5mgzoo8I+ZtSvdFwB0zny6WFCMwxugTwCYSuKMFuqSIAz6Ck8TOSHPRyOi6IFdF84QFbfSRJtcG0u6xVRA213yhyjfPlI73r3ugcLIuPguZChxMsgBIiWyC1kYQHaG7xcpTIrYUzxwa5sZqfVnex26un+Gp2rBmMzGPK6TGbnILRk2owCMD7gYaJyK8EAtgR5bIyB96X8Y6pTWn9qQWt0oSiUk/ShLVnPLNv8UdVIq6PN7XkVVnTI4WS6UVG4DhpOUGO91tG0wiKCfKUiyhETWroO8YZyCdzWjL3g== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: After some adjustments, both architectures have the same implementation so move it to the generic code. Signed-off-by: Alexandre Ghiti --- arch/arm64/include/asm/hugetlb.h | 3 -- arch/arm64/mm/hugetlbpage.c | 56 -------------------------------- arch/riscv/include/asm/hugetlb.h | 5 --- arch/riscv/include/asm/pgtable.h | 13 ++++---- arch/riscv/mm/hugetlbpage.c | 50 ---------------------------- include/linux/hugetlb_contpte.h | 5 +++ mm/hugetlb_contpte.c | 56 ++++++++++++++++++++++++++++++++ 7 files changed, 68 insertions(+), 120 deletions(-) diff --git a/arch/arm64/include/asm/hugetlb.h b/arch/arm64/include/asm/hugetlb.h index 0604e01dca97..cfdc04e11585 100644 --- a/arch/arm64/include/asm/hugetlb.h +++ b/arch/arm64/include/asm/hugetlb.h @@ -35,9 +35,6 @@ static inline void arch_clear_hugetlb_flags(struct folio *folio) pte_t arch_make_huge_pte(pte_t entry, unsigned int shift, vm_flags_t flags); #define arch_make_huge_pte arch_make_huge_pte -#define __HAVE_ARCH_HUGE_SET_HUGE_PTE_AT -extern void set_huge_pte_at(struct mm_struct *mm, unsigned long addr, - pte_t *ptep, pte_t pte, unsigned long sz); #define __HAVE_ARCH_HUGE_PTEP_SET_ACCESS_FLAGS extern int huge_ptep_set_access_flags(struct vm_area_struct *vma, unsigned long addr, pte_t *ptep, diff --git a/arch/arm64/mm/hugetlbpage.c b/arch/arm64/mm/hugetlbpage.c index 60a2bb7575c1..6feb90ed2e7d 100644 --- a/arch/arm64/mm/hugetlbpage.c +++ b/arch/arm64/mm/hugetlbpage.c @@ -128,62 +128,6 @@ static pte_t get_clear_contig_flush(struct mm_struct *mm, return orig_pte; } -/* - * Changing some bits of contiguous entries requires us to follow a - * Break-Before-Make approach, breaking the whole contiguous set - * before we can change any entries. See ARM DDI 0487A.k_iss10775, - * "Misprogramming of the Contiguous bit", page D4-1762. - * - * This helper performs the break step for use cases where the - * original pte is not needed. - */ -static void clear_flush(struct mm_struct *mm, - unsigned long addr, - pte_t *ptep, - unsigned long pgsize, - unsigned long ncontig) -{ - struct vm_area_struct vma = TLB_FLUSH_VMA(mm, 0); - unsigned long i, saddr = addr; - - for (i = 0; i < ncontig; i++, addr += pgsize, ptep++) - __ptep_get_and_clear(mm, addr, ptep); - - flush_tlb_range(&vma, saddr, addr); -} - -void set_huge_pte_at(struct mm_struct *mm, unsigned long addr, - pte_t *ptep, pte_t pte, unsigned long sz) -{ - size_t pgsize; - int i; - int ncontig; - unsigned long pfn, dpfn; - pgprot_t hugeprot; - - ncontig = arch_contpte_get_num_contig(ptep, sz, &pgsize); - - if (!pte_present(pte)) { - for (i = 0; i < ncontig; i++, ptep++, addr += pgsize) - __set_ptes(mm, addr, ptep, pte, 1); - return; - } - - if (!pte_cont(pte)) { - __set_ptes(mm, addr, ptep, pte, 1); - return; - } - - pfn = pte_pfn(pte); - dpfn = pgsize >> PAGE_SHIFT; - hugeprot = pte_pgprot(pte); - - clear_flush(mm, addr, ptep, pgsize, ncontig); - - for (i = 0; i < ncontig; i++, ptep++, addr += pgsize, pfn += dpfn) - __set_ptes(mm, addr, ptep, pfn_pte(pfn, hugeprot), 1); -} - pte_t *huge_pte_alloc(struct mm_struct *mm, struct vm_area_struct *vma, unsigned long addr, unsigned long sz) { diff --git a/arch/riscv/include/asm/hugetlb.h b/arch/riscv/include/asm/hugetlb.h index 69393346ade0..7049a17b819d 100644 --- a/arch/riscv/include/asm/hugetlb.h +++ b/arch/riscv/include/asm/hugetlb.h @@ -24,11 +24,6 @@ bool arch_hugetlb_migration_supported(struct hstate *h); void huge_pte_clear(struct mm_struct *mm, unsigned long addr, pte_t *ptep, unsigned long sz); -#define __HAVE_ARCH_HUGE_SET_HUGE_PTE_AT -void set_huge_pte_at(struct mm_struct *mm, - unsigned long addr, pte_t *ptep, pte_t pte, - unsigned long sz); - #define __HAVE_ARCH_HUGE_PTEP_GET_AND_CLEAR pte_t huge_ptep_get_and_clear(struct mm_struct *mm, unsigned long addr, pte_t *ptep, diff --git a/arch/riscv/include/asm/pgtable.h b/arch/riscv/include/asm/pgtable.h index 286fe1a32ded..5b34b3c9c0f9 100644 --- a/arch/riscv/include/asm/pgtable.h +++ b/arch/riscv/include/asm/pgtable.h @@ -715,9 +715,8 @@ static inline pte_t pte_napot_clear_pfn(pte_t *ptep, pte_t pte) return pte; } -#define __HAVE_ARCH_PTEP_GET_AND_CLEAR -static inline pte_t ptep_get_and_clear(struct mm_struct *mm, - unsigned long address, pte_t *ptep) +static inline pte_t __ptep_get_and_clear(struct mm_struct *mm, + unsigned long address, pte_t *ptep) { pte_t pte = __pte(atomic_long_xchg((atomic_long_t *)ptep, 0)); @@ -775,9 +774,8 @@ static inline void __set_ptes(struct mm_struct *mm, unsigned long addr, #define set_ptes ___set_ptes #define __set_ptes ___set_ptes -#define __HAVE_ARCH_PTEP_GET_AND_CLEAR -static inline pte_t ptep_get_and_clear(struct mm_struct *mm, - unsigned long address, pte_t *ptep) +static inline pte_t __ptep_get_and_clear(struct mm_struct *mm, + unsigned long address, pte_t *ptep) { pte_t pte = __pte(atomic_long_xchg((atomic_long_t *)ptep, 0)); @@ -787,6 +785,9 @@ static inline pte_t ptep_get_and_clear(struct mm_struct *mm, } #endif /* CONFIG_RISCV_ISA_SVNAPOT */ +#define __HAVE_ARCH_PTEP_GET_AND_CLEAR +#define ptep_get_and_clear __ptep_get_and_clear + #define pgprot_nx pgprot_nx static inline pgprot_t pgprot_nx(pgprot_t _prot) { diff --git a/arch/riscv/mm/hugetlbpage.c b/arch/riscv/mm/hugetlbpage.c index b9eb6b7b214d..75faeacc8138 100644 --- a/arch/riscv/mm/hugetlbpage.c +++ b/arch/riscv/mm/hugetlbpage.c @@ -176,56 +176,6 @@ pte_t arch_make_huge_pte(pte_t entry, unsigned int shift, vm_flags_t flags) return entry; } -static void clear_flush(struct mm_struct *mm, - unsigned long addr, - pte_t *ptep, - unsigned long pgsize, - unsigned long ncontig) -{ - struct vm_area_struct vma = TLB_FLUSH_VMA(mm, 0); - unsigned long i, saddr = addr; - - for (i = 0; i < ncontig; i++, addr += pgsize, ptep++) - ptep_get_and_clear(mm, addr, ptep); - - flush_tlb_range(&vma, saddr, addr); -} - -/* - * When dealing with NAPOT mappings, the privileged specification indicates that - * "if an update needs to be made, the OS generally should first mark all of the - * PTEs invalid, then issue SFENCE.VMA instruction(s) covering all 4 KiB regions - * within the range, [...] then update the PTE(s), as described in Section - * 4.2.1.". That's the equivalent of the Break-Before-Make approach used by - * arm64. - */ -void set_huge_pte_at(struct mm_struct *mm, - unsigned long addr, - pte_t *ptep, - pte_t pte, - unsigned long sz) -{ - size_t pgsize; - int i, pte_num; - - pte_num = arch_contpte_get_num_contig(ptep, sz, &pgsize); - - if (!pte_present(pte)) { - for (i = 0; i < pte_num; i++, ptep++, addr += pgsize) - set_ptes(mm, addr, ptep, pte, 1); - return; - } - - if (!pte_napot(pte)) { - set_ptes(mm, addr, ptep, pte, 1); - return; - } - - clear_flush(mm, addr, ptep, pgsize, pte_num); - - set_ptes(mm, addr, ptep, pte, pte_num); -} - int huge_ptep_set_access_flags(struct vm_area_struct *vma, unsigned long addr, pte_t *ptep, diff --git a/include/linux/hugetlb_contpte.h b/include/linux/hugetlb_contpte.h index 2ea17e4fe36b..135b68bd09ca 100644 --- a/include/linux/hugetlb_contpte.h +++ b/include/linux/hugetlb_contpte.h @@ -9,4 +9,9 @@ #define __HAVE_ARCH_HUGE_PTEP_GET extern pte_t huge_ptep_get(struct mm_struct *mm, unsigned long addr, pte_t *ptep); +#define __HAVE_ARCH_HUGE_SET_HUGE_PTE_AT +extern void set_huge_pte_at(struct mm_struct *mm, + unsigned long addr, pte_t *ptep, pte_t pte, + unsigned long sz); + #endif /* _LINUX_HUGETLB_CONTPTE_H */ diff --git a/mm/hugetlb_contpte.c b/mm/hugetlb_contpte.c index 500d0b96a680..cbf93ffcd882 100644 --- a/mm/hugetlb_contpte.c +++ b/mm/hugetlb_contpte.c @@ -30,3 +30,59 @@ pte_t huge_ptep_get(struct mm_struct *mm, unsigned long addr, pte_t *ptep) } return orig_pte; } + +/* + * ARM64: Changing some bits of contiguous entries requires us to follow a + * Break-Before-Make approach, breaking the whole contiguous set + * before we can change any entries. See ARM DDI 0487A.k_iss10775, + * "Misprogramming of the Contiguous bit", page D4-1762. + * + * RISCV: When dealing with NAPOT mappings, the privileged specification + * indicates that "if an update needs to be made, the OS generally should first + * mark all of the PTEs invalid, then issue SFENCE.VMA instruction(s) covering + * all 4 KiB regions within the range, [...] then update the PTE(s), as + * described in Section 4.2.1.". That's the equivalent of the Break-Before-Make + * approach used by arm64. + * + * This helper performs the break step for use cases where the + * original pte is not needed. + */ +static void clear_flush(struct mm_struct *mm, + unsigned long addr, + pte_t *ptep, + unsigned long pgsize, + unsigned long ncontig) +{ + struct vm_area_struct vma = TLB_FLUSH_VMA(mm, 0); + unsigned long i, saddr = addr; + + for (i = 0; i < ncontig; i++, addr += pgsize, ptep++) + __ptep_get_and_clear(mm, addr, ptep); + + flush_tlb_range(&vma, saddr, addr); +} + +void set_huge_pte_at(struct mm_struct *mm, unsigned long addr, + pte_t *ptep, pte_t pte, unsigned long sz) +{ + size_t pgsize; + int i; + int ncontig; + + ncontig = arch_contpte_get_num_contig(ptep, sz, &pgsize); + + if (!pte_present(pte)) { + for (i = 0; i < ncontig; i++, ptep++, addr += pgsize) + __set_ptes(mm, addr, ptep, pte, 1); + return; + } + + if (!pte_cont(pte)) { + __set_ptes(mm, addr, ptep, pte, 1); + return; + } + + clear_flush(mm, addr, ptep, pgsize, ncontig); + + set_contptes(mm, addr, ptep, pte, ncontig, pgsize); +} From patchwork Fri Mar 21 13:06:31 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexandre Ghiti X-Patchwork-Id: 14025410 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 01186C36000 for ; Fri, 21 Mar 2025 13:11:52 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id DB586280007; Fri, 21 Mar 2025 09:11:50 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id D6566280001; Fri, 21 Mar 2025 09:11:50 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C05C5280007; Fri, 21 Mar 2025 09:11:50 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 75F10280001 for ; Fri, 21 Mar 2025 09:11:50 -0400 (EDT) Received: from smtpin27.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 4144AADF5D for ; Fri, 21 Mar 2025 13:11:51 +0000 (UTC) X-FDA: 83245595622.27.4C1021C Received: from mail-wr1-f51.google.com (mail-wr1-f51.google.com [209.85.221.51]) by imf05.hostedemail.com (Postfix) with ESMTP id C9063100014 for ; Fri, 21 Mar 2025 13:11:47 +0000 (UTC) Authentication-Results: imf05.hostedemail.com; dkim=pass header.d=rivosinc-com.20230601.gappssmtp.com header.s=20230601 header.b="y5z/CSJt"; spf=pass (imf05.hostedemail.com: domain of alexghiti@rivosinc.com designates 209.85.221.51 as permitted sender) smtp.mailfrom=alexghiti@rivosinc.com; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1742562707; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=g+Ba9lH6i+4JHX31VX+vvn1rf15Jq1WxTsiq92NtkJo=; b=eLZlp9a/DYMf38iU3wu0xGFZ1k0e9NBJgp5VLZUc8UK5g51Ifvzz7lPbkwwz0KaK67EI5x bMzYoMir4Lx1OS+XyEnXr2WJcguN3x41xDDrR402nsRyJz9i9iZE6bpQdXvnbqkfbSci57 CdAbboLIGjZmVcu8y0rRC9ij20RPzbg= ARC-Authentication-Results: i=1; imf05.hostedemail.com; dkim=pass header.d=rivosinc-com.20230601.gappssmtp.com header.s=20230601 header.b="y5z/CSJt"; spf=pass (imf05.hostedemail.com: domain of alexghiti@rivosinc.com designates 209.85.221.51 as permitted sender) smtp.mailfrom=alexghiti@rivosinc.com; dmarc=none ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1742562707; a=rsa-sha256; cv=none; b=gnFpQ8/ilmXDjllXPlDZZ+9vuM2SmgRtXT1wTmNqOgfrKvOS3tEPXSBitNoFEAxP0wqqBP 5vu8j77gSclVz+6EQ9W4MtppK8TEa4vgWO0NFhUMU9TjfGkFrNlPyfGe+Om0K8EMlZ6JAK gsvkypBDcRNuvP2e0GEVvZbfHAGZ1+k= Received: by mail-wr1-f51.google.com with SMTP id ffacd0b85a97d-39149bccb69so1829420f8f.2 for ; Fri, 21 Mar 2025 06:11:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc-com.20230601.gappssmtp.com; s=20230601; t=1742562706; x=1743167506; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=g+Ba9lH6i+4JHX31VX+vvn1rf15Jq1WxTsiq92NtkJo=; b=y5z/CSJtTrGIulgf4EpWJtLiTBJYypkJXktTytE3DxdSlGbMRMC0RlSDP0+4KW9G02 5vBnIy4JQAXTrYY5UPqz/l1QDPuqjo6Kgwxbw/5KOvHAwXvVlFXiSeRK24vbyKRp8kYE VwSS7bcPpEDadlA7OADFAkMQQGDBWvyc4FIAfDBucP6nWUR6S0ztfYsRNABxGck/Q2qC zAc4rxWRcH61ezSa1pxc64yiafoAxVDsTPeQzciVxI8No6qhcw68p2mhh6QFwEiWUyVB BbmMq0fQHqpIarxfeYateAaxbDL+dmSRWiGXXHKcr6gNdnTITgBZ9RCGAj7/ytLtVGpx eN3A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1742562706; x=1743167506; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=g+Ba9lH6i+4JHX31VX+vvn1rf15Jq1WxTsiq92NtkJo=; b=dSBqUPRZ/AYYlI0sTR0KAlgLv/IuleH1niBUgHdu3xn0vyRoIikdWDpn9qhmThm6Il beHwuiMuXLU3eIZ2W4L73sZM3jEypaBtr44efftbMu55YGoJEPqI7ksIQVEK/Ei1plp1 8Ggb8jJcocjIbX2xrIhXf/CuGwQ31ApubGzYu/fDekWQ1qrjS9qCXzpcA8+o+3713/Wx Sa/nJTblOZ5uSiHGzwtfAGov/5DSItwTmA0XkqWH4+UDamKVtCt13GmDudcY4lK1EaUl s3xouARswulXHxGuWuVX89CkUIPwl2fspGLlnJpCbN6i18mdUmDfdteAxa3m9vrB6e4l bcdw== X-Forwarded-Encrypted: i=1; AJvYcCX7rExsUPHgHVa6JR9X0WloE2TPAXkw+COK9VYOpVx+RNVZOeJHp/Qg0AGkUAiJMcWIEtgNfR7Cdg==@kvack.org X-Gm-Message-State: AOJu0YwParLxpuz818IIkAc4iDGXLmS2i5Ii8zjPEwQs6k5H9S7yHG0Q r/eUYXHhcrClVpyiz051unb6F0OQ8fuc3q4OAaOyF7dKyQE9RsAWshETFnq4S24= X-Gm-Gg: ASbGncsBoQJ/5ynV3lvrQXX9OdJLVBBd88LqKdxl0W8lrNn0GOg4Seph6qeZEgQqd5W nE8rv24Do2J4v9nLobvOfeLSPH9pM1RPyFqLY9BKZeaaxmFaCm2qkak6zLt1/ueAbqkaFE40Ny0 Z9lWrxgWkGZonbv0DM4QivPpasLOhBqIuWWJIZxtGJhNlYWJsTKjRqx5JUtyFkJbX1BCh0F0als 8+Ts4Vmkd85DxbR1648fTHoi34xW4dKmDj6EdtGcJkUNYG8yJ8laz4Lgi6eg8VPSU7LoXziBlkr PcSCxcfB49xaCWSBBudA4eMx8EBOzfakGz/UNvO6+Ku8XhzljHDEZnYJ5koI9g25RxFvrQ== X-Google-Smtp-Source: AGHT+IEGZPDoYAPQGYmkPYx5aU5RVlINcZRG7cNrGB3gR0sMYyiWL8XPHTT0J17da9GbtboJpKfvoQ== X-Received: by 2002:a05:6000:154a:b0:38f:3224:660b with SMTP id ffacd0b85a97d-3997f90a69bmr2874098f8f.22.1742562705888; Fri, 21 Mar 2025 06:11:45 -0700 (PDT) Received: from alex-rivos.ba.rivosinc.com ([2001:861:3382:ef90:3d12:52fe:c1cc:c94]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-3997f9b26a6sm2310275f8f.44.2025.03.21.06.11.45 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 21 Mar 2025 06:11:45 -0700 (PDT) From: Alexandre Ghiti To: Catalin Marinas , Will Deacon , Ryan Roberts , Mark Rutland , Matthew Wilcox , Paul Walmsley , Palmer Dabbelt , Alexandre Ghiti , Andrew Morton , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org, linux-mm@kvack.org Cc: Alexandre Ghiti Subject: [PATCH v5 5/9] mm: Use common huge_pte_clear() function for riscv/arm64 Date: Fri, 21 Mar 2025 14:06:31 +0100 Message-Id: <20250321130635.227011-6-alexghiti@rivosinc.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20250321130635.227011-1-alexghiti@rivosinc.com> References: <20250321130635.227011-1-alexghiti@rivosinc.com> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam09 X-Rspamd-Queue-Id: C9063100014 X-Stat-Signature: zn6m6oh85xnbc5cx61ttpdgm5937pe7a X-HE-Tag: 1742562707-587150 X-HE-Meta: U2FsdGVkX199TALW1G7xfaHczW/T8iZAqbEidjTgovtfabIp91p42eVZyRwjS72Ff7FdYBoJpc7EokDBhNUdCkXEg7zDnQP3kohMpbmF1WnPdgsIxGqniytOHwa32rSHkg1sa5CBIQ+t0U1WyQyQ/vlZkfXzxT1R7nCpqsMewdcIo0+X2KzRvIEFmelvprwHinU18ALQxvldqiE+Wr8lz8RdEddyRlNiXR/BURRIgosVtGLupYheDXLfjOq4350bDarTSxrSKyyRoxMludXpE6z2yZ7bgU29xu3arALUsytuu8pksgEdmNgrJOgJTR+pfl0xdr36FsvJFlQ0EvHqDixmSy077R9ZSzj6YNAxV2lM9Ni6mhn+liKbm4mio9rJqdqxGCTjJ9/et7gM+rVZ2L9WVtU6BpVPrA+uDTlTKnGuHPaV6i4PyaeYMREAzXNqg64zNVsVx06ZbaUYBms3QorNNRrzxwTYEdFKNhF6Z21PkvN24+0yDz457NXDBWIR5uJAQx9nUAyKQ8HDiVBK51cNajyDjcPsBmRiDYMIXYJ3t3AIrdEKqUBdnZw+QIXeeTDX3N1/Em/qBwS/dmkBb6vCcI7XYeyaCUAYVJh25Fv38XvVJZPalgxUX51EZUZUdbJ7knshTuv14PPlaUrF+XzuxpEVnxXfx61Pgu7sR9xzVNdyfZSzWHq8Evf6oaRtvCXubDdKkQCRKP5KdVSR7zphDrKF8HwXR+LjfKYKoxP3+RxcAb99maQqCZzT4zA/xLgzEXjZK9nITrXDw5nRcy3p26Uh6UvSf8R8YfeBrExlsf7/I85PlmrwEW1gzdBZ+LzPbde9IHhdd/HbMRgmSS1RTlSI6erlKREBfoTvR6+Sf+ljeymsn4iM7fdorQWHRC0RSCXHlQdVz7cXCxNfrdU7VsoIsqCpZxASO08jemcg0UxQMDrvZi/y4r1n77H33BTHb0ZNxU2zOYqahkH 3HIB1BBn vsaLXis8Azz1Eph/J00dxvooy9GNeooYxxA4PG77RIU+14UJOVOBLp9ExQBAiOL7g+4WwGljV5QBSf+2eEHKuiLE/TID/43Qy9TtfbfyjGlMdfTeHNeu0RLSRI+AnpVtD1IqMZGdzeuU4/BtCIeYA2Ate9/HG3Bs7AyechNFIrduuAJeha+ulVz/eKQyw8adwHTY6lMem/wTApsDqLluJT4wTL8Gl3w6xgS7Bh+8lYq6ya/30PyJ/ND/7gSJTOk/anMBo3e6/NbHJGSWZEm8GSwT9HrtmJ0NcYqk9u1ISrbCirzFMqOkh9A5xIZCpifcVnIx36whB2gaKcR2Rr7+uLHAmDuiLQkulh17dI99183eArQdcf0mvqQKNIFJckROuyHHbBZ3iSE1933v76UWxPAd9JkEJEUuCaMJITSz/X28sM2WcBmAETH8XkwZskSsIP5Oio/iij0ttGAsOv+KG/OFScg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Both architectures have the same implementation so move it to generic code. Signed-off-by: Alexandre Ghiti --- arch/arm64/include/asm/hugetlb.h | 3 --- arch/arm64/mm/hugetlbpage.c | 12 ------------ arch/riscv/include/asm/hugetlb.h | 4 ---- arch/riscv/include/asm/pgtable.h | 5 +++-- arch/riscv/mm/hugetlbpage.c | 19 ------------------- include/linux/hugetlb_contpte.h | 4 ++++ mm/hugetlb_contpte.c | 12 ++++++++++++ 7 files changed, 19 insertions(+), 40 deletions(-) diff --git a/arch/arm64/include/asm/hugetlb.h b/arch/arm64/include/asm/hugetlb.h index cfdc04e11585..ed75631ad63c 100644 --- a/arch/arm64/include/asm/hugetlb.h +++ b/arch/arm64/include/asm/hugetlb.h @@ -48,9 +48,6 @@ extern void huge_ptep_set_wrprotect(struct mm_struct *mm, #define __HAVE_ARCH_HUGE_PTEP_CLEAR_FLUSH extern pte_t huge_ptep_clear_flush(struct vm_area_struct *vma, unsigned long addr, pte_t *ptep); -#define __HAVE_ARCH_HUGE_PTE_CLEAR -extern void huge_pte_clear(struct mm_struct *mm, unsigned long addr, - pte_t *ptep, unsigned long sz); void __init arm64_hugetlb_cma_reserve(void); diff --git a/arch/arm64/mm/hugetlbpage.c b/arch/arm64/mm/hugetlbpage.c index 6feb90ed2e7d..99728b02a3ca 100644 --- a/arch/arm64/mm/hugetlbpage.c +++ b/arch/arm64/mm/hugetlbpage.c @@ -260,18 +260,6 @@ pte_t arch_make_huge_pte(pte_t entry, unsigned int shift, vm_flags_t flags) return entry; } -void huge_pte_clear(struct mm_struct *mm, unsigned long addr, - pte_t *ptep, unsigned long sz) -{ - int i, ncontig; - size_t pgsize; - - ncontig = arch_contpte_get_num_contig(ptep, sz, &pgsize); - - for (i = 0; i < ncontig; i++, addr += pgsize, ptep++) - __pte_clear(mm, addr, ptep); -} - pte_t huge_ptep_get_and_clear(struct mm_struct *mm, unsigned long addr, pte_t *ptep, unsigned long sz) { diff --git a/arch/riscv/include/asm/hugetlb.h b/arch/riscv/include/asm/hugetlb.h index 7049a17b819d..467bc30c2153 100644 --- a/arch/riscv/include/asm/hugetlb.h +++ b/arch/riscv/include/asm/hugetlb.h @@ -20,10 +20,6 @@ bool arch_hugetlb_migration_supported(struct hstate *h); #endif #ifdef CONFIG_RISCV_ISA_SVNAPOT -#define __HAVE_ARCH_HUGE_PTE_CLEAR -void huge_pte_clear(struct mm_struct *mm, unsigned long addr, - pte_t *ptep, unsigned long sz); - #define __HAVE_ARCH_HUGE_PTEP_GET_AND_CLEAR pte_t huge_ptep_get_and_clear(struct mm_struct *mm, unsigned long addr, pte_t *ptep, diff --git a/arch/riscv/include/asm/pgtable.h b/arch/riscv/include/asm/pgtable.h index 5b34b3c9c0f9..72d3592454d3 100644 --- a/arch/riscv/include/asm/pgtable.h +++ b/arch/riscv/include/asm/pgtable.h @@ -651,8 +651,8 @@ static inline int arch_contpte_get_num_contig(pte_t *ptep, unsigned long size, } #endif -static inline void pte_clear(struct mm_struct *mm, - unsigned long addr, pte_t *ptep) +static inline void __pte_clear(struct mm_struct *mm, + unsigned long addr, pte_t *ptep) { __set_pte_at(mm, ptep, __pte(0)); } @@ -787,6 +787,7 @@ static inline pte_t __ptep_get_and_clear(struct mm_struct *mm, #define __HAVE_ARCH_PTEP_GET_AND_CLEAR #define ptep_get_and_clear __ptep_get_and_clear +#define pte_clear __pte_clear #define pgprot_nx pgprot_nx static inline pgprot_t pgprot_nx(pgprot_t _prot) diff --git a/arch/riscv/mm/hugetlbpage.c b/arch/riscv/mm/hugetlbpage.c index 75faeacc8138..fe82284c3dc4 100644 --- a/arch/riscv/mm/hugetlbpage.c +++ b/arch/riscv/mm/hugetlbpage.c @@ -254,25 +254,6 @@ pte_t huge_ptep_clear_flush(struct vm_area_struct *vma, return get_clear_contig_flush(vma->vm_mm, addr, ptep, pte_num); } -void huge_pte_clear(struct mm_struct *mm, - unsigned long addr, - pte_t *ptep, - unsigned long sz) -{ - size_t pgsize; - pte_t pte = ptep_get(ptep); - int i, pte_num; - - if (!pte_napot(pte)) { - pte_clear(mm, addr, ptep); - return; - } - - pte_num = arch_contpte_get_num_contig(ptep, sz, &pgsize); - for (i = 0; i < pte_num; i++, addr += pgsize, ptep++) - pte_clear(mm, addr, ptep); -} - static bool is_napot_size(unsigned long size) { unsigned long order; diff --git a/include/linux/hugetlb_contpte.h b/include/linux/hugetlb_contpte.h index 135b68bd09ca..e6aa9befa78c 100644 --- a/include/linux/hugetlb_contpte.h +++ b/include/linux/hugetlb_contpte.h @@ -14,4 +14,8 @@ extern void set_huge_pte_at(struct mm_struct *mm, unsigned long addr, pte_t *ptep, pte_t pte, unsigned long sz); +#define __HAVE_ARCH_HUGE_PTE_CLEAR +extern void huge_pte_clear(struct mm_struct *mm, unsigned long addr, + pte_t *ptep, unsigned long sz); + #endif /* _LINUX_HUGETLB_CONTPTE_H */ diff --git a/mm/hugetlb_contpte.c b/mm/hugetlb_contpte.c index cbf93ffcd882..e881b302dd63 100644 --- a/mm/hugetlb_contpte.c +++ b/mm/hugetlb_contpte.c @@ -86,3 +86,15 @@ void set_huge_pte_at(struct mm_struct *mm, unsigned long addr, set_contptes(mm, addr, ptep, pte, ncontig, pgsize); } + +void huge_pte_clear(struct mm_struct *mm, unsigned long addr, + pte_t *ptep, unsigned long sz) +{ + int i, ncontig; + size_t pgsize; + + ncontig = arch_contpte_get_num_contig(ptep, sz, &pgsize); + + for (i = 0; i < ncontig; i++, addr += pgsize, ptep++) + __pte_clear(mm, addr, ptep); +} From patchwork Fri Mar 21 13:06:32 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexandre Ghiti X-Patchwork-Id: 14025420 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id F34F2C36000 for ; Fri, 21 Mar 2025 13:12:51 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id C1E0E280008; Fri, 21 Mar 2025 09:12:49 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id BD020280001; Fri, 21 Mar 2025 09:12:49 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A703C280008; Fri, 21 Mar 2025 09:12:49 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 89F6B280001 for ; Fri, 21 Mar 2025 09:12:49 -0400 (EDT) Received: from smtpin22.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 19A915A07D for ; Fri, 21 Mar 2025 13:12:51 +0000 (UTC) X-FDA: 83245598142.22.44826F3 Received: from mail-wr1-f48.google.com (mail-wr1-f48.google.com [209.85.221.48]) by imf03.hostedemail.com (Postfix) with ESMTP id 1EF472000D for ; Fri, 21 Mar 2025 13:12:48 +0000 (UTC) Authentication-Results: imf03.hostedemail.com; dkim=pass header.d=rivosinc-com.20230601.gappssmtp.com header.s=20230601 header.b=vJSQBUQn; spf=pass (imf03.hostedemail.com: domain of alexghiti@rivosinc.com designates 209.85.221.48 as permitted sender) smtp.mailfrom=alexghiti@rivosinc.com; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1742562769; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=tW5CZGKlc6dgOhT2HYLkHYDVp73Jiwu1Hc2Rif5lDWM=; b=voNrmu5RATlDqlB3eUkuP6h+MJ8FcsHeqbC430XfcJ6DuWX90ao06ODCC+z/fQJyvAhgO4 1j8TOHeFNtvVNdzOuCCO+VoIEGs4gIY+LkI2BEgl16CPZy7UrwBA1PgPrOtNCCScqFDHiV wwtonA9ajxtokToHv7IFCsQecAwC9LY= ARC-Authentication-Results: i=1; imf03.hostedemail.com; dkim=pass header.d=rivosinc-com.20230601.gappssmtp.com header.s=20230601 header.b=vJSQBUQn; spf=pass (imf03.hostedemail.com: domain of alexghiti@rivosinc.com designates 209.85.221.48 as permitted sender) smtp.mailfrom=alexghiti@rivosinc.com; dmarc=none ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1742562769; a=rsa-sha256; cv=none; b=iPcj4MbCXTvI8IDQwpSWYUB8E5X044BL0XhrueEMhG3XRy9mXGTtVLVbqxnwIH/zG20CwE qWMimd3I5P1mwtOoPaR/xjKw+kUGCd4h6KzdhUfBTwiSF6gO5xOqPAEGTnXLfK5qMzFD/y z5Bcv36W1seYZAJfc5HUPXmWMcDirkE= Received: by mail-wr1-f48.google.com with SMTP id ffacd0b85a97d-3913fdd0120so1168100f8f.0 for ; Fri, 21 Mar 2025 06:12:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc-com.20230601.gappssmtp.com; s=20230601; t=1742562768; x=1743167568; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=tW5CZGKlc6dgOhT2HYLkHYDVp73Jiwu1Hc2Rif5lDWM=; b=vJSQBUQnyLzaPHbtsym4gg98xwszuu3W5884jO3DvZ3AVFYik1C4bAWPn17M0KsPAL xHvdbGwMWH5VblHMDoPy/al6jG9c7Ti5D18lpJ/nB/jItu0ZBsW1lr/muQNXB9jknkg3 DKl9O+rMt51Quf4UC63oOgImeiqTJd/rcf1blF7TGHouUZBz1EKPzBgV2zZevq2Lao6a uX3ytN70rzro8NeK/nW4hXYVayDAzd9BX+MutXQVBVziOWAfXU6P1Wg0vuJwhNTMLK9o uWPnF9qdKKCj2EpBJwpbBjUk58i6bnoQ7eDzMwI+r+Z3FabYgdiI41zP3+LRcD6fOh9l tVIA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1742562768; x=1743167568; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=tW5CZGKlc6dgOhT2HYLkHYDVp73Jiwu1Hc2Rif5lDWM=; b=WyHC9DAjrkJroeEzMz1CJ+VPDbvu4+Sygns2FkvKxDWaIyeVBAUQRi11wPQdRPU5xi 7iIu1Cb6G/CXXC3L6kz5rYc8wFsnB7XKtl7QyTy/dLuKIFCKJS/hynRWU5092vpkG2ER 5Ntyb38mc8zH2Tp/aFBsjyP1g7xMSylrKzfSoOja3heS8M5N7IxMMYYqFSRuISRiSp4F ypj3Ai0E71GUHMZgIPhgxSJDIXWMI/CzszRs5LG40khVTxNJLJIfgvlrUHgP1Dp93ylc Rdnu67m8dRBksCxubpKCxRY6BRWICAclF97RM3bERaouP+uFc6TS7hQi+eMLddc8As/4 5vEQ== X-Forwarded-Encrypted: i=1; AJvYcCV4fPnZAxRn/lVOR/YWTiiyu/mAMpMU273NA27qjZVX527nvd86FHwfEoivQVtvYiAoderJoUku1A==@kvack.org X-Gm-Message-State: AOJu0Yxfzj4rPR7YTnGucAgb2CmapZ66h5vFiAeavPcidsoBVs4z9LfU tonEAVRdxEqjAEQSu7jWIEvpu5daWw/mX3WByKBjPXUMxuhug+VD9XJ6xbCO8t8= X-Gm-Gg: ASbGncvracIXUuefk0Pq+YftQQSmMwF9ZYRGc9MKbiwXKPoZ9roQK8I63EIeGvvYeg5 OU2+hg903RT7KmhrZ6cTp+MdfPd4RI2iG2Dvr6iGl70dwPtF7Yyi8T+x1ywav5xO0APt6c0OztH 1S16YLmaIHNg5SX8YgjhAqBNTFjFZK2q3LeM+v4l+OVGTpguXfLvwddYYL92sLToQ4LVSo3u7S8 iBL+jJbAJJl1EyX8jo8GZ35YvW+VyZ1/vBugc781LZzciggT3ktfhWPeEG4ehvTq9wOhjECdZ9c 76Au+RXqx1OZLjtAeA7HpiMIJmhS2T4/n0e2APDgOPG0ZzXhpIWa3x/mG1Ojj4pa5j6M7w== X-Google-Smtp-Source: AGHT+IEUtROIBbu/cy/pImYWjl+q4lRngYo9z7idT6KT9+6QFPXCkP6BZJtyCAPZK3CSSAem4Gc0LA== X-Received: by 2002:a5d:64af:0:b0:391:2db0:2961 with SMTP id ffacd0b85a97d-3997f932efcmr3394631f8f.38.1742562767493; Fri, 21 Mar 2025 06:12:47 -0700 (PDT) Received: from alex-rivos.ba.rivosinc.com ([2001:861:3382:ef90:3d12:52fe:c1cc:c94]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-3997f9e66c7sm2356167f8f.79.2025.03.21.06.12.46 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 21 Mar 2025 06:12:47 -0700 (PDT) From: Alexandre Ghiti To: Catalin Marinas , Will Deacon , Ryan Roberts , Mark Rutland , Matthew Wilcox , Paul Walmsley , Palmer Dabbelt , Alexandre Ghiti , Andrew Morton , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org, linux-mm@kvack.org Cc: Alexandre Ghiti Subject: [PATCH v5 6/9] mm: Use common huge_ptep_get_and_clear() function for riscv/arm64 Date: Fri, 21 Mar 2025 14:06:32 +0100 Message-Id: <20250321130635.227011-7-alexghiti@rivosinc.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20250321130635.227011-1-alexghiti@rivosinc.com> References: <20250321130635.227011-1-alexghiti@rivosinc.com> MIME-Version: 1.0 X-Rspamd-Queue-Id: 1EF472000D X-Stat-Signature: 6geea4zbh8gbiyb9dw7hro5egjat59ga X-Rspam-User: X-Rspamd-Server: rspam12 X-HE-Tag: 1742562768-723644 X-HE-Meta: U2FsdGVkX18NiDG0t9rIXpS6QGlbhKJNSJ0tjn8NILwBlxQRFr/181bm30TKWpVy2Kt/nR7Pd8lSUymeGUcnnybkFwI2lnJPxNiYPK5qfvvxxWQruD9iscNOcTghDBaCa6CkTdnxpr1VPwDmy9BSb0j6naT3YkjEE8ATlF3I7M2R0b6jMil35Bns05mlDzvjm5MtLiKPxCpZ0FuSTUXbiAzTlp760HYznKigat5gQVL6KFEAR/tEdX3Zxyg7lKOtogXLCH8T4z1/XHGLGpAO5kLM6bzrnW9LygHnKQUvez3d22kNrCeMM5ZSwyxz/79Hh3ObNz/LNs+UDOt1I64eAud8FOV5qq6QQYFrakZQi2HorN4F6apA9mFBHe8Kw702d7zQq9TOsvt3XTQtwc0hMvvUJtnefTy+wgH3+qBfloHkZd55aBaga3pff/AxfHruw3vEFvTI9214691glk/H02QI5Fn9x4s9NTFyI1egq6AsM/PJrrwvSs4uLkTrDIwSTKPL3jm3lWhf322QH2qvGPoWKRwuYCeOSOJBub3WKs6KsPgWYFxY0JVb/s56TzVZwuVeYZEdlGARFJTkmw94aKfPGsS5qg1beYhHC1DFt5PlhHlU9aR+2iHV2+B6uw17PfgSEit/p6EV4n2m65MBAivgcvOY7NAuCWrBPXlQyYjDLS+k/3+ddNi2yTFFnDhOCoUrFwoMLEqSZ41gbhgsQ5hePvob/omKAGEdlsnmN7VjKEFiE+l5dT9StkaV1jrc4LTRjCUC0O1SY3V+xMP9AkASgKlLGRMP0/ECKvkXf3D5NCw94GUpowIvnPrWgNPBaLplcH9Dv9ZBDD8PfgdhXM8bssw7tOkijEdVdLRusGWIbo4A/+w213xOfzywo+X6o9IJgZp1FQivZZSt8FjhH7setCo1V9B3ucPP8GIT1eZUCfuUmzY8AOwptKulk7gzi2u+Eerfdr1j+7sbEZp AkMtFj/k Wo9zLefsPYtqBVJu+uxfIbrFBe7RVQQQPlvuX8HhNsKluzAgnTmuYZNRaa9WHoMA7n1hbSSC9VQ2ydcqU5PhsqPS3pSsThrbxeLa3WoRQHVqkHIkYjU+O8Vmzg/7h4i6VjDX8TfycORfflTXpvvWWhlcv+5WABp5ZYWlSiaGOVei12EP9eLMtE4apg/Kt5BdIyVQlnivMdRYIcVSEzdZxKgT5SHYbLzsjIxvWQD0+RYvpfMr+Jy2mQPUgEiXS1hrmKR1ZSc5LrmSdWJferhATrUxC7tNyZ5w+Q+LF5K06mRVnMp4yHegIerbqrMJmwNXfat/0Hi12sWjtw+2TXbtVhnoRiQX+Tj9Y83J+71bA2CyZ5fCR0EixhsQdOgspVhMnzAihVx5Rol/qSo8fgdq6V2+h4n1UByVVNji5T0n1BdrpmPoZsWYMIu/N3YAEXpe3eBsA4zU+zhYYxg95tvvVVAbziw== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: After some adjustments, both architectures have the same implementation so move it to the generic code. Note that get_clear_contig() function is duplicated in the generic and the arm64 code because it is still used by some arm64 functions that will, in the next commits, be moved to the generic code. Once all have been moved, the arm64 version will be removed. Signed-off-by: Alexandre Ghiti --- arch/arm64/include/asm/hugetlb.h | 3 -- arch/arm64/mm/hugetlbpage.c | 10 ------- arch/riscv/include/asm/hugetlb.h | 5 ---- arch/riscv/mm/hugetlbpage.c | 15 ---------- include/linux/hugetlb_contpte.h | 5 ++++ mm/hugetlb_contpte.c | 51 ++++++++++++++++++++++++++++++++ 6 files changed, 56 insertions(+), 33 deletions(-) diff --git a/arch/arm64/include/asm/hugetlb.h b/arch/arm64/include/asm/hugetlb.h index ed75631ad63c..9b1c25775bea 100644 --- a/arch/arm64/include/asm/hugetlb.h +++ b/arch/arm64/include/asm/hugetlb.h @@ -39,9 +39,6 @@ pte_t arch_make_huge_pte(pte_t entry, unsigned int shift, vm_flags_t flags); extern int huge_ptep_set_access_flags(struct vm_area_struct *vma, unsigned long addr, pte_t *ptep, pte_t pte, int dirty); -#define __HAVE_ARCH_HUGE_PTEP_GET_AND_CLEAR -extern pte_t huge_ptep_get_and_clear(struct mm_struct *mm, unsigned long addr, - pte_t *ptep, unsigned long sz); #define __HAVE_ARCH_HUGE_PTEP_SET_WRPROTECT extern void huge_ptep_set_wrprotect(struct mm_struct *mm, unsigned long addr, pte_t *ptep); diff --git a/arch/arm64/mm/hugetlbpage.c b/arch/arm64/mm/hugetlbpage.c index 99728b02a3ca..62a66ce2b2fe 100644 --- a/arch/arm64/mm/hugetlbpage.c +++ b/arch/arm64/mm/hugetlbpage.c @@ -260,16 +260,6 @@ pte_t arch_make_huge_pte(pte_t entry, unsigned int shift, vm_flags_t flags) return entry; } -pte_t huge_ptep_get_and_clear(struct mm_struct *mm, unsigned long addr, - pte_t *ptep, unsigned long sz) -{ - int ncontig; - size_t pgsize; - - ncontig = arch_contpte_get_num_contig(ptep, sz, &pgsize); - return get_clear_contig(mm, addr, ptep, pgsize, ncontig); -} - /* * huge_ptep_set_access_flags will update access flags (dirty, accesssed) * and write permission. diff --git a/arch/riscv/include/asm/hugetlb.h b/arch/riscv/include/asm/hugetlb.h index 467bc30c2153..0fbb6b19df79 100644 --- a/arch/riscv/include/asm/hugetlb.h +++ b/arch/riscv/include/asm/hugetlb.h @@ -20,11 +20,6 @@ bool arch_hugetlb_migration_supported(struct hstate *h); #endif #ifdef CONFIG_RISCV_ISA_SVNAPOT -#define __HAVE_ARCH_HUGE_PTEP_GET_AND_CLEAR -pte_t huge_ptep_get_and_clear(struct mm_struct *mm, - unsigned long addr, pte_t *ptep, - unsigned long sz); - #define __HAVE_ARCH_HUGE_PTEP_CLEAR_FLUSH pte_t huge_ptep_clear_flush(struct vm_area_struct *vma, unsigned long addr, pte_t *ptep); diff --git a/arch/riscv/mm/hugetlbpage.c b/arch/riscv/mm/hugetlbpage.c index fe82284c3dc4..87168123d4a2 100644 --- a/arch/riscv/mm/hugetlbpage.c +++ b/arch/riscv/mm/hugetlbpage.c @@ -203,21 +203,6 @@ int huge_ptep_set_access_flags(struct vm_area_struct *vma, return true; } -pte_t huge_ptep_get_and_clear(struct mm_struct *mm, - unsigned long addr, - pte_t *ptep, unsigned long sz) -{ - pte_t orig_pte = ptep_get(ptep); - int pte_num; - - if (!pte_napot(orig_pte)) - return ptep_get_and_clear(mm, addr, ptep); - - pte_num = arch_contpte_get_num_contig(ptep, sz, NULL); - - return get_clear_contig(mm, addr, ptep, pte_num); -} - void huge_ptep_set_wrprotect(struct mm_struct *mm, unsigned long addr, pte_t *ptep) diff --git a/include/linux/hugetlb_contpte.h b/include/linux/hugetlb_contpte.h index e6aa9befa78c..1c8f46ff95ea 100644 --- a/include/linux/hugetlb_contpte.h +++ b/include/linux/hugetlb_contpte.h @@ -18,4 +18,9 @@ extern void set_huge_pte_at(struct mm_struct *mm, extern void huge_pte_clear(struct mm_struct *mm, unsigned long addr, pte_t *ptep, unsigned long sz); +#define __HAVE_ARCH_HUGE_PTEP_GET_AND_CLEAR +extern pte_t huge_ptep_get_and_clear(struct mm_struct *mm, + unsigned long addr, pte_t *ptep, + unsigned long sz); + #endif /* _LINUX_HUGETLB_CONTPTE_H */ diff --git a/mm/hugetlb_contpte.c b/mm/hugetlb_contpte.c index e881b302dd63..82f49eb79ffb 100644 --- a/mm/hugetlb_contpte.c +++ b/mm/hugetlb_contpte.c @@ -98,3 +98,54 @@ void huge_pte_clear(struct mm_struct *mm, unsigned long addr, for (i = 0; i < ncontig; i++, addr += pgsize, ptep++) __pte_clear(mm, addr, ptep); } + +/* + * ARM: Changing some bits of contiguous entries requires us to follow a + * Break-Before-Make approach, breaking the whole contiguous set + * before we can change any entries. See ARM DDI 0487A.k_iss10775, + * "Misprogramming of the Contiguous bit", page D4-1762. + * + * RISCV: When dealing with NAPOT mappings, the privileged specification + * indicates that "if an update needs to be made, the OS generally should first + * mark all of the PTEs invalid, then issue SFENCE.VMA instruction(s) covering + * all 4 KiB regions within the range, [...] then update the PTE(s), as + * described in Section 4.2.1.". That's the equivalent of the Break-Before-Make + * approach used by arm64. + * + * This helper performs the break step. + */ +static pte_t get_clear_contig(struct mm_struct *mm, + unsigned long addr, + pte_t *ptep, + unsigned long pgsize, + unsigned long ncontig) +{ + pte_t pte, tmp_pte; + bool present; + + pte = __ptep_get_and_clear(mm, addr, ptep); + present = pte_present(pte); + while (--ncontig) { + ptep++; + addr += pgsize; + tmp_pte = __ptep_get_and_clear(mm, addr, ptep); + if (present) { + if (pte_dirty(tmp_pte)) + pte = pte_mkdirty(pte); + if (pte_young(tmp_pte)) + pte = pte_mkyoung(pte); + } + } + return pte; +} + +pte_t huge_ptep_get_and_clear(struct mm_struct *mm, + unsigned long addr, pte_t *ptep, unsigned long sz) +{ + int ncontig; + size_t pgsize; + + ncontig = arch_contpte_get_num_contig(ptep, sz, &pgsize); + + return get_clear_contig(mm, addr, ptep, pgsize, ncontig); +} From patchwork Fri Mar 21 13:06:33 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexandre Ghiti X-Patchwork-Id: 14025421 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id A2375C36002 for ; Fri, 21 Mar 2025 13:13:53 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 482EA280009; Fri, 21 Mar 2025 09:13:51 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 43184280001; Fri, 21 Mar 2025 09:13:51 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 2D300280009; Fri, 21 Mar 2025 09:13:51 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 0D72B280001 for ; Fri, 21 Mar 2025 09:13:51 -0400 (EDT) Received: from smtpin21.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 6C1FFAB0C6 for ; Fri, 21 Mar 2025 13:13:52 +0000 (UTC) X-FDA: 83245600704.21.151D62A Received: from mail-wr1-f52.google.com (mail-wr1-f52.google.com [209.85.221.52]) by imf02.hostedemail.com (Postfix) with ESMTP id 6E8378000D for ; Fri, 21 Mar 2025 13:13:50 +0000 (UTC) Authentication-Results: imf02.hostedemail.com; dkim=pass header.d=rivosinc-com.20230601.gappssmtp.com header.s=20230601 header.b=UsSk2a2U; dmarc=none; spf=pass (imf02.hostedemail.com: domain of alexghiti@rivosinc.com designates 209.85.221.52 as permitted sender) smtp.mailfrom=alexghiti@rivosinc.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1742562830; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=ws5N2/PtP1Eu6ihB8rghsKDx2GXJblSk7INWyBMy7EM=; b=eqskbMSFfCcCWPBKvm9rvuF6gpL9m6KHNJv4R0o2o7Ft7lAQb6fFJhF+UgogQ+dQlQYx+o ASX5kHQH6R/npD3BMtmpO93WzXgZWi3JO0UrJhCyYA4Q4ZdkHtwCKPv1MhzFsQOwYGKnPn LvijfaJiByavF6UHVONEDRNgISikgSo= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1742562830; a=rsa-sha256; cv=none; b=r0SRN9g87+uu01xLpPl08Buuf18J9N7HNVLf3nlH5RX+Tz1KPtgZNTDNFzNJbJ+O5Gyvbz tuwUeW1fostBSiaX7FwiBkh9kcXFa7VaqsfYzUmFcyCjxlPZN1HRdv9JVX8ZHttFg/kNc+ EjSWuUwx01FQ6x2K8rjuZE5p/nPy+Fg= ARC-Authentication-Results: i=1; imf02.hostedemail.com; dkim=pass header.d=rivosinc-com.20230601.gappssmtp.com header.s=20230601 header.b=UsSk2a2U; dmarc=none; spf=pass (imf02.hostedemail.com: domain of alexghiti@rivosinc.com designates 209.85.221.52 as permitted sender) smtp.mailfrom=alexghiti@rivosinc.com Received: by mail-wr1-f52.google.com with SMTP id ffacd0b85a97d-38f2f391864so1095666f8f.3 for ; Fri, 21 Mar 2025 06:13:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc-com.20230601.gappssmtp.com; s=20230601; t=1742562829; x=1743167629; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=ws5N2/PtP1Eu6ihB8rghsKDx2GXJblSk7INWyBMy7EM=; b=UsSk2a2UdEz4J7dqZrvgEpei2tOxDZ/xvYeN41Bz2e0hdelezPOq2eNgSXv6XdbDgj XBJ2XaSBXhNhaWVVE38khDuzXajbnf4dk17APPYT21EbgnINzMr+x/Sc8G8s19Sy5/OW WpvkmZSE+ShGGsT/23vHYx8eDBSXWs0XuPIcHdJ/37o6Hu23iIdsrot3fKNeEFMPuqCD 49sq2+nYyyHAbxOzoNlwsjyPX/0ncFHa48IC9EPa5ekA+XBdQna6GuzsVPonqlaRSOCL SjoAwopWU6Gs4zQKt6SWaMSeIZUcngiycmCMOmDQggM9q1r073WkffUK6nF7APs20Qzd o8iQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1742562829; x=1743167629; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=ws5N2/PtP1Eu6ihB8rghsKDx2GXJblSk7INWyBMy7EM=; b=SVJuJt601UbBkdZTvTCcd/MWQ6PtYwXwii4VOTyFcjwmYcPl28b01w5aZBau0xnppY EGzPal1+ojQX4KZt1PGIg6dOmP0Bv8BPc0sWObve/E4KkAXG+K6LiLXC6jXTPiapyhQ/ pS32mlSoJ54cHcDFJRkzBZ47FCxgl7p/0ZFuqwJKW/zeX/Fiy4rSYwyCYckhA2dlYLLn uaSmc37nrEVISzcFADmBJvnvWFZTMMv00cLaT3G1YeonS7apmOuts80UlE6huNk8xHvm D5U5FE3/wGdDotgWVPb564xeeuDSOmANQKJEikbuHKrfVku9oNKnt4IKtKWxVblJuaVI VClA== X-Forwarded-Encrypted: i=1; AJvYcCXH6cAwZAGF3ehIovn5NAkgK2hLOAmRZFohB6vuhw55/AvCxP3UreU6d1Jzzkh5lV+D0nR36eaMnw==@kvack.org X-Gm-Message-State: AOJu0YxB+VUtLL1FoL9ekaTVExSOg/RIp4S4/D8w+E7Jau/Gzl+rJyIe +WlnXA6k7Oj3uK3MFQAVZCscv4rY+XxKCpZDReVDQfH4Q+dwIrTl7gg227Urj18= X-Gm-Gg: ASbGncvTjJbW+dQEoAt5I4EL54uWdrcmHmiLmaATuTn9MTr1iDlllnPynsG7EH054Nq RjMQNNvgOTjipFiMCba8RPkp6bIz9SFwFxR2mZlNGqqPkji7zo17uwns2ncAfuSg/sIW9lys8Tm xiYY1LMvXciu+HfLkJZPKMVJ9ljQZu933cQTqMDeA/t5gbxiieiUjMnSicCK+G4PjnqNwycwjjL uRWhenW3gKX0SNerKIL1YHoGWa2Xc8cDwF/+1OINgPhZ9CQkzuhZFBtDjQWprcprklIJipOWTzW NsX8RvHoA/XyCfNLdtnYBwjcaA2JMOPmcXXzEDnzHq5vSo0yXYNRPerY225Z3f19RwTHDmjdm/N 1Kb8/ X-Google-Smtp-Source: AGHT+IGUJnxhjyGu7ufB6wyGpdYdbKS7+6k2eyHYgFKxVHRlB/Xm7E8vrXAFhp0hvlUzI9R7KED60g== X-Received: by 2002:a5d:64af:0:b0:391:2db0:2961 with SMTP id ffacd0b85a97d-3997f932efcmr3397837f8f.38.1742562828869; Fri, 21 Mar 2025 06:13:48 -0700 (PDT) Received: from alex-rivos.ba.rivosinc.com ([2001:861:3382:ef90:3d12:52fe:c1cc:c94]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-3997f9e674fsm2292970f8f.80.2025.03.21.06.13.48 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 21 Mar 2025 06:13:48 -0700 (PDT) From: Alexandre Ghiti To: Catalin Marinas , Will Deacon , Ryan Roberts , Mark Rutland , Matthew Wilcox , Paul Walmsley , Palmer Dabbelt , Alexandre Ghiti , Andrew Morton , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org, linux-mm@kvack.org Cc: Alexandre Ghiti Subject: [PATCH v5 7/9] mm: Use common huge_ptep_set_access_flags() function for riscv/arm64 Date: Fri, 21 Mar 2025 14:06:33 +0100 Message-Id: <20250321130635.227011-8-alexghiti@rivosinc.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20250321130635.227011-1-alexghiti@rivosinc.com> References: <20250321130635.227011-1-alexghiti@rivosinc.com> MIME-Version: 1.0 X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: 6E8378000D X-Stat-Signature: kut8i81wtw6qzrbpd8i37ud3jah9ktsg X-Rspam-User: X-HE-Tag: 1742562830-410119 X-HE-Meta: U2FsdGVkX1/IxH7y10jw1GPZUwxkOnsii3wzJw1hyds0wN+BozDwRpTXWAUNWs7RdSmRI1oRnGSAh/BYUrnBbPigoV5db0n0zcYN+J1808tGVDvFPW59E1ivoSOv4QS7UPda+WrobIcEXlQQqfQAajQ8OtscmfiKFmU5FqTIdc1NTH/xSfF1qDb3CFDIWKBdyHqZnKqSBLn3IK5R94VQB2CecWF6XD7n1XaKYnSxOXlFdN3/jqesbC6FYUSIOICESN+hSKpIY9yT5RxQymJ/sqn/UfTfi49mtWytEmVbEFsYbvOWSJFL0x9AiEKgmnSUC+PWUllZBDT6ELdMH5ssf0Ph2Ctp0ERlmddxzVHIRuoc8ZJchYp2ComCdqVxN7PvALXIG63wxh/b87cIeXVdQYQbXzwJCYdCuHyoJp4LL5BAyDvP74wOnDiQyVDYNgTd+DG+P6w+7KRh/IxayY+TL0isuPB05VtDgF3SPHjMnyo1vBzGIfjC4gJKhLl1lRrENzewOY/BRn+adnn8sndTzmgIHde+rAgLQq22rzVH3Gde6RnN3ElnrRdtz8Jbq8wGlMQNdb3mv6trzx/YZ1XoXKfh2K0RGHmxIK16X0PVX6KhCzozFuMlreQumzCwc8abg2hdieRj95EdBUyZ5E8BW4NOkN0g8BYfV8xemoss3U73MxRRREmPOopZQepaJaPteZXWToqmgIAap5ajGslWKgoW5wHTQdYqwCOd0cq3rJK3kJsR5f1++oUA9utEn0Q9fE31X/4RmLanCBTYO4ybg4nuxI/x+BRcjjarG6LZ7MI75veyuZ+Vc0z9j5NmhlHRvS/tEFrBHSiDyd7aSeYPKzEUskNTlKVY0atipxKkYTCJvLp5SG+rXAIUF7rzq5kLzQv/nSvawAGkURJYdEU5R+PXFX+g6GfmjUELszaY6XyJK1zlq3vcSKmnl62TV/jRQN96P6WbUWJWfsQ2Dp6 GwQH1gmr RDI/hyj6S8zAb6FZsNSirHYwhRGZ+S/tPHVr7kolNBjcBI5zH3WWrDpIWhd84qUfOOKxTxtbqqozrg4zmhLNdncVH3VSHhXcDtF01QHQxtGGpIxlokm8Fq8mOEZD3yjPiZWW8Se87rQnnmSatBSmVDPhVGiYO58XGO5ZVhoN+WBCLWGUciGPe50mU9v6Gh5CWZGoSGphtuGCXdfEfjL5rYujBv8M2IYGhoatOUN7aBz8tjb/nPKgX2RlNT0L6BOj9puHpksWhxe/JhPpd/X1GaIOh6vEMBTtNfz0XdZFZ6qv8pgxkfyn9TWKlhOeWYxmSwdRHUAa4jf/vAYj0x3RbUktkjnEwA1quQf3QSEUvePedhYC5et2gt3TnZp3wlwyrBUHIC8YIiX4TW2H8DBcSDRmT20FuWRCyBOxN1qNTU7m4bBebuH3bCI4uJDDK2LlfaIf2DZa6SO+6KkNA8RlSxI4VXA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Both architectures have almost the same implementation: __cont_access_flags_changed() is also correct on riscv and brings the same benefits (ie don't do anything if the flags are unchanged). As in the previous commit, get_clear_contig_flush() is duplicated in both the arch and the generic codes, it will be removed from the arch code when the last reference there gets moved to the generic code. Signed-off-by: Alexandre Ghiti --- arch/arm64/include/asm/hugetlb.h | 4 -- arch/arm64/include/asm/pgtable.h | 15 +++++- arch/arm64/mm/hugetlbpage.c | 69 +-------------------------- arch/riscv/include/asm/hugetlb.h | 5 -- arch/riscv/include/asm/pgtable.h | 11 +++-- arch/riscv/mm/hugetlbpage.c | 32 ++----------- arch/riscv/mm/pgtable.c | 6 +-- include/linux/hugetlb_contpte.h | 5 ++ mm/hugetlb_contpte.c | 81 ++++++++++++++++++++++++++++++-- 9 files changed, 110 insertions(+), 118 deletions(-) diff --git a/arch/arm64/include/asm/hugetlb.h b/arch/arm64/include/asm/hugetlb.h index 9b1c25775bea..29a9dac52cef 100644 --- a/arch/arm64/include/asm/hugetlb.h +++ b/arch/arm64/include/asm/hugetlb.h @@ -35,10 +35,6 @@ static inline void arch_clear_hugetlb_flags(struct folio *folio) pte_t arch_make_huge_pte(pte_t entry, unsigned int shift, vm_flags_t flags); #define arch_make_huge_pte arch_make_huge_pte -#define __HAVE_ARCH_HUGE_PTEP_SET_ACCESS_FLAGS -extern int huge_ptep_set_access_flags(struct vm_area_struct *vma, - unsigned long addr, pte_t *ptep, - pte_t pte, int dirty); #define __HAVE_ARCH_HUGE_PTEP_SET_WRPROTECT extern void huge_ptep_set_wrprotect(struct mm_struct *mm, unsigned long addr, pte_t *ptep); diff --git a/arch/arm64/include/asm/pgtable.h b/arch/arm64/include/asm/pgtable.h index af8156929c1d..9b5c57e56691 100644 --- a/arch/arm64/include/asm/pgtable.h +++ b/arch/arm64/include/asm/pgtable.h @@ -1847,12 +1847,23 @@ static inline bool __hugetlb_valid_size(unsigned long size) return false; } -static inline int arch_contpte_get_num_contig(pte_t *ptep, - unsigned long size, +extern int find_num_contig(struct mm_struct *mm, unsigned long addr, + pte_t *ptep, size_t *pgsize); + +static inline int arch_contpte_get_num_contig(struct mm_struct *mm, + unsigned long addr, + pte_t *ptep, unsigned long size, size_t *pgsize) { int contig_ptes = 1; + /* + * If the size is not passed, we need to go through the page table to + * find out the number of contiguous ptes. + */ + if (size == 0) + return find_num_contig(mm, addr, ptep, pgsize); + if (pgsize) *pgsize = size; diff --git a/arch/arm64/mm/hugetlbpage.c b/arch/arm64/mm/hugetlbpage.c index 62a66ce2b2fe..03cb757f7935 100644 --- a/arch/arm64/mm/hugetlbpage.c +++ b/arch/arm64/mm/hugetlbpage.c @@ -63,8 +63,8 @@ bool arch_hugetlb_migration_supported(struct hstate *h) } #endif -static int find_num_contig(struct mm_struct *mm, unsigned long addr, - pte_t *ptep, size_t *pgsize) +int find_num_contig(struct mm_struct *mm, unsigned long addr, + pte_t *ptep, size_t *pgsize) { pgd_t *pgdp = pgd_offset(mm, addr); p4d_t *p4dp; @@ -260,71 +260,6 @@ pte_t arch_make_huge_pte(pte_t entry, unsigned int shift, vm_flags_t flags) return entry; } -/* - * huge_ptep_set_access_flags will update access flags (dirty, accesssed) - * and write permission. - * - * For a contiguous huge pte range we need to check whether or not write - * permission has to change only on the first pte in the set. Then for - * all the contiguous ptes we need to check whether or not there is a - * discrepancy between dirty or young. - */ -static int __cont_access_flags_changed(pte_t *ptep, pte_t pte, int ncontig) -{ - int i; - - if (pte_write(pte) != pte_write(__ptep_get(ptep))) - return 1; - - for (i = 0; i < ncontig; i++) { - pte_t orig_pte = __ptep_get(ptep + i); - - if (pte_dirty(pte) != pte_dirty(orig_pte)) - return 1; - - if (pte_young(pte) != pte_young(orig_pte)) - return 1; - } - - return 0; -} - -int huge_ptep_set_access_flags(struct vm_area_struct *vma, - unsigned long addr, pte_t *ptep, - pte_t pte, int dirty) -{ - int ncontig, i; - size_t pgsize = 0; - unsigned long pfn = pte_pfn(pte), dpfn; - struct mm_struct *mm = vma->vm_mm; - pgprot_t hugeprot; - pte_t orig_pte; - - if (!pte_cont(pte)) - return __ptep_set_access_flags(vma, addr, ptep, pte, dirty); - - ncontig = find_num_contig(mm, addr, ptep, &pgsize); - dpfn = pgsize >> PAGE_SHIFT; - - if (!__cont_access_flags_changed(ptep, pte, ncontig)) - return 0; - - orig_pte = get_clear_contig_flush(mm, addr, ptep, pgsize, ncontig); - - /* Make sure we don't lose the dirty or young state */ - if (pte_dirty(orig_pte)) - pte = pte_mkdirty(pte); - - if (pte_young(orig_pte)) - pte = pte_mkyoung(pte); - - hugeprot = pte_pgprot(pte); - for (i = 0; i < ncontig; i++, ptep++, addr += pgsize, pfn += dpfn) - __set_ptes(mm, addr, ptep, pfn_pte(pfn, hugeprot), 1); - - return 1; -} - void huge_ptep_set_wrprotect(struct mm_struct *mm, unsigned long addr, pte_t *ptep) { diff --git a/arch/riscv/include/asm/hugetlb.h b/arch/riscv/include/asm/hugetlb.h index 0fbb6b19df79..bf533c2cef84 100644 --- a/arch/riscv/include/asm/hugetlb.h +++ b/arch/riscv/include/asm/hugetlb.h @@ -28,11 +28,6 @@ pte_t huge_ptep_clear_flush(struct vm_area_struct *vma, void huge_ptep_set_wrprotect(struct mm_struct *mm, unsigned long addr, pte_t *ptep); -#define __HAVE_ARCH_HUGE_PTEP_SET_ACCESS_FLAGS -int huge_ptep_set_access_flags(struct vm_area_struct *vma, - unsigned long addr, pte_t *ptep, - pte_t pte, int dirty); - pte_t arch_make_huge_pte(pte_t entry, unsigned int shift, vm_flags_t flags); #define arch_make_huge_pte arch_make_huge_pte diff --git a/arch/riscv/include/asm/pgtable.h b/arch/riscv/include/asm/pgtable.h index 72d3592454d3..081385e0d10a 100644 --- a/arch/riscv/include/asm/pgtable.h +++ b/arch/riscv/include/asm/pgtable.h @@ -613,7 +613,9 @@ static inline void ___set_ptes(struct mm_struct *mm, unsigned long addr, * Some hugetlb functions can be called on !present ptes, so we must use the * size parameter when it is passed. */ -static inline int arch_contpte_get_num_contig(pte_t *ptep, unsigned long size, +static inline int arch_contpte_get_num_contig(struct mm_struct *mm, + unsigned long addr, + pte_t *ptep, unsigned long size, size_t *pgsize) { unsigned long hugepage_shift; @@ -657,9 +659,8 @@ static inline void __pte_clear(struct mm_struct *mm, __set_pte_at(mm, ptep, __pte(0)); } -#define __HAVE_ARCH_PTEP_SET_ACCESS_FLAGS /* defined in mm/pgtable.c */ -extern int ptep_set_access_flags(struct vm_area_struct *vma, unsigned long address, - pte_t *ptep, pte_t entry, int dirty); +extern int __ptep_set_access_flags(struct vm_area_struct *vma, unsigned long address, + pte_t *ptep, pte_t entry, int dirty); #define __HAVE_ARCH_PTEP_TEST_AND_CLEAR_YOUNG /* defined in mm/pgtable.c */ extern int ptep_test_and_clear_young(struct vm_area_struct *vma, unsigned long address, pte_t *ptep); @@ -788,6 +789,8 @@ static inline pte_t __ptep_get_and_clear(struct mm_struct *mm, #define __HAVE_ARCH_PTEP_GET_AND_CLEAR #define ptep_get_and_clear __ptep_get_and_clear #define pte_clear __pte_clear +#define __HAVE_ARCH_PTEP_SET_ACCESS_FLAGS +#define ptep_set_access_flags __ptep_set_access_flags #define pgprot_nx pgprot_nx static inline pgprot_t pgprot_nx(pgprot_t _prot) diff --git a/arch/riscv/mm/hugetlbpage.c b/arch/riscv/mm/hugetlbpage.c index 87168123d4a2..b2046f4bd445 100644 --- a/arch/riscv/mm/hugetlbpage.c +++ b/arch/riscv/mm/hugetlbpage.c @@ -176,33 +176,6 @@ pte_t arch_make_huge_pte(pte_t entry, unsigned int shift, vm_flags_t flags) return entry; } -int huge_ptep_set_access_flags(struct vm_area_struct *vma, - unsigned long addr, - pte_t *ptep, - pte_t pte, - int dirty) -{ - struct mm_struct *mm = vma->vm_mm; - pte_t orig_pte; - int pte_num; - - if (!pte_napot(pte)) - return ptep_set_access_flags(vma, addr, ptep, pte, dirty); - - pte_num = arch_contpte_get_num_contig(ptep, 0, NULL); - orig_pte = get_clear_contig_flush(mm, addr, ptep, pte_num); - - if (pte_dirty(orig_pte)) - pte = pte_mkdirty(pte); - - if (pte_young(orig_pte)) - pte = pte_mkyoung(pte); - - set_ptes(mm, addr, ptep, pte, pte_num); - - return true; -} - void huge_ptep_set_wrprotect(struct mm_struct *mm, unsigned long addr, pte_t *ptep) @@ -216,7 +189,8 @@ void huge_ptep_set_wrprotect(struct mm_struct *mm, return; } - pte_num = arch_contpte_get_num_contig(ptep, 0, NULL); + pte_num = arch_contpte_get_num_contig(mm, addr, ptep, 0, NULL); + orig_pte = get_clear_contig_flush(mm, addr, ptep, pte_num); orig_pte = pte_wrprotect(orig_pte); @@ -234,7 +208,7 @@ pte_t huge_ptep_clear_flush(struct vm_area_struct *vma, if (!pte_napot(pte)) return ptep_clear_flush(vma, addr, ptep); - pte_num = arch_contpte_get_num_contig(ptep, 0, NULL); + pte_num = arch_contpte_get_num_contig(vma->vm_mm, addr, ptep, 0, NULL); return get_clear_contig_flush(vma->vm_mm, addr, ptep, pte_num); } diff --git a/arch/riscv/mm/pgtable.c b/arch/riscv/mm/pgtable.c index 4ae67324f992..af8b3769a349 100644 --- a/arch/riscv/mm/pgtable.c +++ b/arch/riscv/mm/pgtable.c @@ -5,9 +5,9 @@ #include #include -int ptep_set_access_flags(struct vm_area_struct *vma, - unsigned long address, pte_t *ptep, - pte_t entry, int dirty) +int __ptep_set_access_flags(struct vm_area_struct *vma, + unsigned long address, pte_t *ptep, + pte_t entry, int dirty) { asm goto(ALTERNATIVE("nop", "j %l[svvptc]", 0, RISCV_ISA_EXT_SVVPTC, 1) : : : : svvptc); diff --git a/include/linux/hugetlb_contpte.h b/include/linux/hugetlb_contpte.h index 1c8f46ff95ea..e129578f6500 100644 --- a/include/linux/hugetlb_contpte.h +++ b/include/linux/hugetlb_contpte.h @@ -23,4 +23,9 @@ extern pte_t huge_ptep_get_and_clear(struct mm_struct *mm, unsigned long addr, pte_t *ptep, unsigned long sz); +#define __HAVE_ARCH_HUGE_PTEP_SET_ACCESS_FLAGS +extern int huge_ptep_set_access_flags(struct vm_area_struct *vma, + unsigned long addr, pte_t *ptep, + pte_t pte, int dirty); + #endif /* _LINUX_HUGETLB_CONTPTE_H */ diff --git a/mm/hugetlb_contpte.c b/mm/hugetlb_contpte.c index 82f49eb79ffb..b4c409d11195 100644 --- a/mm/hugetlb_contpte.c +++ b/mm/hugetlb_contpte.c @@ -15,7 +15,7 @@ pte_t huge_ptep_get(struct mm_struct *mm, unsigned long addr, pte_t *ptep) if (!pte_present(orig_pte) || !pte_cont(orig_pte)) return orig_pte; - ncontig = arch_contpte_get_num_contig(ptep, + ncontig = arch_contpte_get_num_contig(mm, addr, ptep, page_size(pte_page(orig_pte)), NULL); @@ -69,7 +69,7 @@ void set_huge_pte_at(struct mm_struct *mm, unsigned long addr, int i; int ncontig; - ncontig = arch_contpte_get_num_contig(ptep, sz, &pgsize); + ncontig = arch_contpte_get_num_contig(mm, addr, ptep, sz, &pgsize); if (!pte_present(pte)) { for (i = 0; i < ncontig; i++, ptep++, addr += pgsize) @@ -93,7 +93,7 @@ void huge_pte_clear(struct mm_struct *mm, unsigned long addr, int i, ncontig; size_t pgsize; - ncontig = arch_contpte_get_num_contig(ptep, sz, &pgsize); + ncontig = arch_contpte_get_num_contig(mm, addr, ptep, sz, &pgsize); for (i = 0; i < ncontig; i++, addr += pgsize, ptep++) __pte_clear(mm, addr, ptep); @@ -145,7 +145,80 @@ pte_t huge_ptep_get_and_clear(struct mm_struct *mm, int ncontig; size_t pgsize; - ncontig = arch_contpte_get_num_contig(ptep, sz, &pgsize); + ncontig = arch_contpte_get_num_contig(mm, addr, ptep, sz, &pgsize); return get_clear_contig(mm, addr, ptep, pgsize, ncontig); } + +/* + * huge_ptep_set_access_flags will update access flags (dirty, accesssed) + * and write permission. + * + * For a contiguous huge pte range we need to check whether or not write + * permission has to change only on the first pte in the set. Then for + * all the contiguous ptes we need to check whether or not there is a + * discrepancy between dirty or young. + */ +static int __cont_access_flags_changed(pte_t *ptep, pte_t pte, int ncontig) +{ + int i; + + if (pte_write(pte) != pte_write(__ptep_get(ptep))) + return 1; + + for (i = 0; i < ncontig; i++) { + pte_t orig_pte = __ptep_get(ptep + i); + + if (pte_dirty(pte) != pte_dirty(orig_pte)) + return 1; + + if (pte_young(pte) != pte_young(orig_pte)) + return 1; + } + + return 0; +} + +static pte_t get_clear_contig_flush(struct mm_struct *mm, + unsigned long addr, + pte_t *ptep, + unsigned long pgsize, + unsigned long ncontig) +{ + pte_t orig_pte = get_clear_contig(mm, addr, ptep, pgsize, ncontig); + struct vm_area_struct vma = TLB_FLUSH_VMA(mm, 0); + + flush_tlb_range(&vma, addr, addr + (pgsize * ncontig)); + return orig_pte; +} + +int huge_ptep_set_access_flags(struct vm_area_struct *vma, + unsigned long addr, pte_t *ptep, + pte_t pte, int dirty) +{ + int ncontig; + size_t pgsize = 0; + struct mm_struct *mm = vma->vm_mm; + pte_t orig_pte; + + if (!pte_cont(pte)) + return __ptep_set_access_flags(vma, addr, ptep, pte, dirty); + + ncontig = arch_contpte_get_num_contig(vma->vm_mm, addr, ptep, 0, &pgsize); + + if (!__cont_access_flags_changed(ptep, pte, ncontig)) + return 0; + + orig_pte = get_clear_contig_flush(mm, addr, ptep, pgsize, ncontig); + + /* Make sure we don't lose the dirty or young state */ + if (pte_dirty(orig_pte)) + pte = pte_mkdirty(pte); + + if (pte_young(orig_pte)) + pte = pte_mkyoung(pte); + + set_contptes(mm, addr, ptep, pte, ncontig, pgsize); + + return 1; +} From patchwork Fri Mar 21 13:06:34 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexandre Ghiti X-Patchwork-Id: 14025422 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 13D2EC36000 for ; Fri, 21 Mar 2025 13:14:55 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 9699C28000B; Fri, 21 Mar 2025 09:14:52 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 8CC9F28000A; Fri, 21 Mar 2025 09:14:52 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 7468228000B; Fri, 21 Mar 2025 09:14:52 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 52F1A28000A for ; Fri, 21 Mar 2025 09:14:52 -0400 (EDT) Received: from smtpin30.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 0D72D16250E for ; Fri, 21 Mar 2025 13:14:54 +0000 (UTC) X-FDA: 83245603308.30.B0DB377 Received: from mail-wr1-f42.google.com (mail-wr1-f42.google.com [209.85.221.42]) by imf10.hostedemail.com (Postfix) with ESMTP id D2330C000A for ; Fri, 21 Mar 2025 13:14:51 +0000 (UTC) Authentication-Results: imf10.hostedemail.com; dkim=pass header.d=rivosinc-com.20230601.gappssmtp.com header.s=20230601 header.b=thm+3h9R; spf=pass (imf10.hostedemail.com: domain of alexghiti@rivosinc.com designates 209.85.221.42 as permitted sender) smtp.mailfrom=alexghiti@rivosinc.com; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1742562891; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=JOImW0XTENLZqMOVas1NKgXRQH/ppGnjlH3sp/4Pu3c=; b=N4ikf1V3yxJvcqp0OAeYExqMFQKSev4g4iipE7q1nZd+KSheuugeyz+xZ4x6nkDZ92FLEp XVRCA/QrZMdowE5+mr8Hpj/ofzFUTilpYzK5hgj+Ggu9QeTjYUWdevdMh1FZMhswSUgOdd 2m5G5GCKjdEM406V41LyxAQq4w6o+uY= ARC-Authentication-Results: i=1; imf10.hostedemail.com; dkim=pass header.d=rivosinc-com.20230601.gappssmtp.com header.s=20230601 header.b=thm+3h9R; spf=pass (imf10.hostedemail.com: domain of alexghiti@rivosinc.com designates 209.85.221.42 as permitted sender) smtp.mailfrom=alexghiti@rivosinc.com; dmarc=none ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1742562891; a=rsa-sha256; cv=none; b=wFVdUK+inHmQRyzrAratHJLe5CnxmxPFsUuBiOIeepQZn7SzdOku+a8wjgQ65crRabg0ta X7/R8RCyLMPp0Yr33iAIRNtbSNZs92MOZTqL3M9H6XxN5yOzmMIjtRxB85seEMOsNYDOrU D35hkueuLRWjUrQek7cvSkGiU8IR5NI= Received: by mail-wr1-f42.google.com with SMTP id ffacd0b85a97d-39130ee05b0so1860147f8f.3 for ; Fri, 21 Mar 2025 06:14:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc-com.20230601.gappssmtp.com; s=20230601; t=1742562890; x=1743167690; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=JOImW0XTENLZqMOVas1NKgXRQH/ppGnjlH3sp/4Pu3c=; b=thm+3h9Rki5fw8j53//WWa8CGc71iMmRaP9hmw3Wi71AMpvgTswjMdipcDqz/LZ7lf N0DSHxQbG9TcsUyHz4caxdv7leQWtvNVC2q6iBRdWwH9fHh6fMjUNme57yLvXTyYMm4w irlZcPL6UQ9kw2K222x+ZRQd8kZwdVosl+RiW2T/9MtwE1qT5qffSTklmKCSs32ioamI JoWd1RcxpOryBq2170/vicCFVIneTETBkXt6Jya2jjM1BCPn69UN1tVIUXtVwOLzxJ05 sqCy1+uy9wV2nbzT58+o0xpOR7mjzEjFyqoAaZ55ukq22HcRCSvweGqc+cOlZEIqQHtg GJHg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1742562890; x=1743167690; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=JOImW0XTENLZqMOVas1NKgXRQH/ppGnjlH3sp/4Pu3c=; b=YuoLD/9bvO91kW2Lzqxv6JyF2CdzvXwdg3c8MVoec5pOJe1kN1pu06qAQL2zkR3caM jHbbu9OerK8vI/QIf1HYtNG5uPZNqT8nsHcmiELGu6r4H6fEoTYFEd493EDzMK8uwUcz 9S+pL8dIHBC/HSTxgICcOqwqyU0wN/mNsUfS47J8vrnc08ATJg83oRlmQcrxnwUCs+j/ aeTjRRBr6WaCtYMyi0rirwGBXODlLMxl3s6x1E0IuSdz0EDG8sEPEAeyeKIMe+gK/xN/ UPzoXIZDsh9VzawX4cQ+TDM9GwMOku/W0/Vbc9/0jnTmWSSJX5kArhZPnZNMW3P3fMOo BP/Q== X-Forwarded-Encrypted: i=1; AJvYcCVGNS92j9tAo4mcmq312EdKp0N2HSeQffClbQbnTw2XTb35csgv4UZmNF24+MehiWijSawbnQSQQw==@kvack.org X-Gm-Message-State: AOJu0YwmYnDjajADQZkwaRqztwpRSn3xSATuu/vvA4P3dgZ5ORrYuTLD FL3luvTH00Dn51z6ozfnC1goZkBvaPZ6ncQuoeaWMF/oNduaUB0By8Z6AAr7uH2q087WRgDF2fV C X-Gm-Gg: ASbGncvXTY1OnWOw2tOgVLsz9iLpWYeQ2kBZZCRep8w146V7v34oAkJ2s4N5q61HSe4 TlXyNOoAg3xwwVUPhwlUHdkGkBDR9Zqnk7pv3E9sZkuB3I6XwTYc+Wh1BpS5HSoOdzbhrODigXU uzuMoy9/fP3sPLpi5iuCZidxJQOGOuaPvT9cfDnx4jwG/Fk+b685wTV9f2BbPL5QkhATQo1VJxX lxJyRmuJR+oTz2yBqcQ1vzas64mZWxRx3D26iXGCLsBVf1C3lGjO0toETYn8AYhbcFsF1qNmQxI nL4QztnpJDy6y3jifhGww0k3iCwG7PdJsxcvN8Aj9Q7qNhqYVV+TSEMF+Xt3+b6x//AtWw== X-Google-Smtp-Source: AGHT+IG4Bh4b8ljgMGozvKhj+gGf/zTKUltGlpRU5vbjoRJl/qtI6QCkciAO8DnHWDVRjHELB4GFxw== X-Received: by 2002:a05:6000:381:b0:399:737f:4e02 with SMTP id ffacd0b85a97d-3997f938798mr3602833f8f.39.1742562890297; Fri, 21 Mar 2025 06:14:50 -0700 (PDT) Received: from alex-rivos.ba.rivosinc.com ([2001:861:3382:ef90:3d12:52fe:c1cc:c94]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-3997f9eef37sm2294241f8f.85.2025.03.21.06.14.49 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 21 Mar 2025 06:14:49 -0700 (PDT) From: Alexandre Ghiti To: Catalin Marinas , Will Deacon , Ryan Roberts , Mark Rutland , Matthew Wilcox , Paul Walmsley , Palmer Dabbelt , Alexandre Ghiti , Andrew Morton , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org, linux-mm@kvack.org Cc: Alexandre Ghiti Subject: [PATCH v5 8/9] mm: Use common huge_ptep_set_wrprotect() function for riscv/arm64 Date: Fri, 21 Mar 2025 14:06:34 +0100 Message-Id: <20250321130635.227011-9-alexghiti@rivosinc.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20250321130635.227011-1-alexghiti@rivosinc.com> References: <20250321130635.227011-1-alexghiti@rivosinc.com> MIME-Version: 1.0 X-Rspamd-Queue-Id: D2330C000A X-Rspamd-Server: rspam05 X-Rspam-User: X-Stat-Signature: yecydbjdqz4zq8a67acwh8he5yhsfy9u X-HE-Tag: 1742562891-2133 X-HE-Meta: U2FsdGVkX182bELhvzQKXRv1LqDkmh+t6MQsB6aPV+UFZoAd1DbAALoVpmbgU7pwnhs0/ikcd8HYzAPHSJQSo9qB/xPw2RCM4tEa16qJMGFKWHgHLD31vPQ9Ga116LxrkMeIYf0gsuwi5YjxAxYC1ka+4wk5oT6nEVbuOutj20QOEhUbFdJk5eUGHz9/PuJ/u15YZrfe6B1lEDsj4WH+WLi1nmzRWJajZ2LLPxa06d/U8vd9cY6UtvbiVUt+7pvHLh176wOE5JEloswowOnVqTYglRUW1O+KVbGYliY6I/1qlH9efneq3B04NEcVhdBLMtxkgFQEHHYp8GVfAxleiSEGuXrQYUYwQcgL+uUvF4WtDUAf72KaL8EJXDBxkghaykDgAyHVuDWucJR+gzbCi7kFhuFhdhZ+FOpiaNKtNbuJh5aXkNXgWf616niO0YEGEOKF76yUV5FxkG03Of7SflQqLqhX0/ihulLfYjA9lHpUnxnVVCnZ3Mja5Ow6PPFf1cCtyc4IA/BBU+/cD/EwJgAhTJv+C56kf8pGzQQ+ms6sq3f4/i/kxHtQWecd+jNxLFb4wOuXccqD7BRWNR5yN4P7sZABGEraYJbwB3KjqoE+P7vV/SK6sleM8XxqBvOYE8td+3f4iL/1v6+mD+e/LYchCERpr9dOBSZ4Iw8+3Oa1P+BormM4B2meMM1sRXHPw5wFwVJD/deQ7D6JxpOyLOb5+trWX11akcDleix5f7kytTlcqq97y2rzaBmaNi41HawIK5ui0wPEqXl7v8r+53t0Yl45EI7K/YWThLGJp+OCJRmMME2h98FygAuwxTIYvNMkgmh+VbppbbLxVQFTjjxzn91KK9rxH85AdJpMIMIBjyCbD7yqdYe4vXJGEjzAE5CeqwJAkqPRrOnlErp+tSWkl70cVmXoFT6jT4LB3H1rk2kD36SvNNKjtBx6FHPEBk945tMOL7+5bZeXYjL /3K9zAME TrX4PFHsDTpjAvPPOcmURopVi2BGnKv56YFa4bYMioYx8Su/cZDCJTAJtsh8LgqzGfPdE62ssfG99L/Cvl1knz66JudCDViHTfNHkd6vVZfugjcEq7IVsYMmuyUJgn48CWd9780rCzqAB53aNXd1QfWOLXvCCVvWxyatbC0m5Es+E3bdw7N2bsFTkbMX3i6spdorLjvYN821UtkM8h/4Ub/SRNC3xasItInZJBKVqAU2rM0g6PhfznoGAg1FJ5YSKuTzQPLPf1V2XAnHjURGsWYBAe4wFyumycde8x/6KmnRPqylkfH94Sbh/idyzT0qTuclZN5MLc9bOSESl+6TCedDrLstjEJ/BwRa8bZKoTCDBUB5nS/ZdcaqexWms/xoRm2fUKbWsYlLwMpq0/Kma0UOPehfT4nCV22DPa5gMzeO+CHfrTQUym6DFAjl7Fv4mEfdDSm0JKlIubqL3+mSpz9fNPA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: After some adjustments, both architectures have the same implementation so move it to the generic code. Signed-off-by: Alexandre Ghiti --- arch/arm64/include/asm/hugetlb.h | 3 --- arch/arm64/mm/hugetlbpage.c | 27 --------------------------- arch/riscv/include/asm/hugetlb.h | 4 ---- arch/riscv/include/asm/pgtable.h | 7 ++++--- arch/riscv/mm/hugetlbpage.c | 22 ---------------------- include/linux/hugetlb_contpte.h | 4 ++++ mm/hugetlb_contpte.c | 20 ++++++++++++++++++++ 7 files changed, 28 insertions(+), 59 deletions(-) diff --git a/arch/arm64/include/asm/hugetlb.h b/arch/arm64/include/asm/hugetlb.h index 29a9dac52cef..f568467e8ba2 100644 --- a/arch/arm64/include/asm/hugetlb.h +++ b/arch/arm64/include/asm/hugetlb.h @@ -35,9 +35,6 @@ static inline void arch_clear_hugetlb_flags(struct folio *folio) pte_t arch_make_huge_pte(pte_t entry, unsigned int shift, vm_flags_t flags); #define arch_make_huge_pte arch_make_huge_pte -#define __HAVE_ARCH_HUGE_PTEP_SET_WRPROTECT -extern void huge_ptep_set_wrprotect(struct mm_struct *mm, - unsigned long addr, pte_t *ptep); #define __HAVE_ARCH_HUGE_PTEP_CLEAR_FLUSH extern pte_t huge_ptep_clear_flush(struct vm_area_struct *vma, unsigned long addr, pte_t *ptep); diff --git a/arch/arm64/mm/hugetlbpage.c b/arch/arm64/mm/hugetlbpage.c index 03cb757f7935..17f1ed34356d 100644 --- a/arch/arm64/mm/hugetlbpage.c +++ b/arch/arm64/mm/hugetlbpage.c @@ -260,33 +260,6 @@ pte_t arch_make_huge_pte(pte_t entry, unsigned int shift, vm_flags_t flags) return entry; } -void huge_ptep_set_wrprotect(struct mm_struct *mm, - unsigned long addr, pte_t *ptep) -{ - unsigned long pfn, dpfn; - pgprot_t hugeprot; - int ncontig, i; - size_t pgsize; - pte_t pte; - - if (!pte_cont(__ptep_get(ptep))) { - __ptep_set_wrprotect(mm, addr, ptep); - return; - } - - ncontig = find_num_contig(mm, addr, ptep, &pgsize); - dpfn = pgsize >> PAGE_SHIFT; - - pte = get_clear_contig_flush(mm, addr, ptep, pgsize, ncontig); - pte = pte_wrprotect(pte); - - hugeprot = pte_pgprot(pte); - pfn = pte_pfn(pte); - - for (i = 0; i < ncontig; i++, ptep++, addr += pgsize, pfn += dpfn) - __set_ptes(mm, addr, ptep, pfn_pte(pfn, hugeprot), 1); -} - pte_t huge_ptep_clear_flush(struct vm_area_struct *vma, unsigned long addr, pte_t *ptep) { diff --git a/arch/riscv/include/asm/hugetlb.h b/arch/riscv/include/asm/hugetlb.h index bf533c2cef84..4c692dd82779 100644 --- a/arch/riscv/include/asm/hugetlb.h +++ b/arch/riscv/include/asm/hugetlb.h @@ -24,10 +24,6 @@ bool arch_hugetlb_migration_supported(struct hstate *h); pte_t huge_ptep_clear_flush(struct vm_area_struct *vma, unsigned long addr, pte_t *ptep); -#define __HAVE_ARCH_HUGE_PTEP_SET_WRPROTECT -void huge_ptep_set_wrprotect(struct mm_struct *mm, - unsigned long addr, pte_t *ptep); - pte_t arch_make_huge_pte(pte_t entry, unsigned int shift, vm_flags_t flags); #define arch_make_huge_pte arch_make_huge_pte diff --git a/arch/riscv/include/asm/pgtable.h b/arch/riscv/include/asm/pgtable.h index 081385e0d10a..c41b49948ee9 100644 --- a/arch/riscv/include/asm/pgtable.h +++ b/arch/riscv/include/asm/pgtable.h @@ -665,9 +665,8 @@ extern int __ptep_set_access_flags(struct vm_area_struct *vma, unsigned long add extern int ptep_test_and_clear_young(struct vm_area_struct *vma, unsigned long address, pte_t *ptep); -#define __HAVE_ARCH_PTEP_SET_WRPROTECT -static inline void ptep_set_wrprotect(struct mm_struct *mm, - unsigned long address, pte_t *ptep) +static inline void __ptep_set_wrprotect(struct mm_struct *mm, + unsigned long address, pte_t *ptep) { atomic_long_and(~(unsigned long)_PAGE_WRITE, (atomic_long_t *)ptep); } @@ -791,6 +790,8 @@ static inline pte_t __ptep_get_and_clear(struct mm_struct *mm, #define pte_clear __pte_clear #define __HAVE_ARCH_PTEP_SET_ACCESS_FLAGS #define ptep_set_access_flags __ptep_set_access_flags +#define __HAVE_ARCH_PTEP_SET_WRPROTECT +#define ptep_set_wrprotect __ptep_set_wrprotect #define pgprot_nx pgprot_nx static inline pgprot_t pgprot_nx(pgprot_t _prot) diff --git a/arch/riscv/mm/hugetlbpage.c b/arch/riscv/mm/hugetlbpage.c index b2046f4bd445..db13f7bcdd54 100644 --- a/arch/riscv/mm/hugetlbpage.c +++ b/arch/riscv/mm/hugetlbpage.c @@ -176,28 +176,6 @@ pte_t arch_make_huge_pte(pte_t entry, unsigned int shift, vm_flags_t flags) return entry; } -void huge_ptep_set_wrprotect(struct mm_struct *mm, - unsigned long addr, - pte_t *ptep) -{ - pte_t pte = ptep_get(ptep); - pte_t orig_pte; - int pte_num; - - if (!pte_napot(pte)) { - ptep_set_wrprotect(mm, addr, ptep); - return; - } - - pte_num = arch_contpte_get_num_contig(mm, addr, ptep, 0, NULL); - - orig_pte = get_clear_contig_flush(mm, addr, ptep, pte_num); - - orig_pte = pte_wrprotect(orig_pte); - - set_ptes(mm, addr, ptep, orig_pte, pte_num); -} - pte_t huge_ptep_clear_flush(struct vm_area_struct *vma, unsigned long addr, pte_t *ptep) diff --git a/include/linux/hugetlb_contpte.h b/include/linux/hugetlb_contpte.h index e129578f6500..9ec8792a2f4d 100644 --- a/include/linux/hugetlb_contpte.h +++ b/include/linux/hugetlb_contpte.h @@ -28,4 +28,8 @@ extern int huge_ptep_set_access_flags(struct vm_area_struct *vma, unsigned long addr, pte_t *ptep, pte_t pte, int dirty); +#define __HAVE_ARCH_HUGE_PTEP_SET_WRPROTECT +extern void huge_ptep_set_wrprotect(struct mm_struct *mm, + unsigned long addr, pte_t *ptep); + #endif /* _LINUX_HUGETLB_CONTPTE_H */ diff --git a/mm/hugetlb_contpte.c b/mm/hugetlb_contpte.c index b4c409d11195..629878765081 100644 --- a/mm/hugetlb_contpte.c +++ b/mm/hugetlb_contpte.c @@ -222,3 +222,23 @@ int huge_ptep_set_access_flags(struct vm_area_struct *vma, return 1; } + +void huge_ptep_set_wrprotect(struct mm_struct *mm, + unsigned long addr, pte_t *ptep) +{ + int ncontig; + size_t pgsize; + pte_t pte; + + if (!pte_cont(__ptep_get(ptep))) { + __ptep_set_wrprotect(mm, addr, ptep); + return; + } + + ncontig = arch_contpte_get_num_contig(mm, addr, ptep, 0, &pgsize); + + pte = get_clear_contig_flush(mm, addr, ptep, pgsize, ncontig); + pte = pte_wrprotect(pte); + + set_contptes(mm, addr, ptep, pte, ncontig, pgsize); +} From patchwork Fri Mar 21 13:06:35 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexandre Ghiti X-Patchwork-Id: 14025423 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id EA16FC36000 for ; Fri, 21 Mar 2025 13:15:59 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 5867A280007; Fri, 21 Mar 2025 09:15:57 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 50FD1280006; Fri, 21 Mar 2025 09:15:57 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 3B0EB280007; Fri, 21 Mar 2025 09:15:57 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id CF92A280006 for ; Fri, 21 Mar 2025 09:15:56 -0400 (EDT) Received: from smtpin28.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 531B0C23C9 for ; Fri, 21 Mar 2025 13:15:58 +0000 (UTC) X-FDA: 83245605996.28.7CB0D71 Received: from mail-wm1-f47.google.com (mail-wm1-f47.google.com [209.85.128.47]) by imf19.hostedemail.com (Postfix) with ESMTP id D1ED61A000B for ; Fri, 21 Mar 2025 13:15:53 +0000 (UTC) Authentication-Results: imf19.hostedemail.com; dkim=pass header.d=rivosinc-com.20230601.gappssmtp.com header.s=20230601 header.b=NMp9Jijk; dmarc=none; spf=pass (imf19.hostedemail.com: domain of alexghiti@rivosinc.com designates 209.85.128.47 as permitted sender) smtp.mailfrom=alexghiti@rivosinc.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1742562953; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=Vd9K/fWiMAnzVZgWbX0wvtztqz4Hm8uA/kr1foq/koQ=; b=yevtoGqhR+9fEtMvAJl6fEb7wYbeMGQB/w9cnBDf1ZnUJ0lsZOTTrBuZp5cja9yqtr7Jp2 1InSZf2H05gxYQkb58VYtgMEo3WB2XU0UEUyveqghVukzaxpdSgmSM31CDxoplHfUgUyvy 7hwQzYW/FLPExsPX/IeqmFOy85uQjLo= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1742562953; a=rsa-sha256; cv=none; b=LrfVVOXhXEnXqxmUyi7a+g12EFaC1t6Wa/ltlLJLFbfaJ1XW5VqG4h6+oSOcYtlh4bZVBM puKHaMNtAPJcORzK5fTD71S1fmx6UW3X+jYGYpYkPZ38o8HQxURTS/E7CrgC1DRid/bj+z O2F6b99mwzdh2gxCZCBbhO8boTkxK6A= ARC-Authentication-Results: i=1; imf19.hostedemail.com; dkim=pass header.d=rivosinc-com.20230601.gappssmtp.com header.s=20230601 header.b=NMp9Jijk; dmarc=none; spf=pass (imf19.hostedemail.com: domain of alexghiti@rivosinc.com designates 209.85.128.47 as permitted sender) smtp.mailfrom=alexghiti@rivosinc.com Received: by mail-wm1-f47.google.com with SMTP id 5b1f17b1804b1-43cf848528aso16413635e9.2 for ; Fri, 21 Mar 2025 06:15:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc-com.20230601.gappssmtp.com; s=20230601; t=1742562952; x=1743167752; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Vd9K/fWiMAnzVZgWbX0wvtztqz4Hm8uA/kr1foq/koQ=; b=NMp9JijkSFooxclHIdt0PdjE0cPPLh3jAO1o2+zIa6wzcdgzAkCB+DXkDS/D+D054F eUnk7r5ZA/lck3CcsF4UZOkx9cg+/nxC3WUHitF+5WZzCIzNpuschNhbdAExgwIRs76T 8MdqaFxAC06xl6eUDWOUAht4X+UHTwSZuptMe9Z98uwufoFwu8zAgI+4OEeYGa/2em7i ukB/dE6uXcskvE1h1fiSRzwussa96Ro65NNnw9VMME8kQ1hgrJWRRovqNA+JkURdOrul NPX0fZkGuRnI0FDkCGZ2dVcBfk98h82xLxbKh9MvWDK22j/PA/GQcAfdd0twcOMB9rqJ TNLA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1742562952; x=1743167752; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Vd9K/fWiMAnzVZgWbX0wvtztqz4Hm8uA/kr1foq/koQ=; b=pQkk3OVF/o+fnuQH1BFdxyhmIFlYHfCNvWAsPl5lOd3mOjYUS1Op4xTZgqosgdeBwh YNSrtBZ1ok78nk4xmCbUqk9YbbK1tTXpbjCANLjs2a+6nJXR5EezkmHkMlPYLN+/z9Qc kZWTlfnQkuV5HE88OVwN42cf4/UdadYUogd/+epx4P0cmJOGPullnjeIhkUChAXMEU9s HlQk7iCM2Ln3tDMe4fnA02lnsJT1w0lac3MJy5mG64NH/n3FbqprLTEQCPEhwx/LrK6d 3vjG8uF44ur80/mi/76UzlAd8KvTlMn9b8Pm26EYGVH+evyb/SvbS30/d+DHBCpGCzvv noyA== X-Forwarded-Encrypted: i=1; AJvYcCXfKt8oNdjhIJnGf4Zuvd+KkRL2TR62t3pbxKRp27m5qUpP1UDj+yCNbMfuALzpwdvVLrGebUMu+g==@kvack.org X-Gm-Message-State: AOJu0Ywpp4yhnBswR+eGlxLkYdfuyK3qGvl+EpNUG0T7B9vkd64yH3Mn I3rkxMsXX+1Qfe17g4jyNVU1ENVe8/LKBFLLvHo4+KlbVMr7JBu3+9ZLqWN+1x0= X-Gm-Gg: ASbGnctqoojVi7k/x8nN8VPE2+7ppz2CseSKq4ou9YDuKdo7VsgijiRD17VqKm3NsLZ Aq0TyogrxFRGWJDmq+mXfFvSMRHHViTsBF2t+WYlh6omUVGMPWHhG9/AUL+m8l466BT5+9ahGBC IYvq3PljyIFS1WGynAXa/W8NMoxzyf1uj6PRxBGTm95nCCZKGp8nw8Uc1/faENZMh9tD2O99ByB mHi+X3IOzz1zZ59AVxxuglS0VKZsWOyBhwwl3NQ9qhDZbjoAwC7uQJcH8kIWE5pzjruVHnKTzAy LvneTOTcig9GrvVk5Hqzzi46BN84hoCx6s+L90Eu+XO8Jk6R9HAg4nB1WMzDD9AvEeUrow== X-Google-Smtp-Source: AGHT+IEeSuJJCualMyRGxNrWV663ipohlvvxYL4TpbesKMPlj8TlM2yLM3TCv4QBVXMIE7MjteYhiA== X-Received: by 2002:a05:600c:83cf:b0:43c:ec28:d310 with SMTP id 5b1f17b1804b1-43d509ec838mr36046425e9.10.1742562951783; Fri, 21 Mar 2025 06:15:51 -0700 (PDT) Received: from alex-rivos.ba.rivosinc.com ([2001:861:3382:ef90:3d12:52fe:c1cc:c94]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-43d4fdb06b9sm26435515e9.36.2025.03.21.06.15.50 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 21 Mar 2025 06:15:51 -0700 (PDT) From: Alexandre Ghiti To: Catalin Marinas , Will Deacon , Ryan Roberts , Mark Rutland , Matthew Wilcox , Paul Walmsley , Palmer Dabbelt , Alexandre Ghiti , Andrew Morton , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org, linux-mm@kvack.org Cc: Alexandre Ghiti Subject: [PATCH v5 9/9] mm: Use common huge_ptep_clear_flush() function for riscv/arm64 Date: Fri, 21 Mar 2025 14:06:35 +0100 Message-Id: <20250321130635.227011-10-alexghiti@rivosinc.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20250321130635.227011-1-alexghiti@rivosinc.com> References: <20250321130635.227011-1-alexghiti@rivosinc.com> MIME-Version: 1.0 X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: D1ED61A000B X-Stat-Signature: ij9k8ap1qyto159zk3uyso4okk33fp8a X-Rspam-User: X-HE-Tag: 1742562953-820248 X-HE-Meta: U2FsdGVkX1/M+ThAHGU3Q6AW4RtpeHtmEo8/Cgdy2K8XGVzrAg8ZbrJzQbkuL45g+qvit56yM6M5RU9vaqTHXr7/fzx6bou51xH9pxElAGvYQVUuxKC9sYSbL990kZoAkHQQZxU1K7nwYEOgK+GgKC/qyzQTOm/ub6yHYfc3wp3ObjBA++aR/cyUuquoZMfCy26b4t00gK25t7sVobKowaWv0luuBfv83fyxFz0/nCY5wa56fnZYeuHR4HhdqK5sv6mUHIDHdOxWP0AAW7tQnExzNKJg8i2Rw0u09COjYBFlEQlIAu+P4osdkY9l6/CyQzqIaKxbSiSBdKFt7YrnD6dcWgJZb4Tnezf3/Jn/pRMBNlAZ0lJZCCrC0QBEcShEsCD4zZquCJgTEvxlfEN3EFiYiWrpZM6xLs4gTHrMFpdapcn+9YkL+lD2bBFzoK6dxqBwDoHCXaisgAh4LsyTbta6IJsHR9nIExgWMS9Q7ZJ8auqujbEdEQP7qX8miUqKh1MVaXU8q4ptGgmjFeEmdpDmZRAq3jDXFPuKf/k4kSlf/gVfja0eQEhO+rgrKQHRj/du5KBpozQDTAAA1gMihYKVG6F8CRGDzhA5wd5TjSFLa8TknmYiFMgSYz755ZN5wTRONA6KKKz0sdoKN0udu7X9cSGJ3ECb+TupGcg0j0L8RM19DhfrKedZbshf6CBaYUfgkSktZThxyfQx8Wi5tHyo2CvK8VSzwODny8nyuMVppmYCe8ZAfggQKfcmDAA2eiiT0GfpOa/+Ph8FR5FV2AQJRYAoN2MDyocZD7ErBcXWTfcc8a9qw7ALKGPZd8fwbBWHIectVCaTEBvJiOi7Ld1aumteBv1ND0k0KeLJ0mVyqUcLJU9VCM5LC5Ga4f5KDt8mod80GvAh8ABw5P4Skzpt90aW+WESxempTCxZliadZn+lXa5aKjtzNCTvWcawTw825Gv+GMqeQpmg1vd K3w31XxY H3WPG1jVcTjqClWUtdIK3PyM0BHTESxRoqH+GSNpDOeovuqw5JKWxwQ9yKKXIsdE9Uyh1GS7sYeAP/OJWaYUssTupPMtEKTnPYeJ3aiAbKDBambn9H48GKxouKPq6oOvLCZRNhSvsYx6itGQ+wHAfYRUO4DM9l0p2zWeNLDrsLFikc+NSKIjsnwN3Q1ZSk21NSd7YArHS1pQDcsiPuBGAt2HoIY2J+D1YQJEp3dvLkGn9E+9rPegQgA6QOnibbjai3AccmmUE+yXwUdillhAHkNXD5UxssvIpUuNTJfJoqZ2pZeOV29W9UIp81vKYl5OoXJ9gxQZ9iA9J/3kMvn304EKwzjxU/ipKKXYUxB/qzoqvQ51O+HsqHhGFrOmgbdDtfttr9fwECYkkXyPOkp9fIx1lDrfKodkpXzB8fU6iImQ5fiYNztOIcKj9+IuQomX0aZp2nbTTW9/IfgO8ViI0sLsRgg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: After some adjustments, both architectures have the same implementation so move it to the generic code. Signed-off-by: Alexandre Ghiti --- arch/arm64/include/asm/hugetlb.h | 3 -- arch/arm64/mm/hugetlbpage.c | 60 -------------------------------- arch/riscv/include/asm/hugetlb.h | 7 +--- arch/riscv/mm/hugetlbpage.c | 54 ---------------------------- include/linux/hugetlb_contpte.h | 4 +++ mm/hugetlb_contpte.c | 14 ++++++++ 6 files changed, 19 insertions(+), 123 deletions(-) diff --git a/arch/arm64/include/asm/hugetlb.h b/arch/arm64/include/asm/hugetlb.h index f568467e8ba2..368600764127 100644 --- a/arch/arm64/include/asm/hugetlb.h +++ b/arch/arm64/include/asm/hugetlb.h @@ -35,9 +35,6 @@ static inline void arch_clear_hugetlb_flags(struct folio *folio) pte_t arch_make_huge_pte(pte_t entry, unsigned int shift, vm_flags_t flags); #define arch_make_huge_pte arch_make_huge_pte -#define __HAVE_ARCH_HUGE_PTEP_CLEAR_FLUSH -extern pte_t huge_ptep_clear_flush(struct vm_area_struct *vma, - unsigned long addr, pte_t *ptep); void __init arm64_hugetlb_cma_reserve(void); diff --git a/arch/arm64/mm/hugetlbpage.c b/arch/arm64/mm/hugetlbpage.c index 17f1ed34356d..08316cf4b104 100644 --- a/arch/arm64/mm/hugetlbpage.c +++ b/arch/arm64/mm/hugetlbpage.c @@ -82,52 +82,6 @@ int find_num_contig(struct mm_struct *mm, unsigned long addr, return CONT_PTES; } -/* - * Changing some bits of contiguous entries requires us to follow a - * Break-Before-Make approach, breaking the whole contiguous set - * before we can change any entries. See ARM DDI 0487A.k_iss10775, - * "Misprogramming of the Contiguous bit", page D4-1762. - * - * This helper performs the break step. - */ -static pte_t get_clear_contig(struct mm_struct *mm, - unsigned long addr, - pte_t *ptep, - unsigned long pgsize, - unsigned long ncontig) -{ - pte_t pte, tmp_pte; - bool present; - - pte = __ptep_get_and_clear(mm, addr, ptep); - present = pte_present(pte); - while (--ncontig) { - ptep++; - addr += pgsize; - tmp_pte = __ptep_get_and_clear(mm, addr, ptep); - if (present) { - if (pte_dirty(tmp_pte)) - pte = pte_mkdirty(pte); - if (pte_young(tmp_pte)) - pte = pte_mkyoung(pte); - } - } - return pte; -} - -static pte_t get_clear_contig_flush(struct mm_struct *mm, - unsigned long addr, - pte_t *ptep, - unsigned long pgsize, - unsigned long ncontig) -{ - pte_t orig_pte = get_clear_contig(mm, addr, ptep, pgsize, ncontig); - struct vm_area_struct vma = TLB_FLUSH_VMA(mm, 0); - - flush_tlb_range(&vma, addr, addr + (pgsize * ncontig)); - return orig_pte; -} - pte_t *huge_pte_alloc(struct mm_struct *mm, struct vm_area_struct *vma, unsigned long addr, unsigned long sz) { @@ -260,20 +214,6 @@ pte_t arch_make_huge_pte(pte_t entry, unsigned int shift, vm_flags_t flags) return entry; } -pte_t huge_ptep_clear_flush(struct vm_area_struct *vma, - unsigned long addr, pte_t *ptep) -{ - struct mm_struct *mm = vma->vm_mm; - size_t pgsize; - int ncontig; - - if (!pte_cont(__ptep_get(ptep))) - return ptep_clear_flush(vma, addr, ptep); - - ncontig = find_num_contig(mm, addr, ptep, &pgsize); - return get_clear_contig_flush(mm, addr, ptep, pgsize, ncontig); -} - static int __init hugetlbpage_init(void) { /* diff --git a/arch/riscv/include/asm/hugetlb.h b/arch/riscv/include/asm/hugetlb.h index 4c692dd82779..63c7e4fa342a 100644 --- a/arch/riscv/include/asm/hugetlb.h +++ b/arch/riscv/include/asm/hugetlb.h @@ -20,14 +20,9 @@ bool arch_hugetlb_migration_supported(struct hstate *h); #endif #ifdef CONFIG_RISCV_ISA_SVNAPOT -#define __HAVE_ARCH_HUGE_PTEP_CLEAR_FLUSH -pte_t huge_ptep_clear_flush(struct vm_area_struct *vma, - unsigned long addr, pte_t *ptep); - pte_t arch_make_huge_pte(pte_t entry, unsigned int shift, vm_flags_t flags); #define arch_make_huge_pte arch_make_huge_pte - -#endif /*CONFIG_RISCV_ISA_SVNAPOT*/ +#endif /* CONFIG_RISCV_ISA_SVNAPOT */ #include diff --git a/arch/riscv/mm/hugetlbpage.c b/arch/riscv/mm/hugetlbpage.c index db13f7bcdd54..a6176415432a 100644 --- a/arch/riscv/mm/hugetlbpage.c +++ b/arch/riscv/mm/hugetlbpage.c @@ -121,45 +121,6 @@ unsigned long hugetlb_mask_last_page(struct hstate *h) return 0UL; } -static pte_t get_clear_contig(struct mm_struct *mm, - unsigned long addr, - pte_t *ptep, - unsigned long ncontig) -{ - pte_t pte, tmp_pte; - bool present; - - pte = ptep_get_and_clear(mm, addr, ptep); - present = pte_present(pte); - while (--ncontig) { - ptep++; - addr += PAGE_SIZE; - tmp_pte = ptep_get_and_clear(mm, addr, ptep); - if (present) { - if (pte_dirty(tmp_pte)) - pte = pte_mkdirty(pte); - if (pte_young(tmp_pte)) - pte = pte_mkyoung(pte); - } - } - return pte; -} - -static pte_t get_clear_contig_flush(struct mm_struct *mm, - unsigned long addr, - pte_t *ptep, - unsigned long pte_num) -{ - pte_t orig_pte = get_clear_contig(mm, addr, ptep, pte_num); - struct vm_area_struct vma = TLB_FLUSH_VMA(mm, 0); - bool valid = !pte_none(orig_pte); - - if (valid) - flush_tlb_range(&vma, addr, addr + (PAGE_SIZE * pte_num)); - - return orig_pte; -} - pte_t arch_make_huge_pte(pte_t entry, unsigned int shift, vm_flags_t flags) { unsigned long order; @@ -176,21 +137,6 @@ pte_t arch_make_huge_pte(pte_t entry, unsigned int shift, vm_flags_t flags) return entry; } -pte_t huge_ptep_clear_flush(struct vm_area_struct *vma, - unsigned long addr, - pte_t *ptep) -{ - pte_t pte = ptep_get(ptep); - int pte_num; - - if (!pte_napot(pte)) - return ptep_clear_flush(vma, addr, ptep); - - pte_num = arch_contpte_get_num_contig(vma->vm_mm, addr, ptep, 0, NULL); - - return get_clear_contig_flush(vma->vm_mm, addr, ptep, pte_num); -} - static bool is_napot_size(unsigned long size) { unsigned long order; diff --git a/include/linux/hugetlb_contpte.h b/include/linux/hugetlb_contpte.h index 9ec8792a2f4d..e217a3412b13 100644 --- a/include/linux/hugetlb_contpte.h +++ b/include/linux/hugetlb_contpte.h @@ -32,4 +32,8 @@ extern int huge_ptep_set_access_flags(struct vm_area_struct *vma, extern void huge_ptep_set_wrprotect(struct mm_struct *mm, unsigned long addr, pte_t *ptep); +#define __HAVE_ARCH_HUGE_PTEP_CLEAR_FLUSH +extern pte_t huge_ptep_clear_flush(struct vm_area_struct *vma, + unsigned long addr, pte_t *ptep); + #endif /* _LINUX_HUGETLB_CONTPTE_H */ diff --git a/mm/hugetlb_contpte.c b/mm/hugetlb_contpte.c index 629878765081..1dc211d6fbe1 100644 --- a/mm/hugetlb_contpte.c +++ b/mm/hugetlb_contpte.c @@ -242,3 +242,17 @@ void huge_ptep_set_wrprotect(struct mm_struct *mm, set_contptes(mm, addr, ptep, pte, ncontig, pgsize); } + +pte_t huge_ptep_clear_flush(struct vm_area_struct *vma, + unsigned long addr, pte_t *ptep) +{ + struct mm_struct *mm = vma->vm_mm; + size_t pgsize; + int ncontig; + + if (!pte_cont(__ptep_get(ptep))) + return ptep_clear_flush(vma, addr, ptep); + + ncontig = arch_contpte_get_num_contig(mm, addr, ptep, 0, &pgsize); + return get_clear_contig_flush(mm, addr, ptep, pgsize, ncontig); +}