From patchwork Fri Aug 2 15:14:22 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexandre Ghiti X-Patchwork-Id: 13751614 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id D4F08C52D6F for ; Fri, 2 Aug 2024 15:15:39 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 63FFC6B0089; Fri, 2 Aug 2024 11:15:39 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 5F01F6B008A; Fri, 2 Aug 2024 11:15:39 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 4B7D66B008C; Fri, 2 Aug 2024 11:15:39 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 2B1236B0089 for ; Fri, 2 Aug 2024 11:15:39 -0400 (EDT) Received: from smtpin07.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id CD005141306 for ; Fri, 2 Aug 2024 15:15:38 +0000 (UTC) X-FDA: 82407654756.07.E85D13C Received: from mail-wm1-f50.google.com (mail-wm1-f50.google.com [209.85.128.50]) by imf17.hostedemail.com (Postfix) with ESMTP id C9FE740016 for ; Fri, 2 Aug 2024 15:15:36 +0000 (UTC) Authentication-Results: imf17.hostedemail.com; dkim=pass header.d=rivosinc-com.20230601.gappssmtp.com header.s=20230601 header.b=uCkr1dcM; dmarc=none; spf=pass (imf17.hostedemail.com: domain of alexghiti@rivosinc.com designates 209.85.128.50 as permitted sender) smtp.mailfrom=alexghiti@rivosinc.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1722611693; a=rsa-sha256; cv=none; b=8NH/megg646Mh/0ycL2RuQbFLFN68GoogzrV3bvq6ReRk+8DG0yTZTIcASi72idE9DFpN9 CQGVq00O8Q6JvVch1puVB7ykGpSfIvmHFdrddfGy77SqXh4mHj6wy7uJrOlVo80mSa6pLi k8LK9YYDed9p0lA7jw2wtw6ROJ73sOw= ARC-Authentication-Results: i=1; imf17.hostedemail.com; dkim=pass header.d=rivosinc-com.20230601.gappssmtp.com header.s=20230601 header.b=uCkr1dcM; dmarc=none; spf=pass (imf17.hostedemail.com: domain of alexghiti@rivosinc.com designates 209.85.128.50 as permitted sender) smtp.mailfrom=alexghiti@rivosinc.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1722611693; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=YlgUGAmuTWGVKaOGr6DQU8bhY7gyR7CYcPHPJxmYPTE=; b=JdFvc0qE1lIsKmK9w/C85Q+BUiFcNXw3/7AQoGB1GAC+Q7KzX9e7j+BvMPKWMhRq21nh/f DTlkFnJdNtu9SWpN5yp0G2BpsrBye4l8yMMLd5OeCHIqcnm74A0MErsDqU+RPQWydaRxGE hypd9oV7sn6ziETfq18KnVHh49swSJ8= Received: by mail-wm1-f50.google.com with SMTP id 5b1f17b1804b1-42803bbf842so73475835e9.1 for ; Fri, 02 Aug 2024 08:15:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc-com.20230601.gappssmtp.com; s=20230601; t=1722611735; x=1723216535; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=YlgUGAmuTWGVKaOGr6DQU8bhY7gyR7CYcPHPJxmYPTE=; b=uCkr1dcMEfeDI6vBZjjV2YJSbxGI9I+Ve0vrWLwb+tAyfAxD/KYkNFYVjjmr963elk ORWpFgrkwr9CXp4s2Jsnk22MdpWRrx34o+hJKgYNCSZaXcxdwBX6+DEniRFJ1br1FT7+ G+Yf31F8div+TnXL9LjRkssvdriplvWnLewi14/aWmIeqcsOP05ztpFb48kHAUfShmat gBJ7pXSojn3WuEMnqpXZaP7knuJgLe27CI9lQUM+ERf20xeC09KywypzcNbTLagD9ze0 ZjJRzRfAOvRxpG/5O4fniXf9ndZzIAL7/lBmUGhpAOcRqaO/ovmTtYtBwgzuYSwCtrGS FNWQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1722611735; x=1723216535; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=YlgUGAmuTWGVKaOGr6DQU8bhY7gyR7CYcPHPJxmYPTE=; b=rPA1SMbbp9DVWp5aYGkVLrcilh7IKMECrN+x1pMv2yfEEri8fwelhWvujSC40u+Bm/ srR3ugv65KIpWMAmCU/0JNU0G3NrXwTsTU3X3ZTDwF70aaJC2fOmv/Ueb9X5Xcp+++Ui dLVyLgkcCubH5WZWsvs2gp1Nipe+L/TIRvsORjV3s02Rlc4M8wJDMvCNStWNCWnJ4tbK EALob/j9pFtLLjCSRn9/xchdVEtlrIIZCxp738qWAzEZx0W9sU21yTIqaFU10OT5PjVi HjyN8P0gfA1BUQC3tDUL/X4okmetZMfRWUEUNJV3beXvrz9fPu8en/PuBOdWC3ivXboz DNdQ== X-Forwarded-Encrypted: i=1; AJvYcCXyN9wqI+BSb1qn3yRaHfDF+Wieuk0k2fCz62IEwSRYKCU7JfmJcsvBTp8omfaRzrP+/nTJCr3BLnHM64bal8/RpMo= X-Gm-Message-State: AOJu0YwLBGNc5fcR3PkC4X1S+xYqbGfVkaie1LxPV1dsnJtT4JtkcBrv P0+l0fPlBgcocwHZK+re3mvVDqanYNw9Pdq00gYznhym9IQK0vQ/8CBmHN4MQEXNO25a1b92vhh 7 X-Google-Smtp-Source: AGHT+IH4Zu+tW3v1CbjrzMX8UC2ElJa1MqCyLtpkk5dkYfGnqCXlVjQZLgHhH8VhTadf1xkkn6UFcw== X-Received: by 2002:a05:600c:4713:b0:426:593c:9359 with SMTP id 5b1f17b1804b1-428e6b954cfmr36381225e9.32.1722611734897; Fri, 02 Aug 2024 08:15:34 -0700 (PDT) Received: from alex-rivos.ba.rivosinc.com (amontpellier-656-1-456-62.w92-145.abo.wanadoo.fr. [92.145.124.62]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-36bbd06d078sm2168897f8f.94.2024.08.02.08.15.33 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 02 Aug 2024 08:15:34 -0700 (PDT) From: Alexandre Ghiti To: Catalin Marinas , Will Deacon , Ryan Roberts , Mark Rutland , Paul Walmsley , Palmer Dabbelt , Albert Ou , Andrew Morton , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org, linux-mm@kvack.org Cc: Alexandre Ghiti Subject: [PATCH v3 1/9] riscv: Safely remove huge_pte_offset() when manipulating NAPOT ptes Date: Fri, 2 Aug 2024 17:14:22 +0200 Message-Id: <20240802151430.99114-2-alexghiti@rivosinc.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20240802151430.99114-1-alexghiti@rivosinc.com> References: <20240802151430.99114-1-alexghiti@rivosinc.com> MIME-Version: 1.0 X-Rspamd-Queue-Id: C9FE740016 X-Rspam-User: X-Rspamd-Server: rspam05 X-Stat-Signature: 1gro8zg3k3bamncsx7sxpi9oeahy5g5r X-HE-Tag: 1722611736-712412 X-HE-Meta: U2FsdGVkX197PEfo+c4i4vOKFg06/fjfs42Wh2ieemKHh1EcOpg9BNiJT/5BwjQUHkiQjjHGwsNlFdzCT7fgLAEsg7A5HzTsmH+j4p2TxOet0+HToTsC/AdNzLtZf3fBhnyzuozz4XR0KQAlfs57TgKq/xRMi9xPMJ6pbT484uG/1rFpWEfx9ESVGv9xD8A6+JeN7PCRLNjJ1vE+z9anMuBqqLkVkRXgsZG8A+TR2Ta36uosYX+ErkWGtv9B3xJxdqhwYmw5HhFi2DSi9ZCD2bbHemP96zapdGSpT2s6hFAYMZZEwONe1wAqzYOUSKbF1gLb9URpljAEzuaUFGZi8m68PehJMr4imxcj7iXxhPDiVYv3FS7SR7QXFMWJkzmzEt4PBXuscx+bRH7/jzBiTxoS6PSzt2P0m9Q6wvJWYsPYWGsMC2GLRH2y92YAeTJAoeXc1qIqEKh5jfbiXc3VYx+Zd/MkfvRURnW/WZ3kjyw809ooZDbHJI1+0r7owmKNAtQEuaXQNx3RzUpPetggIZvOPSnOwsCeyGG4wefX55v1HGFK68riTyySeHnPN8LQiK1eotaZKvPVGLawuT0IuSy0nfYJ1wXHxiR9mzhvOsvD+ykg6GygTRVrzGBA+fcSaraSl3kZEopO1FLKFfPZGeOG+izh/HDlwj9eGY58jOs9nvvtkKMMZM/8+2uOFfi6ioKds341VahJPKHQuWKu/6VlJImiykNsMv28VzX3MwwLvq/sIewXGB3bmbQIuryIf5kAgBn3BaOKqh3hokHeAto+HCXAvKc8gKE4eoHQYfz5HmkHR1eqkRoQYDA0NGj8Nw0x64sBfQzC0qJkuOhLycB/l0t/cW4NCV3v7dy8hZ5F1m6gNWEYss70LYaFHPgDHe0HphZxLuA9zpCvI0XWZ+7l8U8pSUhdm2mWws4ie3CUi8eiJAvfw34TZVc9CFAfChhV1SwlZS8kCtWmMuW zme0Z8a1 +0V7rOprYpO6KBfnez0llifc6nJzEBOvzHRKYdtRPN8jf6ME+p2Oq671kDXxoO2tDzbgDpQFVhNK0pCINDyKNeFKE8KJwQYDtFVnE3IQxrj/S/cL+W5pZiz2Us7jYczM2SMLc//cbQQleZ4csMPmx8YvH0NjpuH/ka3LVriP+dysMONs92bwxWOaIAXriPUDuqfjhXV4fNkmbewasFh+Enmlti/bRmVf4PgU51muU01Y9brPMqgPwvzGxFyL2emTqpDTaqgY8G8P9Ey/0848frGBVKokge5fi8XgjsYwMVklc9uSjKS2auadUbmk2IHt6EuoWWdixKQJta3p0woSTvlhYJ/42i0ZSFj9JqEOYi3xaS/OdzV3I3kCTbzkqpF7iImD98jFZVBWprrE/YCbhihY7+w== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: The pte_t pointer is expected to point to the first entry of the NAPOT mapping so no need to use huge_pte_offset(), similarly to what is done in arm64. Signed-off-by: Alexandre Ghiti --- arch/riscv/mm/hugetlbpage.c | 2 -- 1 file changed, 2 deletions(-) diff --git a/arch/riscv/mm/hugetlbpage.c b/arch/riscv/mm/hugetlbpage.c index 42314f093922..6b09cd1ef41c 100644 --- a/arch/riscv/mm/hugetlbpage.c +++ b/arch/riscv/mm/hugetlbpage.c @@ -276,7 +276,6 @@ int huge_ptep_set_access_flags(struct vm_area_struct *vma, order = napot_cont_order(pte); pte_num = napot_pte_num(order); - ptep = huge_pte_offset(mm, addr, napot_cont_size(order)); orig_pte = get_clear_contig_flush(mm, addr, ptep, pte_num); if (pte_dirty(orig_pte)) @@ -322,7 +321,6 @@ void huge_ptep_set_wrprotect(struct mm_struct *mm, order = napot_cont_order(pte); pte_num = napot_pte_num(order); - ptep = huge_pte_offset(mm, addr, napot_cont_size(order)); orig_pte = get_clear_contig_flush(mm, addr, ptep, pte_num); orig_pte = pte_wrprotect(orig_pte); From patchwork Fri Aug 2 15:14:23 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexandre Ghiti X-Patchwork-Id: 13751624 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2FFABC3DA4A for ; Fri, 2 Aug 2024 15:16:41 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id BE5AD6B008C; Fri, 2 Aug 2024 11:16:40 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id B95F26B0092; Fri, 2 Aug 2024 11:16:40 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A38D06B0093; Fri, 2 Aug 2024 11:16:40 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 84FE36B008C for ; Fri, 2 Aug 2024 11:16:40 -0400 (EDT) Received: from smtpin24.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id B12CA41293 for ; Fri, 2 Aug 2024 15:16:39 +0000 (UTC) X-FDA: 82407657318.24.A3785EE Received: from mail-wr1-f47.google.com (mail-wr1-f47.google.com [209.85.221.47]) by imf23.hostedemail.com (Postfix) with ESMTP id C71BE140036 for ; Fri, 2 Aug 2024 15:16:37 +0000 (UTC) Authentication-Results: imf23.hostedemail.com; dkim=pass header.d=rivosinc-com.20230601.gappssmtp.com header.s=20230601 header.b=YcdAaH3B; dmarc=none; spf=pass (imf23.hostedemail.com: domain of alexghiti@rivosinc.com designates 209.85.221.47 as permitted sender) smtp.mailfrom=alexghiti@rivosinc.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1722611750; a=rsa-sha256; cv=none; b=1N073jh0VMmp1kTKAHckXqkfHvoRjN/DCaYSG6ixgraitHzDfj860pp8dnfhli6hPSGAm2 y/qjgFkK2n1c7TFq6nZ/IS6LmCUXXWWcnZFYVGEy0fuMa0Rzv4zFA/zhIHRNx+jRwqh7RH ePCIqo+jl+fpulVA+NkSNN2FG678Ld0= ARC-Authentication-Results: i=1; imf23.hostedemail.com; dkim=pass header.d=rivosinc-com.20230601.gappssmtp.com header.s=20230601 header.b=YcdAaH3B; dmarc=none; spf=pass (imf23.hostedemail.com: domain of alexghiti@rivosinc.com designates 209.85.221.47 as permitted sender) smtp.mailfrom=alexghiti@rivosinc.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1722611750; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=5juqdFHx6bq1L8uGHC8KAcytqmq0yTnlf857QDP6MLo=; b=ZOVy16IJa1Pw8G+mC9oK1QyLRLWYM/9aOUqjh6W8JKXr3GCHIKsKHN8lXANNW2l8Erm1/W kA6kfWcEy/rLBWW0URHFUOHv/+36vePXQVvllV6s5MggY8DUcb7JrXLLw9dWGUrSY7FMeS toRJ54IwWhXGZKa2P49Jpegc6sV6EQg= Received: by mail-wr1-f47.google.com with SMTP id ffacd0b85a97d-3684bea9728so4752805f8f.3 for ; Fri, 02 Aug 2024 08:16:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc-com.20230601.gappssmtp.com; s=20230601; t=1722611796; x=1723216596; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=5juqdFHx6bq1L8uGHC8KAcytqmq0yTnlf857QDP6MLo=; b=YcdAaH3BFN4Dxn3ztO6R31CP8uJ7MEIfIkkyZfYQtkQEg/kOq3Lu3pt4GLYEdo4YRe IgtUKVo7fiv6sFHQZDnXArxq5S4xBurV/n+D8ZuY4RVnhU98UNTbv2p63qj+dxvgLq5J pctiP5isb2e6lyzgb8gJkb53kDqrlUyBDZv0P/l8po7tM7NbgsYkDXlKwfpLTeUouC1X ne92eN6T279hv/6oFE6LjPyg8faxKPf9zzKMZI6o7Ldapsiq0Kp/BrtkIlBxoeO+VJwE zH8A8oJyq16nhixLsCJAJuuW346rti517Oi7oiOKtbdzE/gw+905a5RI3Qw6zg8ogpYR ApbQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1722611796; x=1723216596; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=5juqdFHx6bq1L8uGHC8KAcytqmq0yTnlf857QDP6MLo=; b=LJUwv+gOkwDE4kVdj8vZJk0RGRbruuKfOzlJ8vk60pLpkXhl+vLmLY3VqH2Bm/R5j/ SNR3cZoE2VVv+Gy/spuK7O3pt4q7M8X8LBQ9LhbRXilrwmgxgCpIcIhVIEQWjksbgA+A Sf9uzbLAQMo9Jofq4AVqKvnZ4590OSqDaKJ2t+iKBCm7vK/6v5/kWtSzUtc5BcRiuLky +CuW6mO/bqm1i9AFRp2TO9ez7BuYuUiOlTgcbS1X3RTUr7uGPx/k97lpyxBq9n5ZxCyV UzBHbNq1XQtWwBbIrXGjzKVMhMtREgqn55jGuIU9ZVdLDjWX9ssfbmkTdbT5RqV6iTSx 8bPw== X-Forwarded-Encrypted: i=1; AJvYcCUsF2t2CmTC9RnBFJaQAKsG88Cpe1xrBPKDKrlGhFqPhW4RvaXyKebt4XAzqlHeLxXYiSCqposOg/BK8Do33kLI6mM= X-Gm-Message-State: AOJu0YyfdqBzghJx+z4rkv8tfshjaLVYZl40s4icAqDSpDA2LdXDNMZP Gqt8+ufreqtE1DbXxoNP4cms/4mrIAVlVQEHiytjOyxAB2f8CkAROSwks1jiiJk= X-Google-Smtp-Source: AGHT+IEF5tp7ncv5iXPHoA3R3svUCtTjvmewfdQGfXhbRlwySugbqyJNH05NnKLNTKXkwTci1pBy4Q== X-Received: by 2002:adf:f80c:0:b0:368:12ef:92d8 with SMTP id ffacd0b85a97d-36bbc17ec98mr2408572f8f.52.1722611796133; Fri, 02 Aug 2024 08:16:36 -0700 (PDT) Received: from alex-rivos.ba.rivosinc.com (amontpellier-656-1-456-62.w92-145.abo.wanadoo.fr. [92.145.124.62]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-36bbcf0cc9csm2206159f8f.17.2024.08.02.08.16.35 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 02 Aug 2024 08:16:35 -0700 (PDT) From: Alexandre Ghiti To: Catalin Marinas , Will Deacon , Ryan Roberts , Mark Rutland , Paul Walmsley , Palmer Dabbelt , Albert Ou , Andrew Morton , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org, linux-mm@kvack.org Cc: Alexandre Ghiti Subject: [PATCH v3 2/9] riscv: Restore the pfn in a NAPOT pte when manipulated by core mm code Date: Fri, 2 Aug 2024 17:14:23 +0200 Message-Id: <20240802151430.99114-3-alexghiti@rivosinc.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20240802151430.99114-1-alexghiti@rivosinc.com> References: <20240802151430.99114-1-alexghiti@rivosinc.com> MIME-Version: 1.0 X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: C71BE140036 X-Stat-Signature: 53gag4trnqgrfmujxdbqfi6r75itc4sa X-Rspam-User: X-HE-Tag: 1722611797-770193 X-HE-Meta: U2FsdGVkX1/QeS1cwkpswTd0Xy/XgJnaHZhtdr5NYmPy7SEC3biR4PLPZynubC4QWKz+jGpe/ArDwRpRC4V/WSD+dsAksJJk3TS7YqJ9uPYif2E/X676xQ8NOjU7oZxjXWKJuySl1Ax8yqDj5T5ym/uuSqavB9lFh0805fUmxF2A6PGSI/qZ50F00lGqlFVgr3WzF/+vujY8zB7KnaZIWFmwAEMpidiO+IJw6vkWNdnDmHDC2bJJhSg+jGgm6m01DeMtgX9NKGDeotKtkh5GcO5KaDbWoc1bO3y3XJqWHykXPz3gqbZdwS26Ekdwqq4lA0vIErxqr1Y6S+qmy3kT18fAKf0MuhL1fUkKgefvbF8medEV72zniCgoS1afSsZJ8yRlAcSBhw3y1qWDyokMOYtMpU0a+Cd5cZTFWwFaUK+bBmdcZB52Nks7YBZrNm/JHcgWxDtsZBGkDteIPeGCMnPmW3btV7Jl16BQnKhCM+H5wKOMzlr+C9E5HIoA0fElr1bHl8cfi3FCzUce3iGMzlJBb7WDrmVeCpeK2yk06CWZcbNLoCDFM2xuhO75vaL4r+f+2WOE6dtJPWjmFShR4Bwr4yNJPCvRpn6bTP+dW/tiTmYhsIJvYGd4T7nNajyqTxwQ8F9C3jEkKKQ8eTiBLsG9fqMc8e+PYgapDbP0Ufv88vT82CviLnIlO+RHRnTQD1yhoHgI2jV5kdFoT1fog8ZNoEyDqbapjI8LdZIoSnjgevcG+fAK5noQzqvLENoOXQhRXN2Fz70aoQCFHoLgtUWhRO0U4i8WVERLe8KSp3pTHF2Xa1eBAFuZmhOwUsxtEMtceks7I9O6ipD0WVxNVwZMxmM4yVMoQF/3id8Beb4Cgobyyq2QrC56RiePTbsg3SkhTKVhpMWPn3KSthe08wc0HUtX6L7UixiPXcz4D6yfBC9sAcr9Pf5WENbdkZAR721pDivGYGUZhrQ/7y3 Hi1iPfZl zc5Cq1Eo0e1DWTRpXdLiV/JV6b3tjuLbHEwXNgS9bItvk7OJisLxnrQlNkfhQBimG/J14nOHNnYneqnwA15+lw46F39qTiJcsUAnBZUnoPIJSMobGtFrspdzFMEKfmjGaNXtMVBd4whL9wg4oThC7koyU7GG9+fkP5iPx/zx31QNTC4XHfu5E9so70G3CSbiDIb6jfw7ApyEW4iBLHvqk/ofsXVm5hKMsvpERWEwr7e+zXnSw38CGG4bOhH6GuyuNJnYzSJ1/SxsSB92ZCLopwN0sOL3mgygGTaPM34dQjCkotF87AMV24EdPa36mms0fdzvvYJkdWSf68A/PjERYgOh+DpkB/uSTEbKj X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: The core mm code expects to be able to extract the pfn from a pte. NAPOT mappings work differently since its ptes actually point to the first pfn of the mapping, the other bits being used to encode the size of the mapping. So modify ptep_get() so that it returns a pte value that contains the *real* pfn (which is then different from what the HW expects) and right before storing the ptes to the page table, reset the pfn LSBs to the size of the mapping. And make sure that all NAPOT mappings are set using set_ptes(). Signed-off-by: Alexandre Ghiti --- arch/riscv/include/asm/pgtable-64.h | 11 ++++ arch/riscv/include/asm/pgtable.h | 91 ++++++++++++++++++++++++++--- arch/riscv/mm/hugetlbpage.c | 9 +-- 3 files changed, 96 insertions(+), 15 deletions(-) diff --git a/arch/riscv/include/asm/pgtable-64.h b/arch/riscv/include/asm/pgtable-64.h index 0897dd99ab8d..cddbe426f618 100644 --- a/arch/riscv/include/asm/pgtable-64.h +++ b/arch/riscv/include/asm/pgtable-64.h @@ -104,6 +104,17 @@ enum napot_cont_order { #define napot_cont_mask(order) (~(napot_cont_size(order) - 1UL)) #define napot_pte_num(order) BIT(order) +static inline bool is_napot_order(unsigned int order) +{ + unsigned int napot_order; + + for_each_napot_order(napot_order) + if (order == napot_order) + return true; + + return false; +} + #ifdef CONFIG_RISCV_ISA_SVNAPOT #define HUGE_MAX_HSTATE (2 + (NAPOT_ORDER_MAX - NAPOT_CONT_ORDER_BASE)) #else diff --git a/arch/riscv/include/asm/pgtable.h b/arch/riscv/include/asm/pgtable.h index 089f3c9f56a3..34c4c360d4ce 100644 --- a/arch/riscv/include/asm/pgtable.h +++ b/arch/riscv/include/asm/pgtable.h @@ -300,6 +300,8 @@ static inline unsigned long pte_napot(pte_t pte) return pte_val(pte) & _PAGE_NAPOT; } +#define pte_valid_napot(pte) (pte_present(pte) && pte_napot(pte)) + static inline pte_t pte_mknapot(pte_t pte, unsigned int order) { int pos = order - 1 + _PAGE_PFN_SHIFT; @@ -309,6 +311,12 @@ static inline pte_t pte_mknapot(pte_t pte, unsigned int order) return __pte((pte_val(pte) & napot_mask) | napot_bit | _PAGE_NAPOT); } +/* pte at entry must *not* encode the mapping size in the pfn LSBs. */ +static inline pte_t pte_clear_napot(pte_t pte) +{ + return __pte(pte_val(pte) & ~_PAGE_NAPOT); +} + #else static __always_inline bool has_svnapot(void) { return false; } @@ -318,17 +326,14 @@ static inline unsigned long pte_napot(pte_t pte) return 0; } +#define pte_valid_napot(pte) false + #endif /* CONFIG_RISCV_ISA_SVNAPOT */ /* Yields the page frame number (PFN) of a page table entry */ static inline unsigned long pte_pfn(pte_t pte) { - unsigned long res = __page_val_to_pfn(pte_val(pte)); - - if (has_svnapot() && pte_napot(pte)) - res = res & (res - 1UL); - - return res; + return __page_val_to_pfn(pte_val(pte)); } #define pte_page(x) pfn_to_page(pte_pfn(x)) @@ -553,8 +558,13 @@ static inline void __set_pte_at(struct mm_struct *mm, pte_t *ptep, pte_t pteval) #define PFN_PTE_SHIFT _PAGE_PFN_SHIFT -static inline void set_ptes(struct mm_struct *mm, unsigned long addr, - pte_t *ptep, pte_t pteval, unsigned int nr) +static inline pte_t __ptep_get(pte_t *ptep) +{ + return READ_ONCE(*ptep); +} + +static inline void __set_ptes(struct mm_struct *mm, unsigned long addr, + pte_t *ptep, pte_t pteval, unsigned int nr) { page_table_check_ptes_set(mm, ptep, pteval, nr); @@ -563,10 +573,13 @@ static inline void set_ptes(struct mm_struct *mm, unsigned long addr, if (--nr == 0) break; ptep++; + + if (unlikely(pte_valid_napot(pteval))) + continue; + pte_val(pteval) += 1 << _PAGE_PFN_SHIFT; } } -#define set_ptes set_ptes static inline void pte_clear(struct mm_struct *mm, unsigned long addr, pte_t *ptep) @@ -621,6 +634,66 @@ static inline int ptep_clear_flush_young(struct vm_area_struct *vma, return ptep_test_and_clear_young(vma, address, ptep); } +#ifdef CONFIG_RISCV_ISA_SVNAPOT +static inline void set_ptes(struct mm_struct *mm, unsigned long addr, + pte_t *ptep, pte_t pteval, unsigned int nr) +{ + if (unlikely(pte_valid_napot(pteval))) { + unsigned int order = ilog2(nr); + + if (!is_napot_order(order)) { + /* + * Something's weird, we are given a NAPOT pte but the + * size of the mapping is not a known NAPOT mapping + * size, so clear the NAPOT bit and map this without + * NAPOT support: core mm only manipulates pte with the + * real pfn so we know the pte is valid without the N + * bit. + */ + pr_err("Incorrect NAPOT mapping, resetting.\n"); + pteval = pte_clear_napot(pteval); + } else { + /* + * NAPOT ptes that arrive here only have the N bit set + * and their pfn does not contain the mapping size, so + * set that here. + */ + pteval = pte_mknapot(pteval, order); + } + } + + __set_ptes(mm, addr, ptep, pteval, nr); +} +#define set_ptes set_ptes + +static inline pte_t ptep_get(pte_t *ptep) +{ + pte_t pte = __ptep_get(ptep); + + /* + * The pte we load has the N bit set and the size of the mapping in + * the pfn LSBs: keep the N bit and replace the mapping size with + * the *real* pfn since the core mm code expects to find it there. + * The mapping size will be reset just before being written to the + * page table in set_ptes(). + */ + if (unlikely(pte_valid_napot(pte))) { + unsigned int order = napot_cont_order(pte); + int pos = order - 1 + _PAGE_PFN_SHIFT; + unsigned long napot_mask = ~GENMASK(pos, _PAGE_PFN_SHIFT); + pte_t *orig_ptep = PTR_ALIGN_DOWN(ptep, sizeof(*ptep) * napot_pte_num(order)); + + pte = __pte((pte_val(pte) & napot_mask) + ((ptep - orig_ptep) << _PAGE_PFN_SHIFT)); + } + + return pte; +} +#define ptep_get ptep_get +#else +#define set_ptes __set_ptes +#define ptep_get __ptep_get +#endif /* CONFIG_RISCV_ISA_SVNAPOT */ + #define pgprot_nx pgprot_nx static inline pgprot_t pgprot_nx(pgprot_t _prot) { diff --git a/arch/riscv/mm/hugetlbpage.c b/arch/riscv/mm/hugetlbpage.c index 6b09cd1ef41c..59ed26ce6857 100644 --- a/arch/riscv/mm/hugetlbpage.c +++ b/arch/riscv/mm/hugetlbpage.c @@ -256,8 +256,7 @@ void set_huge_pte_at(struct mm_struct *mm, clear_flush(mm, addr, ptep, pgsize, pte_num); - for (i = 0; i < pte_num; i++, ptep++, addr += pgsize) - set_pte_at(mm, addr, ptep, pte); + set_ptes(mm, addr, ptep, pte, pte_num); } int huge_ptep_set_access_flags(struct vm_area_struct *vma, @@ -284,8 +283,7 @@ int huge_ptep_set_access_flags(struct vm_area_struct *vma, if (pte_young(orig_pte)) pte = pte_mkyoung(pte); - for (i = 0; i < pte_num; i++, addr += PAGE_SIZE, ptep++) - set_pte_at(mm, addr, ptep, pte); + set_ptes(mm, addr, ptep, pte, pte_num); return true; } @@ -325,8 +323,7 @@ void huge_ptep_set_wrprotect(struct mm_struct *mm, orig_pte = pte_wrprotect(orig_pte); - for (i = 0; i < pte_num; i++, addr += PAGE_SIZE, ptep++) - set_pte_at(mm, addr, ptep, orig_pte); + set_ptes(mm, addr, ptep, orig_pte, pte_num); } pte_t huge_ptep_clear_flush(struct vm_area_struct *vma, From patchwork Fri Aug 2 15:14:24 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexandre Ghiti X-Patchwork-Id: 13751625 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 17923C52D6F for ; Fri, 2 Aug 2024 15:17:42 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id A12116B0093; Fri, 2 Aug 2024 11:17:41 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 9C1E46B0095; Fri, 2 Aug 2024 11:17:41 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 862E46B0096; Fri, 2 Aug 2024 11:17:41 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 624266B0093 for ; Fri, 2 Aug 2024 11:17:41 -0400 (EDT) Received: from smtpin25.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 1A36A1A0B03 for ; Fri, 2 Aug 2024 15:17:41 +0000 (UTC) X-FDA: 82407659922.25.E475C31 Received: from mail-wm1-f48.google.com (mail-wm1-f48.google.com [209.85.128.48]) by imf28.hostedemail.com (Postfix) with ESMTP id 0B779C0015 for ; Fri, 2 Aug 2024 15:17:38 +0000 (UTC) Authentication-Results: imf28.hostedemail.com; dkim=pass header.d=rivosinc-com.20230601.gappssmtp.com header.s=20230601 header.b=mqo3sJ6L; spf=pass (imf28.hostedemail.com: domain of alexghiti@rivosinc.com designates 209.85.128.48 as permitted sender) smtp.mailfrom=alexghiti@rivosinc.com; dmarc=none ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1722611829; a=rsa-sha256; cv=none; b=s+6VLif7R8B+V72Y9339DIrYPpPxU9ETnSRgd/8N6ugPJX/lGHA02OjCM04mfEK8gcxxo2 2p5U6NeuElAxfnA97782HYOn9i7dgyffq4f+n5eALL27gJmYtincZAEOCimiGi6RQCFdgR InY0fqtUog0IY+OkWH+I8US+WB5ux8c= ARC-Authentication-Results: i=1; imf28.hostedemail.com; dkim=pass header.d=rivosinc-com.20230601.gappssmtp.com header.s=20230601 header.b=mqo3sJ6L; spf=pass (imf28.hostedemail.com: domain of alexghiti@rivosinc.com designates 209.85.128.48 as permitted sender) smtp.mailfrom=alexghiti@rivosinc.com; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1722611829; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=BBx7bwwM+VNLmAljCjIgeC8HmfBD2FAmTGPMtsL/Kbc=; b=kWfhzL0IwZuHlJGvtXqpVtCHvNHQurVA7yaBSZ0tpmpUf2ry/68P5prR3CWvpmsHgMy7TT Rr0z5ZO89sGUVmQQqPIcSXZ+u1zkTlHJeQVjD4xegxhjXEWRBu8scVgYNIjNjbGvUo6KVK 0ejOyQtMPu04RWF+mNaUZIHJdJwRJQM= Received: by mail-wm1-f48.google.com with SMTP id 5b1f17b1804b1-4280ee5f1e3so55467215e9.0 for ; Fri, 02 Aug 2024 08:17:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc-com.20230601.gappssmtp.com; s=20230601; t=1722611858; x=1723216658; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=BBx7bwwM+VNLmAljCjIgeC8HmfBD2FAmTGPMtsL/Kbc=; b=mqo3sJ6LTQQwwa4XtTSizTM+mC4DKT32WZ/nmEoGqqJ3E+YMiQKNmki7TILAamPNZl 6BiT8qAe6Wbai56TrFQXxWTneidFOXhjRNWlPqpgZnNG8HdUZZRkNc6kS9HPX75bNQ5u MmGpN3JQaqgThQIJZsrX2IZLVJd9qnEh4jhs0cBmQfgsDdBcN9edXdDkq16Yf4SxA9Xf UhNJMexg1CD2K4y5q3OzHnA9AKphw0hbAVKESqoSd9EbC4oG8EaDSqx+HHN2MMDDAXq6 C6vcnx+i3Ai/AHWqlAZgRy4wIuNdEIBgZ7a/g6Stfj7QlckD0RElJKpLVCGRLRFsLIio v4uw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1722611858; x=1723216658; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=BBx7bwwM+VNLmAljCjIgeC8HmfBD2FAmTGPMtsL/Kbc=; b=j2DBDu3iUQQIBcygfOgwzFN5cBQ9qEomqWoc5vz5Ggq/fdcA4Zt4Q3DWYdEdYbESj7 iN9dd3ILF1AH2jVT1ltkCb/f5DVJp/GJ6mIO+8vgLrEpdiS0Wd+ZHGNk10dc3xStDCvi UhilSfEWlKuzpEd1HkHhs95Vr28ZrJVcGa3ZVzs/FxCg5jcMg+7J3At/Ad8BGjY/SDdQ njQfS8+iFjR3h6QkcqF/leSoY7AqtExCIMbNOeCDNgr7CiBhmwIuHAcmGqg/4jK5y2SL hPuTWFtZPcSVPgKS+00A1LRWp8TsiJIDfBmD6IHIsKdzgxvnvSABTBezbGApK6VHTQkx KSAg== X-Forwarded-Encrypted: i=1; AJvYcCXwwqA4Us1ioXRhLQEY3knY23GvA54xPEHSHTHnkmfH5NLIy/aOfaBuQyU1VAH1a9s1CtUUvBP5fuNQa5XrkaH9y5U= X-Gm-Message-State: AOJu0YzRsNtoFLOkTGMVzEMwhrTgigydKDUSUp7oouo5cvJszsj01dWr q3dWX5u8t4ggLPvBPdWvbqAvkSJM9NRB4Six//Qnh+uKNLy5Snnubu7/8YgNGts= X-Google-Smtp-Source: AGHT+IHprdXLf73S6/uvgO6HKNUd/CeSHOZgyKoYo8w2u87HFzJ7dB7zNCokVnrDRI5tmninu2XIMA== X-Received: by 2002:a05:600c:45cd:b0:426:66e9:b844 with SMTP id 5b1f17b1804b1-428e6aeb0b3mr30207865e9.8.1722611857423; Fri, 02 Aug 2024 08:17:37 -0700 (PDT) Received: from alex-rivos.ba.rivosinc.com (amontpellier-656-1-456-62.w92-145.abo.wanadoo.fr. [92.145.124.62]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-428e6d6b8d9sm36362165e9.7.2024.08.02.08.17.36 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 02 Aug 2024 08:17:37 -0700 (PDT) From: Alexandre Ghiti To: Catalin Marinas , Will Deacon , Ryan Roberts , Mark Rutland , Paul Walmsley , Palmer Dabbelt , Albert Ou , Andrew Morton , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org, linux-mm@kvack.org Cc: Alexandre Ghiti Subject: [PATCH v3 3/9] mm: Use common huge_ptep_get() function for riscv/arm64 Date: Fri, 2 Aug 2024 17:14:24 +0200 Message-Id: <20240802151430.99114-4-alexghiti@rivosinc.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20240802151430.99114-1-alexghiti@rivosinc.com> References: <20240802151430.99114-1-alexghiti@rivosinc.com> MIME-Version: 1.0 X-Stat-Signature: gcz36fwiri1f8jnm9c4wb7bfqucnnixp X-Rspamd-Queue-Id: 0B779C0015 X-Rspam-User: X-Rspamd-Server: rspam10 X-HE-Tag: 1722611858-185181 X-HE-Meta: U2FsdGVkX1/Oh2sGxTmTR+omIdKMOX2lpwK2knt2FDFFxyAClKUaMGjTZ/4UR137TAD6ELJycI80wtkJkjNntHsJG0TpcBdXAHhDwhtEhmKBtpV5Gb8U/x2A66et80F+0Xvt6dtPxYYV8Xu7XtFjhibOsSWmNrDEdfnQBdx+nHwYOT6njbO2DhhMwSB4kB3NzCLNsZBBr7RcBVBCszB5nfEMkswzBzvVSIchCyDLpVc1IgERnAxi7LGmVxOqNWkgDe4w8kk2aBVVVne/tN6xWtOpaDHqQ9mfNQKqCOGIwkxt8CByCE3cQg/TD9qjqxfLm84m2oE+Lm3xFhcNlHF41weK1Pr6LRfhJfJ04Rb3sTwrEwJfyfspbbH2ot9zAIjWvn6TANYcp4WNtykp76tH65vMt/KcChq28TvnlKrqoBWHQjBM/983SEhaG8bqDLW67SF+AyMjim3TiN+0lhbkTm5PZdcmivpAkkQJHz+9YMzd4Kj7iC6mL8imuErgE7wEhU0CFMFeXObdA1D8VYwP909w1WdY/7LsbDqOIvPnjWYf6IgRWPo3UoWt0ImjLoMiVJdVuGlobEL/aI2nUym3dQaj2rGcAdDY34EadjlxiMWH8TKz7OD8Uff3ZpEU4kg4uvNgo0cj3pAONbaqmc06tSpd8iIhpD93K3xBwEv/wi4hOQL/nKs8/mGzXieN9uRTZT3BLm8RJBA7xUgEeJOzaLT9NjdMXCIyxfLvFikYOeG6KjzjfxQxWUJZcHdC/tACCElS70mvLRTNIGN3wpVpLxzeI+/PvKnVhTfxl1K4n4h6fjhm986kFN16TYs0RNirhIFFjPoOyY8e/8Sag4jn1ZZ8xFkUpae6Pt22l7M93Zh1mH8vXzqC+xDF+YA4kETzhPob7gcwfnzbj4+klKfGUD2X1OWxma56MFjsC28+rXxGjtqN8eY3MijXupiEeyl2FB5F2czPJZoQaB7CRIN GKGb2RYa 6VbBpIBmsOxl/LkQaAnZSToU5Q1XHzD0/MNiB1p0kgl0CXVpkeUxENK3D2BRmzDU6LrtZj+oatu02ck8bo6M3H4jhkxOiTyzZoeqKd6AcVSHdFu8Hllwav/msN5eGhgNsBkZxc3YXBhnXXBBPoN1r6LMd2fxbZCi183EnXjzk6HWXMnNTgGRBJjeR0zklEDpQPsRAm0zIWaDAzpfCQtjEolU/jNtYGuefYy03gKAeq9rDc5ltsOCLJxPSuz8Z7Wp6SslH8CqGae3CZ1oo6/Qa1ujtFFPbFdX4cI0bwnGZNsvDYzUP1DCmvFUdmhjso3Uh6EFRMYLpTCOn1c0/m+joDGYZ18IC7842szTSjdZYKaGb+YZ6Si91MtVCdiz8tSFaxV5DA5yhcU26jUhqm+diu40+Dg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: After some adjustments, both architectures have the same implementation so move it to the generic code. Signed-off-by: Alexandre Ghiti --- arch/arm64/Kconfig | 1 + arch/arm64/include/asm/hugetlb.h | 3 +- arch/arm64/include/asm/pgtable.h | 48 +++++++++++++++++++++++++--- arch/arm64/mm/hugetlbpage.c | 55 ++------------------------------ arch/riscv/Kconfig | 1 + arch/riscv/include/asm/hugetlb.h | 6 ++-- arch/riscv/include/asm/pgtable.h | 36 +++++++++++++++++++++ arch/riscv/mm/hugetlbpage.c | 45 ++++++-------------------- include/linux/hugetlb_contpte.h | 12 +++++++ mm/Kconfig | 3 ++ mm/Makefile | 1 + mm/hugetlb_contpte.c | 44 +++++++++++++++++++++++++ 12 files changed, 157 insertions(+), 98 deletions(-) create mode 100644 include/linux/hugetlb_contpte.h create mode 100644 mm/hugetlb_contpte.c diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig index b3fc891f1544..0a524959804a 100644 --- a/arch/arm64/Kconfig +++ b/arch/arm64/Kconfig @@ -104,6 +104,7 @@ config ARM64 select ARCH_WANT_DEFAULT_BPF_JIT select ARCH_WANT_DEFAULT_TOPDOWN_MMAP_LAYOUT select ARCH_WANT_FRAME_POINTERS + select ARCH_WANT_GENERAL_HUGETLB_CONTPTE select ARCH_WANT_HUGE_PMD_SHARE if ARM64_4K_PAGES || (ARM64_16K_PAGES && !ARM64_VA_BITS_36) select ARCH_WANT_LD_ORPHAN_WARN select ARCH_WANTS_EXECMEM_LATE if EXECMEM diff --git a/arch/arm64/include/asm/hugetlb.h b/arch/arm64/include/asm/hugetlb.h index 293f880865e8..80d25b4eff25 100644 --- a/arch/arm64/include/asm/hugetlb.h +++ b/arch/arm64/include/asm/hugetlb.h @@ -12,6 +12,7 @@ #include #include +#include #ifdef CONFIG_ARCH_ENABLE_HUGEPAGE_MIGRATION #define arch_hugetlb_migration_supported arch_hugetlb_migration_supported @@ -45,8 +46,6 @@ extern pte_t huge_ptep_clear_flush(struct vm_area_struct *vma, #define __HAVE_ARCH_HUGE_PTE_CLEAR extern void huge_pte_clear(struct mm_struct *mm, unsigned long addr, pte_t *ptep, unsigned long sz); -#define __HAVE_ARCH_HUGE_PTEP_GET -extern pte_t huge_ptep_get(struct mm_struct *mm, unsigned long addr, pte_t *ptep); void __init arm64_hugetlb_cma_reserve(void); diff --git a/arch/arm64/include/asm/pgtable.h b/arch/arm64/include/asm/pgtable.h index 7a4f5604be3f..2a6a01b447df 100644 --- a/arch/arm64/include/asm/pgtable.h +++ b/arch/arm64/include/asm/pgtable.h @@ -397,9 +397,10 @@ static inline pte_t pte_advance_pfn(pte_t pte, unsigned long nr) return pfn_pte(pte_pfn(pte) + nr, pte_pgprot(pte)); } -static inline void __set_ptes(struct mm_struct *mm, - unsigned long __always_unused addr, - pte_t *ptep, pte_t pte, unsigned int nr) +static inline void ___set_ptes(struct mm_struct *mm, + unsigned long __always_unused addr, + pte_t *ptep, pte_t pte, unsigned int nr, + size_t pgsize) { page_table_check_ptes_set(mm, ptep, pte, nr); __sync_cache_and_tags(pte, nr); @@ -410,10 +411,15 @@ static inline void __set_ptes(struct mm_struct *mm, if (--nr == 0) break; ptep++; - pte = pte_advance_pfn(pte, 1); + pte = pte_advance_pfn(pte, pgsize >> PAGE_SHIFT); } } +#define __set_ptes(mm, addr, ptep, pte, nr) \ + ___set_ptes(mm, addr, ptep, pte, nr, PAGE_SIZE) + +#define set_contptes ___set_ptes + /* * Huge pte definitions. */ @@ -1760,6 +1766,40 @@ static inline void clear_young_dirty_ptes(struct vm_area_struct *vma, #endif /* CONFIG_ARM64_CONTPTE */ +static inline int arch_contpte_get_num_contig(pte_t *ptep, + unsigned long size, + size_t *pgsize) +{ + int contig_ptes = 0; + + if (pgsize) + *pgsize = size; + + switch (size) { +#ifndef __PAGETABLE_PMD_FOLDED + case PUD_SIZE: + if (pud_sect_supported()) + contig_ptes = 1; + break; +#endif + case PMD_SIZE: + contig_ptes = 1; + break; + case CONT_PMD_SIZE: + if (pgsize) + *pgsize = PMD_SIZE; + contig_ptes = CONT_PMDS; + break; + case CONT_PTE_SIZE: + if (pgsize) + *pgsize = PAGE_SIZE; + contig_ptes = CONT_PTES; + break; + } + + return contig_ptes; +} + #endif /* !__ASSEMBLY__ */ #endif /* __ASM_PGTABLE_H */ diff --git a/arch/arm64/mm/hugetlbpage.c b/arch/arm64/mm/hugetlbpage.c index 5f1e2103888b..58cb5e06dcb2 100644 --- a/arch/arm64/mm/hugetlbpage.c +++ b/arch/arm64/mm/hugetlbpage.c @@ -98,57 +98,6 @@ static int find_num_contig(struct mm_struct *mm, unsigned long addr, return CONT_PTES; } -static inline int num_contig_ptes(unsigned long size, size_t *pgsize) -{ - int contig_ptes = 0; - - *pgsize = size; - - switch (size) { -#ifndef __PAGETABLE_PMD_FOLDED - case PUD_SIZE: - if (pud_sect_supported()) - contig_ptes = 1; - break; -#endif - case PMD_SIZE: - contig_ptes = 1; - break; - case CONT_PMD_SIZE: - *pgsize = PMD_SIZE; - contig_ptes = CONT_PMDS; - break; - case CONT_PTE_SIZE: - *pgsize = PAGE_SIZE; - contig_ptes = CONT_PTES; - break; - } - - return contig_ptes; -} - -pte_t huge_ptep_get(struct mm_struct *mm, unsigned long addr, pte_t *ptep) -{ - int ncontig, i; - size_t pgsize; - pte_t orig_pte = __ptep_get(ptep); - - if (!pte_present(orig_pte) || !pte_cont(orig_pte)) - return orig_pte; - - ncontig = num_contig_ptes(page_size(pte_page(orig_pte)), &pgsize); - for (i = 0; i < ncontig; i++, ptep++) { - pte_t pte = __ptep_get(ptep); - - if (pte_dirty(pte)) - orig_pte = pte_mkdirty(orig_pte); - - if (pte_young(pte)) - orig_pte = pte_mkyoung(orig_pte); - } - return orig_pte; -} - /* * Changing some bits of contiguous entries requires us to follow a * Break-Before-Make approach, breaking the whole contiguous set @@ -229,7 +178,7 @@ void set_huge_pte_at(struct mm_struct *mm, unsigned long addr, unsigned long pfn, dpfn; pgprot_t hugeprot; - ncontig = num_contig_ptes(sz, &pgsize); + ncontig = arch_contpte_get_num_contig(ptep, sz, &pgsize); if (!pte_present(pte)) { for (i = 0; i < ncontig; i++, ptep++, addr += pgsize) @@ -379,7 +328,7 @@ void huge_pte_clear(struct mm_struct *mm, unsigned long addr, int i, ncontig; size_t pgsize; - ncontig = num_contig_ptes(sz, &pgsize); + ncontig = arch_contpte_get_num_contig(ptep, sz, &pgsize); for (i = 0; i < ncontig; i++, addr += pgsize, ptep++) __pte_clear(mm, addr, ptep); diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig index e860f3e8c702..5f6ed3a43aa7 100644 --- a/arch/riscv/Kconfig +++ b/arch/riscv/Kconfig @@ -73,6 +73,7 @@ config RISCV select ARCH_WANT_DEFAULT_TOPDOWN_MMAP_LAYOUT if MMU select ARCH_WANT_FRAME_POINTERS select ARCH_WANT_GENERAL_HUGETLB if !RISCV_ISA_SVNAPOT + select ARCH_WANT_GENERAL_HUGETLB_CONTPTE if RISCV_ISA_SVNAPOT select ARCH_WANT_HUGE_PMD_SHARE if 64BIT select ARCH_WANT_LD_ORPHAN_WARN if !XIP_KERNEL select ARCH_WANT_OPTIMIZE_DAX_VMEMMAP diff --git a/arch/riscv/include/asm/hugetlb.h b/arch/riscv/include/asm/hugetlb.h index faf3624d8057..d9f9bfb84908 100644 --- a/arch/riscv/include/asm/hugetlb.h +++ b/arch/riscv/include/asm/hugetlb.h @@ -4,6 +4,9 @@ #include #include +#ifdef CONFIG_ARCH_WANT_GENERAL_HUGETLB_CONTPTE +#include +#endif static inline void arch_clear_hugetlb_flags(struct folio *folio) { @@ -43,9 +46,6 @@ int huge_ptep_set_access_flags(struct vm_area_struct *vma, unsigned long addr, pte_t *ptep, pte_t pte, int dirty); -#define __HAVE_ARCH_HUGE_PTEP_GET -pte_t huge_ptep_get(struct mm_struct *mm, unsigned long addr, pte_t *ptep); - pte_t arch_make_huge_pte(pte_t entry, unsigned int shift, vm_flags_t flags); #define arch_make_huge_pte arch_make_huge_pte diff --git a/arch/riscv/include/asm/pgtable.h b/arch/riscv/include/asm/pgtable.h index 34c4c360d4ce..412ccebcdee9 100644 --- a/arch/riscv/include/asm/pgtable.h +++ b/arch/riscv/include/asm/pgtable.h @@ -300,6 +300,8 @@ static inline unsigned long pte_napot(pte_t pte) return pte_val(pte) & _PAGE_NAPOT; } +#define pte_cont pte_napot + #define pte_valid_napot(pte) (pte_present(pte) && pte_napot(pte)) static inline pte_t pte_mknapot(pte_t pte, unsigned int order) @@ -581,6 +583,38 @@ static inline void __set_ptes(struct mm_struct *mm, unsigned long addr, } } +#ifdef CONFIG_RISCV_ISA_SVNAPOT +static inline int arch_contpte_get_num_contig(pte_t *ptep, unsigned long size, + size_t *pgsize) +{ + unsigned long hugepage_shift; + pte_t __pte; + + if (size >= PGDIR_SIZE) + hugepage_shift = PGDIR_SHIFT; + else if (size >= P4D_SIZE) + hugepage_shift = P4D_SHIFT; + else if (size >= PUD_SIZE) + hugepage_shift = PUD_SHIFT; + else if (size >= PMD_SIZE) + hugepage_shift = PMD_SHIFT; + else + hugepage_shift = PAGE_SHIFT; + + if (pgsize) + *pgsize = BIT(hugepage_shift); + + /* We must read the raw value of the pte to get the size of the mapping */ + __pte = __ptep_get(ptep); + + /* Make sure __pte is not a swap entry */ + if (pte_valid_napot(__pte)) + return napot_pte_num(napot_cont_order(__pte)); + + return size >> hugepage_shift; +} +#endif + static inline void pte_clear(struct mm_struct *mm, unsigned long addr, pte_t *ptep) { @@ -665,6 +699,8 @@ static inline void set_ptes(struct mm_struct *mm, unsigned long addr, __set_ptes(mm, addr, ptep, pteval, nr); } #define set_ptes set_ptes +#define set_contptes(mm, addr, ptep, pte, nr, pgsize) \ + set_ptes(mm, addr, ptep, pte, nr) static inline pte_t ptep_get(pte_t *ptep) { diff --git a/arch/riscv/mm/hugetlbpage.c b/arch/riscv/mm/hugetlbpage.c index 59ed26ce6857..d51863824540 100644 --- a/arch/riscv/mm/hugetlbpage.c +++ b/arch/riscv/mm/hugetlbpage.c @@ -3,30 +3,6 @@ #include #ifdef CONFIG_RISCV_ISA_SVNAPOT -pte_t huge_ptep_get(struct mm_struct *mm, unsigned long addr, pte_t *ptep) -{ - unsigned long pte_num; - int i; - pte_t orig_pte = ptep_get(ptep); - - if (!pte_present(orig_pte) || !pte_napot(orig_pte)) - return orig_pte; - - pte_num = napot_pte_num(napot_cont_order(orig_pte)); - - for (i = 0; i < pte_num; i++, ptep++) { - pte_t pte = ptep_get(ptep); - - if (pte_dirty(pte)) - orig_pte = pte_mkdirty(orig_pte); - - if (pte_young(pte)) - orig_pte = pte_mkyoung(orig_pte); - } - - return orig_pte; -} - pte_t *huge_pte_alloc(struct mm_struct *mm, struct vm_area_struct *vma, unsigned long addr, @@ -266,15 +242,13 @@ int huge_ptep_set_access_flags(struct vm_area_struct *vma, int dirty) { struct mm_struct *mm = vma->vm_mm; - unsigned long order; pte_t orig_pte; - int i, pte_num; + int pte_num; if (!pte_napot(pte)) return ptep_set_access_flags(vma, addr, ptep, pte, dirty); - order = napot_cont_order(pte); - pte_num = napot_pte_num(order); + pte_num = arch_contpte_get_num_contig(ptep, 0, NULL); orig_pte = get_clear_contig_flush(mm, addr, ptep, pte_num); if (pte_dirty(orig_pte)) @@ -298,7 +272,7 @@ pte_t huge_ptep_get_and_clear(struct mm_struct *mm, if (!pte_napot(orig_pte)) return ptep_get_and_clear(mm, addr, ptep); - pte_num = napot_pte_num(napot_cont_order(orig_pte)); + pte_num = arch_contpte_get_num_contig(ptep, 0, NULL); return get_clear_contig(mm, addr, ptep, pte_num); } @@ -308,17 +282,15 @@ void huge_ptep_set_wrprotect(struct mm_struct *mm, pte_t *ptep) { pte_t pte = ptep_get(ptep); - unsigned long order; pte_t orig_pte; - int i, pte_num; + int pte_num; if (!pte_napot(pte)) { ptep_set_wrprotect(mm, addr, ptep); return; } - order = napot_cont_order(pte); - pte_num = napot_pte_num(order); + pte_num = arch_contpte_get_num_contig(ptep, 0, NULL); orig_pte = get_clear_contig_flush(mm, addr, ptep, pte_num); orig_pte = pte_wrprotect(orig_pte); @@ -336,7 +308,7 @@ pte_t huge_ptep_clear_flush(struct vm_area_struct *vma, if (!pte_napot(pte)) return ptep_clear_flush(vma, addr, ptep); - pte_num = napot_pte_num(napot_cont_order(pte)); + pte_num = arch_contpte_get_num_contig(ptep, 0, NULL); return get_clear_contig_flush(vma->vm_mm, addr, ptep, pte_num); } @@ -346,6 +318,7 @@ void huge_pte_clear(struct mm_struct *mm, pte_t *ptep, unsigned long sz) { + size_t pgsize; pte_t pte = ptep_get(ptep); int i, pte_num; @@ -354,8 +327,8 @@ void huge_pte_clear(struct mm_struct *mm, return; } - pte_num = napot_pte_num(napot_cont_order(pte)); - for (i = 0; i < pte_num; i++, addr += PAGE_SIZE, ptep++) + pte_num = arch_contpte_get_num_contig(ptep, sz, &pgsize); + for (i = 0; i < pte_num; i++, addr += pgsize, ptep++) pte_clear(mm, addr, ptep); } diff --git a/include/linux/hugetlb_contpte.h b/include/linux/hugetlb_contpte.h new file mode 100644 index 000000000000..ec4189cd65b8 --- /dev/null +++ b/include/linux/hugetlb_contpte.h @@ -0,0 +1,12 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (C) 2024 Rivos Inc. + */ + +#ifndef _LINUX_HUGETLB_CONTPTE_H +#define _LINUX_HUGETLB_CONTPTE_H + +#define __HAVE_ARCH_HUGE_PTEP_GET +extern pte_t huge_ptep_get(struct mm_struct *mm, unsigned long addr, pte_t *ptep); + +#endif /* _LINUX_HUGETLB_CONTPTE_H */ diff --git a/mm/Kconfig b/mm/Kconfig index b72e7d040f78..0bba50d0639a 100644 --- a/mm/Kconfig +++ b/mm/Kconfig @@ -798,6 +798,9 @@ config NOMMU_INITIAL_TRIM_EXCESS config ARCH_WANT_GENERAL_HUGETLB bool +config ARCH_WANT_GENERAL_HUGETLB_CONTPTE + bool + config ARCH_WANTS_THP_SWAP def_bool n diff --git a/mm/Makefile b/mm/Makefile index d2915f8c9dc0..3dd91c4466aa 100644 --- a/mm/Makefile +++ b/mm/Makefile @@ -95,6 +95,7 @@ obj-$(CONFIG_MIGRATION) += migrate.o obj-$(CONFIG_NUMA) += memory-tiers.o obj-$(CONFIG_DEVICE_MIGRATION) += migrate_device.o obj-$(CONFIG_TRANSPARENT_HUGEPAGE) += huge_memory.o khugepaged.o +obj-$(CONFIG_ARCH_WANT_GENERAL_HUGETLB_CONTPTE) += hugetlb_contpte.o obj-$(CONFIG_PAGE_COUNTER) += page_counter.o obj-$(CONFIG_MEMCG_V1) += memcontrol-v1.o obj-$(CONFIG_MEMCG) += memcontrol.o vmpressure.o diff --git a/mm/hugetlb_contpte.c b/mm/hugetlb_contpte.c new file mode 100644 index 000000000000..0e3ba6f97c58 --- /dev/null +++ b/mm/hugetlb_contpte.c @@ -0,0 +1,44 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright 2024 Rivos Inc. + */ + +#include +#include +#include + +/* + * Any arch that wants to use that needs to define: + * - __ptep_get() + * - pte_cont() + * - arch_contpte_get_num_contig() + */ + +/* + * This file implements the following contpte aware API: + * - huge_ptep_get() + */ + +pte_t huge_ptep_get(struct mm_struct *mm, unsigned long addr, pte_t *ptep) +{ + int ncontig, i; + pte_t orig_pte = __ptep_get(ptep); + + if (!pte_present(orig_pte) || !pte_cont(orig_pte)) + return orig_pte; + + ncontig = arch_contpte_get_num_contig(ptep, + page_size(pte_page(orig_pte)), + NULL); + + for (i = 0; i < ncontig; i++, ptep++) { + pte_t pte = __ptep_get(ptep); + + if (pte_dirty(pte)) + orig_pte = pte_mkdirty(orig_pte); + + if (pte_young(pte)) + orig_pte = pte_mkyoung(orig_pte); + } + return orig_pte; +} From patchwork Fri Aug 2 15:14:25 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexandre Ghiti X-Patchwork-Id: 13751626 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id AA9A6C52D6F for ; Fri, 2 Aug 2024 15:18:43 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 35BC96B0088; Fri, 2 Aug 2024 11:18:43 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 30C6B6B0092; Fri, 2 Aug 2024 11:18:43 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 1D4816B0096; Fri, 2 Aug 2024 11:18:43 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id EDC0B6B0088 for ; Fri, 2 Aug 2024 11:18:42 -0400 (EDT) Received: from smtpin28.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 877F31A1213 for ; Fri, 2 Aug 2024 15:18:42 +0000 (UTC) X-FDA: 82407662484.28.E24FF31 Received: from mail-lj1-f169.google.com (mail-lj1-f169.google.com [209.85.208.169]) by imf15.hostedemail.com (Postfix) with ESMTP id 8F577A002E for ; Fri, 2 Aug 2024 15:18:40 +0000 (UTC) Authentication-Results: imf15.hostedemail.com; dkim=pass header.d=rivosinc-com.20230601.gappssmtp.com header.s=20230601 header.b=toj0CO6p; spf=pass (imf15.hostedemail.com: domain of alexghiti@rivosinc.com designates 209.85.208.169 as permitted sender) smtp.mailfrom=alexghiti@rivosinc.com; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1722611914; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=AGB1X3sQQ3LKpfi0jpo5C6FpBS7I6wNMeoGCJ6BimlY=; b=WpvnviRH3nDmi+Z0XVG1tMz0kbcR4yiP/RDnWwUr3mNgLGfeqGAHG4MA02SSTw1y494foi 3L55YmUpzFCj7uun7KLz2tdsFhtuSMKNh1o7p8dTyvEb+ck6DC2v5e/qoMuitaragwAsbJ LkWZeAVWxzXmtNQMmCVaehZZEqN5g24= ARC-Authentication-Results: i=1; imf15.hostedemail.com; dkim=pass header.d=rivosinc-com.20230601.gappssmtp.com header.s=20230601 header.b=toj0CO6p; spf=pass (imf15.hostedemail.com: domain of alexghiti@rivosinc.com designates 209.85.208.169 as permitted sender) smtp.mailfrom=alexghiti@rivosinc.com; dmarc=none ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1722611914; a=rsa-sha256; cv=none; b=3Dwlwd4hpsyYgzmR9rLGuudC1XV/jCWagcPqJyeETL/vpsOSk4cPvt9sWm2X7ipiL5tUUz otsJQ6cPRk3aPuTlc8yBikQR9h5ml2RNIeKk38fPn0NMiHO07Lv7ObhFVyPKvZo+ZJ3p2a nUkqiTxdZvo3QRaQUqzD4R1lydv+79I= Received: by mail-lj1-f169.google.com with SMTP id 38308e7fff4ca-2ef2d96164aso92152141fa.3 for ; Fri, 02 Aug 2024 08:18:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc-com.20230601.gappssmtp.com; s=20230601; t=1722611919; x=1723216719; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=AGB1X3sQQ3LKpfi0jpo5C6FpBS7I6wNMeoGCJ6BimlY=; b=toj0CO6pL+aN0OX/PWMUlb3kdOuOPT+VZ6kt3ublKqDgjR5Vw7rEG+qFe7iFWeLfb+ vjW9bPZTfCoI/2Jr6a+lNK+vLmqKVBHBX7kHsFQZOOll4NgUZXxlkT9/aurvNUAQxNNv 8PHTTIzddb6s8lgE+pH9Owi0u8GxrjVipOpVelPkUyJxngny4azlqkx54m2gPPpUoyP3 dbAOlJuRxiQrGEai7EJoq5CfQtDGr61LL/Fyyui/2dKI7Wls36MHnKb/IfDt9BlJ+oGV dEcqXIh1+2FyVxadwv7gYeZ+w//GkI0CkqnyAtWVBeNyKMWeC5h/UtVQ3MCTbig/496d iBhw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1722611919; x=1723216719; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=AGB1X3sQQ3LKpfi0jpo5C6FpBS7I6wNMeoGCJ6BimlY=; b=wMLiwCcj1xpteGJh+bYX7kOEH9V8QpNlDr8YhKAosZEqXKZl9+Pi027Ma2zIEIBbeT ylLp/gmn3zXAzkpJc+MsF5oqWxcKML6k8/SwfVX+gVePw4DAN7wjN8+iAtZ/Z4rFp+01 9CWnlAWmWrfleWZdVIQqWIEJF0XC6TVhQVPvcQW3pbHlDb786KeNdrRiBtai214N82YI YtPyrrwBjjSdQgEbzhkjJwaOfff0jnP1hreeBQ5dWQAcSwbW3QSxkwCbRnNzMVNm1fxx iyQUmNiJfBcpUVRrtdwhTMsNB6guWQqGCC91BOJruNexzN0OSiwLJIKAjIHJh3XbiWjW EZIQ== X-Forwarded-Encrypted: i=1; AJvYcCX6f1J1plx2IGEn6EluLyb7cqDz9fdohJS5852D7ZxERXTl0rGp0v0R+E453Ob1WWL7zBbtMAdJs5AZi2KFstBtsZ0= X-Gm-Message-State: AOJu0Yw85WibDW1oHOlvqbNUil/eRiU7kl+JT3FI0bCPPoveG0PQDVNy VP8TW0yDxAFtUSZdLKeDcZyRRsUGoPa5H6/zjROoQi0n37dIRQFD06tt4N+57pg= X-Google-Smtp-Source: AGHT+IFLYojfl67dZHCnARw4zAZgsB8CSLUwEfcrwOW6i95dP1Od+Eh6wNfB4OrwDg1Qyouoqg9rsQ== X-Received: by 2002:a2e:b170:0:b0:2ef:1bd5:bac3 with SMTP id 38308e7fff4ca-2f15ab419camr28122861fa.41.1722611918703; Fri, 02 Aug 2024 08:18:38 -0700 (PDT) Received: from alex-rivos.ba.rivosinc.com (amontpellier-656-1-456-62.w92-145.abo.wanadoo.fr. [92.145.124.62]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-42824af5410sm97336045e9.1.2024.08.02.08.18.37 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 02 Aug 2024 08:18:38 -0700 (PDT) From: Alexandre Ghiti To: Catalin Marinas , Will Deacon , Ryan Roberts , Mark Rutland , Paul Walmsley , Palmer Dabbelt , Albert Ou , Andrew Morton , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org, linux-mm@kvack.org Cc: Alexandre Ghiti Subject: [PATCH v3 4/9] mm: Use common set_huge_pte_at() function for riscv/arm64 Date: Fri, 2 Aug 2024 17:14:25 +0200 Message-Id: <20240802151430.99114-5-alexghiti@rivosinc.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20240802151430.99114-1-alexghiti@rivosinc.com> References: <20240802151430.99114-1-alexghiti@rivosinc.com> MIME-Version: 1.0 X-Rspam-User: X-Stat-Signature: g9igkdnkpsausasywc9ku58ae8pj4wg7 X-Rspamd-Queue-Id: 8F577A002E X-Rspamd-Server: rspam11 X-HE-Tag: 1722611920-481796 X-HE-Meta: U2FsdGVkX19ZfGsWsWufiPIs38MgqTX/DGYSYBjIhcXfuc61+b2414HjEwEK0tdIldWB6l1LjsvWL3skee2nVB8U5uRK+0DDXITPzxNZTVIlJM4JNnqMM3hVCluFXpzDnMOHy/cWMSHtc9XAk1bj5ohgHKfPajysXAovp/+MqR/Qy8oJhhxStupexT5nlJL1v1dK5+id6d76G+FoMD2nt1hx28OEL66LcXMdO82oY8vFwwvrH1s6XTCDsKX4oTOJmRP8pWbmjtRdm5BGnqOQQAvoT10sdMvDzFwaUg1YFiZJGWPEIiHA1CJSQv8LjLZh9cbTpuEdgEOOHS+2wsOa95Vxwy8k2Yb/c3xlPH9CUsi1YblCBKBkWe/E8gL1xO+O/Px7xe4ufw1AOA6u/oCLaCWo2dH/JUFWtCHewaJVJPAqya741we1HHv2dM4bYawz4xUTDOoWJO23mok6l6HD0xCpFSmOzYl3f366Nn6g1b0zdKNak2PMoWrJY+kduz2YLIyyijsknZEOVZIUlQhPLsZ8K3rQNvDunIQBTk9CldisvKviQGJ6/iq/uPmjXoVSf03WU1C4hRt1NGX+oW99nLVJq7SVKjp+HB1NmghTKeQEB9ZMgvDYm1DQByEq1gZV7D2zdjPlkRqGi8lPfDfYbHrOIe/Z9iVE31GOtpjWraHC3H+xKSkei66/KhG1vzzbPb9H0HyVQ1N+dmdW8Z6lLC/RRzFOj+MUYH68epMkXMs9Rm2UMYtM/XWNNW/JA9Sp6JtztdQjxqycJqNpoXb4BuXwNr1iz8cG65DEXeIaco6UOm3oFoZ2f5c7jsONqcLBbqlIIx+of8vGvpwGulOAvuwnurvVsoPI6HImyCFlLBe/pJqaN080y5cH42xPV/73HBnuGKfngTPbvsQqS87VsNjz5NvJcLlJgDwG9Gr9NNQlhpRyeAJ9P99pNd/eY+3kIcqfs4blVmUGOa1DwNO yvFJsetd n2p0rh6v5aaLRwip8i8Csn8mXg86bRU6xhEa9msmwXveeuy8s7Oc3ZmOXJnQKzZ3MBz7jpmgTJXOtq83EI39SbCDCmQkSSDE7cZcVO3wKKhCFx9jIODQCuXjNKUCYjDzam4msxIHF7Yw92Suu/nyLHD5iW4QhEQ2IMXeKo+rjCwX4EuiH8AN6fmwgZVulblRoVKVvw/E5CucNP7dJlxtopNnYf93Lyyv5yodqQSyL/shdWmgc38zF0E4UztSDxdIgRf9EvwaeApS2A1aMR3SSCgoiHf7UvMyGTyyJzGuRwu85fz2mslVdiuZUCiofhdzY4IX8ojtPlXpha9WULKZ+cKzQkYOXs0okUra7kiveSPw8ZaqwEPO93mZISTEiSZ7w3P4EYswzoZebls5yArWJhPr1HA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: After some adjustments, both architectures have the same implementation so move it to the generic code. Signed-off-by: Alexandre Ghiti --- arch/arm64/include/asm/hugetlb.h | 3 -- arch/arm64/mm/hugetlbpage.c | 56 ----------------------------- arch/riscv/include/asm/hugetlb.h | 5 --- arch/riscv/include/asm/pgtable.h | 8 +++-- arch/riscv/mm/hugetlbpage.c | 62 -------------------------------- include/linux/hugetlb_contpte.h | 5 +++ mm/hugetlb_contpte.c | 59 ++++++++++++++++++++++++++++++ 7 files changed, 69 insertions(+), 129 deletions(-) diff --git a/arch/arm64/include/asm/hugetlb.h b/arch/arm64/include/asm/hugetlb.h index 80d25b4eff25..d35093b7ab59 100644 --- a/arch/arm64/include/asm/hugetlb.h +++ b/arch/arm64/include/asm/hugetlb.h @@ -27,9 +27,6 @@ static inline void arch_clear_hugetlb_flags(struct folio *folio) pte_t arch_make_huge_pte(pte_t entry, unsigned int shift, vm_flags_t flags); #define arch_make_huge_pte arch_make_huge_pte -#define __HAVE_ARCH_HUGE_SET_HUGE_PTE_AT -extern void set_huge_pte_at(struct mm_struct *mm, unsigned long addr, - pte_t *ptep, pte_t pte, unsigned long sz); #define __HAVE_ARCH_HUGE_PTEP_SET_ACCESS_FLAGS extern int huge_ptep_set_access_flags(struct vm_area_struct *vma, unsigned long addr, pte_t *ptep, diff --git a/arch/arm64/mm/hugetlbpage.c b/arch/arm64/mm/hugetlbpage.c index 58cb5e06dcb2..c7130d1f07c4 100644 --- a/arch/arm64/mm/hugetlbpage.c +++ b/arch/arm64/mm/hugetlbpage.c @@ -145,62 +145,6 @@ static pte_t get_clear_contig_flush(struct mm_struct *mm, return orig_pte; } -/* - * Changing some bits of contiguous entries requires us to follow a - * Break-Before-Make approach, breaking the whole contiguous set - * before we can change any entries. See ARM DDI 0487A.k_iss10775, - * "Misprogramming of the Contiguous bit", page D4-1762. - * - * This helper performs the break step for use cases where the - * original pte is not needed. - */ -static void clear_flush(struct mm_struct *mm, - unsigned long addr, - pte_t *ptep, - unsigned long pgsize, - unsigned long ncontig) -{ - struct vm_area_struct vma = TLB_FLUSH_VMA(mm, 0); - unsigned long i, saddr = addr; - - for (i = 0; i < ncontig; i++, addr += pgsize, ptep++) - __ptep_get_and_clear(mm, addr, ptep); - - flush_tlb_range(&vma, saddr, addr); -} - -void set_huge_pte_at(struct mm_struct *mm, unsigned long addr, - pte_t *ptep, pte_t pte, unsigned long sz) -{ - size_t pgsize; - int i; - int ncontig; - unsigned long pfn, dpfn; - pgprot_t hugeprot; - - ncontig = arch_contpte_get_num_contig(ptep, sz, &pgsize); - - if (!pte_present(pte)) { - for (i = 0; i < ncontig; i++, ptep++, addr += pgsize) - __set_ptes(mm, addr, ptep, pte, 1); - return; - } - - if (!pte_cont(pte)) { - __set_ptes(mm, addr, ptep, pte, 1); - return; - } - - pfn = pte_pfn(pte); - dpfn = pgsize >> PAGE_SHIFT; - hugeprot = pte_pgprot(pte); - - clear_flush(mm, addr, ptep, pgsize, ncontig); - - for (i = 0; i < ncontig; i++, ptep++, addr += pgsize, pfn += dpfn) - __set_ptes(mm, addr, ptep, pfn_pte(pfn, hugeprot), 1); -} - pte_t *huge_pte_alloc(struct mm_struct *mm, struct vm_area_struct *vma, unsigned long addr, unsigned long sz) { diff --git a/arch/riscv/include/asm/hugetlb.h b/arch/riscv/include/asm/hugetlb.h index d9f9bfb84908..28cbf5d761e1 100644 --- a/arch/riscv/include/asm/hugetlb.h +++ b/arch/riscv/include/asm/hugetlb.h @@ -24,11 +24,6 @@ bool arch_hugetlb_migration_supported(struct hstate *h); void huge_pte_clear(struct mm_struct *mm, unsigned long addr, pte_t *ptep, unsigned long sz); -#define __HAVE_ARCH_HUGE_SET_HUGE_PTE_AT -void set_huge_pte_at(struct mm_struct *mm, - unsigned long addr, pte_t *ptep, pte_t pte, - unsigned long sz); - #define __HAVE_ARCH_HUGE_PTEP_GET_AND_CLEAR pte_t huge_ptep_get_and_clear(struct mm_struct *mm, unsigned long addr, pte_t *ptep); diff --git a/arch/riscv/include/asm/pgtable.h b/arch/riscv/include/asm/pgtable.h index 412ccebcdee9..46b409e558b3 100644 --- a/arch/riscv/include/asm/pgtable.h +++ b/arch/riscv/include/asm/pgtable.h @@ -628,9 +628,8 @@ extern int ptep_set_access_flags(struct vm_area_struct *vma, unsigned long addre extern int ptep_test_and_clear_young(struct vm_area_struct *vma, unsigned long address, pte_t *ptep); -#define __HAVE_ARCH_PTEP_GET_AND_CLEAR -static inline pte_t ptep_get_and_clear(struct mm_struct *mm, - unsigned long address, pte_t *ptep) +static inline pte_t __ptep_get_and_clear(struct mm_struct *mm, + unsigned long address, pte_t *ptep) { pte_t pte = __pte(atomic_long_xchg((atomic_long_t *)ptep, 0)); @@ -730,6 +729,9 @@ static inline pte_t ptep_get(pte_t *ptep) #define ptep_get __ptep_get #endif /* CONFIG_RISCV_ISA_SVNAPOT */ +#define __HAVE_ARCH_PTEP_GET_AND_CLEAR +#define ptep_get_and_clear __ptep_get_and_clear + #define pgprot_nx pgprot_nx static inline pgprot_t pgprot_nx(pgprot_t _prot) { diff --git a/arch/riscv/mm/hugetlbpage.c b/arch/riscv/mm/hugetlbpage.c index d51863824540..0ecb2846c3f0 100644 --- a/arch/riscv/mm/hugetlbpage.c +++ b/arch/riscv/mm/hugetlbpage.c @@ -173,68 +173,6 @@ pte_t arch_make_huge_pte(pte_t entry, unsigned int shift, vm_flags_t flags) return entry; } -static void clear_flush(struct mm_struct *mm, - unsigned long addr, - pte_t *ptep, - unsigned long pgsize, - unsigned long ncontig) -{ - struct vm_area_struct vma = TLB_FLUSH_VMA(mm, 0); - unsigned long i, saddr = addr; - - for (i = 0; i < ncontig; i++, addr += pgsize, ptep++) - ptep_get_and_clear(mm, addr, ptep); - - flush_tlb_range(&vma, saddr, addr); -} - -/* - * When dealing with NAPOT mappings, the privileged specification indicates that - * "if an update needs to be made, the OS generally should first mark all of the - * PTEs invalid, then issue SFENCE.VMA instruction(s) covering all 4 KiB regions - * within the range, [...] then update the PTE(s), as described in Section - * 4.2.1.". That's the equivalent of the Break-Before-Make approach used by - * arm64. - */ -void set_huge_pte_at(struct mm_struct *mm, - unsigned long addr, - pte_t *ptep, - pte_t pte, - unsigned long sz) -{ - unsigned long hugepage_shift, pgsize; - int i, pte_num; - - if (sz >= PGDIR_SIZE) - hugepage_shift = PGDIR_SHIFT; - else if (sz >= P4D_SIZE) - hugepage_shift = P4D_SHIFT; - else if (sz >= PUD_SIZE) - hugepage_shift = PUD_SHIFT; - else if (sz >= PMD_SIZE) - hugepage_shift = PMD_SHIFT; - else - hugepage_shift = PAGE_SHIFT; - - pte_num = sz >> hugepage_shift; - pgsize = 1 << hugepage_shift; - - if (!pte_present(pte)) { - for (i = 0; i < pte_num; i++, ptep++, addr += pgsize) - set_ptes(mm, addr, ptep, pte, 1); - return; - } - - if (!pte_napot(pte)) { - set_ptes(mm, addr, ptep, pte, 1); - return; - } - - clear_flush(mm, addr, ptep, pgsize, pte_num); - - set_ptes(mm, addr, ptep, pte, pte_num); -} - int huge_ptep_set_access_flags(struct vm_area_struct *vma, unsigned long addr, pte_t *ptep, diff --git a/include/linux/hugetlb_contpte.h b/include/linux/hugetlb_contpte.h index ec4189cd65b8..7acd734a75e8 100644 --- a/include/linux/hugetlb_contpte.h +++ b/include/linux/hugetlb_contpte.h @@ -9,4 +9,9 @@ #define __HAVE_ARCH_HUGE_PTEP_GET extern pte_t huge_ptep_get(struct mm_struct *mm, unsigned long addr, pte_t *ptep); +#define __HAVE_ARCH_HUGE_SET_HUGE_PTE_AT +extern void set_huge_pte_at(struct mm_struct *mm, + unsigned long addr, pte_t *ptep, pte_t pte, + unsigned long sz); + #endif /* _LINUX_HUGETLB_CONTPTE_H */ diff --git a/mm/hugetlb_contpte.c b/mm/hugetlb_contpte.c index 0e3ba6f97c58..9a3a376784b0 100644 --- a/mm/hugetlb_contpte.c +++ b/mm/hugetlb_contpte.c @@ -10,6 +10,8 @@ /* * Any arch that wants to use that needs to define: * - __ptep_get() + * - __set_ptes() + * - __ptep_get_and_clear() * - pte_cont() * - arch_contpte_get_num_contig() */ @@ -17,6 +19,7 @@ /* * This file implements the following contpte aware API: * - huge_ptep_get() + * - set_huge_pte_at() */ pte_t huge_ptep_get(struct mm_struct *mm, unsigned long addr, pte_t *ptep) @@ -42,3 +45,59 @@ pte_t huge_ptep_get(struct mm_struct *mm, unsigned long addr, pte_t *ptep) } return orig_pte; } + +/* + * ARM64: Changing some bits of contiguous entries requires us to follow a + * Break-Before-Make approach, breaking the whole contiguous set + * before we can change any entries. See ARM DDI 0487A.k_iss10775, + * "Misprogramming of the Contiguous bit", page D4-1762. + * + * RISCV: When dealing with NAPOT mappings, the privileged specification + * indicates that "if an update needs to be made, the OS generally should first + * mark all of the PTEs invalid, then issue SFENCE.VMA instruction(s) covering + * all 4 KiB regions within the range, [...] then update the PTE(s), as + * described in Section 4.2.1.". That's the equivalent of the Break-Before-Make + * approach used by arm64. + * + * This helper performs the break step for use cases where the + * original pte is not needed. + */ +static void clear_flush(struct mm_struct *mm, + unsigned long addr, + pte_t *ptep, + unsigned long pgsize, + unsigned long ncontig) +{ + struct vm_area_struct vma = TLB_FLUSH_VMA(mm, 0); + unsigned long i, saddr = addr; + + for (i = 0; i < ncontig; i++, addr += pgsize, ptep++) + __ptep_get_and_clear(mm, addr, ptep); + + flush_tlb_range(&vma, saddr, addr); +} + +void set_huge_pte_at(struct mm_struct *mm, unsigned long addr, + pte_t *ptep, pte_t pte, unsigned long sz) +{ + size_t pgsize; + int i; + int ncontig; + + ncontig = arch_contpte_get_num_contig(ptep, sz, &pgsize); + + if (!pte_present(pte)) { + for (i = 0; i < ncontig; i++, ptep++, addr += pgsize) + __set_ptes(mm, addr, ptep, pte, 1); + return; + } + + if (!pte_cont(pte)) { + __set_ptes(mm, addr, ptep, pte, 1); + return; + } + + clear_flush(mm, addr, ptep, pgsize, ncontig); + + set_contptes(mm, addr, ptep, pte, ncontig, pgsize); +} From patchwork Fri Aug 2 15:14:26 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexandre Ghiti X-Patchwork-Id: 13751627 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 28B8DC3DA4A for ; Fri, 2 Aug 2024 15:19:46 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id B75D96B008A; Fri, 2 Aug 2024 11:19:45 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id B239B6B0095; Fri, 2 Aug 2024 11:19:45 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 9C42F6B0096; Fri, 2 Aug 2024 11:19:45 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 7CFE76B008A for ; Fri, 2 Aug 2024 11:19:45 -0400 (EDT) Received: from smtpin12.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id EA0CC80BD4 for ; Fri, 2 Aug 2024 15:19:44 +0000 (UTC) X-FDA: 82407665088.12.5147D74 Received: from mail-lj1-f174.google.com (mail-lj1-f174.google.com [209.85.208.174]) by imf22.hostedemail.com (Postfix) with ESMTP id 09381C0005 for ; Fri, 2 Aug 2024 15:19:42 +0000 (UTC) Authentication-Results: imf22.hostedemail.com; dkim=pass header.d=rivosinc-com.20230601.gappssmtp.com header.s=20230601 header.b=0R7WXPJ8; spf=pass (imf22.hostedemail.com: domain of alexghiti@rivosinc.com designates 209.85.208.174 as permitted sender) smtp.mailfrom=alexghiti@rivosinc.com; dmarc=none ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1722611953; a=rsa-sha256; cv=none; b=sC1TKj4ilSQ8gdKYYAKJtpSlLIBHuzxVTgh3AkTi1DGngpByO8gD+jZKlIojaevgAb5Kad 8CUzH32PF6etnmynLdfuxuNVIjf2SpMKZO2GMNu4rKnVSAeWZ7AupIpP5zoHFjr521uNg8 e2JBxOOdcvS1wnrLG2/A1FliF0rLZh4= ARC-Authentication-Results: i=1; imf22.hostedemail.com; dkim=pass header.d=rivosinc-com.20230601.gappssmtp.com header.s=20230601 header.b=0R7WXPJ8; spf=pass (imf22.hostedemail.com: domain of alexghiti@rivosinc.com designates 209.85.208.174 as permitted sender) smtp.mailfrom=alexghiti@rivosinc.com; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1722611953; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=hgy6e6djKCEiajpx++weOAekuViyBMKy0O9RS2EJI5E=; b=IhYQcGHMSPDvzVjCpEygjuVR5RiNrcrBz7MSWkLqPyiGstpb9RO72a5xkV74vg9+r3808D lthpK0wNp7wldN5pg/M8PC9Vnj9KPaR7JSNYtg7eDziuhCf0jRUUg1iwNvjVTUwylgLldN r/Xu6bmOwfa2fR6wdgTn8iVgKmHFRvU= Received: by mail-lj1-f174.google.com with SMTP id 38308e7fff4ca-2f032cb782dso88021281fa.3 for ; Fri, 02 Aug 2024 08:19:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc-com.20230601.gappssmtp.com; s=20230601; t=1722611981; x=1723216781; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=hgy6e6djKCEiajpx++weOAekuViyBMKy0O9RS2EJI5E=; b=0R7WXPJ8lN515k1b0pHmllf43v7BTzmhyb4uREV63fRnlLc7D3mIbot3OSAIZNtpGr C1tFGw+Dg6biu2Hu6NKWErg+qc6BAL88FmBfS7L/2PXfNzQR7Jli8iOli3V+l8gcKiTL nhvFVX7uv14ZTx7grZRm7w/nGpXnJOWP6EgCgqpkEtoCZj4dkJsB8hA8osPDMybizVRE on7IB+tRpV8N+E/vc1cr4V5sx5jBo8ZA/e00Rj/+1IZO0sTCdV0wxTS61hFcaDy/NhUj Htayn1ME4+26lmgjBhQP3dPtopob0MuM6vbn61ja0PqzXNuLc1y88ELrOMu18tIPu7+k GhSA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1722611981; x=1723216781; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=hgy6e6djKCEiajpx++weOAekuViyBMKy0O9RS2EJI5E=; b=wz0wVQvh6x5TTpZTlylZY3fRUe/ujvyDAWJgo0s8v7l+hpFqhsdxZX2TLOC7+i8UJA suiKVFTJlvcCKuNVm5ltVE0qTnGSEsvmDRJ1f+NeCga6Q9dgiDeg72E/CWVfAzmzLf4T kIED1txvcTNmQPuRol67itbriGgQFIFBvW80H+Ms3F4lsijaN2VdNSjdNXIwEt4HHHLF G66g1XP/IIOMvoVigTL3GTSNAk71C7YjgGnw0XMCqZDoSj9i7T56w7UOxjIpGk/r0Mys gr64oJnTWzHiK4Q0dX1iVJgJHaB4E+nebbFNkYSbiA6Gf4qZ/t+/X/l/LpiI41uJq8E/ qftQ== X-Forwarded-Encrypted: i=1; AJvYcCW3+z/9jOoLUD0u83oX5qAW/Rsw4GhDtuZr5Vu50DAfx//cYIjJVl9LS1bEi2lXcrFEnTdF3Fj/gvAHjUZnDy+5kwc= X-Gm-Message-State: AOJu0Yx3whKFZPqbBVFpV4xMAj9cq3Quh8H+tbCzE9/fMjo9eqFhE4GU tXIMERMgQC/KvNDY+xsLVPwCnP3hxA6ykjvZ9fUYxD8bVYjJDUmhSYjNb5MGEMI= X-Google-Smtp-Source: AGHT+IEILpnFvb14vizaqdHNCAsEefKY8W5fJrghMGpJs4NQI541WsQVt7UAPN57wbPSjoIKa73jbQ== X-Received: by 2002:a05:651c:b06:b0:2f1:6a30:6754 with SMTP id 38308e7fff4ca-2f16a3067dbmr1525221fa.12.1722611981105; Fri, 02 Aug 2024 08:19:41 -0700 (PDT) Received: from alex-rivos.ba.rivosinc.com (amontpellier-656-1-456-62.w92-145.abo.wanadoo.fr. [92.145.124.62]) by smtp.gmail.com with ESMTPSA id 38308e7fff4ca-2f15e17e840sm1828141fa.12.2024.08.02.08.19.39 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 02 Aug 2024 08:19:40 -0700 (PDT) From: Alexandre Ghiti To: Catalin Marinas , Will Deacon , Ryan Roberts , Mark Rutland , Paul Walmsley , Palmer Dabbelt , Albert Ou , Andrew Morton , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org, linux-mm@kvack.org Cc: Alexandre Ghiti Subject: [PATCH v3 5/9] mm: Use common huge_pte_clear() function for riscv/arm64 Date: Fri, 2 Aug 2024 17:14:26 +0200 Message-Id: <20240802151430.99114-6-alexghiti@rivosinc.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20240802151430.99114-1-alexghiti@rivosinc.com> References: <20240802151430.99114-1-alexghiti@rivosinc.com> MIME-Version: 1.0 X-Stat-Signature: c8ytk9kg439c41q555hmqcr947uuhifm X-Rspamd-Queue-Id: 09381C0005 X-Rspam-User: X-Rspamd-Server: rspam10 X-HE-Tag: 1722611982-307504 X-HE-Meta: U2FsdGVkX1+PDZBV7Zq4t1EMyk9GToQemRZohAsPLQsrygQj+5jr/9WaQdNiYs4ET1u7I09rkDQ0qnQim1wy8ZiCOL4xlv/T1ZUlzvY+lglGu/Y3e9/N7MQe9fwomLGZJPVRwZCIxx0EGFz8Q83hRMzCI1YazGUmVkrVIj/vRYNLaIR0AIau+NJhz5P1UMCQ99CKY6e21B2jqzMiZQ/ctfj2AfUZPQwWHTzBmPjMQEXPCCw4fqF/+jNJI65klhCmo2JOeOADLECHSthNRH9F5/NRhFhocBhJNE7IaneGBwi0fDPV7TFLp0eR4uRmGJwikWtmkr3KGy5sfcyPpuiYDwffiVDdxoByQIfL4Gwn/bWBrpku2TDGeVBPtUoFepJej/wH3ydqnTAYdXCZsWNFzxF0uovoEN69W2BAMOtzLOqah3Kzo9pDTJOQMO2aZfq1OjtpDKJo0gX17a3m5Xzheez16wbcDrx6AiN+4i5MfYZNoF0VhFqtidW4LyZ/vtAhfkVVtoMbs9M96eqWaPfzOJDHW5mSGn0nU5fqrKxonl9bFlmH6vA/XCYMsDLi8WBw5TSNe3Y1NzwWmGarxORE05GXira3SWc/AjJM3fMaq5dDChhp5xNkm2r5hggzfXT61oEdPg9iLmIehg38ZuWa/5lleuL50fDk4iwTATikefp9Fhn1Hil2qH7IWuRXk9wzokoLvbekR32VHO6XgbEcTNWwgh4rDuqfAIWQoKo1nVEYlEOSoYhkz33c74Ldkhyyvc+ZZgpHQD4WRY0JP7fEvV0JUaRyTpXy2P0mjqZZ0NypCPQRYcZ/53hqdRuD/ojAFFwFQhKvVc3Zu6QrOOEHY1celupwnoG5qViKY6Ks4CPPEtRQtQ8ZknlydmElfR4P/inULyNlsNHgkmnmKw7aU0X8Gvtum2yZhC3WR7mT8BzMISGZNBKb/YmA2Hg4vHLOv3fNeplm/WshgaSPNu1 exsjYlFz jKgsuTplijQn2F5mUzW7JqHLYq3oPSCfcXw5hQJPsWnixtfatB3TVME9M+TKocL7kA4e5u8y2Q4bEcQsVkrv03aUFrkk9VzbWl2uEF4FT3bWLp7Bz3fTBN8RpXRTFCl6L2ErxBzJBWRNuo9kH1idkDWdIM0vd4PZtV9dfL2tVmPOVSQBpc5FjfQWVUzviomTQL1miCktskYTE4YXeyl3onUBaTlTAzCpsmWZ8o9cibFKfRbzhVp2w1WKN4sWIcRJ56JMAIx/ZutATa0czE2qSr4HdzYlkY6nOHbo07UVS129jGtJDM7abRoCWb7bqowXNrnt03CQxsVmGDh6tRjx2bXh57FJl9SCeyiekBw22Z4xyQnr5YGoZAtklqfYGAVryzWrStSStzzEVWn/BpOniWcy2og== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Both architectures have the same implementation so move it to generic code. Signed-off-by: Alexandre Ghiti --- arch/arm64/include/asm/hugetlb.h | 3 --- arch/arm64/mm/hugetlbpage.c | 12 ------------ arch/riscv/include/asm/hugetlb.h | 4 ---- arch/riscv/include/asm/pgtable.h | 5 +++-- arch/riscv/mm/hugetlbpage.c | 19 ------------------- include/linux/hugetlb_contpte.h | 4 ++++ mm/hugetlb_contpte.c | 14 ++++++++++++++ 7 files changed, 21 insertions(+), 40 deletions(-) diff --git a/arch/arm64/include/asm/hugetlb.h b/arch/arm64/include/asm/hugetlb.h index d35093b7ab59..fab0afbe4eea 100644 --- a/arch/arm64/include/asm/hugetlb.h +++ b/arch/arm64/include/asm/hugetlb.h @@ -40,9 +40,6 @@ extern void huge_ptep_set_wrprotect(struct mm_struct *mm, #define __HAVE_ARCH_HUGE_PTEP_CLEAR_FLUSH extern pte_t huge_ptep_clear_flush(struct vm_area_struct *vma, unsigned long addr, pte_t *ptep); -#define __HAVE_ARCH_HUGE_PTE_CLEAR -extern void huge_pte_clear(struct mm_struct *mm, unsigned long addr, - pte_t *ptep, unsigned long sz); void __init arm64_hugetlb_cma_reserve(void); diff --git a/arch/arm64/mm/hugetlbpage.c b/arch/arm64/mm/hugetlbpage.c index c7130d1f07c4..495f706e0170 100644 --- a/arch/arm64/mm/hugetlbpage.c +++ b/arch/arm64/mm/hugetlbpage.c @@ -266,18 +266,6 @@ pte_t arch_make_huge_pte(pte_t entry, unsigned int shift, vm_flags_t flags) return entry; } -void huge_pte_clear(struct mm_struct *mm, unsigned long addr, - pte_t *ptep, unsigned long sz) -{ - int i, ncontig; - size_t pgsize; - - ncontig = arch_contpte_get_num_contig(ptep, sz, &pgsize); - - for (i = 0; i < ncontig; i++, addr += pgsize, ptep++) - __pte_clear(mm, addr, ptep); -} - pte_t huge_ptep_get_and_clear(struct mm_struct *mm, unsigned long addr, pte_t *ptep) { diff --git a/arch/riscv/include/asm/hugetlb.h b/arch/riscv/include/asm/hugetlb.h index 28cbf5d761e1..ca9930cdf2e6 100644 --- a/arch/riscv/include/asm/hugetlb.h +++ b/arch/riscv/include/asm/hugetlb.h @@ -20,10 +20,6 @@ bool arch_hugetlb_migration_supported(struct hstate *h); #endif #ifdef CONFIG_RISCV_ISA_SVNAPOT -#define __HAVE_ARCH_HUGE_PTE_CLEAR -void huge_pte_clear(struct mm_struct *mm, unsigned long addr, - pte_t *ptep, unsigned long sz); - #define __HAVE_ARCH_HUGE_PTEP_GET_AND_CLEAR pte_t huge_ptep_get_and_clear(struct mm_struct *mm, unsigned long addr, pte_t *ptep); diff --git a/arch/riscv/include/asm/pgtable.h b/arch/riscv/include/asm/pgtable.h index 46b409e558b3..5ab0559b0bd2 100644 --- a/arch/riscv/include/asm/pgtable.h +++ b/arch/riscv/include/asm/pgtable.h @@ -615,8 +615,8 @@ static inline int arch_contpte_get_num_contig(pte_t *ptep, unsigned long size, } #endif -static inline void pte_clear(struct mm_struct *mm, - unsigned long addr, pte_t *ptep) +static inline void __pte_clear(struct mm_struct *mm, + unsigned long addr, pte_t *ptep) { __set_pte_at(mm, ptep, __pte(0)); } @@ -731,6 +731,7 @@ static inline pte_t ptep_get(pte_t *ptep) #define __HAVE_ARCH_PTEP_GET_AND_CLEAR #define ptep_get_and_clear __ptep_get_and_clear +#define pte_clear __pte_clear #define pgprot_nx pgprot_nx static inline pgprot_t pgprot_nx(pgprot_t _prot) diff --git a/arch/riscv/mm/hugetlbpage.c b/arch/riscv/mm/hugetlbpage.c index 0ecb2846c3f0..e2093e7266a5 100644 --- a/arch/riscv/mm/hugetlbpage.c +++ b/arch/riscv/mm/hugetlbpage.c @@ -251,25 +251,6 @@ pte_t huge_ptep_clear_flush(struct vm_area_struct *vma, return get_clear_contig_flush(vma->vm_mm, addr, ptep, pte_num); } -void huge_pte_clear(struct mm_struct *mm, - unsigned long addr, - pte_t *ptep, - unsigned long sz) -{ - size_t pgsize; - pte_t pte = ptep_get(ptep); - int i, pte_num; - - if (!pte_napot(pte)) { - pte_clear(mm, addr, ptep); - return; - } - - pte_num = arch_contpte_get_num_contig(ptep, sz, &pgsize); - for (i = 0; i < pte_num; i++, addr += pgsize, ptep++) - pte_clear(mm, addr, ptep); -} - static bool is_napot_size(unsigned long size) { unsigned long order; diff --git a/include/linux/hugetlb_contpte.h b/include/linux/hugetlb_contpte.h index 7acd734a75e8..d9892a047b2b 100644 --- a/include/linux/hugetlb_contpte.h +++ b/include/linux/hugetlb_contpte.h @@ -14,4 +14,8 @@ extern void set_huge_pte_at(struct mm_struct *mm, unsigned long addr, pte_t *ptep, pte_t pte, unsigned long sz); +#define __HAVE_ARCH_HUGE_PTE_CLEAR +extern void huge_pte_clear(struct mm_struct *mm, unsigned long addr, + pte_t *ptep, unsigned long sz); + #endif /* _LINUX_HUGETLB_CONTPTE_H */ diff --git a/mm/hugetlb_contpte.c b/mm/hugetlb_contpte.c index 9a3a376784b0..b9634ffa1bad 100644 --- a/mm/hugetlb_contpte.c +++ b/mm/hugetlb_contpte.c @@ -12,6 +12,7 @@ * - __ptep_get() * - __set_ptes() * - __ptep_get_and_clear() + * - __pte_clear() * - pte_cont() * - arch_contpte_get_num_contig() */ @@ -20,6 +21,7 @@ * This file implements the following contpte aware API: * - huge_ptep_get() * - set_huge_pte_at() + * - huge_pte_clear() */ pte_t huge_ptep_get(struct mm_struct *mm, unsigned long addr, pte_t *ptep) @@ -101,3 +103,15 @@ void set_huge_pte_at(struct mm_struct *mm, unsigned long addr, set_contptes(mm, addr, ptep, pte, ncontig, pgsize); } + +void huge_pte_clear(struct mm_struct *mm, unsigned long addr, + pte_t *ptep, unsigned long sz) +{ + int i, ncontig; + size_t pgsize; + + ncontig = arch_contpte_get_num_contig(ptep, sz, &pgsize); + + for (i = 0; i < ncontig; i++, addr += pgsize, ptep++) + __pte_clear(mm, addr, ptep); +} From patchwork Fri Aug 2 15:14:27 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexandre Ghiti X-Patchwork-Id: 13751628 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1F1C4C52D6F for ; Fri, 2 Aug 2024 15:20:47 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id AB0476B0096; Fri, 2 Aug 2024 11:20:46 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id A619A6B0098; Fri, 2 Aug 2024 11:20:46 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 901FA6B0099; Fri, 2 Aug 2024 11:20:46 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 731726B0096 for ; Fri, 2 Aug 2024 11:20:46 -0400 (EDT) Received: from smtpin05.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 189974129F for ; Fri, 2 Aug 2024 15:20:46 +0000 (UTC) X-FDA: 82407667692.05.9A73749 Received: from mail-wr1-f50.google.com (mail-wr1-f50.google.com [209.85.221.50]) by imf28.hostedemail.com (Postfix) with ESMTP id 2A096C0024 for ; Fri, 2 Aug 2024 15:20:43 +0000 (UTC) Authentication-Results: imf28.hostedemail.com; dkim=pass header.d=rivosinc-com.20230601.gappssmtp.com header.s=20230601 header.b=QKXsyBZ3; spf=pass (imf28.hostedemail.com: domain of alexghiti@rivosinc.com designates 209.85.221.50 as permitted sender) smtp.mailfrom=alexghiti@rivosinc.com; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1722611967; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=fMaYL2MnyzMLNeppKABE7xd1gAuHqwYBgzfb7XbILdY=; b=0pB+kch4RdVyxLFdOkWbsfIXdMaF98igT2fZP/H6FQbjtVU7cjc9B+9Me408PmnvpFU9dI D5KqOOGlRFQlRorzqt3evT0JquTyxACe3YrXbmQkv/WgkorraeTMqIZzxp1fvP24NDmKwb yRXyC7MilgGypea8ioor+nENyP5RF7I= ARC-Authentication-Results: i=1; imf28.hostedemail.com; dkim=pass header.d=rivosinc-com.20230601.gappssmtp.com header.s=20230601 header.b=QKXsyBZ3; spf=pass (imf28.hostedemail.com: domain of alexghiti@rivosinc.com designates 209.85.221.50 as permitted sender) smtp.mailfrom=alexghiti@rivosinc.com; dmarc=none ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1722611967; a=rsa-sha256; cv=none; b=lDEUQgJssPtqijvg33e5Gozf7goOw4HVUFFwUPhWXkZm70YDjQA1Ghg6NQr8ugDeC+qbI1 76IopLLaVeSznfC6AWilEjVcgZS6p+2kjlAT2nEopTBKzJHR1D+JNvdZ2gn2v1+o6FOAU+ XYx5s8o3Lr+4I54Va81SgmtUiSwyJVE= Received: by mail-wr1-f50.google.com with SMTP id ffacd0b85a97d-36887ca3da2so3849490f8f.2 for ; Fri, 02 Aug 2024 08:20:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc-com.20230601.gappssmtp.com; s=20230601; t=1722612043; x=1723216843; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=fMaYL2MnyzMLNeppKABE7xd1gAuHqwYBgzfb7XbILdY=; b=QKXsyBZ3NPGLrJ941lxhCwrCDncCc20X5tGGTbmehcB7IPMvd5vGqXlQz6tPZemi7W 2Ge07nmMhnvJnh1pxupFq99bvBUZXLsVRODE0q9bC7cnkgiMA6FwxgdEssIxvNPRIztL v/IoOnWYPKK9miYhesL1DOUCNlm+580UjTMggDVoornn29RT/YewpWxYMEamVX5fs4IB zsUeezrC/6bAuQuinrzOTJEvHn+bdDtgcbhWeZWdyuGB0E4rQ6LmiISUs/lbURC1NduW cGPpBRtsNsHU/IpbMObdraYhIl6YH2bGKLcTqXE0I5d1RjibSn6F2sd9bxz2vqPCcF7H lZEA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1722612043; x=1723216843; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=fMaYL2MnyzMLNeppKABE7xd1gAuHqwYBgzfb7XbILdY=; b=SdNt4qqRPOJC2Y5DzQQXuCU4ZofmeQ6ZDVZ7T2hfx5vIrr4Iv5egyNllUyeOtI/yZn LNQ6q4xLr/oJvhM/9DhXj6wXqcMNUPuLXBJFymcuRT6EpqEkedjv3SecaRTEhNyqaa4C fXMtEoEBKE1By0NKG1OrxLURLG0nGsSCINZEn2eoiUyLL7eW23/W1Htr4ACMfVq7ksgX EHRl189I/aHEIvRhb4bHJTPijm+HZdiI2i1cbSTVvtIxdShroWPuSzUHYffSRyZ+mHUd SHvGOYc3olheLCCfOcWBdOmyxbL8/v6VIff8m7vvO+LUMq4m4uaBoazwPzx8t+CtNFA6 el1w== X-Forwarded-Encrypted: i=1; AJvYcCWbpCVxFWQqwAaTbRuK9yAN7JLtxbQc/JJ6lil1v8Qk9UBYFxeO621oy6AaucqeJtlgKxHfVkcvLxbIBClwDZbSYcU= X-Gm-Message-State: AOJu0YxWkmLaCtJhT/1ECTQgs1brXMgKTqDduOVRlsm88hNWNq0eIKDs f7M8BYud6PN8mYrmhbPIUTH3nCSm4dF6xxEN25GToO40NAk/bTL/1Jnn9dP3YUo= X-Google-Smtp-Source: AGHT+IGTa0t3MKbLVg4kZPwSxPiJXdJvhDxgkhF/gxXss7ykdb1ycHNk2hMqIUX7HnfDufjyI9Oq4w== X-Received: by 2002:adf:fc03:0:b0:368:37ac:3f95 with SMTP id ffacd0b85a97d-36bbc11bb28mr2076124f8f.31.1722612042502; Fri, 02 Aug 2024 08:20:42 -0700 (PDT) Received: from alex-rivos.ba.rivosinc.com (amontpellier-656-1-456-62.w92-145.abo.wanadoo.fr. [92.145.124.62]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-36bbd06d078sm2177020f8f.94.2024.08.02.08.20.41 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 02 Aug 2024 08:20:42 -0700 (PDT) From: Alexandre Ghiti To: Catalin Marinas , Will Deacon , Ryan Roberts , Mark Rutland , Paul Walmsley , Palmer Dabbelt , Albert Ou , Andrew Morton , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org, linux-mm@kvack.org Cc: Alexandre Ghiti Subject: [PATCH v3 6/9] mm: Use common huge_ptep_get_and_clear() function for riscv/arm64 Date: Fri, 2 Aug 2024 17:14:27 +0200 Message-Id: <20240802151430.99114-7-alexghiti@rivosinc.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20240802151430.99114-1-alexghiti@rivosinc.com> References: <20240802151430.99114-1-alexghiti@rivosinc.com> MIME-Version: 1.0 X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: 2A096C0024 X-Stat-Signature: ogqkxq6puy186tantag3rwtqxxj6zig6 X-Rspam-User: X-HE-Tag: 1722612043-795369 X-HE-Meta: U2FsdGVkX19UbFYG+ad40sRcUfNPAOwkvJUncYlkQtL1T8iDrBK6lra6yohoMZVxox6Jxxy84D2nERqvh3U2/o7rh6amjpzC3rO6B38/uR+wXdmgwYwuCwKbyes+Tjhs4nrnNhi9oljdHVrw4edtwNIXWsEFDbo2O2AuHAx4hJqxSww7AKQI4MuUB/rhpi8CxJhtNjp6qZ6GkcSu6Y9+uMS+OLIPbZvvETmj+QkyTaAyzvFSL6Gjsq7Kg2mFULe+nhQLNS2wKEjLfXFS/17V96I2umjg5d4TQ0sTU3PeuukXPZYfIFYL4HjtvLjVKN7x1AdI42AKs55BtgejUy1cccsKDiv7lJVJ21uVhwRm/Pd4kyPdnaqqDoG7jkmh3dMTZKVuv8oW+ntsFVDge8HcirxLWjxyyEqyk8eSdYj2xM/vRmPtCotkbNwg6c9WlN8I1g1+kPVrDlD7hJj5GF0qEia7b3SZM/21GcAS6UY39E31zDxjPhwKgQfMjG2M7Ma9aFGOQeZ+SaYEVwaXiZPtRIa/uiTgouHFQqtQUC7XglfdCJmS95I/U7860EJC/3vMZm9j3YAZWRf6kX6QyAcXsvLZdRfNK2ROkrdT/MiQjnjBNVOYIXxL6t6+EdygO6RElM46wKJQ1dTBK9u1pgg+4o4antsv4s5NDSnqJNUcer7+T+BGyzN4h5dCquFwolR9mKmNEQHSBU8Cf8Zj2+Ae9MDxSVscDb8DsAB0FoU7h88DECMWg9tDKpxZd21FGgb2Df4E6wUob5M6XzUcwYkc8xgo/yaWFQW0GUJy4SCi6DnuCsktLc+Pzqd3KNkIYc4pCLKElLEELfaS1Fb9naQ/x+U7jPF3z83wniTX9FAbYPb3hRxIphdC7mfCqC5h1uLChkijNmLkQsALoUZnnU3BJ/NOyms5/sq4BWGGpwUMRBdDMiBIgkjZAQeaULm9AC2xtz8TipoWSYeG6PRU4um k75fwU0d Bj1e50uagsrj0P9Wup9ZHUvm0ZocvvI64q39tooHW4YS6aMj8wSdwSnvetuKuRQiKXIhXQgaK5/qnRxYK5yeQe2DXtxaUN2Bgp2h52eLxUxU0VsqCpGktlMnN+1lgBpmu1OxFQFkjUPaOFhkotpulSU7BrvB2JZ461f1JFXi0xn7KGfKC90R8NhhtmHOcdQJSbYkue8u4a6GCsJ67/9dmf3MfH07ufym5XtS7QIcqBt3v8m/x2OnOrITgGYo7wE28ubyOtdSemT0ngis9irP9tXgPmHJofWngzwUcvZL4UuKGDCCpxE/ZYY0bxiG7064/y9USYQ2p4SzzxKBgmctK3bksgDPsZUIFe21q6Vzd5djXqHtta3QFl/88Xy4AfKmO+Y247mbDE3zmlhCFYYuaN6250g== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: After some adjustments, both architectures have the same implementation so move it to the generic code. Note that get_clear_contig() function is duplicated in the generic and the arm64 code because it is still used by some arm64 functions that will, in the next commits, be moved to the generic code. Once all have been moved, the arm64 version will be removed. Signed-off-by: Alexandre Ghiti --- arch/arm64/include/asm/hugetlb.h | 3 -- arch/arm64/include/asm/pgtable.h | 15 ++++++++-- arch/arm64/mm/hugetlbpage.c | 19 ++----------- arch/riscv/include/asm/hugetlb.h | 4 --- arch/riscv/include/asm/pgtable.h | 4 ++- arch/riscv/mm/hugetlbpage.c | 23 ++++----------- include/linux/hugetlb_contpte.h | 4 +++ mm/hugetlb_contpte.c | 48 ++++++++++++++++++++++++++++++-- 8 files changed, 72 insertions(+), 48 deletions(-) diff --git a/arch/arm64/include/asm/hugetlb.h b/arch/arm64/include/asm/hugetlb.h index fab0afbe4eea..04117a84dc04 100644 --- a/arch/arm64/include/asm/hugetlb.h +++ b/arch/arm64/include/asm/hugetlb.h @@ -31,9 +31,6 @@ pte_t arch_make_huge_pte(pte_t entry, unsigned int shift, vm_flags_t flags); extern int huge_ptep_set_access_flags(struct vm_area_struct *vma, unsigned long addr, pte_t *ptep, pte_t pte, int dirty); -#define __HAVE_ARCH_HUGE_PTEP_GET_AND_CLEAR -extern pte_t huge_ptep_get_and_clear(struct mm_struct *mm, - unsigned long addr, pte_t *ptep); #define __HAVE_ARCH_HUGE_PTEP_SET_WRPROTECT extern void huge_ptep_set_wrprotect(struct mm_struct *mm, unsigned long addr, pte_t *ptep); diff --git a/arch/arm64/include/asm/pgtable.h b/arch/arm64/include/asm/pgtable.h index 2a6a01b447df..4b277ff9810d 100644 --- a/arch/arm64/include/asm/pgtable.h +++ b/arch/arm64/include/asm/pgtable.h @@ -1766,12 +1766,23 @@ static inline void clear_young_dirty_ptes(struct vm_area_struct *vma, #endif /* CONFIG_ARM64_CONTPTE */ -static inline int arch_contpte_get_num_contig(pte_t *ptep, - unsigned long size, +extern int find_num_contig(struct mm_struct *mm, unsigned long addr, + pte_t *ptep, size_t *pgsize); + +static inline int arch_contpte_get_num_contig(struct mm_struct *mm, + unsigned long addr, + pte_t *ptep, unsigned long size, size_t *pgsize) { int contig_ptes = 0; + /* + * If the size is not passed, we need to go through the page table to + * find out the number of contiguous ptes. + */ + if (size == 0) + return find_num_contig(mm, addr, ptep, pgsize); + if (pgsize) *pgsize = size; diff --git a/arch/arm64/mm/hugetlbpage.c b/arch/arm64/mm/hugetlbpage.c index 495f706e0170..2f3753c10c1f 100644 --- a/arch/arm64/mm/hugetlbpage.c +++ b/arch/arm64/mm/hugetlbpage.c @@ -79,8 +79,8 @@ bool arch_hugetlb_migration_supported(struct hstate *h) } #endif -static int find_num_contig(struct mm_struct *mm, unsigned long addr, - pte_t *ptep, size_t *pgsize) +int find_num_contig(struct mm_struct *mm, unsigned long addr, + pte_t *ptep, size_t *pgsize) { pgd_t *pgdp = pgd_offset(mm, addr); p4d_t *p4dp; @@ -266,21 +266,6 @@ pte_t arch_make_huge_pte(pte_t entry, unsigned int shift, vm_flags_t flags) return entry; } -pte_t huge_ptep_get_and_clear(struct mm_struct *mm, - unsigned long addr, pte_t *ptep) -{ - int ncontig; - size_t pgsize; - pte_t orig_pte = __ptep_get(ptep); - - if (!pte_cont(orig_pte)) - return __ptep_get_and_clear(mm, addr, ptep); - - ncontig = find_num_contig(mm, addr, ptep, &pgsize); - - return get_clear_contig(mm, addr, ptep, pgsize, ncontig); -} - /* * huge_ptep_set_access_flags will update access flags (dirty, accesssed) * and write permission. diff --git a/arch/riscv/include/asm/hugetlb.h b/arch/riscv/include/asm/hugetlb.h index ca9930cdf2e6..0fbb6b19df79 100644 --- a/arch/riscv/include/asm/hugetlb.h +++ b/arch/riscv/include/asm/hugetlb.h @@ -20,10 +20,6 @@ bool arch_hugetlb_migration_supported(struct hstate *h); #endif #ifdef CONFIG_RISCV_ISA_SVNAPOT -#define __HAVE_ARCH_HUGE_PTEP_GET_AND_CLEAR -pte_t huge_ptep_get_and_clear(struct mm_struct *mm, - unsigned long addr, pte_t *ptep); - #define __HAVE_ARCH_HUGE_PTEP_CLEAR_FLUSH pte_t huge_ptep_clear_flush(struct vm_area_struct *vma, unsigned long addr, pte_t *ptep); diff --git a/arch/riscv/include/asm/pgtable.h b/arch/riscv/include/asm/pgtable.h index 5ab0559b0bd2..13e89348a46a 100644 --- a/arch/riscv/include/asm/pgtable.h +++ b/arch/riscv/include/asm/pgtable.h @@ -584,7 +584,9 @@ static inline void __set_ptes(struct mm_struct *mm, unsigned long addr, } #ifdef CONFIG_RISCV_ISA_SVNAPOT -static inline int arch_contpte_get_num_contig(pte_t *ptep, unsigned long size, +static inline int arch_contpte_get_num_contig(struct mm_struct *mm, + unsigned long addr, + pte_t *ptep, unsigned long size, size_t *pgsize) { unsigned long hugepage_shift; diff --git a/arch/riscv/mm/hugetlbpage.c b/arch/riscv/mm/hugetlbpage.c index e2093e7266a5..b44023336fd9 100644 --- a/arch/riscv/mm/hugetlbpage.c +++ b/arch/riscv/mm/hugetlbpage.c @@ -186,7 +186,8 @@ int huge_ptep_set_access_flags(struct vm_area_struct *vma, if (!pte_napot(pte)) return ptep_set_access_flags(vma, addr, ptep, pte, dirty); - pte_num = arch_contpte_get_num_contig(ptep, 0, NULL); + pte_num = arch_contpte_get_num_contig(vma->vm_mm, addr, ptep, 0, NULL); + orig_pte = get_clear_contig_flush(mm, addr, ptep, pte_num); if (pte_dirty(orig_pte)) @@ -200,21 +201,6 @@ int huge_ptep_set_access_flags(struct vm_area_struct *vma, return true; } -pte_t huge_ptep_get_and_clear(struct mm_struct *mm, - unsigned long addr, - pte_t *ptep) -{ - pte_t orig_pte = ptep_get(ptep); - int pte_num; - - if (!pte_napot(orig_pte)) - return ptep_get_and_clear(mm, addr, ptep); - - pte_num = arch_contpte_get_num_contig(ptep, 0, NULL); - - return get_clear_contig(mm, addr, ptep, pte_num); -} - void huge_ptep_set_wrprotect(struct mm_struct *mm, unsigned long addr, pte_t *ptep) @@ -228,7 +214,8 @@ void huge_ptep_set_wrprotect(struct mm_struct *mm, return; } - pte_num = arch_contpte_get_num_contig(ptep, 0, NULL); + pte_num = arch_contpte_get_num_contig(mm, addr, ptep, 0, NULL); + orig_pte = get_clear_contig_flush(mm, addr, ptep, pte_num); orig_pte = pte_wrprotect(orig_pte); @@ -246,7 +233,7 @@ pte_t huge_ptep_clear_flush(struct vm_area_struct *vma, if (!pte_napot(pte)) return ptep_clear_flush(vma, addr, ptep); - pte_num = arch_contpte_get_num_contig(ptep, 0, NULL); + pte_num = arch_contpte_get_num_contig(vma->vm_mm, addr, ptep, 0, NULL); return get_clear_contig_flush(vma->vm_mm, addr, ptep, pte_num); } diff --git a/include/linux/hugetlb_contpte.h b/include/linux/hugetlb_contpte.h index d9892a047b2b..20d3a3e14e14 100644 --- a/include/linux/hugetlb_contpte.h +++ b/include/linux/hugetlb_contpte.h @@ -18,4 +18,8 @@ extern void set_huge_pte_at(struct mm_struct *mm, extern void huge_pte_clear(struct mm_struct *mm, unsigned long addr, pte_t *ptep, unsigned long sz); +#define __HAVE_ARCH_HUGE_PTEP_GET_AND_CLEAR +extern pte_t huge_ptep_get_and_clear(struct mm_struct *mm, + unsigned long addr, pte_t *ptep); + #endif /* _LINUX_HUGETLB_CONTPTE_H */ diff --git a/mm/hugetlb_contpte.c b/mm/hugetlb_contpte.c index b9634ffa1bad..9df98d1a59a1 100644 --- a/mm/hugetlb_contpte.c +++ b/mm/hugetlb_contpte.c @@ -22,6 +22,7 @@ * - huge_ptep_get() * - set_huge_pte_at() * - huge_pte_clear() + * - huge_ptep_get_and_clear() */ pte_t huge_ptep_get(struct mm_struct *mm, unsigned long addr, pte_t *ptep) @@ -32,7 +33,7 @@ pte_t huge_ptep_get(struct mm_struct *mm, unsigned long addr, pte_t *ptep) if (!pte_present(orig_pte) || !pte_cont(orig_pte)) return orig_pte; - ncontig = arch_contpte_get_num_contig(ptep, + ncontig = arch_contpte_get_num_contig(mm, addr, ptep, page_size(pte_page(orig_pte)), NULL); @@ -86,7 +87,7 @@ void set_huge_pte_at(struct mm_struct *mm, unsigned long addr, int i; int ncontig; - ncontig = arch_contpte_get_num_contig(ptep, sz, &pgsize); + ncontig = arch_contpte_get_num_contig(mm, addr, ptep, sz, &pgsize); if (!pte_present(pte)) { for (i = 0; i < ncontig; i++, ptep++, addr += pgsize) @@ -110,8 +111,49 @@ void huge_pte_clear(struct mm_struct *mm, unsigned long addr, int i, ncontig; size_t pgsize; - ncontig = arch_contpte_get_num_contig(ptep, sz, &pgsize); + ncontig = arch_contpte_get_num_contig(mm, addr, ptep, sz, &pgsize); for (i = 0; i < ncontig; i++, addr += pgsize, ptep++) __pte_clear(mm, addr, ptep); } + +static pte_t get_clear_contig(struct mm_struct *mm, + unsigned long addr, + pte_t *ptep, + unsigned long pgsize, + unsigned long ncontig) +{ + pte_t orig_pte = __ptep_get(ptep); + unsigned long i; + + for (i = 0; i < ncontig; i++, addr += pgsize, ptep++) { + pte_t pte = __ptep_get_and_clear(mm, addr, ptep); + + /* + * If HW_AFDBM (arm64) or Svadu (riscv) is enabled, then the HW + * could turn on the dirty or accessed bit for any page in the + * set, so check them all. + */ + if (pte_dirty(pte)) + orig_pte = pte_mkdirty(orig_pte); + + if (pte_young(pte)) + orig_pte = pte_mkyoung(orig_pte); + } + return orig_pte; +} + +pte_t huge_ptep_get_and_clear(struct mm_struct *mm, + unsigned long addr, pte_t *ptep) +{ + int ncontig; + size_t pgsize; + pte_t orig_pte = __ptep_get(ptep); + + if (!pte_cont(orig_pte)) + return __ptep_get_and_clear(mm, addr, ptep); + + ncontig = arch_contpte_get_num_contig(mm, addr, ptep, 0, &pgsize); + + return get_clear_contig(mm, addr, ptep, pgsize, ncontig); +} From patchwork Fri Aug 2 15:14:28 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexandre Ghiti X-Patchwork-Id: 13751652 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4261DC52D6F for ; Fri, 2 Aug 2024 15:21:48 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D49346B0089; Fri, 2 Aug 2024 11:21:47 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id CF93F6B0093; Fri, 2 Aug 2024 11:21:47 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id BC1226B0099; Fri, 2 Aug 2024 11:21:47 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id A105C6B0089 for ; Fri, 2 Aug 2024 11:21:47 -0400 (EDT) Received: from smtpin18.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 546351C4480 for ; Fri, 2 Aug 2024 15:21:47 +0000 (UTC) X-FDA: 82407670254.18.8B0864F Received: from mail-wm1-f46.google.com (mail-wm1-f46.google.com [209.85.128.46]) by imf10.hostedemail.com (Postfix) with ESMTP id 6394AC0029 for ; Fri, 2 Aug 2024 15:21:45 +0000 (UTC) Authentication-Results: imf10.hostedemail.com; dkim=pass header.d=rivosinc-com.20230601.gappssmtp.com header.s=20230601 header.b="Ppzy/dJR"; dmarc=none; spf=pass (imf10.hostedemail.com: domain of alexghiti@rivosinc.com designates 209.85.128.46 as permitted sender) smtp.mailfrom=alexghiti@rivosinc.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1722612062; a=rsa-sha256; cv=none; b=Wq4LayvrTdsK3EHpBiBXOhQb6EEGr0YAUMZxCWdN5Ru73J9ZJIr0VPHtxpakvxRDay+Sad upinJCbpARbxXuFttYe8IoANO77gwRsUlKtISmX8dibycTx+cR4/FZbrGnYX//+dKt4cj2 hvFbldZqBeCKYrhkiZZgmBV/CfcFqF8= ARC-Authentication-Results: i=1; imf10.hostedemail.com; dkim=pass header.d=rivosinc-com.20230601.gappssmtp.com header.s=20230601 header.b="Ppzy/dJR"; dmarc=none; spf=pass (imf10.hostedemail.com: domain of alexghiti@rivosinc.com designates 209.85.128.46 as permitted sender) smtp.mailfrom=alexghiti@rivosinc.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1722612062; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=dJwxj4Q/epTCsuuoDx5TluW633PRo5bWcwGJkWSnI+E=; b=SGmTX3jP6Hclgwrh6C5/vMpj7h+KtfXPdxIkEAGBo2EjhqIYF7WBVYaZT1P8LKbJQqvoxI uJuGL4k4qs8qRPuJtSv7+YwhkauihZeYuDYN0cT1uCNSTXHftTP+rlm4GJrCxd9d1Cz67o vMvwtOQMuNE1rv58M1HiASR92rwTDYQ= Received: by mail-wm1-f46.google.com with SMTP id 5b1f17b1804b1-42816ca782dso54237795e9.2 for ; Fri, 02 Aug 2024 08:21:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc-com.20230601.gappssmtp.com; s=20230601; t=1722612104; x=1723216904; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=dJwxj4Q/epTCsuuoDx5TluW633PRo5bWcwGJkWSnI+E=; b=Ppzy/dJRcKxZRp8ZPnSd268QxgJ0MsJQO0laJjHGbvSvrzGQfYb6P+f2EEvBma8VDb XkJZhW1+QM0MRIGdYUo2hCHkP+LvqflUVKHBLkYjC8uf5yT7ib97oM/BnKjgk6tG2RkU DtQBXxPZaUwF1s2CsYrXF6krc9vPKd8kKi30IPT8Iw86uc93X6jQB4eH7CVrpI2l80y3 GUUEaMPfkYkBlqnLkhJILQ/BbRwwLO39Ma145WX1dFXlwMM4jcc8jzb05C4uog7RP61x GdhJgxp4PCl9+2bc4D8BmJsFBJDpt/oEyfplUt2dyRQHChHF8xx2SshvJFcshpA9rh/9 QwMg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1722612104; x=1723216904; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=dJwxj4Q/epTCsuuoDx5TluW633PRo5bWcwGJkWSnI+E=; b=gtHfR54GSblA8YZNcY/hOZQs/MiIGCToxjvqsdisMllLjoO3oY1l6rXiQBWt+Fq/4E N/F/7kO+ZJjSxbs9n71sLOxO9ry8KVyIaj3lbaUDzvbYUzlECNACBxxKYQy/B2uhLmbD QZd4TW+C1U5KtAk5QEwS2o7caM0UPXeZXWfA1El5x5S0Na+AbEJYAi12r6gAEkAomkiO djyHCx3fhfD/aw/6DXoyNDGZg1M10CmCP9wO5/mL9bBIK6lGAmdpS4xsRGwW7Xoy0LPg 1BWffDdMcgqQww7pyvsG0CF36iD1XUJVL03S9NlaZMl3T+x03k5bHCaU/r7E8rMQ0K/M UoRg== X-Forwarded-Encrypted: i=1; AJvYcCWjmiI9wGHFT04D0vPol+S9d53yvJmN0WenkUWzsQ6a1/O3PXx1zEtiHhu6pAi61SHY3Wx9x5yeYi0pakEEMRqhq64= X-Gm-Message-State: AOJu0Yxd1x50uzVPdqLHgn8q6dlaiPhzGZc/lrIOTbYEPX34myzsSy6k lZSFprpOwt6aK/vBJVCMveNcDyFAGkn4X5/9XLprnCbcSj/t7zfq8dbvUAf1BcE= X-Google-Smtp-Source: AGHT+IHs8c35/MVJPJWHbhThNLdZDWmclnMPyTlHp9e2Mj7pB+l3jgAdGFRRGTJ474zJ6huZaRujOg== X-Received: by 2002:a05:600c:a45:b0:428:1694:bd8c with SMTP id 5b1f17b1804b1-428e6b938dcmr22007775e9.37.1722612103839; Fri, 02 Aug 2024 08:21:43 -0700 (PDT) Received: from alex-rivos.ba.rivosinc.com (amontpellier-656-1-456-62.w92-145.abo.wanadoo.fr. [92.145.124.62]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-4282bb9d464sm98619985e9.42.2024.08.02.08.21.43 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 02 Aug 2024 08:21:43 -0700 (PDT) From: Alexandre Ghiti To: Catalin Marinas , Will Deacon , Ryan Roberts , Mark Rutland , Paul Walmsley , Palmer Dabbelt , Albert Ou , Andrew Morton , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org, linux-mm@kvack.org Cc: Alexandre Ghiti Subject: [PATCH v3 7/9] mm: Use common huge_ptep_set_access_flags() function for riscv/arm64 Date: Fri, 2 Aug 2024 17:14:28 +0200 Message-Id: <20240802151430.99114-8-alexghiti@rivosinc.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20240802151430.99114-1-alexghiti@rivosinc.com> References: <20240802151430.99114-1-alexghiti@rivosinc.com> MIME-Version: 1.0 X-Rspamd-Queue-Id: 6394AC0029 X-Rspam-User: X-Rspamd-Server: rspam05 X-Stat-Signature: wx8d13y4hrxjdab9sz5odwdiidfanm8h X-HE-Tag: 1722612105-281742 X-HE-Meta: U2FsdGVkX1/32UT83lgue38ZVACsZgOX4RJocrl3jl6g58HTYV7BzNpMR5+08LPO08a8z5t2ThCZBtwg66DitE2HT2nje8JSgocD3vEkfUjdnYDr54clUbubP1a9j3pL8z8EFIJdspReF54gxe7wz1JQ9O4sQxZtviVgU6V7pL+kSKooBR5aESLCagavrKmTQmXvCnQN7j6LbQssB00/S4G9mPfNOtkuQKe+IYX0z2H0mqPMvXO1xq+93LeOfXxLDT27Z5rHilvVJrH9Nx6apqKmxURxbPi7clVkKNvu76120LDmaI4+Gpwp3FqsyLpYBmQkOA7xBI+j8V1J6EjGNp23ays14Pgm3LK13eNXMPC/eTs4j889XjLsvb0KQ4UuYKdsDM4OzzHkBLmorGjn9jTZnRu2o3WnBnAyrUPR4MUaTh2mYyfjRRg1zDCBJ1ZMMU+ptbtfWY4zJ7guASPtx+bADHm5AjdQqCaRy8kOJ+hD7UMQYEslHYCX7HaVGA96C+c1QqECnPTUGRrMRT9LISFsKdTgNBol6OJ6IQz2pktfrI9Qg8N2BKgYrYu9l4LdjURYNnzHQ/EDOi9OloHtlSwjqS5AI1PmM3lASmsiwv2BELmVK9JCYIJbYt4//EWAu8i4aZzNEEjbl3sdLjMVYjzZhx78Zp1vnX5kyf/BCHiYM4/b6Za/GWyVPDwaJPTCcti1TZ/dgahIdq9cy2562PaQAKg4NhQTfLyKiIf/qobCSZLVJGZwG59mfcl2EbAXWfHUq6eHw7l/qcLgKO4e7roh+BJ1+MW8zFV9h/n1JjcBiIm1juIzlEozYU+GZCtW8cHXJMGQGXIg0TLABxYlFa3shnrt8oHKpGdlE0oxg6+tHgBtmpnZ376AZlwC5dkdacZNyTCGd2yIbfjlFevLCy0wlnQpbGt5nQRifegTjKzIw+M7Q6dhhoF0AulCFvzsbxCD8cXe6trehdg/yXD 1riizJCQ /u7pGoRzvJUwuHCQTql3GODc5HvK5dgIQoKjWwjDJPEN5aAThB23414IBFGhSKJuftJ7ILqJ23XkaoSEo3HqjfeVD7pGWzGvLoDCOusa+STRMaQEYRemUw66zNEn2EzFH68bt4PgtvZXG1i53BoMZVGqpyH57Gwmr/iPZDS2PIW2KMQSpz7OX2JdsZ29pZ3uZhiXXKircfg02/9ZGJdCgBP7XDArVnpK3S+XHRlgzMQ+8opkewM5IsVRKJbrmhHfu71X5HPzSO/IUYdsKlL2NrUTAIMA6yncyfGwAHOFLukJD22Q+TCZ0HUpZfCSuXZQq1Kh/QbfqrxEewCb2ZTcReQgBUnsYnkC2KVO9sAuNsSGjFSuYC+o8ebKsVmnh1qbiJQbuQD99eUNN/zjw5M1I6dRWGQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Both architectures have almost the same implementation: __cont_access_flags_changed() is also correct on riscv and brings the same benefits (ie don't do anything if the flags are unchanged). As in the previous commit, get_clear_contig_flush() is duplicated in both the arch and the generic codes, it will be removed from the arch code when the last reference there gets moved to the generic code. Signed-off-by: Alexandre Ghiti --- arch/arm64/include/asm/hugetlb.h | 4 -- arch/arm64/mm/hugetlbpage.c | 65 --------------------------- arch/riscv/include/asm/hugetlb.h | 5 --- arch/riscv/include/asm/pgtable.h | 7 +-- arch/riscv/mm/hugetlbpage.c | 28 ------------ arch/riscv/mm/pgtable.c | 6 +-- include/linux/hugetlb_contpte.h | 5 +++ mm/hugetlb_contpte.c | 75 ++++++++++++++++++++++++++++++++ 8 files changed, 87 insertions(+), 108 deletions(-) diff --git a/arch/arm64/include/asm/hugetlb.h b/arch/arm64/include/asm/hugetlb.h index 04117a84dc04..6b84e287b72d 100644 --- a/arch/arm64/include/asm/hugetlb.h +++ b/arch/arm64/include/asm/hugetlb.h @@ -27,10 +27,6 @@ static inline void arch_clear_hugetlb_flags(struct folio *folio) pte_t arch_make_huge_pte(pte_t entry, unsigned int shift, vm_flags_t flags); #define arch_make_huge_pte arch_make_huge_pte -#define __HAVE_ARCH_HUGE_PTEP_SET_ACCESS_FLAGS -extern int huge_ptep_set_access_flags(struct vm_area_struct *vma, - unsigned long addr, pte_t *ptep, - pte_t pte, int dirty); #define __HAVE_ARCH_HUGE_PTEP_SET_WRPROTECT extern void huge_ptep_set_wrprotect(struct mm_struct *mm, unsigned long addr, pte_t *ptep); diff --git a/arch/arm64/mm/hugetlbpage.c b/arch/arm64/mm/hugetlbpage.c index 2f3753c10c1f..658635022e4d 100644 --- a/arch/arm64/mm/hugetlbpage.c +++ b/arch/arm64/mm/hugetlbpage.c @@ -266,71 +266,6 @@ pte_t arch_make_huge_pte(pte_t entry, unsigned int shift, vm_flags_t flags) return entry; } -/* - * huge_ptep_set_access_flags will update access flags (dirty, accesssed) - * and write permission. - * - * For a contiguous huge pte range we need to check whether or not write - * permission has to change only on the first pte in the set. Then for - * all the contiguous ptes we need to check whether or not there is a - * discrepancy between dirty or young. - */ -static int __cont_access_flags_changed(pte_t *ptep, pte_t pte, int ncontig) -{ - int i; - - if (pte_write(pte) != pte_write(__ptep_get(ptep))) - return 1; - - for (i = 0; i < ncontig; i++) { - pte_t orig_pte = __ptep_get(ptep + i); - - if (pte_dirty(pte) != pte_dirty(orig_pte)) - return 1; - - if (pte_young(pte) != pte_young(orig_pte)) - return 1; - } - - return 0; -} - -int huge_ptep_set_access_flags(struct vm_area_struct *vma, - unsigned long addr, pte_t *ptep, - pte_t pte, int dirty) -{ - int ncontig, i; - size_t pgsize = 0; - unsigned long pfn = pte_pfn(pte), dpfn; - struct mm_struct *mm = vma->vm_mm; - pgprot_t hugeprot; - pte_t orig_pte; - - if (!pte_cont(pte)) - return __ptep_set_access_flags(vma, addr, ptep, pte, dirty); - - ncontig = find_num_contig(mm, addr, ptep, &pgsize); - dpfn = pgsize >> PAGE_SHIFT; - - if (!__cont_access_flags_changed(ptep, pte, ncontig)) - return 0; - - orig_pte = get_clear_contig_flush(mm, addr, ptep, pgsize, ncontig); - - /* Make sure we don't lose the dirty or young state */ - if (pte_dirty(orig_pte)) - pte = pte_mkdirty(pte); - - if (pte_young(orig_pte)) - pte = pte_mkyoung(pte); - - hugeprot = pte_pgprot(pte); - for (i = 0; i < ncontig; i++, ptep++, addr += pgsize, pfn += dpfn) - __set_ptes(mm, addr, ptep, pfn_pte(pfn, hugeprot), 1); - - return 1; -} - void huge_ptep_set_wrprotect(struct mm_struct *mm, unsigned long addr, pte_t *ptep) { diff --git a/arch/riscv/include/asm/hugetlb.h b/arch/riscv/include/asm/hugetlb.h index 0fbb6b19df79..bf533c2cef84 100644 --- a/arch/riscv/include/asm/hugetlb.h +++ b/arch/riscv/include/asm/hugetlb.h @@ -28,11 +28,6 @@ pte_t huge_ptep_clear_flush(struct vm_area_struct *vma, void huge_ptep_set_wrprotect(struct mm_struct *mm, unsigned long addr, pte_t *ptep); -#define __HAVE_ARCH_HUGE_PTEP_SET_ACCESS_FLAGS -int huge_ptep_set_access_flags(struct vm_area_struct *vma, - unsigned long addr, pte_t *ptep, - pte_t pte, int dirty); - pte_t arch_make_huge_pte(pte_t entry, unsigned int shift, vm_flags_t flags); #define arch_make_huge_pte arch_make_huge_pte diff --git a/arch/riscv/include/asm/pgtable.h b/arch/riscv/include/asm/pgtable.h index 13e89348a46a..5d9b051ac51c 100644 --- a/arch/riscv/include/asm/pgtable.h +++ b/arch/riscv/include/asm/pgtable.h @@ -623,9 +623,8 @@ static inline void __pte_clear(struct mm_struct *mm, __set_pte_at(mm, ptep, __pte(0)); } -#define __HAVE_ARCH_PTEP_SET_ACCESS_FLAGS /* defined in mm/pgtable.c */ -extern int ptep_set_access_flags(struct vm_area_struct *vma, unsigned long address, - pte_t *ptep, pte_t entry, int dirty); +extern int __ptep_set_access_flags(struct vm_area_struct *vma, unsigned long address, + pte_t *ptep, pte_t entry, int dirty); #define __HAVE_ARCH_PTEP_TEST_AND_CLEAR_YOUNG /* defined in mm/pgtable.c */ extern int ptep_test_and_clear_young(struct vm_area_struct *vma, unsigned long address, pte_t *ptep); @@ -734,6 +733,8 @@ static inline pte_t ptep_get(pte_t *ptep) #define __HAVE_ARCH_PTEP_GET_AND_CLEAR #define ptep_get_and_clear __ptep_get_and_clear #define pte_clear __pte_clear +#define __HAVE_ARCH_PTEP_SET_ACCESS_FLAGS +#define ptep_set_access_flags __ptep_set_access_flags #define pgprot_nx pgprot_nx static inline pgprot_t pgprot_nx(pgprot_t _prot) diff --git a/arch/riscv/mm/hugetlbpage.c b/arch/riscv/mm/hugetlbpage.c index b44023336fd9..0e2ca7327479 100644 --- a/arch/riscv/mm/hugetlbpage.c +++ b/arch/riscv/mm/hugetlbpage.c @@ -173,34 +173,6 @@ pte_t arch_make_huge_pte(pte_t entry, unsigned int shift, vm_flags_t flags) return entry; } -int huge_ptep_set_access_flags(struct vm_area_struct *vma, - unsigned long addr, - pte_t *ptep, - pte_t pte, - int dirty) -{ - struct mm_struct *mm = vma->vm_mm; - pte_t orig_pte; - int pte_num; - - if (!pte_napot(pte)) - return ptep_set_access_flags(vma, addr, ptep, pte, dirty); - - pte_num = arch_contpte_get_num_contig(vma->vm_mm, addr, ptep, 0, NULL); - - orig_pte = get_clear_contig_flush(mm, addr, ptep, pte_num); - - if (pte_dirty(orig_pte)) - pte = pte_mkdirty(pte); - - if (pte_young(orig_pte)) - pte = pte_mkyoung(pte); - - set_ptes(mm, addr, ptep, pte, pte_num); - - return true; -} - void huge_ptep_set_wrprotect(struct mm_struct *mm, unsigned long addr, pte_t *ptep) diff --git a/arch/riscv/mm/pgtable.c b/arch/riscv/mm/pgtable.c index 533ec9055fa0..e86df7ef193c 100644 --- a/arch/riscv/mm/pgtable.c +++ b/arch/riscv/mm/pgtable.c @@ -5,9 +5,9 @@ #include #include -int ptep_set_access_flags(struct vm_area_struct *vma, - unsigned long address, pte_t *ptep, - pte_t entry, int dirty) +int __ptep_set_access_flags(struct vm_area_struct *vma, + unsigned long address, pte_t *ptep, + pte_t entry, int dirty) { if (!pte_same(ptep_get(ptep), entry)) __set_pte_at(vma->vm_mm, ptep, entry); diff --git a/include/linux/hugetlb_contpte.h b/include/linux/hugetlb_contpte.h index 20d3a3e14e14..fea47035ac38 100644 --- a/include/linux/hugetlb_contpte.h +++ b/include/linux/hugetlb_contpte.h @@ -22,4 +22,9 @@ extern void huge_pte_clear(struct mm_struct *mm, unsigned long addr, extern pte_t huge_ptep_get_and_clear(struct mm_struct *mm, unsigned long addr, pte_t *ptep); +#define __HAVE_ARCH_HUGE_PTEP_SET_ACCESS_FLAGS +extern int huge_ptep_set_access_flags(struct vm_area_struct *vma, + unsigned long addr, pte_t *ptep, + pte_t pte, int dirty); + #endif /* _LINUX_HUGETLB_CONTPTE_H */ diff --git a/mm/hugetlb_contpte.c b/mm/hugetlb_contpte.c index 9df98d1a59a1..21f5c89daa6b 100644 --- a/mm/hugetlb_contpte.c +++ b/mm/hugetlb_contpte.c @@ -13,6 +13,7 @@ * - __set_ptes() * - __ptep_get_and_clear() * - __pte_clear() + * - __ptep_set_access_flags() * - pte_cont() * - arch_contpte_get_num_contig() */ @@ -23,6 +24,7 @@ * - set_huge_pte_at() * - huge_pte_clear() * - huge_ptep_get_and_clear() + * - huge_ptep_set_access_flags() */ pte_t huge_ptep_get(struct mm_struct *mm, unsigned long addr, pte_t *ptep) @@ -157,3 +159,76 @@ pte_t huge_ptep_get_and_clear(struct mm_struct *mm, return get_clear_contig(mm, addr, ptep, pgsize, ncontig); } + +/* + * huge_ptep_set_access_flags will update access flags (dirty, accesssed) + * and write permission. + * + * For a contiguous huge pte range we need to check whether or not write + * permission has to change only on the first pte in the set. Then for + * all the contiguous ptes we need to check whether or not there is a + * discrepancy between dirty or young. + */ +static int __cont_access_flags_changed(pte_t *ptep, pte_t pte, int ncontig) +{ + int i; + + if (pte_write(pte) != pte_write(__ptep_get(ptep))) + return 1; + + for (i = 0; i < ncontig; i++) { + pte_t orig_pte = __ptep_get(ptep + i); + + if (pte_dirty(pte) != pte_dirty(orig_pte)) + return 1; + + if (pte_young(pte) != pte_young(orig_pte)) + return 1; + } + + return 0; +} + +static pte_t get_clear_contig_flush(struct mm_struct *mm, + unsigned long addr, + pte_t *ptep, + unsigned long pgsize, + unsigned long ncontig) +{ + pte_t orig_pte = get_clear_contig(mm, addr, ptep, pgsize, ncontig); + struct vm_area_struct vma = TLB_FLUSH_VMA(mm, 0); + + flush_tlb_range(&vma, addr, addr + (pgsize * ncontig)); + return orig_pte; +} + +int huge_ptep_set_access_flags(struct vm_area_struct *vma, + unsigned long addr, pte_t *ptep, + pte_t pte, int dirty) +{ + int ncontig; + size_t pgsize = 0; + struct mm_struct *mm = vma->vm_mm; + pte_t orig_pte; + + if (!pte_cont(pte)) + return __ptep_set_access_flags(vma, addr, ptep, pte, dirty); + + ncontig = arch_contpte_get_num_contig(vma->vm_mm, addr, ptep, 0, &pgsize); + + if (!__cont_access_flags_changed(ptep, pte, ncontig)) + return 0; + + orig_pte = get_clear_contig_flush(mm, addr, ptep, pgsize, ncontig); + + /* Make sure we don't lose the dirty or young state */ + if (pte_dirty(orig_pte)) + pte = pte_mkdirty(pte); + + if (pte_young(orig_pte)) + pte = pte_mkyoung(pte); + + set_contptes(mm, addr, ptep, pte, ncontig, pgsize); + + return 1; +} From patchwork Fri Aug 2 15:14:29 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexandre Ghiti X-Patchwork-Id: 13751653 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 711F8C3DA4A for ; Fri, 2 Aug 2024 15:22:49 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 078C56B0099; Fri, 2 Aug 2024 11:22:49 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 029056B009A; Fri, 2 Aug 2024 11:22:48 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E0C6A6B009B; Fri, 2 Aug 2024 11:22:48 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id C42CD6B0099 for ; Fri, 2 Aug 2024 11:22:48 -0400 (EDT) Received: from smtpin03.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 8313312103A for ; Fri, 2 Aug 2024 15:22:48 +0000 (UTC) X-FDA: 82407672816.03.EF08E09 Received: from mail-wm1-f51.google.com (mail-wm1-f51.google.com [209.85.128.51]) by imf15.hostedemail.com (Postfix) with ESMTP id A4D14A0035 for ; Fri, 2 Aug 2024 15:22:46 +0000 (UTC) Authentication-Results: imf15.hostedemail.com; dkim=pass header.d=rivosinc-com.20230601.gappssmtp.com header.s=20230601 header.b=FWlb8k1c; spf=pass (imf15.hostedemail.com: domain of alexghiti@rivosinc.com designates 209.85.128.51 as permitted sender) smtp.mailfrom=alexghiti@rivosinc.com; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1722612089; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=y1cBbOZt36mdcwgOJ6aPskHFTTaexGabCNqRxjg6Oh0=; b=LlBhobQc2f7wIE2Pc4lBprW+OzvriWCtpDoJiagUOMKzXzEU9vdvoWFfpBPx679iS1sLV9 YeEpP90twy+pdMXeRAKCmJb+wJXvv1LzDNuLkk0kMPv9ryT6YiYv25iOVc5IJE56OKPqQA s3qk+LW2H5JywUrEHa2y76n8ZiYR7IQ= ARC-Authentication-Results: i=1; imf15.hostedemail.com; dkim=pass header.d=rivosinc-com.20230601.gappssmtp.com header.s=20230601 header.b=FWlb8k1c; spf=pass (imf15.hostedemail.com: domain of alexghiti@rivosinc.com designates 209.85.128.51 as permitted sender) smtp.mailfrom=alexghiti@rivosinc.com; dmarc=none ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1722612089; a=rsa-sha256; cv=none; b=4GJqU3NyZMiR7sTbZ5uPNi7Z5R15IgPM10iHHqD1y00+TPcCdYxs/WvogOiK6K8YV2SAIf zLAeZrUDlbJLJGaaZ5ueSi6hVH4LwajNAoC5TL90iuiknbAhTqfraWfVyRVjgwmLTc/EbQ XByY8oPRPAgsnN8O1PfVQYYQgOvnLcM= Received: by mail-wm1-f51.google.com with SMTP id 5b1f17b1804b1-428035c0bb2so19340545e9.1 for ; Fri, 02 Aug 2024 08:22:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc-com.20230601.gappssmtp.com; s=20230601; t=1722612165; x=1723216965; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=y1cBbOZt36mdcwgOJ6aPskHFTTaexGabCNqRxjg6Oh0=; b=FWlb8k1cA3xO5dAU6H29zlPFwUbMOhaa0SaP61PTMB8w4vhRWEVlbJ9H2XpFiE7cHw lFLhXhrvcBXXnFwUQESvLi7aeWujnA4RFOZrTysD4FWcidLslHmLwpciCBgp5ScX6Qvc KQmDOYLD8IK7wBnB3ukKumw1V8V1YEbRta67ZlT+Ek08ixoFtF+dl9UZ42bZE7ge373g J1diV31sxyV/YjYhpJjFxLSk2Idt9Fv4V9d7ZH8HyoOVo2w/qmb+yMUsSsD0k5fmJRVA tta+UG4LT+jrF2ML4pGB9UGgWUoLkZGzeyJ1xjweUj17AjCF7UKCi+g/53weqmHf8qFF ZEnQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1722612165; x=1723216965; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=y1cBbOZt36mdcwgOJ6aPskHFTTaexGabCNqRxjg6Oh0=; b=jE/A3D3EyHsQz62ujqF99ns9RTuJBRgEQxD6ViIkHtDUhwTr3OmkiVpppYU/1JjdCX TgyH5a97wOiHMKWkeIiTVAJ1U6z1rf2aQPbughxNJc1/Oz6C69PFVGtHvrvEvQzzZfL1 UkrvAgBfKZuOfuOGmNTw34XdsGm68QrHBDrkBcUE3Bdo7gDR+7YUocWHXBYdaFD4XkPG GDogwCopaffS2/TPCklFSCBtCag9+Hauk9DZ0Ytl6HVfa6BPCHFxg8dCMNqy7o0MZ+0I 7w6YK6oXSgSda7Lxrh4pke2wwgCXvudfhGf8ZxT8QkD+xxk0llOTQF4tUsLXPiHkndBZ sWMQ== X-Forwarded-Encrypted: i=1; AJvYcCUEltbIIncTP0428RHd4dZ2mHdSzyH6YQEOack7yvWMOKpkJb9VspF6tgj4kT8jdaKNz6XJtpnB5jC+i2C2kPvN1eQ= X-Gm-Message-State: AOJu0Yw8Wpl7Do65eqthOUQFBhBuhkOW8DYype/GEUvKdKwVb/0vvGFZ e22xaa4VwSYYb+1kWBd8KAbejV98T1UngB3KHINbbS+xZSTGyppQ+i6RTS3uqK0= X-Google-Smtp-Source: AGHT+IGevEgshEH6AuLTp8Bo7gjdyVs7Lhu/RR9IyqwSflPacf3rHkteywexmhQW20qeFqo6GEOMhQ== X-Received: by 2002:a05:6000:400d:b0:367:8f89:f7c9 with SMTP id ffacd0b85a97d-36bbbef1261mr3849953f8f.33.1722612165098; Fri, 02 Aug 2024 08:22:45 -0700 (PDT) Received: from alex-rivos.ba.rivosinc.com (amontpellier-656-1-456-62.w92-145.abo.wanadoo.fr. [92.145.124.62]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-36bbd01619csm2202892f8f.36.2024.08.02.08.22.44 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 02 Aug 2024 08:22:44 -0700 (PDT) From: Alexandre Ghiti To: Catalin Marinas , Will Deacon , Ryan Roberts , Mark Rutland , Paul Walmsley , Palmer Dabbelt , Albert Ou , Andrew Morton , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org, linux-mm@kvack.org Cc: Alexandre Ghiti Subject: [PATCH v3 8/9] mm: Use common huge_ptep_set_wrprotect() function for riscv/arm64 Date: Fri, 2 Aug 2024 17:14:29 +0200 Message-Id: <20240802151430.99114-9-alexghiti@rivosinc.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20240802151430.99114-1-alexghiti@rivosinc.com> References: <20240802151430.99114-1-alexghiti@rivosinc.com> MIME-Version: 1.0 X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: A4D14A0035 X-Stat-Signature: aqqgha6xw5xg5x66x9xczfq19xmx1naz X-Rspam-User: X-HE-Tag: 1722612166-418616 X-HE-Meta: U2FsdGVkX18tJONIodawMp+yXxfZVuZTQHpMDPZsSDEJYRXRqhS90NWuf4zzxe2jGPKevSofVKhDqJeHaJjcQfOzRNVVv0kgWnhMwbaNzPwdBTnqGZFWJDJat+oYrn9VJeNME1PB8L19K+oy4Xyunzyc44KAP7YP/N/OVckyVZS8ZG4JK2VTGS99dMVjyk9n8qw1ht4Onn/rSGAMwMkjmHTgjBpK+goJV+372ND2aVz2prjnNwZEJlT03jCJcWlK64OWVVamSsWpoKOOjrN3JqOmWexT//zg5zdZX14iosGvSQhYVn9kyKw/IZnNG1w7iuhWGVIRhLfcMxwUPz1i5dmnirElc0DJqDDdc+OiEK7vHvb9hA978CsMT8TRWlzOK5DDs1aj80ou/ZD70gdDqOCORWu9ajFQ4DL2YLIph8FqAQt2hAZK4Iy4OUnR4EK1bZYrXN5UWlIHG2Q0g/5ArY9oIlaDN2XtbzLtAeDRL/rdh8umj/mMGqK0eUiBWD83tQJB+EQqH2+wNk8SZCYtrZu1FK+4erjpbag1kMW44bgiTC2OHatAVAPZ66AuOzHvVcZh+NDpUH/iipPt1KKjEdMZuFkxxWZ4fmy/1dO4S5qjlWOZ4Lti4wEgDlnPNVpzf9/6FrVKVIP4xHbp3cFxS5HCEBUXvUPsZnlaYqCF9I5FUDWBtKQaDY4vJn6LYVr9WgN4w+Mhue6xAb9loq8WpfyprljHEDGCNPBw5IPuNnrj9un1r026/fbc10a2uBmAZ+WMMEClsOhkKZrL3WPchqZ3qzFlwqTOvxKC/hiAfqJAUq5Jm4MVeHsk6gK0gvvTKN3U6gvS/vINYg8Z5ZrGKhy4jwOSHtrTHfs4uS1IO12BO1796cjyCjynFXewZw5JvklLkQdJV7WIRzVELa2W2aoBLJjntc8SVaCH8uBQ2VFXCTMkhwW6x8wGO49mpZRsCEVXan04fiOAMZcv9HM teZ/OFAu ywc+gLb00Bki9BPEGsGPhNxkBEMjwStctdEAf9lLLb72NfdxHv/10aDJjk6FgXg+o5ZwEmeziTag0rjRwWIwjAFc4grrL1dcuYupBjPyBV0PJhCEzaOTCD4Gkc3DrCmhN82D9yJ6xG74A3oPahNtzyBijyVKKTHAGzoPntoQ8f51vliarYt3Kh6H8IBGN1CQMKot2cD5o+RWaQdHGApLuEsHEkocioWkGtSX4fdmtL/HoJUmhxn4ZAmJLvdTKZXUIGfiGrU5lvaigsbLbMh7hkBJrVUjAOacHKGrpDREuTBeRJGV5v7VRxtVUQSHFdluf00qaROsEV2Un6L+HZJndMXjCnyPWcuqCN+NuY3fByvZsouyiyUq8sn4CR+qqya5D6hD7AwMAYKlUcTWJ/kQPqmXLkw== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: After some adjustments, both architectures have the same implementation so move it to the generic code. Signed-off-by: Alexandre Ghiti --- arch/arm64/include/asm/hugetlb.h | 3 --- arch/arm64/mm/hugetlbpage.c | 27 --------------------------- arch/riscv/include/asm/hugetlb.h | 4 ---- arch/riscv/include/asm/pgtable.h | 7 ++++--- arch/riscv/mm/hugetlbpage.c | 22 ---------------------- include/linux/hugetlb_contpte.h | 4 ++++ mm/hugetlb_contpte.c | 22 ++++++++++++++++++++++ 7 files changed, 30 insertions(+), 59 deletions(-) diff --git a/arch/arm64/include/asm/hugetlb.h b/arch/arm64/include/asm/hugetlb.h index 6b84e287b72d..4ceb6cb1bec5 100644 --- a/arch/arm64/include/asm/hugetlb.h +++ b/arch/arm64/include/asm/hugetlb.h @@ -27,9 +27,6 @@ static inline void arch_clear_hugetlb_flags(struct folio *folio) pte_t arch_make_huge_pte(pte_t entry, unsigned int shift, vm_flags_t flags); #define arch_make_huge_pte arch_make_huge_pte -#define __HAVE_ARCH_HUGE_PTEP_SET_WRPROTECT -extern void huge_ptep_set_wrprotect(struct mm_struct *mm, - unsigned long addr, pte_t *ptep); #define __HAVE_ARCH_HUGE_PTEP_CLEAR_FLUSH extern pte_t huge_ptep_clear_flush(struct vm_area_struct *vma, unsigned long addr, pte_t *ptep); diff --git a/arch/arm64/mm/hugetlbpage.c b/arch/arm64/mm/hugetlbpage.c index 658635022e4d..8d271e175848 100644 --- a/arch/arm64/mm/hugetlbpage.c +++ b/arch/arm64/mm/hugetlbpage.c @@ -266,33 +266,6 @@ pte_t arch_make_huge_pte(pte_t entry, unsigned int shift, vm_flags_t flags) return entry; } -void huge_ptep_set_wrprotect(struct mm_struct *mm, - unsigned long addr, pte_t *ptep) -{ - unsigned long pfn, dpfn; - pgprot_t hugeprot; - int ncontig, i; - size_t pgsize; - pte_t pte; - - if (!pte_cont(__ptep_get(ptep))) { - __ptep_set_wrprotect(mm, addr, ptep); - return; - } - - ncontig = find_num_contig(mm, addr, ptep, &pgsize); - dpfn = pgsize >> PAGE_SHIFT; - - pte = get_clear_contig_flush(mm, addr, ptep, pgsize, ncontig); - pte = pte_wrprotect(pte); - - hugeprot = pte_pgprot(pte); - pfn = pte_pfn(pte); - - for (i = 0; i < ncontig; i++, ptep++, addr += pgsize, pfn += dpfn) - __set_ptes(mm, addr, ptep, pfn_pte(pfn, hugeprot), 1); -} - pte_t huge_ptep_clear_flush(struct vm_area_struct *vma, unsigned long addr, pte_t *ptep) { diff --git a/arch/riscv/include/asm/hugetlb.h b/arch/riscv/include/asm/hugetlb.h index bf533c2cef84..4c692dd82779 100644 --- a/arch/riscv/include/asm/hugetlb.h +++ b/arch/riscv/include/asm/hugetlb.h @@ -24,10 +24,6 @@ bool arch_hugetlb_migration_supported(struct hstate *h); pte_t huge_ptep_clear_flush(struct vm_area_struct *vma, unsigned long addr, pte_t *ptep); -#define __HAVE_ARCH_HUGE_PTEP_SET_WRPROTECT -void huge_ptep_set_wrprotect(struct mm_struct *mm, - unsigned long addr, pte_t *ptep); - pte_t arch_make_huge_pte(pte_t entry, unsigned int shift, vm_flags_t flags); #define arch_make_huge_pte arch_make_huge_pte diff --git a/arch/riscv/include/asm/pgtable.h b/arch/riscv/include/asm/pgtable.h index 5d9b051ac51c..f4bb0c971bd8 100644 --- a/arch/riscv/include/asm/pgtable.h +++ b/arch/riscv/include/asm/pgtable.h @@ -639,9 +639,8 @@ static inline pte_t __ptep_get_and_clear(struct mm_struct *mm, return pte; } -#define __HAVE_ARCH_PTEP_SET_WRPROTECT -static inline void ptep_set_wrprotect(struct mm_struct *mm, - unsigned long address, pte_t *ptep) +static inline void __ptep_set_wrprotect(struct mm_struct *mm, + unsigned long address, pte_t *ptep) { atomic_long_and(~(unsigned long)_PAGE_WRITE, (atomic_long_t *)ptep); } @@ -735,6 +734,8 @@ static inline pte_t ptep_get(pte_t *ptep) #define pte_clear __pte_clear #define __HAVE_ARCH_PTEP_SET_ACCESS_FLAGS #define ptep_set_access_flags __ptep_set_access_flags +#define __HAVE_ARCH_PTEP_SET_WRPROTECT +#define ptep_set_wrprotect __ptep_set_wrprotect #define pgprot_nx pgprot_nx static inline pgprot_t pgprot_nx(pgprot_t _prot) diff --git a/arch/riscv/mm/hugetlbpage.c b/arch/riscv/mm/hugetlbpage.c index 0e2ca7327479..8963a4e77742 100644 --- a/arch/riscv/mm/hugetlbpage.c +++ b/arch/riscv/mm/hugetlbpage.c @@ -173,28 +173,6 @@ pte_t arch_make_huge_pte(pte_t entry, unsigned int shift, vm_flags_t flags) return entry; } -void huge_ptep_set_wrprotect(struct mm_struct *mm, - unsigned long addr, - pte_t *ptep) -{ - pte_t pte = ptep_get(ptep); - pte_t orig_pte; - int pte_num; - - if (!pte_napot(pte)) { - ptep_set_wrprotect(mm, addr, ptep); - return; - } - - pte_num = arch_contpte_get_num_contig(mm, addr, ptep, 0, NULL); - - orig_pte = get_clear_contig_flush(mm, addr, ptep, pte_num); - - orig_pte = pte_wrprotect(orig_pte); - - set_ptes(mm, addr, ptep, orig_pte, pte_num); -} - pte_t huge_ptep_clear_flush(struct vm_area_struct *vma, unsigned long addr, pte_t *ptep) diff --git a/include/linux/hugetlb_contpte.h b/include/linux/hugetlb_contpte.h index fea47035ac38..02bce0ed93d8 100644 --- a/include/linux/hugetlb_contpte.h +++ b/include/linux/hugetlb_contpte.h @@ -27,4 +27,8 @@ extern int huge_ptep_set_access_flags(struct vm_area_struct *vma, unsigned long addr, pte_t *ptep, pte_t pte, int dirty); +#define __HAVE_ARCH_HUGE_PTEP_SET_WRPROTECT +extern void huge_ptep_set_wrprotect(struct mm_struct *mm, + unsigned long addr, pte_t *ptep); + #endif /* _LINUX_HUGETLB_CONTPTE_H */ diff --git a/mm/hugetlb_contpte.c b/mm/hugetlb_contpte.c index 21f5c89daa6b..b00bbcc2d939 100644 --- a/mm/hugetlb_contpte.c +++ b/mm/hugetlb_contpte.c @@ -14,6 +14,7 @@ * - __ptep_get_and_clear() * - __pte_clear() * - __ptep_set_access_flags() + * - __ptep_set_wrprotect() * - pte_cont() * - arch_contpte_get_num_contig() */ @@ -25,6 +26,7 @@ * - huge_pte_clear() * - huge_ptep_get_and_clear() * - huge_ptep_set_access_flags() + * - huge_ptep_set_wrprotect() */ pte_t huge_ptep_get(struct mm_struct *mm, unsigned long addr, pte_t *ptep) @@ -232,3 +234,23 @@ int huge_ptep_set_access_flags(struct vm_area_struct *vma, return 1; } + +void huge_ptep_set_wrprotect(struct mm_struct *mm, + unsigned long addr, pte_t *ptep) +{ + int ncontig; + size_t pgsize; + pte_t pte; + + if (!pte_cont(__ptep_get(ptep))) { + __ptep_set_wrprotect(mm, addr, ptep); + return; + } + + ncontig = arch_contpte_get_num_contig(mm, addr, ptep, 0, &pgsize); + + pte = get_clear_contig_flush(mm, addr, ptep, pgsize, ncontig); + pte = pte_wrprotect(pte); + + set_contptes(mm, addr, ptep, pte, ncontig, pgsize); +} From patchwork Fri Aug 2 15:14:30 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexandre Ghiti X-Patchwork-Id: 13751654 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 079E7C52D6F for ; Fri, 2 Aug 2024 15:23:51 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 8DE236B009B; Fri, 2 Aug 2024 11:23:50 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 88DE86B009C; Fri, 2 Aug 2024 11:23:50 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 72F3F6B009D; Fri, 2 Aug 2024 11:23:50 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 55E896B009B for ; Fri, 2 Aug 2024 11:23:50 -0400 (EDT) Received: from smtpin27.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 05ADAA6163 for ; Fri, 2 Aug 2024 15:23:50 +0000 (UTC) X-FDA: 82407675420.27.190CBBE Received: from mail-wr1-f43.google.com (mail-wr1-f43.google.com [209.85.221.43]) by imf07.hostedemail.com (Postfix) with ESMTP id 11A0D40037 for ; Fri, 2 Aug 2024 15:23:47 +0000 (UTC) Authentication-Results: imf07.hostedemail.com; dkim=pass header.d=rivosinc-com.20230601.gappssmtp.com header.s=20230601 header.b=IGk0kobg; spf=pass (imf07.hostedemail.com: domain of alexghiti@rivosinc.com designates 209.85.221.43 as permitted sender) smtp.mailfrom=alexghiti@rivosinc.com; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1722612151; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=vQeh/2IBWHHfu2c6+4XfBIZxNp4bqCqqEIWNYF9wjYg=; b=24xx4eTHfUecPPReu2D+1I3MTwcUbiEKZK9G+yQl2GQQ6/xzlv8din8UGSm9NjivQlcoq/ rUQSRUcEGgoInHrLvb+O8cyuPU5Zp4gWxZl6zE78k23f4Q6h7odJjYsFufGLf5X2tC60Kx 5U8a7YRq/5Qju5yJi9VCYyI14CjxmQk= ARC-Authentication-Results: i=1; imf07.hostedemail.com; dkim=pass header.d=rivosinc-com.20230601.gappssmtp.com header.s=20230601 header.b=IGk0kobg; spf=pass (imf07.hostedemail.com: domain of alexghiti@rivosinc.com designates 209.85.221.43 as permitted sender) smtp.mailfrom=alexghiti@rivosinc.com; dmarc=none ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1722612151; a=rsa-sha256; cv=none; b=gIZ+NddGrMkR+HQNA0cKFcCZ7gbmWkiD5CSnOY+LCS3KuNzWwDnAqquzpEtfjsFBQMO8VE S+uo651qn7ep+zPGh3sSqS0c6xHzFNGNj1mGcYoo0vKs+GrzhYYHpg4FWIwdmcB5+TGE/u EA+KVZRihWTAV1oSL5IkrIujT6P+6p0= Received: by mail-wr1-f43.google.com with SMTP id ffacd0b85a97d-3683178b226so4012844f8f.1 for ; Fri, 02 Aug 2024 08:23:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc-com.20230601.gappssmtp.com; s=20230601; t=1722612227; x=1723217027; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=vQeh/2IBWHHfu2c6+4XfBIZxNp4bqCqqEIWNYF9wjYg=; b=IGk0kobg5sD+TL0w2aXIEunCV5+CuL2QThEXTpxqhxqpTpxOHAifcgzDG4RqkegVjt T7uJijQHp7b7Aydrdr+shoqoMJF05ohP3cIrGfGPLgKuUWcGqAgwh01NvBrFEzGaUCBX 8OiF+06eRYMbUUp130Aw1se3KXW4+PDDpQDuB3hg9eUfsbtoMP/gf1KRxeN2Z3xrpyHN 7rQsJLFVT1pcaXrhOVy9jWi8/9p3tKi2lfOvPE6ODzcKanv+hgLkKNyLggfnRVkMZ/A5 ktMblZznxyyk3xeqKstDejbAqnOg32f2eIaiSx9g1DJdprv6hc0eEGKH77EfUm50j93z tKsg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1722612227; x=1723217027; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=vQeh/2IBWHHfu2c6+4XfBIZxNp4bqCqqEIWNYF9wjYg=; b=OIfhPRRzF7fuU2GrHW/w2HwGIL180PDVWvGDbAyJqOvDa0yko4lo4oqX+SOuodOXgN 3aeR+2zbLDpiHiTDwHwJ6MZnHgwDhKYtzzDtXfiMwRl9cC6K26HNF55G+Ayx4N5Z3PK6 7o9+umX392/ML57ed7S7lT+nTBCzENgqWmsn2XUxnqEjgf9wsElIY7pGNLCBYNdJ7T6m ad0wXWEaDctCbzJkiJsQ2ZnjdyD9mqov44/VOtCLgMqyRalCJM8YYeiFn5bIsNNDunov 4VshiOEuXAzTMbELUGEr395KCCT3itZN6MZWGw7WTmOxDUGB+HYjEwmx2C07VOEUE9Bx zqjw== X-Forwarded-Encrypted: i=1; AJvYcCVlSpeXN/IMzXiYf2wQxubDEyt8YepWaGt/fB9Kp7gihuNUPRHwHUk0SoDAd5TV/AJ+eodYncC05cwsvv7yP1ZknL0= X-Gm-Message-State: AOJu0YyzcDbbjaoyIoRndNWc6shx4uhCByhj/Rh+hdSw9bgIzRZ44Gah Zkvi+IHB64DG7c7/3DR2RCl/RabJQ2QL9n7IVPz3bZq4LUxChz9egexOdE5vb8U= X-Google-Smtp-Source: AGHT+IHTN3sk4f5D6D5TLgsFPV/oHaKP/CTwEIdjag4arFIbTitJxkFdq6uXQ6FKZRCqULys8NHSbg== X-Received: by 2002:a5d:5c87:0:b0:368:460d:d942 with SMTP id ffacd0b85a97d-36bc5a007bamr859962f8f.24.1722612226380; Fri, 02 Aug 2024 08:23:46 -0700 (PDT) Received: from alex-rivos.ba.rivosinc.com (amontpellier-656-1-456-62.w92-145.abo.wanadoo.fr. [92.145.124.62]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-36bbd01efb1sm2194291f8f.52.2024.08.02.08.23.45 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 02 Aug 2024 08:23:46 -0700 (PDT) From: Alexandre Ghiti To: Catalin Marinas , Will Deacon , Ryan Roberts , Mark Rutland , Paul Walmsley , Palmer Dabbelt , Albert Ou , Andrew Morton , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org, linux-mm@kvack.org Cc: Alexandre Ghiti Subject: [PATCH v3 9/9] mm: Use common huge_ptep_clear_flush() function for riscv/arm64 Date: Fri, 2 Aug 2024 17:14:30 +0200 Message-Id: <20240802151430.99114-10-alexghiti@rivosinc.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20240802151430.99114-1-alexghiti@rivosinc.com> References: <20240802151430.99114-1-alexghiti@rivosinc.com> MIME-Version: 1.0 X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: 11A0D40037 X-Stat-Signature: 9cify8zu7jptegpfta7yczdomderbssz X-Rspam-User: X-HE-Tag: 1722612227-190982 X-HE-Meta: U2FsdGVkX190z5gYbphiFe20KWAqhC7QPuqQsHKWudDEOpspWM8gqX3tc0g84DbXcHDi53lltnU4CGmc/njt3nJpcQazKL7uy+HQPWAneoPKCpAmK2TaE979g2gtIPHQKsdm6EHntU9LAV4e3WWQkRR3Gi4ph6GHECwaLX2nt1Zu1+5TQhYCxEsn7ujbHnddGQtJegqUQWjt8/SZH08rVZioDeiKEnUxmVQeJKQBbUyZ6qJxflPZ0frudHdt5LHfV6hF9r35G0VpUYYokW+Y71TCUHM6IEhGtAfsajisM60bXjY6GallYxPVn7vKMTmamdIQ3/mRCNBEr0LOJ5hy11A4WYsw4z0KzFJj3bVKjkfxWjoYNkzc0IQta1JJexHyvGidhJfUCZ36+EvJxC5msX1eYP2vk3WcBEBalgc417sqS191ctDOXJ0mGo+Er68TE7cgtszvUZ/gfWZzC1SN8RRnO4k/SlO6eSUNqPdRUe/yRSE6KB6QznNwN70yoooPXz6nVo3tH92gBd/iQzU1K+9qXM2VHaqQK5u4XAAZNYOgc6DBXgb4y6SmDZJhIvTRbIEd+/oGbl4FVcM/fu15wahiaN49pqdabjvZ/QwOZT3yFkwl5eWuPg/1h3GrGcJcEsynfwhgihpNOC2u8VFihDkth8w28V85bfaH0WY/E7XjVMxm086nOkCRBhtrc6GT74voml78pVgAO5mcXf3B1zgYuE6wThycUti6md7LhT7WwRuUkBcZKSQfIwRAOV8BHUJrnf1DZvr54AtceB7/Sn54bpybJ7X2ulactsZ6ZtJjqc098u/JNq2NF8BfwHmmJ/LwjXd9LAkflNdIufUX2lKOgyplyNIpHEIZlJMRswPUXCkFHRhuDaheoTD7CtfjbMsUJtOV4bzTosYtLQ/L4yZKN06RLLWePfp3JXVFn0G8vXfk1FMno8I7kjMOBWYrIahQ1sT/HUYg+ulNoe6 y9CBM4E0 lI6tx0eO1tWljXrX5y/SDJYyhpDdwE//du92Nr9q5eXUEH32G5MsweN5R7tLtMhtvliIAMAaWjGNDHUIbaHuP8RGZj6TI7MIUGMl1YO0/oXyqkd2gmASxoe+VcVvsLFcWD4Wb46IF5qrxwRON8hZ1HWK3XovRk7l069a0KhGq6IPNQaZGGximqayCjD8qq159CjJjynk5njZApOns0mZ5JzlAVdLOnSH6/5qWnr/VwHMe31MPj2Za+nBRwxJaHu7MPIw43PyzceEIrFpuZPAb4LY3pPzAR5q++Sc9HUdKd9LPg5S0L897M3hH66PeqXHW0LDdAdDQdqX2LaSZhiwWxaaVZbDfoJeXoawPRPs5QY4a2PS6Xw8C0+9gZmBizUtyv1GHn1Hwy1uClc2sUvsirI+lhg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: After some adjustments, both architectures have the same implementation so move it to the generic code. Signed-off-by: Alexandre Ghiti --- arch/arm64/include/asm/hugetlb.h | 3 -- arch/arm64/mm/hugetlbpage.c | 61 -------------------------------- arch/riscv/include/asm/hugetlb.h | 7 +--- arch/riscv/mm/hugetlbpage.c | 51 -------------------------- include/linux/hugetlb_contpte.h | 4 +++ mm/hugetlb_contpte.c | 15 ++++++++ 6 files changed, 20 insertions(+), 121 deletions(-) diff --git a/arch/arm64/include/asm/hugetlb.h b/arch/arm64/include/asm/hugetlb.h index 4ceb6cb1bec5..93c44a76c994 100644 --- a/arch/arm64/include/asm/hugetlb.h +++ b/arch/arm64/include/asm/hugetlb.h @@ -27,9 +27,6 @@ static inline void arch_clear_hugetlb_flags(struct folio *folio) pte_t arch_make_huge_pte(pte_t entry, unsigned int shift, vm_flags_t flags); #define arch_make_huge_pte arch_make_huge_pte -#define __HAVE_ARCH_HUGE_PTEP_CLEAR_FLUSH -extern pte_t huge_ptep_clear_flush(struct vm_area_struct *vma, - unsigned long addr, pte_t *ptep); void __init arm64_hugetlb_cma_reserve(void); diff --git a/arch/arm64/mm/hugetlbpage.c b/arch/arm64/mm/hugetlbpage.c index 8d271e175848..ad04641261dc 100644 --- a/arch/arm64/mm/hugetlbpage.c +++ b/arch/arm64/mm/hugetlbpage.c @@ -98,53 +98,6 @@ int find_num_contig(struct mm_struct *mm, unsigned long addr, return CONT_PTES; } -/* - * Changing some bits of contiguous entries requires us to follow a - * Break-Before-Make approach, breaking the whole contiguous set - * before we can change any entries. See ARM DDI 0487A.k_iss10775, - * "Misprogramming of the Contiguous bit", page D4-1762. - * - * This helper performs the break step. - */ -static pte_t get_clear_contig(struct mm_struct *mm, - unsigned long addr, - pte_t *ptep, - unsigned long pgsize, - unsigned long ncontig) -{ - pte_t orig_pte = __ptep_get(ptep); - unsigned long i; - - for (i = 0; i < ncontig; i++, addr += pgsize, ptep++) { - pte_t pte = __ptep_get_and_clear(mm, addr, ptep); - - /* - * If HW_AFDBM is enabled, then the HW could turn on - * the dirty or accessed bit for any page in the set, - * so check them all. - */ - if (pte_dirty(pte)) - orig_pte = pte_mkdirty(orig_pte); - - if (pte_young(pte)) - orig_pte = pte_mkyoung(orig_pte); - } - return orig_pte; -} - -static pte_t get_clear_contig_flush(struct mm_struct *mm, - unsigned long addr, - pte_t *ptep, - unsigned long pgsize, - unsigned long ncontig) -{ - pte_t orig_pte = get_clear_contig(mm, addr, ptep, pgsize, ncontig); - struct vm_area_struct vma = TLB_FLUSH_VMA(mm, 0); - - flush_tlb_range(&vma, addr, addr + (pgsize * ncontig)); - return orig_pte; -} - pte_t *huge_pte_alloc(struct mm_struct *mm, struct vm_area_struct *vma, unsigned long addr, unsigned long sz) { @@ -266,20 +219,6 @@ pte_t arch_make_huge_pte(pte_t entry, unsigned int shift, vm_flags_t flags) return entry; } -pte_t huge_ptep_clear_flush(struct vm_area_struct *vma, - unsigned long addr, pte_t *ptep) -{ - struct mm_struct *mm = vma->vm_mm; - size_t pgsize; - int ncontig; - - if (!pte_cont(__ptep_get(ptep))) - return ptep_clear_flush(vma, addr, ptep); - - ncontig = find_num_contig(mm, addr, ptep, &pgsize); - return get_clear_contig_flush(mm, addr, ptep, pgsize, ncontig); -} - static int __init hugetlbpage_init(void) { if (pud_sect_supported()) diff --git a/arch/riscv/include/asm/hugetlb.h b/arch/riscv/include/asm/hugetlb.h index 4c692dd82779..63c7e4fa342a 100644 --- a/arch/riscv/include/asm/hugetlb.h +++ b/arch/riscv/include/asm/hugetlb.h @@ -20,14 +20,9 @@ bool arch_hugetlb_migration_supported(struct hstate *h); #endif #ifdef CONFIG_RISCV_ISA_SVNAPOT -#define __HAVE_ARCH_HUGE_PTEP_CLEAR_FLUSH -pte_t huge_ptep_clear_flush(struct vm_area_struct *vma, - unsigned long addr, pte_t *ptep); - pte_t arch_make_huge_pte(pte_t entry, unsigned int shift, vm_flags_t flags); #define arch_make_huge_pte arch_make_huge_pte - -#endif /*CONFIG_RISCV_ISA_SVNAPOT*/ +#endif /* CONFIG_RISCV_ISA_SVNAPOT */ #include diff --git a/arch/riscv/mm/hugetlbpage.c b/arch/riscv/mm/hugetlbpage.c index 8963a4e77742..ea1ae3a43d45 100644 --- a/arch/riscv/mm/hugetlbpage.c +++ b/arch/riscv/mm/hugetlbpage.c @@ -121,42 +121,6 @@ unsigned long hugetlb_mask_last_page(struct hstate *h) return 0UL; } -static pte_t get_clear_contig(struct mm_struct *mm, - unsigned long addr, - pte_t *ptep, - unsigned long pte_num) -{ - pte_t orig_pte = ptep_get(ptep); - unsigned long i; - - for (i = 0; i < pte_num; i++, addr += PAGE_SIZE, ptep++) { - pte_t pte = ptep_get_and_clear(mm, addr, ptep); - - if (pte_dirty(pte)) - orig_pte = pte_mkdirty(orig_pte); - - if (pte_young(pte)) - orig_pte = pte_mkyoung(orig_pte); - } - - return orig_pte; -} - -static pte_t get_clear_contig_flush(struct mm_struct *mm, - unsigned long addr, - pte_t *ptep, - unsigned long pte_num) -{ - pte_t orig_pte = get_clear_contig(mm, addr, ptep, pte_num); - struct vm_area_struct vma = TLB_FLUSH_VMA(mm, 0); - bool valid = !pte_none(orig_pte); - - if (valid) - flush_tlb_range(&vma, addr, addr + (PAGE_SIZE * pte_num)); - - return orig_pte; -} - pte_t arch_make_huge_pte(pte_t entry, unsigned int shift, vm_flags_t flags) { unsigned long order; @@ -173,21 +137,6 @@ pte_t arch_make_huge_pte(pte_t entry, unsigned int shift, vm_flags_t flags) return entry; } -pte_t huge_ptep_clear_flush(struct vm_area_struct *vma, - unsigned long addr, - pte_t *ptep) -{ - pte_t pte = ptep_get(ptep); - int pte_num; - - if (!pte_napot(pte)) - return ptep_clear_flush(vma, addr, ptep); - - pte_num = arch_contpte_get_num_contig(vma->vm_mm, addr, ptep, 0, NULL); - - return get_clear_contig_flush(vma->vm_mm, addr, ptep, pte_num); -} - static bool is_napot_size(unsigned long size) { unsigned long order; diff --git a/include/linux/hugetlb_contpte.h b/include/linux/hugetlb_contpte.h index 02bce0ed93d8..911b9cd4aa4d 100644 --- a/include/linux/hugetlb_contpte.h +++ b/include/linux/hugetlb_contpte.h @@ -31,4 +31,8 @@ extern int huge_ptep_set_access_flags(struct vm_area_struct *vma, extern void huge_ptep_set_wrprotect(struct mm_struct *mm, unsigned long addr, pte_t *ptep); +#define __HAVE_ARCH_HUGE_PTEP_CLEAR_FLUSH +extern pte_t huge_ptep_clear_flush(struct vm_area_struct *vma, + unsigned long addr, pte_t *ptep); + #endif /* _LINUX_HUGETLB_CONTPTE_H */ diff --git a/mm/hugetlb_contpte.c b/mm/hugetlb_contpte.c index b00bbcc2d939..c8ea81883184 100644 --- a/mm/hugetlb_contpte.c +++ b/mm/hugetlb_contpte.c @@ -27,6 +27,7 @@ * - huge_ptep_get_and_clear() * - huge_ptep_set_access_flags() * - huge_ptep_set_wrprotect() + * - huge_ptep_clear_flush() */ pte_t huge_ptep_get(struct mm_struct *mm, unsigned long addr, pte_t *ptep) @@ -254,3 +255,17 @@ void huge_ptep_set_wrprotect(struct mm_struct *mm, set_contptes(mm, addr, ptep, pte, ncontig, pgsize); } + +pte_t huge_ptep_clear_flush(struct vm_area_struct *vma, + unsigned long addr, pte_t *ptep) +{ + struct mm_struct *mm = vma->vm_mm; + size_t pgsize; + int ncontig; + + if (!pte_cont(__ptep_get(ptep))) + return ptep_clear_flush(vma, addr, ptep); + + ncontig = arch_contpte_get_num_contig(mm, addr, ptep, 0, &pgsize); + return get_clear_contig_flush(mm, addr, ptep, pgsize, ncontig); +}