From patchwork Mon Jan 27 09:35:22 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexandre Ghiti X-Patchwork-Id: 13951117 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0D6DFC0218C for ; Mon, 27 Jan 2025 09:36:45 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 8EDAE6B0117; Mon, 27 Jan 2025 04:36:44 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 89CD62800D1; Mon, 27 Jan 2025 04:36:44 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 73DF66B0119; Mon, 27 Jan 2025 04:36:44 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 548546B0117 for ; Mon, 27 Jan 2025 04:36:44 -0500 (EST) Received: from smtpin08.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id F2108C20C8 for ; Mon, 27 Jan 2025 09:36:43 +0000 (UTC) X-FDA: 83052727086.08.599DDF4 Received: from mail-wr1-f41.google.com (mail-wr1-f41.google.com [209.85.221.41]) by imf26.hostedemail.com (Postfix) with ESMTP id 1AB9E14000B for ; Mon, 27 Jan 2025 09:36:41 +0000 (UTC) Authentication-Results: imf26.hostedemail.com; dkim=pass header.d=rivosinc-com.20230601.gappssmtp.com header.s=20230601 header.b=GJt8qxwJ; dmarc=none; spf=pass (imf26.hostedemail.com: domain of alexghiti@rivosinc.com designates 209.85.221.41 as permitted sender) smtp.mailfrom=alexghiti@rivosinc.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1737970602; a=rsa-sha256; cv=none; b=RxWPpbj/8LaPXmbo0I6hNKPudc3FRzEm0lguBBGVJGRw+hYkXvYSiEpiyAeOWnysXDu6R4 dCGKiLYiWEXVvh0xuR0RzRH3TGo1v1oGfHECc/XIQniWL4ISgmUF9Va56xYBifw+Vf5ptx Meh1y3UCV12LqsQLKOsLaymoDph57QY= ARC-Authentication-Results: i=1; imf26.hostedemail.com; dkim=pass header.d=rivosinc-com.20230601.gappssmtp.com header.s=20230601 header.b=GJt8qxwJ; dmarc=none; spf=pass (imf26.hostedemail.com: domain of alexghiti@rivosinc.com designates 209.85.221.41 as permitted sender) smtp.mailfrom=alexghiti@rivosinc.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1737970602; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=YlgUGAmuTWGVKaOGr6DQU8bhY7gyR7CYcPHPJxmYPTE=; b=Uz8uHyKmZsDabWPtOUSaVCIGTxfQUMFurP9O78liAHqgCmoh0Nh2MR8QClq63XvLpdhDrp byWcVsnttBXC856YH76wDZKXXjtefZHNJ/VtAboCxWGw6jE1RDHTHn0Ynv+i5zMonRXQOB zwdGp22jizx0iei0ui900J8IPxdCYBE= Received: by mail-wr1-f41.google.com with SMTP id ffacd0b85a97d-3863c36a731so3492546f8f.1 for ; Mon, 27 Jan 2025 01:36:41 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc-com.20230601.gappssmtp.com; s=20230601; t=1737970601; x=1738575401; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=YlgUGAmuTWGVKaOGr6DQU8bhY7gyR7CYcPHPJxmYPTE=; b=GJt8qxwJEivfkDPm3YEhl2yUoS4mZI3ixW7Y2n/PHg9nzhjUDwdSsw8TYNp2NEgFEd WoU7Y6J1i2hvcSNhV91mD0trmk01aLdv40N18ec1slPNQLPK/XwlidTJFOHL8jmWlgIM +R1OuD0Osplv2M3APzEupeSKnf9n4QXD5YVjsj06RySUy9MJYt/kDma2XsOYPW82je3N WCXUQD2oUEODVtd44PsvSkRhM16gnQuzD2wotUwxO0lBbJsKiGoRqQit7KNBH+4zxGXH gkAn26yWDglKM2aSbJvai5hsZjP7aECtSGhHMXiiK+OZWWBAffhkjdUX/lF3LYb8ayJd gtHA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1737970601; x=1738575401; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=YlgUGAmuTWGVKaOGr6DQU8bhY7gyR7CYcPHPJxmYPTE=; b=P/vHa1Y0M/Yx93Mm+sNDnRM8BVUPhwPgT/aTa0/MH3MP5XBQubbh4OWefYCDRqeCy3 JCFcg1kN0f/5OUa9jhml1fwavDPtJYi21sU73vNgnWBabbnGw0pI9nljw9MayCNZBLa0 BMsLM5QxFv1VkJ9t2Os6ouKPV+1MoZwStoMW7g2YU3bh9SxWU1a1ivJGGAnfO7gKut97 8brq6TyPwsfV3KzR3g6nXXiprEHoJSd1A4hiTC84P+mtWrwAL0qP5PCrmp04S6XXAYkD 9vYKNSaKYC2Qf1XrEBv843ZhuBIL2xa7tRCFJkIqAoNQ80Xd8b4VS5LW4fi1eBvGKBE2 Bbkw== X-Forwarded-Encrypted: i=1; AJvYcCVFE0IAK6itz8qvI/qqn5l8yKJN5+cF4LnKVcVAI9NJMt+ze7Nczu5nU5Rixbf0u48Nc8Ul5yX91A==@kvack.org X-Gm-Message-State: AOJu0YwQsMuA8T+sBFNc85RNcWu5fkt/UwDy3pNcVP0JyrYPJwhRI/Ju 7rEOvYNR9+YNdzCyvI/caCdaW7lbdkNoO8d3h/7qjzrFkFNruSs6vilGf1OMvN0= X-Gm-Gg: ASbGnct+LWsx8fKzTXSNxcx1i06S3McjqhqQ6bP/aW9kLmO2uskCw9tDt+CMWPQ4RRr a8yf89d8rcFvOjWJ8uYfie7oFaZIowxPmo6agLETFgPlSVP6vxDgLhRUCT9msov4nPoxNCl4vQP P2I/TH8wK3aSBZ0RyBZwzThmGAjE8ZGRpqD77K2giVUDPz9Ophy9CMulQnd9aNuC41dovgirqGa GSmmvagFva5/NnEjdjgTtVi9rju+Vx4XNNlKUJWvxl+tHh+FbFpgbxd1qku0yt7qXW4SFARn0bD bDIQrqmOyI11ZZBvlWgz3PEqEKZQJu8= X-Google-Smtp-Source: AGHT+IHUUXHJ5coMzJl68ATLI/2WB7Q+b+nPXfERL8/PDwCduzlj4e0CTpfsp9bAQ08qd/jskgeAvg== X-Received: by 2002:a05:6000:1445:b0:386:37f5:99f6 with SMTP id ffacd0b85a97d-38bf59ed62fmr33321912f8f.53.1737970600263; Mon, 27 Jan 2025 01:36:40 -0800 (PST) Received: from alex-rivos.ba.rivosinc.com ([2001:861:3382:ef90:3e22:3e78:ce5a:32c3]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-38c2a176490sm10314205f8f.1.2025.01.27.01.36.38 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 27 Jan 2025 01:36:39 -0800 (PST) From: Alexandre Ghiti To: Catalin Marinas , Will Deacon , Ryan Roberts , Mark Rutland , Paul Walmsley , Palmer Dabbelt , Albert Ou , Andrew Morton , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org, linux-mm@kvack.org Cc: Alexandre Ghiti Subject: [PATCH v4 1/9] riscv: Safely remove huge_pte_offset() when manipulating NAPOT ptes Date: Mon, 27 Jan 2025 10:35:22 +0100 Message-Id: <20250127093530.19548-2-alexghiti@rivosinc.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20250127093530.19548-1-alexghiti@rivosinc.com> References: <20250127093530.19548-1-alexghiti@rivosinc.com> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Queue-Id: 1AB9E14000B X-Rspamd-Server: rspam10 X-Stat-Signature: ffbmdbbkwy99ypo37fkhw6a6buaqrqom X-HE-Tag: 1737970601-335683 X-HE-Meta: U2FsdGVkX1/aAzIoloEZdtqb+jxxSv1bTBmVoiFC2+zOsk2+V3UXvo9IFf75rboMmU8nW2XXNpZc1ZazPhdRWYMjvxYbYwuDdhoscq7m7WTyxjs1SwHE6VCHEHXkAm0aDTEvqIKkIKWvx1mIkAbdYmykbMZ5FACLvVNbDRiz7sLPMNhNVXSXoyDFgE0wVUS40NWFeIqERBN/y0vVqecw6iXjRnr2m7ao0S7l8fz3ucPUzhPRLSQ045del/RNIfhF5tcIvylLh559AOBZd1xP/RvUwmbbJm84jIOaWGS0kpv8eblcNz5NiKB6bgmWBMVQX1kR77jnE+u2j0MKMK2yUAMrrDRNKAXJYsYQ8mZYSh5KEVbt7Cio5+JIQaWD5D86eNiAZ2vw6EADtXhCOgBf9IcAmn62IxddR2u3/Uy7JbjxswVCnDU0hCM5J1NF7NoqvP3SKMCTLZtF0+8XuGsFWkoEGOHOOXjIY4VvIgCykxB41zbwnPT3W1DEY1II4pyLjPdzb8g+BYgp1KTfJBV/pC/E0SylyWXOaNzUnffg1biorp8zDvLSzFG2RLjwTE04dObe/O8QDJv8m4HqRsICyZk1pLy+AwcJGe4NsEmDG0Vbn7s8jd/Bd0imkEbI5PZ50JuAjZO4kdI+v39HqUNLJMWnm9x78lpWH7gF9F6ekFcSeZdFDOnWaXuZn/gpyWaSFYavNslj/tszMH5f0vbnGpgYuIYvGz1MvA1a8HCXwSEskfLc/IwCTpjFoFEl0g5nit2Amz1H4Dd00jqeW+0eLwhV10AY6QuG5/UxV8ltyScHcFvXdDOW10gJjAlY30eo36fWtVtmr8xlmlDfWwvMiUjUJd52okKmyJCNCbg43XSLQNz11z1EkMXkNVA04Vo1zr1JuOfiXRMACrxUfEztPWALYWENi7PWDjXNLackMspVQEt9f914PVkMhmHb6uVEHbBXJUNmj/bThd0vD5B 9yEzUdil AtpkvPFVgMyFIAxqrMOtJtEx44VCIz6C5rBfaPdkuuAWj7Y7MRe06Qf9O1/4Rn1FN8lLZNRU/EwW4Z8msF5HaG5FcBOWitAFNrbdzwzXmp+AYDatzOywLFnR6ieLRL9Wz+ngkxGoezak/ji2pn+U4Mm4RA2XHSnxJjXYOihqLOJ5xuqWj+ZsdJoWV9Ooi7Yxba2Rs7zbvaQ2X3GrawDZgPkdo2zPz5ZR7gRKnha5aincrD7fdfA2wNd8lsBJ68vfvo/oIFxJLovhagrYf9uraAC976L77P2c/f/x2TbJdUBHKiV8J12axKiD4VUq8pvZsTWGI/bK/Gkcl/0jjNx25DbD8GBbOkgzamWj9IlKrDWyjVDfNH03x8KI90nIyu63mBE9MFzxp9N5BIHLfRRTLjTY8BbvYGG9KJZBQUL7zmk2dC/v4f4gVh8IGelveEAfLOHGWA1DxFIz1yJbZwI/kRby9fw== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: The pte_t pointer is expected to point to the first entry of the NAPOT mapping so no need to use huge_pte_offset(), similarly to what is done in arm64. Signed-off-by: Alexandre Ghiti --- arch/riscv/mm/hugetlbpage.c | 2 -- 1 file changed, 2 deletions(-) diff --git a/arch/riscv/mm/hugetlbpage.c b/arch/riscv/mm/hugetlbpage.c index 42314f093922..6b09cd1ef41c 100644 --- a/arch/riscv/mm/hugetlbpage.c +++ b/arch/riscv/mm/hugetlbpage.c @@ -276,7 +276,6 @@ int huge_ptep_set_access_flags(struct vm_area_struct *vma, order = napot_cont_order(pte); pte_num = napot_pte_num(order); - ptep = huge_pte_offset(mm, addr, napot_cont_size(order)); orig_pte = get_clear_contig_flush(mm, addr, ptep, pte_num); if (pte_dirty(orig_pte)) @@ -322,7 +321,6 @@ void huge_ptep_set_wrprotect(struct mm_struct *mm, order = napot_cont_order(pte); pte_num = napot_pte_num(order); - ptep = huge_pte_offset(mm, addr, napot_cont_size(order)); orig_pte = get_clear_contig_flush(mm, addr, ptep, pte_num); orig_pte = pte_wrprotect(orig_pte); From patchwork Mon Jan 27 09:35:23 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexandre Ghiti X-Patchwork-Id: 13951118 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id C5066C0218C for ; Mon, 27 Jan 2025 09:37:46 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 607A56B011B; Mon, 27 Jan 2025 04:37:46 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 5B7AF28012F; Mon, 27 Jan 2025 04:37:46 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 47F9E6B011D; Mon, 27 Jan 2025 04:37:46 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 2A5736B011B for ; Mon, 27 Jan 2025 04:37:46 -0500 (EST) Received: from smtpin19.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id D8A3A1620FD for ; Mon, 27 Jan 2025 09:37:45 +0000 (UTC) X-FDA: 83052729690.19.A37A8D7 Received: from mail-wm1-f50.google.com (mail-wm1-f50.google.com [209.85.128.50]) by imf07.hostedemail.com (Postfix) with ESMTP id F217040002 for ; Mon, 27 Jan 2025 09:37:43 +0000 (UTC) Authentication-Results: imf07.hostedemail.com; dkim=pass header.d=rivosinc-com.20230601.gappssmtp.com header.s=20230601 header.b=yk8BvksE; spf=pass (imf07.hostedemail.com: domain of alexghiti@rivosinc.com designates 209.85.128.50 as permitted sender) smtp.mailfrom=alexghiti@rivosinc.com; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1737970664; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=Pa3kN5Pj/U9dm1V/qwQhRimY+zw1O6zUoccW70uHpSw=; b=x2IrGWjlWrG7D16ccckR1lwO5TAhkS3VkL4vP2hm+1jIn7DyHPIkQ3OOfj+xP07oZ1Ax5e D59A28+0/ac2GKpNsLD7fLbI3V/B1+gVtUYP4M6QEg530i/y2BuulvnL/pQRGnC1h5WXBs k2P+/TuaE0m/dUU7KcMEj0yoAZhyBzQ= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1737970664; a=rsa-sha256; cv=none; b=eghMFXYEWTUK9S4VUQSDmwolcz4v2Yi61A1dFLUYa50fVh/MKh60dEPH8WoetmOO4v2TYS MaTVrIggqeX9X27zslHwBDlC4owEg3x+Ror+CIeob5Hi2g6nwL2mAATZJeOUomY64GyM1t BNzKOrRcVHUPBObwzT09nXzeVIjOpX0= ARC-Authentication-Results: i=1; imf07.hostedemail.com; dkim=pass header.d=rivosinc-com.20230601.gappssmtp.com header.s=20230601 header.b=yk8BvksE; spf=pass (imf07.hostedemail.com: domain of alexghiti@rivosinc.com designates 209.85.128.50 as permitted sender) smtp.mailfrom=alexghiti@rivosinc.com; dmarc=none Received: by mail-wm1-f50.google.com with SMTP id 5b1f17b1804b1-4362f61757fso44483605e9.2 for ; Mon, 27 Jan 2025 01:37:43 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc-com.20230601.gappssmtp.com; s=20230601; t=1737970662; x=1738575462; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Pa3kN5Pj/U9dm1V/qwQhRimY+zw1O6zUoccW70uHpSw=; b=yk8BvksEWFYgRU+RTNoZGXZPm+9QEuvtVa3afbeqPNKLF5LTjk3dOWURo/KNTv2nl4 9eEgDa8VGkdTYmU8CdkaJv2oRxWhjnWjtPXZnOuGiX8+CJ09AicS6tPSdOTCLhWxrwcO WgEZfN9fxqnOkWUhtj+JPaXuv+hIlp8HVdrKBBBTtQV73PuH+9HqyNBWHpaoL2V3nBDQ kXK3BN3EHBKVVrZnYc8tvFydDhQwY+gQAvjEm/QedJL/k98YSH+glP39CYhIsOwF9vXY CoIoRu9qK/dftmxeJmzKcEtKIcf3LIUvoNHCn8Qn6FSTaa2VcvSoG/EHmsdYGlNaXNif fW/A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1737970662; x=1738575462; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Pa3kN5Pj/U9dm1V/qwQhRimY+zw1O6zUoccW70uHpSw=; b=bpC5ByhjPkMInWC8bKcxBWoMZQ/IMgQ5yCw+OD9FIKzLBa12XuZMNVoqLuCKvnUz13 HIyieb7rvgITiqqmmaaLW4pRkoLZVwtRSgLP3Ia4CL71yoaLELKB/K/BMpH0xXFhgdSY a6MhhBpkc1YZKcr3xfu7UEQggiBKO/+rTYDLluZvnU+B11iCw9KaCNR8LLgV2IOofYQA Ww/L4g5fqPoaZPGmEqt1P7L589/0znuFBD/lQoaC7gyalMTGZG0eswoXjjCiF/gskg2G SzchP1mdnuK8hvJA0He9uhn/WcuIO6qXzeRZKebpWoQYpkqmGf8JVAWXk8IevTJCuda2 XSlQ== X-Forwarded-Encrypted: i=1; AJvYcCWBZo7XwstYllJReLPsC7HZ2LCpcZbOAw6Ezr174YM6VUlbIg8nKOHy21a/Swge0K0INOLSwAs33g==@kvack.org X-Gm-Message-State: AOJu0YxkcAZaN6zIjjFWVs7wWHdcoI5LfQb/HfUDGXRKODi4n+917yrJ twJLkyn/UK4X3wjXJa0V8ItxtWNk+d4T20bemvierIVll0XywbYcEXwqV4g7PI4= X-Gm-Gg: ASbGncveVAlr3wxBcs/VDRcii0ZvbL/d1BjxZWzPGerT8eQz5/HmCq4UzEKN779LT4F NE08kXLY7JCurCyFE2WVfWPyBuDr+LVhDaXEQ9CSxBFng3jduU/a0izq6LkJZAX5Nt1ZC6J6R5b 5YiFa+DktOHmUSYtGUezp7kB8m1FdBIKFKzJqV8ARQLHvgy4gmWg0il+xY4gUWnBAVhl7CegNgr aiZgFagSTeAoOA2wx9pkEgkK2+oGEyYUVDdetmisBX8Z9qTd8p04E1NYJ7pkwV290nFiAMOfKSe C1Ugcu3aCq+ocCJwZ4yMnxLltNTT8Ug= X-Google-Smtp-Source: AGHT+IHlygR1Rem42sW1brqnRYg+b7cppikaFDk94wrGvPPrCI76J4t5Wx9vdAYW3HFrhkgLyStnBg== X-Received: by 2002:a05:600c:4f05:b0:434:fe62:28c1 with SMTP id 5b1f17b1804b1-438913ef83bmr346516855e9.18.1737970662539; Mon, 27 Jan 2025 01:37:42 -0800 (PST) Received: from alex-rivos.ba.rivosinc.com ([2001:861:3382:ef90:3e22:3e78:ce5a:32c3]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-438b1718741sm100844135e9.0.2025.01.27.01.37.41 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 27 Jan 2025 01:37:42 -0800 (PST) From: Alexandre Ghiti To: Catalin Marinas , Will Deacon , Ryan Roberts , Mark Rutland , Paul Walmsley , Palmer Dabbelt , Albert Ou , Andrew Morton , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org, linux-mm@kvack.org Cc: Alexandre Ghiti Subject: [PATCH v4 2/9] riscv: Restore the pfn in a NAPOT pte when manipulated by core mm code Date: Mon, 27 Jan 2025 10:35:23 +0100 Message-Id: <20250127093530.19548-3-alexghiti@rivosinc.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20250127093530.19548-1-alexghiti@rivosinc.com> References: <20250127093530.19548-1-alexghiti@rivosinc.com> MIME-Version: 1.0 X-Stat-Signature: y9mndwshsts1phxu1risa1zjyyy63moc X-Rspam-User: X-Rspamd-Queue-Id: F217040002 X-Rspamd-Server: rspam03 X-HE-Tag: 1737970663-907803 X-HE-Meta: U2FsdGVkX1+GG4s2klA6wVOm9CrXoSZhaYbrHdtOflbNlJbpkZZnPqrTtrrF4jfHXvlGa7oZAMXICbwjLyEHsR7+eR4q4imqj0nirg3q1Cdl/HGSh3pklTqN1G0tbk9wEQDH35Sn16iPHGDjyFcDGCVugehG06+0WHJGhSdovmtLrAJzZqjcpRpQteSwSLGKpf88sbIh7qVcEncjHR8cj/Mst3vHL8DiuqDVYq3Ru3dSFJfq+HLPDrN3/W6zi8+YH2s/aQP95e4Uoq9RFovLJDEmbR0GnTz2uzYe74+u+X2YFFMtWLtQDYLndg8eGq0iwnKwYIvcmM8aiokoTsq/IoSxGGFz9vA8C+TTfL/3GfHayjoSDh1/CU1IJvbyCtgc4lAwk1eDOv6jjn05oX3KnjskbiO+V3JeeEEQuDfj56lLrXs9jri6R9U5QOPou98sXk7E/42PBcvjzGeFan2YVg0CIjVyfiarT4PGsFta9WRIicylfY/dvL1UYmJBuZbV3V5QwMqGMFD0VgKrxmw8mtxKqVgB7ubhSypbiD3Kdc4/N6jCqJLU3ojGnoWTfUheiMGUdFWj3rp2gKuF3SkKzIvsdAopAia1DJOTI09sMCCITr327tEjPFYfGpYBQ82DUMWNsKN6rXh/mLCY/IyArW9K3DczaHkOFzaSYamsbZRnx5D57n9sxWtS+cj9Y8HfHsTPLUKNMkVn8/7sD1hAEr2V0na65AV7gpciVFzIKlfIrZmbAfjVqh/3S3ovow+9ri25qrEEJxkNegdE1b41NVGLrE70HoDGGVguNWkQb6QYCgAl2ulFz1NFW5NjMvcKmH9OtMlokSEzoTDHkuXI2sak+fCNgiB4Q7t7Waiw32CpL4D5neE/QAuTssMyD5RPmS1fXDfnooltvS8WOo2/mfuu2ZcnyDnDR2dcD9MHslzYALUSKd67tRgi1LRbt6QGEIYa9Q6acS7gNzIowq5 k8N+ZJIz qWa7Yqnrv0hsC0XU4g9RlOvGgmefwnEhOuGUWSkkOla9ogkV+XWpsSXjCOrNAXl8+tSTyZVaqWfSBt5nCjEGEWqkmuhpaOo9M891erdC9UlmtKycGgaFSVv0igRkSzbnOpAaq+30ZY59R5LiEa4i0PE9LdFM7DbnDx2inVlGnbCdrUXwUSfrzPBb9xIXUV0nzNiw2cMmntrTVFNdlnouVmPWTujfFIwxSKjUaBnLFyd+6V9t6Gn+G+dHbjGIJd351K2+j2dbW8ExxmZsXKg/ZsQSq/7QfJner+xAoEeUPXgmdthB5aHsG1bQX7A1KpuTi4D9Ljuhid5LlblVH+IbnzHmkIuyZTNH8+BasP59WpQUWzIRpDhpngeU1A8JmRjA3ttt/UTTZ/luCkhH+DOHpsN3Slb8Y2c3+GazPc55osApNWOY= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: The core mm code expects to be able to extract the pfn from a pte. NAPOT mappings work differently since its ptes actually point to the first pfn of the mapping, the other bits being used to encode the size of the mapping. So modify ptep_get() so that it returns a pte value that contains the *real* pfn (which is then different from what the HW expects) and right before storing the ptes to the page table, reset the pfn LSBs to the size of the mapping. And make sure that all NAPOT mappings are set using set_ptes(). Signed-off-by: Alexandre Ghiti --- arch/riscv/include/asm/pgtable-64.h | 11 ++++ arch/riscv/include/asm/pgtable.h | 91 ++++++++++++++++++++++++++--- arch/riscv/mm/hugetlbpage.c | 9 +-- 3 files changed, 96 insertions(+), 15 deletions(-) diff --git a/arch/riscv/include/asm/pgtable-64.h b/arch/riscv/include/asm/pgtable-64.h index 0897dd99ab8d..cddbe426f618 100644 --- a/arch/riscv/include/asm/pgtable-64.h +++ b/arch/riscv/include/asm/pgtable-64.h @@ -104,6 +104,17 @@ enum napot_cont_order { #define napot_cont_mask(order) (~(napot_cont_size(order) - 1UL)) #define napot_pte_num(order) BIT(order) +static inline bool is_napot_order(unsigned int order) +{ + unsigned int napot_order; + + for_each_napot_order(napot_order) + if (order == napot_order) + return true; + + return false; +} + #ifdef CONFIG_RISCV_ISA_SVNAPOT #define HUGE_MAX_HSTATE (2 + (NAPOT_ORDER_MAX - NAPOT_CONT_ORDER_BASE)) #else diff --git a/arch/riscv/include/asm/pgtable.h b/arch/riscv/include/asm/pgtable.h index 050fdc49b5ad..82b264423b25 100644 --- a/arch/riscv/include/asm/pgtable.h +++ b/arch/riscv/include/asm/pgtable.h @@ -296,6 +296,8 @@ static inline unsigned long pte_napot(pte_t pte) return pte_val(pte) & _PAGE_NAPOT; } +#define pte_valid_napot(pte) (pte_present(pte) && pte_napot(pte)) + static inline pte_t pte_mknapot(pte_t pte, unsigned int order) { int pos = order - 1 + _PAGE_PFN_SHIFT; @@ -305,6 +307,12 @@ static inline pte_t pte_mknapot(pte_t pte, unsigned int order) return __pte((pte_val(pte) & napot_mask) | napot_bit | _PAGE_NAPOT); } +/* pte at entry must *not* encode the mapping size in the pfn LSBs. */ +static inline pte_t pte_clear_napot(pte_t pte) +{ + return __pte(pte_val(pte) & ~_PAGE_NAPOT); +} + #else static __always_inline bool has_svnapot(void) { return false; } @@ -314,17 +322,14 @@ static inline unsigned long pte_napot(pte_t pte) return 0; } +#define pte_valid_napot(pte) false + #endif /* CONFIG_RISCV_ISA_SVNAPOT */ /* Yields the page frame number (PFN) of a page table entry */ static inline unsigned long pte_pfn(pte_t pte) { - unsigned long res = __page_val_to_pfn(pte_val(pte)); - - if (has_svnapot() && pte_napot(pte)) - res = res & (res - 1UL); - - return res; + return __page_val_to_pfn(pte_val(pte)); } #define pte_page(x) pfn_to_page(pte_pfn(x)) @@ -559,8 +564,13 @@ static inline void __set_pte_at(struct mm_struct *mm, pte_t *ptep, pte_t pteval) #define PFN_PTE_SHIFT _PAGE_PFN_SHIFT -static inline void set_ptes(struct mm_struct *mm, unsigned long addr, - pte_t *ptep, pte_t pteval, unsigned int nr) +static inline pte_t __ptep_get(pte_t *ptep) +{ + return READ_ONCE(*ptep); +} + +static inline void __set_ptes(struct mm_struct *mm, unsigned long addr, + pte_t *ptep, pte_t pteval, unsigned int nr) { page_table_check_ptes_set(mm, ptep, pteval, nr); @@ -569,10 +579,13 @@ static inline void set_ptes(struct mm_struct *mm, unsigned long addr, if (--nr == 0) break; ptep++; + + if (unlikely(pte_valid_napot(pteval))) + continue; + pte_val(pteval) += 1 << _PAGE_PFN_SHIFT; } } -#define set_ptes set_ptes static inline void pte_clear(struct mm_struct *mm, unsigned long addr, pte_t *ptep) @@ -627,6 +640,66 @@ static inline int ptep_clear_flush_young(struct vm_area_struct *vma, return ptep_test_and_clear_young(vma, address, ptep); } +#ifdef CONFIG_RISCV_ISA_SVNAPOT +static inline void set_ptes(struct mm_struct *mm, unsigned long addr, + pte_t *ptep, pte_t pteval, unsigned int nr) +{ + if (unlikely(pte_valid_napot(pteval))) { + unsigned int order = ilog2(nr); + + if (!is_napot_order(order)) { + /* + * Something's weird, we are given a NAPOT pte but the + * size of the mapping is not a known NAPOT mapping + * size, so clear the NAPOT bit and map this without + * NAPOT support: core mm only manipulates pte with the + * real pfn so we know the pte is valid without the N + * bit. + */ + pr_err("Incorrect NAPOT mapping, resetting.\n"); + pteval = pte_clear_napot(pteval); + } else { + /* + * NAPOT ptes that arrive here only have the N bit set + * and their pfn does not contain the mapping size, so + * set that here. + */ + pteval = pte_mknapot(pteval, order); + } + } + + __set_ptes(mm, addr, ptep, pteval, nr); +} +#define set_ptes set_ptes + +static inline pte_t ptep_get(pte_t *ptep) +{ + pte_t pte = __ptep_get(ptep); + + /* + * The pte we load has the N bit set and the size of the mapping in + * the pfn LSBs: keep the N bit and replace the mapping size with + * the *real* pfn since the core mm code expects to find it there. + * The mapping size will be reset just before being written to the + * page table in set_ptes(). + */ + if (unlikely(pte_valid_napot(pte))) { + unsigned int order = napot_cont_order(pte); + int pos = order - 1 + _PAGE_PFN_SHIFT; + unsigned long napot_mask = ~GENMASK(pos, _PAGE_PFN_SHIFT); + pte_t *orig_ptep = PTR_ALIGN_DOWN(ptep, sizeof(*ptep) * napot_pte_num(order)); + + pte = __pte((pte_val(pte) & napot_mask) + ((ptep - orig_ptep) << _PAGE_PFN_SHIFT)); + } + + return pte; +} +#define ptep_get ptep_get +#else +#define set_ptes __set_ptes +#define ptep_get __ptep_get +#endif /* CONFIG_RISCV_ISA_SVNAPOT */ + #define pgprot_nx pgprot_nx static inline pgprot_t pgprot_nx(pgprot_t _prot) { diff --git a/arch/riscv/mm/hugetlbpage.c b/arch/riscv/mm/hugetlbpage.c index 6b09cd1ef41c..59ed26ce6857 100644 --- a/arch/riscv/mm/hugetlbpage.c +++ b/arch/riscv/mm/hugetlbpage.c @@ -256,8 +256,7 @@ void set_huge_pte_at(struct mm_struct *mm, clear_flush(mm, addr, ptep, pgsize, pte_num); - for (i = 0; i < pte_num; i++, ptep++, addr += pgsize) - set_pte_at(mm, addr, ptep, pte); + set_ptes(mm, addr, ptep, pte, pte_num); } int huge_ptep_set_access_flags(struct vm_area_struct *vma, @@ -284,8 +283,7 @@ int huge_ptep_set_access_flags(struct vm_area_struct *vma, if (pte_young(orig_pte)) pte = pte_mkyoung(pte); - for (i = 0; i < pte_num; i++, addr += PAGE_SIZE, ptep++) - set_pte_at(mm, addr, ptep, pte); + set_ptes(mm, addr, ptep, pte, pte_num); return true; } @@ -325,8 +323,7 @@ void huge_ptep_set_wrprotect(struct mm_struct *mm, orig_pte = pte_wrprotect(orig_pte); - for (i = 0; i < pte_num; i++, addr += PAGE_SIZE, ptep++) - set_pte_at(mm, addr, ptep, orig_pte); + set_ptes(mm, addr, ptep, orig_pte, pte_num); } pte_t huge_ptep_clear_flush(struct vm_area_struct *vma, From patchwork Mon Jan 27 09:35:24 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexandre Ghiti X-Patchwork-Id: 13951119 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id B8525C0218C for ; Mon, 27 Jan 2025 09:38:48 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 59362280136; Mon, 27 Jan 2025 04:38:48 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 543E428012F; Mon, 27 Jan 2025 04:38:48 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 3E4A1280136; Mon, 27 Jan 2025 04:38:48 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 1B1F528012F for ; Mon, 27 Jan 2025 04:38:48 -0500 (EST) Received: from smtpin03.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 9E2FF1A212C for ; Mon, 27 Jan 2025 09:38:47 +0000 (UTC) X-FDA: 83052732294.03.4338E46 Received: from mail-wr1-f46.google.com (mail-wr1-f46.google.com [209.85.221.46]) by imf13.hostedemail.com (Postfix) with ESMTP id B3C332000B for ; Mon, 27 Jan 2025 09:38:45 +0000 (UTC) Authentication-Results: imf13.hostedemail.com; dkim=pass header.d=rivosinc-com.20230601.gappssmtp.com header.s=20230601 header.b=fqZR03qs; spf=pass (imf13.hostedemail.com: domain of alexghiti@rivosinc.com designates 209.85.221.46 as permitted sender) smtp.mailfrom=alexghiti@rivosinc.com; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1737970725; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=kgdyYq7XD2Yilje0JoNAbIWlWrTDbtDHltbsUNmCh+Y=; b=sKtnMFFErzfGT7bH/7UphjruX/IMxge6zLgrYMJHpf89rfiYbk9EOYOG/Jc0IFDx7phsJo dPsWIQFndlO7dEFTi1YFUpAGHsc60rCux7WRuV1+koced2xxTAxlii42AACTcyqg5cDaSN GDn1pJCj/kucKFF07lUGvwCoWFvCEz0= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1737970725; a=rsa-sha256; cv=none; b=tns4bMw4V34907oCPvlnu1uElwjFmBUT8y4lxhH3qFkmMnpVuvlbFjkJX4Cte93RoChUCm buXQvIgEzhx5+WeamKpbrOC55m6uEUrLx+ox5A32gHPVVyI/IHJhfSR0kTixmce0pMYZ/g rMwTkxfwMLhni1/oYcT88RaK7MB/YWc= ARC-Authentication-Results: i=1; imf13.hostedemail.com; dkim=pass header.d=rivosinc-com.20230601.gappssmtp.com header.s=20230601 header.b=fqZR03qs; spf=pass (imf13.hostedemail.com: domain of alexghiti@rivosinc.com designates 209.85.221.46 as permitted sender) smtp.mailfrom=alexghiti@rivosinc.com; dmarc=none Received: by mail-wr1-f46.google.com with SMTP id ffacd0b85a97d-38634c35129so3842786f8f.3 for ; Mon, 27 Jan 2025 01:38:45 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc-com.20230601.gappssmtp.com; s=20230601; t=1737970724; x=1738575524; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=kgdyYq7XD2Yilje0JoNAbIWlWrTDbtDHltbsUNmCh+Y=; b=fqZR03qsr/oebmlEk6zkr0OxAxx1epgyEK+oHly9NuOwvLkET3evFC74fN1TiEEaS+ yXixvIyHh8v/pZXtru/eSp+MX70FhVWd07+gCIfuK+H3QkJUeGw3bU0D+0G/aDX7L3AX 6Cha6DL9sfB0353mhYDvOVMNHwgleXIs7PDKFnyYkXQuncEXJ3eJOVglNpLyhgVNcOSF LYMNwLv2ttYTvWYpcOrBPvFAN9rERjX+THNZ3ePjBic6r+BnvhjxJNEiXMdPXox1mo1a /iP0qraCtM7NnEXxj//+Q8sI2VDCKGyhwbfoJ0/YyX/ejdavhvCTDh6EhjqaOanXlCBv 7wFg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1737970724; x=1738575524; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=kgdyYq7XD2Yilje0JoNAbIWlWrTDbtDHltbsUNmCh+Y=; b=RYRsmeju9IEkxI6VrfE29DJxHQgi9sn95ZCaAkGWjPlajxWtE/olUMmBhNFUZC2PRR 3wb9/kryhQ/0Xh+peM8XO8FFqUHxxIViRZYcL1EG+ODA8cLz/bhuXgWCiENCqD75s3pH L9HffNL4BSf6xaTLEmKBSFeC3c96bPF/N5TSbUglh0B51pJlot2wWXleNniHEPrD/rR3 2+w4mtLdHSIzGVTOhTc9zy4B7FVvc6tABpCUTDzr9rOcqTAMrQJW9222HmVnzyYm/cYg 61Cz/8DsREur8Xb4zNObX1om5allNqIdwxea/zA1DO9SsTMJFQxq2Z5ukqWnPXD/rmfU yTpA== X-Forwarded-Encrypted: i=1; AJvYcCXPT/j/B0mRLoqAEtOvw63AZBqiT/20zEwIXtc2gEyG0C/JM3937E7oHGUbyhAdHmo7zSMVEbPTBg==@kvack.org X-Gm-Message-State: AOJu0YxHt8l0xdtRgSmd2X0uFzViKgDeXCDoajCrLu15RBy2qYnm3Lod nt31oYUieSAq5DSbW7ZmtZ0XDrUHYmAVNjecfa0q+mCDcUNVXRv4P3GqnYnfWxs= X-Gm-Gg: ASbGncuWwpVxZiUgHkXTSPlPzhA6sM5g2os2X0fwoZgttn44egIXchTa/fJ9vxRdr17 3pGGBeN2NnpWDCUYiTca7cA6n5++l4uTNlsgx/Qqff9LBIsWwlqyzDTduucgaw3tp1WXhsHJryp rv6T1vw7PxZkY23cpm3EHAg/RX9tgrGRoLXNYKzQygm1W6FigM0SKKRmHLW+hsrNPtz1I3LvR5Q ARX0CwJxPHrAKrQgwpinGvE1jrmVNfd9EVg1FNyKf1SSd9YmFynZMMkJswOrp9bkm/snSh5wr8G UUzOFGYIjIpmrjww8pTjKE1UESa7YMA= X-Google-Smtp-Source: AGHT+IHF4Wz5gcDgdKwuRpBVbD6zHBbXFDhSzsPT6V4k0QqTqrpmiB0ymgyz8XwZE3Dv5j9kSPHaVA== X-Received: by 2002:a5d:4312:0:b0:388:e377:8a1b with SMTP id ffacd0b85a97d-38bf57950eemr28290614f8f.28.1737970724282; Mon, 27 Jan 2025 01:38:44 -0800 (PST) Received: from alex-rivos.ba.rivosinc.com ([2001:861:3382:ef90:3e22:3e78:ce5a:32c3]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-38c2a1baf65sm10687893f8f.64.2025.01.27.01.38.43 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 27 Jan 2025 01:38:43 -0800 (PST) From: Alexandre Ghiti To: Catalin Marinas , Will Deacon , Ryan Roberts , Mark Rutland , Paul Walmsley , Palmer Dabbelt , Albert Ou , Andrew Morton , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org, linux-mm@kvack.org Cc: Alexandre Ghiti Subject: [PATCH v4 3/9] mm: Use common huge_ptep_get() function for riscv/arm64 Date: Mon, 27 Jan 2025 10:35:24 +0100 Message-Id: <20250127093530.19548-4-alexghiti@rivosinc.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20250127093530.19548-1-alexghiti@rivosinc.com> References: <20250127093530.19548-1-alexghiti@rivosinc.com> MIME-Version: 1.0 X-Stat-Signature: krtu5abn1eoepbzbnbk5my7tr1kskyhi X-Rspamd-Queue-Id: B3C332000B X-Rspam-User: X-Rspamd-Server: rspam06 X-HE-Tag: 1737970725-475154 X-HE-Meta: U2FsdGVkX1+Fx2tgRxvwmaOt2xO1f/up9Jt/rcX6/l4VGmw2m70M3oH4s9UadLkd5UVLMjms65j5Uaqkd8m412fJOu70PnWXrggB3gsHwAvOfLp/aDuvpNiOtjH4OSoGiPRDYeAwvI9z8cVky4o30hbAOtzNlR3m6d5w9zkaG3E70RIL2VXed42/D3qZ50lUE+TV5a4E4NSfYcciFLb9Wc486v0gX+mAToL00X379q6aU55MqzfYA3ZrXi4lGwbVLwPaINSqj5OWHaZvx6h9yd2In8EcPJv7jCA3AHQNjlRBRn8LWDuw/VoflexGoJfhl59/5RF4QIk6d4BCfTTD1i1bixoFQcb4/9IVfhSvJRvAwOoBryWatMrBjU2Hc+6iM+tYusgYBHtb+X88Ipob79KlHhg66TcPave7IvMbMgA2537wgCcQz3tN+MFki8UlQZtu9X76fl2RIYLmGQNw67f25ZCOLB8n5XtE/JOkuyC3b5kG046lJJwCN8ouPSXI0Y0LQgDg5Fu/KkqWPFVNEpY3CE6o2Vj2vfO1mBp+dHqLIXDvrXlIj+d3ZaHyhAOsOHM4I9L49BxjFet0OucBbB6RcnVicp6BkokPsGUxONpdZK73kippoISP/deJkFlWxInpbTUmLwCki/zQsgyH5dQIQmxQvCjREJTPOoNZ7m4hK+jHSdzkFFXKB5crOHWpiNVhZv5/lVgEkCzmbhVIbaJCUvdQJWWQfyG1s1Vit3RFlw8soNEqYqIKDKcYoSXGe4hq+6lDXbD5HtcT/j6UDLE9AB8tGoqizg1xbgzFT7rVij1SMcK5H58izCiLpBEfeZ40FYCHK1ZuXyKXRdK/iMV2/WgRi0GIwLsU+MvmH4XZKwzSwyJGEW0T/kCzQmKWO75TwF839B1cK2ULKDTwUKNZR96bfFlwnxkXnGEQO9Ngl9htfqPumYOiafljSiV9y70Cbymw3oHmXh1B+ay KM64tQxs 60uyDEVv3691SqZbcTwb/wiF/YgkJoR2zE1CIfXkIPKYP3nPCVivc3ZD4jJJogfWOIH+j+kzUkYH7oasA5QPY7RQ0q2Dt2/A0TN6TWQoy2Xz4VkY8gtym9ZxizF/1MZl88WF1A+q9rteWtV+ncG3OjzgSHRElIhZmLvI10qIo3vAi357uL5tEyml31AdRU6R7g6D92XxMw1HzKolrZclu+zmFMLtsXpaU4Gl5nmfjPUIyg5tdw8sDKHHV1hbsf/uQRsK0PhIFFtdf9NM6Uk7CR/EyyhnkvN3dmxOM5TNOKIKiQqh99BKS+5qLb8NXc7POUSYgfi9HLLMOBYGPfXkz05YPa7D1+V8/bL+olUKfNnC3LVLig5OqhY9v7diGH7o+yDefYk4l0C8GpPU5XRgFosK7S5jXwQtA1SBDLXQnTgp/lwpg1Pf1kEoL8Go3Nux+k0Lvo3kz+FOcwXRvY3slk0regg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: After some adjustments, both architectures have the same implementation so move it to the generic code. Signed-off-by: Alexandre Ghiti --- arch/arm64/Kconfig | 1 + arch/arm64/include/asm/hugetlb.h | 3 +- arch/arm64/include/asm/pgtable.h | 48 +++++++++++++++++++++++++--- arch/arm64/mm/hugetlbpage.c | 55 ++------------------------------ arch/riscv/Kconfig | 1 + arch/riscv/include/asm/hugetlb.h | 6 ++-- arch/riscv/include/asm/pgtable.h | 36 +++++++++++++++++++++ arch/riscv/mm/hugetlbpage.c | 45 ++++++-------------------- include/linux/hugetlb_contpte.h | 12 +++++++ mm/Kconfig | 3 ++ mm/Makefile | 1 + mm/hugetlb_contpte.c | 44 +++++++++++++++++++++++++ 12 files changed, 157 insertions(+), 98 deletions(-) create mode 100644 include/linux/hugetlb_contpte.h create mode 100644 mm/hugetlb_contpte.c diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig index 100570a048c5..fb85d33bfe98 100644 --- a/arch/arm64/Kconfig +++ b/arch/arm64/Kconfig @@ -111,6 +111,7 @@ config ARM64 select ARCH_WANT_DEFAULT_BPF_JIT select ARCH_WANT_DEFAULT_TOPDOWN_MMAP_LAYOUT select ARCH_WANT_FRAME_POINTERS + select ARCH_WANT_GENERAL_HUGETLB_CONTPTE select ARCH_WANT_HUGE_PMD_SHARE if ARM64_4K_PAGES || (ARM64_16K_PAGES && !ARM64_VA_BITS_36) select ARCH_WANT_LD_ORPHAN_WARN select ARCH_WANTS_EXECMEM_LATE if EXECMEM diff --git a/arch/arm64/include/asm/hugetlb.h b/arch/arm64/include/asm/hugetlb.h index c6dff3e69539..27d7f4bdd724 100644 --- a/arch/arm64/include/asm/hugetlb.h +++ b/arch/arm64/include/asm/hugetlb.h @@ -13,6 +13,7 @@ #include #include #include +#include #ifdef CONFIG_ARCH_ENABLE_HUGEPAGE_MIGRATION #define arch_hugetlb_migration_supported arch_hugetlb_migration_supported @@ -53,8 +54,6 @@ extern pte_t huge_ptep_clear_flush(struct vm_area_struct *vma, #define __HAVE_ARCH_HUGE_PTE_CLEAR extern void huge_pte_clear(struct mm_struct *mm, unsigned long addr, pte_t *ptep, unsigned long sz); -#define __HAVE_ARCH_HUGE_PTEP_GET -extern pte_t huge_ptep_get(struct mm_struct *mm, unsigned long addr, pte_t *ptep); void __init arm64_hugetlb_cma_reserve(void); diff --git a/arch/arm64/include/asm/pgtable.h b/arch/arm64/include/asm/pgtable.h index 6986345b537a..cebbfcfb0e53 100644 --- a/arch/arm64/include/asm/pgtable.h +++ b/arch/arm64/include/asm/pgtable.h @@ -420,9 +420,10 @@ static inline pte_t pte_advance_pfn(pte_t pte, unsigned long nr) return pfn_pte(pte_pfn(pte) + nr, pte_pgprot(pte)); } -static inline void __set_ptes(struct mm_struct *mm, - unsigned long __always_unused addr, - pte_t *ptep, pte_t pte, unsigned int nr) +static inline void ___set_ptes(struct mm_struct *mm, + unsigned long __always_unused addr, + pte_t *ptep, pte_t pte, unsigned int nr, + size_t pgsize) { page_table_check_ptes_set(mm, ptep, pte, nr); __sync_cache_and_tags(pte, nr); @@ -433,10 +434,15 @@ static inline void __set_ptes(struct mm_struct *mm, if (--nr == 0) break; ptep++; - pte = pte_advance_pfn(pte, 1); + pte = pte_advance_pfn(pte, pgsize >> PAGE_SHIFT); } } +#define __set_ptes(mm, addr, ptep, pte, nr) \ + ___set_ptes(mm, addr, ptep, pte, nr, PAGE_SIZE) + +#define set_contptes ___set_ptes + /* * Hugetlb definitions. */ @@ -1825,6 +1831,40 @@ static inline void clear_young_dirty_ptes(struct vm_area_struct *vma, #endif /* CONFIG_ARM64_CONTPTE */ +static inline int arch_contpte_get_num_contig(pte_t *ptep, + unsigned long size, + size_t *pgsize) +{ + int contig_ptes = 0; + + if (pgsize) + *pgsize = size; + + switch (size) { +#ifndef __PAGETABLE_PMD_FOLDED + case PUD_SIZE: + if (pud_sect_supported()) + contig_ptes = 1; + break; +#endif + case PMD_SIZE: + contig_ptes = 1; + break; + case CONT_PMD_SIZE: + if (pgsize) + *pgsize = PMD_SIZE; + contig_ptes = CONT_PMDS; + break; + case CONT_PTE_SIZE: + if (pgsize) + *pgsize = PAGE_SIZE; + contig_ptes = CONT_PTES; + break; + } + + return contig_ptes; +} + #endif /* !__ASSEMBLY__ */ #endif /* __ASM_PGTABLE_H */ diff --git a/arch/arm64/mm/hugetlbpage.c b/arch/arm64/mm/hugetlbpage.c index 3215adf48a1b..3458461adb90 100644 --- a/arch/arm64/mm/hugetlbpage.c +++ b/arch/arm64/mm/hugetlbpage.c @@ -98,57 +98,6 @@ static int find_num_contig(struct mm_struct *mm, unsigned long addr, return CONT_PTES; } -static inline int num_contig_ptes(unsigned long size, size_t *pgsize) -{ - int contig_ptes = 0; - - *pgsize = size; - - switch (size) { -#ifndef __PAGETABLE_PMD_FOLDED - case PUD_SIZE: - if (pud_sect_supported()) - contig_ptes = 1; - break; -#endif - case PMD_SIZE: - contig_ptes = 1; - break; - case CONT_PMD_SIZE: - *pgsize = PMD_SIZE; - contig_ptes = CONT_PMDS; - break; - case CONT_PTE_SIZE: - *pgsize = PAGE_SIZE; - contig_ptes = CONT_PTES; - break; - } - - return contig_ptes; -} - -pte_t huge_ptep_get(struct mm_struct *mm, unsigned long addr, pte_t *ptep) -{ - int ncontig, i; - size_t pgsize; - pte_t orig_pte = __ptep_get(ptep); - - if (!pte_present(orig_pte) || !pte_cont(orig_pte)) - return orig_pte; - - ncontig = num_contig_ptes(page_size(pte_page(orig_pte)), &pgsize); - for (i = 0; i < ncontig; i++, ptep++) { - pte_t pte = __ptep_get(ptep); - - if (pte_dirty(pte)) - orig_pte = pte_mkdirty(orig_pte); - - if (pte_young(pte)) - orig_pte = pte_mkyoung(orig_pte); - } - return orig_pte; -} - /* * Changing some bits of contiguous entries requires us to follow a * Break-Before-Make approach, breaking the whole contiguous set @@ -229,7 +178,7 @@ void set_huge_pte_at(struct mm_struct *mm, unsigned long addr, unsigned long pfn, dpfn; pgprot_t hugeprot; - ncontig = num_contig_ptes(sz, &pgsize); + ncontig = arch_contpte_get_num_contig(ptep, sz, &pgsize); if (!pte_present(pte)) { for (i = 0; i < ncontig; i++, ptep++, addr += pgsize) @@ -390,7 +339,7 @@ void huge_pte_clear(struct mm_struct *mm, unsigned long addr, int i, ncontig; size_t pgsize; - ncontig = num_contig_ptes(sz, &pgsize); + ncontig = arch_contpte_get_num_contig(ptep, sz, &pgsize); for (i = 0; i < ncontig; i++, addr += pgsize, ptep++) __pte_clear(mm, addr, ptep); diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig index d4a7ca0388c0..2fe8c68fba85 100644 --- a/arch/riscv/Kconfig +++ b/arch/riscv/Kconfig @@ -78,6 +78,7 @@ config RISCV select ARCH_WANT_DEFAULT_TOPDOWN_MMAP_LAYOUT if MMU select ARCH_WANT_FRAME_POINTERS select ARCH_WANT_GENERAL_HUGETLB if !RISCV_ISA_SVNAPOT + select ARCH_WANT_GENERAL_HUGETLB_CONTPTE if RISCV_ISA_SVNAPOT select ARCH_WANT_HUGE_PMD_SHARE if 64BIT select ARCH_WANT_LD_ORPHAN_WARN if !XIP_KERNEL select ARCH_WANT_OPTIMIZE_DAX_VMEMMAP diff --git a/arch/riscv/include/asm/hugetlb.h b/arch/riscv/include/asm/hugetlb.h index faf3624d8057..d9f9bfb84908 100644 --- a/arch/riscv/include/asm/hugetlb.h +++ b/arch/riscv/include/asm/hugetlb.h @@ -4,6 +4,9 @@ #include #include +#ifdef CONFIG_ARCH_WANT_GENERAL_HUGETLB_CONTPTE +#include +#endif static inline void arch_clear_hugetlb_flags(struct folio *folio) { @@ -43,9 +46,6 @@ int huge_ptep_set_access_flags(struct vm_area_struct *vma, unsigned long addr, pte_t *ptep, pte_t pte, int dirty); -#define __HAVE_ARCH_HUGE_PTEP_GET -pte_t huge_ptep_get(struct mm_struct *mm, unsigned long addr, pte_t *ptep); - pte_t arch_make_huge_pte(pte_t entry, unsigned int shift, vm_flags_t flags); #define arch_make_huge_pte arch_make_huge_pte diff --git a/arch/riscv/include/asm/pgtable.h b/arch/riscv/include/asm/pgtable.h index 82b264423b25..d4e6427b8ca9 100644 --- a/arch/riscv/include/asm/pgtable.h +++ b/arch/riscv/include/asm/pgtable.h @@ -296,6 +296,8 @@ static inline unsigned long pte_napot(pte_t pte) return pte_val(pte) & _PAGE_NAPOT; } +#define pte_cont pte_napot + #define pte_valid_napot(pte) (pte_present(pte) && pte_napot(pte)) static inline pte_t pte_mknapot(pte_t pte, unsigned int order) @@ -587,6 +589,38 @@ static inline void __set_ptes(struct mm_struct *mm, unsigned long addr, } } +#ifdef CONFIG_RISCV_ISA_SVNAPOT +static inline int arch_contpte_get_num_contig(pte_t *ptep, unsigned long size, + size_t *pgsize) +{ + unsigned long hugepage_shift; + pte_t __pte; + + if (size >= PGDIR_SIZE) + hugepage_shift = PGDIR_SHIFT; + else if (size >= P4D_SIZE) + hugepage_shift = P4D_SHIFT; + else if (size >= PUD_SIZE) + hugepage_shift = PUD_SHIFT; + else if (size >= PMD_SIZE) + hugepage_shift = PMD_SHIFT; + else + hugepage_shift = PAGE_SHIFT; + + if (pgsize) + *pgsize = BIT(hugepage_shift); + + /* We must read the raw value of the pte to get the size of the mapping */ + __pte = __ptep_get(ptep); + + /* Make sure __pte is not a swap entry */ + if (pte_valid_napot(__pte)) + return napot_pte_num(napot_cont_order(__pte)); + + return size >> hugepage_shift; +} +#endif + static inline void pte_clear(struct mm_struct *mm, unsigned long addr, pte_t *ptep) { @@ -671,6 +705,8 @@ static inline void set_ptes(struct mm_struct *mm, unsigned long addr, __set_ptes(mm, addr, ptep, pteval, nr); } #define set_ptes set_ptes +#define set_contptes(mm, addr, ptep, pte, nr, pgsize) \ + set_ptes(mm, addr, ptep, pte, nr) static inline pte_t ptep_get(pte_t *ptep) { diff --git a/arch/riscv/mm/hugetlbpage.c b/arch/riscv/mm/hugetlbpage.c index 59ed26ce6857..d51863824540 100644 --- a/arch/riscv/mm/hugetlbpage.c +++ b/arch/riscv/mm/hugetlbpage.c @@ -3,30 +3,6 @@ #include #ifdef CONFIG_RISCV_ISA_SVNAPOT -pte_t huge_ptep_get(struct mm_struct *mm, unsigned long addr, pte_t *ptep) -{ - unsigned long pte_num; - int i; - pte_t orig_pte = ptep_get(ptep); - - if (!pte_present(orig_pte) || !pte_napot(orig_pte)) - return orig_pte; - - pte_num = napot_pte_num(napot_cont_order(orig_pte)); - - for (i = 0; i < pte_num; i++, ptep++) { - pte_t pte = ptep_get(ptep); - - if (pte_dirty(pte)) - orig_pte = pte_mkdirty(orig_pte); - - if (pte_young(pte)) - orig_pte = pte_mkyoung(orig_pte); - } - - return orig_pte; -} - pte_t *huge_pte_alloc(struct mm_struct *mm, struct vm_area_struct *vma, unsigned long addr, @@ -266,15 +242,13 @@ int huge_ptep_set_access_flags(struct vm_area_struct *vma, int dirty) { struct mm_struct *mm = vma->vm_mm; - unsigned long order; pte_t orig_pte; - int i, pte_num; + int pte_num; if (!pte_napot(pte)) return ptep_set_access_flags(vma, addr, ptep, pte, dirty); - order = napot_cont_order(pte); - pte_num = napot_pte_num(order); + pte_num = arch_contpte_get_num_contig(ptep, 0, NULL); orig_pte = get_clear_contig_flush(mm, addr, ptep, pte_num); if (pte_dirty(orig_pte)) @@ -298,7 +272,7 @@ pte_t huge_ptep_get_and_clear(struct mm_struct *mm, if (!pte_napot(orig_pte)) return ptep_get_and_clear(mm, addr, ptep); - pte_num = napot_pte_num(napot_cont_order(orig_pte)); + pte_num = arch_contpte_get_num_contig(ptep, 0, NULL); return get_clear_contig(mm, addr, ptep, pte_num); } @@ -308,17 +282,15 @@ void huge_ptep_set_wrprotect(struct mm_struct *mm, pte_t *ptep) { pte_t pte = ptep_get(ptep); - unsigned long order; pte_t orig_pte; - int i, pte_num; + int pte_num; if (!pte_napot(pte)) { ptep_set_wrprotect(mm, addr, ptep); return; } - order = napot_cont_order(pte); - pte_num = napot_pte_num(order); + pte_num = arch_contpte_get_num_contig(ptep, 0, NULL); orig_pte = get_clear_contig_flush(mm, addr, ptep, pte_num); orig_pte = pte_wrprotect(orig_pte); @@ -336,7 +308,7 @@ pte_t huge_ptep_clear_flush(struct vm_area_struct *vma, if (!pte_napot(pte)) return ptep_clear_flush(vma, addr, ptep); - pte_num = napot_pte_num(napot_cont_order(pte)); + pte_num = arch_contpte_get_num_contig(ptep, 0, NULL); return get_clear_contig_flush(vma->vm_mm, addr, ptep, pte_num); } @@ -346,6 +318,7 @@ void huge_pte_clear(struct mm_struct *mm, pte_t *ptep, unsigned long sz) { + size_t pgsize; pte_t pte = ptep_get(ptep); int i, pte_num; @@ -354,8 +327,8 @@ void huge_pte_clear(struct mm_struct *mm, return; } - pte_num = napot_pte_num(napot_cont_order(pte)); - for (i = 0; i < pte_num; i++, addr += PAGE_SIZE, ptep++) + pte_num = arch_contpte_get_num_contig(ptep, sz, &pgsize); + for (i = 0; i < pte_num; i++, addr += pgsize, ptep++) pte_clear(mm, addr, ptep); } diff --git a/include/linux/hugetlb_contpte.h b/include/linux/hugetlb_contpte.h new file mode 100644 index 000000000000..ec4189cd65b8 --- /dev/null +++ b/include/linux/hugetlb_contpte.h @@ -0,0 +1,12 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (C) 2024 Rivos Inc. + */ + +#ifndef _LINUX_HUGETLB_CONTPTE_H +#define _LINUX_HUGETLB_CONTPTE_H + +#define __HAVE_ARCH_HUGE_PTEP_GET +extern pte_t huge_ptep_get(struct mm_struct *mm, unsigned long addr, pte_t *ptep); + +#endif /* _LINUX_HUGETLB_CONTPTE_H */ diff --git a/mm/Kconfig b/mm/Kconfig index 84000b016808..8cd38de612ce 100644 --- a/mm/Kconfig +++ b/mm/Kconfig @@ -810,6 +810,9 @@ config NOMMU_INITIAL_TRIM_EXCESS config ARCH_WANT_GENERAL_HUGETLB bool +config ARCH_WANT_GENERAL_HUGETLB_CONTPTE + bool + config ARCH_WANTS_THP_SWAP def_bool n diff --git a/mm/Makefile b/mm/Makefile index dba52bb0da8a..1c1250fbb020 100644 --- a/mm/Makefile +++ b/mm/Makefile @@ -96,6 +96,7 @@ obj-$(CONFIG_MIGRATION) += migrate.o obj-$(CONFIG_NUMA) += memory-tiers.o obj-$(CONFIG_DEVICE_MIGRATION) += migrate_device.o obj-$(CONFIG_TRANSPARENT_HUGEPAGE) += huge_memory.o khugepaged.o +obj-$(CONFIG_ARCH_WANT_GENERAL_HUGETLB_CONTPTE) += hugetlb_contpte.o obj-$(CONFIG_PAGE_COUNTER) += page_counter.o obj-$(CONFIG_MEMCG_V1) += memcontrol-v1.o obj-$(CONFIG_MEMCG) += memcontrol.o vmpressure.o diff --git a/mm/hugetlb_contpte.c b/mm/hugetlb_contpte.c new file mode 100644 index 000000000000..a03e91d3efb1 --- /dev/null +++ b/mm/hugetlb_contpte.c @@ -0,0 +1,44 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright 2025 Rivos Inc. + */ + +#include +#include +#include + +/* + * Any arch that wants to use that needs to define: + * - __ptep_get() + * - pte_cont() + * - arch_contpte_get_num_contig() + */ + +/* + * This file implements the following contpte aware API: + * - huge_ptep_get() + */ + +pte_t huge_ptep_get(struct mm_struct *mm, unsigned long addr, pte_t *ptep) +{ + int ncontig, i; + pte_t orig_pte = __ptep_get(ptep); + + if (!pte_present(orig_pte) || !pte_cont(orig_pte)) + return orig_pte; + + ncontig = arch_contpte_get_num_contig(ptep, + page_size(pte_page(orig_pte)), + NULL); + + for (i = 0; i < ncontig; i++, ptep++) { + pte_t pte = __ptep_get(ptep); + + if (pte_dirty(pte)) + orig_pte = pte_mkdirty(orig_pte); + + if (pte_young(pte)) + orig_pte = pte_mkyoung(orig_pte); + } + return orig_pte; +} From patchwork Mon Jan 27 09:35:25 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexandre Ghiti X-Patchwork-Id: 13951127 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id BCC06C02190 for ; Mon, 27 Jan 2025 09:39:50 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 4A9FE280139; Mon, 27 Jan 2025 04:39:50 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 45AED28012F; Mon, 27 Jan 2025 04:39:50 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 2FB67280139; Mon, 27 Jan 2025 04:39:50 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 1114128012F for ; Mon, 27 Jan 2025 04:39:50 -0500 (EST) Received: from smtpin14.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id C2984122101 for ; Mon, 27 Jan 2025 09:39:49 +0000 (UTC) X-FDA: 83052734898.14.3F18052 Received: from mail-wm1-f43.google.com (mail-wm1-f43.google.com [209.85.128.43]) by imf14.hostedemail.com (Postfix) with ESMTP id CEF93100014 for ; Mon, 27 Jan 2025 09:39:47 +0000 (UTC) Authentication-Results: imf14.hostedemail.com; dkim=pass header.d=rivosinc-com.20230601.gappssmtp.com header.s=20230601 header.b=zaAFNvbN; spf=pass (imf14.hostedemail.com: domain of alexghiti@rivosinc.com designates 209.85.128.43 as permitted sender) smtp.mailfrom=alexghiti@rivosinc.com; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1737970787; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=8mhN3d8PsNley3p53cGFp5ftMvicJk5x5kFgKHD2MsE=; b=wN8v8McnhkTskRAoV7So6QCfCvLg+oMb7E1kg8A9vaBW++Ha5Cdho2VFJDbv82ddrmzxxi FdzE0ILi+x70bBBWcH4rCdyJM4ooMqiemhQx+XqVLQu9uP1anYXHxOSINNslxlwPmWxH+m TJqzymxQJzwGdFUiC7+fEYB8DPo27ac= ARC-Authentication-Results: i=1; imf14.hostedemail.com; dkim=pass header.d=rivosinc-com.20230601.gappssmtp.com header.s=20230601 header.b=zaAFNvbN; spf=pass (imf14.hostedemail.com: domain of alexghiti@rivosinc.com designates 209.85.128.43 as permitted sender) smtp.mailfrom=alexghiti@rivosinc.com; dmarc=none ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1737970787; a=rsa-sha256; cv=none; b=bxMczWGepS4Fz+LBYTvh7sUObp3dLGr4cjiARQIihQ/tdJ5bqi9bbTLwBK5xBCf2W8TVrw Rv4g4NSMyPdC8icMSEw7AbAcl9VY/kR/R9XfdS0gepF3pZ5u5YDw5HKp0KpmR3rjgH1uTJ Jn8vX2dhs8m54zeiD6GITtAY+IqLhok= Received: by mail-wm1-f43.google.com with SMTP id 5b1f17b1804b1-4361815b96cso27341565e9.1 for ; Mon, 27 Jan 2025 01:39:47 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc-com.20230601.gappssmtp.com; s=20230601; t=1737970786; x=1738575586; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=8mhN3d8PsNley3p53cGFp5ftMvicJk5x5kFgKHD2MsE=; b=zaAFNvbNWOffshaNH6a2zpUM+XMi4MpgUpMDNbo4I9ZNuBCXm46SaQYLN6uOXSf5m2 7shVfF6iiCb9PaierSO7NJ5bC7s0zJ6qv4qFVRzP4buC7N666m05I10tVxJvQTHZTMul OvewO/SjFUYWHTl2QNm3FAOlsqcJ7FP2U8goieASnVpUSPuFAqPuciyAvltPIqfdRuaU hL2JDhb7Q6N/g6i6Fyw2MeiTFtZOp5Z1T3TX2N0Kif0YPJm0cYFsn51cgsGwWvT6koox m1YUOTkd+INRLRhej+WqqwzKZBTzB7eZtnrdM3tPmOOo36nInaGCwwzpFe2Wzln+brEw x2Bw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1737970786; x=1738575586; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=8mhN3d8PsNley3p53cGFp5ftMvicJk5x5kFgKHD2MsE=; b=dVzNUImYCBEscH0hC5rdErXXtB3RZ5htr3lzMYoChRWrvMFNiBiGwfrLKJxSGgvQVB 7uFixC/fCpNg654sVcqK2ixwdwkscJt2I2EOhaNCJ58BR/bs3LAD/lxeCI7tgeRSc9cb WtF5t1Mzqw4PJgaWGoSEwKzV+9+y//8GaYWqR6fFyoyRavNK9+I8jXV3/P1coHmNq1h7 j89DbU7yX2M8B1aRKik/aqqT3vcpBk/1aJZEI3FVUaG8DZPK5zbYvIgRc5jBAhuquEH5 x9W0MaGHWaEOr0KHRndP7oL8endfJ+BeOj0WmhA3ahkEORPq+/Nfa2UggJ3CgkhJ4Xj+ jvmQ== X-Forwarded-Encrypted: i=1; AJvYcCVleHsRR06GNOTT6NT7YGjdQgwaQZXH0rKoHIbIeyu04Kno926p+OAinWQpTciHbQbI6uwT5C7ltw==@kvack.org X-Gm-Message-State: AOJu0Yx10Q/cS17NlhhUzqstGHgGv2DpF+9PxaQvJ0tGjqoQDOl+y3Ji j9P70EuVmedGyvdbMSoTtlUfNkmJkfLRF1/FGDKq8i/9WH+FqIOJvhWp1GwmTVA= X-Gm-Gg: ASbGncttUFldFuLtes8+tPqsQ2jZYMbqqKxuEIYzhhD6O2+4oyd+nmZzchyP2jLnjrl xMXjRlQ7aEgm+Q39yInRaD9PF8Di7dMNIPF5YNfrQMcb4Li5LbEDmg6suUVwlhe006hr6SyvCId Xdi5nRLQL+Rb7RKa3lJr5TLxMZAGBivTryo18PT6rrFqFBYWv7aeiF7xhkzHIqPmYp6Lu8zXLQ/ EKkEAkHX5oO+3asFxXBSL6tGf9yzIfUDZKGOoNxMbtLrjuDMFAjDY/kdvxGdLODR4YBSgJx4DG5 YULHmYJfh2O5FHExXQa14yLeoFmEvPM= X-Google-Smtp-Source: AGHT+IFb60eslNQdePOaLfXv36Z0OpFdyDEXUXYChc5sJwAPhp8i6cvPTkFCm9BP5tmum6xqjAkWnA== X-Received: by 2002:a05:6000:4022:b0:385:e013:73f6 with SMTP id ffacd0b85a97d-38bf59eff21mr37315469f8f.50.1737970786287; Mon, 27 Jan 2025 01:39:46 -0800 (PST) Received: from alex-rivos.ba.rivosinc.com ([2001:861:3382:ef90:3e22:3e78:ce5a:32c3]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-38c2a1bb040sm10590537f8f.67.2025.01.27.01.39.45 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 27 Jan 2025 01:39:46 -0800 (PST) From: Alexandre Ghiti To: Catalin Marinas , Will Deacon , Ryan Roberts , Mark Rutland , Paul Walmsley , Palmer Dabbelt , Albert Ou , Andrew Morton , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org, linux-mm@kvack.org Cc: Alexandre Ghiti Subject: [PATCH v4 4/9] mm: Use common set_huge_pte_at() function for riscv/arm64 Date: Mon, 27 Jan 2025 10:35:25 +0100 Message-Id: <20250127093530.19548-5-alexghiti@rivosinc.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20250127093530.19548-1-alexghiti@rivosinc.com> References: <20250127093530.19548-1-alexghiti@rivosinc.com> MIME-Version: 1.0 X-Rspamd-Server: rspam02 X-Rspamd-Queue-Id: CEF93100014 X-Stat-Signature: xz9fk5igod4nfp7o4gbrt66qpythgmdt X-Rspam-User: X-HE-Tag: 1737970787-579782 X-HE-Meta: U2FsdGVkX18DEKz0F2POj4u/tK3ENlah8GvFXDIbACAxxm49ylPDoWIDbd4mNWO+HLgj7GzuztASHL/qhSy7Z3LcaOn/dTobZhz6COfB9KrKKHbWa/8Wdyqhi75HC6faH0eu8+YAWL4nacVp2GDLlYE+klXZAxcygnCmeguMUIgKBgWhry9x4B3bAmkNXl7RHEMy9i8L3dldoM2CXBpxv8X5kStH1sjSuC/73pCaz+Bgh7I2sh/lDHyTsIxycbrLNWYiQz6ugsWYKUjMY8SjJKEu4l+Lka+Lv/9ney/UwyYGuP2MOBX3DJZ8csBImVFiIhWv4WUGFQVfqKyAawIUY4xpBKuKCrQ527w3C23JJ31aOWM0thQ5J7dwgHqqubzO8ZH7gzIV2dtNe+M1xJa3Pc+G1Q5GHRUS8V4ZvjOGNL8QEsFso3khrEjWKbBaEgi2ltNAuPgfYi5pC6vp2Ujh9lmoFNmU9bbAhWWx8LK51oKpw6TuIaBERCqIDTqTqvZhYfxKgATQIl8jslN837/tNBedJXdPxEOSJZAsSags6nxIGk26c9CFRbzK2XPV36olLCDQQ1kf9ok4BqHcmDNH/bewbVt79Ov+cOjvqb9GMX7Pmr9tY20n++rqTY26Z1JBujrGFx9tRYeLLzyPx8DnsdvQZ+NWG4KGMfH+EQP20uKBLxqSKmbHIOJyI/Q7qCHGe1UsgbxcaZjrgKx0g9wkQTQcwoERNg9TLS8pnsNIo7wlorNwFNjYHQU9nm7tREgeXQbTLiwUFCr/M6D5MF9GMLwA3raLB1JmvLR2aMn0RcRPL6CMXO8erfRHKBxu9KDa+G2ic+3KsJT4ajQ59RSpZWL7pRnoXZ3I2fiq77Lvc8/SDgQ6q5fli/jEFYEvACvq2vLimhutdf/dxJI5QRXP+3+vUFfCsR9LWgi+3ctMJzmg1i6v7Zmg/O8k95P3BKiGnnjYLMv4MaP9tianl/z Yn1v14J/ jm396gpl/XY/JnOzjkS3icyojDVdTECruYz9FGpNhm3PhSOMaPPbumBfF9TCqpmTWHPqu0I+amblWDE7jbz/Hpevs84GYJ8nUAT+e8SQfw0dPQ7BoJFVT+mZDBzgfRat+Vms2UCO5sjlb/DAXh/YBwaXiP92s6oqnMaTesjH92ibv55GMd75xcURYasO5MZuRUvP0RsDUXvIGLDXhO3H3TjEszNVaNxt3vLwVuTtVDVDFiszHrUNJVaK0+zwIb2IsuML9ljkwr2dbrWQo1pAq4h5+ciXNnCdcNm5dmifMEyxo/BZZEA5WKJ6uFPj49hGT9gdoxKBqcLxbaBsOtYv9jZrSCKO4zZOyPwnpukLA380JsmChuNEN3b3JZ3O7DgbdiPrQzWygkmvn3ca77Vcc19JIMALwUSX+4oPlz7+Ci21M0i82E0huS6lPNLbYIZ+1l+OQORaxQCrMTcdOWODIvqbd/A== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: After some adjustments, both architectures have the same implementation so move it to the generic code. Signed-off-by: Alexandre Ghiti --- arch/arm64/include/asm/hugetlb.h | 3 -- arch/arm64/mm/hugetlbpage.c | 56 ----------------------------- arch/riscv/include/asm/hugetlb.h | 5 --- arch/riscv/include/asm/pgtable.h | 8 +++-- arch/riscv/mm/hugetlbpage.c | 62 -------------------------------- include/linux/hugetlb_contpte.h | 5 +++ mm/hugetlb_contpte.c | 59 ++++++++++++++++++++++++++++++ 7 files changed, 69 insertions(+), 129 deletions(-) diff --git a/arch/arm64/include/asm/hugetlb.h b/arch/arm64/include/asm/hugetlb.h index 27d7f4bdd724..40d87a563093 100644 --- a/arch/arm64/include/asm/hugetlb.h +++ b/arch/arm64/include/asm/hugetlb.h @@ -35,9 +35,6 @@ static inline void arch_clear_hugetlb_flags(struct folio *folio) pte_t arch_make_huge_pte(pte_t entry, unsigned int shift, vm_flags_t flags); #define arch_make_huge_pte arch_make_huge_pte -#define __HAVE_ARCH_HUGE_SET_HUGE_PTE_AT -extern void set_huge_pte_at(struct mm_struct *mm, unsigned long addr, - pte_t *ptep, pte_t pte, unsigned long sz); #define __HAVE_ARCH_HUGE_PTEP_SET_ACCESS_FLAGS extern int huge_ptep_set_access_flags(struct vm_area_struct *vma, unsigned long addr, pte_t *ptep, diff --git a/arch/arm64/mm/hugetlbpage.c b/arch/arm64/mm/hugetlbpage.c index 3458461adb90..02de680a6a0d 100644 --- a/arch/arm64/mm/hugetlbpage.c +++ b/arch/arm64/mm/hugetlbpage.c @@ -145,62 +145,6 @@ static pte_t get_clear_contig_flush(struct mm_struct *mm, return orig_pte; } -/* - * Changing some bits of contiguous entries requires us to follow a - * Break-Before-Make approach, breaking the whole contiguous set - * before we can change any entries. See ARM DDI 0487A.k_iss10775, - * "Misprogramming of the Contiguous bit", page D4-1762. - * - * This helper performs the break step for use cases where the - * original pte is not needed. - */ -static void clear_flush(struct mm_struct *mm, - unsigned long addr, - pte_t *ptep, - unsigned long pgsize, - unsigned long ncontig) -{ - struct vm_area_struct vma = TLB_FLUSH_VMA(mm, 0); - unsigned long i, saddr = addr; - - for (i = 0; i < ncontig; i++, addr += pgsize, ptep++) - __ptep_get_and_clear(mm, addr, ptep); - - flush_tlb_range(&vma, saddr, addr); -} - -void set_huge_pte_at(struct mm_struct *mm, unsigned long addr, - pte_t *ptep, pte_t pte, unsigned long sz) -{ - size_t pgsize; - int i; - int ncontig; - unsigned long pfn, dpfn; - pgprot_t hugeprot; - - ncontig = arch_contpte_get_num_contig(ptep, sz, &pgsize); - - if (!pte_present(pte)) { - for (i = 0; i < ncontig; i++, ptep++, addr += pgsize) - __set_ptes(mm, addr, ptep, pte, 1); - return; - } - - if (!pte_cont(pte)) { - __set_ptes(mm, addr, ptep, pte, 1); - return; - } - - pfn = pte_pfn(pte); - dpfn = pgsize >> PAGE_SHIFT; - hugeprot = pte_pgprot(pte); - - clear_flush(mm, addr, ptep, pgsize, ncontig); - - for (i = 0; i < ncontig; i++, ptep++, addr += pgsize, pfn += dpfn) - __set_ptes(mm, addr, ptep, pfn_pte(pfn, hugeprot), 1); -} - pte_t *huge_pte_alloc(struct mm_struct *mm, struct vm_area_struct *vma, unsigned long addr, unsigned long sz) { diff --git a/arch/riscv/include/asm/hugetlb.h b/arch/riscv/include/asm/hugetlb.h index d9f9bfb84908..28cbf5d761e1 100644 --- a/arch/riscv/include/asm/hugetlb.h +++ b/arch/riscv/include/asm/hugetlb.h @@ -24,11 +24,6 @@ bool arch_hugetlb_migration_supported(struct hstate *h); void huge_pte_clear(struct mm_struct *mm, unsigned long addr, pte_t *ptep, unsigned long sz); -#define __HAVE_ARCH_HUGE_SET_HUGE_PTE_AT -void set_huge_pte_at(struct mm_struct *mm, - unsigned long addr, pte_t *ptep, pte_t pte, - unsigned long sz); - #define __HAVE_ARCH_HUGE_PTEP_GET_AND_CLEAR pte_t huge_ptep_get_and_clear(struct mm_struct *mm, unsigned long addr, pte_t *ptep); diff --git a/arch/riscv/include/asm/pgtable.h b/arch/riscv/include/asm/pgtable.h index d4e6427b8ca9..74d29d0af172 100644 --- a/arch/riscv/include/asm/pgtable.h +++ b/arch/riscv/include/asm/pgtable.h @@ -634,9 +634,8 @@ extern int ptep_set_access_flags(struct vm_area_struct *vma, unsigned long addre extern int ptep_test_and_clear_young(struct vm_area_struct *vma, unsigned long address, pte_t *ptep); -#define __HAVE_ARCH_PTEP_GET_AND_CLEAR -static inline pte_t ptep_get_and_clear(struct mm_struct *mm, - unsigned long address, pte_t *ptep) +static inline pte_t __ptep_get_and_clear(struct mm_struct *mm, + unsigned long address, pte_t *ptep) { pte_t pte = __pte(atomic_long_xchg((atomic_long_t *)ptep, 0)); @@ -736,6 +735,9 @@ static inline pte_t ptep_get(pte_t *ptep) #define ptep_get __ptep_get #endif /* CONFIG_RISCV_ISA_SVNAPOT */ +#define __HAVE_ARCH_PTEP_GET_AND_CLEAR +#define ptep_get_and_clear __ptep_get_and_clear + #define pgprot_nx pgprot_nx static inline pgprot_t pgprot_nx(pgprot_t _prot) { diff --git a/arch/riscv/mm/hugetlbpage.c b/arch/riscv/mm/hugetlbpage.c index d51863824540..0ecb2846c3f0 100644 --- a/arch/riscv/mm/hugetlbpage.c +++ b/arch/riscv/mm/hugetlbpage.c @@ -173,68 +173,6 @@ pte_t arch_make_huge_pte(pte_t entry, unsigned int shift, vm_flags_t flags) return entry; } -static void clear_flush(struct mm_struct *mm, - unsigned long addr, - pte_t *ptep, - unsigned long pgsize, - unsigned long ncontig) -{ - struct vm_area_struct vma = TLB_FLUSH_VMA(mm, 0); - unsigned long i, saddr = addr; - - for (i = 0; i < ncontig; i++, addr += pgsize, ptep++) - ptep_get_and_clear(mm, addr, ptep); - - flush_tlb_range(&vma, saddr, addr); -} - -/* - * When dealing with NAPOT mappings, the privileged specification indicates that - * "if an update needs to be made, the OS generally should first mark all of the - * PTEs invalid, then issue SFENCE.VMA instruction(s) covering all 4 KiB regions - * within the range, [...] then update the PTE(s), as described in Section - * 4.2.1.". That's the equivalent of the Break-Before-Make approach used by - * arm64. - */ -void set_huge_pte_at(struct mm_struct *mm, - unsigned long addr, - pte_t *ptep, - pte_t pte, - unsigned long sz) -{ - unsigned long hugepage_shift, pgsize; - int i, pte_num; - - if (sz >= PGDIR_SIZE) - hugepage_shift = PGDIR_SHIFT; - else if (sz >= P4D_SIZE) - hugepage_shift = P4D_SHIFT; - else if (sz >= PUD_SIZE) - hugepage_shift = PUD_SHIFT; - else if (sz >= PMD_SIZE) - hugepage_shift = PMD_SHIFT; - else - hugepage_shift = PAGE_SHIFT; - - pte_num = sz >> hugepage_shift; - pgsize = 1 << hugepage_shift; - - if (!pte_present(pte)) { - for (i = 0; i < pte_num; i++, ptep++, addr += pgsize) - set_ptes(mm, addr, ptep, pte, 1); - return; - } - - if (!pte_napot(pte)) { - set_ptes(mm, addr, ptep, pte, 1); - return; - } - - clear_flush(mm, addr, ptep, pgsize, pte_num); - - set_ptes(mm, addr, ptep, pte, pte_num); -} - int huge_ptep_set_access_flags(struct vm_area_struct *vma, unsigned long addr, pte_t *ptep, diff --git a/include/linux/hugetlb_contpte.h b/include/linux/hugetlb_contpte.h index ec4189cd65b8..7acd734a75e8 100644 --- a/include/linux/hugetlb_contpte.h +++ b/include/linux/hugetlb_contpte.h @@ -9,4 +9,9 @@ #define __HAVE_ARCH_HUGE_PTEP_GET extern pte_t huge_ptep_get(struct mm_struct *mm, unsigned long addr, pte_t *ptep); +#define __HAVE_ARCH_HUGE_SET_HUGE_PTE_AT +extern void set_huge_pte_at(struct mm_struct *mm, + unsigned long addr, pte_t *ptep, pte_t pte, + unsigned long sz); + #endif /* _LINUX_HUGETLB_CONTPTE_H */ diff --git a/mm/hugetlb_contpte.c b/mm/hugetlb_contpte.c index a03e91d3efb1..677d714fd10d 100644 --- a/mm/hugetlb_contpte.c +++ b/mm/hugetlb_contpte.c @@ -10,6 +10,8 @@ /* * Any arch that wants to use that needs to define: * - __ptep_get() + * - __set_ptes() + * - __ptep_get_and_clear() * - pte_cont() * - arch_contpte_get_num_contig() */ @@ -17,6 +19,7 @@ /* * This file implements the following contpte aware API: * - huge_ptep_get() + * - set_huge_pte_at() */ pte_t huge_ptep_get(struct mm_struct *mm, unsigned long addr, pte_t *ptep) @@ -42,3 +45,59 @@ pte_t huge_ptep_get(struct mm_struct *mm, unsigned long addr, pte_t *ptep) } return orig_pte; } + +/* + * ARM64: Changing some bits of contiguous entries requires us to follow a + * Break-Before-Make approach, breaking the whole contiguous set + * before we can change any entries. See ARM DDI 0487A.k_iss10775, + * "Misprogramming of the Contiguous bit", page D4-1762. + * + * RISCV: When dealing with NAPOT mappings, the privileged specification + * indicates that "if an update needs to be made, the OS generally should first + * mark all of the PTEs invalid, then issue SFENCE.VMA instruction(s) covering + * all 4 KiB regions within the range, [...] then update the PTE(s), as + * described in Section 4.2.1.". That's the equivalent of the Break-Before-Make + * approach used by arm64. + * + * This helper performs the break step for use cases where the + * original pte is not needed. + */ +static void clear_flush(struct mm_struct *mm, + unsigned long addr, + pte_t *ptep, + unsigned long pgsize, + unsigned long ncontig) +{ + struct vm_area_struct vma = TLB_FLUSH_VMA(mm, 0); + unsigned long i, saddr = addr; + + for (i = 0; i < ncontig; i++, addr += pgsize, ptep++) + __ptep_get_and_clear(mm, addr, ptep); + + flush_tlb_range(&vma, saddr, addr); +} + +void set_huge_pte_at(struct mm_struct *mm, unsigned long addr, + pte_t *ptep, pte_t pte, unsigned long sz) +{ + size_t pgsize; + int i; + int ncontig; + + ncontig = arch_contpte_get_num_contig(ptep, sz, &pgsize); + + if (!pte_present(pte)) { + for (i = 0; i < ncontig; i++, ptep++, addr += pgsize) + __set_ptes(mm, addr, ptep, pte, 1); + return; + } + + if (!pte_cont(pte)) { + __set_ptes(mm, addr, ptep, pte, 1); + return; + } + + clear_flush(mm, addr, ptep, pgsize, ncontig); + + set_contptes(mm, addr, ptep, pte, ncontig, pgsize); +} From patchwork Mon Jan 27 09:35:26 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexandre Ghiti X-Patchwork-Id: 13951128 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8FEC7C0218C for ; Mon, 27 Jan 2025 09:40:52 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 2D70D28013B; Mon, 27 Jan 2025 04:40:52 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 286A628012F; Mon, 27 Jan 2025 04:40:52 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 14EEB28013B; Mon, 27 Jan 2025 04:40:52 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id ED50628012F for ; Mon, 27 Jan 2025 04:40:51 -0500 (EST) Received: from smtpin24.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 7C129820D9 for ; Mon, 27 Jan 2025 09:40:51 +0000 (UTC) X-FDA: 83052737502.24.343CDAE Received: from mail-wm1-f46.google.com (mail-wm1-f46.google.com [209.85.128.46]) by imf18.hostedemail.com (Postfix) with ESMTP id 79D0C1C0005 for ; Mon, 27 Jan 2025 09:40:49 +0000 (UTC) Authentication-Results: imf18.hostedemail.com; dkim=pass header.d=rivosinc-com.20230601.gappssmtp.com header.s=20230601 header.b=FdddVpqS; spf=pass (imf18.hostedemail.com: domain of alexghiti@rivosinc.com designates 209.85.128.46 as permitted sender) smtp.mailfrom=alexghiti@rivosinc.com; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1737970849; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=WRFnXJIF4/2q4SeWU0XB4ffUEopf6Q+wLHMkPdh3cGw=; b=yKMl6Gbx03+Ia2lHp9hvAs3dtfKPG6w2ESZtJzEdsu1vt3DKGJjyet54kqHTGIr16tqA0s tGr9dI79Q9pGSiSvhed3Q1w3o2FcoHalyRVYnWF5jQP3D+N4mR9OZGeD83wUvurnFyoohO Rgzzg3U1Ryf9DITl1+EYl4jauLuW+nM= ARC-Authentication-Results: i=1; imf18.hostedemail.com; dkim=pass header.d=rivosinc-com.20230601.gappssmtp.com header.s=20230601 header.b=FdddVpqS; spf=pass (imf18.hostedemail.com: domain of alexghiti@rivosinc.com designates 209.85.128.46 as permitted sender) smtp.mailfrom=alexghiti@rivosinc.com; dmarc=none ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1737970849; a=rsa-sha256; cv=none; b=RE+G3CcOv0LV9qrs5RT8YxS26LiilfntLgUsoT31VeDM3UZll5G9QfORclRYi3C6gSOnAf /TzWEwHWtxvVXpYX2YrZc9hrUq9oFTYGY5FFBzNoxvtOof9RUDNW/kYK3uz3OtPII+W+hr jxLhqScFkquUSs4aTZyuWbqaMMEOqng= Received: by mail-wm1-f46.google.com with SMTP id 5b1f17b1804b1-43634b570c1so29153615e9.0 for ; Mon, 27 Jan 2025 01:40:49 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc-com.20230601.gappssmtp.com; s=20230601; t=1737970848; x=1738575648; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=WRFnXJIF4/2q4SeWU0XB4ffUEopf6Q+wLHMkPdh3cGw=; b=FdddVpqSMjW0QAkn8DKmT3Uiw5+bC7JVULvPAU197Z+CuR3npGIg5i8T6RdS8kZ0pG ov7c/xa7zBaJKYskgNw4lhZlhxchzPSxOTDwZLJtnrbXhG6zFcQxb1E7/hE72VNJKasJ tTvT6uJD8bxrX6jhpdqxqn2q2c5zm0PeptwpzwqTXtNN/XMYyuBFPYmBh6q/L6JCUA8i 8ve0sJsN4prCHOcMBP2Yqt5w16ygJFkxDs3mepv1X75R5NSmjhnKAC0rIVWCUQ1jO8or dtBqUsaTnHkJ0eZP5D+qn70k3H+ISQO9Jz93CYA4cPRxYRh8qKPC3s6xuV9QUgDnG5ah QFQw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1737970848; x=1738575648; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=WRFnXJIF4/2q4SeWU0XB4ffUEopf6Q+wLHMkPdh3cGw=; b=AmxPxmmzj7ulqJnGBQk64NTe1mcOXche4JW0vzh7wlqd3CN3/4uivkOIVBBSdLeOfS IiiKAGN3Yns0Ec4qVf6XQEFJ3iB44XuWcgMvAam/s28f/DHWpOhF4pSNPmmuQ9YXiw/g 65uuqGRHyQyhLMY+eu5wacFlomINYz7aalI7ClvvFbCWypgAXwPIAms0D6tIkoVxPPAn npeuvfWpBfYCx1jicPFYU+H8u+soffZcxlDHu7vImV+zTEi9MT62Dqg8NKns35vNJn7I J9HfdY90yhjCipHi5tpLstc5HaD92owkmjADiooLBBE/Cde3/UlumfICnrSjceYqKzt8 lqvg== X-Forwarded-Encrypted: i=1; AJvYcCXWDDvjXicvwU+vLX4tT5nveUNkDFUlLMtwk3JGV3q8i0THqT5naXwh6L5rUCPqHHKQwnDVfEuMbg==@kvack.org X-Gm-Message-State: AOJu0YxU3YV0aDjev5yZBwLSaJ4p8Hr73rnv17OFtqM+MMU4p5Xm2RpJ 4w+rdoOXGxTyt8YJtFuwDESRST3/z/MB/xmqppL5l2aXCP9uq3Oqvo6WhXYBPjQ= X-Gm-Gg: ASbGnctys0TOIar5QSzTpGBxbhxFEDyjyeplBoiuKXTlYD9D4RSi/z4eggzzFS9MwtY +EIFkqeQiLelLgzb6hk38THeT+DhO/UMX32NzL4FSOxZ7dWG0UKulbSZdCp639B1Shz60QJDPZ+ exUbRJ+NffWNak02VJRkMERwvYeC3rDqauy+5rXC+eSOUxrHqfY5kWdqtOAMNk8z68P37NOMAbC xhzc2v8pj9zthinS2N1J8yuQnnLcZCUMle5QUu4r40XsN6KylMOFziHo1kdzu3RJoKmznvi/fKG AmOIDsToQnkxC/VeKQc1CwLvYHrODDQ= X-Google-Smtp-Source: AGHT+IHaxudRMH2a6X9Kau2FejbNNa+aNZH7OQ4Z4KpCCzwiphcNEJX34OXuuFlJfN/dPYRcVT3pPw== X-Received: by 2002:a05:600c:6c9a:b0:438:a240:c54 with SMTP id 5b1f17b1804b1-438a2400d95mr279659705e9.9.1737970847952; Mon, 27 Jan 2025 01:40:47 -0800 (PST) Received: from alex-rivos.ba.rivosinc.com ([2001:861:3382:ef90:3e22:3e78:ce5a:32c3]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-438bd57517fsm122917795e9.38.2025.01.27.01.40.46 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 27 Jan 2025 01:40:47 -0800 (PST) From: Alexandre Ghiti To: Catalin Marinas , Will Deacon , Ryan Roberts , Mark Rutland , Paul Walmsley , Palmer Dabbelt , Albert Ou , Andrew Morton , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org, linux-mm@kvack.org Cc: Alexandre Ghiti Subject: [PATCH v4 5/9] mm: Use common huge_pte_clear() function for riscv/arm64 Date: Mon, 27 Jan 2025 10:35:26 +0100 Message-Id: <20250127093530.19548-6-alexghiti@rivosinc.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20250127093530.19548-1-alexghiti@rivosinc.com> References: <20250127093530.19548-1-alexghiti@rivosinc.com> MIME-Version: 1.0 X-Rspamd-Queue-Id: 79D0C1C0005 X-Stat-Signature: 3ej9yhyq5r5wdc4juh5eq4qj4jyxcneu X-Rspamd-Server: rspam08 X-Rspam-User: X-HE-Tag: 1737970849-807954 X-HE-Meta: U2FsdGVkX19DgTgrFdA8QjOt0vTzmqBj3iainT57dqTEykeoT96Xf8gAJzT+dztHj5SC4xLQE+KY49PEFK24r0gkBrMCws8QNyPVstoANTAXPxW5ZqkcIdkvc8PTIUmRmxkEE4levHsgbTW7Z2rGEanrIo/fKx/jmnWAfvwNhPgAFOtYN64sipq8Yp16tvQ6A8d5XQVqshHrg4rkzl9ZOgs8KBJ3MLx0bakkJPXnfhW1g87RUejlmOLpaqEOZEPt0RUkCHTHFu98Ce5lsxSAE9ZiTjB2MYfFbsVwdU69qU1LiWuOTPJRFGGuQEYgERmLhCXNGinSrlZv1Mun+zFcPKkkRlwnZdIPRWNMgKNg6YrpNqT/SlN+8caKwGLcN0B9m76FIyN9RVeqCY2++bf1+ieN67HuTfHEGKJcDEecl22i012Q9+FYCCCOle3LN+jEaWxIJQREHlRAzDMrGOVXgFRxNgMLmXLmpKuNUu+aZqkSZrxeIvSofU+nePQZmkM+Tda3NGkrrx5E/No9lyGClWotXLVCpXUXVRUYcftUMVQI1M4KSmzxeonRX8SWAFFTo/kElXEmMcLDq7JtjcDtEEavmAdS6sPC0/uI+74db0oM9TP7q91p1dcoFD7eEdTICWzed/xuIMVvDI2lhlvs7Bl6M/JqGJAKyU2FDpqTG23Fij/Y4wkGYxxyU3pe0rGYyUlRXmlwPNQJRqe9m12jt1LdVjdtCMVi4KM8dss0ea4fcejc8Vd2dwUOYuCDokfc2ft7FTQB4KUzJ77p/9h39K9fjFcOJgEsXrv11POlinH2wUAEWPwMFthtG9+wBFjes3qc+VQtKo/KDSNjUMt5iyJfcEDKl+GPjTRUVAxsNCQ8K2qR78k2IktaeR1n0EZE4dx2hq9aZyT8fxSFteykB9oH386Jcz96Z9b4gIQaZOmY++qEG3HlbcZiGoG1pEqr9sGXjLT0Uy2TNpeqBPT 1zsPLomE 45F4xA+IdPCLCXDnrslClGxx8PTAhnVju0GlKkLkF/Oq6Mx0maLEhZx0WtavSNDmpw87sWFu29tdXKUzqehg3Kq38ZBQgB72eEo/rKrWXkgFwrADKqI3s/MQ3PwnCBybyLFuu5vqL6/AMMOjKl1uGZTWjkZ/tIEI4+Aqrr4+jwau004IG6zvX7XZEiHKEE5nRH5OStbGVp3zq53JEWgzgrz2ROjOjZm5CiTX+eDZiSqV9h02CW0Wxm5CcnhkjZ5IlsnqvODXHolEUuovD/cKgdfMwcja4zEvgE8q2/kvVW4B43xBZH7WV+krXgQuyHGX1yKW4MAb5hbFdycELc5eHEMDPqkfkDqHgQH0c77Rl12npkG+f2iMEQ3RazAGU5T9YyEWuDLQOOkLErRALP3X4a/3uync3HSomMUhc0CSdSB6PL//Y2t7Qh3t91Xg0fcIeddbCDWEQQpxnjjjyRPGS5svjlg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Both architectures have the same implementation so move it to generic code. Signed-off-by: Alexandre Ghiti --- arch/arm64/include/asm/hugetlb.h | 3 --- arch/arm64/mm/hugetlbpage.c | 12 ------------ arch/riscv/include/asm/hugetlb.h | 4 ---- arch/riscv/include/asm/pgtable.h | 5 +++-- arch/riscv/mm/hugetlbpage.c | 19 ------------------- include/linux/hugetlb_contpte.h | 4 ++++ mm/hugetlb_contpte.c | 14 ++++++++++++++ 7 files changed, 21 insertions(+), 40 deletions(-) diff --git a/arch/arm64/include/asm/hugetlb.h b/arch/arm64/include/asm/hugetlb.h index 40d87a563093..e4acaedea149 100644 --- a/arch/arm64/include/asm/hugetlb.h +++ b/arch/arm64/include/asm/hugetlb.h @@ -48,9 +48,6 @@ extern void huge_ptep_set_wrprotect(struct mm_struct *mm, #define __HAVE_ARCH_HUGE_PTEP_CLEAR_FLUSH extern pte_t huge_ptep_clear_flush(struct vm_area_struct *vma, unsigned long addr, pte_t *ptep); -#define __HAVE_ARCH_HUGE_PTE_CLEAR -extern void huge_pte_clear(struct mm_struct *mm, unsigned long addr, - pte_t *ptep, unsigned long sz); void __init arm64_hugetlb_cma_reserve(void); diff --git a/arch/arm64/mm/hugetlbpage.c b/arch/arm64/mm/hugetlbpage.c index 02de680a6a0d..541358f50b64 100644 --- a/arch/arm64/mm/hugetlbpage.c +++ b/arch/arm64/mm/hugetlbpage.c @@ -277,18 +277,6 @@ pte_t arch_make_huge_pte(pte_t entry, unsigned int shift, vm_flags_t flags) return entry; } -void huge_pte_clear(struct mm_struct *mm, unsigned long addr, - pte_t *ptep, unsigned long sz) -{ - int i, ncontig; - size_t pgsize; - - ncontig = arch_contpte_get_num_contig(ptep, sz, &pgsize); - - for (i = 0; i < ncontig; i++, addr += pgsize, ptep++) - __pte_clear(mm, addr, ptep); -} - pte_t huge_ptep_get_and_clear(struct mm_struct *mm, unsigned long addr, pte_t *ptep) { diff --git a/arch/riscv/include/asm/hugetlb.h b/arch/riscv/include/asm/hugetlb.h index 28cbf5d761e1..ca9930cdf2e6 100644 --- a/arch/riscv/include/asm/hugetlb.h +++ b/arch/riscv/include/asm/hugetlb.h @@ -20,10 +20,6 @@ bool arch_hugetlb_migration_supported(struct hstate *h); #endif #ifdef CONFIG_RISCV_ISA_SVNAPOT -#define __HAVE_ARCH_HUGE_PTE_CLEAR -void huge_pte_clear(struct mm_struct *mm, unsigned long addr, - pte_t *ptep, unsigned long sz); - #define __HAVE_ARCH_HUGE_PTEP_GET_AND_CLEAR pte_t huge_ptep_get_and_clear(struct mm_struct *mm, unsigned long addr, pte_t *ptep); diff --git a/arch/riscv/include/asm/pgtable.h b/arch/riscv/include/asm/pgtable.h index 74d29d0af172..08b24c0a579b 100644 --- a/arch/riscv/include/asm/pgtable.h +++ b/arch/riscv/include/asm/pgtable.h @@ -621,8 +621,8 @@ static inline int arch_contpte_get_num_contig(pte_t *ptep, unsigned long size, } #endif -static inline void pte_clear(struct mm_struct *mm, - unsigned long addr, pte_t *ptep) +static inline void __pte_clear(struct mm_struct *mm, + unsigned long addr, pte_t *ptep) { __set_pte_at(mm, ptep, __pte(0)); } @@ -737,6 +737,7 @@ static inline pte_t ptep_get(pte_t *ptep) #define __HAVE_ARCH_PTEP_GET_AND_CLEAR #define ptep_get_and_clear __ptep_get_and_clear +#define pte_clear __pte_clear #define pgprot_nx pgprot_nx static inline pgprot_t pgprot_nx(pgprot_t _prot) diff --git a/arch/riscv/mm/hugetlbpage.c b/arch/riscv/mm/hugetlbpage.c index 0ecb2846c3f0..e2093e7266a5 100644 --- a/arch/riscv/mm/hugetlbpage.c +++ b/arch/riscv/mm/hugetlbpage.c @@ -251,25 +251,6 @@ pte_t huge_ptep_clear_flush(struct vm_area_struct *vma, return get_clear_contig_flush(vma->vm_mm, addr, ptep, pte_num); } -void huge_pte_clear(struct mm_struct *mm, - unsigned long addr, - pte_t *ptep, - unsigned long sz) -{ - size_t pgsize; - pte_t pte = ptep_get(ptep); - int i, pte_num; - - if (!pte_napot(pte)) { - pte_clear(mm, addr, ptep); - return; - } - - pte_num = arch_contpte_get_num_contig(ptep, sz, &pgsize); - for (i = 0; i < pte_num; i++, addr += pgsize, ptep++) - pte_clear(mm, addr, ptep); -} - static bool is_napot_size(unsigned long size) { unsigned long order; diff --git a/include/linux/hugetlb_contpte.h b/include/linux/hugetlb_contpte.h index 7acd734a75e8..d9892a047b2b 100644 --- a/include/linux/hugetlb_contpte.h +++ b/include/linux/hugetlb_contpte.h @@ -14,4 +14,8 @@ extern void set_huge_pte_at(struct mm_struct *mm, unsigned long addr, pte_t *ptep, pte_t pte, unsigned long sz); +#define __HAVE_ARCH_HUGE_PTE_CLEAR +extern void huge_pte_clear(struct mm_struct *mm, unsigned long addr, + pte_t *ptep, unsigned long sz); + #endif /* _LINUX_HUGETLB_CONTPTE_H */ diff --git a/mm/hugetlb_contpte.c b/mm/hugetlb_contpte.c index 677d714fd10d..c76d6b3d0121 100644 --- a/mm/hugetlb_contpte.c +++ b/mm/hugetlb_contpte.c @@ -12,6 +12,7 @@ * - __ptep_get() * - __set_ptes() * - __ptep_get_and_clear() + * - __pte_clear() * - pte_cont() * - arch_contpte_get_num_contig() */ @@ -20,6 +21,7 @@ * This file implements the following contpte aware API: * - huge_ptep_get() * - set_huge_pte_at() + * - huge_pte_clear() */ pte_t huge_ptep_get(struct mm_struct *mm, unsigned long addr, pte_t *ptep) @@ -101,3 +103,15 @@ void set_huge_pte_at(struct mm_struct *mm, unsigned long addr, set_contptes(mm, addr, ptep, pte, ncontig, pgsize); } + +void huge_pte_clear(struct mm_struct *mm, unsigned long addr, + pte_t *ptep, unsigned long sz) +{ + int i, ncontig; + size_t pgsize; + + ncontig = arch_contpte_get_num_contig(ptep, sz, &pgsize); + + for (i = 0; i < ncontig; i++, addr += pgsize, ptep++) + __pte_clear(mm, addr, ptep); +} From patchwork Mon Jan 27 09:35:27 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexandre Ghiti X-Patchwork-Id: 13951129 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id DC24BC0218C for ; Mon, 27 Jan 2025 09:41:53 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 7444C28011C; Mon, 27 Jan 2025 04:41:53 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 7257C280115; Mon, 27 Jan 2025 04:41:53 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 5E3CA28011C; Mon, 27 Jan 2025 04:41:53 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 3FBE7280115 for ; Mon, 27 Jan 2025 04:41:53 -0500 (EST) Received: from smtpin02.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id C0BD8122181 for ; Mon, 27 Jan 2025 09:41:52 +0000 (UTC) X-FDA: 83052740064.02.909F559 Received: from mail-wr1-f51.google.com (mail-wr1-f51.google.com [209.85.221.51]) by imf30.hostedemail.com (Postfix) with ESMTP id CA23D80008 for ; Mon, 27 Jan 2025 09:41:50 +0000 (UTC) Authentication-Results: imf30.hostedemail.com; dkim=pass header.d=rivosinc-com.20230601.gappssmtp.com header.s=20230601 header.b=zez3wvdC; spf=pass (imf30.hostedemail.com: domain of alexghiti@rivosinc.com designates 209.85.221.51 as permitted sender) smtp.mailfrom=alexghiti@rivosinc.com; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1737970910; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=/Ib7BO4DbRcQ5HoSRv1ee1R7qySu3xzgQBO2hzqpTQY=; b=fHtrFDecOi9tJRynWLX1m5Ajit1g9b1wnvA4hbjZAdKuBMQ3u7YBwCbCtFfW1ix51VpUmU QRIxy49/9Lx8+UOSXrmxAhjLrWUC9VQQjiYr3JHVsUgCxvqeaHMUvUGA6e81BsSaHziMHv 6urVqfQmi6XJAYPm6WVTKtGgkS7X41A= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1737970910; a=rsa-sha256; cv=none; b=LxWbffQgjdj9OmMf9skeHLYU+88HYvDS+bo2VQAqU72D6Pb4mga1+RxlAmPPXuRq/yOFp3 xDXMuy6QCHj8Qg9VmLNQ5MNQraSatq4XsXOlExyi/TuRmlwqqcm3MkdoQ1rURBAZ2OyWlo KyMV1JTWjSdcJGEVMZcvTDZ1/KZcCro= ARC-Authentication-Results: i=1; imf30.hostedemail.com; dkim=pass header.d=rivosinc-com.20230601.gappssmtp.com header.s=20230601 header.b=zez3wvdC; spf=pass (imf30.hostedemail.com: domain of alexghiti@rivosinc.com designates 209.85.221.51 as permitted sender) smtp.mailfrom=alexghiti@rivosinc.com; dmarc=none Received: by mail-wr1-f51.google.com with SMTP id ffacd0b85a97d-38633b5dbcfso4694927f8f.2 for ; Mon, 27 Jan 2025 01:41:50 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc-com.20230601.gappssmtp.com; s=20230601; t=1737970909; x=1738575709; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=/Ib7BO4DbRcQ5HoSRv1ee1R7qySu3xzgQBO2hzqpTQY=; b=zez3wvdC6ekjC0gTE6GHzYHtogmjFK05XIPG7gOtG/qNSjdd6QdjH6q4XO4nHtjXF0 kFuV1VqnJXJf7QPiqZRXj0Daq5tVOE/MfRKNotAsxeO9l7QhOF9kCB+Yf/E0zEe0/i3n DMKd9Eq3ZI8/idiPIAwPAOexNOffrGtc2QzdhIJ+U8tK/WVAUJltDraI0X5piZsjQsPD Y7GM6EsWfDVtZbjons4Ccicsju0GifZK7omJnWKh9WwjCiw+v/mJ64Vb/wO0kKEWOo+4 6gb0fUJkeciaPRzXbtYP2gXNnfsk+RIJ+VHB/wzCiv3aAlgRzgfQsDsX6Ff8s8/lDStJ S+Dg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1737970909; x=1738575709; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=/Ib7BO4DbRcQ5HoSRv1ee1R7qySu3xzgQBO2hzqpTQY=; b=ORZMXwR76FoBQlNwC2tUAalOyTvaRb9Lw5NZvSscQmroO/uUm7+xhpOX6ww6X8qdXE v0X+WhX7bAfDK840TwHSk5PdlvQlO+uiOfzfeUEofMf1sBCGwvqmbOxB5g2s8hEsErj7 nLxXmhAmEtN0+zDDuWhuWBAJ32TG2s7GABx7y4+4zYFGLOFADVJ2TeblwK/ZtBiJFT2G 02cU4rF+jHQITExcOtaLRI27YM/1w5YlL9wi4qpKG2PuvRehloUUZfZxteobvcYOAxx6 8TGGbJFRwFT8ra7hT+pbDcwCULeFjX3arqLkBM3tY1+EdDXJwIdLiLq3sqwCAretjsyM KH/A== X-Forwarded-Encrypted: i=1; AJvYcCXfIZWdlZAeMbQ7p8A7mCqjUT4euvMqOMG/zHAUxjFHwCNfIK7B8opuB5JXHbvaypty/CgvpYa2Bg==@kvack.org X-Gm-Message-State: AOJu0YziKmz47Kg+oWlSx1tYxlNBvx7ri8I9VwGhrGeOlFBnARiW1Lue GDiUsuqfreR9dzWmkdX5MdvAVEQiYb/lqwvLLJY6ams4AlHDYwiTx9sMmNSNXwc= X-Gm-Gg: ASbGncv2ZIxzd4BxfLfRTxR3cZs6cpxfDvGbmKjmaTCR5WG35XloxMNbl42Go3Y/7oN /bvhLGliCnTVHtExgf374HxD9LQnlEQh51enG5U3uSn0SkKA1DvORiG7lnXS9lIOz5t/R2DBKRJ 8gKXSzDFgYjeEbJl8HgtX31Ekbp/RhzsMiF3CXOL18e3ij0rbPgvm0UsSKssbgV9bo+Z/AJqQOY 5JSwVtsAkACGCmQf26elIZjWsB21HqBMh1WtdcCo1sND4XnVsSnNxf4Bg8xXy7gPm/LVIxRJ6ni OGx+0ZOUjzxhtm8BUPZiN3M3t2YS5ps= X-Google-Smtp-Source: AGHT+IE0wvAkHX9Lz2nK7eyxGx0auM1eq+y4AS6IYBZCccECBQROh0UBY1MLjeLntJwSdARX6LIQ6w== X-Received: by 2002:a5d:64a3:0:b0:385:f909:eb2c with SMTP id ffacd0b85a97d-38bf57a77a2mr48324115f8f.38.1737970909255; Mon, 27 Jan 2025 01:41:49 -0800 (PST) Received: from alex-rivos.ba.rivosinc.com ([2001:861:3382:ef90:3e22:3e78:ce5a:32c3]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-38c2a1c42fcsm10707051f8f.96.2025.01.27.01.41.48 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 27 Jan 2025 01:41:49 -0800 (PST) From: Alexandre Ghiti To: Catalin Marinas , Will Deacon , Ryan Roberts , Mark Rutland , Paul Walmsley , Palmer Dabbelt , Albert Ou , Andrew Morton , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org, linux-mm@kvack.org Cc: Alexandre Ghiti Subject: [PATCH v4 6/9] mm: Use common huge_ptep_get_and_clear() function for riscv/arm64 Date: Mon, 27 Jan 2025 10:35:27 +0100 Message-Id: <20250127093530.19548-7-alexghiti@rivosinc.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20250127093530.19548-1-alexghiti@rivosinc.com> References: <20250127093530.19548-1-alexghiti@rivosinc.com> MIME-Version: 1.0 X-Stat-Signature: gb7bn13jw6bwy6hd3nqwuxybsg46uymf X-Rspamd-Queue-Id: CA23D80008 X-Rspam-User: X-Rspamd-Server: rspam06 X-HE-Tag: 1737970910-922952 X-HE-Meta: U2FsdGVkX19jiGxkO8m6bNbl13ecBpHOOh9wT7x5uND9nzy8tibdMw2hK8HkJy4uXqHJiHAgJTELT6msP2wWnEmX95rwrFq8Km/ppBJYC8Dh10xVTWng3Np0H9HTNozYvOsfGO2vrIvHmKHdRgIBV56goPvPpG4zW3frKfNIFpWCyWok4K9u3MBxZQXtjXuRh2RFCGNSxO5MIIkrZbLLvAieuIzbvScnCNpwOp5UGNI59ZncOolaVjaiY21ppOztegNoyKM2VPpXF3aMX8ydSrvEac5NeRAkGIGF2wNvv/KZJbF+IyrzfVhf05/qs2aRnMBX+D010uwey6lB2WkMHdzzUgiZ3VprJ+RQGarVvSGcXbIExanUyvTpknwGb+EIk0wcQfHyugh+Kf+ZjVQJ/nmU3GtJAi1LzLprBg5MvWTGPi/NsvxSYBEz35ByrqXwnHvu2+S/Wwa9HPFJc2jz1CXjkKuF02P+Nt6cTm9j5P8w6a4CndXVUsJHqQYJ1uBcsrWH06pb/IiQG7Ex1KAzZNjRVJbZiami9/bw64tAHT6/UUZ4Ok7b8q714pUATzOKgH4izN61gRCcRP11B+M2wWeG8RIu6SlsOT4QfzcAxAeFYs4oHBPQpEqK7C2hfG2vrwPpoC/WlCOW75oVv+Db5BMMdXNW2RAtZRttOfQZ17/Tmgz72TZrYtbDKqinOr8MUnPjyIx4BbkXFmb0W/TKpC78udgbYQjp/vVJrCI7PPoeQ8dtvaHw5xRRI2LQnafuARINZTtaCsE27b8Gf+GCBxiwI1OATxUOSmZodOd8ooB+fLYERA21l+XCfNb6/aPze+vlVplosYM959Y27Hw52e2l5uvAuOQSHni/Y5zfZZwttz4HOq3ZBdA2IpJZAAHY8m12G7JPnaHKvn3Y+SU7UZh9imgjcQjNzTbxus5Q7GUa2OoYcx/BTYPKJrGNpOcXRH7ODGW6q78CCxEOFd3 NZ3cw/Ww p2JD7TDBa5z5Hdpw00YAkg8isQH9ivb/16RvljhLTejoF/bOesssAhZiESDtX1zAI8IOeUsQb1gl4CVgYjEpvYEQ4GW5iqmwwXvDNGMbrSjVPEVn6S13Ln7XpOZ51MtIOVff3GDhMXmC7yAYpXkUOXynC9J0w12Pgm1dpUmAiDywVnDKh+ZoxXHc/s+tLRBa+Iq+u+S1y0xOuSq5TfTf2VLD4axn/uGRYpdcd5833PHL85+/VDu+ung2zhOAso3In+Iw902rHxHz2wu9y/Z6Dq6ZEYCUB44glAgGwfo/LmmPULlRw4U+5o8ROcab2xuPThRnqzB9KD6HULNOfLoNxJjujO3UIsM1ZH62hM99Tv7/ub+aOVcSmQUGsgszPHzQb1Z+Ws2OqZps3WGlPftYm3clGOs8PsapxmM7WqMIP0F029076/c07qM3SlkHTEh6EvMReMkH5gQqvID6n8bpeksGElg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: After some adjustments, both architectures have the same implementation so move it to the generic code. Note that get_clear_contig() function is duplicated in the generic and the arm64 code because it is still used by some arm64 functions that will, in the next commits, be moved to the generic code. Once all have been moved, the arm64 version will be removed. Signed-off-by: Alexandre Ghiti --- arch/arm64/include/asm/hugetlb.h | 3 -- arch/arm64/include/asm/pgtable.h | 15 ++++++++-- arch/arm64/mm/hugetlbpage.c | 19 ++----------- arch/riscv/include/asm/hugetlb.h | 4 --- arch/riscv/include/asm/pgtable.h | 4 ++- arch/riscv/mm/hugetlbpage.c | 23 ++++----------- include/linux/hugetlb_contpte.h | 4 +++ mm/hugetlb_contpte.c | 48 ++++++++++++++++++++++++++++++-- 8 files changed, 72 insertions(+), 48 deletions(-) diff --git a/arch/arm64/include/asm/hugetlb.h b/arch/arm64/include/asm/hugetlb.h index e4acaedea149..5c605a0a2017 100644 --- a/arch/arm64/include/asm/hugetlb.h +++ b/arch/arm64/include/asm/hugetlb.h @@ -39,9 +39,6 @@ pte_t arch_make_huge_pte(pte_t entry, unsigned int shift, vm_flags_t flags); extern int huge_ptep_set_access_flags(struct vm_area_struct *vma, unsigned long addr, pte_t *ptep, pte_t pte, int dirty); -#define __HAVE_ARCH_HUGE_PTEP_GET_AND_CLEAR -extern pte_t huge_ptep_get_and_clear(struct mm_struct *mm, - unsigned long addr, pte_t *ptep); #define __HAVE_ARCH_HUGE_PTEP_SET_WRPROTECT extern void huge_ptep_set_wrprotect(struct mm_struct *mm, unsigned long addr, pte_t *ptep); diff --git a/arch/arm64/include/asm/pgtable.h b/arch/arm64/include/asm/pgtable.h index cebbfcfb0e53..c339b568ac51 100644 --- a/arch/arm64/include/asm/pgtable.h +++ b/arch/arm64/include/asm/pgtable.h @@ -1831,12 +1831,23 @@ static inline void clear_young_dirty_ptes(struct vm_area_struct *vma, #endif /* CONFIG_ARM64_CONTPTE */ -static inline int arch_contpte_get_num_contig(pte_t *ptep, - unsigned long size, +extern int find_num_contig(struct mm_struct *mm, unsigned long addr, + pte_t *ptep, size_t *pgsize); + +static inline int arch_contpte_get_num_contig(struct mm_struct *mm, + unsigned long addr, + pte_t *ptep, unsigned long size, size_t *pgsize) { int contig_ptes = 0; + /* + * If the size is not passed, we need to go through the page table to + * find out the number of contiguous ptes. + */ + if (size == 0) + return find_num_contig(mm, addr, ptep, pgsize); + if (pgsize) *pgsize = size; diff --git a/arch/arm64/mm/hugetlbpage.c b/arch/arm64/mm/hugetlbpage.c index 541358f50b64..0b7a53fee55d 100644 --- a/arch/arm64/mm/hugetlbpage.c +++ b/arch/arm64/mm/hugetlbpage.c @@ -79,8 +79,8 @@ bool arch_hugetlb_migration_supported(struct hstate *h) } #endif -static int find_num_contig(struct mm_struct *mm, unsigned long addr, - pte_t *ptep, size_t *pgsize) +int find_num_contig(struct mm_struct *mm, unsigned long addr, + pte_t *ptep, size_t *pgsize) { pgd_t *pgdp = pgd_offset(mm, addr); p4d_t *p4dp; @@ -277,21 +277,6 @@ pte_t arch_make_huge_pte(pte_t entry, unsigned int shift, vm_flags_t flags) return entry; } -pte_t huge_ptep_get_and_clear(struct mm_struct *mm, - unsigned long addr, pte_t *ptep) -{ - int ncontig; - size_t pgsize; - pte_t orig_pte = __ptep_get(ptep); - - if (!pte_cont(orig_pte)) - return __ptep_get_and_clear(mm, addr, ptep); - - ncontig = find_num_contig(mm, addr, ptep, &pgsize); - - return get_clear_contig(mm, addr, ptep, pgsize, ncontig); -} - /* * huge_ptep_set_access_flags will update access flags (dirty, accesssed) * and write permission. diff --git a/arch/riscv/include/asm/hugetlb.h b/arch/riscv/include/asm/hugetlb.h index ca9930cdf2e6..0fbb6b19df79 100644 --- a/arch/riscv/include/asm/hugetlb.h +++ b/arch/riscv/include/asm/hugetlb.h @@ -20,10 +20,6 @@ bool arch_hugetlb_migration_supported(struct hstate *h); #endif #ifdef CONFIG_RISCV_ISA_SVNAPOT -#define __HAVE_ARCH_HUGE_PTEP_GET_AND_CLEAR -pte_t huge_ptep_get_and_clear(struct mm_struct *mm, - unsigned long addr, pte_t *ptep); - #define __HAVE_ARCH_HUGE_PTEP_CLEAR_FLUSH pte_t huge_ptep_clear_flush(struct vm_area_struct *vma, unsigned long addr, pte_t *ptep); diff --git a/arch/riscv/include/asm/pgtable.h b/arch/riscv/include/asm/pgtable.h index 08b24c0a579b..705d666e014d 100644 --- a/arch/riscv/include/asm/pgtable.h +++ b/arch/riscv/include/asm/pgtable.h @@ -590,7 +590,9 @@ static inline void __set_ptes(struct mm_struct *mm, unsigned long addr, } #ifdef CONFIG_RISCV_ISA_SVNAPOT -static inline int arch_contpte_get_num_contig(pte_t *ptep, unsigned long size, +static inline int arch_contpte_get_num_contig(struct mm_struct *mm, + unsigned long addr, + pte_t *ptep, unsigned long size, size_t *pgsize) { unsigned long hugepage_shift; diff --git a/arch/riscv/mm/hugetlbpage.c b/arch/riscv/mm/hugetlbpage.c index e2093e7266a5..b44023336fd9 100644 --- a/arch/riscv/mm/hugetlbpage.c +++ b/arch/riscv/mm/hugetlbpage.c @@ -186,7 +186,8 @@ int huge_ptep_set_access_flags(struct vm_area_struct *vma, if (!pte_napot(pte)) return ptep_set_access_flags(vma, addr, ptep, pte, dirty); - pte_num = arch_contpte_get_num_contig(ptep, 0, NULL); + pte_num = arch_contpte_get_num_contig(vma->vm_mm, addr, ptep, 0, NULL); + orig_pte = get_clear_contig_flush(mm, addr, ptep, pte_num); if (pte_dirty(orig_pte)) @@ -200,21 +201,6 @@ int huge_ptep_set_access_flags(struct vm_area_struct *vma, return true; } -pte_t huge_ptep_get_and_clear(struct mm_struct *mm, - unsigned long addr, - pte_t *ptep) -{ - pte_t orig_pte = ptep_get(ptep); - int pte_num; - - if (!pte_napot(orig_pte)) - return ptep_get_and_clear(mm, addr, ptep); - - pte_num = arch_contpte_get_num_contig(ptep, 0, NULL); - - return get_clear_contig(mm, addr, ptep, pte_num); -} - void huge_ptep_set_wrprotect(struct mm_struct *mm, unsigned long addr, pte_t *ptep) @@ -228,7 +214,8 @@ void huge_ptep_set_wrprotect(struct mm_struct *mm, return; } - pte_num = arch_contpte_get_num_contig(ptep, 0, NULL); + pte_num = arch_contpte_get_num_contig(mm, addr, ptep, 0, NULL); + orig_pte = get_clear_contig_flush(mm, addr, ptep, pte_num); orig_pte = pte_wrprotect(orig_pte); @@ -246,7 +233,7 @@ pte_t huge_ptep_clear_flush(struct vm_area_struct *vma, if (!pte_napot(pte)) return ptep_clear_flush(vma, addr, ptep); - pte_num = arch_contpte_get_num_contig(ptep, 0, NULL); + pte_num = arch_contpte_get_num_contig(vma->vm_mm, addr, ptep, 0, NULL); return get_clear_contig_flush(vma->vm_mm, addr, ptep, pte_num); } diff --git a/include/linux/hugetlb_contpte.h b/include/linux/hugetlb_contpte.h index d9892a047b2b..20d3a3e14e14 100644 --- a/include/linux/hugetlb_contpte.h +++ b/include/linux/hugetlb_contpte.h @@ -18,4 +18,8 @@ extern void set_huge_pte_at(struct mm_struct *mm, extern void huge_pte_clear(struct mm_struct *mm, unsigned long addr, pte_t *ptep, unsigned long sz); +#define __HAVE_ARCH_HUGE_PTEP_GET_AND_CLEAR +extern pte_t huge_ptep_get_and_clear(struct mm_struct *mm, + unsigned long addr, pte_t *ptep); + #endif /* _LINUX_HUGETLB_CONTPTE_H */ diff --git a/mm/hugetlb_contpte.c b/mm/hugetlb_contpte.c index c76d6b3d0121..0c86c6f77c29 100644 --- a/mm/hugetlb_contpte.c +++ b/mm/hugetlb_contpte.c @@ -22,6 +22,7 @@ * - huge_ptep_get() * - set_huge_pte_at() * - huge_pte_clear() + * - huge_ptep_get_and_clear() */ pte_t huge_ptep_get(struct mm_struct *mm, unsigned long addr, pte_t *ptep) @@ -32,7 +33,7 @@ pte_t huge_ptep_get(struct mm_struct *mm, unsigned long addr, pte_t *ptep) if (!pte_present(orig_pte) || !pte_cont(orig_pte)) return orig_pte; - ncontig = arch_contpte_get_num_contig(ptep, + ncontig = arch_contpte_get_num_contig(mm, addr, ptep, page_size(pte_page(orig_pte)), NULL); @@ -86,7 +87,7 @@ void set_huge_pte_at(struct mm_struct *mm, unsigned long addr, int i; int ncontig; - ncontig = arch_contpte_get_num_contig(ptep, sz, &pgsize); + ncontig = arch_contpte_get_num_contig(mm, addr, ptep, sz, &pgsize); if (!pte_present(pte)) { for (i = 0; i < ncontig; i++, ptep++, addr += pgsize) @@ -110,8 +111,49 @@ void huge_pte_clear(struct mm_struct *mm, unsigned long addr, int i, ncontig; size_t pgsize; - ncontig = arch_contpte_get_num_contig(ptep, sz, &pgsize); + ncontig = arch_contpte_get_num_contig(mm, addr, ptep, sz, &pgsize); for (i = 0; i < ncontig; i++, addr += pgsize, ptep++) __pte_clear(mm, addr, ptep); } + +static pte_t get_clear_contig(struct mm_struct *mm, + unsigned long addr, + pte_t *ptep, + unsigned long pgsize, + unsigned long ncontig) +{ + pte_t orig_pte = __ptep_get(ptep); + unsigned long i; + + for (i = 0; i < ncontig; i++, addr += pgsize, ptep++) { + pte_t pte = __ptep_get_and_clear(mm, addr, ptep); + + /* + * If HW_AFDBM (arm64) or Svadu (riscv) is enabled, then the HW + * could turn on the dirty or accessed bit for any page in the + * set, so check them all. + */ + if (pte_dirty(pte)) + orig_pte = pte_mkdirty(orig_pte); + + if (pte_young(pte)) + orig_pte = pte_mkyoung(orig_pte); + } + return orig_pte; +} + +pte_t huge_ptep_get_and_clear(struct mm_struct *mm, + unsigned long addr, pte_t *ptep) +{ + int ncontig; + size_t pgsize; + pte_t orig_pte = __ptep_get(ptep); + + if (!pte_cont(orig_pte)) + return __ptep_get_and_clear(mm, addr, ptep); + + ncontig = arch_contpte_get_num_contig(mm, addr, ptep, 0, &pgsize); + + return get_clear_contig(mm, addr, ptep, pgsize, ncontig); +} From patchwork Mon Jan 27 09:35:28 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexandre Ghiti X-Patchwork-Id: 13951130 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 66CC6C0218C for ; Mon, 27 Jan 2025 09:42:55 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id F3497280140; Mon, 27 Jan 2025 04:42:54 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id EBE1628013F; Mon, 27 Jan 2025 04:42:54 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D37F9280140; Mon, 27 Jan 2025 04:42:54 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id B5CD328013F for ; Mon, 27 Jan 2025 04:42:54 -0500 (EST) Received: from smtpin01.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 6401BA20C6 for ; Mon, 27 Jan 2025 09:42:54 +0000 (UTC) X-FDA: 83052742668.01.9CD8620 Received: from mail-wr1-f51.google.com (mail-wr1-f51.google.com [209.85.221.51]) by imf22.hostedemail.com (Postfix) with ESMTP id 67417C0003 for ; Mon, 27 Jan 2025 09:42:52 +0000 (UTC) Authentication-Results: imf22.hostedemail.com; dkim=pass header.d=rivosinc-com.20230601.gappssmtp.com header.s=20230601 header.b=bikNPKUH; dmarc=none; spf=pass (imf22.hostedemail.com: domain of alexghiti@rivosinc.com designates 209.85.221.51 as permitted sender) smtp.mailfrom=alexghiti@rivosinc.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1737970972; a=rsa-sha256; cv=none; b=Yg+mHP7DCguI+viSxfBDITJuzo2K75ItrRafPEFs9zxhS6O08jjpEdpgHfBTquNtdFmSEI mB+K0tQD5q0fRCvzsGnAvfYrC/Ot+BgVX2MtU0a7EafUw1YvRUia7zViP/FAklqIjCrGK4 X8gDyzwhCyraF3B3oSNZsNaAr8fXhiM= ARC-Authentication-Results: i=1; imf22.hostedemail.com; dkim=pass header.d=rivosinc-com.20230601.gappssmtp.com header.s=20230601 header.b=bikNPKUH; dmarc=none; spf=pass (imf22.hostedemail.com: domain of alexghiti@rivosinc.com designates 209.85.221.51 as permitted sender) smtp.mailfrom=alexghiti@rivosinc.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1737970972; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=GkmyUQAgkdeDfCGOVLuC8IWvmfEC8o5jHyW1gtJjCzs=; b=y1nrTxVIBzfymLsx4xQdXRgyHlRR/R/1g+nMwCVVI8dM8oJN7jaLqiedrmVoAiii3cKh77 UzJtsDppzXUcdEkpMZukM+CNsXVy4Yx7NjjNXx30uAGgQs61HMixeH2W3KdOYkLl74x4YX tPlelePdvdcxOW87RZWa5Fhda5lZOJQ= Received: by mail-wr1-f51.google.com with SMTP id ffacd0b85a97d-3862a921123so2762325f8f.3 for ; Mon, 27 Jan 2025 01:42:52 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc-com.20230601.gappssmtp.com; s=20230601; t=1737970971; x=1738575771; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=GkmyUQAgkdeDfCGOVLuC8IWvmfEC8o5jHyW1gtJjCzs=; b=bikNPKUHLaPsLZk8YpNSUappgX1/UJmIwmWKuqcFz062dZ3gm1cCKgHyWMg5a3htI7 HzVuehQJn17eaYUOLTv4wmOGNIwrOJg/JiKCyVVBKrPDlUtQkG5+C9eLU0jpeILV3YWL 89L9ORDfyTfeD8fShCb1ky8gePf9FlZZxmdXR3XDmW3ucKv4CcAK2v6PCwjNdUcMcB5Z PZhJ8cVqdc1ZtewtpI8wvpmhS2YrNHMC9DsbjVqyIJoNyJ7Mlz3zC4PRNQX3lOBkw91w sRhzodwY/MJ8SEJVmlXrgOsb9sEAJvoNY6xHTti/71useAQfbEL25GacY1s7lmpNIdIg 4vlA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1737970971; x=1738575771; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=GkmyUQAgkdeDfCGOVLuC8IWvmfEC8o5jHyW1gtJjCzs=; b=qEHmJWus0dKLrnDMPP7ER+G05TYS+VsVdHpoxLkewJLPRRtLQdKCTN/L6Lwk08PQJ0 STu38o/GLXRzkEm24zUOC3IkoAHFaB0PlqaHfZjsKLGXG4ggLlsZ/SYwAI8jmCU8H1Cp Frb7eB8B8/uxsHiHhibFuI8PKx9z9YSPp7wjDZjDPtZJi5Yw6wPceQoVgpFNGIx4LckF LEwPcZsCdNFoJqrTCownQIXeMZnr5Wpv3eQErg9Q3vbuVdBDdjpja7WE96tJVTyGw6FL pHnSCzkjZ1W0Sn16LVowoWg72u21o246RSqCViwnsqpCz+MkOStZ9tVT7qE5PJsXZhNy YWdg== X-Forwarded-Encrypted: i=1; AJvYcCU6h/qmFXKnEDt/1Uo1zfNCiCEXFABxDF4JiUa8lCKCDkOrh734fudrjP69nDOg/ptZ35AKloCUMQ==@kvack.org X-Gm-Message-State: AOJu0Yy12OeuzOClBI/e4HA7sEhEO041lSjc5AeKMwB/rbyGJ2KMzesq +FYzcdBHYiwCkgvEB9HrR8vZbsE+UaWlMbbRuX8E6o3P4ct4Vq5252u+MgFjO/k= X-Gm-Gg: ASbGnctzD7L9FfRicBngBZt0UQmBE0Q/Mjd3QonUghGEiIhmr2MZX2icxu1V+Fq3bW4 Dfw4idopxIUZg+B27+OM7vPPYYBqpaQFeBBNCn92+sWwCv+6YFiBP84EO1dAaojC3eCHGWNSkZY bIbv2bdFmPcURsBNy4YNpkwX5S53X6a8ayFLvMdrcYyberQutRY/AZx2yxUOkO/DvPY2VypPfZV NxrOctx/TeMx85ByTHDkEpXzzN/vR8ui0r3sBNb27+B8mmGxi0p5AsAJ8kDw2poFnq+PT0f2F96 pJCAr1KY19n7+7rrZCSy55QwoXocbGM= X-Google-Smtp-Source: AGHT+IFkGD+876PDgJO8tAmQAHU0ol+3F5plCgX8+RfXXi3SGv5kqaXmgh3Sp+SaS77G8Nc/FNbA7g== X-Received: by 2002:a05:6000:1ac6:b0:385:f07b:93da with SMTP id ffacd0b85a97d-38bf57bf69bmr33979049f8f.47.1737970970947; Mon, 27 Jan 2025 01:42:50 -0800 (PST) Received: from alex-rivos.ba.rivosinc.com ([2001:861:3382:ef90:3e22:3e78:ce5a:32c3]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-38c2a17d6edsm10380540f8f.40.2025.01.27.01.42.49 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 27 Jan 2025 01:42:50 -0800 (PST) From: Alexandre Ghiti To: Catalin Marinas , Will Deacon , Ryan Roberts , Mark Rutland , Paul Walmsley , Palmer Dabbelt , Albert Ou , Andrew Morton , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org, linux-mm@kvack.org Cc: Alexandre Ghiti Subject: [PATCH v4 7/9] mm: Use common huge_ptep_set_access_flags() function for riscv/arm64 Date: Mon, 27 Jan 2025 10:35:28 +0100 Message-Id: <20250127093530.19548-8-alexghiti@rivosinc.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20250127093530.19548-1-alexghiti@rivosinc.com> References: <20250127093530.19548-1-alexghiti@rivosinc.com> MIME-Version: 1.0 X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 67417C0003 X-Stat-Signature: cfaakn4h4fwh9e5qxz65qntwhnnamat9 X-Rspam-User: X-HE-Tag: 1737970972-65377 X-HE-Meta: U2FsdGVkX190+855t4uls+nocLIvxnZPXbSeBX/u0mQfjFAW1/V/O32InJRVuayCNdsHUnAup8igAR4cl8gWtZOp/bqxv2RXacgviu/5zp9BnRWqUedYaNVKpwRuomOFmClhidkwVepLYvDuPEx3uYpBsYlOfzswQkNjUJYH+MZcz8HW0WpDs0oPP1VhXpHWXScJvyZEOfJibGCIb2Vxu5h0kf6SHcBDIi54MZK3PCFRZqO96rqsLt5uoTlUSbl0zjLI3cay+sdBdUx0S45ShLSrLAnpfBt9RmWPKaxb0TNvMCLH5CXX8vcp06eV9wAmA06Up73wxKh+UW0mBNhsbLS5lHyaQnYSkZcl+N15YIgOYePt6aZO7OCnG6g/FwJceZTrmbOQ2xdliVmi3y1gUOJ0YfUP8hvqCKzPUMHqoq/K3EWYgWYZLbZ0EqHvknysFtCPaz7zJWnaFoZwJBG7m1/VLEyqHf370sj6bK2HEb154omFRky1LKthtkZkLE6bW4YvCGvcEgxBHH4wWLrF6WlAbS3UAcWH90D2VMlC37P1xa6UoI55RnN8AaJFChU/vnX3FTCDEU1jcceP/uTkBZUO6CQyorOn1dS+kds+jawEm7ijy4qEhMlALAIUqnwR2zSSwe1wPQ9kAM4MRb2H3msREe7ppw9fAtSA9Vs9+CdKPHbIFOYw3K+vqjattjOMwKcucmcDUEnxQAApGVmo4tKoH+9oAV8gZRFHizr7XKHaNhNggOJF46LeNaBnUf5tqwz38J2FK0TWdeMYno/B5kBBbIC6h2+DqTJmnhOOuTyUE0umTPIKLDERchBlCpZ8/sP3ThTJt9Ft+QR8MSWOVKiQnaPVsNNDbCrNKTriUwSp8aACTcBL2R6KPq1sXwDYQ5AQDWco8ppXI0GZsFZVI6a0kehCdb/6DzQvK6Vyy1hyK3kS2/FG+PKwaIm9I4hLeS8liiwKd/eAyhQJx7a YtBR+pEg X7eeneGCAD8oNzjZfe1Zx0DZG5hOdI8Pf7Ba+LOJy9XVXwPIyg7YkdTieYErQcLKXkr6fZMehLsMIBGZQjhc6x39G10H/W6sfb32X81rBQAcyZ+aH3ubhJIYaE0wNHmR9eedMuW3v7XkIsoxYcCpjvr+BuNgHT8yG+4TTOnZmkXZ74BlobozUGUmXmJYx2pChirec+Vn0GlMVsqTriu7oa7OH2vLwETWJjykeq/jsC9YTKz3kp81wMe9LuPNlcBBpJRJTc1qJjIjIBVjrAAMr9VxqDz9RwKry+gRlFrlEpViIhtwM+TXzNshwGO63qjNc5OnxImEXxQ8ghZCgbjfXpw8ZDjleUgqLN0h1TeR6dOEcFMoaSTD5R7OUExGvJSzlE8fuf8I7kkY8XLYWuECbl23mLkaEfWNPZz56RPp/wBX8PU29uk+rftrYV8VFMBDjs5WzvlFPzApdeOLUlyr2Vjde1A== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Both architectures have almost the same implementation: __cont_access_flags_changed() is also correct on riscv and brings the same benefits (ie don't do anything if the flags are unchanged). As in the previous commit, get_clear_contig_flush() is duplicated in both the arch and the generic codes, it will be removed from the arch code when the last reference there gets moved to the generic code. Signed-off-by: Alexandre Ghiti --- arch/arm64/include/asm/hugetlb.h | 4 -- arch/arm64/mm/hugetlbpage.c | 65 --------------------------- arch/riscv/include/asm/hugetlb.h | 5 --- arch/riscv/include/asm/pgtable.h | 7 +-- arch/riscv/mm/hugetlbpage.c | 28 ------------ arch/riscv/mm/pgtable.c | 6 +-- include/linux/hugetlb_contpte.h | 5 +++ mm/hugetlb_contpte.c | 75 ++++++++++++++++++++++++++++++++ 8 files changed, 87 insertions(+), 108 deletions(-) diff --git a/arch/arm64/include/asm/hugetlb.h b/arch/arm64/include/asm/hugetlb.h index 5c605a0a2017..654f5f2f03a3 100644 --- a/arch/arm64/include/asm/hugetlb.h +++ b/arch/arm64/include/asm/hugetlb.h @@ -35,10 +35,6 @@ static inline void arch_clear_hugetlb_flags(struct folio *folio) pte_t arch_make_huge_pte(pte_t entry, unsigned int shift, vm_flags_t flags); #define arch_make_huge_pte arch_make_huge_pte -#define __HAVE_ARCH_HUGE_PTEP_SET_ACCESS_FLAGS -extern int huge_ptep_set_access_flags(struct vm_area_struct *vma, - unsigned long addr, pte_t *ptep, - pte_t pte, int dirty); #define __HAVE_ARCH_HUGE_PTEP_SET_WRPROTECT extern void huge_ptep_set_wrprotect(struct mm_struct *mm, unsigned long addr, pte_t *ptep); diff --git a/arch/arm64/mm/hugetlbpage.c b/arch/arm64/mm/hugetlbpage.c index 0b7a53fee55d..643ba2043f0f 100644 --- a/arch/arm64/mm/hugetlbpage.c +++ b/arch/arm64/mm/hugetlbpage.c @@ -277,71 +277,6 @@ pte_t arch_make_huge_pte(pte_t entry, unsigned int shift, vm_flags_t flags) return entry; } -/* - * huge_ptep_set_access_flags will update access flags (dirty, accesssed) - * and write permission. - * - * For a contiguous huge pte range we need to check whether or not write - * permission has to change only on the first pte in the set. Then for - * all the contiguous ptes we need to check whether or not there is a - * discrepancy between dirty or young. - */ -static int __cont_access_flags_changed(pte_t *ptep, pte_t pte, int ncontig) -{ - int i; - - if (pte_write(pte) != pte_write(__ptep_get(ptep))) - return 1; - - for (i = 0; i < ncontig; i++) { - pte_t orig_pte = __ptep_get(ptep + i); - - if (pte_dirty(pte) != pte_dirty(orig_pte)) - return 1; - - if (pte_young(pte) != pte_young(orig_pte)) - return 1; - } - - return 0; -} - -int huge_ptep_set_access_flags(struct vm_area_struct *vma, - unsigned long addr, pte_t *ptep, - pte_t pte, int dirty) -{ - int ncontig, i; - size_t pgsize = 0; - unsigned long pfn = pte_pfn(pte), dpfn; - struct mm_struct *mm = vma->vm_mm; - pgprot_t hugeprot; - pte_t orig_pte; - - if (!pte_cont(pte)) - return __ptep_set_access_flags(vma, addr, ptep, pte, dirty); - - ncontig = find_num_contig(mm, addr, ptep, &pgsize); - dpfn = pgsize >> PAGE_SHIFT; - - if (!__cont_access_flags_changed(ptep, pte, ncontig)) - return 0; - - orig_pte = get_clear_contig_flush(mm, addr, ptep, pgsize, ncontig); - - /* Make sure we don't lose the dirty or young state */ - if (pte_dirty(orig_pte)) - pte = pte_mkdirty(pte); - - if (pte_young(orig_pte)) - pte = pte_mkyoung(pte); - - hugeprot = pte_pgprot(pte); - for (i = 0; i < ncontig; i++, ptep++, addr += pgsize, pfn += dpfn) - __set_ptes(mm, addr, ptep, pfn_pte(pfn, hugeprot), 1); - - return 1; -} - void huge_ptep_set_wrprotect(struct mm_struct *mm, unsigned long addr, pte_t *ptep) { diff --git a/arch/riscv/include/asm/hugetlb.h b/arch/riscv/include/asm/hugetlb.h index 0fbb6b19df79..bf533c2cef84 100644 --- a/arch/riscv/include/asm/hugetlb.h +++ b/arch/riscv/include/asm/hugetlb.h @@ -28,11 +28,6 @@ pte_t huge_ptep_clear_flush(struct vm_area_struct *vma, void huge_ptep_set_wrprotect(struct mm_struct *mm, unsigned long addr, pte_t *ptep); -#define __HAVE_ARCH_HUGE_PTEP_SET_ACCESS_FLAGS -int huge_ptep_set_access_flags(struct vm_area_struct *vma, - unsigned long addr, pte_t *ptep, - pte_t pte, int dirty); - pte_t arch_make_huge_pte(pte_t entry, unsigned int shift, vm_flags_t flags); #define arch_make_huge_pte arch_make_huge_pte diff --git a/arch/riscv/include/asm/pgtable.h b/arch/riscv/include/asm/pgtable.h index 705d666e014d..290d5fbfe031 100644 --- a/arch/riscv/include/asm/pgtable.h +++ b/arch/riscv/include/asm/pgtable.h @@ -629,9 +629,8 @@ static inline void __pte_clear(struct mm_struct *mm, __set_pte_at(mm, ptep, __pte(0)); } -#define __HAVE_ARCH_PTEP_SET_ACCESS_FLAGS /* defined in mm/pgtable.c */ -extern int ptep_set_access_flags(struct vm_area_struct *vma, unsigned long address, - pte_t *ptep, pte_t entry, int dirty); +extern int __ptep_set_access_flags(struct vm_area_struct *vma, unsigned long address, + pte_t *ptep, pte_t entry, int dirty); #define __HAVE_ARCH_PTEP_TEST_AND_CLEAR_YOUNG /* defined in mm/pgtable.c */ extern int ptep_test_and_clear_young(struct vm_area_struct *vma, unsigned long address, pte_t *ptep); @@ -740,6 +739,8 @@ static inline pte_t ptep_get(pte_t *ptep) #define __HAVE_ARCH_PTEP_GET_AND_CLEAR #define ptep_get_and_clear __ptep_get_and_clear #define pte_clear __pte_clear +#define __HAVE_ARCH_PTEP_SET_ACCESS_FLAGS +#define ptep_set_access_flags __ptep_set_access_flags #define pgprot_nx pgprot_nx static inline pgprot_t pgprot_nx(pgprot_t _prot) diff --git a/arch/riscv/mm/hugetlbpage.c b/arch/riscv/mm/hugetlbpage.c index b44023336fd9..0e2ca7327479 100644 --- a/arch/riscv/mm/hugetlbpage.c +++ b/arch/riscv/mm/hugetlbpage.c @@ -173,34 +173,6 @@ pte_t arch_make_huge_pte(pte_t entry, unsigned int shift, vm_flags_t flags) return entry; } -int huge_ptep_set_access_flags(struct vm_area_struct *vma, - unsigned long addr, - pte_t *ptep, - pte_t pte, - int dirty) -{ - struct mm_struct *mm = vma->vm_mm; - pte_t orig_pte; - int pte_num; - - if (!pte_napot(pte)) - return ptep_set_access_flags(vma, addr, ptep, pte, dirty); - - pte_num = arch_contpte_get_num_contig(vma->vm_mm, addr, ptep, 0, NULL); - - orig_pte = get_clear_contig_flush(mm, addr, ptep, pte_num); - - if (pte_dirty(orig_pte)) - pte = pte_mkdirty(pte); - - if (pte_young(orig_pte)) - pte = pte_mkyoung(pte); - - set_ptes(mm, addr, ptep, pte, pte_num); - - return true; -} - void huge_ptep_set_wrprotect(struct mm_struct *mm, unsigned long addr, pte_t *ptep) diff --git a/arch/riscv/mm/pgtable.c b/arch/riscv/mm/pgtable.c index 4ae67324f992..af8b3769a349 100644 --- a/arch/riscv/mm/pgtable.c +++ b/arch/riscv/mm/pgtable.c @@ -5,9 +5,9 @@ #include #include -int ptep_set_access_flags(struct vm_area_struct *vma, - unsigned long address, pte_t *ptep, - pte_t entry, int dirty) +int __ptep_set_access_flags(struct vm_area_struct *vma, + unsigned long address, pte_t *ptep, + pte_t entry, int dirty) { asm goto(ALTERNATIVE("nop", "j %l[svvptc]", 0, RISCV_ISA_EXT_SVVPTC, 1) : : : : svvptc); diff --git a/include/linux/hugetlb_contpte.h b/include/linux/hugetlb_contpte.h index 20d3a3e14e14..fea47035ac38 100644 --- a/include/linux/hugetlb_contpte.h +++ b/include/linux/hugetlb_contpte.h @@ -22,4 +22,9 @@ extern void huge_pte_clear(struct mm_struct *mm, unsigned long addr, extern pte_t huge_ptep_get_and_clear(struct mm_struct *mm, unsigned long addr, pte_t *ptep); +#define __HAVE_ARCH_HUGE_PTEP_SET_ACCESS_FLAGS +extern int huge_ptep_set_access_flags(struct vm_area_struct *vma, + unsigned long addr, pte_t *ptep, + pte_t pte, int dirty); + #endif /* _LINUX_HUGETLB_CONTPTE_H */ diff --git a/mm/hugetlb_contpte.c b/mm/hugetlb_contpte.c index 0c86c6f77c29..49950c1ce615 100644 --- a/mm/hugetlb_contpte.c +++ b/mm/hugetlb_contpte.c @@ -13,6 +13,7 @@ * - __set_ptes() * - __ptep_get_and_clear() * - __pte_clear() + * - __ptep_set_access_flags() * - pte_cont() * - arch_contpte_get_num_contig() */ @@ -23,6 +24,7 @@ * - set_huge_pte_at() * - huge_pte_clear() * - huge_ptep_get_and_clear() + * - huge_ptep_set_access_flags() */ pte_t huge_ptep_get(struct mm_struct *mm, unsigned long addr, pte_t *ptep) @@ -157,3 +159,76 @@ pte_t huge_ptep_get_and_clear(struct mm_struct *mm, return get_clear_contig(mm, addr, ptep, pgsize, ncontig); } + +/* + * huge_ptep_set_access_flags will update access flags (dirty, accesssed) + * and write permission. + * + * For a contiguous huge pte range we need to check whether or not write + * permission has to change only on the first pte in the set. Then for + * all the contiguous ptes we need to check whether or not there is a + * discrepancy between dirty or young. + */ +static int __cont_access_flags_changed(pte_t *ptep, pte_t pte, int ncontig) +{ + int i; + + if (pte_write(pte) != pte_write(__ptep_get(ptep))) + return 1; + + for (i = 0; i < ncontig; i++) { + pte_t orig_pte = __ptep_get(ptep + i); + + if (pte_dirty(pte) != pte_dirty(orig_pte)) + return 1; + + if (pte_young(pte) != pte_young(orig_pte)) + return 1; + } + + return 0; +} + +static pte_t get_clear_contig_flush(struct mm_struct *mm, + unsigned long addr, + pte_t *ptep, + unsigned long pgsize, + unsigned long ncontig) +{ + pte_t orig_pte = get_clear_contig(mm, addr, ptep, pgsize, ncontig); + struct vm_area_struct vma = TLB_FLUSH_VMA(mm, 0); + + flush_tlb_range(&vma, addr, addr + (pgsize * ncontig)); + return orig_pte; +} + +int huge_ptep_set_access_flags(struct vm_area_struct *vma, + unsigned long addr, pte_t *ptep, + pte_t pte, int dirty) +{ + int ncontig; + size_t pgsize = 0; + struct mm_struct *mm = vma->vm_mm; + pte_t orig_pte; + + if (!pte_cont(pte)) + return __ptep_set_access_flags(vma, addr, ptep, pte, dirty); + + ncontig = arch_contpte_get_num_contig(vma->vm_mm, addr, ptep, 0, &pgsize); + + if (!__cont_access_flags_changed(ptep, pte, ncontig)) + return 0; + + orig_pte = get_clear_contig_flush(mm, addr, ptep, pgsize, ncontig); + + /* Make sure we don't lose the dirty or young state */ + if (pte_dirty(orig_pte)) + pte = pte_mkdirty(pte); + + if (pte_young(orig_pte)) + pte = pte_mkyoung(pte); + + set_contptes(mm, addr, ptep, pte, ncontig, pgsize); + + return 1; +} From patchwork Mon Jan 27 09:35:29 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexandre Ghiti X-Patchwork-Id: 13951131 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id D0204C0218C for ; Mon, 27 Jan 2025 09:43:56 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 6AA25280141; Mon, 27 Jan 2025 04:43:56 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 6338728013F; Mon, 27 Jan 2025 04:43:56 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 4AD58280141; Mon, 27 Jan 2025 04:43:56 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 2A22828013F for ; Mon, 27 Jan 2025 04:43:56 -0500 (EST) Received: from smtpin14.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id DA56E1A213D for ; Mon, 27 Jan 2025 09:43:55 +0000 (UTC) X-FDA: 83052745230.14.89E7B47 Received: from mail-wr1-f43.google.com (mail-wr1-f43.google.com [209.85.221.43]) by imf07.hostedemail.com (Postfix) with ESMTP id DFDA140002 for ; Mon, 27 Jan 2025 09:43:53 +0000 (UTC) Authentication-Results: imf07.hostedemail.com; dkim=pass header.d=rivosinc-com.20230601.gappssmtp.com header.s=20230601 header.b=xT4n3iaK; spf=pass (imf07.hostedemail.com: domain of alexghiti@rivosinc.com designates 209.85.221.43 as permitted sender) smtp.mailfrom=alexghiti@rivosinc.com; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1737971034; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=C5hVtVN5ts8qRfTru3U6IxaXR2nRBQvNb9/UQuUojb8=; b=vRzhYT04RC/5DfGufKZrgPRjc1CSH0dJj66B12Yd4/B2JVCg3TY7rz2bTdbGs3KksWLKsS YcYFMJK74dUP6P/s28omS4ewqq6LOqOCmY+SuQ+O4BiSShPwKNhYi9LbG9dGRvChHHIilS xTGZ8H/AZtRQ0hLEonEifZ7yNvf+kmc= ARC-Authentication-Results: i=1; imf07.hostedemail.com; dkim=pass header.d=rivosinc-com.20230601.gappssmtp.com header.s=20230601 header.b=xT4n3iaK; spf=pass (imf07.hostedemail.com: domain of alexghiti@rivosinc.com designates 209.85.221.43 as permitted sender) smtp.mailfrom=alexghiti@rivosinc.com; dmarc=none ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1737971034; a=rsa-sha256; cv=none; b=XaheYWCyp4HXU28A6DKbsZJLJb/QOuutjEL+Pdymj7C6vHzysyZW+By49tFhyPTt2CCRt2 odo8VgIgYoG7POexOcsPIAWGKVwGt6R2HloWDbJ67FjTkQgGk77pr1DQsBjEGtbAoC5ZLW KWQ8NJuC9AVULZLPmlWbyUJzdXIqgtU= Received: by mail-wr1-f43.google.com with SMTP id ffacd0b85a97d-388cae9eb9fso2395011f8f.3 for ; Mon, 27 Jan 2025 01:43:53 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc-com.20230601.gappssmtp.com; s=20230601; t=1737971032; x=1738575832; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=C5hVtVN5ts8qRfTru3U6IxaXR2nRBQvNb9/UQuUojb8=; b=xT4n3iaK7VJwhxRtIng+lflfZIhZR02Dsd6endAFAv3abuUHRMo6CNiqsUeCgSej58 FNw6+jw3vdhz1zQo7Azo2B3r1vI/sJC47Iuc3hCbepmRheahLyKIcc8ZOfC01vzc8+4M wT7veFhc5XINfgXwcGIVohF16YSkQbPAtmqeSpJIRpJ1YYOlttrqL8QUUoOulysh2xZw bfT7g1lD+ja1oBC1wnnWDj6BYzeMDmydbozr4AipNFdIytxWo6vjr+OA7WrzAEZty5LE 3f6N/RVHuQ/i8k3UqCSTgomU116ZNMDj3sjmv/hnredo/Hrp+bTW9j4hl5QNWVN86XQR Jg+Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1737971032; x=1738575832; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=C5hVtVN5ts8qRfTru3U6IxaXR2nRBQvNb9/UQuUojb8=; b=t8h7QyjaCNNI/qhFbaBWjD9kycQVa2D9lq5a4f4cJlu9HrNGZ5Gfa2VM8wShxyIEC0 dcEz0qRrM3nw1JukDZMWykYnfJs9S25HyU9XIbUgMLSodxYS01mstKqoEKu1nEZaIj1B FbrI4uB4jEy9SCDJems6I5n74Q7YFPT4fh+Jcmpojk4K10k+n/QjD7HgUKGHDAgVUuE/ 44K3XEdsOAvud0L6PzlXOTta9IqakdZ0sZZdF35w90dkNLONKyqWYvkcL3hBCCl5wWiu v11stsxBQ/eq56eykAHdBhjY4YQK3RcJiX3xF8/Z/UYk3+n/ENXCv6MlzsZIR5Qt+/z1 9exg== X-Forwarded-Encrypted: i=1; AJvYcCXifsIVPKgXaLcvFGlzJ/9bEOxUOJEOyyZ7ygEq1zHE7fHOnD4zcds22lV0bzTwiff/g9+dKbfpYg==@kvack.org X-Gm-Message-State: AOJu0YzVwUWdgQZtzkB2eg17OJ809ztdKBSgDVNqp0aBYYViGwfmrctJ SKwuJNL/rGZ8PRh+XEnyERe72yKdtlc5WexWzaBP9fhteRtz+w4jfoAJB7mGva8= X-Gm-Gg: ASbGnctPQaKqbeKjcNujDQrMiFh2TGZCsI8Ha70K3xbCWsfHq1Ac5iBrSwJo5Au+TKM VdPh0patuzP5QqzgJbuVy9fcwOnSvON+GFg2i9+1DthEMHsIG/iLfZL7ricjymEDJXR8k1NOZ7T wvV4gd2l/VgWr2lVXIri4XRqLiXJbWdlKs0dfd3FcN/WhUEeaV5eDcrd9z7L2bjQUDc7X0zP0Tr 38fgf3njnx9MgQm5cPl9MIrNvDZUm4rPSBeN58GHypuoh4ViN/g9H4fvAD4IT+LcVSJKxpziEwi DpF3ApSRRDJ5Qs3FSMC/yr30SzY2kXQ= X-Google-Smtp-Source: AGHT+IFXkE/IMZA1k4v5FDtRJn2j5Onh6MT51lwkeaLHVsBqz96ACklS2Q489STBeMVgKrxepukPbg== X-Received: by 2002:a05:6000:18a8:b0:385:f69a:7e5f with SMTP id ffacd0b85a97d-38bf57a9a86mr46329622f8f.38.1737971032476; Mon, 27 Jan 2025 01:43:52 -0800 (PST) Received: from alex-rivos.ba.rivosinc.com ([2001:861:3382:ef90:3e22:3e78:ce5a:32c3]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-38c2a17d6a6sm10405111f8f.26.2025.01.27.01.43.51 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 27 Jan 2025 01:43:52 -0800 (PST) From: Alexandre Ghiti To: Catalin Marinas , Will Deacon , Ryan Roberts , Mark Rutland , Paul Walmsley , Palmer Dabbelt , Albert Ou , Andrew Morton , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org, linux-mm@kvack.org Cc: Alexandre Ghiti Subject: [PATCH v4 8/9] mm: Use common huge_ptep_set_wrprotect() function for riscv/arm64 Date: Mon, 27 Jan 2025 10:35:29 +0100 Message-Id: <20250127093530.19548-9-alexghiti@rivosinc.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20250127093530.19548-1-alexghiti@rivosinc.com> References: <20250127093530.19548-1-alexghiti@rivosinc.com> MIME-Version: 1.0 X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: DFDA140002 X-Stat-Signature: eqbnr45bynpzai47wqpwoms7jsqtxe9a X-Rspam-User: X-HE-Tag: 1737971033-15454 X-HE-Meta: U2FsdGVkX1/3E68qgZlLp0QzhsW7A2JDBU9GznJxf5U0MjggHOWi3fLlPQMU70LOy1i0QFxOgc9WOFJUGvfGrE7Ped72R/tHRxCHmzIIGFn4ogeNZROrRUVTNDe5VrYfjMuE4KwlHqjmBebq/90qEa2wA7hAAL4UCZMXddb7rBBUynFdX6BhxcQcMB9/fm1fJawUTb5Bw+VHHCZmef6GRM9919+r1tDPBXAzYj85qz6QQ/ZxnfMQ4szaGmr0APMfXeE/77ATttBNelBnduJEBZG7coEif1001EvZ9MueZN5RxuRgob6CnmGd5pYz7Jl/kITF7N0CedhGTvOR1fAw5UjFNV4UP0v1TXVP09+uIlttiBhrHjM4rt68F8FoKAJyEnnsyU4XdjWQMvrZBGAQALs8YzcTOANOjXwdNBsOL8261xDfxo4bBuj3dnLs4I72rYtSLzrTCCsAuDjFA9tFkJRHWxDFkDwC0dnLay0Yru24xGRmek6T+XOne2o0MYi/99NI/e/F3ves+dU4YbQ0TRyQPiP8rb/xmRQ7VGnLKl8sTsBOgQuEHG6v8BJ5X/pmFLl9MRoor/oaZRkc37FuXvYV/Qsd65ZY8kL2yON0BgmO4we4LYbgDFi2dnsUq2QLg3eqzObagmpCT376lROJ0E86fiXkTwLTUOB4qH5KgcUX5kasM7JBcxuwRdhWpncElsQGhD3d0XGZCqUAg73TIr+LKaceOH/JscU39AvJhhJMBRKbTisLKK0Anq6vgFqrZEhTQBQ3LSuGalqYmml5KyBpZM4xs+GYclLilSzFr80dwxV+nr83nZLdJcBMxkb6fKyzOx15gPyFYvY+ebxf5xc44gthGDbeDx/ETAXqUEFyu+cmISMPmw301vAC147filOVWO7yencCIzADaHNXSnwPH3L4SWSH1ZBuK9Eg7aFK0jeOIMq2J/YeOhzldwbrY57YA461eidLyfsRPhY 1eeFuMu1 Z9t5ce/1wpJYF7DG8TGD58x7BCE/VcqA3nHt+Jk8DRsu0Nq0LnKdHP9aXa4muX4NqYf1iLJKH/r2qJYC3WcMdCmDmDwPeVvusw1t3Fq+/y9MmSsATtK0xRNKEmalCtWswr2UYZxychzrvyt2dqseRcSYgbeH4i6h2y2Q3nCkgvKXKZiOt1nnNmHetPwI5ng143/BF7OLUZ/w1t5zienRBFCsoAaSQX88QMDHAx0KnVz6yEg87JDq6Wv2UHHi7k4F8DKb2wSG2cnG77UU80+8/0yfsInpjcRpHklrV7+n1M0rrPZcMQsKdSyc2GAXTS20ntu2jJg1oniyAcDzFtAEzNdpQoDGFXXVM3RhsIOTR1h/9QcVA1MkISOyawQ5k5+vNEmnThEAl927ggWeOtk7anF294AeGwzTS356bclZV14c3nOjztjyX30P3flvaUO6S8k5KpobivRmcuI8A1aBr1gvz8g== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: After some adjustments, both architectures have the same implementation so move it to the generic code. Signed-off-by: Alexandre Ghiti --- arch/arm64/include/asm/hugetlb.h | 3 --- arch/arm64/mm/hugetlbpage.c | 27 --------------------------- arch/riscv/include/asm/hugetlb.h | 4 ---- arch/riscv/include/asm/pgtable.h | 7 ++++--- arch/riscv/mm/hugetlbpage.c | 22 ---------------------- include/linux/hugetlb_contpte.h | 4 ++++ mm/hugetlb_contpte.c | 22 ++++++++++++++++++++++ 7 files changed, 30 insertions(+), 59 deletions(-) diff --git a/arch/arm64/include/asm/hugetlb.h b/arch/arm64/include/asm/hugetlb.h index 654f5f2f03a3..fd1de0caad3f 100644 --- a/arch/arm64/include/asm/hugetlb.h +++ b/arch/arm64/include/asm/hugetlb.h @@ -35,9 +35,6 @@ static inline void arch_clear_hugetlb_flags(struct folio *folio) pte_t arch_make_huge_pte(pte_t entry, unsigned int shift, vm_flags_t flags); #define arch_make_huge_pte arch_make_huge_pte -#define __HAVE_ARCH_HUGE_PTEP_SET_WRPROTECT -extern void huge_ptep_set_wrprotect(struct mm_struct *mm, - unsigned long addr, pte_t *ptep); #define __HAVE_ARCH_HUGE_PTEP_CLEAR_FLUSH extern pte_t huge_ptep_clear_flush(struct vm_area_struct *vma, unsigned long addr, pte_t *ptep); diff --git a/arch/arm64/mm/hugetlbpage.c b/arch/arm64/mm/hugetlbpage.c index 643ba2043f0f..0430cb41f381 100644 --- a/arch/arm64/mm/hugetlbpage.c +++ b/arch/arm64/mm/hugetlbpage.c @@ -277,33 +277,6 @@ pte_t arch_make_huge_pte(pte_t entry, unsigned int shift, vm_flags_t flags) return entry; } -void huge_ptep_set_wrprotect(struct mm_struct *mm, - unsigned long addr, pte_t *ptep) -{ - unsigned long pfn, dpfn; - pgprot_t hugeprot; - int ncontig, i; - size_t pgsize; - pte_t pte; - - if (!pte_cont(__ptep_get(ptep))) { - __ptep_set_wrprotect(mm, addr, ptep); - return; - } - - ncontig = find_num_contig(mm, addr, ptep, &pgsize); - dpfn = pgsize >> PAGE_SHIFT; - - pte = get_clear_contig_flush(mm, addr, ptep, pgsize, ncontig); - pte = pte_wrprotect(pte); - - hugeprot = pte_pgprot(pte); - pfn = pte_pfn(pte); - - for (i = 0; i < ncontig; i++, ptep++, addr += pgsize, pfn += dpfn) - __set_ptes(mm, addr, ptep, pfn_pte(pfn, hugeprot), 1); -} - pte_t huge_ptep_clear_flush(struct vm_area_struct *vma, unsigned long addr, pte_t *ptep) { diff --git a/arch/riscv/include/asm/hugetlb.h b/arch/riscv/include/asm/hugetlb.h index bf533c2cef84..4c692dd82779 100644 --- a/arch/riscv/include/asm/hugetlb.h +++ b/arch/riscv/include/asm/hugetlb.h @@ -24,10 +24,6 @@ bool arch_hugetlb_migration_supported(struct hstate *h); pte_t huge_ptep_clear_flush(struct vm_area_struct *vma, unsigned long addr, pte_t *ptep); -#define __HAVE_ARCH_HUGE_PTEP_SET_WRPROTECT -void huge_ptep_set_wrprotect(struct mm_struct *mm, - unsigned long addr, pte_t *ptep); - pte_t arch_make_huge_pte(pte_t entry, unsigned int shift, vm_flags_t flags); #define arch_make_huge_pte arch_make_huge_pte diff --git a/arch/riscv/include/asm/pgtable.h b/arch/riscv/include/asm/pgtable.h index 290d5fbfe031..5a29153a4013 100644 --- a/arch/riscv/include/asm/pgtable.h +++ b/arch/riscv/include/asm/pgtable.h @@ -645,9 +645,8 @@ static inline pte_t __ptep_get_and_clear(struct mm_struct *mm, return pte; } -#define __HAVE_ARCH_PTEP_SET_WRPROTECT -static inline void ptep_set_wrprotect(struct mm_struct *mm, - unsigned long address, pte_t *ptep) +static inline void __ptep_set_wrprotect(struct mm_struct *mm, + unsigned long address, pte_t *ptep) { atomic_long_and(~(unsigned long)_PAGE_WRITE, (atomic_long_t *)ptep); } @@ -741,6 +740,8 @@ static inline pte_t ptep_get(pte_t *ptep) #define pte_clear __pte_clear #define __HAVE_ARCH_PTEP_SET_ACCESS_FLAGS #define ptep_set_access_flags __ptep_set_access_flags +#define __HAVE_ARCH_PTEP_SET_WRPROTECT +#define ptep_set_wrprotect __ptep_set_wrprotect #define pgprot_nx pgprot_nx static inline pgprot_t pgprot_nx(pgprot_t _prot) diff --git a/arch/riscv/mm/hugetlbpage.c b/arch/riscv/mm/hugetlbpage.c index 0e2ca7327479..8963a4e77742 100644 --- a/arch/riscv/mm/hugetlbpage.c +++ b/arch/riscv/mm/hugetlbpage.c @@ -173,28 +173,6 @@ pte_t arch_make_huge_pte(pte_t entry, unsigned int shift, vm_flags_t flags) return entry; } -void huge_ptep_set_wrprotect(struct mm_struct *mm, - unsigned long addr, - pte_t *ptep) -{ - pte_t pte = ptep_get(ptep); - pte_t orig_pte; - int pte_num; - - if (!pte_napot(pte)) { - ptep_set_wrprotect(mm, addr, ptep); - return; - } - - pte_num = arch_contpte_get_num_contig(mm, addr, ptep, 0, NULL); - - orig_pte = get_clear_contig_flush(mm, addr, ptep, pte_num); - - orig_pte = pte_wrprotect(orig_pte); - - set_ptes(mm, addr, ptep, orig_pte, pte_num); -} - pte_t huge_ptep_clear_flush(struct vm_area_struct *vma, unsigned long addr, pte_t *ptep) diff --git a/include/linux/hugetlb_contpte.h b/include/linux/hugetlb_contpte.h index fea47035ac38..02bce0ed93d8 100644 --- a/include/linux/hugetlb_contpte.h +++ b/include/linux/hugetlb_contpte.h @@ -27,4 +27,8 @@ extern int huge_ptep_set_access_flags(struct vm_area_struct *vma, unsigned long addr, pte_t *ptep, pte_t pte, int dirty); +#define __HAVE_ARCH_HUGE_PTEP_SET_WRPROTECT +extern void huge_ptep_set_wrprotect(struct mm_struct *mm, + unsigned long addr, pte_t *ptep); + #endif /* _LINUX_HUGETLB_CONTPTE_H */ diff --git a/mm/hugetlb_contpte.c b/mm/hugetlb_contpte.c index 49950c1ce615..de505350ef48 100644 --- a/mm/hugetlb_contpte.c +++ b/mm/hugetlb_contpte.c @@ -14,6 +14,7 @@ * - __ptep_get_and_clear() * - __pte_clear() * - __ptep_set_access_flags() + * - __ptep_set_wrprotect() * - pte_cont() * - arch_contpte_get_num_contig() */ @@ -25,6 +26,7 @@ * - huge_pte_clear() * - huge_ptep_get_and_clear() * - huge_ptep_set_access_flags() + * - huge_ptep_set_wrprotect() */ pte_t huge_ptep_get(struct mm_struct *mm, unsigned long addr, pte_t *ptep) @@ -232,3 +234,23 @@ int huge_ptep_set_access_flags(struct vm_area_struct *vma, return 1; } + +void huge_ptep_set_wrprotect(struct mm_struct *mm, + unsigned long addr, pte_t *ptep) +{ + int ncontig; + size_t pgsize; + pte_t pte; + + if (!pte_cont(__ptep_get(ptep))) { + __ptep_set_wrprotect(mm, addr, ptep); + return; + } + + ncontig = arch_contpte_get_num_contig(mm, addr, ptep, 0, &pgsize); + + pte = get_clear_contig_flush(mm, addr, ptep, pgsize, ncontig); + pte = pte_wrprotect(pte); + + set_contptes(mm, addr, ptep, pte, ncontig, pgsize); +} From patchwork Mon Jan 27 09:35:30 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexandre Ghiti X-Patchwork-Id: 13951149 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 547FCC0218F for ; Mon, 27 Jan 2025 09:44:58 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id DB0A9280142; Mon, 27 Jan 2025 04:44:57 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id D608628013F; Mon, 27 Jan 2025 04:44:57 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C00E1280142; Mon, 27 Jan 2025 04:44:57 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id A177528013F for ; Mon, 27 Jan 2025 04:44:57 -0500 (EST) Received: from smtpin12.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 63C14B2985 for ; Mon, 27 Jan 2025 09:44:57 +0000 (UTC) X-FDA: 83052747834.12.6404107 Received: from mail-wm1-f43.google.com (mail-wm1-f43.google.com [209.85.128.43]) by imf20.hostedemail.com (Postfix) with ESMTP id 7424C1C000A for ; Mon, 27 Jan 2025 09:44:55 +0000 (UTC) Authentication-Results: imf20.hostedemail.com; dkim=pass header.d=rivosinc-com.20230601.gappssmtp.com header.s=20230601 header.b=eW+Kjmtu; dmarc=none; spf=pass (imf20.hostedemail.com: domain of alexghiti@rivosinc.com designates 209.85.128.43 as permitted sender) smtp.mailfrom=alexghiti@rivosinc.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1737971095; a=rsa-sha256; cv=none; b=aeZWGAuIpQxjqyOHEEP9/OKU3LYAEac4dpktxCaNx38A6QWW8UUIwUOU+Ra59KYDucnB74 qWcvahw1Qln53AE9Dlh3sBSdzH3/UZmH9ST7jmLN0O+x73PMaW29wPlsfgHD5c+b03HO8g Rkyfk+9kV+yDsz6ohmOl0dyWTNuhi4Y= ARC-Authentication-Results: i=1; imf20.hostedemail.com; dkim=pass header.d=rivosinc-com.20230601.gappssmtp.com header.s=20230601 header.b=eW+Kjmtu; dmarc=none; spf=pass (imf20.hostedemail.com: domain of alexghiti@rivosinc.com designates 209.85.128.43 as permitted sender) smtp.mailfrom=alexghiti@rivosinc.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1737971095; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=jsGp3DbjRPLkoj2kwFjv1+EJYiVm/DSEknC/OoKntbQ=; b=rnAemXc/k7bNd379lB/p686MToCZHjNRFUEgs27/ZFh9F0kHbL945tRA/hMv4hrZAsDYtW aH6cm8Hz7QpPQJP2gEVB5zeGYjwGLLyljgP3Ik05RXZ+zIcNkpuoTL8eAG8TrlSKVE8oJz zqreBTXq32wxJhdtwKvbRnoFu3EtN1s= Received: by mail-wm1-f43.google.com with SMTP id 5b1f17b1804b1-4364a37a1d7so43285495e9.3 for ; Mon, 27 Jan 2025 01:44:55 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc-com.20230601.gappssmtp.com; s=20230601; t=1737971094; x=1738575894; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=jsGp3DbjRPLkoj2kwFjv1+EJYiVm/DSEknC/OoKntbQ=; b=eW+KjmtuQr8NlAJK/1BD+kajk7HD91lhGu3uY/MtmRTCP7ZTv1pD3bmiXh0+Pl4Jkk wL906wZ8voX/06Ustz5VpdfUmh6dBJmzSGQYVGzbPcpxNypFvuw6CspbgnXgqk92CCXy 13jo5roq7THWk6f5FGHRILpmU4QnfZMPKd25H+4AtRUw14ExmcAp15euYK0BcZT3qjCj ec/qNpRasQdaEXbVKPyoF5RieXAu7MrGqCJJnL6++8cTGkwN/nupuffZqR2O4xCX1fGj GX41zpV8pZu8J6AyWclmebHnrp0BzkcDCrHZzECj4GpWkMypXl4T9z3ZyKyLYgWIBkTi HqdQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1737971094; x=1738575894; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=jsGp3DbjRPLkoj2kwFjv1+EJYiVm/DSEknC/OoKntbQ=; b=H8e2IJJOIL0IhyHP2WU55dHvxTmHe5L73nH7maAd9+l3/NtwdBREd5kjF8wmADFy02 bu/Mn/RQzVfXT8j6ohk6Vjz5qewixgCyKrQ500xQjN11/fh2BxAQuUhwB5peumKH4B9p 8dM2UwCPwSgv8nvbGsV5147anzpB/j8b2VZyrSVloRa0Pk0GhfBzke71vhyiYF5HHhdb TYCs6auQ3MfN7NNVmt287ZDtPeJ8nxPh7ET4aWfvzYIDhPLY7HWRn3/J1NlipryaM/ms 5M0JManRLSxMilxb7rsY8GolSvK8ejzu6xic8hwMQTT8caF+HB2IUhALbKdvULqSZmB5 X+1A== X-Forwarded-Encrypted: i=1; AJvYcCUo30HbAD+mNYp/jurZ8E3Avu9TmxLKjqTBgWz3nwsc8KYOWX12GH1ZBeGKGgeaVP3D1qZzeadYnw==@kvack.org X-Gm-Message-State: AOJu0Yw+j+rB6PRvNjq8EhMkASEnpHuQiCfRWPAiQDKsZCR6M06cCK/i CqHUYIfqueOHgKe1odPFMfbjtlq6ysAQrrZob78sO9btN/yvBn7Gh9a1Sq5gWzg= X-Gm-Gg: ASbGncuMDqioffFe/swja7NmJ3D6Wqwtz72C5+ZdT1ZIzcHyi9o4oRJ8p8E08qRuJ6U JRn1VZNlwtZIAVczZDd/8m5mF42JBq4YpuJ5uZO7gxlEmgbW8EdqeaX4d0LoJM/szrqXvjYmUJ3 VOeDzIKXLqBGnkmb6kkjbWxWjtVvaNNTTFx5T5nuHJqPnhZOSmdDNRVM5CV1RQdarb8OkKRyRZW PADxaXNTHCg2k3IQo1CZfOcWRuuLpzBQB/gncCeuCw0Mvj58m+dZKSF3g59l7KG6Llz0bvHncBq NNV1JSW77pvLfULPRveRXwx7CFdWzBw= X-Google-Smtp-Source: AGHT+IFDaK9cNMo23MROO6bkzC2nQlBGexsaacNWVhEd87bHI/6OEZGcK91epmkFjxQ01HSL43+DVA== X-Received: by 2002:a05:600c:1ca9:b0:434:f297:8e78 with SMTP id 5b1f17b1804b1-438913cb65fmr390606995e9.7.1737971094091; Mon, 27 Jan 2025 01:44:54 -0800 (PST) Received: from alex-rivos.ba.rivosinc.com ([2001:861:3382:ef90:3e22:3e78:ce5a:32c3]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-438bd47eecasm124006275e9.6.2025.01.27.01.44.53 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 27 Jan 2025 01:44:53 -0800 (PST) From: Alexandre Ghiti To: Catalin Marinas , Will Deacon , Ryan Roberts , Mark Rutland , Paul Walmsley , Palmer Dabbelt , Albert Ou , Andrew Morton , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org, linux-mm@kvack.org Cc: Alexandre Ghiti Subject: [PATCH v4 9/9] mm: Use common huge_ptep_clear_flush() function for riscv/arm64 Date: Mon, 27 Jan 2025 10:35:30 +0100 Message-Id: <20250127093530.19548-10-alexghiti@rivosinc.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20250127093530.19548-1-alexghiti@rivosinc.com> References: <20250127093530.19548-1-alexghiti@rivosinc.com> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam01 X-Rspamd-Queue-Id: 7424C1C000A X-Stat-Signature: kxe7jj5epnkob5979stjq1o674pd6uqd X-HE-Tag: 1737971095-498855 X-HE-Meta: U2FsdGVkX1+xase6SCuQLMTVzfGofU4LIlcs8pbPg8g+u+jxXzzsh0b1/k1B1VSpmMr1IGo7cfvcllErWlvQLLgVZAtPQcqUxl3yF3QVJpejpy/15FPD9kAdjA5NvuWPP7s7reA3j69gbg+XdNvrtWjzyDl79s7uKFqx7N2/AymkQMZKodeueYG2L0/20+PCoeS5Nq69y6vwhFUyDRuR4Gh6oeMbhe3WvfHZLAo9c7f0xBNueiI2I+01fdIv+ks2sZwlPrYmX2MRTbcnr5Xudn87XClJxrtBl/d2PrK0lpr8dBDglBiM/zAbHaKB+ltiPxTUxqFtbVCaJQzMd+KSLrEfj8Wxks4OpW19Ln5hj6E6tK9aXmnwlbhFqJ7O0l3keSsJ/JsertbXhW5IK29VYCGH/ckAJnpoKQ7qF3qvfNMQfIzAETk1dIbEhl9sBQ1bb70ZuTVUhVdCTng8YunKJY9QHrI1hKv3CNm990G9oSZscK0w+YpIx/FQXQMb0jG+JxIYDYTl1PwVJeADzpulL+ouuO7wGY4ME/3x26GLmr3IHVkx3OvSgZwzgHu5J7KjsAMe0nBeIbD29H4tLBXS32bn0IYuS+Vz41MWS61S0fh6CzwyDObOSisMRozk56rsz2zp2Z3TUScWCHvr/uQdY2PNbC5BwxSXWyiJjDwcHkLb2chgS0r70jhnehOzuvuDLN1ygYjBjVzggHBNwGwFz1iTpWLuTFcz52WqWLweRuRsAvWEZmIzU5egUb1LfGNehgIGbMVzEW4BRxJaERgXWA/IA0a9mPHI01hYezps048CHU2vL8WuS9jsyG7soC5cipJkDty2KULLjX1B1rghD6ti1ygFGw0xwyDfQuS3dLPdvXU31BkVEIeB8bm6bC9Ilcw2aRV05FwxMhLVK/SSwkGFVLRLXsPGOdMXQ72qimO2MrECtSk8ra3JWW1PUOu/h/RyhiyJymFX2+XtiiM 7sECplVV QdQ6OJomEBn1/7rOSh06gCTSoY3aJSSwwFTCJBmx6MjR9XQ1Yxz0TNg2lWx05kM7alVU+BlaikmZfgcuJzEheuxYJm+dmqDsRKbDOnhxVn3Wh9aeD+gxqXOnTut245BitOw7fmva1LU58M47bexXyty1PT5MCBZ7Z43YjxgBylaCTD++nW/n8cVUC/5cDjG13U4pUvLCv/I5x/sYEHE80OZcjnrmXj6AAkSSmjiVv2Sui4p7oj4a9z9RYsW6gaNY7IykkGaCU93M7rvJeesEDBNBtjG0LaF1aeDPMcw3YmDNCcFiZxWtcgA2nmEfp8bprL8ykYGDS2s8TTrBBRIihyvlXNKzbVNkYlQNdjnQTpOj0vCTjMpIe6NPdG7j/BmeC/1X6ShtbS67r3MZsNtFkI7b5L4dm2JNyCLyvYfKsuv8eR6Vl45M8wdsDCwzubpWbVg6E1SWb5faG6wPl5zn9BNS3eQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: After some adjustments, both architectures have the same implementation so move it to the generic code. Signed-off-by: Alexandre Ghiti --- arch/arm64/include/asm/hugetlb.h | 3 -- arch/arm64/mm/hugetlbpage.c | 61 -------------------------------- arch/riscv/include/asm/hugetlb.h | 7 +--- arch/riscv/mm/hugetlbpage.c | 51 -------------------------- include/linux/hugetlb_contpte.h | 4 +++ mm/hugetlb_contpte.c | 15 ++++++++ 6 files changed, 20 insertions(+), 121 deletions(-) diff --git a/arch/arm64/include/asm/hugetlb.h b/arch/arm64/include/asm/hugetlb.h index fd1de0caad3f..3f79e4b76711 100644 --- a/arch/arm64/include/asm/hugetlb.h +++ b/arch/arm64/include/asm/hugetlb.h @@ -35,9 +35,6 @@ static inline void arch_clear_hugetlb_flags(struct folio *folio) pte_t arch_make_huge_pte(pte_t entry, unsigned int shift, vm_flags_t flags); #define arch_make_huge_pte arch_make_huge_pte -#define __HAVE_ARCH_HUGE_PTEP_CLEAR_FLUSH -extern pte_t huge_ptep_clear_flush(struct vm_area_struct *vma, - unsigned long addr, pte_t *ptep); void __init arm64_hugetlb_cma_reserve(void); diff --git a/arch/arm64/mm/hugetlbpage.c b/arch/arm64/mm/hugetlbpage.c index 0430cb41f381..270e4580e12a 100644 --- a/arch/arm64/mm/hugetlbpage.c +++ b/arch/arm64/mm/hugetlbpage.c @@ -98,53 +98,6 @@ int find_num_contig(struct mm_struct *mm, unsigned long addr, return CONT_PTES; } -/* - * Changing some bits of contiguous entries requires us to follow a - * Break-Before-Make approach, breaking the whole contiguous set - * before we can change any entries. See ARM DDI 0487A.k_iss10775, - * "Misprogramming of the Contiguous bit", page D4-1762. - * - * This helper performs the break step. - */ -static pte_t get_clear_contig(struct mm_struct *mm, - unsigned long addr, - pte_t *ptep, - unsigned long pgsize, - unsigned long ncontig) -{ - pte_t orig_pte = __ptep_get(ptep); - unsigned long i; - - for (i = 0; i < ncontig; i++, addr += pgsize, ptep++) { - pte_t pte = __ptep_get_and_clear(mm, addr, ptep); - - /* - * If HW_AFDBM is enabled, then the HW could turn on - * the dirty or accessed bit for any page in the set, - * so check them all. - */ - if (pte_dirty(pte)) - orig_pte = pte_mkdirty(orig_pte); - - if (pte_young(pte)) - orig_pte = pte_mkyoung(orig_pte); - } - return orig_pte; -} - -static pte_t get_clear_contig_flush(struct mm_struct *mm, - unsigned long addr, - pte_t *ptep, - unsigned long pgsize, - unsigned long ncontig) -{ - pte_t orig_pte = get_clear_contig(mm, addr, ptep, pgsize, ncontig); - struct vm_area_struct vma = TLB_FLUSH_VMA(mm, 0); - - flush_tlb_range(&vma, addr, addr + (pgsize * ncontig)); - return orig_pte; -} - pte_t *huge_pte_alloc(struct mm_struct *mm, struct vm_area_struct *vma, unsigned long addr, unsigned long sz) { @@ -277,20 +230,6 @@ pte_t arch_make_huge_pte(pte_t entry, unsigned int shift, vm_flags_t flags) return entry; } -pte_t huge_ptep_clear_flush(struct vm_area_struct *vma, - unsigned long addr, pte_t *ptep) -{ - struct mm_struct *mm = vma->vm_mm; - size_t pgsize; - int ncontig; - - if (!pte_cont(__ptep_get(ptep))) - return ptep_clear_flush(vma, addr, ptep); - - ncontig = find_num_contig(mm, addr, ptep, &pgsize); - return get_clear_contig_flush(mm, addr, ptep, pgsize, ncontig); -} - static int __init hugetlbpage_init(void) { if (pud_sect_supported()) diff --git a/arch/riscv/include/asm/hugetlb.h b/arch/riscv/include/asm/hugetlb.h index 4c692dd82779..63c7e4fa342a 100644 --- a/arch/riscv/include/asm/hugetlb.h +++ b/arch/riscv/include/asm/hugetlb.h @@ -20,14 +20,9 @@ bool arch_hugetlb_migration_supported(struct hstate *h); #endif #ifdef CONFIG_RISCV_ISA_SVNAPOT -#define __HAVE_ARCH_HUGE_PTEP_CLEAR_FLUSH -pte_t huge_ptep_clear_flush(struct vm_area_struct *vma, - unsigned long addr, pte_t *ptep); - pte_t arch_make_huge_pte(pte_t entry, unsigned int shift, vm_flags_t flags); #define arch_make_huge_pte arch_make_huge_pte - -#endif /*CONFIG_RISCV_ISA_SVNAPOT*/ +#endif /* CONFIG_RISCV_ISA_SVNAPOT */ #include diff --git a/arch/riscv/mm/hugetlbpage.c b/arch/riscv/mm/hugetlbpage.c index 8963a4e77742..ea1ae3a43d45 100644 --- a/arch/riscv/mm/hugetlbpage.c +++ b/arch/riscv/mm/hugetlbpage.c @@ -121,42 +121,6 @@ unsigned long hugetlb_mask_last_page(struct hstate *h) return 0UL; } -static pte_t get_clear_contig(struct mm_struct *mm, - unsigned long addr, - pte_t *ptep, - unsigned long pte_num) -{ - pte_t orig_pte = ptep_get(ptep); - unsigned long i; - - for (i = 0; i < pte_num; i++, addr += PAGE_SIZE, ptep++) { - pte_t pte = ptep_get_and_clear(mm, addr, ptep); - - if (pte_dirty(pte)) - orig_pte = pte_mkdirty(orig_pte); - - if (pte_young(pte)) - orig_pte = pte_mkyoung(orig_pte); - } - - return orig_pte; -} - -static pte_t get_clear_contig_flush(struct mm_struct *mm, - unsigned long addr, - pte_t *ptep, - unsigned long pte_num) -{ - pte_t orig_pte = get_clear_contig(mm, addr, ptep, pte_num); - struct vm_area_struct vma = TLB_FLUSH_VMA(mm, 0); - bool valid = !pte_none(orig_pte); - - if (valid) - flush_tlb_range(&vma, addr, addr + (PAGE_SIZE * pte_num)); - - return orig_pte; -} - pte_t arch_make_huge_pte(pte_t entry, unsigned int shift, vm_flags_t flags) { unsigned long order; @@ -173,21 +137,6 @@ pte_t arch_make_huge_pte(pte_t entry, unsigned int shift, vm_flags_t flags) return entry; } -pte_t huge_ptep_clear_flush(struct vm_area_struct *vma, - unsigned long addr, - pte_t *ptep) -{ - pte_t pte = ptep_get(ptep); - int pte_num; - - if (!pte_napot(pte)) - return ptep_clear_flush(vma, addr, ptep); - - pte_num = arch_contpte_get_num_contig(vma->vm_mm, addr, ptep, 0, NULL); - - return get_clear_contig_flush(vma->vm_mm, addr, ptep, pte_num); -} - static bool is_napot_size(unsigned long size) { unsigned long order; diff --git a/include/linux/hugetlb_contpte.h b/include/linux/hugetlb_contpte.h index 02bce0ed93d8..911b9cd4aa4d 100644 --- a/include/linux/hugetlb_contpte.h +++ b/include/linux/hugetlb_contpte.h @@ -31,4 +31,8 @@ extern int huge_ptep_set_access_flags(struct vm_area_struct *vma, extern void huge_ptep_set_wrprotect(struct mm_struct *mm, unsigned long addr, pte_t *ptep); +#define __HAVE_ARCH_HUGE_PTEP_CLEAR_FLUSH +extern pte_t huge_ptep_clear_flush(struct vm_area_struct *vma, + unsigned long addr, pte_t *ptep); + #endif /* _LINUX_HUGETLB_CONTPTE_H */ diff --git a/mm/hugetlb_contpte.c b/mm/hugetlb_contpte.c index de505350ef48..d27c7599ce74 100644 --- a/mm/hugetlb_contpte.c +++ b/mm/hugetlb_contpte.c @@ -27,6 +27,7 @@ * - huge_ptep_get_and_clear() * - huge_ptep_set_access_flags() * - huge_ptep_set_wrprotect() + * - huge_ptep_clear_flush() */ pte_t huge_ptep_get(struct mm_struct *mm, unsigned long addr, pte_t *ptep) @@ -254,3 +255,17 @@ void huge_ptep_set_wrprotect(struct mm_struct *mm, set_contptes(mm, addr, ptep, pte, ncontig, pgsize); } + +pte_t huge_ptep_clear_flush(struct vm_area_struct *vma, + unsigned long addr, pte_t *ptep) +{ + struct mm_struct *mm = vma->vm_mm; + size_t pgsize; + int ncontig; + + if (!pte_cont(__ptep_get(ptep))) + return ptep_clear_flush(vma, addr, ptep); + + ncontig = arch_contpte_get_num_contig(mm, addr, ptep, 0, &pgsize); + return get_clear_contig_flush(mm, addr, ptep, pgsize, ncontig); +}