From patchwork Thu Nov 14 06:59:56 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Qi Zheng X-Patchwork-Id: 13874614 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id AEEE9D65C52 for ; Thu, 14 Nov 2024 07:01:10 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 465236B0099; Thu, 14 Nov 2024 02:01:10 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 3C6256B009A; Thu, 14 Nov 2024 02:01:10 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 2197F6B009C; Thu, 14 Nov 2024 02:01:10 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id F18616B0099 for ; Thu, 14 Nov 2024 02:01:09 -0500 (EST) Received: from smtpin28.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id A895DC0F7C for ; Thu, 14 Nov 2024 07:01:09 +0000 (UTC) X-FDA: 82783803438.28.31DDA9A Received: from mail-pl1-f172.google.com (mail-pl1-f172.google.com [209.85.214.172]) by imf04.hostedemail.com (Postfix) with ESMTP id 41FC740449 for ; Thu, 14 Nov 2024 07:00:12 +0000 (UTC) Authentication-Results: imf04.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=SgB39wUL; spf=pass (imf04.hostedemail.com: domain of zhengqi.arch@bytedance.com designates 209.85.214.172 as permitted sender) smtp.mailfrom=zhengqi.arch@bytedance.com; dmarc=pass (policy=quarantine) header.from=bytedance.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1731567492; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=jUqyGtITukFY0TNyC3digbboCysogNiC1m+PxqBi3vg=; b=MaR6xX8Mn75VyDBveehX4VKFB0IjuOqykD7XZYdxRjVd7KTDof4U0hxBfmXU2aN6gl/Jix t7tH0Thv/B89HyhWknBSSOMvDdHl0SNGQHmVgliRmTAWQ5upZ4kaVSah8z9VbhAIWL+JkT yiYhcjnesMPc+SJwBykQaVUj9/zLYhE= ARC-Authentication-Results: i=1; imf04.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=SgB39wUL; spf=pass (imf04.hostedemail.com: domain of zhengqi.arch@bytedance.com designates 209.85.214.172 as permitted sender) smtp.mailfrom=zhengqi.arch@bytedance.com; dmarc=pass (policy=quarantine) header.from=bytedance.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1731567492; a=rsa-sha256; cv=none; b=0Jzvy1Z1/T3K/FehL955SZTCLgsX6vqnI0G2yxc/zgmLu9I4HV8EhtiGi2ka2cCpEXzfnC e2qJ9b46nAXZlF0vSkhCysJroabhWNtv0m3IQourFfM/xk0CCbk1MBGpjEmizQ7ndk09j1 1yPnyWgYa3snNJkliCSovgw30YKmPlY= Received: by mail-pl1-f172.google.com with SMTP id d9443c01a7336-20cdda5cfb6so2310685ad.3 for ; Wed, 13 Nov 2024 23:01:07 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1731567666; x=1732172466; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=jUqyGtITukFY0TNyC3digbboCysogNiC1m+PxqBi3vg=; b=SgB39wULZFLF1lyrwjnqGJwYbb+7oytgQEqeFwzASrHKG7KlG5GkuCW1Vm5TgPWQy+ jfd7QZuhsyFaJ4VAjPK7dUEjTmQrzor5qLb8InAA6nmRAUOwiWYAhKzi151B2wdemWLs FKrEfBCEDj//f9ujpU/AYskH7LFxlFH7yDLr/OZ2lfVB7YXX3POogCYfkWQcYhmSW2Xj UkI1TxQ+RZXpgfmB28Z4DIpEXbRU33omKqZiMwaa3fu7Uk2X2Yw6Iscf99Fh5kuaosfa BbNdj0tbu0STsUiNFKfEGBF/ItzlhUAAkx20sBH9LEMB1qKIIjWThAc8iUJHNUFp5fOW IpMA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1731567666; x=1732172466; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=jUqyGtITukFY0TNyC3digbboCysogNiC1m+PxqBi3vg=; b=jGIZXI9TvmrMbCu2pRXiW8JghwNNCbsYYHMe0BmrUKtPeBpW+yT/gSYhGtqPO6xtrf nLI67wPjEvd4Kdapd0WPqeYIlPdwjL63rXhJlGZV2QYaXIpj2lf2c1COAw1W6bHXfFFA oekO5QJCLIDXk9Tvq6zdjAfDmOjBHK0S/zvAAVZxisgVXenayKrLD3Sh2E1FF8koIP3p aKJwNA/jtL0SD/l9m0wFvkLlfAI4C5D7HmBFfo6eWsr80dWfT0CgsJpC5oeyR/1vVwSz f7nvZO3W/XQeNUjtL6WVtcc51dv9dGVjZwKgi88QIS12keK/Dj08qYchAiJP6Arvy1FR LFew== X-Forwarded-Encrypted: i=1; AJvYcCWANQFwPY5Lc4jkpNy1I+CfrpoYKOdUWOu1sxN554MTqmSOVya+EpIKinxW8aBNhXXWfVZIGcq8sg==@kvack.org X-Gm-Message-State: AOJu0YzymXuge8rh4lNA4TfbKBHY8iicdDReXW45dA1Q6+vKWJLX40Bv XH01nrt5xfCj1AgsUxhpqTgcfS478wZ82uGua0VLLa5NvPUGpoZGGNjXYFsajQ4= X-Google-Smtp-Source: AGHT+IH+CIyFvL8Lk36CjibW+Kiayt53v1Ia4QUhn9aVanblqL6M7a/wyMFzo1loiDImRID/mm6uDQ== X-Received: by 2002:a17:902:ec82:b0:20c:d469:ba95 with SMTP id d9443c01a7336-211b6609df6mr63772575ad.16.1731567666242; Wed, 13 Nov 2024 23:01:06 -0800 (PST) Received: from C02DW0BEMD6R.bytedance.net ([63.216.146.178]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-211c7d389c2sm4119065ad.268.2024.11.13.23.00.58 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 13 Nov 2024 23:01:05 -0800 (PST) From: Qi Zheng To: david@redhat.com, jannh@google.com, hughd@google.com, willy@infradead.org, muchun.song@linux.dev, vbabka@kernel.org, akpm@linux-foundation.org, peterx@redhat.com Cc: mgorman@suse.de, catalin.marinas@arm.com, will@kernel.org, dave.hansen@linux.intel.com, luto@kernel.org, peterz@infradead.org, x86@kernel.org, lorenzo.stoakes@oracle.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org, zokeefe@google.com, rientjes@google.com, Qi Zheng Subject: [PATCH v3 5/9] mm: introduce do_zap_pte_range() Date: Thu, 14 Nov 2024 14:59:56 +0800 Message-Id: <1d8467c1428573cc666ca3150ba66877f7b316cf.1731566457.git.zhengqi.arch@bytedance.com> X-Mailer: git-send-email 2.24.3 (Apple Git-128) In-Reply-To: References: MIME-Version: 1.0 X-Rspamd-Server: rspam09 X-Rspamd-Queue-Id: 41FC740449 X-Stat-Signature: 41a5oh66ucdq6ngn578fpcou7h1tc17z X-Rspam-User: X-HE-Tag: 1731567612-825255 X-HE-Meta: U2FsdGVkX18ovusbkt5ZoXzbJ42GVP6oEyXz7nQbxmmQxFTWZLZDh48c3FB87Avm9/wJBs60eb6x46QughW8b1/o7S2ImE6qXkNV79zDXVqKxtzEv6yBMjs/c/0udqsVm1clUdicU8vlbPnMMYTBqk8ODC6zsgcU+zrC5VCQUjcUEgieg/pQj5HWsavY6wpdA/wCh9Yem5fOEO9KA/MlG4U89ub6w6zFRAWs3kWYlhaMASZKkjRiyzjK8UCuBvt6Ckm3M6ZNRihxxquSF042UXI1UOjF2TtlagUN/CARfERc95sO3GsWMHMhdKJINY+txO0ddgL459jWO2YS/nvNH1hQmQp5P4fk7hT6q4YQyPibxwPoUNzmWOHvWCkP0ycSJ99wE4grzsRX9PxQNbgTLurmEK4XnmPynEk9VjWH4McxbBV3nJCpXmcEEdqoCUwV+Szp+c1KgqnRvtLoMF70ZxA/4VqfO8SUQGyW/sqf6EJGloMcvqlldjpKjZiDY2bXav4Ebdj19fPhHW74WfxqRUvo0apkQ0PxjuIG6DRp0nyLnEyY5DK/1opBt9M76jVUxuHpTo9owCq9NgD0S6g6LzDpor8rLvK1zfyOzub0nbDi7jgkdAdkUJD7pywFFgL/eYCXPdMz4aA/Jz02Bkpovdqcmg7d76U3iqB0tK9+t953OCrFttE3f3cWXq1VCChAmGUf6mcIjF6MtZgVZPYCf1A8P+CNXPydF+WQUh++2qNPNFuWQD422YCKQbSRAa2tjresJlioxkmtel3WCf+s481csbNHvlO9U3dtU2267NuZctHVKyPs8xC9zCAeBQpxnowOFHUjL7mDUN7rEurOrhlMJObkTN9LhBMKGCl+k+R+ZHYbbs690+sWFv2dpoP8u8+T+v8L9UKo0lcjRVYjOuoKzNN1QPDmONeY6U4rVcVNB5bQM1pHcoHLjn3K9A326d7GrVJguhSdOsZ8IMT gterx7Rt fDHmzNTuNFvOMcBro1qNCsXjDe0qEm11XHn5HXAgRSAzHzqlJQDzWgN3XbzjFKpOOpUAoKUBNnK55xh8RFzgqSZUn+VzTGjdYyz4edKSrQzlnZuQL7sDaeoTPcX3oljnnIsYl9fnowH1DIatgdAEfLJuPpEeZZ05d7fbHfsEShTXllhvIqcRYn1qgsXFrECwwUrrf+B4pOE2vgUzCzF6tkO7VbhvFVEGhOr/5mJJSrZ64UuRx+GNEOd4ssdOHhwVQujIumf9vnHO99xXc+q4B2nuxazt4PMqgM1jm85IRtaS35+hw432BKxCHvQMCB9pKsXL4H3GQV9s6ztx3Y83pLNLf3vLSePhNz85TfkpLrqXYByj/wZYezRxMmcCBh3FHosPBcfbutP1BvxtWSbioq5itAg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: This commit introduces do_zap_pte_range() to actually zap the PTEs, which will help improve code readability and facilitate secondary checking of the processed PTEs in the future. No functional change. Signed-off-by: Qi Zheng Reviewed-by: Jann Horn Acked-by: David Hildenbrand --- mm/memory.c | 39 ++++++++++++++++++++++++--------------- 1 file changed, 24 insertions(+), 15 deletions(-) diff --git a/mm/memory.c b/mm/memory.c index 24633d0e1445a..bf5ac8e0b4656 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -1679,6 +1679,25 @@ static inline int skip_none_ptes(pte_t *pte, unsigned long addr, return nr; } +/* If PTE_MARKER_UFFD_WP is enabled, the uffd-wp PTEs may be re-installed. */ +static inline int do_zap_pte_range(struct mmu_gather *tlb, + struct vm_area_struct *vma, pte_t *pte, + unsigned long addr, unsigned long end, + struct zap_details *details, int *rss, + bool *force_flush, bool *force_break) +{ + pte_t ptent = ptep_get(pte); + int max_nr = (end - addr) / PAGE_SIZE; + + if (pte_present(ptent)) + return zap_present_ptes(tlb, vma, pte, ptent, max_nr, + addr, details, rss, force_flush, + force_break); + + return zap_nonpresent_ptes(tlb, vma, pte, ptent, max_nr, addr, + details, rss); +} + static unsigned long zap_pte_range(struct mmu_gather *tlb, struct vm_area_struct *vma, pmd_t *pmd, unsigned long addr, unsigned long end, @@ -1701,9 +1720,6 @@ static unsigned long zap_pte_range(struct mmu_gather *tlb, flush_tlb_batched_pending(mm); arch_enter_lazy_mmu_mode(); do { - pte_t ptent = ptep_get(pte); - int max_nr; - if (need_resched()) break; @@ -1715,18 +1731,11 @@ static unsigned long zap_pte_range(struct mmu_gather *tlb, pte += nr; } - max_nr = (end - addr) / PAGE_SIZE; - if (pte_present(ptent)) { - nr = zap_present_ptes(tlb, vma, pte, ptent, max_nr, - addr, details, rss, &force_flush, - &force_break); - if (unlikely(force_break)) { - addr += nr * PAGE_SIZE; - break; - } - } else { - nr = zap_nonpresent_ptes(tlb, vma, pte, ptent, max_nr, - addr, details, rss); + nr = do_zap_pte_range(tlb, vma, pte, addr, end, details, + rss, &force_flush, &force_break); + if (unlikely(force_break)) { + addr += nr * PAGE_SIZE; + break; } } while (pte += nr, addr += PAGE_SIZE * nr, addr != end);