From patchwork Wed Dec 4 11:09:44 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Qi Zheng X-Patchwork-Id: 13893585 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8AC02E7716D for ; Wed, 4 Dec 2024 11:10:53 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 1EC316B0095; Wed, 4 Dec 2024 06:10:53 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 19C716B0098; Wed, 4 Dec 2024 06:10:53 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 03C196B0099; Wed, 4 Dec 2024 06:10:52 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id CCA306B0095 for ; Wed, 4 Dec 2024 06:10:52 -0500 (EST) Received: from smtpin24.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 06EAE1A0F2A for ; Wed, 4 Dec 2024 11:10:52 +0000 (UTC) X-FDA: 82857008724.24.2DCBDA0 Received: from mail-pl1-f177.google.com (mail-pl1-f177.google.com [209.85.214.177]) by imf03.hostedemail.com (Postfix) with ESMTP id 7D2CA2000B for ; Wed, 4 Dec 2024 11:10:43 +0000 (UTC) Authentication-Results: imf03.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=Tu05LzB9; spf=pass (imf03.hostedemail.com: domain of zhengqi.arch@bytedance.com designates 209.85.214.177 as permitted sender) smtp.mailfrom=zhengqi.arch@bytedance.com; dmarc=pass (policy=quarantine) header.from=bytedance.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1733310640; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=J2X5SMV19Fsn2OjJZllhusee1Lj+v0wZR5QDlA7SZpw=; b=T3NBkteLetKIGDkmnL29z1eKHHd/R6r1iiYQvTuqw6yMaes4i6VC9i2Y56SnbATdlNHwLg sbRU4vQkgaREuXdoayh/PWk4ZhPFQV/78mz97LiyyoCBYVgdqE5k+wAy9YYhtZEDp7ChbP 040uvNV1O3lhrgGFqm2W3mtjqIFBzIo= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1733310640; a=rsa-sha256; cv=none; b=0BTGYBFl25/fL0/ao7s8/yY14DV8F8cDkBnA3D+TT7FdGWky0XCvjGUzeGlWwhithPbBwf vjqFvmNhfClWoO+R2PznQ5/R60MhcNKgXcBghOtVe/g/y75dW8OuCPQxwljOGidb/+e2Lo J2YpRhrjsI5+7ZhdkgRwW7VeaknhlEk= ARC-Authentication-Results: i=1; imf03.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=Tu05LzB9; spf=pass (imf03.hostedemail.com: domain of zhengqi.arch@bytedance.com designates 209.85.214.177 as permitted sender) smtp.mailfrom=zhengqi.arch@bytedance.com; dmarc=pass (policy=quarantine) header.from=bytedance.com Received: by mail-pl1-f177.google.com with SMTP id d9443c01a7336-215ac560292so26299215ad.2 for ; Wed, 04 Dec 2024 03:10:49 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1733310649; x=1733915449; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=J2X5SMV19Fsn2OjJZllhusee1Lj+v0wZR5QDlA7SZpw=; b=Tu05LzB9rc7/sf1zybd0NF2z9heTcpJv5mNJC2cSVnnGhKGqz05LEDSryt8OekUdhk GVC5CMVbSSagVnfTAEIF7Emn99lwSlvNCgdUAeCLrrvKTNq+4eXvEpAFsdrU3+FjBkjL IbW9W3vAtQ1TtCq0Wci5Jm0EO+gaZR5UpRNawVqGfdrk1xn8r9Ap5FhbVHuPkOxcZVbp Y8drbHjlkq+0I+gtobhGRsDeQ6HGd0osjQW1RSpYKTSwuFw0rkRoUrGAkdz38i33N/2t VUcEQAX+RXz1Rcg0Kw1hnP9wj4EkPoR0Tb0XSLJMd8EAIF+Zqyk8HoF9BYUQqTTFgBFR rr1A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1733310649; x=1733915449; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=J2X5SMV19Fsn2OjJZllhusee1Lj+v0wZR5QDlA7SZpw=; b=hwHQt35UZz4d04twGiyQmJEynCEw0cTmQYyi4mj84bwVG1MZYrPGqYapFfOWfkwAIM IMIl3KSa9AKqpUKP20/JRtfn5TARt6aDasLlU6NVDxkR5jgTEG5hAjspWojJKdVijZsr Jhj1fbN5tqRHowQ80Vpshxt67lpwIBVFUy4C3/K6QmRJrJVrTHobjJeP1wheWaXVJMve xzVgGEV67CEozllGk0tMhu0kId6p1MTCam0KGXfLoytTwgFwNv7ezPmQfxoX2J/M+JnV t0PPqqOoAJzaiWa9b6Ar5wV4+hmiuL7I1QzEElN+A/X1ynVqarVL9ORR+f/aep2HdWvD MDkQ== X-Forwarded-Encrypted: i=1; AJvYcCV2gz/DBUxSX2JptYcXum/MHHyP8D1iWeavIePLrmpvJdAEydZwx9cbqRrAeD5oOO/bTqqR9Cnoag==@kvack.org X-Gm-Message-State: AOJu0Yy5vqbvBv0WYOH/5RnP3/VttZGT3SWdxTnKSdgWNJ8gzQKDQg7U D26YcQjIiiXNvVHhHMnL1UZ2eGyLcs3gQB8SDf4H2t0ZJ4KxIhkt4PynihdyTCYWRtVuRvRl0Sf X X-Gm-Gg: ASbGncsLiIJI2Ab/u27jThzB7fBABOR3eymGD2wvRRCPoqvl6JXgZeNQtY3afgEs3DY oNw6eLrNiSEXF5GvbBTiOoTsHdPm835xxTJ9W0LLWAw9aJvGKu58f16HmlKS2J6jaXFfwtBVJki F4IXTN0uxGlCU/Ahg7XzNRWlHrIAnVCzHRhhRgpdbuwBoAiFuujDswq7DztrvsDsyXEENEvzMIz PCB6Udy5S6IfglaMMVkiYPfWo7pmytXC5XbIRXwJdt3uYzfiJxVnzQyIUthWJtZVwLRbPawxspq 6A/to3ECmLPo7mA= X-Google-Smtp-Source: AGHT+IH7KMftfdbT7IJ/SYECv1Z0b6kIRJQ0pshXMsGiPcHLBFGh7gTwpZlD1z3c4LTYAe+5oKH33A== X-Received: by 2002:a17:902:d2c7:b0:215:b18d:e1 with SMTP id d9443c01a7336-215bcfc2ab1mr102416785ad.24.1733310648745; Wed, 04 Dec 2024 03:10:48 -0800 (PST) Received: from C02DW0BEMD6R.bytedance.net ([203.208.167.148]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-21527515731sm107447495ad.192.2024.12.04.03.10.40 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 04 Dec 2024 03:10:47 -0800 (PST) From: Qi Zheng To: david@redhat.com, jannh@google.com, hughd@google.com, willy@infradead.org, muchun.song@linux.dev, vbabka@kernel.org, peterx@redhat.com, akpm@linux-foundation.org Cc: mgorman@suse.de, catalin.marinas@arm.com, will@kernel.org, dave.hansen@linux.intel.com, luto@kernel.org, peterz@infradead.org, x86@kernel.org, lorenzo.stoakes@oracle.com, zokeefe@google.com, rientjes@google.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Qi Zheng Subject: [PATCH v4 04/11] mm: introduce do_zap_pte_range() Date: Wed, 4 Dec 2024 19:09:44 +0800 Message-Id: X-Mailer: git-send-email 2.24.3 (Apple Git-128) In-Reply-To: References: MIME-Version: 1.0 X-Rspamd-Server: rspam02 X-Rspamd-Queue-Id: 7D2CA2000B X-Stat-Signature: kcyt9uckyqryc75ya43mjc78mqqexnna X-Rspam-User: X-HE-Tag: 1733310643-681152 X-HE-Meta: U2FsdGVkX1/Wv+hnzCOe2P7EhdJZ418f44wmdDHVswA2tf5aelN2d8lWrLqR1yFPGT9e/v+jDVZEhVyH5T44CrVCnGUKG3MdExKtNeVlzQ6dBVYIsl8dpviPznSkrjQ1+upOdfzsR/dF5cN0GNVRAuuk92EJi02bQx2ea+DywKRUwxyHaUEhnHmNPY2kVxZON4g9fzWA/jEmqaVAeFq2P9n09EfR7n7YOl2jlQBecvBDRoXqx6hcuaar+TxpBSAGnmcHXKzNB6n5A/RWPMOWCQS3ez/9mqAZLUwkStpRCKXHSdz4hiUzt+5EvHvb5MJNf2WY17FV+nYy/F3AItSqj5CmToCjEl//6jUTROhnr5ryEh/tWQL3rZG5VVW7SHj6UBN1+JFpAaL2gQR4ZQebqGVZpeS6qFpuELBjCmqQx0DspsAzKA3Nx4z8MSYxc02y8wCnDkS5ROvFXWk7cMzoY1jNWcw2x6gsHSqqqjWC4E9BJQhz2R6FaH9wGkPXUOzxN2dWtD1oM3Wsb8ZZfRwCoH2yKd5StS8F5WyRqAiyPGf7nwKSjhG4fvYmS+o0zVJf1BTrbPxPG4huISVOKnS5MvMCZ1bDu4crVKGzcB9gzHpN2MUlLaG3Z1NAUUPVu6nJp8kks0IHvMa2c9EU16umDhex/+KUQ+JXLcMkqneZgI+65hYlcFRqsxPdkJ6217Veq4Icv/URczsmJ3xhScbz7wHOu+hps6XVBtdazHdX2Sz+hqCORinXmEKT8la3lBono4ZyeeU6o+4cKQWbSYWNKhCDw4okY2S5OwvB49/hXWSsH6M9rPDwVnowi0VO+MkfE1jLlGAlyAO3kgzDBdopUKUKdsMEK7O4VY2cPx8BABSAn1lJdMHeFFUTvSBn1SWiy2NljxIhxk8z1DePoR+uc3VbXrZgdBKbLnETH8O4DoJBgfykg9KFQfRaobnPQqOaskdzGqRuapndVDpCQwv waw5zQVR P/4NCx/tOQ/rl654BrrhcwOSEEmU9wCJiFwDwBtaYj7by9xPgiksAZnHredq9YuWmDo11k/yat9DhRab9zVbtRlGcwKyWHUs5Rm5RosvG1CQ83UmBbVwJxgNFoGi2/Vw6XIihtappgL8D/J+kD0/WxYglA7ap8jENkVRhCi02Gc7n8+fAJoEBly0j/vLUW2Z6+hU3W1qaTWwilGSXBPy+Jz5DNpAEgEgNEk34HjcIDg4cWZEj9F+S1x471ccdfyvxUm7+9twvQT2DyXEyGsjOQja8ZLZvMB2lasXQ0s6twx516FnSKgvGs6m/h86eYJIr9XPzcjL8zD34g4hqyw7yIGNCb/4dj5mWGFtJ0kSn4DjXJ/lENzGIOmMSasHEXf8cdZlF8razTEyBH4ABIzweI/wPYA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: This commit introduces do_zap_pte_range() to actually zap the PTEs, which will help improve code readability and facilitate secondary checking of the processed PTEs in the future. No functional change. Signed-off-by: Qi Zheng Reviewed-by: Jann Horn Acked-by: David Hildenbrand --- mm/memory.c | 45 ++++++++++++++++++++++++++------------------- 1 file changed, 26 insertions(+), 19 deletions(-) diff --git a/mm/memory.c b/mm/memory.c index 5624c22bb03cf..abe07e6bdd1bb 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -1657,6 +1657,27 @@ static inline int zap_nonpresent_ptes(struct mmu_gather *tlb, return nr; } +static inline int do_zap_pte_range(struct mmu_gather *tlb, + struct vm_area_struct *vma, pte_t *pte, + unsigned long addr, unsigned long end, + struct zap_details *details, int *rss, + bool *force_flush, bool *force_break) +{ + pte_t ptent = ptep_get(pte); + int max_nr = (end - addr) / PAGE_SIZE; + + if (pte_none(ptent)) + return 1; + + if (pte_present(ptent)) + return zap_present_ptes(tlb, vma, pte, ptent, max_nr, + addr, details, rss, force_flush, + force_break); + + return zap_nonpresent_ptes(tlb, vma, pte, ptent, max_nr, addr, + details, rss); +} + static unsigned long zap_pte_range(struct mmu_gather *tlb, struct vm_area_struct *vma, pmd_t *pmd, unsigned long addr, unsigned long end, @@ -1679,28 +1700,14 @@ static unsigned long zap_pte_range(struct mmu_gather *tlb, flush_tlb_batched_pending(mm); arch_enter_lazy_mmu_mode(); do { - pte_t ptent = ptep_get(pte); - int max_nr; - - nr = 1; - if (pte_none(ptent)) - continue; - if (need_resched()) break; - max_nr = (end - addr) / PAGE_SIZE; - if (pte_present(ptent)) { - nr = zap_present_ptes(tlb, vma, pte, ptent, max_nr, - addr, details, rss, &force_flush, - &force_break); - if (unlikely(force_break)) { - addr += nr * PAGE_SIZE; - break; - } - } else { - nr = zap_nonpresent_ptes(tlb, vma, pte, ptent, max_nr, - addr, details, rss); + nr = do_zap_pte_range(tlb, vma, pte, addr, end, details, rss, + &force_flush, &force_break); + if (unlikely(force_break)) { + addr += nr * PAGE_SIZE; + break; } } while (pte += nr, addr += PAGE_SIZE * nr, addr != end);