From patchwork Mon Jul 18 12:02:04 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nadav Amit X-Patchwork-Id: 12921684 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3954CC43334 for ; Mon, 18 Jul 2022 19:37:15 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D52026B0078; Mon, 18 Jul 2022 15:37:14 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id CD89E6B007B; Mon, 18 Jul 2022 15:37:14 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B50FD8E0001; Mon, 18 Jul 2022 15:37:14 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id A2EEE6B0078 for ; Mon, 18 Jul 2022 15:37:14 -0400 (EDT) Received: from smtpin30.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 67AEC9E8 for ; Mon, 18 Jul 2022 19:37:14 +0000 (UTC) X-FDA: 79701229188.30.39FCCBD Received: from relay4.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by imf30.hostedemail.com (Postfix) with ESMTP id 0135280088 for ; Mon, 18 Jul 2022 19:37:13 +0000 (UTC) Received: from smtpin30.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay11.hostedemail.com (Postfix) with ESMTP id B05C68015F for ; Mon, 18 Jul 2022 19:37:13 +0000 (UTC) X-FDA: 79701229146.30.6F670F0 Received: from mail-pg1-f178.google.com (mail-pg1-f178.google.com [209.85.215.178]) by imf02.hostedemail.com (Postfix) with ESMTP id 3F2F480048 for ; Mon, 18 Jul 2022 19:37:13 +0000 (UTC) Received: by mail-pg1-f178.google.com with SMTP id 72so11548812pge.0 for ; Mon, 18 Jul 2022 12:37:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=VCDxDoUSCrnTZdtdcyuvlf+NNtVJJIcoOQ0MDquBzAY=; b=WpzOa1eCI6gKdwOYRlP/7uAcBkP9V49tEy6TRy1l2Khiu+GZnm10Q5jyVbyYM1tYJ0 r1e+kLptD6I/s6ViDo6K3ZWQxtQqTivN9316p9KbowhoCjz6+CLLmK18EEI8JLQoLMTw 6hJP+pqwUio5gv5yYp+Q5f9oq+Nlk4vL7U3Nr2uzyliGndyYJu/aty4nLWwUOlzgE7f3 XKi59Yz5xUZS3IH7VwQS7RSyx2mkvafbnQ3f/hs9aloww2hNMJj8gDsL4ymdSB9RE3Xb MzJM2PoJh3hUc1G7Ka+jmv39doe16fxqrhkTV1xozTVG+hkSSQ0CDqVkr23SSAAypVrA ptew== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=VCDxDoUSCrnTZdtdcyuvlf+NNtVJJIcoOQ0MDquBzAY=; b=JJX8wuNuVvEFQEzJWK24LBosMYqf5eYzSTOooNnh1byb/0WoTk3PASShZE3qYuAndo S+/fQe+svwmG1kLoyKV6vmWsvhcHXpbAz431v8STM03EvICyUuNTLKpHqYEPLMRHqBKK Nbg5AB0HcJuVXIaAIpYRd6jxhJm/2ez0oWtqZkZrJLEDvuhzVcuYtKNeJC/pcJdDSyNU 9i9vRWw/nGEVE38gtuH7zvZEr8AdeJVSflkYUfVjyLw1iS6rbnOv8zlCXgDiCisHj0eL PTh1pqHOs5k665SLOm1p/kL4t0pEuYUK6Z1DdwiGdY4npKpicyEbKV3ZyqeptzCaJj7k m3gA== X-Gm-Message-State: AJIora9SG1qsCGKAbyzrv7aVtoUa0jnGdkXItg/rDB+mfcmNRgVG9g9x N3uJ/4K/VC9gLC1NxtFmpD/T3ibdfGv1vg== X-Google-Smtp-Source: AGRyM1vGJivctvX5ZkdLIJ4O9MXoDMilDP3XL2pp+bNJOG2Zy6lN3VLUtfwzhoCDKo97VJCK/R1iTA== X-Received: by 2002:a63:fc48:0:b0:40d:ad0a:a868 with SMTP id r8-20020a63fc48000000b0040dad0aa868mr25533373pgk.204.1658173031839; Mon, 18 Jul 2022 12:37:11 -0700 (PDT) Received: from sc2-haas01-esx0118.eng.vmware.com ([66.170.99.1]) by smtp.gmail.com with ESMTPSA id q6-20020a170902a3c600b0016bc4a6ce28sm9907887plb.98.2022.07.18.12.37.10 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 18 Jul 2022 12:37:11 -0700 (PDT) From: Nadav Amit X-Google-Original-From: Nadav Amit To: linux-mm@kvack.org Cc: linux-kernel@vger.kernel.org, Andrew Morton , Mike Rapoport , Axel Rasmussen , Nadav Amit , Andrea Arcangeli , Andrew Cooper , Andy Lutomirski , Dave Hansen , David Hildenbrand , Peter Xu , Peter Zijlstra , Thomas Gleixner , Will Deacon , Yu Zhao , Nick Piggin Subject: [RFC PATCH 06/14] mm/rmap: avoid flushing on page_vma_mkclean_one() when possible Date: Mon, 18 Jul 2022 05:02:04 -0700 Message-Id: <20220718120212.3180-7-namit@vmware.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220718120212.3180-1-namit@vmware.com> References: <20220718120212.3180-1-namit@vmware.com> MIME-Version: 1.0 ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1658173034; a=rsa-sha256; cv=none; b=42joCuQNU6DadOs1uQy0TWio77iMse9jrMEcIrx2f0B5b/09XDJn3kl4x5KhSlafw4qXHG 4ZgRPLhervFqm7bg2D5FO23Z/Kq8mKu3U50FaIi+w7Itrei6/AgTeVXEz1qUR3na0ZgwhC hdPEVWVaEm7c5bcrft7vFn31fm4P2v0= ARC-Authentication-Results: i=1; imf30.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=WpzOa1eC; dmarc=pass (policy=none) header.from=gmail.com; spf=none (imf30.hostedemail.com: domain of MAILER-DAEMON@hostedemail.com has no SPF policy when checking 216.40.44.17) smtp.mailfrom=MAILER-DAEMON@hostedemail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1658173034; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=VCDxDoUSCrnTZdtdcyuvlf+NNtVJJIcoOQ0MDquBzAY=; b=5oMKLUMq4HkmhXyF3rpxOj52u6E1LHb7DB0f9MX0dLHq8IoGYfhL8jCBDWG8jCIvY/Ydc8 4pS0nShvOmpc27mfNGH5Fvl0UrOYFY2y5FMoSpLRXG3sN6GEfxm6Y1xnQ5y05/yqnZhQbS So9cxJ8xSYSjb6qtU0vRuOD8WDUpeKc= X-Rspamd-Queue-Id: 0135280088 X-Rspam-User: X-HE-Tag-Orig: 1658173033-655444 Authentication-Results: imf30.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=WpzOa1eC; dmarc=pass (policy=none) header.from=gmail.com; spf=none (imf30.hostedemail.com: domain of MAILER-DAEMON@hostedemail.com has no SPF policy when checking 216.40.44.17) smtp.mailfrom=MAILER-DAEMON@hostedemail.com X-Rspamd-Server: rspam08 X-Stat-Signature: 1zhztpqsuretrmaen4g65tj6imd9r1p4 X-HE-Tag: 1658173033-875377 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Nadav Amit x86 is capable to avoid TLB flush on clean writable entries. page_vma_mkclean_one() does not take advantage of this behavior. Adapt it. Cc: Andrea Arcangeli Cc: Andrew Cooper Cc: Andrew Morton Cc: Andy Lutomirski Cc: Dave Hansen Cc: David Hildenbrand Cc: Peter Xu Cc: Peter Zijlstra Cc: Thomas Gleixner Cc: Will Deacon Cc: Yu Zhao Cc: Nick Piggin Signed-off-by: Nadav Amit --- mm/rmap.c | 16 ++++++++++++---- 1 file changed, 12 insertions(+), 4 deletions(-) diff --git a/mm/rmap.c b/mm/rmap.c index 83172ee0ea35..23997c387858 100644 --- a/mm/rmap.c +++ b/mm/rmap.c @@ -961,17 +961,25 @@ static int page_vma_mkclean_one(struct page_vma_mapped_walk *pvmw) address = pvmw->address; if (pvmw->pte) { - pte_t entry; + pte_t entry, oldpte; pte_t *pte = pvmw->pte; if (!pte_dirty(*pte) && !pte_write(*pte)) continue; flush_cache_page(vma, address, pte_pfn(*pte)); - entry = ptep_clear_flush(vma, address, pte); - entry = pte_wrprotect(entry); + oldpte = ptep_modify_prot_start(pvmw->vma, address, + pte); + + entry = pte_wrprotect(oldpte); entry = pte_mkclean(entry); - set_pte_at(vma->vm_mm, address, pte, entry); + + if (pte_needs_flush(oldpte, entry) || + mm_tlb_flush_pending(vma->vm_mm)) + flush_tlb_page(vma, address); + + ptep_modify_prot_commit(vma, address, pte, oldpte, + entry); ret = 1; } else { #ifdef CONFIG_TRANSPARENT_HUGEPAGE