From patchwork Thu Feb 17 06:07:45 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Xu X-Patchwork-Id: 12749421 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 67895C433F5 for ; Thu, 17 Feb 2022 06:08:15 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id F0FEB6B007B; Thu, 17 Feb 2022 01:08:14 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id E985A6B007D; Thu, 17 Feb 2022 01:08:14 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D12E96B007E; Thu, 17 Feb 2022 01:08:14 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0222.hostedemail.com [216.40.44.222]) by kanga.kvack.org (Postfix) with ESMTP id C09046B007B for ; Thu, 17 Feb 2022 01:08:14 -0500 (EST) Received: from smtpin12.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id 83409181AC9C6 for ; Thu, 17 Feb 2022 06:08:14 +0000 (UTC) X-FDA: 79151241708.12.BE42C86 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by imf26.hostedemail.com (Postfix) with ESMTP id 0608F140003 for ; Thu, 17 Feb 2022 06:08:13 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1645078093; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=A1vbZxsyzMifaaJqgkhHlizU7SHJnrILT5bLrmNnzMQ=; b=UF3otFFdlgED7oMXBGZHUacxQsvcyU15vFgTclZxvD8KhqpovrLqxowtUkt3LjzOvtcTMe WaA+oE035x+VA79LDH5E02m8Pkf7h2UwAzpnY1hqsvjf1t9OIpnJpvccqmS5E1y0iOGz2V Lj/jomIXCARDsErcL02s3/aQGIoPj0E= Received: from mail-pj1-f72.google.com (mail-pj1-f72.google.com [209.85.216.72]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-448-hB8rWLKKPlum9-3xlzb0RA-1; Thu, 17 Feb 2022 01:08:12 -0500 X-MC-Unique: hB8rWLKKPlum9-3xlzb0RA-1 Received: by mail-pj1-f72.google.com with SMTP id c6-20020a17090a020600b001b9b44c18aaso5840959pjc.4 for ; Wed, 16 Feb 2022 22:08:12 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=A1vbZxsyzMifaaJqgkhHlizU7SHJnrILT5bLrmNnzMQ=; b=pbbGxiv+w7Edae5lzWApha/SHC2wcEnIfBdozWmGjY5cceq/fiVfpWpLHwgQqV6oWo s6QWnhJgBloCxwOkXgsKcy1Ss39Ph4Sx27yEiDDiuE/QR5Mdikh+ncNIxetUc5VVR/pb wJsWz15eRl+249u7p36fghudS63n1PMdFX+rS4v4MkyjuEoY3QnKINQGn3jsIkyJ0+yc EtyviMYnppV/uX/7AUFJmw3PQKyhDO3+Sm8rFRJrN5DqFFvyrVQy/bc1dRVjIfwZ4nVM qsALbe5hzs0RZHMUKyzo2O1gYaX3NeR5Tv+4rXVbwjelSEy5sj90BIy0RyzIgPuqTFld jnvg== X-Gm-Message-State: AOAM531VvNKEWTDk9p3TEiU0+zXX4WzlM5hygqg8UR8sAvz95qvSzcI5 h+opV5V6EPoKSM6WtLAdgv81vYFb0oKmFRsLHgBv/0/zyJPp9dCdOCnCddhV/hS/qUOWMeDShsx Rmh9ksKyLbl7fRtfR7NPlOhkFBn+cmP91oUWWJ18hqcQK65MpyFZbEWK9nn2z X-Received: by 2002:a17:90a:ec10:b0:1b8:9da7:3d13 with SMTP id l16-20020a17090aec1000b001b89da73d13mr1531885pjy.194.1645078091273; Wed, 16 Feb 2022 22:08:11 -0800 (PST) X-Google-Smtp-Source: ABdhPJxJ+MRoSejjpjSfq3CLN/Ow3BCwyndgoRTeQRVbvX65z98rR9y1roji3xMcenHn/prKAwWNxg== X-Received: by 2002:a17:90a:ec10:b0:1b8:9da7:3d13 with SMTP id l16-20020a17090aec1000b001b89da73d13mr1531852pjy.194.1645078090846; Wed, 16 Feb 2022 22:08:10 -0800 (PST) Received: from localhost.localdomain ([94.177.118.126]) by smtp.gmail.com with ESMTPSA id j8sm224230pjc.11.2022.02.16.22.08.06 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Wed, 16 Feb 2022 22:08:10 -0800 (PST) From: Peter Xu To: linux-mm@kvack.org, linux-kernel@vger.kernel.org Cc: Alistair Popple , Matthew Wilcox , peterx@redhat.com, David Hildenbrand , Andrea Arcangeli , Hugh Dickins , Yang Shi , Vlastimil Babka , John Hubbard , Andrew Morton , "Kirill A . Shutemov" Subject: [PATCH v5 3/4] mm: Change zap_details.zap_mapping into even_cows Date: Thu, 17 Feb 2022 14:07:45 +0800 Message-Id: <20220217060746.71256-4-peterx@redhat.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20220217060746.71256-1-peterx@redhat.com> References: <20220217060746.71256-1-peterx@redhat.com> MIME-Version: 1.0 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-Rspamd-Server: rspam09 X-Rspamd-Queue-Id: 0608F140003 X-Rspam-User: Authentication-Results: imf26.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=UF3otFFd; spf=none (imf26.hostedemail.com: domain of peterx@redhat.com has no SPF policy when checking 170.10.133.124) smtp.mailfrom=peterx@redhat.com; dmarc=pass (policy=none) header.from=redhat.com X-Stat-Signature: wzatub9u8bedu6i3rpxffrgu3e39eyzz X-HE-Tag: 1645078093-658189 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Currently we have a zap_mapping pointer maintained in zap_details, when it is specified we only want to zap the pages that has the same mapping with what the caller has specified. But what we want to do is actually simpler: we want to skip zapping private (COW-ed) pages in some cases. It comes from unmap_mapping_pages() where we could have passed in different even_cows values. The other user is unmap_mapping_folio() where we always want to skip private pages. According to Hugh, we used a mapping pointer for historical reason, as explained here: https://lore.kernel.org/lkml/391aa58d-ce84-9d4-d68d-d98a9c533255@google.com/ Quoting partly from Hugh: Which raises the question again of why I did not just use a boolean flag there originally: aah, I think I've found why. In those days there was a horrible "optimization", for better performance on some benchmark I guess, which when you read from /dev/zero into a private mapping, would map the zero page there (look up read_zero_pagealigned() and zeromap_page_range() if you dare). So there was another category of page to be skipped along with the anon COWs, and I didn't want multiple tests in the zap loop, so checking check_mapping against page->mapping did both. I think nowadays you could do it by checking for PageAnon page (or genuine swap entry) instead. This patch replaced the zap_details.zap_mapping pointer into the even_cows boolean, then we check it against PageAnon. Suggested-by: Hugh Dickins Reviewed-by: John Hubbard Signed-off-by: Peter Xu --- mm/memory.c | 16 +++++++--------- 1 file changed, 7 insertions(+), 9 deletions(-) diff --git a/mm/memory.c b/mm/memory.c index 3728632ea993..c2defe8a1472 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -1309,8 +1309,8 @@ copy_page_range(struct vm_area_struct *dst_vma, struct vm_area_struct *src_vma) * Parameter block passed down to zap_pte_range in exceptional cases. */ struct zap_details { - struct address_space *zap_mapping; /* Check page->mapping if set */ struct folio *single_folio; /* Locked folio to be unmapped */ + bool even_cows; /* Zap COWed private pages too? */ }; /* Whether we should zap all COWed (private) pages too */ @@ -1321,13 +1321,10 @@ static inline bool should_zap_cows(struct zap_details *details) return true; /* Or, we zap COWed pages only if the caller wants to */ - return !details->zap_mapping; + return details->even_cows; } -/* - * We set details->zap_mapping when we want to unmap shared but keep private - * pages. Return true if we should zap this page, false otherwise. - */ +/* Decides whether we should zap this page with the page pointer specified */ static inline bool should_zap_page(struct zap_details *details, struct page *page) { /* If we can make a decision without *page.. */ @@ -1338,7 +1335,8 @@ static inline bool should_zap_page(struct zap_details *details, struct page *pag if (!page) return true; - return details->zap_mapping == page_rmapping(page); + /* Otherwise we should only zap non-anon pages */ + return !PageAnon(page); } static unsigned long zap_pte_range(struct mmu_gather *tlb, @@ -3398,7 +3396,7 @@ void unmap_mapping_folio(struct folio *folio) first_index = folio->index; last_index = folio->index + folio_nr_pages(folio) - 1; - details.zap_mapping = mapping; + details.even_cows = false; details.single_folio = folio; i_mmap_lock_write(mapping); @@ -3427,7 +3425,7 @@ void unmap_mapping_pages(struct address_space *mapping, pgoff_t start, pgoff_t first_index = start; pgoff_t last_index = start + nr - 1; - details.zap_mapping = even_cows ? NULL : mapping; + details.even_cows = even_cows; if (last_index < first_index) last_index = ULONG_MAX;