From patchwork Wed Sep 8 16:36:28 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Xu X-Patchwork-Id: 12481531 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id D6985C433EF for ; Wed, 8 Sep 2021 16:36:34 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 58A4461139 for ; Wed, 8 Sep 2021 16:36:34 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 58A4461139 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=kvack.org Received: by kanga.kvack.org (Postfix) id 000CE6B0073; Wed, 8 Sep 2021 12:36:34 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id EA6B26B0074; Wed, 8 Sep 2021 12:36:33 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C3409900002; Wed, 8 Sep 2021 12:36:33 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0122.hostedemail.com [216.40.44.122]) by kanga.kvack.org (Postfix) with ESMTP id B1D926B0073 for ; Wed, 8 Sep 2021 12:36:33 -0400 (EDT) Received: from smtpin37.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id 6DB16181A0D97 for ; Wed, 8 Sep 2021 16:36:33 +0000 (UTC) X-FDA: 78564959466.37.ED5D302 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by imf27.hostedemail.com (Postfix) with ESMTP id 1633D700009B for ; Wed, 8 Sep 2021 16:36:32 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1631118992; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=awAUVK0W17m50EUHlrWKEaUEucIpwyDPaQz4TREBEy8=; b=gIdAQ6ANwrVwAX/izpXWx9pTlAHuHZCWFKtTwgi5CbjcITOX9esn9f1BGom9/zXGuR5xuy JGXzQ/+URzrPLH8w71qvzbxPX1H0yHg/BXgM83XqqfHiJv2UD8SmoiXS1l2yAQLgn0GjU7 PeoOFE7Etp8XuOhQfiVDeH4qOeUkGU8= Received: from mail-io1-f72.google.com (mail-io1-f72.google.com [209.85.166.72]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-584-Uc82gApgMw-5jsIQs2R_dg-1; Wed, 08 Sep 2021 12:36:31 -0400 X-MC-Unique: Uc82gApgMw-5jsIQs2R_dg-1 Received: by mail-io1-f72.google.com with SMTP id x26-20020a6bd01a000000b005c23c703629so2381124ioa.10 for ; Wed, 08 Sep 2021 09:36:31 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=awAUVK0W17m50EUHlrWKEaUEucIpwyDPaQz4TREBEy8=; b=1GPfJl/rsDn265d8Uku2DkMbMHOPe0yJw6FdxwY/OxKpS8Tq5tAsKijCHqzjRUpHiK 7LnE7AJ9sWSI6HNIsGGfEJeFF6vWmtN4OYZ5Pj4tA0LqWtoYABCDoNNsh3JFTZvqxI9E P1WlB1T33N2+a/1I9wAcrbt+UyWPhAhboPKWHEHS2Us3sIRFNt01XzAtedAyS+/3i5+u 15nFayzcVyd9xv2Rut+1wkNagzqvV4dUQ+LjPLNDKcXkUd0+ippJkqdoe7GwljdBcUk8 yG2MrumIER8eD38lTFVQ2zK5laDdgSp0j2sMku26LHtvFuGn7hfJnSeJcjax781H7gap +6/g== X-Gm-Message-State: AOAM533a6wSZwXestLlkPTmcnnLikWLx4rC7UM5OrmrG07AoPHzUrufr ZQa56FGjkVNup91H1QCqCUL3eCI8Pkkf8gcunnpBrJvrc7UDgtG9XD8y6+G/tTFZYLjyLUFC3Cf 8vShKdFxUst8= X-Received: by 2002:a05:6602:38e:: with SMTP id f14mr679944iov.62.1631118990920; Wed, 08 Sep 2021 09:36:30 -0700 (PDT) X-Google-Smtp-Source: ABdhPJwPBgjnxcRmDnznmj8Z8D2m8laj1F/cKtZgajr2y1D7LR0ubX5FPyZ9OQcONNyn/24a8NcA/Q== X-Received: by 2002:a05:6602:38e:: with SMTP id f14mr679909iov.62.1631118990674; Wed, 08 Sep 2021 09:36:30 -0700 (PDT) Received: from t490s.phub.net.cable.rogers.com ([2607:fea8:56a3:500::ad7f]) by smtp.gmail.com with ESMTPSA id q14sm1340458ilj.34.2021.09.08.09.36.29 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 08 Sep 2021 09:36:30 -0700 (PDT) From: Peter Xu To: linux-kernel@vger.kernel.org, Andrew Morton , Hugh Dickins , linux-mm@kvack.org Cc: Miaohe Lin , David Hildenbrand , Andrea Arcangeli , Yang Shi , Alistair Popple , Matthew Wilcox , "Kirill A . Shutemov" , Jerome Glisse , peterx@redhat.com, Liam Howlett , Mike Rapoport Subject: [PATCH v3 5/5] mm: Add ZAP_FLAG_SKIP_SWAP and zap_flags Date: Wed, 8 Sep 2021 12:36:28 -0400 Message-Id: <20210908163628.215052-1-peterx@redhat.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210908163516.214441-1-peterx@redhat.com> References: <20210908163516.214441-1-peterx@redhat.com> MIME-Version: 1.0 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Authentication-Results: imf27.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=gIdAQ6AN; spf=none (imf27.hostedemail.com: domain of peterx@redhat.com has no SPF policy when checking 170.10.133.124) smtp.mailfrom=peterx@redhat.com; dmarc=pass (policy=none) header.from=redhat.com X-Stat-Signature: q4x1nrz6fbind3dbctqqxjcgmjynahgc X-Rspamd-Server: rspam02 X-Rspamd-Queue-Id: 1633D700009B X-HE-Tag: 1631118992-671499 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Firstly, the comment in zap_pte_range() is misleading because it checks against details rather than check_mappings, so it's against what the code did. Meanwhile, there's no explicit reason why passing in the details pointer should mean to skip all swap entries. New user of zap_details could very possibly miss this fact if they don't read deep until zap_pte_range() because there's no comment at zap_details talking about it at all, so swap entries could be erroneously skipped without being noticed. This partly reverts 3e8715fdc03e ("mm: drop zap_details::check_swap_entries"), but introduce ZAP_FLAG_SKIP_SWAP flag, which means the opposite of previous "details" parameter: the caller should explicitly set this to skip swap entries, otherwise swap entries will always be considered (which should still be the major case here). We may want to look into when exactly we need ZAP_FLAG_SKIP_SWAP and we should have it in a synchronous manner, e.g., currently even if ZAP_FLAG_SKIP_SWAP is set we'll still look into swap pmds no matter what. But that should be a separate effort of this patch. The flag introduced in this patch will be a preparation for more bits defined in the future, e.g., for a new bit in flag to show whether to persist the upcoming uffd-wp bit in pgtable entries. Cc: Kirill A. Shutemov Cc: Hugh Dickins Signed-off-by: Peter Xu --- include/linux/mm.h | 16 ++++++++++++++++ mm/memory.c | 6 +++--- 2 files changed, 19 insertions(+), 3 deletions(-) diff --git a/include/linux/mm.h b/include/linux/mm.h index ed44f31615d9..beb784ce35b9 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -1717,12 +1717,18 @@ static inline bool can_do_mlock(void) { return false; } extern int user_shm_lock(size_t, struct ucounts *); extern void user_shm_unlock(size_t, struct ucounts *); +typedef unsigned int __bitwise zap_flags_t; + +/* Whether to skip zapping swap entries */ +#define ZAP_FLAG_SKIP_SWAP ((__force zap_flags_t) BIT(0)) + /* * Parameter block passed down to zap_pte_range in exceptional cases. */ struct zap_details { struct address_space *zap_mapping; /* Check page->mapping if set */ struct page *single_page; /* Locked page to be unmapped */ + zap_flags_t zap_flags; /* Extra flags for zapping */ }; /* @@ -1739,6 +1745,16 @@ zap_skip_check_mapping(struct zap_details *details, struct page *page) (details->zap_mapping != page_rmapping(page)); } +/* Return true if skip swap entries, false otherwise */ +static inline bool +zap_skip_swap(struct zap_details *details) +{ + if (!details) + return false; + + return details->zap_flags & ZAP_FLAG_SKIP_SWAP; +} + struct page *vm_normal_page(struct vm_area_struct *vma, unsigned long addr, pte_t pte); struct page *vm_normal_page_pmd(struct vm_area_struct *vma, unsigned long addr, diff --git a/mm/memory.c b/mm/memory.c index e5ee8399d270..26e37bef1888 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -1379,8 +1379,7 @@ static unsigned long zap_pte_range(struct mmu_gather *tlb, continue; } - /* If details->check_mapping, we leave swap entries. */ - if (unlikely(details)) + if (unlikely(zap_skip_swap(details))) continue; if (!non_swap_entry(entry)) @@ -3353,6 +3352,7 @@ void unmap_mapping_page(struct page *page) details.zap_mapping = mapping; details.single_page = page; + details.zap_flags = ZAP_FLAG_SKIP_SWAP; i_mmap_lock_write(mapping); if (unlikely(!RB_EMPTY_ROOT(&mapping->i_mmap.rb_root))) @@ -3377,7 +3377,7 @@ void unmap_mapping_pages(struct address_space *mapping, pgoff_t start, pgoff_t nr, bool even_cows) { pgoff_t first_index = start, last_index = start + nr - 1; - struct zap_details details = { }; + struct zap_details details = { .zap_flags = ZAP_FLAG_SKIP_SWAP }; details.zap_mapping = even_cows ? NULL : mapping; if (last_index < first_index)