From patchwork Wed Sep 1 20:56:18 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Xu X-Patchwork-Id: 12470315 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id DFFB8C43214 for ; Wed, 1 Sep 2021 20:56:31 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 74F3A6102A for ; Wed, 1 Sep 2021 20:56:31 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 74F3A6102A Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=kvack.org Received: by kanga.kvack.org (Postfix) id 0BFCB8D0001; Wed, 1 Sep 2021 16:56:31 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 06F54900002; Wed, 1 Sep 2021 16:56:31 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E2C188D0003; Wed, 1 Sep 2021 16:56:30 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0238.hostedemail.com [216.40.44.238]) by kanga.kvack.org (Postfix) with ESMTP id D17D78D0001 for ; Wed, 1 Sep 2021 16:56:30 -0400 (EDT) Received: from smtpin10.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id 88BA08249980 for ; Wed, 1 Sep 2021 20:56:30 +0000 (UTC) X-FDA: 78540212940.10.FF8D86A Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [216.205.24.124]) by imf02.hostedemail.com (Postfix) with ESMTP id 33BBC7001A08 for ; Wed, 1 Sep 2021 20:56:30 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1630529789; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=uUFjqTEK/Z5AKo1Agn+ijWYpHJREXXiYmYHdAc5BzAA=; b=JdUwMn5M+++y/r/8dj28OA75MXlmH1Sg8d0pNPmELyYsqrOdEmR0mWTx/o3b4SGofCrZ1Q oimfxjzQHJnI7ehT99KZA+YbI9NBY9htG+lNSOSYVRDoGhTGF056JJABxXIQIwrZizc5+G cP8qf+YAQXbMZcxiuOYLJDeUHlQlbSA= Received: from mail-qt1-f199.google.com (mail-qt1-f199.google.com [209.85.160.199]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-227-KyLw8rUhPsKxWgb94PD4pg-1; Wed, 01 Sep 2021 16:56:28 -0400 X-MC-Unique: KyLw8rUhPsKxWgb94PD4pg-1 Received: by mail-qt1-f199.google.com with SMTP id m6-20020ac807c6000000b0029994381c5fso896683qth.5 for ; Wed, 01 Sep 2021 13:56:28 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=uUFjqTEK/Z5AKo1Agn+ijWYpHJREXXiYmYHdAc5BzAA=; b=unG6/y74oEyEEZSwWF0BPgGw7/HULgYUmMiNhr1S+mUE6Iz10BpwrnY8uaj6FKL+NF WieKv0LYV87jmcvt+yx61AYReWZCIOYaE4AGfU9cbjp7DIVd8rHwzCM65SiirUEOroES nJkF3ekijrX8yKHWvy56XLFgmSgbyDyay13+CTfxQ4F0X/QOZGbN3In+ixKWHKiunzkN fbTQcWYzoHcuBBCw+sVNfh9jxplziTVtYU4qFP2FUF8HWBAWxu0KrxzluzUTxy+jhQ5R +FuNgcX1BkVUp6DVuicVwrpQAb4/puPt3Z8qwNI/0MZBgdp4EZwnDH9UP7o7cNxy5jjN 1znw== X-Gm-Message-State: AOAM533efR29vEYoSjQJnBGapSDRsv0eWxK+y31+XRVlz+TBqnnqn6T3 hIwt18AGYnSGOKN+N46saMEQoWs2EjANYLRTR2oivlBaoG5eAzgnVCCMD0CL1A8jzIyuKXZQiaR d3NJDxHUm1hY= X-Received: by 2002:a37:741:: with SMTP id 62mr1642009qkh.490.1630529788065; Wed, 01 Sep 2021 13:56:28 -0700 (PDT) X-Google-Smtp-Source: ABdhPJyg27bfuu/yWSrJR+ACgldtYrxoQNg/w2ohixTNUzohW/oZaLbtRecN5m3ghvaicA92Bsp6WQ== X-Received: by 2002:a37:741:: with SMTP id 62mr1641988qkh.490.1630529787864; Wed, 01 Sep 2021 13:56:27 -0700 (PDT) Received: from t490s.redhat.com ([2607:fea8:56a3:500::ad7f]) by smtp.gmail.com with ESMTPSA id f3sm624865qti.65.2021.09.01.13.56.26 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 01 Sep 2021 13:56:27 -0700 (PDT) From: Peter Xu To: linux-kernel@vger.kernel.org, linux-mm@kvack.org Cc: Andrea Arcangeli , Mike Rapoport , peterx@redhat.com, Jerome Glisse , Alistair Popple , Yang Shi , Andrew Morton , David Hildenbrand , Miaohe Lin , "Kirill A . Shutemov" , Matthew Wilcox , Hugh Dickins , Axel Rasmussen Subject: [PATCH 1/5] mm/shmem: Unconditionally set pte dirty in mfill_atomic_install_pte Date: Wed, 1 Sep 2021 16:56:18 -0400 Message-Id: <20210901205622.6935-2-peterx@redhat.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210901205622.6935-1-peterx@redhat.com> References: <20210901205622.6935-1-peterx@redhat.com> MIME-Version: 1.0 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Authentication-Results: imf02.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=JdUwMn5M; spf=none (imf02.hostedemail.com: domain of peterx@redhat.com has no SPF policy when checking 216.205.24.124) smtp.mailfrom=peterx@redhat.com; dmarc=pass (policy=none) header.from=redhat.com X-Stat-Signature: r94j3axyc68thsnohmu7mrqc7z11u3ge X-Rspamd-Queue-Id: 33BBC7001A08 X-Rspamd-Server: rspam04 X-HE-Tag: 1630529790-878931 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: It was conditionally done previously, as there's one shmem special case that we use SetPageDirty() instead. However that's not necessary and it should be easier and cleaner to do it unconditionally in mfill_atomic_install_pte(). The most recent discussion about this is here, where Hugh explained the history of SetPageDirty() and why it's possible that it's not required at all: https://lore.kernel.org/lkml/alpine.LSU.2.11.2104121657050.1097@eggly.anvils/ Currently mfill_atomic_install_pte() has three callers: 1. shmem_mfill_atomic_pte 2. mcopy_atomic_pte 3. mcontinue_atomic_pte After the change: case (1) should have its SetPageDirty replaced by the dirty bit on pte (so we unify them together, finally), case (2) should have no functional change at all as it has page_in_cache==false, case (3) may add a dirty bit to the pte. However since case (3) is UFFDIO_CONTINUE for shmem, it's merely 100% sure the page is dirty after all, so should not make a real difference either. This should make it much easier to follow on which case will set dirty for uffd, as we'll simply set it all now for all uffd related ioctls. Meanwhile, no special handling of SetPageDirty() if there's no need. Cc: Hugh Dickins Cc: Axel Rasmussen Cc: Andrea Arcangeli Signed-off-by: Peter Xu Reviewed-by: Axel Rasmussen --- mm/shmem.c | 1 - mm/userfaultfd.c | 3 +-- 2 files changed, 1 insertion(+), 3 deletions(-) diff --git a/mm/shmem.c b/mm/shmem.c index dacda7463d54..3f91c8ce4d02 100644 --- a/mm/shmem.c +++ b/mm/shmem.c @@ -2437,7 +2437,6 @@ int shmem_mfill_atomic_pte(struct mm_struct *dst_mm, shmem_recalc_inode(inode); spin_unlock_irq(&info->lock); - SetPageDirty(page); unlock_page(page); return 0; out_delete_from_cache: diff --git a/mm/userfaultfd.c b/mm/userfaultfd.c index 0e2132834bc7..b30a3724c701 100644 --- a/mm/userfaultfd.c +++ b/mm/userfaultfd.c @@ -69,10 +69,9 @@ int mfill_atomic_install_pte(struct mm_struct *dst_mm, pmd_t *dst_pmd, pgoff_t offset, max_off; _dst_pte = mk_pte(page, dst_vma->vm_page_prot); + _dst_pte = pte_mkdirty(_dst_pte); if (page_in_cache && !vm_shared) writable = false; - if (writable || !page_in_cache) - _dst_pte = pte_mkdirty(_dst_pte); if (writable) { if (wp_copy) _dst_pte = pte_mkuffd_wp(_dst_pte); From patchwork Wed Sep 1 20:56:19 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Xu X-Patchwork-Id: 12470317 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id D77D9C432BE for ; Wed, 1 Sep 2021 20:56:33 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 6C8566102A for ; Wed, 1 Sep 2021 20:56:33 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 6C8566102A Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=kvack.org Received: by kanga.kvack.org (Postfix) id D9166900003; Wed, 1 Sep 2021 16:56:32 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id D41E6900002; Wed, 1 Sep 2021 16:56:32 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B44CC900003; Wed, 1 Sep 2021 16:56:32 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0171.hostedemail.com [216.40.44.171]) by kanga.kvack.org (Postfix) with ESMTP id A464F900002 for ; Wed, 1 Sep 2021 16:56:32 -0400 (EDT) Received: from smtpin35.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id 54CFC252C9 for ; Wed, 1 Sep 2021 20:56:32 +0000 (UTC) X-FDA: 78540213024.35.5F7418D Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by imf16.hostedemail.com (Postfix) with ESMTP id F336CF00008C for ; Wed, 1 Sep 2021 20:56:31 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1630529791; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=H80HJYI0qyzkG5zbmWICcaBUu/kpdbMbG4peNGDuQ4A=; b=LBPbANKYJ0PJAKhGN4WDQG6XV2mWwaOgcZCDenl4WRKfdTE3pKGkv1RTF7kYBIBZX9NxlG NzzR8s/07XbAJQ1fOJeLwfLi4mOK+EHjEkWOKDXSQwXMGDJg0W/oMCCeihYg5hRuEPYHF/ JJkCLYc7aCb7XK4XmsZmH5+QNWEG8kU= Received: from mail-qv1-f70.google.com (mail-qv1-f70.google.com [209.85.219.70]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-460-Db0PLC4SPBKL7vyL3_8RVA-1; Wed, 01 Sep 2021 16:56:30 -0400 X-MC-Unique: Db0PLC4SPBKL7vyL3_8RVA-1 Received: by mail-qv1-f70.google.com with SMTP id dv7-20020ad44ee7000000b0036fa79fd337so984933qvb.6 for ; Wed, 01 Sep 2021 13:56:30 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=H80HJYI0qyzkG5zbmWICcaBUu/kpdbMbG4peNGDuQ4A=; b=pRCY3tqqEhpPHQaAYqRZ8rk5Z5vHUzR7roVVLJ+CaXrlJsfipUc2JdJg4/T9u5c8od xviCA0hEyShUgYIUnSdiVv5h0aSSnOrBRnCN9dElRhRNDdmtukGt+YXgqBpUypg0ex+b eFKUDdRv/KK/jqt1/OJ7gPYQJThSvwXBTs7X38W96k1JaXE/4fk+gBTZdB/BvbG9lGss FLMpnfd9XIDqkzhV0VhiGQbJ/fr9K/WwlSBrpTc0OhjgHrwGYYMRSplEqCfRQd/MmIR7 CKwxEaWXRzEtMaojls7YEeVorZD1AQVsIzmOLZQ0bt+9a/ptzKZvN+I3Mjd4KYgKgmDU 3DYQ== X-Gm-Message-State: AOAM530g4FrFGeMUvhz/Z6FPBYCVVsm4POpC+U0Y1VDs4UUDxq3n+hGq msWcNBqNScbSBOgw949ofMr9jpNRjhdfAR7Y10yRUhzEOPvFAm3k3NNoYwMVlHrWATxX7f3CDA9 L5kreuGGj+dA= X-Received: by 2002:ac8:4241:: with SMTP id r1mr433337qtm.296.1630529790194; Wed, 01 Sep 2021 13:56:30 -0700 (PDT) X-Google-Smtp-Source: ABdhPJxfX94m0Q5rz7prZMT3zggqIDbw5ktE9Et7MQf17IWYgA/u195BGGhxvkI+Dg/upwPF/Y2ASQ== X-Received: by 2002:ac8:4241:: with SMTP id r1mr433306qtm.296.1630529789960; Wed, 01 Sep 2021 13:56:29 -0700 (PDT) Received: from t490s.redhat.com ([2607:fea8:56a3:500::ad7f]) by smtp.gmail.com with ESMTPSA id f3sm624865qti.65.2021.09.01.13.56.28 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 01 Sep 2021 13:56:29 -0700 (PDT) From: Peter Xu To: linux-kernel@vger.kernel.org, linux-mm@kvack.org Cc: Andrea Arcangeli , Mike Rapoport , peterx@redhat.com, Jerome Glisse , Alistair Popple , Yang Shi , Andrew Morton , David Hildenbrand , Miaohe Lin , "Kirill A . Shutemov" , Matthew Wilcox , Hugh Dickins Subject: [PATCH 2/5] mm: Clear vmf->pte after pte_unmap_same() returns Date: Wed, 1 Sep 2021 16:56:19 -0400 Message-Id: <20210901205622.6935-3-peterx@redhat.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210901205622.6935-1-peterx@redhat.com> References: <20210901205622.6935-1-peterx@redhat.com> MIME-Version: 1.0 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Authentication-Results: imf16.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=LBPbANKY; spf=none (imf16.hostedemail.com: domain of peterx@redhat.com has no SPF policy when checking 170.10.133.124) smtp.mailfrom=peterx@redhat.com; dmarc=pass (policy=none) header.from=redhat.com X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: F336CF00008C X-Stat-Signature: 8tq3wsqd3ijs49xkkzsq4hkbyk3ucfd5 X-HE-Tag: 1630529791-772150 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: pte_unmap_same() will always unmap the pte pointer. After the unmap, vmf->pte will not be valid any more, we should clear it. It was safe only because no one is accessing vmf->pte after pte_unmap_same() returns, since the only caller of pte_unmap_same() (so far) is do_swap_page(), where vmf->pte will in most cases be overwritten very soon. Directly pass in vmf into pte_unmap_same() and then we can also avoid the long parameter list too, which should be a nice cleanup. Reviewed-by: Miaohe Lin Signed-off-by: Peter Xu Reviewed-by: David Hildenbrand --- mm/memory.c | 13 +++++++------ 1 file changed, 7 insertions(+), 6 deletions(-) diff --git a/mm/memory.c b/mm/memory.c index 25fc46e87214..204141e8a53d 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -2724,19 +2724,20 @@ EXPORT_SYMBOL_GPL(apply_to_existing_page_range); * proceeding (but do_wp_page is only called after already making such a check; * and do_anonymous_page can safely check later on). */ -static inline int pte_unmap_same(struct mm_struct *mm, pmd_t *pmd, - pte_t *page_table, pte_t orig_pte) +static inline int pte_unmap_same(struct vm_fault *vmf) { int same = 1; #if defined(CONFIG_SMP) || defined(CONFIG_PREEMPTION) if (sizeof(pte_t) > sizeof(unsigned long)) { - spinlock_t *ptl = pte_lockptr(mm, pmd); + spinlock_t *ptl = pte_lockptr(vmf->vma->vm_mm, vmf->pmd); spin_lock(ptl); - same = pte_same(*page_table, orig_pte); + same = pte_same(*vmf->pte, vmf->orig_pte); spin_unlock(ptl); } #endif - pte_unmap(page_table); + pte_unmap(vmf->pte); + /* After unmap of pte, the pointer is invalid now - clear it. */ + vmf->pte = NULL; return same; } @@ -3487,7 +3488,7 @@ vm_fault_t do_swap_page(struct vm_fault *vmf) vm_fault_t ret = 0; void *shadow = NULL; - if (!pte_unmap_same(vma->vm_mm, vmf->pmd, vmf->pte, vmf->orig_pte)) + if (!pte_unmap_same(vmf)) goto out; entry = pte_to_swp_entry(vmf->orig_pte); From patchwork Wed Sep 1 20:56:20 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Xu X-Patchwork-Id: 12470319 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id D45ABC4320A for ; Wed, 1 Sep 2021 20:56:39 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 814AD61027 for ; Wed, 1 Sep 2021 20:56:39 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 814AD61027 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=kvack.org Received: by kanga.kvack.org (Postfix) id 1827F940007; Wed, 1 Sep 2021 16:56:39 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 10BD6900002; Wed, 1 Sep 2021 16:56:39 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id F14DA940007; Wed, 1 Sep 2021 16:56:38 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0244.hostedemail.com [216.40.44.244]) by kanga.kvack.org (Postfix) with ESMTP id E3C64900002 for ; Wed, 1 Sep 2021 16:56:38 -0400 (EDT) Received: from smtpin34.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id 9905D8249980 for ; Wed, 1 Sep 2021 20:56:38 +0000 (UTC) X-FDA: 78540213276.34.6FED65A Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by imf30.hostedemail.com (Postfix) with ESMTP id 32CC9E0016B0 for ; Wed, 1 Sep 2021 20:56:38 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1630529797; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=wyvbaiG0U5DtzGIoOe88A1mEk7hd6gPkepgpidPNJQ0=; b=QYf7fnCK3r3e4sYCNbBPMGV/KfMurIVBUoKLSmS4Fg8yCSiLOcCJzGw9jxHgA8Tost3hZP /ubLi3GQ3f7Tp0uf/9cWCkTzUczQF4GHKfAV8Ffw96+cMoYl7Wrv+Vc/hHpvryL/QpSUMr qafZQZGxy7r0I4J/sYNLjEH7DHYTvYQ= Received: from mail-qv1-f72.google.com (mail-qv1-f72.google.com [209.85.219.72]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-308-MCik064nMjSx0m15pwoT6Q-1; Wed, 01 Sep 2021 16:56:36 -0400 X-MC-Unique: MCik064nMjSx0m15pwoT6Q-1 Received: by mail-qv1-f72.google.com with SMTP id b8-20020a0562141148b02902f1474ce8b7so936942qvt.20 for ; Wed, 01 Sep 2021 13:56:36 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=wyvbaiG0U5DtzGIoOe88A1mEk7hd6gPkepgpidPNJQ0=; b=Tfmb36L03hrEMSx5pXMpF6i34OBCDE+K2jga7aKmgDA5Rx/RXkh69kdCVIJS3+eiyJ xjT4lLZt2F683/58A1I3hzdadEfkmTuC4plqVeY8pL3OTFZGxK/tGh4Ppb/jUU3VllNT Ybmdi1f7xBSeioXXSA9pt+/F+LjR9vf+dpl+zw3rq04OqZ67QEdN4D4XKNxefpOhbqOX rHmU1O483uB+KI2pqImR768wML0V9wjT1pKqNxeN5Zo+Q2N9vyW6qdNylT3BFvgUGdQI cANhhc1Wlb4KIMsigV9Jc/cXXqVFnkX7Bw3SnxN0WxYCo2MsWk9WlCzi146fOaz0Be+1 qYOg== X-Gm-Message-State: AOAM533vRoKHF4bVubYMtMzbe6MwE9H41nQCf2ISoPCA2tbD9gNmUP/D 1g8WfPQyMzvJwbBGvYcZdP3xuyfM+DDAFvt8wD3zJFmfQ5vyZwJqfXYpXPtKtlqyWEdumG52ily +E/wRLcq5WaU= X-Received: by 2002:ac8:4d90:: with SMTP id a16mr1427468qtw.190.1630529792096; Wed, 01 Sep 2021 13:56:32 -0700 (PDT) X-Google-Smtp-Source: ABdhPJzA6t7HDUB1HpglkSgCMBRTVio39DwymdQJ8VWpp6mifRs9LMiBKPHS8XxadRgIFxFNScBadw== X-Received: by 2002:ac8:4d90:: with SMTP id a16mr1427452qtw.190.1630529791909; Wed, 01 Sep 2021 13:56:31 -0700 (PDT) Received: from t490s.redhat.com ([2607:fea8:56a3:500::ad7f]) by smtp.gmail.com with ESMTPSA id f3sm624865qti.65.2021.09.01.13.56.30 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 01 Sep 2021 13:56:31 -0700 (PDT) From: Peter Xu To: linux-kernel@vger.kernel.org, linux-mm@kvack.org Cc: Andrea Arcangeli , Mike Rapoport , peterx@redhat.com, Jerome Glisse , Alistair Popple , Yang Shi , Andrew Morton , David Hildenbrand , Miaohe Lin , "Kirill A . Shutemov" , Matthew Wilcox , Hugh Dickins Subject: [PATCH 3/5] mm: Drop first_index/last_index in zap_details Date: Wed, 1 Sep 2021 16:56:20 -0400 Message-Id: <20210901205622.6935-4-peterx@redhat.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210901205622.6935-1-peterx@redhat.com> References: <20210901205622.6935-1-peterx@redhat.com> MIME-Version: 1.0 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Authentication-Results: imf30.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=QYf7fnCK; dmarc=pass (policy=none) header.from=redhat.com; spf=none (imf30.hostedemail.com: domain of peterx@redhat.com has no SPF policy when checking 170.10.133.124) smtp.mailfrom=peterx@redhat.com X-Rspamd-Server: rspam03 X-Rspamd-Queue-Id: 32CC9E0016B0 X-Stat-Signature: rq4udmx43wsjg53mf7b3wbqh8rftr961 X-HE-Tag: 1630529798-251196 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: The first_index/last_index parameters in zap_details are actually only used in unmap_mapping_range_tree(). At the meantime, this function is only called by unmap_mapping_pages() once. Instead of passing these two variables through the whole stack of page zapping code, remove them from zap_details and let them simply be parameters of unmap_mapping_range_tree(), which is inlined. Reviewed-by: Alistair Popple Signed-off-by: Peter Xu Reviewed-by: David Hildenbrand --- include/linux/mm.h | 2 -- mm/memory.c | 29 ++++++++++++++++------------- 2 files changed, 16 insertions(+), 15 deletions(-) diff --git a/include/linux/mm.h b/include/linux/mm.h index 7ca22e6e694a..69259229f090 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -1721,8 +1721,6 @@ extern void user_shm_unlock(size_t, struct ucounts *); */ struct zap_details { struct address_space *check_mapping; /* Check page->mapping if set */ - pgoff_t first_index; /* Lowest page->index to unmap */ - pgoff_t last_index; /* Highest page->index to unmap */ struct page *single_page; /* Locked page to be unmapped */ }; diff --git a/mm/memory.c b/mm/memory.c index 204141e8a53d..3b860f6a51ac 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -3322,20 +3322,20 @@ static void unmap_mapping_range_vma(struct vm_area_struct *vma, } static inline void unmap_mapping_range_tree(struct rb_root_cached *root, + pgoff_t first_index, + pgoff_t last_index, struct zap_details *details) { struct vm_area_struct *vma; pgoff_t vba, vea, zba, zea; - vma_interval_tree_foreach(vma, root, - details->first_index, details->last_index) { - + vma_interval_tree_foreach(vma, root, first_index, last_index) { vba = vma->vm_pgoff; vea = vba + vma_pages(vma) - 1; - zba = details->first_index; + zba = first_index; if (zba < vba) zba = vba; - zea = details->last_index; + zea = last_index; if (zea > vea) zea = vea; @@ -3361,18 +3361,21 @@ void unmap_mapping_page(struct page *page) { struct address_space *mapping = page->mapping; struct zap_details details = { }; + pgoff_t first_index, last_index; VM_BUG_ON(!PageLocked(page)); VM_BUG_ON(PageTail(page)); + first_index = page->index; + last_index = page->index + thp_nr_pages(page) - 1; + details.check_mapping = mapping; - details.first_index = page->index; - details.last_index = page->index + thp_nr_pages(page) - 1; details.single_page = page; i_mmap_lock_write(mapping); if (unlikely(!RB_EMPTY_ROOT(&mapping->i_mmap.rb_root))) - unmap_mapping_range_tree(&mapping->i_mmap, &details); + unmap_mapping_range_tree(&mapping->i_mmap, first_index, + last_index, &details); i_mmap_unlock_write(mapping); } @@ -3391,17 +3394,17 @@ void unmap_mapping_page(struct page *page) void unmap_mapping_pages(struct address_space *mapping, pgoff_t start, pgoff_t nr, bool even_cows) { + pgoff_t first_index = start, last_index = start + nr - 1; struct zap_details details = { }; details.check_mapping = even_cows ? NULL : mapping; - details.first_index = start; - details.last_index = start + nr - 1; - if (details.last_index < details.first_index) - details.last_index = ULONG_MAX; + if (last_index < first_index) + last_index = ULONG_MAX; i_mmap_lock_write(mapping); if (unlikely(!RB_EMPTY_ROOT(&mapping->i_mmap.rb_root))) - unmap_mapping_range_tree(&mapping->i_mmap, &details); + unmap_mapping_range_tree(&mapping->i_mmap, first_index, + last_index, &details); i_mmap_unlock_write(mapping); } From patchwork Wed Sep 1 20:57:22 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Xu X-Patchwork-Id: 12470321 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7F49DC432BE for ; Wed, 1 Sep 2021 20:57:31 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 1E2A06108E for ; Wed, 1 Sep 2021 20:57:31 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 1E2A06108E Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=kvack.org Received: by kanga.kvack.org (Postfix) id AD04E940008; Wed, 1 Sep 2021 16:57:30 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id A7FBA900002; Wed, 1 Sep 2021 16:57:30 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 920B7940008; Wed, 1 Sep 2021 16:57:30 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0184.hostedemail.com [216.40.44.184]) by kanga.kvack.org (Postfix) with ESMTP id 7E514900002 for ; Wed, 1 Sep 2021 16:57:30 -0400 (EDT) Received: from smtpin36.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id 23A7218211CCC for ; Wed, 1 Sep 2021 20:57:30 +0000 (UTC) X-FDA: 78540215460.36.9352E15 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [216.205.24.124]) by imf20.hostedemail.com (Postfix) with ESMTP id C0182D0000AA for ; Wed, 1 Sep 2021 20:57:29 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1630529849; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Hq9ANBDJCBArw0iPPGA88rqVQpepBRZET67OXkoqBVo=; b=SA5j9O6611Ph6jJTxg197IcBVHyZh5oGBKc2vao/BWr1jAYiQobXn27G7n9DPzDUw5ppH2 82Hnwaj8RCKxrUZ6u+O51isf2RFlmuUaQG6FlJhlYkM7FzvoW2ADX3L5eTbfB4fkEnlHba 7aDwNyeya9V2gY0Tl+O/wNqKy8XhS7g= Received: from mail-qv1-f69.google.com (mail-qv1-f69.google.com [209.85.219.69]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-235-IcCU2xmPMoynlJlHytVwTw-1; Wed, 01 Sep 2021 16:57:26 -0400 X-MC-Unique: IcCU2xmPMoynlJlHytVwTw-1 Received: by mail-qv1-f69.google.com with SMTP id t12-20020ad45bcc000000b003772069d04aso940514qvt.19 for ; Wed, 01 Sep 2021 13:57:26 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Hq9ANBDJCBArw0iPPGA88rqVQpepBRZET67OXkoqBVo=; b=BNcKMCo22JyFd5akdzow9ik+atSXK6l5IfirTwVh7lWjkv5ra+AG+hBRmZFTzlieAE EIhuh+9C2XPeQzCivVOt73Ax01wCHXzjRUsPpaUuBUDXyUjuSUB2wvTYo/XmvQVRE+rr ejzMU13Z2QN0h+KUHC8UwNVNTzhL0ZYjrAYJlrCelGpPm1DLa21t4NK5Ik8a1+53eT2f R54f7NxtLJVQgesOnP2ba/eIsBDKYaU5TTGV0BHjDXDehs4ZjnVKlBdsADYMf+HZbwK/ 4BE4mR6I7Crf6klTf1SK/HpJqS0BoQJZBIKtIcJfbL3QyU5FcoVB296fpF/ITgux7b1s HYXQ== X-Gm-Message-State: AOAM532ltjWq8+t/ygnx4cggc0JWnlyBrFCw2C+yVILXCvIMnkIkit+r LmX7DBDKKw+bEcKs/euMLHMpBgZHH2TGpaGJzhkUHZ0gI9wF/ODwq7r9cHnLi/eVL2gXIEM8S3s rHe7kh9c3PPg= X-Received: by 2002:a05:6214:c23:: with SMTP id a3mr1816160qvd.34.1630529845442; Wed, 01 Sep 2021 13:57:25 -0700 (PDT) X-Google-Smtp-Source: ABdhPJy5+HjiwKW042yLAjnBDFXBymG7aWJsJmhLIvkEnkATvTWvbu1mTRN3D2vspZ0i+cjMM8yc8Q== X-Received: by 2002:a05:6214:c23:: with SMTP id a3mr1816129qvd.34.1630529845236; Wed, 01 Sep 2021 13:57:25 -0700 (PDT) Received: from t490s.redhat.com ([2607:fea8:56a3:500::ad7f]) by smtp.gmail.com with ESMTPSA id t28sm529163qkt.70.2021.09.01.13.57.23 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 01 Sep 2021 13:57:24 -0700 (PDT) From: Peter Xu To: linux-kernel@vger.kernel.org, linux-mm@kvack.org Cc: Andrew Morton , Matthew Wilcox , Yang Shi , Miaohe Lin , Hugh Dickins , David Hildenbrand , peterx@redhat.com, Mike Rapoport , Andrea Arcangeli , "Kirill A . Shutemov" , Jerome Glisse , Alistair Popple Subject: [PATCH 4/5] mm: Introduce zap_details.zap_flags Date: Wed, 1 Sep 2021 16:57:22 -0400 Message-Id: <20210901205722.7328-1-peterx@redhat.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210901205622.6935-1-peterx@redhat.com> References: <20210901205622.6935-1-peterx@redhat.com> MIME-Version: 1.0 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-Rspamd-Queue-Id: C0182D0000AA Authentication-Results: imf20.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=SA5j9O66; dmarc=pass (policy=none) header.from=redhat.com; spf=none (imf20.hostedemail.com: domain of peterx@redhat.com has no SPF policy when checking 216.205.24.124) smtp.mailfrom=peterx@redhat.com X-Rspamd-Server: rspam01 X-Stat-Signature: b8arwx543nbxnydsaqg6ac4y77seucm3 X-HE-Tag: 1630529849-373921 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Instead of trying to introduce one variable for every new zap_details fields, let's introduce a flag so that it can start to encode true/false informations. Let's start to use this flag first to clean up the only check_mapping variable. Firstly, the name "check_mapping" implies this is a "boolean", but actually it stores the mapping inside, just in a way that it won't be set if we don't want to check the mapping. To make things clearer, introduce the 1st zap flag ZAP_FLAG_CHECK_MAPPING, so that we only check against the mapping if this bit set. At the same time, we can rename check_mapping into zap_mapping and set it always. Since at it, introduce another helper zap_check_mapping_skip() and use it in zap_pte_range() properly. Some old comments have been removed in zap_pte_range() because they're duplicated, and since now we're with ZAP_FLAG_CHECK_MAPPING flag, it'll be very easy to grep this information by simply grepping the flag. It'll also make life easier when we want to e.g. pass in zap_flags into the callers like unmap_mapping_pages() (instead of adding new booleans besides the even_cows parameter). Signed-off-by: Peter Xu --- include/linux/mm.h | 19 ++++++++++++++++++- mm/memory.c | 34 ++++++++++------------------------ 2 files changed, 28 insertions(+), 25 deletions(-) diff --git a/include/linux/mm.h b/include/linux/mm.h index 69259229f090..fcbc1c4f8e8e 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -1716,14 +1716,31 @@ static inline bool can_do_mlock(void) { return false; } extern int user_shm_lock(size_t, struct ucounts *); extern void user_shm_unlock(size_t, struct ucounts *); +/* Whether to check page->mapping when zapping */ +#define ZAP_FLAG_CHECK_MAPPING BIT(0) + /* * Parameter block passed down to zap_pte_range in exceptional cases. */ struct zap_details { - struct address_space *check_mapping; /* Check page->mapping if set */ + struct address_space *zap_mapping; struct page *single_page; /* Locked page to be unmapped */ + unsigned long zap_flags; }; +/* Return true if skip zapping this page, false otherwise */ +static inline bool +zap_skip_check_mapping(struct zap_details *details, struct page *page) +{ + if (!details || !page) + return false; + + if (!(details->zap_flags & ZAP_FLAG_CHECK_MAPPING)) + return false; + + return details->zap_mapping != page_rmapping(page); +} + struct page *vm_normal_page(struct vm_area_struct *vma, unsigned long addr, pte_t pte); struct page *vm_normal_page_pmd(struct vm_area_struct *vma, unsigned long addr, diff --git a/mm/memory.c b/mm/memory.c index 3b860f6a51ac..05ccacda4fe9 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -1333,16 +1333,8 @@ static unsigned long zap_pte_range(struct mmu_gather *tlb, struct page *page; page = vm_normal_page(vma, addr, ptent); - if (unlikely(details) && page) { - /* - * unmap_shared_mapping_pages() wants to - * invalidate cache without truncating: - * unmap shared but keep private pages. - */ - if (details->check_mapping && - details->check_mapping != page_rmapping(page)) - continue; - } + if (unlikely(zap_skip_check_mapping(details, page))) + continue; ptent = ptep_get_and_clear_full(mm, addr, pte, tlb->fullmm); tlb_remove_tlb_entry(tlb, pte, addr); @@ -1375,17 +1367,8 @@ static unsigned long zap_pte_range(struct mmu_gather *tlb, is_device_exclusive_entry(entry)) { struct page *page = pfn_swap_entry_to_page(entry); - if (unlikely(details && details->check_mapping)) { - /* - * unmap_shared_mapping_pages() wants to - * invalidate cache without truncating: - * unmap shared but keep private pages. - */ - if (details->check_mapping != - page_rmapping(page)) - continue; - } - + if (unlikely(zap_skip_check_mapping(details, page))) + continue; pte_clear_not_present_full(mm, addr, pte, tlb->fullmm); rss[mm_counter(page)]--; @@ -3369,8 +3352,9 @@ void unmap_mapping_page(struct page *page) first_index = page->index; last_index = page->index + thp_nr_pages(page) - 1; - details.check_mapping = mapping; + details.zap_mapping = mapping; details.single_page = page; + details.zap_flags = ZAP_FLAG_CHECK_MAPPING; i_mmap_lock_write(mapping); if (unlikely(!RB_EMPTY_ROOT(&mapping->i_mmap.rb_root))) @@ -3395,9 +3379,11 @@ void unmap_mapping_pages(struct address_space *mapping, pgoff_t start, pgoff_t nr, bool even_cows) { pgoff_t first_index = start, last_index = start + nr - 1; - struct zap_details details = { }; + struct zap_details details = { .zap_mapping = mapping }; + + if (!even_cows) + details.zap_flags |= ZAP_FLAG_CHECK_MAPPING; - details.check_mapping = even_cows ? NULL : mapping; if (last_index < first_index) last_index = ULONG_MAX; From patchwork Wed Sep 1 20:57:25 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Xu X-Patchwork-Id: 12470323 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9BD9BC43214 for ; Wed, 1 Sep 2021 20:57:32 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 4BE9061027 for ; Wed, 1 Sep 2021 20:57:32 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 4BE9061027 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=kvack.org Received: by kanga.kvack.org (Postfix) id EA13F900002; Wed, 1 Sep 2021 16:57:31 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id E02EA940009; Wed, 1 Sep 2021 16:57:31 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C7C7D900004; Wed, 1 Sep 2021 16:57:31 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0032.hostedemail.com [216.40.44.32]) by kanga.kvack.org (Postfix) with ESMTP id BB2BF900002 for ; Wed, 1 Sep 2021 16:57:31 -0400 (EDT) Received: from smtpin14.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id 8324729E17 for ; Wed, 1 Sep 2021 20:57:31 +0000 (UTC) X-FDA: 78540215502.14.EE74A55 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [216.205.24.124]) by imf04.hostedemail.com (Postfix) with ESMTP id 28FC750000A8 for ; Wed, 1 Sep 2021 20:57:30 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1630529850; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=pWrZC5R796B6AxPjLXJOHfEspvF6TQvhNiM+3hpUqTo=; b=jNuAztBmTiKaWlmyT9CavBq8w3xioiisRYhp3/WNKBPY2oF22Gd9qpLU4Bs2JdQUc4sYex fbE/Cpm9ChG3LpzqVFvO6ZXuIyHaqhLusPHeLOMcDwR69SoOWrELkUw68pKoCeN7YFlcNm HCStAbmIH1t7RIScOZ3ephJtK0pHaVY= Received: from mail-qk1-f197.google.com (mail-qk1-f197.google.com [209.85.222.197]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-253-DgkgPDblNzeO1Kfs_qxIuA-1; Wed, 01 Sep 2021 16:57:29 -0400 X-MC-Unique: DgkgPDblNzeO1Kfs_qxIuA-1 Received: by mail-qk1-f197.google.com with SMTP id e22-20020a05620a209600b003d5ff97bff7so871931qka.1 for ; Wed, 01 Sep 2021 13:57:29 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=pWrZC5R796B6AxPjLXJOHfEspvF6TQvhNiM+3hpUqTo=; b=bqsCHFA74eYPQSU652AVYsMVMMFA/57wQLMEtYerFYSsQmQs3H1RrAtUAj+2UAsOPv 63xjudVMNDNvihQYR2w0nSnR5TnV8lCL+Dmpy3ooKdMr0J4gxSPDMuneHWePXONv8xyn vXuzGnNNxKsE+fhHJLgi+I9mMPiad6v1Fx+K22Y7Vf7ANQpu1+c2M6U5lLMkGCOl3mXH hRIa2/zhjP3h5bQ9DeO3AUHmsRvt/MdCdsKedcqhBRDyu4Yq2Tu90zoSYZG5i3+t5Vkc NibC/KStrr4s0GkG583pAsxxZrNc0bCa5S73LbwkdjT5jbi/uSWWj6HeQABhAq670pj2 18LQ== X-Gm-Message-State: AOAM53053uRSzqt7Q0GHpdCdyyVH+Jk6o7uAdq6rQ1Xzv6/lsWsgRSc4 6atlleF4EeGub4TUX0X9CEHKzwpcAr+QugezJzdE7ASCVI2WkFoJq+LY+xYZ4E5rZ77g0PpAnkk wIf9H7K8yNIA= X-Received: by 2002:a37:6c2:: with SMTP id 185mr1619298qkg.260.1630529849096; Wed, 01 Sep 2021 13:57:29 -0700 (PDT) X-Google-Smtp-Source: ABdhPJwAT0cwh0fiCuGBlVwkSIV+oBHm4p8ri9MOokNJ6Nf3K4EOtcbAo8KK7ymA6ffmkzusdLOapw== X-Received: by 2002:a37:6c2:: with SMTP id 185mr1619264qkg.260.1630529848869; Wed, 01 Sep 2021 13:57:28 -0700 (PDT) Received: from t490s.redhat.com ([2607:fea8:56a3:500::ad7f]) by smtp.gmail.com with ESMTPSA id h17sm562359qtu.68.2021.09.01.13.57.26 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 01 Sep 2021 13:57:28 -0700 (PDT) From: Peter Xu To: linux-kernel@vger.kernel.org, linux-mm@kvack.org Cc: Andrew Morton , Matthew Wilcox , Yang Shi , Miaohe Lin , Hugh Dickins , David Hildenbrand , peterx@redhat.com, Mike Rapoport , Andrea Arcangeli , "Kirill A . Shutemov" , Jerome Glisse , Alistair Popple Subject: [PATCH 5/5] mm: Introduce ZAP_FLAG_SKIP_SWAP Date: Wed, 1 Sep 2021 16:57:25 -0400 Message-Id: <20210901205725.7381-1-peterx@redhat.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210901205622.6935-1-peterx@redhat.com> References: <20210901205622.6935-1-peterx@redhat.com> MIME-Version: 1.0 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Authentication-Results: imf04.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=jNuAztBm; spf=none (imf04.hostedemail.com: domain of peterx@redhat.com has no SPF policy when checking 216.205.24.124) smtp.mailfrom=peterx@redhat.com; dmarc=pass (policy=none) header.from=redhat.com X-Rspamd-Server: rspam02 X-Rspamd-Queue-Id: 28FC750000A8 X-Stat-Signature: fiq7ugq71efqyeeqau3pq3tf61nj4ciw X-HE-Tag: 1630529850-535462 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Firstly, the comment in zap_pte_range() is misleading because it checks against details rather than check_mappings, so it's against what the code did. Meanwhile, it's confusing too on not explaining why passing in the details pointer would mean to skip all swap entries. New user of zap_details could very possibly miss this fact if they don't read deep until zap_pte_range() because there's no comment at zap_details talking about it at all, so swap entries could be errornously skipped without being noticed. This partly reverts 3e8715fdc03e ("mm: drop zap_details::check_swap_entries"), but introduce ZAP_FLAG_SKIP_SWAP flag, which means the opposite of previous "details" parameter: the caller should explicitly set this to skip swap entries, otherwise swap entries will always be considered (which is still the major case here). Cc: Kirill A. Shutemov Reviewed-by: Alistair Popple Signed-off-by: Peter Xu --- include/linux/mm.h | 12 ++++++++++++ mm/memory.c | 8 +++++--- 2 files changed, 17 insertions(+), 3 deletions(-) diff --git a/include/linux/mm.h b/include/linux/mm.h index fcbc1c4f8e8e..f798f5e4baa5 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -1718,6 +1718,8 @@ extern void user_shm_unlock(size_t, struct ucounts *); /* Whether to check page->mapping when zapping */ #define ZAP_FLAG_CHECK_MAPPING BIT(0) +/* Whether to skip zapping swap entries */ +#define ZAP_FLAG_SKIP_SWAP BIT(1) /* * Parameter block passed down to zap_pte_range in exceptional cases. @@ -1741,6 +1743,16 @@ zap_skip_check_mapping(struct zap_details *details, struct page *page) return details->zap_mapping != page_rmapping(page); } +/* Return true if skip swap entries, false otherwise */ +static inline bool +zap_skip_swap(struct zap_details *details) +{ + if (!details) + return false; + + return details->zap_flags & ZAP_FLAG_SKIP_SWAP; +} + struct page *vm_normal_page(struct vm_area_struct *vma, unsigned long addr, pte_t pte); struct page *vm_normal_page_pmd(struct vm_area_struct *vma, unsigned long addr, diff --git a/mm/memory.c b/mm/memory.c index 05ccacda4fe9..79957265afb4 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -1379,8 +1379,7 @@ static unsigned long zap_pte_range(struct mmu_gather *tlb, continue; } - /* If details->check_mapping, we leave swap entries. */ - if (unlikely(details)) + if (unlikely(zap_skip_swap(details))) continue; if (!non_swap_entry(entry)) @@ -3379,7 +3378,10 @@ void unmap_mapping_pages(struct address_space *mapping, pgoff_t start, pgoff_t nr, bool even_cows) { pgoff_t first_index = start, last_index = start + nr - 1; - struct zap_details details = { .zap_mapping = mapping }; + struct zap_details details = { + .zap_mapping = mapping, + .zap_flags = ZAP_FLAG_SKIP_SWAP, + }; if (!even_cows) details.zap_flags |= ZAP_FLAG_CHECK_MAPPING;