From patchwork Wed Sep 8 16:35:12 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Xu X-Patchwork-Id: 12481523 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id DDBB0C433EF for ; Wed, 8 Sep 2021 16:35:28 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 8279261164 for ; Wed, 8 Sep 2021 16:35:28 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 8279261164 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=kvack.org Received: by kanga.kvack.org (Postfix) id 2E72F6B0071; Wed, 8 Sep 2021 12:35:28 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 2BD9B900002; Wed, 8 Sep 2021 12:35:28 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 1862A6B0073; Wed, 8 Sep 2021 12:35:28 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0190.hostedemail.com [216.40.44.190]) by kanga.kvack.org (Postfix) with ESMTP id 071896B0071 for ; Wed, 8 Sep 2021 12:35:28 -0400 (EDT) Received: from smtpin01.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id AE315182C161D for ; Wed, 8 Sep 2021 16:35:27 +0000 (UTC) X-FDA: 78564956694.01.D8E8C90 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by imf05.hostedemail.com (Postfix) with ESMTP id 58F975051D66 for ; Wed, 8 Sep 2021 16:35:27 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1631118926; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=v0jOEqk3duu/S7ny3QESOOgnrnVwgkDjlQfF0n80v8I=; b=QED/3y7RxxnpfkJJGR4NBgKPG/g063gpx8kcAWqQdAi+WkS2ZYkf+SYrrjXE+1wc2FRooc 4HkOLFw3vk9Hc+PEUyCAJIreyOHjOSWbTFTgG0J8U7CTGYaPcggti5DyPd9vEqwM1h9L+N LvZ0+msa8ZL61OgVtoK5uJQpbTxMeWY= Received: from mail-il1-f199.google.com (mail-il1-f199.google.com [209.85.166.199]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-518-qQZvFqbcPjiFHTSGm__R0Q-1; Wed, 08 Sep 2021 12:35:25 -0400 X-MC-Unique: qQZvFqbcPjiFHTSGm__R0Q-1 Received: by mail-il1-f199.google.com with SMTP id v9-20020a92c6c9000000b00226d10082a6so2078876ilm.23 for ; Wed, 08 Sep 2021 09:35:25 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=v0jOEqk3duu/S7ny3QESOOgnrnVwgkDjlQfF0n80v8I=; b=If9LE5sZ8Nx0Njak6zUT9USrWetdNgcd9cyTzIVgJ1QHU1pSYk9BBke/34G9ySRsv/ J4T4sCDP7GJ02UxCQddAkUzlK6iYim8s5kVd+M7dkCsZwiFoEM9fZt1Mv9aLCufTwboK LTecL6D3Fv5uGNrGKMHiw1rozdma5LHfYfJOVB5q5rPwLNffz76uvtW26vDsxY/04EaJ kNQWAuPLkKM2glQg9TovSP4Dun9M4ZNtNdy2ZyEqgLYAJIyDXHogkvk3Ys2nVl9R3IAE 5lMx8zl9mpdAUd0J1hj15UCYXJl2mRU4yfsKyp7eGYpJpeHF0bvujO+CTuxCye3EnDv2 YlYg== X-Gm-Message-State: AOAM532byQ/0TMt3fAr4q0pjacShR9pAfsfKR50jS/VytvU8YNU6MMIu /XadkSopZcMrMYQAgtXrGXU5crElSobplHCbz6etMjSjsOBncBmKZuBiT/S4Yh6BCDMvSlk51Ab N1KoAe/op3ZM= X-Received: by 2002:a92:c9cd:: with SMTP id k13mr518810ilq.169.1631118921708; Wed, 08 Sep 2021 09:35:21 -0700 (PDT) X-Google-Smtp-Source: ABdhPJyaDhl3W2lZk1HaXU2PU7Ni0ECC9fyLGs82p9dRaw7bfzwNvMEP/+GFrjJV2NjJwJoK0CCEOQ== X-Received: by 2002:a92:c9cd:: with SMTP id k13mr518786ilq.169.1631118921445; Wed, 08 Sep 2021 09:35:21 -0700 (PDT) Received: from t490s.phub.net.cable.rogers.com ([2607:fea8:56a3:500::ad7f]) by smtp.gmail.com with ESMTPSA id b10sm1336961ils.13.2021.09.08.09.35.19 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 08 Sep 2021 09:35:20 -0700 (PDT) From: Peter Xu To: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Hugh Dickins , Andrew Morton Cc: Yang Shi , Miaohe Lin , Alistair Popple , Matthew Wilcox , David Hildenbrand , Jerome Glisse , "Kirill A . Shutemov" , Andrea Arcangeli , Liam Howlett , Mike Rapoport , peterx@redhat.com, Axel Rasmussen Subject: [PATCH v3 1/5] mm/shmem: Unconditionally set pte dirty in mfill_atomic_install_pte Date: Wed, 8 Sep 2021 12:35:12 -0400 Message-Id: <20210908163516.214441-2-peterx@redhat.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210908163516.214441-1-peterx@redhat.com> References: <20210908163516.214441-1-peterx@redhat.com> MIME-Version: 1.0 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Authentication-Results: imf05.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b="QED/3y7R"; spf=none (imf05.hostedemail.com: domain of peterx@redhat.com has no SPF policy when checking 170.10.133.124) smtp.mailfrom=peterx@redhat.com; dmarc=pass (policy=none) header.from=redhat.com X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: 58F975051D66 X-Stat-Signature: jai6hscq1b81fkk3wia79rfatnkbqw4t X-HE-Tag: 1631118927-977638 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: It was conditionally done previously, as there's one shmem special case that we use SetPageDirty() instead. However that's not necessary and it should be easier and cleaner to do it unconditionally in mfill_atomic_install_pte(). The most recent discussion about this is here, where Hugh explained the history of SetPageDirty() and why it's possible that it's not required at all: https://lore.kernel.org/lkml/alpine.LSU.2.11.2104121657050.1097@eggly.anvils/ Currently mfill_atomic_install_pte() has three callers: 1. shmem_mfill_atomic_pte 2. mcopy_atomic_pte 3. mcontinue_atomic_pte After the change: case (1) should have its SetPageDirty replaced by the dirty bit on pte (so we unify them together, finally), case (2) should have no functional change at all as it has page_in_cache==false, case (3) may add a dirty bit to the pte. However since case (3) is UFFDIO_CONTINUE for shmem, it's merely 100% sure the page is dirty after all, so should not make a real difference either. This should make it much easier to follow on which case will set dirty for uffd, as we'll simply set it all now for all uffd related ioctls. Meanwhile, no special handling of SetPageDirty() if there's no need. Cc: Hugh Dickins Cc: Axel Rasmussen Cc: Andrea Arcangeli Reviewed-by: Axel Rasmussen Signed-off-by: Peter Xu --- mm/shmem.c | 1 - mm/userfaultfd.c | 3 +-- 2 files changed, 1 insertion(+), 3 deletions(-) diff --git a/mm/shmem.c b/mm/shmem.c index 88742953532c..96ccf6e941aa 100644 --- a/mm/shmem.c +++ b/mm/shmem.c @@ -2424,7 +2424,6 @@ int shmem_mfill_atomic_pte(struct mm_struct *dst_mm, shmem_recalc_inode(inode); spin_unlock_irq(&info->lock); - SetPageDirty(page); unlock_page(page); return 0; out_delete_from_cache: diff --git a/mm/userfaultfd.c b/mm/userfaultfd.c index 7a9008415534..caf6dfff2a60 100644 --- a/mm/userfaultfd.c +++ b/mm/userfaultfd.c @@ -69,10 +69,9 @@ int mfill_atomic_install_pte(struct mm_struct *dst_mm, pmd_t *dst_pmd, pgoff_t offset, max_off; _dst_pte = mk_pte(page, dst_vma->vm_page_prot); + _dst_pte = pte_mkdirty(_dst_pte); if (page_in_cache && !vm_shared) writable = false; - if (writable || !page_in_cache) - _dst_pte = pte_mkdirty(_dst_pte); if (writable) { if (wp_copy) _dst_pte = pte_mkuffd_wp(_dst_pte); From patchwork Wed Sep 8 16:36:19 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Xu X-Patchwork-Id: 12481525 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E4BB8C433F5 for ; Wed, 8 Sep 2021 16:36:27 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 8263261139 for ; Wed, 8 Sep 2021 16:36:27 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 8263261139 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=kvack.org Received: by kanga.kvack.org (Postfix) id 2E5A86B006C; Wed, 8 Sep 2021 12:36:27 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 26E886B0072; Wed, 8 Sep 2021 12:36:27 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 10F836B0073; Wed, 8 Sep 2021 12:36:27 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0124.hostedemail.com [216.40.44.124]) by kanga.kvack.org (Postfix) with ESMTP id 02C0D6B006C for ; Wed, 8 Sep 2021 12:36:27 -0400 (EDT) Received: from smtpin02.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id BC6362FD88 for ; Wed, 8 Sep 2021 16:36:26 +0000 (UTC) X-FDA: 78564959172.02.8B39569 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [216.205.24.124]) by imf27.hostedemail.com (Postfix) with ESMTP id 6CF6270000A0 for ; Wed, 8 Sep 2021 16:36:26 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1631118985; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=yCBcgdQfSQO0vdlLaUvPYqHDlVPUHtkRpfsoC2DOf1s=; b=VOug+yt+rtSnFCpb+o2ehs/JosfLWaw2JI2Efn6LKBEh5E+k5RjvEdfS8T7OjvHTYlLZg9 XLFFtYvwInz9+xrmTKrAmv0DJP4ogtrgYbbchpI5GICGiHtf+VmnRen4vFleiKKBDO/wnA tzly3PHmkqbNflwyO7L+JS7gW8XRu7Q= Received: from mail-il1-f199.google.com (mail-il1-f199.google.com [209.85.166.199]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-175-EW6hI-rANzSyBKF_ss5Jvg-1; Wed, 08 Sep 2021 12:36:22 -0400 X-MC-Unique: EW6hI-rANzSyBKF_ss5Jvg-1 Received: by mail-il1-f199.google.com with SMTP id r6-20020a92c506000000b002246015b2a4so2090720ilg.21 for ; Wed, 08 Sep 2021 09:36:22 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=yCBcgdQfSQO0vdlLaUvPYqHDlVPUHtkRpfsoC2DOf1s=; b=jlWmRKk5CdgJT10CU03ALN3ccT13na2rxsTWIxGjRsP6crpaJ08/FyY8D9kei4MOgx 9Rd0nWunZgf8AP89tsMq00mibUPUKUUQ+pMGRaZGaBPZ7+UdPVQU3LcNq4XdmtH0w5W8 PJ/7VKIMEQOBzdC8IJ5Ab/aFdDZUMCz6gyctd1e4DcM4ykTWdMLSBEIX7/DzQaczDEp9 AqaU1iTdgEGqUyMCN6GPRoafXChy7PcxhrJp1OjMh5gjBt7nvyqwQlGvhCoZqgOcjPOs ogw0trxivRPDQLvoHeH6+Z1LzobXQAYwEHgocuZPtRmcsSlA69o4kJz8XyzgJ7gEHLnK u9WQ== X-Gm-Message-State: AOAM530L0m9F07YibThbAJuNz/Zeb3rkSK00rWmHVVLpRLDvu9RumSBi 2uVG1YBdt+hXFXoxK0em7m1TbKvLOe9X6Fv4AMjcrNUZNqtq/EuhQ9q7j898nrWM65uc8RP9jsR eyfKCQ0m5srk= X-Received: by 2002:a05:6602:d9:: with SMTP id z25mr645545ioe.154.1631118981961; Wed, 08 Sep 2021 09:36:21 -0700 (PDT) X-Google-Smtp-Source: ABdhPJx1EbGLCnuAcfZACCBnyvgCaW1SFMtQKrjHJRqiyYZqaaZQiJRFzEufdRaFsqV34fbIanuG2w== X-Received: by 2002:a05:6602:d9:: with SMTP id z25mr645522ioe.154.1631118981774; Wed, 08 Sep 2021 09:36:21 -0700 (PDT) Received: from t490s.phub.net.cable.rogers.com ([2607:fea8:56a3:500::ad7f]) by smtp.gmail.com with ESMTPSA id n37sm1268406ioz.5.2021.09.08.09.36.20 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 08 Sep 2021 09:36:21 -0700 (PDT) From: Peter Xu To: linux-kernel@vger.kernel.org, Andrew Morton , Hugh Dickins , linux-mm@kvack.org Cc: Miaohe Lin , David Hildenbrand , Andrea Arcangeli , Yang Shi , Alistair Popple , Matthew Wilcox , "Kirill A . Shutemov" , Jerome Glisse , peterx@redhat.com, Liam Howlett , Mike Rapoport Subject: [PATCH v3 2/5] mm: Clear vmf->pte after pte_unmap_same() returns Date: Wed, 8 Sep 2021 12:36:19 -0400 Message-Id: <20210908163619.214901-1-peterx@redhat.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210908163516.214441-1-peterx@redhat.com> References: <20210908163516.214441-1-peterx@redhat.com> MIME-Version: 1.0 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: 6CF6270000A0 X-Stat-Signature: 9eg71m4jojauissanif71m9ijg1odhya Authentication-Results: imf27.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=VOug+yt+; spf=none (imf27.hostedemail.com: domain of peterx@redhat.com has no SPF policy when checking 216.205.24.124) smtp.mailfrom=peterx@redhat.com; dmarc=pass (policy=none) header.from=redhat.com X-HE-Tag: 1631118986-713958 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: pte_unmap_same() will always unmap the pte pointer. After the unmap, vmf->pte will not be valid any more, we should clear it. It was safe only because no one is accessing vmf->pte after pte_unmap_same() returns, since the only caller of pte_unmap_same() (so far) is do_swap_page(), where vmf->pte will in most cases be overwritten very soon. Directly pass in vmf into pte_unmap_same() and then we can also avoid the long parameter list too, which should be a nice cleanup. Reviewed-by: Miaohe Lin Reviewed-by: David Hildenbrand Reviewed-by: Liam Howlett Signed-off-by: Peter Xu --- mm/memory.c | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/mm/memory.c b/mm/memory.c index 25fc46e87214..7b095f07c4ef 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -2724,19 +2724,19 @@ EXPORT_SYMBOL_GPL(apply_to_existing_page_range); * proceeding (but do_wp_page is only called after already making such a check; * and do_anonymous_page can safely check later on). */ -static inline int pte_unmap_same(struct mm_struct *mm, pmd_t *pmd, - pte_t *page_table, pte_t orig_pte) +static inline int pte_unmap_same(struct vm_fault *vmf) { int same = 1; #if defined(CONFIG_SMP) || defined(CONFIG_PREEMPTION) if (sizeof(pte_t) > sizeof(unsigned long)) { - spinlock_t *ptl = pte_lockptr(mm, pmd); + spinlock_t *ptl = pte_lockptr(vmf->vma->vm_mm, vmf->pmd); spin_lock(ptl); - same = pte_same(*page_table, orig_pte); + same = pte_same(*vmf->pte, vmf->orig_pte); spin_unlock(ptl); } #endif - pte_unmap(page_table); + pte_unmap(vmf->pte); + vmf->pte = NULL; return same; } @@ -3487,7 +3487,7 @@ vm_fault_t do_swap_page(struct vm_fault *vmf) vm_fault_t ret = 0; void *shadow = NULL; - if (!pte_unmap_same(vma->vm_mm, vmf->pmd, vmf->pte, vmf->orig_pte)) + if (!pte_unmap_same(vmf)) goto out; entry = pte_to_swp_entry(vmf->orig_pte); From patchwork Wed Sep 8 16:36:22 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Xu X-Patchwork-Id: 12481527 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 37197C433FE for ; Wed, 8 Sep 2021 16:36:29 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id CEA2D60F45 for ; Wed, 8 Sep 2021 16:36:28 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org CEA2D60F45 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=kvack.org Received: by kanga.kvack.org (Postfix) id 7061C6B0072; Wed, 8 Sep 2021 12:36:28 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 63F156B0073; Wed, 8 Sep 2021 12:36:28 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 380CB6B0074; Wed, 8 Sep 2021 12:36:28 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0181.hostedemail.com [216.40.44.181]) by kanga.kvack.org (Postfix) with ESMTP id 22F4E6B0072 for ; Wed, 8 Sep 2021 12:36:28 -0400 (EDT) Received: from smtpin23.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id D2DC382499A8 for ; Wed, 8 Sep 2021 16:36:27 +0000 (UTC) X-FDA: 78564959214.23.81C7CF9 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [216.205.24.124]) by imf11.hostedemail.com (Postfix) with ESMTP id 748ECF0000BD for ; Wed, 8 Sep 2021 16:36:27 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1631118986; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=iu2AY6e6Zg6u66NXQnGBsoZxC9WFtdy2lzYxTQM+jPs=; b=avaIrwkIHfU3uPrylymeNnUZXKF4CPx9JBnEuZJ4EUWwh9vIhGFZcKg2qZP+eMvaJN+JQq vYEZ+y+fLZJupvmdGMhKZJfL5ruJchl8dxdbasnCcDIy01fCG6IZIu8oRfQai2tvryqzJ7 wTv+6/6N66ySrH/nutl8DjotUyUTB1Y= Received: from mail-io1-f69.google.com (mail-io1-f69.google.com [209.85.166.69]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-443-n0q-sgTwNreAFwj0pEAMgA-1; Wed, 08 Sep 2021 12:36:25 -0400 X-MC-Unique: n0q-sgTwNreAFwj0pEAMgA-1 Received: by mail-io1-f69.google.com with SMTP id g2-20020a6b7602000000b005be59530196so2427389iom.0 for ; Wed, 08 Sep 2021 09:36:25 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=iu2AY6e6Zg6u66NXQnGBsoZxC9WFtdy2lzYxTQM+jPs=; b=mjYCGYvexAWqQdyoWHX03pk97RSedm0ToQpaFDhgYNLw9rUORxoQXInOQpxOAOlN6r fw1ZqMjpwdW1HAE/Hgah3bPriDP1ZVHCLj0jo5BJGPscYgHxxJHDmg4G51wIQo0wVbNf hX42nGcqCQ5RL5Ey2WSCfNL1Qub0L9mHWx7Rd6UmoAGCE2IFZzwdeRZ/UvXx0/dZsLpR MRzHvafN3j9A6M9OtwLNgVEsj78isc1nuHouiksuVRbXj/8es2w1UvkSyj1joWMaacCS 6eSuPjui9+u7XLP3OgtMe9pZRQ84b6W6pmEI4hzm46N9Bzn/4rv2XFewpTo2stgf/FeU Lgwg== X-Gm-Message-State: AOAM533sDzmmLqhb4ucgnbNpCdsZbIfCKmwLP7+PxtUrSb1ZyxQRTUdn 8/oalctsrS5L0/TYMfdFOak0cOTmFr2QujAvRYMtkm1w8JDO0cmaq9E/GzGfRnx7FViy7exbk1R 18LE+GsG4qyM= X-Received: by 2002:a92:d646:: with SMTP id x6mr564582ilp.280.1631118984919; Wed, 08 Sep 2021 09:36:24 -0700 (PDT) X-Google-Smtp-Source: ABdhPJzYlGL1ZYPDtYUFtupEgcY3T/OH0AOHOi+Li/h9KZCbrHinuN5nXt6Vb53doT6b7HQ13cvx5A== X-Received: by 2002:a92:d646:: with SMTP id x6mr564564ilp.280.1631118984685; Wed, 08 Sep 2021 09:36:24 -0700 (PDT) Received: from t490s.phub.net.cable.rogers.com ([2607:fea8:56a3:500::ad7f]) by smtp.gmail.com with ESMTPSA id t11sm1363966ilf.16.2021.09.08.09.36.23 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 08 Sep 2021 09:36:24 -0700 (PDT) From: Peter Xu To: linux-kernel@vger.kernel.org, Andrew Morton , Hugh Dickins , linux-mm@kvack.org Cc: Miaohe Lin , David Hildenbrand , Andrea Arcangeli , Yang Shi , Alistair Popple , Matthew Wilcox , "Kirill A . Shutemov" , Jerome Glisse , peterx@redhat.com, Liam Howlett , Mike Rapoport Subject: [PATCH v3 3/5] mm: Drop first_index/last_index in zap_details Date: Wed, 8 Sep 2021 12:36:22 -0400 Message-Id: <20210908163622.214951-1-peterx@redhat.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210908163516.214441-1-peterx@redhat.com> References: <20210908163516.214441-1-peterx@redhat.com> MIME-Version: 1.0 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 748ECF0000BD X-Stat-Signature: ue7a3j1114nur7zzum9rnzfn3yswwnb8 Authentication-Results: imf11.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=avaIrwkI; spf=none (imf11.hostedemail.com: domain of peterx@redhat.com has no SPF policy when checking 216.205.24.124) smtp.mailfrom=peterx@redhat.com; dmarc=pass (policy=none) header.from=redhat.com X-HE-Tag: 1631118987-412311 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: The first_index/last_index parameters in zap_details are actually only used in unmap_mapping_range_tree(). At the meantime, this function is only called by unmap_mapping_pages() once. Instead of passing these two variables through the whole stack of page zapping code, remove them from zap_details and let them simply be parameters of unmap_mapping_range_tree(), which is inlined. Reviewed-by: Alistair Popple Reviewed-by: David Hildenbrand Signed-off-by: Peter Xu Reviewed-by: Liam R. Howlett --- include/linux/mm.h | 2 -- mm/memory.c | 29 ++++++++++++++++------------- 2 files changed, 16 insertions(+), 15 deletions(-) diff --git a/include/linux/mm.h b/include/linux/mm.h index 73a52aba448f..d1126f731221 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -1722,8 +1722,6 @@ extern void user_shm_unlock(size_t, struct ucounts *); */ struct zap_details { struct address_space *check_mapping; /* Check page->mapping if set */ - pgoff_t first_index; /* Lowest page->index to unmap */ - pgoff_t last_index; /* Highest page->index to unmap */ struct page *single_page; /* Locked page to be unmapped */ }; diff --git a/mm/memory.c b/mm/memory.c index 7b095f07c4ef..6bba3b9fef7c 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -3321,20 +3321,20 @@ static void unmap_mapping_range_vma(struct vm_area_struct *vma, } static inline void unmap_mapping_range_tree(struct rb_root_cached *root, + pgoff_t first_index, + pgoff_t last_index, struct zap_details *details) { struct vm_area_struct *vma; pgoff_t vba, vea, zba, zea; - vma_interval_tree_foreach(vma, root, - details->first_index, details->last_index) { - + vma_interval_tree_foreach(vma, root, first_index, last_index) { vba = vma->vm_pgoff; vea = vba + vma_pages(vma) - 1; - zba = details->first_index; + zba = first_index; if (zba < vba) zba = vba; - zea = details->last_index; + zea = last_index; if (zea > vea) zea = vea; @@ -3360,18 +3360,21 @@ void unmap_mapping_page(struct page *page) { struct address_space *mapping = page->mapping; struct zap_details details = { }; + pgoff_t first_index, last_index; VM_BUG_ON(!PageLocked(page)); VM_BUG_ON(PageTail(page)); + first_index = page->index; + last_index = page->index + thp_nr_pages(page) - 1; + details.check_mapping = mapping; - details.first_index = page->index; - details.last_index = page->index + thp_nr_pages(page) - 1; details.single_page = page; i_mmap_lock_write(mapping); if (unlikely(!RB_EMPTY_ROOT(&mapping->i_mmap.rb_root))) - unmap_mapping_range_tree(&mapping->i_mmap, &details); + unmap_mapping_range_tree(&mapping->i_mmap, first_index, + last_index, &details); i_mmap_unlock_write(mapping); } @@ -3390,17 +3393,17 @@ void unmap_mapping_page(struct page *page) void unmap_mapping_pages(struct address_space *mapping, pgoff_t start, pgoff_t nr, bool even_cows) { + pgoff_t first_index = start, last_index = start + nr - 1; struct zap_details details = { }; details.check_mapping = even_cows ? NULL : mapping; - details.first_index = start; - details.last_index = start + nr - 1; - if (details.last_index < details.first_index) - details.last_index = ULONG_MAX; + if (last_index < first_index) + last_index = ULONG_MAX; i_mmap_lock_write(mapping); if (unlikely(!RB_EMPTY_ROOT(&mapping->i_mmap.rb_root))) - unmap_mapping_range_tree(&mapping->i_mmap, &details); + unmap_mapping_range_tree(&mapping->i_mmap, first_index, + last_index, &details); i_mmap_unlock_write(mapping); } From patchwork Wed Sep 8 16:36:25 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Xu X-Patchwork-Id: 12481529 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id DF068C433F5 for ; Wed, 8 Sep 2021 16:36:31 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 8CB7060F45 for ; Wed, 8 Sep 2021 16:36:31 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 8CB7060F45 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=kvack.org Received: by kanga.kvack.org (Postfix) id 334816B0071; Wed, 8 Sep 2021 12:36:31 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 2BD1C6B0073; Wed, 8 Sep 2021 12:36:31 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 0E9A16B0074; Wed, 8 Sep 2021 12:36:31 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id F21376B0071 for ; Wed, 8 Sep 2021 12:36:30 -0400 (EDT) Received: from smtpin23.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id B4509181E5161 for ; Wed, 8 Sep 2021 16:36:30 +0000 (UTC) X-FDA: 78564959340.23.F98BFAB Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by imf09.hostedemail.com (Postfix) with ESMTP id 643B03000103 for ; Wed, 8 Sep 2021 16:36:30 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1631118989; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=56oQXq3TKm+twCnaNWrYfutLR31xlw31ilsGJKLLAyo=; b=g8DGYI6/fASSfY6bMNvvSvHRoXLTf2RYI1FWxGftSW7JgtrDeE033sVCzjIi7p0wv37BSx RXUqmhBLXkR2d5Jrs2qpjwLK8u9oib9ds1m2k8qzUxXyKsoMcQr/ri0SxvCHyj7gyy5P+v FC7vDjvf8UgZtThucgVCK1vW0la2m3Q= Received: from mail-io1-f69.google.com (mail-io1-f69.google.com [209.85.166.69]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-204-u2ynIVkhO-iWG1mm0uc7SQ-1; Wed, 08 Sep 2021 12:36:29 -0400 X-MC-Unique: u2ynIVkhO-iWG1mm0uc7SQ-1 Received: by mail-io1-f69.google.com with SMTP id e18-20020a6b7312000000b005be766a70dbso2192171ioh.19 for ; Wed, 08 Sep 2021 09:36:29 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=56oQXq3TKm+twCnaNWrYfutLR31xlw31ilsGJKLLAyo=; b=x9tZJ+jGi3TTyFOcfJkN4MIuZsBMG3U/KcRny4TewrdGRe1QOuA1kWRacM0TlsDLwn ozN3/bbbXHwTMJCL4TQcLipCu1XOeoSuvsLJkz4oLNKxNLLGeabi1M9gXABl0KEaAzHm ildH7Hp0/hjjo2Jt7zQj3O/ezAkQXmhj0WdtN7Ub/AL8iVfU//Y+KCDjS5BHuJfNN7AQ 853GvmFufmoVftGp0uySSa7+FsP8Ux6ZS5k8xTwX9CnS9MHpSxFc28oNb3kIgc+7sVJV 5ZL/qIydcloz+Ss9c0mOY3LIep4IuOwaCoSI2YBpGm1KxAebTBkfy5hO0qKYdKCw7PIO 0P/g== X-Gm-Message-State: AOAM530jWjSrOicaDkd1afiyGWTpz/5e9YXibEf5QBZXBxR1yt5VuzQm bXU14+mPpBdBTqlrVSrKbeoQ/SvqVDB/6Tz3ZQz9e67PpgGLhIbyTAEKxXyuM7VfVqkP8ZKluZE duB8XzsoVmlo= X-Received: by 2002:a02:ca0b:: with SMTP id i11mr4618043jak.84.1631118987968; Wed, 08 Sep 2021 09:36:27 -0700 (PDT) X-Google-Smtp-Source: ABdhPJzrX//krJNo8s8HBTqOX6r+b8T4NK3eiRYEGoQxukDEVUburkR0mXbCkaE95ZsNYN1utMzy5A== X-Received: by 2002:a02:ca0b:: with SMTP id i11mr4618027jak.84.1631118987784; Wed, 08 Sep 2021 09:36:27 -0700 (PDT) Received: from t490s.phub.net.cable.rogers.com ([2607:fea8:56a3:500::ad7f]) by smtp.gmail.com with ESMTPSA id m10sm1399415ilg.20.2021.09.08.09.36.26 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 08 Sep 2021 09:36:27 -0700 (PDT) From: Peter Xu To: linux-kernel@vger.kernel.org, Andrew Morton , Hugh Dickins , linux-mm@kvack.org Cc: Miaohe Lin , David Hildenbrand , Andrea Arcangeli , Yang Shi , Alistair Popple , Matthew Wilcox , "Kirill A . Shutemov" , Jerome Glisse , peterx@redhat.com, Liam Howlett , Mike Rapoport Subject: [PATCH v3 4/5] mm: Add zap_skip_check_mapping() helper Date: Wed, 8 Sep 2021 12:36:25 -0400 Message-Id: <20210908163625.215001-1-peterx@redhat.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210908163516.214441-1-peterx@redhat.com> References: <20210908163516.214441-1-peterx@redhat.com> MIME-Version: 1.0 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-Stat-Signature: g1qtqsrfpxcw7x7rtt4x6ztmmuwb5nmu Authentication-Results: imf09.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b="g8DGYI6/"; dmarc=pass (policy=none) header.from=redhat.com; spf=none (imf09.hostedemail.com: domain of peterx@redhat.com has no SPF policy when checking 170.10.133.124) smtp.mailfrom=peterx@redhat.com X-Rspamd-Server: rspam03 X-Rspamd-Queue-Id: 643B03000103 X-HE-Tag: 1631118990-570257 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Use the helper for the checks. Rename "check_mapping" into "zap_mapping" because "check_mapping" looks like a bool but in fact it stores the mapping itself. When it's set, we check the mapping (it must be non-NULL). When it's cleared we skip the check, which works like the old way. Move the duplicated comments to the helper too. Signed-off-by: Peter Xu Reviewed-by: Alistair Popple --- include/linux/mm.h | 16 +++++++++++++++- mm/memory.c | 29 ++++++----------------------- 2 files changed, 21 insertions(+), 24 deletions(-) diff --git a/include/linux/mm.h b/include/linux/mm.h index d1126f731221..ed44f31615d9 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -1721,10 +1721,24 @@ extern void user_shm_unlock(size_t, struct ucounts *); * Parameter block passed down to zap_pte_range in exceptional cases. */ struct zap_details { - struct address_space *check_mapping; /* Check page->mapping if set */ + struct address_space *zap_mapping; /* Check page->mapping if set */ struct page *single_page; /* Locked page to be unmapped */ }; +/* + * We set details->zap_mappings when we want to unmap shared but keep private + * pages. Return true if skip zapping this page, false otherwise. + */ +static inline bool +zap_skip_check_mapping(struct zap_details *details, struct page *page) +{ + if (!details || !page) + return false; + + return details->zap_mapping && + (details->zap_mapping != page_rmapping(page)); +} + struct page *vm_normal_page(struct vm_area_struct *vma, unsigned long addr, pte_t pte); struct page *vm_normal_page_pmd(struct vm_area_struct *vma, unsigned long addr, diff --git a/mm/memory.c b/mm/memory.c index 6bba3b9fef7c..e5ee8399d270 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -1333,16 +1333,8 @@ static unsigned long zap_pte_range(struct mmu_gather *tlb, struct page *page; page = vm_normal_page(vma, addr, ptent); - if (unlikely(details) && page) { - /* - * unmap_shared_mapping_pages() wants to - * invalidate cache without truncating: - * unmap shared but keep private pages. - */ - if (details->check_mapping && - details->check_mapping != page_rmapping(page)) - continue; - } + if (unlikely(zap_skip_check_mapping(details, page))) + continue; ptent = ptep_get_and_clear_full(mm, addr, pte, tlb->fullmm); tlb_remove_tlb_entry(tlb, pte, addr); @@ -1375,17 +1367,8 @@ static unsigned long zap_pte_range(struct mmu_gather *tlb, is_device_exclusive_entry(entry)) { struct page *page = pfn_swap_entry_to_page(entry); - if (unlikely(details && details->check_mapping)) { - /* - * unmap_shared_mapping_pages() wants to - * invalidate cache without truncating: - * unmap shared but keep private pages. - */ - if (details->check_mapping != - page_rmapping(page)) - continue; - } - + if (unlikely(zap_skip_check_mapping(details, page))) + continue; pte_clear_not_present_full(mm, addr, pte, tlb->fullmm); rss[mm_counter(page)]--; @@ -3368,7 +3351,7 @@ void unmap_mapping_page(struct page *page) first_index = page->index; last_index = page->index + thp_nr_pages(page) - 1; - details.check_mapping = mapping; + details.zap_mapping = mapping; details.single_page = page; i_mmap_lock_write(mapping); @@ -3396,7 +3379,7 @@ void unmap_mapping_pages(struct address_space *mapping, pgoff_t start, pgoff_t first_index = start, last_index = start + nr - 1; struct zap_details details = { }; - details.check_mapping = even_cows ? NULL : mapping; + details.zap_mapping = even_cows ? NULL : mapping; if (last_index < first_index) last_index = ULONG_MAX; From patchwork Wed Sep 8 16:36:28 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Xu X-Patchwork-Id: 12481531 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id D6985C433EF for ; Wed, 8 Sep 2021 16:36:34 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 58A4461139 for ; Wed, 8 Sep 2021 16:36:34 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 58A4461139 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=kvack.org Received: by kanga.kvack.org (Postfix) id 000CE6B0073; Wed, 8 Sep 2021 12:36:34 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id EA6B26B0074; Wed, 8 Sep 2021 12:36:33 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C3409900002; Wed, 8 Sep 2021 12:36:33 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0122.hostedemail.com [216.40.44.122]) by kanga.kvack.org (Postfix) with ESMTP id B1D926B0073 for ; Wed, 8 Sep 2021 12:36:33 -0400 (EDT) Received: from smtpin37.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id 6DB16181A0D97 for ; Wed, 8 Sep 2021 16:36:33 +0000 (UTC) X-FDA: 78564959466.37.ED5D302 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by imf27.hostedemail.com (Postfix) with ESMTP id 1633D700009B for ; Wed, 8 Sep 2021 16:36:32 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1631118992; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=awAUVK0W17m50EUHlrWKEaUEucIpwyDPaQz4TREBEy8=; b=gIdAQ6ANwrVwAX/izpXWx9pTlAHuHZCWFKtTwgi5CbjcITOX9esn9f1BGom9/zXGuR5xuy JGXzQ/+URzrPLH8w71qvzbxPX1H0yHg/BXgM83XqqfHiJv2UD8SmoiXS1l2yAQLgn0GjU7 PeoOFE7Etp8XuOhQfiVDeH4qOeUkGU8= Received: from mail-io1-f72.google.com (mail-io1-f72.google.com [209.85.166.72]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-584-Uc82gApgMw-5jsIQs2R_dg-1; Wed, 08 Sep 2021 12:36:31 -0400 X-MC-Unique: Uc82gApgMw-5jsIQs2R_dg-1 Received: by mail-io1-f72.google.com with SMTP id x26-20020a6bd01a000000b005c23c703629so2381124ioa.10 for ; Wed, 08 Sep 2021 09:36:31 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=awAUVK0W17m50EUHlrWKEaUEucIpwyDPaQz4TREBEy8=; b=1GPfJl/rsDn265d8Uku2DkMbMHOPe0yJw6FdxwY/OxKpS8Tq5tAsKijCHqzjRUpHiK 7LnE7AJ9sWSI6HNIsGGfEJeFF6vWmtN4OYZ5Pj4tA0LqWtoYABCDoNNsh3JFTZvqxI9E P1WlB1T33N2+a/1I9wAcrbt+UyWPhAhboPKWHEHS2Us3sIRFNt01XzAtedAyS+/3i5+u 15nFayzcVyd9xv2Rut+1wkNagzqvV4dUQ+LjPLNDKcXkUd0+ippJkqdoe7GwljdBcUk8 yG2MrumIER8eD38lTFVQ2zK5laDdgSp0j2sMku26LHtvFuGn7hfJnSeJcjax781H7gap +6/g== X-Gm-Message-State: AOAM533a6wSZwXestLlkPTmcnnLikWLx4rC7UM5OrmrG07AoPHzUrufr ZQa56FGjkVNup91H1QCqCUL3eCI8Pkkf8gcunnpBrJvrc7UDgtG9XD8y6+G/tTFZYLjyLUFC3Cf 8vShKdFxUst8= X-Received: by 2002:a05:6602:38e:: with SMTP id f14mr679944iov.62.1631118990920; Wed, 08 Sep 2021 09:36:30 -0700 (PDT) X-Google-Smtp-Source: ABdhPJwPBgjnxcRmDnznmj8Z8D2m8laj1F/cKtZgajr2y1D7LR0ubX5FPyZ9OQcONNyn/24a8NcA/Q== X-Received: by 2002:a05:6602:38e:: with SMTP id f14mr679909iov.62.1631118990674; Wed, 08 Sep 2021 09:36:30 -0700 (PDT) Received: from t490s.phub.net.cable.rogers.com ([2607:fea8:56a3:500::ad7f]) by smtp.gmail.com with ESMTPSA id q14sm1340458ilj.34.2021.09.08.09.36.29 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 08 Sep 2021 09:36:30 -0700 (PDT) From: Peter Xu To: linux-kernel@vger.kernel.org, Andrew Morton , Hugh Dickins , linux-mm@kvack.org Cc: Miaohe Lin , David Hildenbrand , Andrea Arcangeli , Yang Shi , Alistair Popple , Matthew Wilcox , "Kirill A . Shutemov" , Jerome Glisse , peterx@redhat.com, Liam Howlett , Mike Rapoport Subject: [PATCH v3 5/5] mm: Add ZAP_FLAG_SKIP_SWAP and zap_flags Date: Wed, 8 Sep 2021 12:36:28 -0400 Message-Id: <20210908163628.215052-1-peterx@redhat.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210908163516.214441-1-peterx@redhat.com> References: <20210908163516.214441-1-peterx@redhat.com> MIME-Version: 1.0 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Authentication-Results: imf27.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=gIdAQ6AN; spf=none (imf27.hostedemail.com: domain of peterx@redhat.com has no SPF policy when checking 170.10.133.124) smtp.mailfrom=peterx@redhat.com; dmarc=pass (policy=none) header.from=redhat.com X-Stat-Signature: q4x1nrz6fbind3dbctqqxjcgmjynahgc X-Rspamd-Server: rspam02 X-Rspamd-Queue-Id: 1633D700009B X-HE-Tag: 1631118992-671499 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Firstly, the comment in zap_pte_range() is misleading because it checks against details rather than check_mappings, so it's against what the code did. Meanwhile, there's no explicit reason why passing in the details pointer should mean to skip all swap entries. New user of zap_details could very possibly miss this fact if they don't read deep until zap_pte_range() because there's no comment at zap_details talking about it at all, so swap entries could be erroneously skipped without being noticed. This partly reverts 3e8715fdc03e ("mm: drop zap_details::check_swap_entries"), but introduce ZAP_FLAG_SKIP_SWAP flag, which means the opposite of previous "details" parameter: the caller should explicitly set this to skip swap entries, otherwise swap entries will always be considered (which should still be the major case here). We may want to look into when exactly we need ZAP_FLAG_SKIP_SWAP and we should have it in a synchronous manner, e.g., currently even if ZAP_FLAG_SKIP_SWAP is set we'll still look into swap pmds no matter what. But that should be a separate effort of this patch. The flag introduced in this patch will be a preparation for more bits defined in the future, e.g., for a new bit in flag to show whether to persist the upcoming uffd-wp bit in pgtable entries. Cc: Kirill A. Shutemov Cc: Hugh Dickins Signed-off-by: Peter Xu --- include/linux/mm.h | 16 ++++++++++++++++ mm/memory.c | 6 +++--- 2 files changed, 19 insertions(+), 3 deletions(-) diff --git a/include/linux/mm.h b/include/linux/mm.h index ed44f31615d9..beb784ce35b9 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -1717,12 +1717,18 @@ static inline bool can_do_mlock(void) { return false; } extern int user_shm_lock(size_t, struct ucounts *); extern void user_shm_unlock(size_t, struct ucounts *); +typedef unsigned int __bitwise zap_flags_t; + +/* Whether to skip zapping swap entries */ +#define ZAP_FLAG_SKIP_SWAP ((__force zap_flags_t) BIT(0)) + /* * Parameter block passed down to zap_pte_range in exceptional cases. */ struct zap_details { struct address_space *zap_mapping; /* Check page->mapping if set */ struct page *single_page; /* Locked page to be unmapped */ + zap_flags_t zap_flags; /* Extra flags for zapping */ }; /* @@ -1739,6 +1745,16 @@ zap_skip_check_mapping(struct zap_details *details, struct page *page) (details->zap_mapping != page_rmapping(page)); } +/* Return true if skip swap entries, false otherwise */ +static inline bool +zap_skip_swap(struct zap_details *details) +{ + if (!details) + return false; + + return details->zap_flags & ZAP_FLAG_SKIP_SWAP; +} + struct page *vm_normal_page(struct vm_area_struct *vma, unsigned long addr, pte_t pte); struct page *vm_normal_page_pmd(struct vm_area_struct *vma, unsigned long addr, diff --git a/mm/memory.c b/mm/memory.c index e5ee8399d270..26e37bef1888 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -1379,8 +1379,7 @@ static unsigned long zap_pte_range(struct mmu_gather *tlb, continue; } - /* If details->check_mapping, we leave swap entries. */ - if (unlikely(details)) + if (unlikely(zap_skip_swap(details))) continue; if (!non_swap_entry(entry)) @@ -3353,6 +3352,7 @@ void unmap_mapping_page(struct page *page) details.zap_mapping = mapping; details.single_page = page; + details.zap_flags = ZAP_FLAG_SKIP_SWAP; i_mmap_lock_write(mapping); if (unlikely(!RB_EMPTY_ROOT(&mapping->i_mmap.rb_root))) @@ -3377,7 +3377,7 @@ void unmap_mapping_pages(struct address_space *mapping, pgoff_t start, pgoff_t nr, bool even_cows) { pgoff_t first_index = start, last_index = start + nr - 1; - struct zap_details details = { }; + struct zap_details details = { .zap_flags = ZAP_FLAG_SKIP_SWAP }; details.zap_mapping = even_cows ? NULL : mapping; if (last_index < first_index)