From patchwork Thu Sep 2 20:17:17 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Xu X-Patchwork-Id: 12472435 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 50297C433FE for ; Thu, 2 Sep 2021 20:17:31 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id E6ECA6112F for ; Thu, 2 Sep 2021 20:17:30 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org E6ECA6112F Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=kvack.org Received: by kanga.kvack.org (Postfix) id 8ED496B0074; Thu, 2 Sep 2021 16:17:30 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 89BFE6B0073; Thu, 2 Sep 2021 16:17:30 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 73BF48D0001; Thu, 2 Sep 2021 16:17:30 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0205.hostedemail.com [216.40.44.205]) by kanga.kvack.org (Postfix) with ESMTP id 634BE6B0072 for ; Thu, 2 Sep 2021 16:17:30 -0400 (EDT) Received: from smtpin08.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id 113DC250A9 for ; Thu, 2 Sep 2021 20:17:30 +0000 (UTC) X-FDA: 78543743460.08.76A83D5 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [216.205.24.124]) by imf10.hostedemail.com (Postfix) with ESMTP id A84FD6001989 for ; Thu, 2 Sep 2021 20:17:29 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1630613849; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=uUFjqTEK/Z5AKo1Agn+ijWYpHJREXXiYmYHdAc5BzAA=; b=Z6qEfofjm51JWDcdvlkG7yT+cluxLiNcmIbB9fjakupIWZUXhaY/pCK16i6ejV1kEPfaoh aVKksy5OtKg7M4hsAhm4Kex6n1BY8id+4RSSZDxHH6o7IOZJI8wEppn2MN8qQSTeuq+LCi 0s21ENSoxyanDNu64aSok0IkzxOkxZE= Received: from mail-qv1-f72.google.com (mail-qv1-f72.google.com [209.85.219.72]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-451-iqWFpY24OLKP9fKxaHbWxA-1; Thu, 02 Sep 2021 16:17:26 -0400 X-MC-Unique: iqWFpY24OLKP9fKxaHbWxA-1 Received: by mail-qv1-f72.google.com with SMTP id u8-20020a0cee88000000b00363b89e1c50so3420400qvr.16 for ; Thu, 02 Sep 2021 13:17:26 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=uUFjqTEK/Z5AKo1Agn+ijWYpHJREXXiYmYHdAc5BzAA=; b=hhKXB6ZvCWL9mv9Nr77xmJwobJsvXell/npkGjlcQ0S4HfeqtZzN6M3I3S6nwZDLGo aUHXVhXfNAN9YpH2HjvFoLTRlG++MTAQU2GYD3DLCDUFABJPrJ767CCZssIWP0AnEexU /oSeWxoU3Fz2DsMYCtEraeaNCccZGMg5qWIyz2uYZGZH8/MIJvjgJk/uTYHPd6v8Szfr gKsiOUYkm8K+VLGS3n72ehp22sa0JdooAK587h1io97xASWwLRnJyqSi7NtxiXNPdxNn NGSv0I2crcV95k7uL5V5oiHJQXjR2P6k4zM94T70XpTzUGTLaFiAG4BlzpLaXZEBJG/A xl3Q== X-Gm-Message-State: AOAM5319kr0GA62jbASTIYlHb5UUgfjolzYmvaMgMd5HCvSO9IwTmS7H myv+6MnKnMgtcz+s1yQI2FHBdbTmwGzbOw7qfU95dJ/sANYY/Of9iH6Sz7FXPDoKojmmgLXadOQ 8jy6NE5uo0ek= X-Received: by 2002:ac8:4891:: with SMTP id i17mr124470qtq.321.1630613845666; Thu, 02 Sep 2021 13:17:25 -0700 (PDT) X-Google-Smtp-Source: ABdhPJywigrvnlxKpAN78Q/CgPMddaGPkdQXUBfT3wFLvHKkKpWp9wEzSSYfVzG1sJzo6z/t9ArXKA== X-Received: by 2002:ac8:4891:: with SMTP id i17mr124450qtq.321.1630613845467; Thu, 02 Sep 2021 13:17:25 -0700 (PDT) Received: from t490s.redhat.com ([2607:fea8:56a3:500::ad7f]) by smtp.gmail.com with ESMTPSA id m8sm2219962qkk.130.2021.09.02.13.17.23 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 02 Sep 2021 13:17:24 -0700 (PDT) From: Peter Xu To: linux-kernel@vger.kernel.org, Andrew Morton , Hugh Dickins , linux-mm@kvack.org Cc: Andrea Arcangeli , Yang Shi , Matthew Wilcox , peterx@redhat.com, Jerome Glisse , Mike Rapoport , "Kirill A . Shutemov" , Miaohe Lin , David Hildenbrand , Alistair Popple , Axel Rasmussen Subject: [PATCH v2 1/5] mm/shmem: Unconditionally set pte dirty in mfill_atomic_install_pte Date: Thu, 2 Sep 2021 16:17:17 -0400 Message-Id: <20210902201721.52796-2-peterx@redhat.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210902201721.52796-1-peterx@redhat.com> References: <20210902201721.52796-1-peterx@redhat.com> MIME-Version: 1.0 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Authentication-Results: imf10.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=Z6qEfofj; spf=none (imf10.hostedemail.com: domain of peterx@redhat.com has no SPF policy when checking 216.205.24.124) smtp.mailfrom=peterx@redhat.com; dmarc=pass (policy=none) header.from=redhat.com X-Stat-Signature: nr9df8ks3g69ce3mfz9yagfkhk5zsmkc X-Rspamd-Queue-Id: A84FD6001989 X-Rspamd-Server: rspam04 X-HE-Tag: 1630613849-287949 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: It was conditionally done previously, as there's one shmem special case that we use SetPageDirty() instead. However that's not necessary and it should be easier and cleaner to do it unconditionally in mfill_atomic_install_pte(). The most recent discussion about this is here, where Hugh explained the history of SetPageDirty() and why it's possible that it's not required at all: https://lore.kernel.org/lkml/alpine.LSU.2.11.2104121657050.1097@eggly.anvils/ Currently mfill_atomic_install_pte() has three callers: 1. shmem_mfill_atomic_pte 2. mcopy_atomic_pte 3. mcontinue_atomic_pte After the change: case (1) should have its SetPageDirty replaced by the dirty bit on pte (so we unify them together, finally), case (2) should have no functional change at all as it has page_in_cache==false, case (3) may add a dirty bit to the pte. However since case (3) is UFFDIO_CONTINUE for shmem, it's merely 100% sure the page is dirty after all, so should not make a real difference either. This should make it much easier to follow on which case will set dirty for uffd, as we'll simply set it all now for all uffd related ioctls. Meanwhile, no special handling of SetPageDirty() if there's no need. Cc: Hugh Dickins Cc: Axel Rasmussen Cc: Andrea Arcangeli Signed-off-by: Peter Xu --- mm/shmem.c | 1 - mm/userfaultfd.c | 3 +-- 2 files changed, 1 insertion(+), 3 deletions(-) diff --git a/mm/shmem.c b/mm/shmem.c index dacda7463d54..3f91c8ce4d02 100644 --- a/mm/shmem.c +++ b/mm/shmem.c @@ -2437,7 +2437,6 @@ int shmem_mfill_atomic_pte(struct mm_struct *dst_mm, shmem_recalc_inode(inode); spin_unlock_irq(&info->lock); - SetPageDirty(page); unlock_page(page); return 0; out_delete_from_cache: diff --git a/mm/userfaultfd.c b/mm/userfaultfd.c index 0e2132834bc7..b30a3724c701 100644 --- a/mm/userfaultfd.c +++ b/mm/userfaultfd.c @@ -69,10 +69,9 @@ int mfill_atomic_install_pte(struct mm_struct *dst_mm, pmd_t *dst_pmd, pgoff_t offset, max_off; _dst_pte = mk_pte(page, dst_vma->vm_page_prot); + _dst_pte = pte_mkdirty(_dst_pte); if (page_in_cache && !vm_shared) writable = false; - if (writable || !page_in_cache) - _dst_pte = pte_mkdirty(_dst_pte); if (writable) { if (wp_copy) _dst_pte = pte_mkuffd_wp(_dst_pte); From patchwork Thu Sep 2 20:17:18 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Xu X-Patchwork-Id: 12472437 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id F00A0C4332F for ; Thu, 2 Sep 2021 20:17:32 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id A55F96112F for ; Thu, 2 Sep 2021 20:17:32 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org A55F96112F Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=kvack.org Received: by kanga.kvack.org (Postfix) id 3BA626B0072; Thu, 2 Sep 2021 16:17:32 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 36A8B8D0002; Thu, 2 Sep 2021 16:17:32 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 147F48D0001; Thu, 2 Sep 2021 16:17:32 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0164.hostedemail.com [216.40.44.164]) by kanga.kvack.org (Postfix) with ESMTP id 01E446B0072 for ; Thu, 2 Sep 2021 16:17:32 -0400 (EDT) Received: from smtpin18.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id BC35F18206D51 for ; Thu, 2 Sep 2021 20:17:31 +0000 (UTC) X-FDA: 78543743502.18.7BB65E0 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by imf13.hostedemail.com (Postfix) with ESMTP id 6353B1025AAA for ; Thu, 2 Sep 2021 20:17:31 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1630613850; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=cdgHKq2iasOqwEaJuLz/zWtoLSwDiCtJqQzcku8x6HU=; b=TwPQ00gODxU40fRAGWPm3usf2penegvakzc7y4mpHYoEOI+L92/RehVinwo+0vBqFzR99g h/sXM0WjOacUEuZxxC2wbkPLgilwT96cESkY8LUI7S5Mfw/jfxogijwUAefUT/UYgrM7pI iOFSoFszouaQJaM5r4F8y2uPDmfKbjk= Received: from mail-qv1-f69.google.com (mail-qv1-f69.google.com [209.85.219.69]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-532-4Xmd3b0jPk6JXukw1y3Zcw-1; Thu, 02 Sep 2021 16:17:27 -0400 X-MC-Unique: 4Xmd3b0jPk6JXukw1y3Zcw-1 Received: by mail-qv1-f69.google.com with SMTP id l12-20020a0cc20c000000b0037766e5daaeso3435012qvh.10 for ; Thu, 02 Sep 2021 13:17:27 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=cdgHKq2iasOqwEaJuLz/zWtoLSwDiCtJqQzcku8x6HU=; b=bF/eMfy2OSapZY5i7qS6MUfLkmTE/AcoI5g+qqYguqOpXWWTzNJECWGiPCab4q5K9f MWMKf2LPuNYgrbOpvhq0exDNdNik0LMoJ8Xa6Q3di/E66Ur0eSuiksDQfs8O1TniLT3X psl9MRngwewZ+C51RIsi1ORHPvUbNMK0qAgnfv3qJtDc62VOp1xPz+6tjkOELZa3nRt1 /OT4DcBtBTFnNGFC6UPjyF+tEYPsxnWIAx+RLt1NB+ji+cBZtdkc+cvDBF+2hOqy4Yoe c5F+eVzS9c/W+nibCB56sDuE+j6Oswf/m/NZfqQ+M93xb9vnsPKgpNPvQKurNPxuyU7P v0nw== X-Gm-Message-State: AOAM530YfUFeTYT+YsO08ZnErhF38BPNoVM+T8nnX7AXOHfesYeseEek N9SbKTajDF5ObgciOiyZgP7jyB/SkPzIl3dYn67seFFVC00scz/JXyMld7TTFqFlQQ8qJXiE6yf khnJ58kAdVvI= X-Received: by 2002:ae9:dd43:: with SMTP id r64mr5073148qkf.225.1630613847435; Thu, 02 Sep 2021 13:17:27 -0700 (PDT) X-Google-Smtp-Source: ABdhPJy538vhBDKUaB7nJWBgP4Dij+Fv0SB/UrIvpqCgIyHMHGnI8luhqctobjoYIX8mKB8YeK+nsw== X-Received: by 2002:ae9:dd43:: with SMTP id r64mr5073117qkf.225.1630613847180; Thu, 02 Sep 2021 13:17:27 -0700 (PDT) Received: from t490s.redhat.com ([2607:fea8:56a3:500::ad7f]) by smtp.gmail.com with ESMTPSA id m8sm2219962qkk.130.2021.09.02.13.17.25 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 02 Sep 2021 13:17:26 -0700 (PDT) From: Peter Xu To: linux-kernel@vger.kernel.org, Andrew Morton , Hugh Dickins , linux-mm@kvack.org Cc: Andrea Arcangeli , Yang Shi , Matthew Wilcox , peterx@redhat.com, Jerome Glisse , Mike Rapoport , "Kirill A . Shutemov" , Miaohe Lin , David Hildenbrand , Alistair Popple Subject: [PATCH v2 2/5] mm: Clear vmf->pte after pte_unmap_same() returns Date: Thu, 2 Sep 2021 16:17:18 -0400 Message-Id: <20210902201721.52796-3-peterx@redhat.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210902201721.52796-1-peterx@redhat.com> References: <20210902201721.52796-1-peterx@redhat.com> MIME-Version: 1.0 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Authentication-Results: imf13.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=TwPQ00gO; dmarc=pass (policy=none) header.from=redhat.com; spf=none (imf13.hostedemail.com: domain of peterx@redhat.com has no SPF policy when checking 170.10.133.124) smtp.mailfrom=peterx@redhat.com X-Rspamd-Server: rspam03 X-Rspamd-Queue-Id: 6353B1025AAA X-Stat-Signature: zen4tr91usc1s6btnj348gf7o83jonp8 X-HE-Tag: 1630613851-613337 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: pte_unmap_same() will always unmap the pte pointer. After the unmap, vmf->pte will not be valid any more, we should clear it. It was safe only because no one is accessing vmf->pte after pte_unmap_same() returns, since the only caller of pte_unmap_same() (so far) is do_swap_page(), where vmf->pte will in most cases be overwritten very soon. Directly pass in vmf into pte_unmap_same() and then we can also avoid the long parameter list too, which should be a nice cleanup. Reviewed-by: Miaohe Lin Reviewed-by: David Hildenbrand Signed-off-by: Peter Xu Reviewed-by: Liam R. Howlett --- mm/memory.c | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/mm/memory.c b/mm/memory.c index 25fc46e87214..7b095f07c4ef 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -2724,19 +2724,19 @@ EXPORT_SYMBOL_GPL(apply_to_existing_page_range); * proceeding (but do_wp_page is only called after already making such a check; * and do_anonymous_page can safely check later on). */ -static inline int pte_unmap_same(struct mm_struct *mm, pmd_t *pmd, - pte_t *page_table, pte_t orig_pte) +static inline int pte_unmap_same(struct vm_fault *vmf) { int same = 1; #if defined(CONFIG_SMP) || defined(CONFIG_PREEMPTION) if (sizeof(pte_t) > sizeof(unsigned long)) { - spinlock_t *ptl = pte_lockptr(mm, pmd); + spinlock_t *ptl = pte_lockptr(vmf->vma->vm_mm, vmf->pmd); spin_lock(ptl); - same = pte_same(*page_table, orig_pte); + same = pte_same(*vmf->pte, vmf->orig_pte); spin_unlock(ptl); } #endif - pte_unmap(page_table); + pte_unmap(vmf->pte); + vmf->pte = NULL; return same; } @@ -3487,7 +3487,7 @@ vm_fault_t do_swap_page(struct vm_fault *vmf) vm_fault_t ret = 0; void *shadow = NULL; - if (!pte_unmap_same(vma->vm_mm, vmf->pmd, vmf->pte, vmf->orig_pte)) + if (!pte_unmap_same(vmf)) goto out; entry = pte_to_swp_entry(vmf->orig_pte); From patchwork Thu Sep 2 20:17:19 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Xu X-Patchwork-Id: 12472439 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9E16AC433FE for ; Thu, 2 Sep 2021 20:17:34 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 54FD461131 for ; Thu, 2 Sep 2021 20:17:34 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 54FD461131 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=kvack.org Received: by kanga.kvack.org (Postfix) id 4D10D8D0001; Thu, 2 Sep 2021 16:17:33 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 3E0646B0075; Thu, 2 Sep 2021 16:17:33 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 2592C8D0001; Thu, 2 Sep 2021 16:17:33 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0214.hostedemail.com [216.40.44.214]) by kanga.kvack.org (Postfix) with ESMTP id 0B6DC6B0073 for ; Thu, 2 Sep 2021 16:17:33 -0400 (EDT) Received: from smtpin25.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id B2EC8183DEA84 for ; Thu, 2 Sep 2021 20:17:32 +0000 (UTC) X-FDA: 78543743544.25.63A11C4 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [216.205.24.124]) by imf21.hostedemail.com (Postfix) with ESMTP id 5D4DDD0257B8 for ; Thu, 2 Sep 2021 20:17:32 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1630613851; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=RIXlqNC3vu4lq1txSwh5vWDNYVkPD87pLVis9u4Q2wY=; b=MhT0FYGDSeGuwSZPVzG2sRiTLhFu1azjGGAji1P3oblt77jdG6gbVLYzxoc+xVonqkp/Sl Fu/QgQapwAA+866CUVTPFV0ktRGI7AS/Td0PzRnYB9iwCalirMlbEl76lEpPho69vTBz6h cGIuXKduSrHHcnITHnoLRNhtj5i+89Q= Received: from mail-qv1-f71.google.com (mail-qv1-f71.google.com [209.85.219.71]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-51-JvFvkZYvOq-mUyIsXUSsrQ-1; Thu, 02 Sep 2021 16:17:31 -0400 X-MC-Unique: JvFvkZYvOq-mUyIsXUSsrQ-1 Received: by mail-qv1-f71.google.com with SMTP id j11-20020a0cf9cb000000b00375f0642d2dso3455549qvo.12 for ; Thu, 02 Sep 2021 13:17:31 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=RIXlqNC3vu4lq1txSwh5vWDNYVkPD87pLVis9u4Q2wY=; b=X6DNTbgN76pQD0Nd44FVOuljnesl+6bZL+GvMWVTeAo1LDTACee+6B4X/+GqjOsziS crAMEr/NtG7klG6t6cRcH1NJqKmtNWJX3LUdGFQNFrkn6lGo/Q5uxoak26whC4XsO246 /z6SJ85clXOtty0fuewmz9yiGdnDr964YOSC4o4ow88jeubhUJCpK+/cVyQ28SX/nmFu 2phHkkHkylggVT7YQK9iieCA5ywnc7cdgrhRvH4ohaogKWg6kNy92XfIEP+k7vOyx3kG c2GyRhhbhHBLQJbr9Iqmw/7YwAFPSgiQd0X4iyttGIYdUBMY11hM3DGiHNQH5gmp5oYE 24Rg== X-Gm-Message-State: AOAM531rtzBRYPrjgRthueMycVH0R2CsScTNOvGgdgPyJr1dV++/U+IQ /764RFPtTIMRAEIPt0Qu2bDrqR2DuiNCZCyXVPxCDMcHmB3yZngbeVy7hzckRsT4yHTg7o0UURF g29lDlRtolZc= X-Received: by 2002:ac8:7516:: with SMTP id u22mr144385qtq.119.1630613849398; Thu, 02 Sep 2021 13:17:29 -0700 (PDT) X-Google-Smtp-Source: ABdhPJwiZpw2RoD+jqSjYVE5EubqqLTA6EzNyQnnsyCqxpeuFnkF3mZbnsg1EEivZeiha6W/LYzBQg== X-Received: by 2002:ac8:7516:: with SMTP id u22mr144359qtq.119.1630613849206; Thu, 02 Sep 2021 13:17:29 -0700 (PDT) Received: from t490s.redhat.com ([2607:fea8:56a3:500::ad7f]) by smtp.gmail.com with ESMTPSA id m8sm2219962qkk.130.2021.09.02.13.17.27 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 02 Sep 2021 13:17:28 -0700 (PDT) From: Peter Xu To: linux-kernel@vger.kernel.org, Andrew Morton , Hugh Dickins , linux-mm@kvack.org Cc: Andrea Arcangeli , Yang Shi , Matthew Wilcox , peterx@redhat.com, Jerome Glisse , Mike Rapoport , "Kirill A . Shutemov" , Miaohe Lin , David Hildenbrand , Alistair Popple Subject: [PATCH v2 3/5] mm: Drop first_index/last_index in zap_details Date: Thu, 2 Sep 2021 16:17:19 -0400 Message-Id: <20210902201721.52796-4-peterx@redhat.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210902201721.52796-1-peterx@redhat.com> References: <20210902201721.52796-1-peterx@redhat.com> MIME-Version: 1.0 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Authentication-Results: imf21.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=MhT0FYGD; spf=none (imf21.hostedemail.com: domain of peterx@redhat.com has no SPF policy when checking 216.205.24.124) smtp.mailfrom=peterx@redhat.com; dmarc=pass (policy=none) header.from=redhat.com X-Rspamd-Server: rspam02 X-Rspamd-Queue-Id: 5D4DDD0257B8 X-Stat-Signature: 7c5cxnswzqpbhm96jq8zm59memydtq3g X-HE-Tag: 1630613852-775966 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: The first_index/last_index parameters in zap_details are actually only used in unmap_mapping_range_tree(). At the meantime, this function is only called by unmap_mapping_pages() once. Instead of passing these two variables through the whole stack of page zapping code, remove them from zap_details and let them simply be parameters of unmap_mapping_range_tree(), which is inlined. Reviewed-by: Alistair Popple Reviewed-by: David Hildenbrand Signed-off-by: Peter Xu --- include/linux/mm.h | 2 -- mm/memory.c | 29 ++++++++++++++++------------- 2 files changed, 16 insertions(+), 15 deletions(-) diff --git a/include/linux/mm.h b/include/linux/mm.h index 7ca22e6e694a..69259229f090 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -1721,8 +1721,6 @@ extern void user_shm_unlock(size_t, struct ucounts *); */ struct zap_details { struct address_space *check_mapping; /* Check page->mapping if set */ - pgoff_t first_index; /* Lowest page->index to unmap */ - pgoff_t last_index; /* Highest page->index to unmap */ struct page *single_page; /* Locked page to be unmapped */ }; diff --git a/mm/memory.c b/mm/memory.c index 7b095f07c4ef..6bba3b9fef7c 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -3321,20 +3321,20 @@ static void unmap_mapping_range_vma(struct vm_area_struct *vma, } static inline void unmap_mapping_range_tree(struct rb_root_cached *root, + pgoff_t first_index, + pgoff_t last_index, struct zap_details *details) { struct vm_area_struct *vma; pgoff_t vba, vea, zba, zea; - vma_interval_tree_foreach(vma, root, - details->first_index, details->last_index) { - + vma_interval_tree_foreach(vma, root, first_index, last_index) { vba = vma->vm_pgoff; vea = vba + vma_pages(vma) - 1; - zba = details->first_index; + zba = first_index; if (zba < vba) zba = vba; - zea = details->last_index; + zea = last_index; if (zea > vea) zea = vea; @@ -3360,18 +3360,21 @@ void unmap_mapping_page(struct page *page) { struct address_space *mapping = page->mapping; struct zap_details details = { }; + pgoff_t first_index, last_index; VM_BUG_ON(!PageLocked(page)); VM_BUG_ON(PageTail(page)); + first_index = page->index; + last_index = page->index + thp_nr_pages(page) - 1; + details.check_mapping = mapping; - details.first_index = page->index; - details.last_index = page->index + thp_nr_pages(page) - 1; details.single_page = page; i_mmap_lock_write(mapping); if (unlikely(!RB_EMPTY_ROOT(&mapping->i_mmap.rb_root))) - unmap_mapping_range_tree(&mapping->i_mmap, &details); + unmap_mapping_range_tree(&mapping->i_mmap, first_index, + last_index, &details); i_mmap_unlock_write(mapping); } @@ -3390,17 +3393,17 @@ void unmap_mapping_page(struct page *page) void unmap_mapping_pages(struct address_space *mapping, pgoff_t start, pgoff_t nr, bool even_cows) { + pgoff_t first_index = start, last_index = start + nr - 1; struct zap_details details = { }; details.check_mapping = even_cows ? NULL : mapping; - details.first_index = start; - details.last_index = start + nr - 1; - if (details.last_index < details.first_index) - details.last_index = ULONG_MAX; + if (last_index < first_index) + last_index = ULONG_MAX; i_mmap_lock_write(mapping); if (unlikely(!RB_EMPTY_ROOT(&mapping->i_mmap.rb_root))) - unmap_mapping_range_tree(&mapping->i_mmap, &details); + unmap_mapping_range_tree(&mapping->i_mmap, first_index, + last_index, &details); i_mmap_unlock_write(mapping); } From patchwork Thu Sep 2 20:18:19 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Xu X-Patchwork-Id: 12472441 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1D156C433F5 for ; Thu, 2 Sep 2021 20:18:27 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id C77F86113C for ; Thu, 2 Sep 2021 20:18:26 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org C77F86113C Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=kvack.org Received: by kanga.kvack.org (Postfix) id 69FC3900002; Thu, 2 Sep 2021 16:18:26 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 64FDC6B0075; Thu, 2 Sep 2021 16:18:26 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 53E98900002; Thu, 2 Sep 2021 16:18:26 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0164.hostedemail.com [216.40.44.164]) by kanga.kvack.org (Postfix) with ESMTP id 459B26B0073 for ; Thu, 2 Sep 2021 16:18:26 -0400 (EDT) Received: from smtpin20.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id 1318C180AD5DA for ; Thu, 2 Sep 2021 20:18:26 +0000 (UTC) X-FDA: 78543745812.20.55E8754 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [216.205.24.124]) by imf23.hostedemail.com (Postfix) with ESMTP id A0AF390000A5 for ; Thu, 2 Sep 2021 20:18:25 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1630613905; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=qa2t+mRnex8UWtPa+3EibP8DEZqC9c+ivTe+/7D5DBg=; b=DSuVf5+oX2SIgqr11ovEu2Ar31WQRDdtWSdarlrVP5SkPx7Eipk9DcgNg893VubwbZo+u9 E0Ae2232qrnpAIUOE13hrvr3hhrEy0cJmo6irfc/TWja/R7Ppecu6axuOVlB5KKPA4Z+Ah VORPC1ROrsme9m7roPwfoPFi6bkTXH0= Received: from mail-qt1-f197.google.com (mail-qt1-f197.google.com [209.85.160.197]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-551-umolfnXkNRq5_RNWAd4tdQ-1; Thu, 02 Sep 2021 16:18:24 -0400 X-MC-Unique: umolfnXkNRq5_RNWAd4tdQ-1 Received: by mail-qt1-f197.google.com with SMTP id m6-20020ac807c6000000b0029994381c5fso2824582qth.5 for ; Thu, 02 Sep 2021 13:18:24 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=qa2t+mRnex8UWtPa+3EibP8DEZqC9c+ivTe+/7D5DBg=; b=WZQzJnKPJus6khxaVz5sKO99Ow4FI9jjp6pf5EGLbx7mWSWKvHOYb6UMw99GkB2Bgo GlMWl9vxqbKIOcmpnpOKoebj/p9JwqIa7lzw/khTG8/jNAm7NWKeyRlvFjB2ecO6XQtw v+t7Ty5mBlG54b2/l+gtygl/FnxVlxI9ZQkSzIJA7LVdjBfT1kG03q41duukqdC1CXMo zbfwE6mA3f5IyG+BG1/xSFXdS67gtDdng2PMCjXHaeHlbr9urIyWpA3ITgbsEg1q2MVH XxI8ECw3I0tDbNVk9k2TANPTUk1NJ8Jqz65TfakSNzH2cyVOyKKa/gHOyrMggjoxAkzH dDPA== X-Gm-Message-State: AOAM532Cfk0JVbKTuSn2490ZZYzx/vYrVeE80p644PIgT6DvLN8jyKTY GppS8YnE16D2RPkZ9ljvjWAJLVH/LHXghFn+OXXK+7Puu0sO6bgTQK1F5Oib7FAqYGEes8wD+ON q+fQiB9pThpc= X-Received: by 2002:a05:620a:9dc:: with SMTP id y28mr4818036qky.456.1630613901841; Thu, 02 Sep 2021 13:18:21 -0700 (PDT) X-Google-Smtp-Source: ABdhPJyxSSUlX72Wtt37bFhaHAG9fsAucLiqInaoLBA2qPxUGl8ambA+3CiEsyV5pXsHgu6akQvj9g== X-Received: by 2002:a05:620a:9dc:: with SMTP id y28mr4818012qky.456.1630613901634; Thu, 02 Sep 2021 13:18:21 -0700 (PDT) Received: from t490s.redhat.com ([2607:fea8:56a3:500::ad7f]) by smtp.gmail.com with ESMTPSA id x29sm1677629qtv.74.2021.09.02.13.18.20 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 02 Sep 2021 13:18:21 -0700 (PDT) From: Peter Xu To: Hugh Dickins , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org Cc: "Kirill A . Shutemov" , Jerome Glisse , peterx@redhat.com, Andrea Arcangeli , Miaohe Lin , Yang Shi , Matthew Wilcox , Alistair Popple , Mike Rapoport , David Hildenbrand Subject: [PATCH v2 4/5] mm: Add zap_skip_check_mapping() helper Date: Thu, 2 Sep 2021 16:18:19 -0400 Message-Id: <20210902201819.53343-1-peterx@redhat.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210902201721.52796-1-peterx@redhat.com> References: <20210902201721.52796-1-peterx@redhat.com> MIME-Version: 1.0 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Authentication-Results: imf23.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=DSuVf5+o; spf=none (imf23.hostedemail.com: domain of peterx@redhat.com has no SPF policy when checking 216.205.24.124) smtp.mailfrom=peterx@redhat.com; dmarc=pass (policy=none) header.from=redhat.com X-Stat-Signature: cgxb47i5aybtqd67n4xca76obawwoja1 X-Rspamd-Queue-Id: A0AF390000A5 X-Rspamd-Server: rspam04 X-HE-Tag: 1630613905-390979 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Use the helper for the checks. Rename "check_mapping" into "zap_mapping" because "check_mapping" looks like a bool but in fact it stores the mapping itself. When it's set, we check the mapping (it must be non-NULL). When it's cleared we skip the check, which works like the old way. Move the duplicated comments to the helper too. Signed-off-by: Peter Xu --- include/linux/mm.h | 15 ++++++++++++++- mm/memory.c | 29 ++++++----------------------- 2 files changed, 20 insertions(+), 24 deletions(-) diff --git a/include/linux/mm.h b/include/linux/mm.h index 69259229f090..81e402a5fbc9 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -1720,10 +1720,23 @@ extern void user_shm_unlock(size_t, struct ucounts *); * Parameter block passed down to zap_pte_range in exceptional cases. */ struct zap_details { - struct address_space *check_mapping; /* Check page->mapping if set */ + struct address_space *zap_mapping; /* Check page->mapping if set */ struct page *single_page; /* Locked page to be unmapped */ }; +/* + * We set details->zap_mappings when we want to unmap shared but keep private + * pages. Return true if skip zapping this page, false otherwise. + */ +static inline bool +zap_skip_check_mapping(struct zap_details *details, struct page *page) +{ + if (!details || !page) + return false; + + return details->zap_mapping != page_rmapping(page); +} + struct page *vm_normal_page(struct vm_area_struct *vma, unsigned long addr, pte_t pte); struct page *vm_normal_page_pmd(struct vm_area_struct *vma, unsigned long addr, diff --git a/mm/memory.c b/mm/memory.c index 6bba3b9fef7c..e5ee8399d270 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -1333,16 +1333,8 @@ static unsigned long zap_pte_range(struct mmu_gather *tlb, struct page *page; page = vm_normal_page(vma, addr, ptent); - if (unlikely(details) && page) { - /* - * unmap_shared_mapping_pages() wants to - * invalidate cache without truncating: - * unmap shared but keep private pages. - */ - if (details->check_mapping && - details->check_mapping != page_rmapping(page)) - continue; - } + if (unlikely(zap_skip_check_mapping(details, page))) + continue; ptent = ptep_get_and_clear_full(mm, addr, pte, tlb->fullmm); tlb_remove_tlb_entry(tlb, pte, addr); @@ -1375,17 +1367,8 @@ static unsigned long zap_pte_range(struct mmu_gather *tlb, is_device_exclusive_entry(entry)) { struct page *page = pfn_swap_entry_to_page(entry); - if (unlikely(details && details->check_mapping)) { - /* - * unmap_shared_mapping_pages() wants to - * invalidate cache without truncating: - * unmap shared but keep private pages. - */ - if (details->check_mapping != - page_rmapping(page)) - continue; - } - + if (unlikely(zap_skip_check_mapping(details, page))) + continue; pte_clear_not_present_full(mm, addr, pte, tlb->fullmm); rss[mm_counter(page)]--; @@ -3368,7 +3351,7 @@ void unmap_mapping_page(struct page *page) first_index = page->index; last_index = page->index + thp_nr_pages(page) - 1; - details.check_mapping = mapping; + details.zap_mapping = mapping; details.single_page = page; i_mmap_lock_write(mapping); @@ -3396,7 +3379,7 @@ void unmap_mapping_pages(struct address_space *mapping, pgoff_t start, pgoff_t first_index = start, last_index = start + nr - 1; struct zap_details details = { }; - details.check_mapping = even_cows ? NULL : mapping; + details.zap_mapping = even_cows ? NULL : mapping; if (last_index < first_index) last_index = ULONG_MAX; From patchwork Thu Sep 2 20:18:36 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Xu X-Patchwork-Id: 12472443 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 66D16C433F5 for ; Thu, 2 Sep 2021 20:18:42 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 1333961131 for ; Thu, 2 Sep 2021 20:18:42 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 1333961131 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=kvack.org Received: by kanga.kvack.org (Postfix) id B338E8D0001; Thu, 2 Sep 2021 16:18:41 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id AE48E6B0072; Thu, 2 Sep 2021 16:18:41 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 9D45E8D0001; Thu, 2 Sep 2021 16:18:41 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0178.hostedemail.com [216.40.44.178]) by kanga.kvack.org (Postfix) with ESMTP id 8D69B6B0071 for ; Thu, 2 Sep 2021 16:18:41 -0400 (EDT) Received: from smtpin37.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id 50C652922E for ; Thu, 2 Sep 2021 20:18:41 +0000 (UTC) X-FDA: 78543746442.37.BF09298 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [216.205.24.124]) by imf19.hostedemail.com (Postfix) with ESMTP id E7394B000093 for ; Thu, 2 Sep 2021 20:18:40 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1630613920; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=TOAJdCsxdWRdNb0ovj0EkYvPsElPPpeSCrnOTkOMPhM=; b=eynOuJ1S0pLU1CHJ8Ll4IzlJgv1hHyi6gvrQ9ahWvdanBFGGRR+VYX0mZCwW6CzCKtMqtt 3IneTl8jgRUkAoTH65Rv9ouzr5oUFLOj8knHyRghqwKxk74qYUuDc0IvNNSyB7fbP37/Oh /q7kRrTVU7G+MQwYhxRERPh0/CTfQAk= Received: from mail-qk1-f200.google.com (mail-qk1-f200.google.com [209.85.222.200]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-487-x3dnSdzFPYmh0XXPFmoWzQ-1; Thu, 02 Sep 2021 16:18:39 -0400 X-MC-Unique: x3dnSdzFPYmh0XXPFmoWzQ-1 Received: by mail-qk1-f200.google.com with SMTP id 70-20020a370b49000000b003d2f5f0dcc6so3490813qkl.9 for ; Thu, 02 Sep 2021 13:18:39 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=TOAJdCsxdWRdNb0ovj0EkYvPsElPPpeSCrnOTkOMPhM=; b=eRiMMJOg4DzNxZW66sgWNeBZcun0xxQPFJL3A6/621NnxgjKNPHAD1vfWdy1B7jTcK vlf1y0GehbWprg9CK4cN+tMXxuAxh+hqPhklBTEnvo2rkl1JkJps4nmyvZpjDmrQn1pX t1cPg61DKa+tK8nHHfwdJEBj7HEb5XfLGn8R5Mjf5g/W79LJ22VC1uFhhyuD1zf2JUVx ZOBsoPG7ogOo2C4ZsLWYbNZNubnoBZfeebokTKDS5J44rFqOemcXQtjCUSx9woUBo/e+ QLckhQtUdTTome7b/pZejxGEzyCdGcC6Tw7V7NaqPuIfJ0D4cv96oZn1ybbngtX4CTd9 +TCQ== X-Gm-Message-State: AOAM530Ynr71wp77zDREAXRVRD8d/CLy7scKM90V+STonZ6CtLJ3syd8 mbS1wnnNIh7SEuKWeARrJTU8+ImthRVOKNReFAwaNPIPIHUaiE6VSndBEUVzrL1Yk6UPeWQArbU vW+f6UxkEFhCBpYg3++mrosAsoC407GCl+zJ1u7/oaq0z3bxhhTLMlXwej9/G X-Received: by 2002:ac8:6601:: with SMTP id c1mr143881qtp.179.1630613918644; Thu, 02 Sep 2021 13:18:38 -0700 (PDT) X-Google-Smtp-Source: ABdhPJx++efQTIvasHbdh2PP0MzWi88Mh6mwdgtTP9JqFrJvmkxBs6mFuxVsRBJhqxz9qndbwxPGvg== X-Received: by 2002:ac8:6601:: with SMTP id c1mr143838qtp.179.1630613918342; Thu, 02 Sep 2021 13:18:38 -0700 (PDT) Received: from t490s.redhat.com ([2607:fea8:56a3:500::ad7f]) by smtp.gmail.com with ESMTPSA id c4sm2381805qkf.122.2021.09.02.13.18.36 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 02 Sep 2021 13:18:37 -0700 (PDT) From: Peter Xu To: linux-mm@kvack.org, Hugh Dickins , Andrew Morton , linux-kernel@vger.kernel.org Cc: Miaohe Lin , Matthew Wilcox , David Hildenbrand , Yang Shi , "Kirill A . Shutemov" , peterx@redhat.com, Jerome Glisse , Alistair Popple , Andrea Arcangeli , Mike Rapoport Subject: [PATCH v2 5/5] mm: Add ZAP_FLAG_SKIP_SWAP and zap_flags Date: Thu, 2 Sep 2021 16:18:36 -0400 Message-Id: <20210902201836.53605-1-peterx@redhat.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210902201721.52796-1-peterx@redhat.com> References: <20210902201721.52796-1-peterx@redhat.com> MIME-Version: 1.0 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Authentication-Results: imf19.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=eynOuJ1S; spf=none (imf19.hostedemail.com: domain of peterx@redhat.com has no SPF policy when checking 216.205.24.124) smtp.mailfrom=peterx@redhat.com; dmarc=pass (policy=none) header.from=redhat.com X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: E7394B000093 X-Stat-Signature: ttgqjm9c1nfowq91ge3k5f55k6cpwis7 X-HE-Tag: 1630613920-860489 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Firstly, the comment in zap_pte_range() is misleading because it checks against details rather than check_mappings, so it's against what the code did. Meanwhile, it's confusing too on not explaining why passing in the details pointer would mean to skip all swap entries. New user of zap_details could very possibly miss this fact if they don't read deep until zap_pte_range() because there's no comment at zap_details talking about it at all, so swap entries could be errornously skipped without being noticed. Actually very recently we introduced unmap_mapping_page() in 22061a1ffabd, I think that should also look into swap entries. Add a comment there. IOW, this patch will be a functional change to unmap_mapping_page() but hopefully in the right way to do it. This partly reverts 3e8715fdc03e ("mm: drop zap_details::check_swap_entries"), but introduce ZAP_FLAG_SKIP_SWAP flag, which means the opposite of previous "details" parameter: the caller should explicitly set this to skip swap entries, otherwise swap entries will always be considered (which should still be the major case here). Cc: Kirill A. Shutemov Cc: Hugh Dickins Signed-off-by: Peter Xu --- include/linux/mm.h | 16 ++++++++++++++++ mm/memory.c | 6 +++--- 2 files changed, 19 insertions(+), 3 deletions(-) diff --git a/include/linux/mm.h b/include/linux/mm.h index 81e402a5fbc9..a7bcdb2ec956 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -1716,12 +1716,18 @@ static inline bool can_do_mlock(void) { return false; } extern int user_shm_lock(size_t, struct ucounts *); extern void user_shm_unlock(size_t, struct ucounts *); +typedef unsigned int __bitwise zap_flags_t; + +/* Whether to skip zapping swap entries */ +#define ZAP_FLAG_SKIP_SWAP ((__force zap_flags_t) BIT(0)) + /* * Parameter block passed down to zap_pte_range in exceptional cases. */ struct zap_details { struct address_space *zap_mapping; /* Check page->mapping if set */ struct page *single_page; /* Locked page to be unmapped */ + zap_flags_t zap_flags; /* Extra flags for zapping */ }; /* @@ -1737,6 +1743,16 @@ zap_skip_check_mapping(struct zap_details *details, struct page *page) return details->zap_mapping != page_rmapping(page); } +/* Return true if skip swap entries, false otherwise */ +static inline bool +zap_skip_swap(struct zap_details *details) +{ + if (!details) + return false; + + return details->zap_flags & ZAP_FLAG_SKIP_SWAP; +} + struct page *vm_normal_page(struct vm_area_struct *vma, unsigned long addr, pte_t pte); struct page *vm_normal_page_pmd(struct vm_area_struct *vma, unsigned long addr, diff --git a/mm/memory.c b/mm/memory.c index e5ee8399d270..4cb269ca8249 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -1379,8 +1379,7 @@ static unsigned long zap_pte_range(struct mmu_gather *tlb, continue; } - /* If details->check_mapping, we leave swap entries. */ - if (unlikely(details)) + if (unlikely(zap_skip_swap(details))) continue; if (!non_swap_entry(entry)) @@ -3351,6 +3350,7 @@ void unmap_mapping_page(struct page *page) first_index = page->index; last_index = page->index + thp_nr_pages(page) - 1; + /* Keep ZAP_FLAG_SKIP_SWAP cleared because we're truncating */ details.zap_mapping = mapping; details.single_page = page; @@ -3377,7 +3377,7 @@ void unmap_mapping_pages(struct address_space *mapping, pgoff_t start, pgoff_t nr, bool even_cows) { pgoff_t first_index = start, last_index = start + nr - 1; - struct zap_details details = { }; + struct zap_details details = { .zap_flags = ZAP_FLAG_SKIP_SWAP }; details.zap_mapping = even_cows ? NULL : mapping; if (last_index < first_index)