From patchwork Wed Sep 15 18:14:53 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Xu X-Patchwork-Id: 12497123 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8F7E6C433F5 for ; Wed, 15 Sep 2021 18:15:05 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 0DD37600D4 for ; Wed, 15 Sep 2021 18:15:05 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 0DD37600D4 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=kvack.org Received: by kanga.kvack.org (Postfix) id A331C6B0072; Wed, 15 Sep 2021 14:15:04 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 993E76B0073; Wed, 15 Sep 2021 14:15:04 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 80D796B0074; Wed, 15 Sep 2021 14:15:04 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0005.hostedemail.com [216.40.44.5]) by kanga.kvack.org (Postfix) with ESMTP id 70C516B0072 for ; Wed, 15 Sep 2021 14:15:04 -0400 (EDT) Received: from smtpin25.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id 1686530C89 for ; Wed, 15 Sep 2021 18:15:04 +0000 (UTC) X-FDA: 78590609328.25.FCE4DDB Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by imf30.hostedemail.com (Postfix) with ESMTP id B96B7E001987 for ; Wed, 15 Sep 2021 18:15:03 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1631729703; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=/JtLLJZ1Bt4HNZ1/rns47BurBkMZGMTII6QW5YE/blU=; b=cKOlx5KAZ7I96NCCJVjLiIpPIq0Q2CI1rZbCgo67elHTFXQ3A/runrEHX6te5rNN3Un8jk 7fIHaKtHAUzHYpDpBijhkW1cZq9toTx+HR5PSwAJ4esUajslAQHT2MDaysriS34RTlHDEF CqP6sbt5y2Ao3rVTO02kxvKNhbuozLs= Received: from mail-qt1-f199.google.com (mail-qt1-f199.google.com [209.85.160.199]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-378-_DGx4e--PS-RIEvr4UNZUQ-1; Wed, 15 Sep 2021 14:15:02 -0400 X-MC-Unique: _DGx4e--PS-RIEvr4UNZUQ-1 Received: by mail-qt1-f199.google.com with SMTP id q19-20020ac87353000000b0029a09eca2afso5496188qtp.21 for ; Wed, 15 Sep 2021 11:15:01 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=/JtLLJZ1Bt4HNZ1/rns47BurBkMZGMTII6QW5YE/blU=; b=SOxIY4KT48xQMPrvAcR9rmJsG1YFL+Ejwg/0IcCa0cQLgT85pwtrVbpGsdzpnDooFa 3Es3juqhduiSilTWbE2ynuRGMzhcK9MpP4A2X3TRvJ7Z7B2F9sOlXY6Uc4FhV3Arm9fz mL+Wkoqkj2CPwJbuDQgCLu4Pr+d5x32+JjaoIwY2Ri63aJos7KxALW5FtX/bIhOBwXz7 tfO/y7dobFXKyiD1LA6/k0Dmo3MSALr+I7YDwlc8TZalz8StUJg18BJrF07vEC+/XH5M WByZbT1/3dVpVJwOn9yEo0+6i2kbF/lD5S9eiyjT6X56NcqtqnCleQ3L1kB7zeI+QB6j gFEA== X-Gm-Message-State: AOAM530h+6qXDWKtM9w9J+mlFcNEoOOqGL2VBdB6k68nWotiecbQ65Ur 0kkvM2OFUzcnxvygiP3+dnNaxOnxy7R63r2/K5Vqui4oxiZDzJIrhgva0Z+pBxzXBNh+ZfIckDL ME96U5svL9Ks= X-Received: by 2002:a37:9445:: with SMTP id w66mr1329162qkd.410.1631729700630; Wed, 15 Sep 2021 11:15:00 -0700 (PDT) X-Google-Smtp-Source: ABdhPJxUlP/bFYTOYGW/AZ/geeOWP7U0B/+waHTXzFWcNOT4AybXR2GPW/I4aEVdSb5hzua//G11kw== X-Received: by 2002:a37:9445:: with SMTP id w66mr1329128qkd.410.1631729700388; Wed, 15 Sep 2021 11:15:00 -0700 (PDT) Received: from t490s.redhat.com ([2607:fea8:56a2:9100::d35a]) by smtp.gmail.com with ESMTPSA id n18sm578663qkn.63.2021.09.15.11.14.58 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 15 Sep 2021 11:14:59 -0700 (PDT) From: Peter Xu To: Andrew Morton , linux-mm@kvack.org, linux-kernel@vger.kernel.org Cc: peterx@redhat.com, Andrea Arcangeli , Liam Howlett , Hugh Dickins , Mike Rapoport , Yang Shi , David Hildenbrand , "Kirill A . Shutemov" , Jerome Glisse , Alistair Popple , Miaohe Lin , Matthew Wilcox , Axel Rasmussen Subject: [PATCH v4 1/4] mm/shmem: Unconditionally set pte dirty in mfill_atomic_install_pte Date: Wed, 15 Sep 2021 14:14:53 -0400 Message-Id: <20210915181456.10739-2-peterx@redhat.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210915181456.10739-1-peterx@redhat.com> References: <20210915181456.10739-1-peterx@redhat.com> MIME-Version: 1.0 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Authentication-Results: imf30.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=cKOlx5KA; spf=none (imf30.hostedemail.com: domain of peterx@redhat.com has no SPF policy when checking 170.10.133.124) smtp.mailfrom=peterx@redhat.com; dmarc=pass (policy=none) header.from=redhat.com X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: B96B7E001987 X-Stat-Signature: pjjeu138j5kbc89dm156ujwcztmn1kxx X-HE-Tag: 1631729703-285733 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: It was conditionally done previously, as there's one shmem special case that we use SetPageDirty() instead. However that's not necessary and it should be easier and cleaner to do it unconditionally in mfill_atomic_install_pte(). The most recent discussion about this is here, where Hugh explained the history of SetPageDirty() and why it's possible that it's not required at all: https://lore.kernel.org/lkml/alpine.LSU.2.11.2104121657050.1097@eggly.anvils/ Currently mfill_atomic_install_pte() has three callers: 1. shmem_mfill_atomic_pte 2. mcopy_atomic_pte 3. mcontinue_atomic_pte After the change: case (1) should have its SetPageDirty replaced by the dirty bit on pte (so we unify them together, finally), case (2) should have no functional change at all as it has page_in_cache==false, case (3) may add a dirty bit to the pte. However since case (3) is UFFDIO_CONTINUE for shmem, it's merely 100% sure the page is dirty after all because UFFDIO_CONTINUE normally requires another process to modify the page cache and kick the faulted thread, so should not make a real difference either. This should make it much easier to follow on which case will set dirty for uffd, as we'll simply set it all now for all uffd related ioctls. Meanwhile, no special handling of SetPageDirty() if there's no need. Cc: Hugh Dickins Cc: Axel Rasmussen Cc: Andrea Arcangeli Reviewed-by: Axel Rasmussen Signed-off-by: Peter Xu --- mm/shmem.c | 1 - mm/userfaultfd.c | 3 +-- 2 files changed, 1 insertion(+), 3 deletions(-) diff --git a/mm/shmem.c b/mm/shmem.c index 88742953532c..96ccf6e941aa 100644 --- a/mm/shmem.c +++ b/mm/shmem.c @@ -2424,7 +2424,6 @@ int shmem_mfill_atomic_pte(struct mm_struct *dst_mm, shmem_recalc_inode(inode); spin_unlock_irq(&info->lock); - SetPageDirty(page); unlock_page(page); return 0; out_delete_from_cache: diff --git a/mm/userfaultfd.c b/mm/userfaultfd.c index 7a9008415534..caf6dfff2a60 100644 --- a/mm/userfaultfd.c +++ b/mm/userfaultfd.c @@ -69,10 +69,9 @@ int mfill_atomic_install_pte(struct mm_struct *dst_mm, pmd_t *dst_pmd, pgoff_t offset, max_off; _dst_pte = mk_pte(page, dst_vma->vm_page_prot); + _dst_pte = pte_mkdirty(_dst_pte); if (page_in_cache && !vm_shared) writable = false; - if (writable || !page_in_cache) - _dst_pte = pte_mkdirty(_dst_pte); if (writable) { if (wp_copy) _dst_pte = pte_mkuffd_wp(_dst_pte); From patchwork Wed Sep 15 18:15:33 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Xu X-Patchwork-Id: 12497125 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.5 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9E4C1C433F5 for ; Wed, 15 Sep 2021 18:15:39 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 49867611C4 for ; Wed, 15 Sep 2021 18:15:39 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 49867611C4 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=kvack.org Received: by kanga.kvack.org (Postfix) id E504D6B0073; Wed, 15 Sep 2021 14:15:38 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id DFFFB6B0074; Wed, 15 Sep 2021 14:15:38 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id CA1356B0075; Wed, 15 Sep 2021 14:15:38 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0020.hostedemail.com [216.40.44.20]) by kanga.kvack.org (Postfix) with ESMTP id BAE756B0073 for ; Wed, 15 Sep 2021 14:15:38 -0400 (EDT) Received: from smtpin05.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id BDEA018019575 for ; Wed, 15 Sep 2021 18:15:37 +0000 (UTC) X-FDA: 78590610714.05.CD41757 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by imf25.hostedemail.com (Postfix) with ESMTP id 6137DB000186 for ; Wed, 15 Sep 2021 18:15:37 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1631729736; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=yCBcgdQfSQO0vdlLaUvPYqHDlVPUHtkRpfsoC2DOf1s=; b=Mf/QXWxkd/4ede/TsgVli6b5AUOTiC3X3ODlHnZta/hMSiDROXn/fR5xh+echarxA5MiEN ZICCnXYsn+tCvFtZgniwIM6+Sh3YYF1CHg4hdthGISHMix8ECW2AGurt/udvG9f2lLIjrC lrjVsMdTHSF780yptUHIOfzngif8Ack= Received: from mail-qv1-f72.google.com (mail-qv1-f72.google.com [209.85.219.72]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-42-HVylwxcsMgqlBfRKfut6og-1; Wed, 15 Sep 2021 14:15:36 -0400 X-MC-Unique: HVylwxcsMgqlBfRKfut6og-1 Received: by mail-qv1-f72.google.com with SMTP id ib9-20020a0562141c8900b003671c3a1243so6447116qvb.21 for ; Wed, 15 Sep 2021 11:15:35 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=yCBcgdQfSQO0vdlLaUvPYqHDlVPUHtkRpfsoC2DOf1s=; b=dvbtYwqQJ8zYkDMYgHDmMEtgtIfIyf7TiRXzjlb3apRB76zHG6bObtJCg65t6Y4knW aB8f/7LzR4YOR5GXoWWRdLR+aObE4su6hSeQzZV0iLC5BAAKp58CB/rPj/hJCHFlCsNR nX9fbAGbAywnTTxT62+bdb4pf27ah+1CRaQPlgd+6Xyq+F9z1OI2T3aCZ1QQayKxxMUj fmGMZ0EAQg7bm6aIauGm5GwvoF4XF0ZSL2HcDTp9aJAQTgpbiNEzhrHgPbhmibe5hF2D dh9Mk0wMy30nIM6MQoND6BzVrJ/qMs4Ge9jiWNqn7ApIsiq+66ui3z3vhsl0XJhLBwf6 jBNA== X-Gm-Message-State: AOAM532R8kiLM3awbaK8RstdW+P1Y66R6VDeBMWDdmhly40BNPxLlpxH DgflBf+UmGYb4/2pALaTJbuZGQbR1gVAp4O9cjfouYqZZcJkTvP+AB/sfFmj12InE2oxN59RYLZ pjjBJFlpoRnA= X-Received: by 2002:ae9:e858:: with SMTP id a85mr1259140qkg.97.1631729735530; Wed, 15 Sep 2021 11:15:35 -0700 (PDT) X-Google-Smtp-Source: ABdhPJyNOKoWdJ3QZ+W2MJPuiiol1pHZKM860TkZUW+41Zl1wmq7UcHBNIiRCGRpEP4med2lISCytQ== X-Received: by 2002:ae9:e858:: with SMTP id a85mr1259106qkg.97.1631729735316; Wed, 15 Sep 2021 11:15:35 -0700 (PDT) Received: from t490s.redhat.com ([2607:fea8:56a2:9100::d35a]) by smtp.gmail.com with ESMTPSA id a9sm554724qko.27.2021.09.15.11.15.34 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 15 Sep 2021 11:15:34 -0700 (PDT) From: Peter Xu To: linux-kernel@vger.kernel.org, Andrew Morton , linux-mm@kvack.org Cc: Hugh Dickins , David Hildenbrand , Liam Howlett , Mike Rapoport , Alistair Popple , Matthew Wilcox , peterx@redhat.com, Yang Shi , "Kirill A . Shutemov" , Jerome Glisse , Miaohe Lin , Andrea Arcangeli Subject: [PATCH v4 2/4] mm: Clear vmf->pte after pte_unmap_same() returns Date: Wed, 15 Sep 2021 14:15:33 -0400 Message-Id: <20210915181533.11188-1-peterx@redhat.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210915181456.10739-1-peterx@redhat.com> References: <20210915181456.10739-1-peterx@redhat.com> MIME-Version: 1.0 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Authentication-Results: imf25.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b="Mf/QXWxk"; spf=none (imf25.hostedemail.com: domain of peterx@redhat.com has no SPF policy when checking 170.10.133.124) smtp.mailfrom=peterx@redhat.com; dmarc=pass (policy=none) header.from=redhat.com X-Stat-Signature: 7oqasd1eg5i1oa3unatb5b4ikma6qj98 X-Rspamd-Server: rspam02 X-Rspamd-Queue-Id: 6137DB000186 X-HE-Tag: 1631729737-953404 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: pte_unmap_same() will always unmap the pte pointer. After the unmap, vmf->pte will not be valid any more, we should clear it. It was safe only because no one is accessing vmf->pte after pte_unmap_same() returns, since the only caller of pte_unmap_same() (so far) is do_swap_page(), where vmf->pte will in most cases be overwritten very soon. Directly pass in vmf into pte_unmap_same() and then we can also avoid the long parameter list too, which should be a nice cleanup. Reviewed-by: Miaohe Lin Reviewed-by: David Hildenbrand Reviewed-by: Liam Howlett Signed-off-by: Peter Xu Acked-by: Hugh Dickins --- mm/memory.c | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/mm/memory.c b/mm/memory.c index 25fc46e87214..7b095f07c4ef 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -2724,19 +2724,19 @@ EXPORT_SYMBOL_GPL(apply_to_existing_page_range); * proceeding (but do_wp_page is only called after already making such a check; * and do_anonymous_page can safely check later on). */ -static inline int pte_unmap_same(struct mm_struct *mm, pmd_t *pmd, - pte_t *page_table, pte_t orig_pte) +static inline int pte_unmap_same(struct vm_fault *vmf) { int same = 1; #if defined(CONFIG_SMP) || defined(CONFIG_PREEMPTION) if (sizeof(pte_t) > sizeof(unsigned long)) { - spinlock_t *ptl = pte_lockptr(mm, pmd); + spinlock_t *ptl = pte_lockptr(vmf->vma->vm_mm, vmf->pmd); spin_lock(ptl); - same = pte_same(*page_table, orig_pte); + same = pte_same(*vmf->pte, vmf->orig_pte); spin_unlock(ptl); } #endif - pte_unmap(page_table); + pte_unmap(vmf->pte); + vmf->pte = NULL; return same; } @@ -3487,7 +3487,7 @@ vm_fault_t do_swap_page(struct vm_fault *vmf) vm_fault_t ret = 0; void *shadow = NULL; - if (!pte_unmap_same(vma->vm_mm, vmf->pmd, vmf->pte, vmf->orig_pte)) + if (!pte_unmap_same(vmf)) goto out; entry = pte_to_swp_entry(vmf->orig_pte); From patchwork Wed Sep 15 18:15:35 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Xu X-Patchwork-Id: 12497127 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.5 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 612CDC433F5 for ; Wed, 15 Sep 2021 18:15:43 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id F0BA8600D4 for ; Wed, 15 Sep 2021 18:15:42 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org F0BA8600D4 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=kvack.org Received: by kanga.kvack.org (Postfix) id 9958E6B0074; Wed, 15 Sep 2021 14:15:42 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 945486B0075; Wed, 15 Sep 2021 14:15:42 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 80DD96B0078; Wed, 15 Sep 2021 14:15:42 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0176.hostedemail.com [216.40.44.176]) by kanga.kvack.org (Postfix) with ESMTP id 713DE6B0074 for ; Wed, 15 Sep 2021 14:15:42 -0400 (EDT) Received: from smtpin17.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id 1C43E827CB77 for ; Wed, 15 Sep 2021 18:15:42 +0000 (UTC) X-FDA: 78590610924.17.D29C513 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by imf25.hostedemail.com (Postfix) with ESMTP id B6279B000185 for ; Wed, 15 Sep 2021 18:15:41 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1631729741; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=DZHPDTQPmmT4S/5dP0KaRo8sVu/1OOr/SVcw78xFuwQ=; b=HTq3HH5HZHZMHEbCBv3zhXK5ciX7kxVoGE0BpaYBipwl7J3yxH6LXBqif7LaLgp1pjhV0g pg3u2OEO4Vs5YL//zLcZq+7gcM5cUsKqDMampZM3GFWlR1LVAP9JmonBdaBdT/yRDwA0yH AK9IOMOxjsnTKeeAHVxsaNvJ4IQk86E= Received: from mail-qk1-f197.google.com (mail-qk1-f197.google.com [209.85.222.197]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-478-ZPVPw-hlNsqsftfRq75tCw-1; Wed, 15 Sep 2021 14:15:38 -0400 X-MC-Unique: ZPVPw-hlNsqsftfRq75tCw-1 Received: by mail-qk1-f197.google.com with SMTP id q13-20020a05620a038d00b003d38f784161so6054308qkm.8 for ; Wed, 15 Sep 2021 11:15:38 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=DZHPDTQPmmT4S/5dP0KaRo8sVu/1OOr/SVcw78xFuwQ=; b=sS5cLidYug5UnJy4Qa3tncg8A9rycBhH4zWd1vmcr31jMkLx7rQazwv3xnp3fEF4cC 0wZfxebwuKHWMA/GaM3Q1hJcVBkZkkiIvVgxOc9WkxQeMZAAsDCPqxlnZ9e1iYhifwAQ ZzQY+gTEgn+/3XsHhcXU5B9APusTLqw0F3G+szZedTwySbt0LWdo5U/sRmTE3hFcT67k 6Ch0m9AECvV3HKDeu1EnDaogucLimho9dK/XKFexu9HE1hUMWv2TlqonrvnZuKjB9+Yx pSfexTbAnel5wAJvJDrURuBaDQ7SCMDu8P0XFeyFCJudDZgkIdcGyMEwfRfkDb4RD/mz QyJQ== X-Gm-Message-State: AOAM533OcK7JhzwpwFJzTPVXSFI9OhMFFQ3WbCkGNLPNxnRAwn3Ps3sH wbhxUtODVfzdsPKeOao4OKimNiNih1XHRxUfT2lxa77Elgz/FN3aUje2aPQc5y0k6xXVwA7N5uG Wa1joEeMV7qc= X-Received: by 2002:a05:620a:9c9:: with SMTP id y9mr1254070qky.207.1631729737848; Wed, 15 Sep 2021 11:15:37 -0700 (PDT) X-Google-Smtp-Source: ABdhPJyCDiDDdOWKMQa0nt8YhHUXf0hiyQUvKuJ0Xc/zbcBLQR8zWPpath9jFglOz4LhTP3RVO95bA== X-Received: by 2002:a05:620a:9c9:: with SMTP id y9mr1254052qky.207.1631729737639; Wed, 15 Sep 2021 11:15:37 -0700 (PDT) Received: from t490s.redhat.com ([2607:fea8:56a2:9100::d35a]) by smtp.gmail.com with ESMTPSA id c15sm531826qka.46.2021.09.15.11.15.36 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 15 Sep 2021 11:15:37 -0700 (PDT) From: Peter Xu To: linux-kernel@vger.kernel.org, Andrew Morton , linux-mm@kvack.org Cc: Hugh Dickins , David Hildenbrand , Liam Howlett , Mike Rapoport , Alistair Popple , Matthew Wilcox , peterx@redhat.com, Yang Shi , "Kirill A . Shutemov" , Jerome Glisse , Miaohe Lin , Andrea Arcangeli Subject: [PATCH v4 3/4] mm: Drop first_index/last_index in zap_details Date: Wed, 15 Sep 2021 14:15:35 -0400 Message-Id: <20210915181535.11238-1-peterx@redhat.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210915181456.10739-1-peterx@redhat.com> References: <20210915181456.10739-1-peterx@redhat.com> MIME-Version: 1.0 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Authentication-Results: imf25.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=HTq3HH5H; spf=none (imf25.hostedemail.com: domain of peterx@redhat.com has no SPF policy when checking 170.10.133.124) smtp.mailfrom=peterx@redhat.com; dmarc=pass (policy=none) header.from=redhat.com X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: B6279B000185 X-Stat-Signature: tgd1tndmbidpicyexj5szdgqyqwi8qa3 X-HE-Tag: 1631729741-987427 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: The first_index/last_index parameters in zap_details are actually only used in unmap_mapping_range_tree(). At the meantime, this function is only called by unmap_mapping_pages() once. Instead of passing these two variables through the whole stack of page zapping code, remove them from zap_details and let them simply be parameters of unmap_mapping_range_tree(), which is inlined. Reviewed-by: Alistair Popple Reviewed-by: David Hildenbrand Reviewed-by: Liam Howlett Signed-off-by: Peter Xu Acked-by: Hugh Dickins --- include/linux/mm.h | 2 -- mm/memory.c | 31 ++++++++++++++++++------------- 2 files changed, 18 insertions(+), 15 deletions(-) diff --git a/include/linux/mm.h b/include/linux/mm.h index 73a52aba448f..d1126f731221 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -1722,8 +1722,6 @@ extern void user_shm_unlock(size_t, struct ucounts *); */ struct zap_details { struct address_space *check_mapping; /* Check page->mapping if set */ - pgoff_t first_index; /* Lowest page->index to unmap */ - pgoff_t last_index; /* Highest page->index to unmap */ struct page *single_page; /* Locked page to be unmapped */ }; diff --git a/mm/memory.c b/mm/memory.c index 7b095f07c4ef..a7e427177817 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -3321,20 +3321,20 @@ static void unmap_mapping_range_vma(struct vm_area_struct *vma, } static inline void unmap_mapping_range_tree(struct rb_root_cached *root, + pgoff_t first_index, + pgoff_t last_index, struct zap_details *details) { struct vm_area_struct *vma; pgoff_t vba, vea, zba, zea; - vma_interval_tree_foreach(vma, root, - details->first_index, details->last_index) { - + vma_interval_tree_foreach(vma, root, first_index, last_index) { vba = vma->vm_pgoff; vea = vba + vma_pages(vma) - 1; - zba = details->first_index; + zba = first_index; if (zba < vba) zba = vba; - zea = details->last_index; + zea = last_index; if (zea > vea) zea = vea; @@ -3360,18 +3360,22 @@ void unmap_mapping_page(struct page *page) { struct address_space *mapping = page->mapping; struct zap_details details = { }; + pgoff_t first_index; + pgoff_t last_index; VM_BUG_ON(!PageLocked(page)); VM_BUG_ON(PageTail(page)); + first_index = page->index; + last_index = page->index + thp_nr_pages(page) - 1; + details.check_mapping = mapping; - details.first_index = page->index; - details.last_index = page->index + thp_nr_pages(page) - 1; details.single_page = page; i_mmap_lock_write(mapping); if (unlikely(!RB_EMPTY_ROOT(&mapping->i_mmap.rb_root))) - unmap_mapping_range_tree(&mapping->i_mmap, &details); + unmap_mapping_range_tree(&mapping->i_mmap, first_index, + last_index, &details); i_mmap_unlock_write(mapping); } @@ -3391,16 +3395,17 @@ void unmap_mapping_pages(struct address_space *mapping, pgoff_t start, pgoff_t nr, bool even_cows) { struct zap_details details = { }; + pgoff_t first_index = start; + pgoff_t last_index = start + nr - 1; details.check_mapping = even_cows ? NULL : mapping; - details.first_index = start; - details.last_index = start + nr - 1; - if (details.last_index < details.first_index) - details.last_index = ULONG_MAX; + if (last_index < first_index) + last_index = ULONG_MAX; i_mmap_lock_write(mapping); if (unlikely(!RB_EMPTY_ROOT(&mapping->i_mmap.rb_root))) - unmap_mapping_range_tree(&mapping->i_mmap, &details); + unmap_mapping_range_tree(&mapping->i_mmap, first_index, + last_index, &details); i_mmap_unlock_write(mapping); } From patchwork Wed Sep 15 18:15:38 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Xu X-Patchwork-Id: 12497129 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.5 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 54AF8C433FE for ; Wed, 15 Sep 2021 18:15:44 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 07CD6611C8 for ; Wed, 15 Sep 2021 18:15:44 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 07CD6611C8 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=kvack.org Received: by kanga.kvack.org (Postfix) id 2DF356B0075; Wed, 15 Sep 2021 14:15:43 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 269A16B0078; Wed, 15 Sep 2021 14:15:43 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 130BB900002; Wed, 15 Sep 2021 14:15:43 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0120.hostedemail.com [216.40.44.120]) by kanga.kvack.org (Postfix) with ESMTP id 05A7F6B0075 for ; Wed, 15 Sep 2021 14:15:43 -0400 (EDT) Received: from smtpin19.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id BD3A8827CB89 for ; Wed, 15 Sep 2021 18:15:42 +0000 (UTC) X-FDA: 78590610924.19.5C170AB Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by imf27.hostedemail.com (Postfix) with ESMTP id 7236B70000A7 for ; Wed, 15 Sep 2021 18:15:42 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1631729742; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=zoZpGeMMIEvvXfo6qC7dlVaUPLxDIKE3MfxvS4uRdds=; b=djmRIoDEjEahe3PJl5fB5Q5Qtlh5rPP6YdrL8kespaLujAtLZu/FQQuJVPgRiJLLmXy1N5 oESAlas8ywcHVagpFcFbhO+XM+w/mmcodOEYerSE9jZUtG7EcIN52wjoHKh38swgWAwnvg W1B31UotEdqdx9D43iCzovj36vtLWmA= Received: from mail-qt1-f198.google.com (mail-qt1-f198.google.com [209.85.160.198]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-76-YgPuYLOqOq-Qg7GN3OFR1A-1; Wed, 15 Sep 2021 14:15:41 -0400 X-MC-Unique: YgPuYLOqOq-Qg7GN3OFR1A-1 Received: by mail-qt1-f198.google.com with SMTP id o3-20020a05622a008300b002a480a6b8f4so5534832qtw.15 for ; Wed, 15 Sep 2021 11:15:40 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=zoZpGeMMIEvvXfo6qC7dlVaUPLxDIKE3MfxvS4uRdds=; b=1ZUH2sBpqKtMyMUI/1oYU+MbYNyYquv7Y/J+4Wvkxg9KOdqUSPDT4Mhbw6g+3S+DD2 ti33e4q088lEVNpprCXPF9+YopZiP8rcF+OmAVMxVHV4HcvzybJJ9uMwfQDr1UWY050U Z/3+SkDh/3VkSib8s8Apfz/EHMKJ10sVGtY1bYmLjvbblhyMiveJVo6PnuoIJH6R0Jz9 frnflwD+LeqA0IOSsgMXp3cNCyqXuKfmG9hLLPK0ZzgTOcXDdgaYxAxRHSIUjnL2TlKz pSTLP1/o/XxhrMV5WwaDnlSYVAC4kGFfUinSA2gC7glbHvEIma+Aghe9DJubEU2q4ddY cUFg== X-Gm-Message-State: AOAM532l/e0YJA7NsbyFYWAGLxpfCnn2AtrWd92qXJGKS22qslM4YljB uFW7rcXwPkVo6CNJiMS8+3JB8mRxu0/5qilqRHbU3WSj1j37rNA71fntoPBjYUW3iHhKKUK6nPD ivuVe3g2zVXk= X-Received: by 2002:a0c:dd87:: with SMTP id v7mr1295504qvk.45.1631729740338; Wed, 15 Sep 2021 11:15:40 -0700 (PDT) X-Google-Smtp-Source: ABdhPJz6Lyt46ecPogmobxQZSQVhoZPuKmRpDao/8QgvngGXdt4WSs2hAmQxzkWJfSwJwCmhKbjYdA== X-Received: by 2002:a0c:dd87:: with SMTP id v7mr1295473qvk.45.1631729740083; Wed, 15 Sep 2021 11:15:40 -0700 (PDT) Received: from t490s.redhat.com ([2607:fea8:56a2:9100::d35a]) by smtp.gmail.com with ESMTPSA id 13sm534908qka.56.2021.09.15.11.15.38 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 15 Sep 2021 11:15:39 -0700 (PDT) From: Peter Xu To: linux-kernel@vger.kernel.org, Andrew Morton , linux-mm@kvack.org Cc: Hugh Dickins , David Hildenbrand , Liam Howlett , Mike Rapoport , Alistair Popple , Matthew Wilcox , peterx@redhat.com, Yang Shi , "Kirill A . Shutemov" , Jerome Glisse , Miaohe Lin , Andrea Arcangeli Subject: [PATCH v4 4/4] mm: Add zap_skip_check_mapping() helper Date: Wed, 15 Sep 2021 14:15:38 -0400 Message-Id: <20210915181538.11288-1-peterx@redhat.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210915181456.10739-1-peterx@redhat.com> References: <20210915181456.10739-1-peterx@redhat.com> MIME-Version: 1.0 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Authentication-Results: imf27.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=djmRIoDE; spf=none (imf27.hostedemail.com: domain of peterx@redhat.com has no SPF policy when checking 170.10.133.124) smtp.mailfrom=peterx@redhat.com; dmarc=pass (policy=none) header.from=redhat.com X-Stat-Signature: ai19oobrdrceujyndwa48kba6k5t8mbu X-Rspamd-Server: rspam02 X-Rspamd-Queue-Id: 7236B70000A7 X-HE-Tag: 1631729742-791755 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Use the helper for the checks. Rename "check_mapping" into "zap_mapping" because "check_mapping" looks like a bool but in fact it stores the mapping itself. When it's set, we check the mapping (it must be non-NULL). When it's cleared we skip the check, which works like the old way. Move the duplicated comments to the helper too. Reviewed-by: Alistair Popple Signed-off-by: Peter Xu --- include/linux/mm.h | 16 +++++++++++++++- mm/memory.c | 29 ++++++----------------------- 2 files changed, 21 insertions(+), 24 deletions(-) diff --git a/include/linux/mm.h b/include/linux/mm.h index d1126f731221..ed44f31615d9 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -1721,10 +1721,24 @@ extern void user_shm_unlock(size_t, struct ucounts *); * Parameter block passed down to zap_pte_range in exceptional cases. */ struct zap_details { - struct address_space *check_mapping; /* Check page->mapping if set */ + struct address_space *zap_mapping; /* Check page->mapping if set */ struct page *single_page; /* Locked page to be unmapped */ }; +/* + * We set details->zap_mappings when we want to unmap shared but keep private + * pages. Return true if skip zapping this page, false otherwise. + */ +static inline bool +zap_skip_check_mapping(struct zap_details *details, struct page *page) +{ + if (!details || !page) + return false; + + return details->zap_mapping && + (details->zap_mapping != page_rmapping(page)); +} + struct page *vm_normal_page(struct vm_area_struct *vma, unsigned long addr, pte_t pte); struct page *vm_normal_page_pmd(struct vm_area_struct *vma, unsigned long addr, diff --git a/mm/memory.c b/mm/memory.c index a7e427177817..8db8ce0ca6ce 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -1333,16 +1333,8 @@ static unsigned long zap_pte_range(struct mmu_gather *tlb, struct page *page; page = vm_normal_page(vma, addr, ptent); - if (unlikely(details) && page) { - /* - * unmap_shared_mapping_pages() wants to - * invalidate cache without truncating: - * unmap shared but keep private pages. - */ - if (details->check_mapping && - details->check_mapping != page_rmapping(page)) - continue; - } + if (unlikely(zap_skip_check_mapping(details, page))) + continue; ptent = ptep_get_and_clear_full(mm, addr, pte, tlb->fullmm); tlb_remove_tlb_entry(tlb, pte, addr); @@ -1375,17 +1367,8 @@ static unsigned long zap_pte_range(struct mmu_gather *tlb, is_device_exclusive_entry(entry)) { struct page *page = pfn_swap_entry_to_page(entry); - if (unlikely(details && details->check_mapping)) { - /* - * unmap_shared_mapping_pages() wants to - * invalidate cache without truncating: - * unmap shared but keep private pages. - */ - if (details->check_mapping != - page_rmapping(page)) - continue; - } - + if (unlikely(zap_skip_check_mapping(details, page))) + continue; pte_clear_not_present_full(mm, addr, pte, tlb->fullmm); rss[mm_counter(page)]--; @@ -3369,7 +3352,7 @@ void unmap_mapping_page(struct page *page) first_index = page->index; last_index = page->index + thp_nr_pages(page) - 1; - details.check_mapping = mapping; + details.zap_mapping = mapping; details.single_page = page; i_mmap_lock_write(mapping); @@ -3398,7 +3381,7 @@ void unmap_mapping_pages(struct address_space *mapping, pgoff_t start, pgoff_t first_index = start; pgoff_t last_index = start + nr - 1; - details.check_mapping = even_cows ? NULL : mapping; + details.zap_mapping = even_cows ? NULL : mapping; if (last_index < first_index) last_index = ULONG_MAX;