From patchwork Fri Mar 4 05:16:50 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Xu X-Patchwork-Id: 12768460 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0E47AC433EF for ; Fri, 4 Mar 2022 05:18:07 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 9CE1B8D0008; Fri, 4 Mar 2022 00:18:06 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 97CE38D0001; Fri, 4 Mar 2022 00:18:06 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 844CC8D0008; Fri, 4 Mar 2022 00:18:06 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0135.hostedemail.com [216.40.44.135]) by kanga.kvack.org (Postfix) with ESMTP id 773468D0001 for ; Fri, 4 Mar 2022 00:18:06 -0500 (EST) Received: from smtpin26.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id 2C9BF181C43D5 for ; Fri, 4 Mar 2022 05:18:06 +0000 (UTC) X-FDA: 79205547372.26.C065020 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by imf04.hostedemail.com (Postfix) with ESMTP id 8C8E340002 for ; Fri, 4 Mar 2022 05:18:05 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1646371085; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=J4WHOp4As9jU9WPKckCAt0i8DCpDD3uwt3O/JTJCiOk=; b=RktsEAEDP/2f4cOdtwXBsjhK8pSK4gjhzAHMGFPlipEIJRVhmCXZm4tTYqgQkzuFaCXGQR RBD59udbfJQQPf5D/qcTlqn3xNyad93bV2VsRzTvooCdCMjd3nwfGWzQF73G8bjvVBW01U RBu2yeaXWZwACt6Ee3luIc4WCqcrHV4= Received: from mail-pj1-f71.google.com (mail-pj1-f71.google.com [209.85.216.71]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-163-Cz-7GtBdMvW0eAudAKlSWQ-1; Fri, 04 Mar 2022 00:18:04 -0500 X-MC-Unique: Cz-7GtBdMvW0eAudAKlSWQ-1 Received: by mail-pj1-f71.google.com with SMTP id t10-20020a17090a5d8a00b001bed9556134so6714702pji.5 for ; Thu, 03 Mar 2022 21:18:04 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=J4WHOp4As9jU9WPKckCAt0i8DCpDD3uwt3O/JTJCiOk=; b=fLHNJtbsuKDi2NhEA7zO7Km3EBGaiyj7zAZBdZUgCEPzUB8aPsctru0fopc4Tg6lBA l2syM1QMQh2JLclbh6OD+LX6ssfoXjR96nc2Ds+TMTkV7COZFMGfdampi9m5EJtKIt4q AF6VGP5p3GaZM1zryqWXQQGmlCzuwQ4G2x6Loj8eVn/F/fEqOxdG6I6KCJCoNvMqDRTO UkJsOB5vIed1N0MwkR+JvpT76ORGVtD03zwgHaApaWsFDDS2GeqSqS9aJpMnRuunXHfW trruFBH0HP/x8nAMBGgg3diDSmIg7gwOzoX6IeN0sHtQPJOSmQXDimgi0CJHzC4R62nn Kfuw== X-Gm-Message-State: AOAM532HkqY9YhjmrZmXj7YOjpXyuFMgYn6Oa/hNEd6jFuob5PQUW1H+ gfIlBi0x5cNjKgfvXXeBJcxux1/rJNfrkiD9u+xIRXou6XlioBRpRXM5DtwK1qvbrBW4ADI6b8v H4oROcFHVns5EzkRki9Fdigfoyifw0I7AAJqGmG20JoHHVtNjtnro15cDy/O9 X-Received: by 2002:a63:8648:0:b0:37c:8fc8:ae4 with SMTP id x69-20020a638648000000b0037c8fc80ae4mr2421613pgd.482.1646371083010; Thu, 03 Mar 2022 21:18:03 -0800 (PST) X-Google-Smtp-Source: ABdhPJznlFXBVvf7x27XrfMAEWi1fvHMcNPQxFuHlxUyI/+59HbIcVzbyw8INKOOkmh2hJfkrZdmyg== X-Received: by 2002:a63:8648:0:b0:37c:8fc8:ae4 with SMTP id x69-20020a638648000000b0037c8fc80ae4mr2421585pgd.482.1646371082478; Thu, 03 Mar 2022 21:18:02 -0800 (PST) Received: from localhost.localdomain ([94.177.118.59]) by smtp.gmail.com with ESMTPSA id p16-20020a056a000b5000b004f669806cd9sm4323865pfo.87.2022.03.03.21.17.54 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Thu, 03 Mar 2022 21:18:02 -0800 (PST) From: Peter Xu To: linux-mm@kvack.org, linux-kernel@vger.kernel.org Cc: peterx@redhat.com, Nadav Amit , Hugh Dickins , David Hildenbrand , Axel Rasmussen , Matthew Wilcox , Alistair Popple , Mike Rapoport , Andrew Morton , Jerome Glisse , Mike Kravetz , "Kirill A . Shutemov" , Andrea Arcangeli Subject: [PATCH v7 05/23] mm/shmem: Take care of UFFDIO_COPY_MODE_WP Date: Fri, 4 Mar 2022 13:16:50 +0800 Message-Id: <20220304051708.86193-6-peterx@redhat.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20220304051708.86193-1-peterx@redhat.com> References: <20220304051708.86193-1-peterx@redhat.com> MIME-Version: 1.0 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-Rspamd-Server: rspam11 X-Rspamd-Queue-Id: 8C8E340002 X-Rspam-User: Authentication-Results: imf04.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=RktsEAED; dmarc=pass (policy=none) header.from=redhat.com; spf=none (imf04.hostedemail.com: domain of peterx@redhat.com has no SPF policy when checking 170.10.129.124) smtp.mailfrom=peterx@redhat.com X-Stat-Signature: 9531ypi7mn7r3nsrkr3rtrj4nyh3ffxh X-HE-Tag: 1646371085-827044 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Pass wp_copy into shmem_mfill_atomic_pte() through the stack, then apply the UFFD_WP bit properly when the UFFDIO_COPY on shmem is with UFFDIO_COPY_MODE_WP. wp_copy lands mfill_atomic_install_pte() finally. Note: we must do pte_wrprotect() if !writable in mfill_atomic_install_pte(), as mk_pte() could return a writable pte (e.g., when VM_SHARED on a shmem file). Signed-off-by: Peter Xu --- include/linux/shmem_fs.h | 4 ++-- mm/shmem.c | 4 ++-- mm/userfaultfd.c | 23 ++++++++++++++++++----- 3 files changed, 22 insertions(+), 9 deletions(-) diff --git a/include/linux/shmem_fs.h b/include/linux/shmem_fs.h index ab51d3cd39bd..02d23ce5f979 100644 --- a/include/linux/shmem_fs.h +++ b/include/linux/shmem_fs.h @@ -145,11 +145,11 @@ extern int shmem_mfill_atomic_pte(struct mm_struct *dst_mm, pmd_t *dst_pmd, struct vm_area_struct *dst_vma, unsigned long dst_addr, unsigned long src_addr, - bool zeropage, + bool zeropage, bool wp_copy, struct page **pagep); #else /* !CONFIG_SHMEM */ #define shmem_mfill_atomic_pte(dst_mm, dst_pmd, dst_vma, dst_addr, \ - src_addr, zeropage, pagep) ({ BUG(); 0; }) + src_addr, zeropage, wp_copy, pagep) ({ BUG(); 0; }) #endif /* CONFIG_SHMEM */ #endif /* CONFIG_USERFAULTFD */ diff --git a/mm/shmem.c b/mm/shmem.c index 81a69bd247b4..3f0332c1c1e9 100644 --- a/mm/shmem.c +++ b/mm/shmem.c @@ -2317,7 +2317,7 @@ int shmem_mfill_atomic_pte(struct mm_struct *dst_mm, struct vm_area_struct *dst_vma, unsigned long dst_addr, unsigned long src_addr, - bool zeropage, + bool zeropage, bool wp_copy, struct page **pagep) { struct inode *inode = file_inode(dst_vma->vm_file); @@ -2390,7 +2390,7 @@ int shmem_mfill_atomic_pte(struct mm_struct *dst_mm, goto out_release; ret = mfill_atomic_install_pte(dst_mm, dst_pmd, dst_vma, dst_addr, - page, true, false); + page, true, wp_copy); if (ret) goto out_delete_from_cache; diff --git a/mm/userfaultfd.c b/mm/userfaultfd.c index e9bb6db002aa..ef418a48b121 100644 --- a/mm/userfaultfd.c +++ b/mm/userfaultfd.c @@ -77,10 +77,19 @@ int mfill_atomic_install_pte(struct mm_struct *dst_mm, pmd_t *dst_pmd, * Always mark a PTE as write-protected when needed, regardless of * VM_WRITE, which the user might change. */ - if (wp_copy) + if (wp_copy) { _dst_pte = pte_mkuffd_wp(_dst_pte); - else if (writable) + writable = false; + } + + if (writable) _dst_pte = pte_mkwrite(_dst_pte); + else + /* + * We need this to make sure write bit removed; as mk_pte() + * could return a pte with write bit set. + */ + _dst_pte = pte_wrprotect(_dst_pte); dst_pte = pte_offset_map_lock(dst_mm, dst_pmd, dst_addr, &ptl); @@ -95,7 +104,12 @@ int mfill_atomic_install_pte(struct mm_struct *dst_mm, pmd_t *dst_pmd, } ret = -EEXIST; - if (!pte_none(*dst_pte)) + /* + * We allow to overwrite a pte marker: consider when both MISSING|WP + * registered, we firstly wr-protect a none pte which has no page cache + * page backing it, then access the page. + */ + if (!pte_none_mostly(*dst_pte)) goto out_unlock; if (page_in_cache) { @@ -479,11 +493,10 @@ static __always_inline ssize_t mfill_atomic_pte(struct mm_struct *dst_mm, err = mfill_zeropage_pte(dst_mm, dst_pmd, dst_vma, dst_addr); } else { - VM_WARN_ON_ONCE(wp_copy); err = shmem_mfill_atomic_pte(dst_mm, dst_pmd, dst_vma, dst_addr, src_addr, mode != MCOPY_ATOMIC_NORMAL, - page); + wp_copy, page); } return err;