From patchwork Tue Apr 5 01:48:41 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Xu X-Patchwork-Id: 12800961 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8EA96C433F5 for ; Tue, 5 Apr 2022 01:50:23 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 4D9266B0074; Mon, 4 Apr 2022 21:48:57 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 488F86B0078; Mon, 4 Apr 2022 21:48:57 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 350BE6B007B; Mon, 4 Apr 2022 21:48:57 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (relay.hostedemail.com [64.99.140.26]) by kanga.kvack.org (Postfix) with ESMTP id 28ACD6B0074 for ; Mon, 4 Apr 2022 21:48:57 -0400 (EDT) Received: from smtpin11.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay11.hostedemail.com (Postfix) with ESMTP id 0CA4A80942 for ; Tue, 5 Apr 2022 01:48:47 +0000 (UTC) X-FDA: 79321141494.11.8CD638A Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by imf17.hostedemail.com (Postfix) with ESMTP id 6C0764002E for ; Tue, 5 Apr 2022 01:48:46 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1649123326; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=U7N++YETabzz5IcvkFNQ6jfBTPxFyQ6g10gwAR5sfFo=; b=FSASRzSia783uJ+4TvkkkzXdC0fjMQPBAjTyOA0tpmlwvlTRZcJ6wP2LWKzl/b/Hqr03eq ROB0JQr31O+bE5Mh8I0IT1PLlk8XyuTq9jlIkKMF86rmM25cC2v51Y+dRh0LsqvxNxSosv zxV11vRqF1OFqvuIAJR7QWmj+zHkUPo= Received: from mail-il1-f199.google.com (mail-il1-f199.google.com [209.85.166.199]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-403-G9SBHhaFNbKTcVCV7l8nLw-1; Mon, 04 Apr 2022 21:48:45 -0400 X-MC-Unique: G9SBHhaFNbKTcVCV7l8nLw-1 Received: by mail-il1-f199.google.com with SMTP id r16-20020a056e02109000b002ca35f87493so3272833ilj.22 for ; Mon, 04 Apr 2022 18:48:45 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=U7N++YETabzz5IcvkFNQ6jfBTPxFyQ6g10gwAR5sfFo=; b=lkbQf6pNuihGmd70R89VXoqYsvoWjIUUrgxelRQ/5vmaHFiRegVrzzL8K2CWo0pExf GFLq/sR76h5S4GgqTcMk0ROc705ebH14Exhdyj/H/DVzNZf614B33VztqewYJkf++UPL l1L9TkUA5A5f1ArjDMWSrFa5/jWe8YLGHqpuKk9JgV//LMDXvAlDC1UW7W0Np1loiaOU FnofHk3W6y7YivWkJe24jCKmZHTI5BgSMlixGHBDtp+97KxzAHJLyb9oTGBJpu8xZU/V oqqbE7reZNyUnvmDfcz1jLUEH4p04xfAvHct6rz1r8ZAo12ADr+uJ5RNDE47pWxtNY8i Q+lw== X-Gm-Message-State: AOAM53354XbjTo3M/Xbe3OKSvFlLDZczg1BvlTkfytiBeYEQyAFdC4rV NOR9VTyvqBvoLF2wkvTXEqeQdXbsjbccnJ10IT5V0xSqfrW9/C2gVqTF+KIsSgE0fuRgGZf0maZ 0lG/AlhRh3pI= X-Received: by 2002:a02:ccdb:0:b0:321:2cf8:8c70 with SMTP id k27-20020a02ccdb000000b003212cf88c70mr736086jaq.32.1649123324364; Mon, 04 Apr 2022 18:48:44 -0700 (PDT) X-Google-Smtp-Source: ABdhPJwdVvGICqrYSIFeMsXS/EOPSh7BPG/ifQSrmjaNr8HpCpVJTxhXefsGedxvCcLNau5XeR/6Pg== X-Received: by 2002:a02:ccdb:0:b0:321:2cf8:8c70 with SMTP id k27-20020a02ccdb000000b003212cf88c70mr736070jaq.32.1649123324113; Mon, 04 Apr 2022 18:48:44 -0700 (PDT) Received: from localhost.localdomain (cpec09435e3e0ee-cmc09435e3e0ec.cpe.net.cable.rogers.com. [99.241.198.116]) by smtp.gmail.com with ESMTPSA id ay18-20020a5d9d92000000b0064c77f6aaecsm7925169iob.3.2022.04.04.18.48.42 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Mon, 04 Apr 2022 18:48:43 -0700 (PDT) From: Peter Xu To: linux-kernel@vger.kernel.org, linux-mm@kvack.org Cc: Mike Kravetz , Nadav Amit , Matthew Wilcox , Mike Rapoport , David Hildenbrand , Hugh Dickins , Jerome Glisse , "Kirill A . Shutemov" , Andrea Arcangeli , Andrew Morton , Axel Rasmussen , Alistair Popple , peterx@redhat.com Subject: [PATCH v8 05/23] mm/shmem: Take care of UFFDIO_COPY_MODE_WP Date: Mon, 4 Apr 2022 21:48:41 -0400 Message-Id: <20220405014841.14185-1-peterx@redhat.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20220405014646.13522-1-peterx@redhat.com> References: <20220405014646.13522-1-peterx@redhat.com> MIME-Version: 1.0 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Authentication-Results: imf17.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=FSASRzSi; spf=none (imf17.hostedemail.com: domain of peterx@redhat.com has no SPF policy when checking 170.10.129.124) smtp.mailfrom=peterx@redhat.com; dmarc=pass (policy=none) header.from=redhat.com X-Stat-Signature: 3rfk9wb8ryj33rw7pa7up8xy5fryfaep X-Rspam-User: X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: 6C0764002E X-HE-Tag: 1649123326-883986 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Pass wp_copy into shmem_mfill_atomic_pte() through the stack, then apply the UFFD_WP bit properly when the UFFDIO_COPY on shmem is with UFFDIO_COPY_MODE_WP. wp_copy lands mfill_atomic_install_pte() finally. Note: we must do pte_wrprotect() if !writable in mfill_atomic_install_pte(), as mk_pte() could return a writable pte (e.g., when VM_SHARED on a shmem file). Signed-off-by: Peter Xu --- include/linux/shmem_fs.h | 4 ++-- mm/shmem.c | 4 ++-- mm/userfaultfd.c | 23 ++++++++++++++++++----- 3 files changed, 22 insertions(+), 9 deletions(-) diff --git a/include/linux/shmem_fs.h b/include/linux/shmem_fs.h index 3e915cc550bc..a68f982f22d1 100644 --- a/include/linux/shmem_fs.h +++ b/include/linux/shmem_fs.h @@ -145,11 +145,11 @@ extern int shmem_mfill_atomic_pte(struct mm_struct *dst_mm, pmd_t *dst_pmd, struct vm_area_struct *dst_vma, unsigned long dst_addr, unsigned long src_addr, - bool zeropage, + bool zeropage, bool wp_copy, struct page **pagep); #else /* !CONFIG_SHMEM */ #define shmem_mfill_atomic_pte(dst_mm, dst_pmd, dst_vma, dst_addr, \ - src_addr, zeropage, pagep) ({ BUG(); 0; }) + src_addr, zeropage, wp_copy, pagep) ({ BUG(); 0; }) #endif /* CONFIG_SHMEM */ #endif /* CONFIG_USERFAULTFD */ diff --git a/mm/shmem.c b/mm/shmem.c index 7004c7f55716..9efb8a96d75e 100644 --- a/mm/shmem.c +++ b/mm/shmem.c @@ -2319,7 +2319,7 @@ int shmem_mfill_atomic_pte(struct mm_struct *dst_mm, struct vm_area_struct *dst_vma, unsigned long dst_addr, unsigned long src_addr, - bool zeropage, + bool zeropage, bool wp_copy, struct page **pagep) { struct inode *inode = file_inode(dst_vma->vm_file); @@ -2392,7 +2392,7 @@ int shmem_mfill_atomic_pte(struct mm_struct *dst_mm, goto out_release; ret = mfill_atomic_install_pte(dst_mm, dst_pmd, dst_vma, dst_addr, - page, true, false); + page, true, wp_copy); if (ret) goto out_delete_from_cache; diff --git a/mm/userfaultfd.c b/mm/userfaultfd.c index dae25d985d15..b1c875b77fbb 100644 --- a/mm/userfaultfd.c +++ b/mm/userfaultfd.c @@ -77,10 +77,19 @@ int mfill_atomic_install_pte(struct mm_struct *dst_mm, pmd_t *dst_pmd, * Always mark a PTE as write-protected when needed, regardless of * VM_WRITE, which the user might change. */ - if (wp_copy) + if (wp_copy) { _dst_pte = pte_mkuffd_wp(_dst_pte); - else if (writable) + writable = false; + } + + if (writable) _dst_pte = pte_mkwrite(_dst_pte); + else + /* + * We need this to make sure write bit removed; as mk_pte() + * could return a pte with write bit set. + */ + _dst_pte = pte_wrprotect(_dst_pte); dst_pte = pte_offset_map_lock(dst_mm, dst_pmd, dst_addr, &ptl); @@ -95,7 +104,12 @@ int mfill_atomic_install_pte(struct mm_struct *dst_mm, pmd_t *dst_pmd, } ret = -EEXIST; - if (!pte_none(*dst_pte)) + /* + * We allow to overwrite a pte marker: consider when both MISSING|WP + * registered, we firstly wr-protect a none pte which has no page cache + * page backing it, then access the page. + */ + if (!pte_none_mostly(*dst_pte)) goto out_unlock; if (page_in_cache) { @@ -479,11 +493,10 @@ static __always_inline ssize_t mfill_atomic_pte(struct mm_struct *dst_mm, err = mfill_zeropage_pte(dst_mm, dst_pmd, dst_vma, dst_addr); } else { - VM_WARN_ON_ONCE(wp_copy); err = shmem_mfill_atomic_pte(dst_mm, dst_pmd, dst_vma, dst_addr, src_addr, mode != MCOPY_ATOMIC_NORMAL, - page); + wp_copy, page); } return err;