From patchwork Tue Mar 23 00:49:02 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Xu X-Patchwork-Id: 12156501 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id CE4A3C433C1 for ; Tue, 23 Mar 2021 00:49:53 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 766976196C for ; Tue, 23 Mar 2021 00:49:53 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 766976196C Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id B71A96B012E; Mon, 22 Mar 2021 20:49:40 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id B22466B0130; Mon, 22 Mar 2021 20:49:40 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8FF2B6B0131; Mon, 22 Mar 2021 20:49:40 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0131.hostedemail.com [216.40.44.131]) by kanga.kvack.org (Postfix) with ESMTP id 65A456B012E for ; Mon, 22 Mar 2021 20:49:40 -0400 (EDT) Received: from smtpin03.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id 2D90362F2 for ; Tue, 23 Mar 2021 00:49:40 +0000 (UTC) X-FDA: 77949306120.03.132D71F Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [216.205.24.124]) by imf13.hostedemail.com (Postfix) with ESMTP id 88016E0011C9 for ; Tue, 23 Mar 2021 00:49:39 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1616460579; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Eo6GR1szVdaE50+5HSQqPNu3QULjjOofe28eTN5phnY=; b=QTKagX470a4SOSZBfUMNNC0czFPhCfXLBRf+Vl4vZZep1RTwPrRSYtMnSBUCRra+iqmYT8 4/EtssL5KW9lElCc8uGZAnLt6NSHTO2imXiMonh4FzIkG0/T2t2IS2xxkGJpPjXJvq28G+ lZ+TwuEBnhsAEwHcAYSoWvQAkT9S9eA= Received: from mail-qt1-f200.google.com (mail-qt1-f200.google.com [209.85.160.200]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-431--A5az_gHPhi-EHXYQdf9qw-1; Mon, 22 Mar 2021 20:49:38 -0400 X-MC-Unique: -A5az_gHPhi-EHXYQdf9qw-1 Received: by mail-qt1-f200.google.com with SMTP id t5so410628qti.5 for ; Mon, 22 Mar 2021 17:49:38 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Eo6GR1szVdaE50+5HSQqPNu3QULjjOofe28eTN5phnY=; b=MxHHwRJnAyKrLx0ZHd8hGoQp+l/ukfZ8VdFfLZ1/kdbx31amTMVxbkVNlS2lJa1yaP 7DgiKmQomNpVThd36pNp/LGlyGYQonf0w3LeDDqnVmkJNUc/VMUZUMxVqEXYDy/l2cJ4 X7GHv82/hzZGLDz3I6K5Tcbw05k3TwzoinCREymD7uH31gCw5OAJ4929uhz9vJXUyirn Oy0vLjw6iZqSfcMvujP+38R6V6bkxYhv+rRhTPayBG/uwbUqTwrMPTbcXi96wvS2tazh SzlKOZix4LW50zuWbbSLS1w4/Dq75Kr6PSKccyFAWB6BKS7kA18KKp4ORczsevm5n2c/ Vt6Q== X-Gm-Message-State: AOAM532Jc4nzjLceb9gHA98Jkp2yabILeebdU5L5z8qcMNCS/QrOa9sb pCGE50eyaGuZEoShIw/ndxWgcojSQbG2HqtFccMWHmOko0Pi46jNdrryn8eoQOJCE+K2pCvWwpG v/Kp6Xwc2aXQ= X-Received: by 2002:a0c:f805:: with SMTP id r5mr2752058qvn.45.1616460577099; Mon, 22 Mar 2021 17:49:37 -0700 (PDT) X-Google-Smtp-Source: ABdhPJyf8TDGDMVgqtBSuLbWSmrCugVffrpBoNx4NuHOZZ6VA9Ugvm4eHQ0tm4uAnu2RFfyiBQGo4A== X-Received: by 2002:a0c:f805:: with SMTP id r5mr2752046qvn.45.1616460576859; Mon, 22 Mar 2021 17:49:36 -0700 (PDT) Received: from localhost.localdomain (bras-base-toroon474qw-grc-82-174-91-135-175.dsl.bell.ca. [174.91.135.175]) by smtp.gmail.com with ESMTPSA id n6sm5031793qtx.22.2021.03.22.17.49.35 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 22 Mar 2021 17:49:36 -0700 (PDT) From: Peter Xu To: linux-kernel@vger.kernel.org, linux-mm@kvack.org Cc: "Kirill A . Shutemov" , Jerome Glisse , Mike Kravetz , Matthew Wilcox , Andrew Morton , Axel Rasmussen , Hugh Dickins , peterx@redhat.com, Nadav Amit , Andrea Arcangeli , Mike Rapoport Subject: [PATCH 13/23] shmem/userfaultfd: Handle the left-overed special swap ptes Date: Mon, 22 Mar 2021 20:49:02 -0400 Message-Id: <20210323004912.35132-14-peterx@redhat.com> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20210323004912.35132-1-peterx@redhat.com> References: <20210323004912.35132-1-peterx@redhat.com> MIME-Version: 1.0 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=peterx@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 88016E0011C9 X-Stat-Signature: qjmoyh9qwtdrwdcpbwjm5po16xnciceg Received-SPF: none (redhat.com>: No applicable sender policy available) receiver=imf13; identity=mailfrom; envelope-from=""; helo=us-smtp-delivery-124.mimecast.com; client-ip=216.205.24.124 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1616460579-774471 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Note that the special uffd-wp swap pte can be left over even if the page under the pte got evicted. Normally when evict a page, we will unmap the ptes by walking through the reverse mapping. However we never tracked such information for the special swap ptes because they're not real mappings but just markers. So we need to take care of that when we see a marker but when it's actually meaningless (the page behind it got evicted). We have already taken care of that in e.g. alloc_set_pte() where we'll treat the special swap pte as pte_none() when necessary. However we need to also teach userfaultfd itself on either UFFDIO_COPY or handling page faults, so that everything will still work as expected. Signed-off-by: Peter Xu --- fs/userfaultfd.c | 15 +++++++++++++++ mm/shmem.c | 13 ++++++++++++- 2 files changed, 27 insertions(+), 1 deletion(-) diff --git a/fs/userfaultfd.c b/fs/userfaultfd.c index bd83379d4dd2..72956f9cc892 100644 --- a/fs/userfaultfd.c +++ b/fs/userfaultfd.c @@ -329,6 +329,21 @@ static inline bool userfaultfd_must_wait(struct userfaultfd_ctx *ctx, */ if (pte_none(*pte)) ret = true; + /* + * We also treat the swap special uffd-wp pte as the pte_none() here. + * This should in most cases be a missing event, as we never handle + * wr-protect upon a special uffd-wp swap pte - it should first be + * converted into a normal read request before handling wp. It just + * means the page/swap cache that backing this pte is gone, so this + * special pte is leftover. + * + * We can't simply replace it with a none pte because we're not with + * the pgtable lock here. Instead of taking it and clearing the pte, + * the easy way is to let UFFDIO_COPY understand this pte too when + * trying to install a new page onto it. + */ + if (pte_swp_uffd_wp_special(*pte)) + ret = true; if (!pte_write(*pte) && (reason & VM_UFFD_WP)) ret = true; pte_unmap(pte); diff --git a/mm/shmem.c b/mm/shmem.c index e88aaabaeb27..90d67406af66 100644 --- a/mm/shmem.c +++ b/mm/shmem.c @@ -2469,7 +2469,18 @@ int shmem_mcopy_atomic_pte(struct mm_struct *dst_mm, pmd_t *dst_pmd, goto out_release_unlock; ret = -EEXIST; - if (!pte_none(*dst_pte)) + /* + * Besides the none pte, we also allow UFFDIO_COPY to install a pte + * onto the uffd-wp swap special pte, because that pte should be the + * same as a pte_none() just in that it contains wr-protect information + * (which could only be dropped when unmap the memory). + * + * It's safe to drop that marker because we know this is part of a + * MISSING fault, and the caller is very clear about this page missing + * rather than wr-protected. Then we're sure the wr-protect bit is + * just a leftover so it's useless already. + */ + if (!pte_none(*dst_pte) && !pte_swp_uffd_wp_special(*dst_pte)) goto out_release_unlock; if (!is_continue) {