From patchwork Sat Aug 7 03:25:20 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Xu X-Patchwork-Id: 12424103 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 89286C4320A for ; Sat, 7 Aug 2021 03:25:35 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 17477611C5 for ; Sat, 7 Aug 2021 03:25:34 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 17477611C5 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=kvack.org Received: by kanga.kvack.org (Postfix) id 751176B0073; Fri, 6 Aug 2021 23:25:32 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 7025E6B0074; Fri, 6 Aug 2021 23:25:32 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 57BD36B0075; Fri, 6 Aug 2021 23:25:32 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 347AE6B0073 for ; Fri, 6 Aug 2021 23:25:32 -0400 (EDT) Received: from smtpin25.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id D481218043080 for ; Sat, 7 Aug 2021 03:25:31 +0000 (UTC) X-FDA: 78446844462.25.75CC92C Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [216.205.24.124]) by imf18.hostedemail.com (Postfix) with ESMTP id 22A16400A591 for ; Sat, 7 Aug 2021 03:25:30 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1628306730; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=bjEVjVuqoZmR4E24RTtF6/Pc/e6BiHg9xZTg6exvfsA=; b=hJq9yjY9PB4GzF00cttOp9kX1Ylb3zFYhLjCZ4HFrFx/J0T0mIa2OY9JftUTmAxITTyYLh lJkXcph6e29uzEWcIy6cr0xj32LtQRJRqTujavGSY1rhbP2jY2RdLY9k2HvvAydKHaHUIG 0KoUglUdglZhiasrvKXhOeg02p4cPJ0= Received: from mail-qv1-f69.google.com (mail-qv1-f69.google.com [209.85.219.69]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-189-Fnd6socTNj-6nOKW5lPq2A-1; Fri, 06 Aug 2021 23:25:29 -0400 X-MC-Unique: Fnd6socTNj-6nOKW5lPq2A-1 Received: by mail-qv1-f69.google.com with SMTP id z25-20020a0ca9590000b029033ba243ffa1so7796906qva.0 for ; Fri, 06 Aug 2021 20:25:29 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=bjEVjVuqoZmR4E24RTtF6/Pc/e6BiHg9xZTg6exvfsA=; b=JcLpCrK1KHA06lC7D6BFzjDIvPEaHIG3sMe6hIq3Z8Nm1MHwV+zE6DYN4iWvuE5DpN x+qwpH6SzgJ6PqMs5txnUOkG1thL1SGbaAf7Lu3qOhwfYlFpPAhlxM/zM3V3nhog98J/ niaIvE8NlWfOEukRuE7rSIcJyOGZuo9CW/4h5MDi08DOUqOWOHL64fbEINsIpdnYAY8c V8vWso1JrU6VatixYC6hcLO9uhms1UQYpJa/HxkDT4qaacpG08t9Oeg5dexgis119tEn 59Vvf2yp3Aes/Vo+G73JcrLCB93iLtsQhRRzfZ7fNX5HNVfhgH9OM263CE9nuMMUQvio HehA== X-Gm-Message-State: AOAM530esnpcd0FXvpYIP2csae9egtStn+APb9D+CHe+adToE9Kay1Xf tfxnDD15rmWMzaPsWtkvdHqeusY628iPic9U9724qKbxEV/6CEOpk7rd7NKrNvLIFbtNIKqzWa+ PdQAtCNzesds= X-Received: by 2002:a05:620a:205e:: with SMTP id d30mr13550308qka.365.1628306728897; Fri, 06 Aug 2021 20:25:28 -0700 (PDT) X-Google-Smtp-Source: ABdhPJzpGYk7qN5OVUrrQMIOkOmPcFH65ctEbWhaV9MLhg4CSFYW2Fa7e9x20fuo0S2Px9Hm+mTA0w== X-Received: by 2002:a05:620a:205e:: with SMTP id d30mr13550297qka.365.1628306728688; Fri, 06 Aug 2021 20:25:28 -0700 (PDT) Received: from localhost.localdomain (bras-base-toroon474qw-grc-92-76-70-75-133.dsl.bell.ca. [76.70.75.133]) by smtp.gmail.com with ESMTPSA id a5sm5514875qkk.92.2021.08.06.20.25.27 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Fri, 06 Aug 2021 20:25:28 -0700 (PDT) From: Peter Xu To: linux-kernel@vger.kernel.org, linux-mm@kvack.org Cc: Alistair Popple , Tiberiu Georgescu , ivan.teterevkov@nutanix.com, Mike Rapoport , Hugh Dickins , peterx@redhat.com, Matthew Wilcox , Andrea Arcangeli , David Hildenbrand , "Kirill A . Shutemov" , Andrew Morton , Mike Kravetz Subject: [PATCH RFC 3/4] mm: Handle PTE_MARKER page faults Date: Fri, 6 Aug 2021 23:25:20 -0400 Message-Id: <20210807032521.7591-4-peterx@redhat.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20210807032521.7591-1-peterx@redhat.com> References: <20210807032521.7591-1-peterx@redhat.com> MIME-Version: 1.0 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-Rspamd-Server: rspam03 X-Rspamd-Queue-Id: 22A16400A591 Authentication-Results: imf18.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=hJq9yjY9; dmarc=pass (policy=none) header.from=redhat.com; spf=none (imf18.hostedemail.com: domain of peterx@redhat.com has no SPF policy when checking 216.205.24.124) smtp.mailfrom=peterx@redhat.com X-Stat-Signature: abnrmxpp67dqyn9w6jruutt89c81ij6m X-HE-Tag: 1628306730-354638 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: handle_pte_marker() is the function that will parse and handle all the pte marker faults. For PAGEOUT marker, it's as simple as dropping the pte and do the fault just like a none pte. The other solution should be that we clear the pte to none pte and retry the fault, however that'll be slower than handling it right now. Signed-off-by: Peter Xu --- mm/memory.c | 41 +++++++++++++++++++++++++++++++++++++++++ 1 file changed, 41 insertions(+) diff --git a/mm/memory.c b/mm/memory.c index 7288f585544a..47f8ca064459 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -98,6 +98,8 @@ struct page *mem_map; EXPORT_SYMBOL(mem_map); #endif +static vm_fault_t do_fault(struct vm_fault *vmf); + /* * A number of key systems in x86 including ioremap() rely on the assumption * that high_memory defines the upper bound on direct map memory, then end @@ -1394,6 +1396,10 @@ static unsigned long zap_pte_range(struct mmu_gather *tlb, put_page(page); continue; + } else if (is_pte_marker_entry(entry)) { + /* Drop PTE_MARKER_PAGEOUT when zapped */ + pte_clear_not_present_full(mm, addr, pte, tlb->fullmm); + continue; } /* If details->check_mapping, we leave swap entries. */ @@ -3467,6 +3473,39 @@ static vm_fault_t remove_device_exclusive_entry(struct vm_fault *vmf) return 0; } +/* + * This function parses PTE markers and handle the faults. Returns true if we + * finished the fault, and we should have put the return value into "*ret". + * Otherwise it means we want to continue the swap path, and "*ret" untouched. + */ +static vm_fault_t handle_pte_marker(struct vm_fault *vmf) +{ + swp_entry_t entry = pte_to_swp_entry(vmf->orig_pte); + unsigned long marker; + + marker = pte_marker_get(entry); + + /* + * PTE markers should always be with file-backed memories, and the + * marker should never be empty. If anything weird happened, the best + * thing to do is to kill the process along with its mm. + */ + if (WARN_ON_ONCE(vma_is_anonymous(vmf->vma) || !marker)) + return VM_FAULT_SIGBUS; + +#ifdef CONFIG_PTE_MARKER_PAGEOUT + if (marker == PTE_MARKER_PAGEOUT) + /* + * This pte is previously zapped for swap, the PAGEOUT is only + * a flag before it's accessed again. Safe to drop it now. + */ + return do_fault(vmf); +#endif + + /* We see some marker that we can't handle */ + return VM_FAULT_SIGBUS; +} + /* * We enter with non-exclusive mmap_lock (to exclude vma changes, * but allow concurrent faults), and pte mapped but not yet locked. @@ -3503,6 +3542,8 @@ vm_fault_t do_swap_page(struct vm_fault *vmf) ret = vmf->page->pgmap->ops->migrate_to_ram(vmf); } else if (is_hwpoison_entry(entry)) { ret = VM_FAULT_HWPOISON; + } else if (is_pte_marker_entry(entry)) { + ret = handle_pte_marker(vmf); } else { print_bad_pte(vma, vmf->address, vmf->orig_pte, NULL); ret = VM_FAULT_SIGBUS;