From patchwork Sat Aug 7 03:25:21 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Xu X-Patchwork-Id: 12424105 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 60239C4338F for ; Sat, 7 Aug 2021 03:25:37 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id EB4BE61186 for ; Sat, 7 Aug 2021 03:25:36 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org EB4BE61186 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=kvack.org Received: by kanga.kvack.org (Postfix) id 46D746B0074; Fri, 6 Aug 2021 23:25:35 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 41E936B0075; Fri, 6 Aug 2021 23:25:35 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 2E57B6B0078; Fri, 6 Aug 2021 23:25:35 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0115.hostedemail.com [216.40.44.115]) by kanga.kvack.org (Postfix) with ESMTP id 09AB16B0074 for ; Fri, 6 Aug 2021 23:25:35 -0400 (EDT) Received: from smtpin01.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id BB06B1D975 for ; Sat, 7 Aug 2021 03:25:34 +0000 (UTC) X-FDA: 78446844588.01.6B5B28B Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by imf02.hostedemail.com (Postfix) with ESMTP id 51DC270088DA for ; Sat, 7 Aug 2021 03:25:34 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1628306733; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=OK+bQTiUipxL6Jk7xja/DmcKUDIieuX4Ee7RUXkwoH4=; b=XTs5ywsZ11ianwrs4VxU/wY7wBOlmdy4gviLhd0Oh1+fqfSAw/Y0dJnMo2Rl+svnzY8ZRD l+q6+AACLhtCQuYloSVcYGqtixWBC6++ybSlsv11MpzQaUZ4ngK8YUCgJ6REvu3oRhqgsI CiPdMvhGIOct6e5Ofo1+YOKk/GeTxVg= Received: from mail-qv1-f70.google.com (mail-qv1-f70.google.com [209.85.219.70]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-272-kXvlViTlO2qV5DRKjhFXCg-1; Fri, 06 Aug 2021 23:25:30 -0400 X-MC-Unique: kXvlViTlO2qV5DRKjhFXCg-1 Received: by mail-qv1-f70.google.com with SMTP id a2-20020a0562141302b02903303839b843so7751697qvv.13 for ; Fri, 06 Aug 2021 20:25:30 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=OK+bQTiUipxL6Jk7xja/DmcKUDIieuX4Ee7RUXkwoH4=; b=piko//2m0LcPRwUAe8EKnW9MV59e6sVPsVagc5CTs+BTpeFrqXB24HHWJzn3P41pBo oRwBJL4QGWgfyqIO5Wy3T2+Ttr3465IgxYMpQIckgtHyqxrKh79lHu42HkbUWwKfDzTs ColNgY+oDQjEte9zzu51/mX1NVSNzXm9ATM5k2Xoij+Rk0fxm8sOM26RUxPcK+W5++Mp GssEEgPo162OZ3EZ6nVdlc2E7GdShNTAJonjb2TIF7G+YN0nXYSwXYfP92vQvhjUAC6e ujQrR2ewdSyBv9aKL9dutHrEnE+AMPVki/YrFW3kTxXCLCghGZwY4VFSQLIFYhV8EXPu r4bw== X-Gm-Message-State: AOAM530+0RgryMz/j9qmNBssyNbWCp25Mc8LUMHYuWa6oOD0hPQdo49I IZ7vcTiSfrEBbFp4FIYcgb1l+yEdrZKlUqliRRhtNPs7zrJ21K1cHCSKrrRf+nQqJj9bxyOiOFC N2L3k+bQljtc= X-Received: by 2002:aed:2163:: with SMTP id 90mr11658369qtc.186.1628306730378; Fri, 06 Aug 2021 20:25:30 -0700 (PDT) X-Google-Smtp-Source: ABdhPJzCK0I1XtqNK7P9nzzv0g1D1B3S1najqoV6hQLfL5gLFgAFA6yE5Kw0DBDbjYav/08UMiA3oA== X-Received: by 2002:aed:2163:: with SMTP id 90mr11658356qtc.186.1628306730149; Fri, 06 Aug 2021 20:25:30 -0700 (PDT) Received: from localhost.localdomain (bras-base-toroon474qw-grc-92-76-70-75-133.dsl.bell.ca. [76.70.75.133]) by smtp.gmail.com with ESMTPSA id a5sm5514875qkk.92.2021.08.06.20.25.28 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Fri, 06 Aug 2021 20:25:29 -0700 (PDT) From: Peter Xu To: linux-kernel@vger.kernel.org, linux-mm@kvack.org Cc: Alistair Popple , Tiberiu Georgescu , ivan.teterevkov@nutanix.com, Mike Rapoport , Hugh Dickins , peterx@redhat.com, Matthew Wilcox , Andrea Arcangeli , David Hildenbrand , "Kirill A . Shutemov" , Andrew Morton , Mike Kravetz Subject: [PATCH RFC 4/4] mm: Install marker pte when page out for shmem pages Date: Fri, 6 Aug 2021 23:25:21 -0400 Message-Id: <20210807032521.7591-5-peterx@redhat.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20210807032521.7591-1-peterx@redhat.com> References: <20210807032521.7591-1-peterx@redhat.com> MIME-Version: 1.0 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: 51DC270088DA Authentication-Results: imf02.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=XTs5ywsZ; dmarc=pass (policy=none) header.from=redhat.com; spf=none (imf02.hostedemail.com: domain of peterx@redhat.com has no SPF policy when checking 170.10.133.124) smtp.mailfrom=peterx@redhat.com X-Stat-Signature: d6kogqppp856g5c5pirktrouxjjsykh1 X-HE-Tag: 1628306734-927926 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: When shmem pages are swapped out, instead of clearing the pte entry, we leave a marker pte showing that this page is swapped out as a hint for pagemap. A new TTU flag is introduced to identify this case. This can be useful for detecting swapped out cold shmem pages. Then after some memory background scanning work (which will fault in the shmem page and confusing page reclaim), we can do MADV_PAGEOUT explicitly on this page to swap it out again as we know it was cold. For pagemap, we don't need to explicitly set PM_SWAP bit, because by nature SWP_PTE_MARKER ptes are already counted as PM_SWAP due to it's format as swap. Signed-off-by: Peter Xu --- fs/proc/task_mmu.c | 1 + include/linux/rmap.h | 1 + mm/rmap.c | 19 +++++++++++++++++++ mm/vmscan.c | 2 +- 4 files changed, 22 insertions(+), 1 deletion(-) diff --git a/fs/proc/task_mmu.c b/fs/proc/task_mmu.c index eb97468dfe4c..21b8594abc1d 100644 --- a/fs/proc/task_mmu.c +++ b/fs/proc/task_mmu.c @@ -1384,6 +1384,7 @@ static pagemap_entry_t pte_to_pagemap_entry(struct pagemapread *pm, if (pm->show_pfn) frame = swp_type(entry) | (swp_offset(entry) << MAX_SWAPFILES_SHIFT); + /* NOTE: this covers PTE_MARKER_PAGEOUT too */ flags |= PM_SWAP; if (is_pfn_swap_entry(entry)) page = pfn_swap_entry_to_page(entry); diff --git a/include/linux/rmap.h b/include/linux/rmap.h index c976cc6de257..318a0e95c7fb 100644 --- a/include/linux/rmap.h +++ b/include/linux/rmap.h @@ -95,6 +95,7 @@ enum ttu_flags { * do a final flush if necessary */ TTU_RMAP_LOCKED = 0x80, /* do not grab rmap lock: * caller holds it */ + TTU_HINT_PAGEOUT = 0x100, /* Hint for pageout operation */ }; #ifdef CONFIG_MMU diff --git a/mm/rmap.c b/mm/rmap.c index b9eb5c12f3fe..24a70b36b6da 100644 --- a/mm/rmap.c +++ b/mm/rmap.c @@ -1384,6 +1384,22 @@ void page_remove_rmap(struct page *page, bool compound) unlock_page_memcg(page); } +static inline void +pte_marker_install(struct vm_area_struct *vma, pte_t *pte, + struct page *page, unsigned long address) +{ +#ifdef CONFIG_PTE_MARKER_PAGEOUT + swp_entry_t entry; + pte_t pteval; + + if (vma_is_shmem(vma) && !PageAnon(page) && pte_none(*pte)) { + entry = make_pte_marker_entry(PTE_MARKER_PAGEOUT); + pteval = swp_entry_to_pte(entry); + set_pte_at(vma->vm_mm, address, pte, pteval); + } +#endif +} + /* * @arg: enum ttu_flags will be passed to this argument */ @@ -1628,6 +1644,9 @@ static bool try_to_unmap_one(struct page *page, struct vm_area_struct *vma, */ dec_mm_counter(mm, mm_counter_file(page)); } + + if (flags & TTU_HINT_PAGEOUT) + pte_marker_install(vma, pvmw.pte, page, address); discard: /* * No need to call mmu_notifier_invalidate_range() it has be diff --git a/mm/vmscan.c b/mm/vmscan.c index 4620df62f0ff..4754af6fa24b 100644 --- a/mm/vmscan.c +++ b/mm/vmscan.c @@ -1493,7 +1493,7 @@ static unsigned int shrink_page_list(struct list_head *page_list, * processes. Try to unmap it here. */ if (page_mapped(page)) { - enum ttu_flags flags = TTU_BATCH_FLUSH; + enum ttu_flags flags = TTU_BATCH_FLUSH | TTU_HINT_PAGEOUT; bool was_swapbacked = PageSwapBacked(page); if (unlikely(PageTransHuge(page)))