From patchwork Fri Sep 1 08:20:26 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vlastimil Babka X-Patchwork-Id: 13372241 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2D9A3CA0FE9 for ; Fri, 1 Sep 2023 08:20:36 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 7AC408E0007; Fri, 1 Sep 2023 04:20:35 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 75D548D0002; Fri, 1 Sep 2023 04:20:35 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 649CC8E0007; Fri, 1 Sep 2023 04:20:35 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 565738D0002 for ; Fri, 1 Sep 2023 04:20:35 -0400 (EDT) Received: from smtpin18.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 1514D1C9414 for ; Fri, 1 Sep 2023 08:20:35 +0000 (UTC) X-FDA: 81187332030.18.3BDC34D Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.220.28]) by imf21.hostedemail.com (Postfix) with ESMTP id 1B6621C0019 for ; Fri, 1 Sep 2023 08:20:32 +0000 (UTC) Authentication-Results: imf21.hostedemail.com; dkim=pass header.d=suse.cz header.s=susede2_rsa header.b=r4Avfguv; dkim=pass header.d=suse.cz header.s=susede2_ed25519 header.b=LDBeIJn7; dmarc=none; spf=pass (imf21.hostedemail.com: domain of vbabka@suse.cz designates 195.135.220.28 as permitted sender) smtp.mailfrom=vbabka@suse.cz ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1693556433; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:references:dkim-signature; bh=0WI96qvBSKjcD8skIYDzJyfhIXrX2VNlBg1bDAXSO7w=; b=5PV7se/FzF5injA8Y3AVlBb5abAGxi0wTSeJr6cGRWHa5lHlqOp9HnVvO8TIYdS44ypN3g h8WHCqsWGMXlrUzWNQyhgqtT1ipiq5mQU+dbd2ziqfhYC/cQ8eYTMmiqSBTpP9eMZYyH6M xh+mItenmvnf43J2zs7M3GHXvJbOaXo= ARC-Authentication-Results: i=1; imf21.hostedemail.com; dkim=pass header.d=suse.cz header.s=susede2_rsa header.b=r4Avfguv; dkim=pass header.d=suse.cz header.s=susede2_ed25519 header.b=LDBeIJn7; dmarc=none; spf=pass (imf21.hostedemail.com: domain of vbabka@suse.cz designates 195.135.220.28 as permitted sender) smtp.mailfrom=vbabka@suse.cz ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1693556433; a=rsa-sha256; cv=none; b=gUhOvEcGdWzIPF3oHAdVVoIsLhOPY8oBsxkvL+TyvU8CNbL84mVfnM9qHIKu/ezX+OgW4E 2zo8N8xTgfBZi0+hH/OJR8GgdwifBGCEmdFIZb7mm9h9xb0d1Lc0MGXwd0vfaoxJZFeFim hnbySFuhNlJknJe3feGhpiae/IvNyo0= Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id B16B32185C; Fri, 1 Sep 2023 08:20:30 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_rsa; t=1693556430; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding; bh=0WI96qvBSKjcD8skIYDzJyfhIXrX2VNlBg1bDAXSO7w=; b=r4AvfguvEQvHBhS4L8lDsEaveieeJTkNCYi5f+9IRURkHs9QHXGXU55bgZKYP/UIN96eLm Su7gVEZFmwF5EXNaNlCErp7Hgj8GQqdQewKT6RBq+dcvYVgxoLUZe9WovHMiX8iQwt+NsA NGR32KTwfyM1j9J07mti1C+WlqZc3DM= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_ed25519; t=1693556430; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding; bh=0WI96qvBSKjcD8skIYDzJyfhIXrX2VNlBg1bDAXSO7w=; b=LDBeIJn7DjHmpeWPe/JUiDen1A1u3hmJEXiXvjRyh5pbOBnd20g14NvrGZHMhmcFppSIaI jb3axaR9hQtT2aCA== Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 27C7D1358B; Fri, 1 Sep 2023 08:20:30 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id M/8FCc6e8WSYGwAAMHmgww (envelope-from ); Fri, 01 Sep 2023 08:20:30 +0000 From: Vlastimil Babka To: seanjc@google.com Cc: ackerleytng@google.com, akpm@linux-foundation.org, anup@brainfault.org, aou@eecs.berkeley.edu, chao.p.peng@linux.intel.com, chenhuacai@kernel.org, david@redhat.com, isaku.yamahata@gmail.com, jarkko@kernel.org, jmorris@namei.org, kirill.shutemov@linux.intel.com, kvm-riscv@lists.infradead.org, kvm@vger.kernel.org, kvmarm@lists.linux.dev, liam.merwick@oracle.com, linux-arm-kernel@lists.infradead.org, linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mips@vger.kernel.org, linux-mm@kvack.org, linux-riscv@lists.infradead.org, linux-security-module@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, mail@maciej.szmigiero.name, maz@kernel.org, michael.roth@amd.com, mpe@ellerman.id.au, oliver.upton@linux.dev, palmer@dabbelt.com, paul.walmsley@sifive.com, paul@paul-moore.com, pbonzini@redhat.com, qperret@google.com, serge@hallyn.com, tabba@google.com, vannapurve@google.com, vbabka@suse.cz, wei.w.wang@intel.com, willy@infradead.org, yu.c.zhang@linux.intel.com Subject: [PATCH gmem FIXUP] mm, compaction: make testing mapping_unmovable() safe Date: Fri, 1 Sep 2023 10:20:26 +0200 Message-ID: <20230901082025.20548-2-vbabka@suse.cz> X-Mailer: git-send-email 2.41.0 MIME-Version: 1.0 X-Rspamd-Server: rspam09 X-Rspamd-Queue-Id: 1B6621C0019 X-Stat-Signature: yrhu89bgczuw8q4mzmzko315cc8jcne1 X-Rspam-User: X-HE-Tag: 1693556432-579308 X-HE-Meta: U2FsdGVkX191zukMzrnTiuwRyr9kvW995qxkLCqs+VktcQSE7/9dUIMwP6viHUK3EUwCCE83LDL8EEIjibZCon+e4RMJZ7az08ZwowO0HmWLHtQChFYqCiLoJ+ozMDClPPBnWhLDS7bf0Ptaiic5MHwsOqBw3ZDcixIM8ASnow4ULxnYxUIhapXulOFM4i7WMiTbzN6MiTd/+ctHLF7H8SKCbSk8axpLs7ocKtgKD7MbFz9iMjCXm4V3Y+GfxrEfkIdJbb500dNIbaXsQG2cd1PTb+EQ0Zv4gL7bOJe4X5Fx6abBXem9cKr4GtA5gJNM9jsmz66RVVUiGB/j8za8u3qEW8YaVT59gxS8aF4w1IRIqpz5oX6eP6t5nCZlQIagYn+QfjTx9Ip4PAUlBFMdlR9dfmYQwEduVoMFo2D+SFUctc9Fzu+FW+RPzKfNLugGJ3Ia5cDtu+7DbgOl3MahMTNmX6NJca02c/UvSlVbd2KIHf7pA9wY1cmGnmuYKXIItrYw3V5KNyc6YlXnkIRnKSBRF7fA6Q0CCN6T6kOSuVwyEnZf/om5VhQXU31n3QhJgdPSr1V+MT+66C683q0YQNWsBrB30qFsJfwOAC+k6DPr4cqkQcn+J7dO3MEDK9K4dlBN2Y64MokBif2M9S0zdleX/J6vKyqxloMON9WsrTPF8KocOWC9bfGvoqOr1/tnic1GPmmWJTqvgqyfr9chtxFUiSQMCyAVK36JjkLw7wVaFQu4iIdtr0HFw6C7ikvJA2r8rXubJcHJxFFc3+l36zhRRpNf76dx8dY2Y9cVOyopLQHhM90Jz5Vu1qyWndbgkP4ucqnpNKxyORCTRQY+7iqgWPslPYuhfJ6GWD38FxuTY3zKZc+hfM4pkUQP0kNLe3SgspYv50FaruswrleOOAVxuoA+QfX+kLVA3NBLX9oBYiPpWXw8EovyYvVa7ye7H2FroirFnowkk8NDLni QUlIDFCo yeyTjLu4pfkrx45fGRIzCbZ9ngyyot1aMMjfH5+RD8bUxMAN4ivOJXeis13GRlDoOXS0std/su0h5NmH3OC9EZmVrumz5XQle6IjgMonKKDec4lLcIDkFp32FEwRRFlaHmPwiWFQWfv02Lc9va7D/Xg6vrcT+ntjIk7Zgx1QJEbML8MNuX6t46xC6ZYhZKXW7Q+eXO6XK3gVCZnuqiMylrX16RpWuVeWsi/S7+gAlVCLE4n0iFlFrdqcmUN8iuNtXQfIqyVKp904xrumOytUMf5JtnVjHFDDbCFblrGDq1/sR+ynGt0rkkRYssQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: As Kirill pointed out, mapping can be removed under us due to truncation. Test it under folio lock as already done for the async compaction / dirty folio case. To prevent locking every folio with mapping to do the test, do it only for unevictable folios, as we can expect the unmovable mapping folios are also unevictable - it is the case for guest memfd folios. Also incorporate comment update suggested by Matthew. Fixes: 3424873596ce ("mm: Add AS_UNMOVABLE to mark mapping as completely unmovable") Signed-off-by: Vlastimil Babka --- Feel free to squash into 3424873596ce. mm/compaction.c | 49 ++++++++++++++++++++++++++++++++----------------- 1 file changed, 32 insertions(+), 17 deletions(-) diff --git a/mm/compaction.c b/mm/compaction.c index a3d2b132df52..e0e439b105b5 100644 --- a/mm/compaction.c +++ b/mm/compaction.c @@ -862,6 +862,7 @@ isolate_migratepages_block(struct compact_control *cc, unsigned long low_pfn, /* Time to isolate some pages for migration */ for (; low_pfn < end_pfn; low_pfn++) { + bool is_dirty, is_unevictable; if (skip_on_failure && low_pfn >= next_skip_pfn) { /* @@ -1047,10 +1048,6 @@ isolate_migratepages_block(struct compact_control *cc, unsigned long low_pfn, if (!mapping && (folio_ref_count(folio) - 1) > folio_mapcount(folio)) goto isolate_fail_put; - /* The mapping truly isn't movable. */ - if (mapping && mapping_unmovable(mapping)) - goto isolate_fail_put; - /* * Only allow to migrate anonymous pages in GFP_NOFS context * because those do not depend on fs locks. @@ -1062,8 +1059,10 @@ isolate_migratepages_block(struct compact_control *cc, unsigned long low_pfn, if (!folio_test_lru(folio)) goto isolate_fail_put; + is_unevictable = folio_test_unevictable(folio); + /* Compaction might skip unevictable pages but CMA takes them */ - if (!(mode & ISOLATE_UNEVICTABLE) && folio_test_unevictable(folio)) + if (!(mode & ISOLATE_UNEVICTABLE) && is_unevictable) goto isolate_fail_put; /* @@ -1075,26 +1074,42 @@ isolate_migratepages_block(struct compact_control *cc, unsigned long low_pfn, if ((mode & ISOLATE_ASYNC_MIGRATE) && folio_test_writeback(folio)) goto isolate_fail_put; - if ((mode & ISOLATE_ASYNC_MIGRATE) && folio_test_dirty(folio)) { - bool migrate_dirty; + is_dirty = folio_test_dirty(folio); + + if (((mode & ISOLATE_ASYNC_MIGRATE) && is_dirty) + || (mapping && is_unevictable)) { + bool migrate_dirty = true; + bool is_unmovable; /* - * Only pages without mappings or that have a - * ->migrate_folio callback are possible to migrate - * without blocking. However, we can be racing with - * truncation so it's necessary to lock the page - * to stabilise the mapping as truncation holds - * the page lock until after the page is removed - * from the page cache. + * Only folios without mappings or that have + * a ->migrate_folio callback are possible to migrate + * without blocking. + * + * Folios from unmovable mappings are not migratable. + * + * However, we can be racing with truncation, which can + * free the mapping that we need to check. Truncation + * holds the folio lock until after the folio is removed + * from the page so holding it ourselves is sufficient. + * + * To avoid this folio locking to inspect every folio + * with mapping for being unmovable, we assume every + * such folio is also unevictable, which is a cheaper + * test. If our assumption goes wrong, it's not a bug, + * just potentially wasted cycles. */ if (!folio_trylock(folio)) goto isolate_fail_put; mapping = folio_mapping(folio); - migrate_dirty = !mapping || - mapping->a_ops->migrate_folio; + if ((mode & ISOLATE_ASYNC_MIGRATE) && is_dirty) { + migrate_dirty = !mapping || + mapping->a_ops->migrate_folio; + } + is_unmovable = mapping && mapping_unmovable(mapping); folio_unlock(folio); - if (!migrate_dirty) + if (!migrate_dirty || is_unmovable) goto isolate_fail_put; }