From patchwork Thu Dec 14 04:58:41 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Wilcox X-Patchwork-Id: 13492269 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 96510C4332F for ; Thu, 14 Dec 2023 04:58:59 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 1135C8D0093; Wed, 13 Dec 2023 23:58:59 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 0C4218D0083; Wed, 13 Dec 2023 23:58:59 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id EF4438D0093; Wed, 13 Dec 2023 23:58:58 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id E10388D0083 for ; Wed, 13 Dec 2023 23:58:58 -0500 (EST) Received: from smtpin30.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id B43EE4044C for ; Thu, 14 Dec 2023 04:58:58 +0000 (UTC) X-FDA: 81564219156.30.DB0F25D Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf04.hostedemail.com (Postfix) with ESMTP id BBD3040012 for ; Thu, 14 Dec 2023 04:58:55 +0000 (UTC) Authentication-Results: imf04.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=gWQQQQVB; spf=none (imf04.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1702529937; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:references:dkim-signature; bh=8gEfLFdKleS9iC7c1Er8MCq4Ctx2eRife9quGZ3/RDg=; b=02TKB34KlqBUtSxqxFFswVdU2385nYux4dBfFJPvEHlOI0ygpUSO6IxoZcTAZnV0XRWVVb rZo7xCAFWp+ski8EP4sxU1FQvEp1NhmNaw9drfrLUKUaYSREEQVW0P9rehuYbYn0OmyLB9 9/F3hnZYVi6tYjCvE3SYL00oczDIFSM= ARC-Authentication-Results: i=1; imf04.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=gWQQQQVB; spf=none (imf04.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org; dmarc=none ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1702529937; a=rsa-sha256; cv=none; b=R/Q0VDw10/3j7CChhQCraywoalbi7+tMDRa0zB5p5JoRs9kNZ1FTddhTx/F9dQ8A9MksSf 6v7Gf2DBlXxlHAwLiOt6hTrcPVu60YVwN7hDlV2J8myVDiZBUTGf08geYi/PFrDZHt5tGY tlAnO44GvFxUGP73C7KeR37msJpktrY= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To:Content-Type:Content-ID: Content-Description:In-Reply-To:References; bh=8gEfLFdKleS9iC7c1Er8MCq4Ctx2eRife9quGZ3/RDg=; b=gWQQQQVBAcfyYvywq4mIoA7+NB s7NjXShjKBI76MB2tWTtlR5bg5fyJ+jwsK27g4KdoB9XQU3AC+D2u3NtayHzCPphHt8TTloCYXz+Y meaEGARqEnaOadplFBk5tugmGT7H9XrgciuRQaMNTG4SmHVVOtnmk1H2U962RcuYwYvvdqkDM/fe/ ca6dJDzcKvpeREKRMqpJut1gotKFX5346UZWnXpYDMqSkeTLbFcDB/KAH0NHIPWg6OrtXsS5tJSbj /1oToluHXrkSQoxVUSfJz8A6BJACFX6zJcSYxela5MPW1w0mF8unlO1U+HykfN9COV3a4urtf0jd/ amRVBmFg==; Received: from willy by casper.infradead.org with local (Exim 4.94.2 #2 (Red Hat Linux)) id 1rDdoB-0042X6-0e; Thu, 14 Dec 2023 04:58:51 +0000 From: "Matthew Wilcox (Oracle)" To: Andrew Morton , linux-mm@kvack.org Cc: Charan Teja Kalla , Matthew Wilcox Subject: [PATCH] mm: Migrate high-order folios in swap cache correctly Date: Thu, 14 Dec 2023 04:58:41 +0000 Message-Id: <20231214045841.961776-1-willy@infradead.org> X-Mailer: git-send-email 2.37.1 MIME-Version: 1.0 X-Rspamd-Queue-Id: BBD3040012 X-Rspam-User: X-Stat-Signature: ocf8rs9fa38igqsxdypnuho8hg1ybfap X-Rspamd-Server: rspam01 X-HE-Tag: 1702529935-480617 X-HE-Meta: U2FsdGVkX1/HvqYnOEbuFzcHGyPbk9fDNeJ1YeETmR+53L3Y5twNU/OaTwedhQ39yyjebrANOZAOlRRReoPiyXvnhKoiEZZHf6NUiaDaAHzJE/CYWXqabR6ED3YvN0QAATPVPLbfdWjNOKjmeuZ/yfcPcz0Teozxu/aTsuaFW0RD2Yic+AKKzC9wfvldQJiOsZTJ5RpMLjaSE48EI7XvEqu3PCBlmal8jApwrXPBqrkcxnf8AI34/iHAW6Fb4YACXsZqSjdyqmw1UsHShIblNt9aHjgQVZ2sIcLVVLsUT2xdrH4MBe7U9I0cNsJFg7s1u5VQh7wWSVOjGpo7McqdKr0kvrKP71owOvvDdYsm+p58ABu9qH3AxksRZwO3ftLPzxFvKJ0r7v3D8ZgsjCP+3cQbvZTzFunUpVmTXiKO9DhqQc4lQR1XwMbhCBP85HCz9X6CrWRayWqJeVAEwKSVVW4Mn188OLv73Q6ZyZDFaMoQ/223StcA/VgqYBrmBZEIA/yD3o26IaI6f4r7Co/ry+vT/JBQhTo7LNaUMdITlf47BuajuUnv6ry+sYS0O8LgwrjtYDvi+MRY0k8hKoXQVZEe8BF+9SMGRRk7LMKAQtKh5NODBTeoI7Y3JOxihiW6qkaMAJltEcfWA4wdd/QBoetChaNQ2XxoTKbcDZC+LtZhUeDV3+LxfWwN36c1mEjIdEv+jRxYsCkBePlF0wXyKm8dEIUCaZ0yh036RcEs7QTW3mIgo1Gt4eVysM36xQxwCjY0Dq0vD/Nb88+7PkuGq0sWGHf48Ub+6l1ugzl/6XUwAJIi8QX05ONunUl6NR1JjZUAnDJ9VJ5pdOzLWdKPo8HOaVDbMfaguQG+kAaI90mtVF48QAQWgpPqVYH4fIpU6kTSbOXWyc7e6P2QZ9lTyemGPcQvb9L4vu3SEgQAbzVr5OOQ5mI8wXfC4RDzhF3tjCJAfAWdMgg0XCKQc4F 2ncdOl+/ jwEz9FzRKo4+IrOqMGBbqvewr2uIfkc4tHQv6wPsqhvav5QU2EKi9WL2nPkP9Qj5YoW88/TTgi/OEtKuv9MfUGbKHvnLlxXbW5QS1RqQYil8HoTES4CX8+2kZ3tytGXz5N0g6dHkjluV6YkDHxpraaz+roEeBD+7AugILLPy8uT+vCquzTvjM9pNsEpPY+QXFSg2lU55vhPpZvrhjCvuTvcjTFuCZwpjadtC70RQbYi1RhuvVH7G7hz6IjA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Charan Teja Kalla Large folios occupy N consecutive entries in the swap cache instead of using multi-index entries like the page cache. However, if a large folio is re-added to the LRU list, it can be migrated. The migration code was not aware of the difference between the swap cache and the page cache and assumed that a single xas_store() would be sufficient. This leaves potentially many stale pointers to the now-migrated folio in the swap cache, which can lead to almost arbitrary data corruption in the future. This can also manifest as infinite loops with the RCU read lock held. Signed-off-by: Charan Teja Kalla [modifications to the changelog & tweaked the fix] Signed-off-by: Matthew Wilcox (Oracle) Reported-by: Charan Teja Kalla --- mm/migrate.c | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/mm/migrate.c b/mm/migrate.c index d9d2b9432e81..2d67ca47d2e2 100644 --- a/mm/migrate.c +++ b/mm/migrate.c @@ -405,6 +405,7 @@ int folio_migrate_mapping(struct address_space *mapping, int dirty; int expected_count = folio_expected_refs(mapping, folio) + extra_count; long nr = folio_nr_pages(folio); + long entries, i; if (!mapping) { /* Anonymous page without mapping */ @@ -442,8 +443,10 @@ int folio_migrate_mapping(struct address_space *mapping, folio_set_swapcache(newfolio); newfolio->private = folio_get_private(folio); } + entries = nr; } else { VM_BUG_ON_FOLIO(folio_test_swapcache(folio), folio); + entries = 1; } /* Move dirty while page refs frozen and newpage not yet exposed */ @@ -453,7 +456,11 @@ int folio_migrate_mapping(struct address_space *mapping, folio_set_dirty(newfolio); } - xas_store(&xas, newfolio); + /* Swap cache still stores N entries instead of a high-order entry */ + for (i = 0; i < entries; i++) { + xas_store(&xas, newfolio); + xas_next(&xas); + } /* * Drop cache reference from old page by unfreezing