From patchwork Fri Aug 2 15:55:19 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Hildenbrand X-Patchwork-Id: 13751749 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id BDEA9C3DA4A for ; Fri, 2 Aug 2024 15:56:24 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 574FC6B0099; Fri, 2 Aug 2024 11:56:24 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 525CE6B009A; Fri, 2 Aug 2024 11:56:24 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 3C56A6B009B; Fri, 2 Aug 2024 11:56:24 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 181876B0099 for ; Fri, 2 Aug 2024 11:56:24 -0400 (EDT) Received: from smtpin15.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id BC3F51A1272 for ; Fri, 2 Aug 2024 15:56:23 +0000 (UTC) X-FDA: 82407757446.15.B20F4AC Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by imf23.hostedemail.com (Postfix) with ESMTP id EF706140017 for ; Fri, 2 Aug 2024 15:56:21 +0000 (UTC) Authentication-Results: imf23.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=fguStQ63; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf23.hostedemail.com: domain of david@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=david@redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1722614139; a=rsa-sha256; cv=none; b=A1g4oVZdB565IW0lx7Ss+8uo5Agu83EdvAHfjT5Pdz5p0hmf3k81HweqFxWoSqyxYFmFYU 3h90LEpPmpwZM53cmzCOJJ5PW2kCIt9GWZvb29jZ0/y9rQAzJ74Q09r5SsTnn/ZThc+ysG +0KogSpAgZCFK6RrLFGsDvN5w1Xa5Mk= ARC-Authentication-Results: i=1; imf23.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=fguStQ63; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf23.hostedemail.com: domain of david@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=david@redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1722614138; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=9RoWh69jXPnUan+groy9VerJmyWjUE6OshAG0jiN7k8=; b=Vgs52DMYPxVeJgoLOJmCrmwjp2AsEMBMei+5a3XBFO07IFDxmQK459DNK2jPbwD2sL+2eB 75lPRdUEQ1UK9R7FTXdbah0tie2VttdNmp38IC5F3P6b4r2xF4Hgn2UfH+AKXoFSt6xNNk 4fFXpuEJBG1vn73q6oViCktBIZoETtU= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1722614181; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=9RoWh69jXPnUan+groy9VerJmyWjUE6OshAG0jiN7k8=; b=fguStQ63bEwP1E4QgRiIR8nyFqDMRUGXx7m0BKZg+BfQMa7MIUAgk6en1Y7CyWY2QKJIRK hUlMbisgOxYvWDYEKE1yIuU8HVVWTra+m1HjrCDHxmnA6awLFB82Aa5TgUH6YTkHy8/S/9 lGUmC6P1qbvZvJv2rRv7QQ3sZvqfa1Y= Received: from mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-654-6rd6U-xePRWdI5xPe_dR3w-1; Fri, 02 Aug 2024 11:56:15 -0400 X-MC-Unique: 6rd6U-xePRWdI5xPe_dR3w-1 Received: from mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 45CAA18B65ED; Fri, 2 Aug 2024 15:56:13 +0000 (UTC) Received: from t14s.redhat.com (unknown [10.39.192.113]) by mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 8A4CE300018D; Fri, 2 Aug 2024 15:56:06 +0000 (UTC) From: David Hildenbrand To: linux-kernel@vger.kernel.org Cc: linux-mm@kvack.org, linux-doc@vger.kernel.org, kvm@vger.kernel.org, linux-s390@vger.kernel.org, linux-fsdevel@vger.kernel.org, David Hildenbrand , Andrew Morton , "Matthew Wilcox (Oracle)" , Jonathan Corbet , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Heiko Carstens , Vasily Gorbik , Alexander Gordeev , Sven Schnelle , Gerald Schaefer Subject: [PATCH v1 06/11] mm/ksm: convert scan_get_next_rmap_item() from follow_page() to folio_walk Date: Fri, 2 Aug 2024 17:55:19 +0200 Message-ID: <20240802155524.517137-7-david@redhat.com> In-Reply-To: <20240802155524.517137-1-david@redhat.com> References: <20240802155524.517137-1-david@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.4 X-Rspamd-Queue-Id: EF706140017 X-Rspam-User: X-Rspamd-Server: rspam05 X-Stat-Signature: izhbsb9enooowcobmifu1f8bq3zs8dqq X-HE-Tag: 1722614181-158697 X-HE-Meta: U2FsdGVkX1/vH/NAjfEDa3VdranV30wlqs8OWlK3sI0DlVOPYNGJoGb+ok52ekXWcnk3bWgDlSomrtAvfaMH/qvuLZRfxTHlCGsj0ng3NRuj3CfNmGnT93X5FUmuzGND6c67vP6R5L5Ee5rPOeQrB86f4KKl/qZx0YrZBtuxUgV5OSNDhMkfvse+5U64zQzq4W/43rUC0QmTZWevtiZ/9zdc4xN+n3a0UD88FiCv28CpK5BZPatdhWt4FY8FSn39Dq/OE0BBB7FCkFHZp/MYoLBP9ZdRYimkZKbR/ouCdcXwSUjc/3tjlNMzzQd/UM5dIYUT7nVz7lWMzEPQNTXhr90hce3t6ZQLkyNMF/Ib8a6iSyJw+NRhDPAAugLPHlSusvmzgflCaHhrLNr+B2h6Ry3k2Wr87yZZ0Co5I7Wc2ol1hVlR6foEw48zbu4bCzVLGGzbbNoIy8m5VjFq/4wTeQQV1T6Akd6PQE/Mj6lJYGqpFrmGyWc0J3y+KgyGVEemTvHXJpJmiLOQgr8KFPht3+mHyRDkMmvoZ4Ww5/gJSe9rWQeAEerLgTWmP4Xp98a98NFRUu7HgkFuDgwKc4xz+m/coI6LppM+ZF2/mAh8jxzC29jEcY/EYoLj2WFg1yUgiLBp5+bVYc7WZ+Ar194OERA9OBQVmqSVTEovb8UVDKg+oUs6XbsB8MIgitmtrCRmM81fsYTj+msj21iZrOmhURLMDpHeHuAxEl1pia3n0BLNJNcHiISRa//XGC+iH+lg3Cq1nWrh8AdpvfGup6XxjFil46fs/yVeUN8D8LRTwDyhIaEM/ZPP2PFIeoKqFszNFIpriHW/m+ZxwnB1zT3AzA1trK5A9DBriU69hQ1moh81QTmobgDCLFdjLR/7WDXc19HGjje8M3ZDnHsuMlkfgNCUeZWc0bvKsa2ZIGCtxwtCIRDJgQI/xFtQuA5YytNh3AJtskGzZ+uHFAgRmtY 5+zTYyeQ kxBIGywcc3DfGPoBsuWaQ0TDJ/5xzD5kgnrUx3cx1Hf6NLgtZ22dHcFbvdwYe33uWlXNf+QYRfp5A0L78Zg42izyZ0D/Uvx6qxFwzrZDtYDay03tGRiVxo/R8KjebVSu530n8H55q5nfH/Hrw2BlWhVQWwnP38CYuGA7t9jgx5lVk76YBGwrcowBDJZSz2OtQ8YwAbHeR7ugRw7WWFe1qs/8hrWqZHIqbwfj0vnzQ51Ykn3D50sjNEx4QU/43eYCTi0yzyS9xcMZCugrJlsvxuRvbSN6g75uRJJWEN/5zBooV8EcSbs26SoG9VzfBJbn3XzUWOu5UHIQSDckwTN1L2dH14Q== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Let's use folio_walk instead, for example avoiding taking temporary folio references if the folio does obviously not even apply and getting rid of one more follow_page() user. We cannot move all handling under the PTL, so leave the rmap handling (which implies an allocation) out. Note that zeropages obviously don't apply: old code could just have specified FOLL_DUMP. Further, we don't care about losing the secretmem check in follow_page(): these are never anon pages and vma_ksm_compatible() would never consider secretmem vmas (VM_SHARED | VM_MAYSHARE must be set for secretmem, see secretmem_mmap()). Signed-off-by: David Hildenbrand --- mm/ksm.c | 38 ++++++++++++++++++++++++-------------- 1 file changed, 24 insertions(+), 14 deletions(-) diff --git a/mm/ksm.c b/mm/ksm.c index 742b005f3f77..0f5b2bba4ef0 100644 --- a/mm/ksm.c +++ b/mm/ksm.c @@ -2564,36 +2564,46 @@ static struct ksm_rmap_item *scan_get_next_rmap_item(struct page **page) ksm_scan.address = vma->vm_end; while (ksm_scan.address < vma->vm_end) { + struct page *tmp_page = NULL; + struct folio_walk fw; + struct folio *folio; + if (ksm_test_exit(mm)) break; - *page = follow_page(vma, ksm_scan.address, FOLL_GET); - if (IS_ERR_OR_NULL(*page)) { - ksm_scan.address += PAGE_SIZE; - cond_resched(); - continue; + + folio = folio_walk_start(&fw, vma, ksm_scan.address, 0); + if (folio) { + if (!folio_is_zone_device(folio) && + folio_test_anon(folio)) { + folio_get(folio); + tmp_page = fw.page; + } + folio_walk_end(&fw, vma); } - if (is_zone_device_page(*page)) - goto next_page; - if (PageAnon(*page)) { - flush_anon_page(vma, *page, ksm_scan.address); - flush_dcache_page(*page); + + if (tmp_page) { + flush_anon_page(vma, tmp_page, ksm_scan.address); + flush_dcache_page(tmp_page); rmap_item = get_next_rmap_item(mm_slot, ksm_scan.rmap_list, ksm_scan.address); if (rmap_item) { ksm_scan.rmap_list = &rmap_item->rmap_list; - if (should_skip_rmap_item(*page, rmap_item)) + if (should_skip_rmap_item(tmp_page, rmap_item)) { + folio_put(folio); goto next_page; + } ksm_scan.address += PAGE_SIZE; - } else - put_page(*page); + *page = tmp_page; + } else { + folio_put(folio); + } mmap_read_unlock(mm); return rmap_item; } next_page: - put_page(*page); ksm_scan.address += PAGE_SIZE; cond_resched(); }