From patchwork Thu Jul 4 04:31:15 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oscar Salvador X-Patchwork-Id: 13723206 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7C715C3065C for ; Thu, 4 Jul 2024 04:33:11 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 4FDEE6B00B5; Thu, 4 Jul 2024 00:32:29 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 486866B00B7; Thu, 4 Jul 2024 00:32:29 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 2B30E6B00B8; Thu, 4 Jul 2024 00:32:29 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 07A5F6B00B5 for ; Thu, 4 Jul 2024 00:32:29 -0400 (EDT) Received: from smtpin08.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id C46EE1A1051 for ; Thu, 4 Jul 2024 04:32:28 +0000 (UTC) X-FDA: 82300798776.08.481CF2C Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.223.130]) by imf13.hostedemail.com (Postfix) with ESMTP id C534720006 for ; Thu, 4 Jul 2024 04:32:26 +0000 (UTC) Authentication-Results: imf13.hostedemail.com; dkim=none; spf=pass (imf13.hostedemail.com: domain of osalvador@suse.de designates 195.135.223.130 as permitted sender) smtp.mailfrom=osalvador@suse.de; dmarc=pass (policy=none) header.from=suse.de ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1720067522; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=9NTITUgNzypalmbjIIhE1wxYVO8aTaBxoagSewHtJpo=; b=b8S7FhzpZPVRcAOsDKNrDgGxYAETTg8vxCpKAYxoJk9T8ZeL0L6cH7pwt9VKlsJ+y9YXWf URcgSnvwwA7Vq9BD4n/62nxxObA4vgUo1jguTs89XGShmd2fBHJZFMKOuxRvUb94lC93MQ g4AE2+BIDBvl4fzcmQf4zOOoFCMf3SU= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1720067522; a=rsa-sha256; cv=none; b=ux89J8JKpx6ShaJhJxArbGL7Ut+Bd8OPVsGMkpz75DDOpGl+IEU2uUPk1Eq8BXn9gqbU2Z nieJyP1YfTMlpse0p0nPXwXfgCgoYsqbqdC+wZhf01FkboNHRlKtSrTOTf6pRJZB6yAdVI gUn3GpI2IsBj3QyY/MR1sBcnfVxtoII= ARC-Authentication-Results: i=1; imf13.hostedemail.com; dkim=none; spf=pass (imf13.hostedemail.com: domain of osalvador@suse.de designates 195.135.223.130 as permitted sender) smtp.mailfrom=osalvador@suse.de; dmarc=pass (policy=none) header.from=suse.de Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [IPv6:2a07:de40:b281:104:10:150:64:97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id 94F3A21C10; Thu, 4 Jul 2024 04:32:25 +0000 (UTC) Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 6893D13889; Thu, 4 Jul 2024 04:32:24 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id EK8YFNglhmbnDAAAD6G6ig (envelope-from ); Thu, 04 Jul 2024 04:32:24 +0000 From: Oscar Salvador To: Andrew Morton Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Peter Xu , Muchun Song , David Hildenbrand , SeongJae Park , Miaohe Lin , Michal Hocko , Matthew Wilcox , Christophe Leroy , Oscar Salvador Subject: [PATCH 28/45] mm/damon: Enable damon_mkold_pmd_entry to handle hugetlb vmas Date: Thu, 4 Jul 2024 06:31:15 +0200 Message-ID: <20240704043132.28501-29-osalvador@suse.de> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240704043132.28501-1-osalvador@suse.de> References: <20240704043132.28501-1-osalvador@suse.de> MIME-Version: 1.0 X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Rspamd-Action: no action X-Rspam-User: X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: C534720006 X-Stat-Signature: tsgdmcpgghbucpuupp784nczngue7f74 X-HE-Tag: 1720067546-266168 X-HE-Meta: U2FsdGVkX185KMx6LkKwJk4xhanSrZ6syaiH2+sDyT+ffiMIoae7nxNnxhRudwhqGKJyXe9v/VhxkyPoaspyQa3lXrJXjK5dB+J+KCZ04s9pkmFc/3ceam9YUvLV2YZrXsYkstRt6iYb9oESHWAVv+vUAZu3swaY4/rSmi//taKxcQi1sE3PJ+kG7/baQDfflLdVc8B8wSGMc8/nKEBdqk9NFFAB4FKl5Hk9cZZtM0FhKxOmITdLegtpRQs4OvZTSC1pjCe4OohBLLaKDYUKYmNZmmrtnA3MsJ21i6l7pmfD14HySzYh+kO7RHJSq5l3d/QdyRY3kpFeioLCf6UbrCI2HFiELEEKBxpEWdDhPH1Dme4ZkEjgZddhNMqtRntl22dAu15NwTX50vN95G8sWEHayncm0Jscu0yjQtXUIeLHasxYDPfqyipvgeapf9zOs6FVdkTHtGsg95HvH8M6eYoCpJtcconePKIRLWPJQ+iLkUJsLovfLQs4HFNT/YRoJUDwjoicjiflL0K7i4oZPs8rg7MyO4TylpZK8hWJcDpgB557ZFcuSwAMBgJOyKTYaGiKZIQFJUkkFXPgjn6giRJiJJgFs+vJH5tKOxQT6hxfVb7P9vBWHXOOtBgGFiRTzVxEZNU/NEoEse4uqWp0FmsQcpCHJzXtU7bpSF9YosFKgtl79aB9y7PBfjMe8zl9BspEOyVLHKqo0T7119SDp6ORwW45YVSqGJQLDuEZawd7plIrour7r+fIm1wUiItSIKLb/qYjnz3IwwsTk+FooD57WHEEBM8/jvmpku9oxHUSh28bFeWcRmb0h9yM4YZ89P8UdrY6pgV3f65tUYASJm8Wk3iGdoTC1aY40w02I/IahKYwy+13ohR9eX4hUJz0V1uCCmYxRFiHQjKY78zJ9uqJNAkyS77C93/2D0zi0AppG/ax5H5zNKLIZWxJZ6dM6cavBCaA/bHMVHLKhxl guSsn2xw bYM+2XhzCJ6QrcNXiXizGAPTLbO2R2vGbSKQBmRJ+RSGJukNmY32zPXivohRzP2BrjlrsrZlBHIRYkdYv86N4FTz2F+waY2JZ65oUBN5ZJMaDlVsBwWGs1uIEALm4TW/+nNSpCQsovv3cF2+xiRFPPD9vgkSU9j9hGdS137KpP0Eshyo= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: PMD-mapped hugetlb vmas will also reach damon_mkold_pmd_entry. Add the required code so it knows how to handle those there. Signed-off-by: Oscar Salvador --- mm/damon/ops-common.c | 21 ++++++++++++++++----- mm/damon/vaddr.c | 15 +++++---------- 2 files changed, 21 insertions(+), 15 deletions(-) diff --git a/mm/damon/ops-common.c b/mm/damon/ops-common.c index d25d99cb5f2b..6727658a3ef5 100644 --- a/mm/damon/ops-common.c +++ b/mm/damon/ops-common.c @@ -53,18 +53,29 @@ void damon_ptep_mkold(pte_t *pte, struct vm_area_struct *vma, unsigned long addr void damon_pmdp_mkold(pmd_t *pmd, struct vm_area_struct *vma, unsigned long addr) { -#ifdef CONFIG_TRANSPARENT_HUGEPAGE - struct folio *folio = damon_get_folio(pmd_pfn(pmdp_get(pmd))); +#ifdef CONFIG_PGTABLE_HAS_HUGE_LEAVES + struct folio *folio; + unsigned long size; + + if (is_vm_hugetlb_page(vma)) { + folio = pfn_folio(pdm_pfn(*pmd)) + folio_get(folio); + size = huge_page_size(hstate_vma(vma)); + } else { + folio = damon_get_folio(pmd_pfn(*pmd)); + size = PMD_SIZE; + } if (!folio) - return; + return 0; - if (pmdp_clear_young_notify(vma, addr, pmd)) + if (pmdp_test_and_clear_young(vma, addr, pmd) || + mmu_notifier_clear_young(mm, addr, addr + size)) folio_set_young(folio); folio_set_idle(folio); folio_put(folio); -#endif /* CONFIG_TRANSPARENT_HUGEPAGE */ +#endif /*CONFIG_PGTABLE_HAS_HUGE_LEAVES */ } #define DAMON_MAX_SUBSCORE (100) diff --git a/mm/damon/vaddr.c b/mm/damon/vaddr.c index 2d5ad47b9dae..47c84cdda32c 100644 --- a/mm/damon/vaddr.c +++ b/mm/damon/vaddr.c @@ -304,21 +304,16 @@ static int damon_mkold_pmd_entry(pmd_t *pmd, unsigned long addr, pmd_t pmde; spinlock_t *ptl; - if (pmd_trans_huge(pmdp_get(pmd))) { - ptl = pmd_lock(walk->mm, pmd); - pmde = pmdp_get(pmd); - - if (!pmd_present(pmde)) { + ptl = pmd_huge_lock(walk->vma, pmd); + if (ptl) { + if (!pmd_present(*pmd)) { spin_unlock(ptl); return 0; } - if (pmd_trans_huge(pmde)) { - damon_pmdp_mkold(pmd, walk->vma, addr); - spin_unlock(ptl); - return 0; - } + damon_pmdp_mkold(pmd, walk->vma, addr); spin_unlock(ptl); + return 0; } pte = pte_offset_map_lock(walk->mm, pmd, addr, &ptl);