From patchwork Thu Jul 4 04:30:48 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oscar Salvador X-Patchwork-Id: 13723179 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id B91F7C3271F for ; Thu, 4 Jul 2024 04:31:55 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id E9AAA6B0088; Thu, 4 Jul 2024 00:31:54 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id E226A6B008A; Thu, 4 Jul 2024 00:31:54 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id BD8556B008C; Thu, 4 Jul 2024 00:31:54 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id A03566B0088 for ; Thu, 4 Jul 2024 00:31:54 -0400 (EDT) Received: from smtpin10.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 29F621A105E for ; Thu, 4 Jul 2024 04:31:54 +0000 (UTC) X-FDA: 82300797348.10.54186E8 Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.223.131]) by imf26.hostedemail.com (Postfix) with ESMTP id F170E140008 for ; Thu, 4 Jul 2024 04:31:51 +0000 (UTC) Authentication-Results: imf26.hostedemail.com; dkim=pass header.d=suse.de header.s=susede2_rsa header.b=TnzHIbUf; dkim=pass header.d=suse.de header.s=susede2_ed25519 header.b=bF+fGAUR; dkim=pass header.d=suse.de header.s=susede2_rsa header.b=TnzHIbUf; dkim=pass header.d=suse.de header.s=susede2_ed25519 header.b=bF+fGAUR; dmarc=pass (policy=none) header.from=suse.de; spf=pass (imf26.hostedemail.com: domain of osalvador@suse.de designates 195.135.223.131 as permitted sender) smtp.mailfrom=osalvador@suse.de ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1720067492; a=rsa-sha256; cv=none; b=7lGp3260Unqu5BfHzHcBF/V+G1ZtfL/ZEAEMMKCXeFp2z5sAiTx7MEE+SRoWS2Ep/CE0FD evWWt8YhzB3vNERDaVklc7ry27IuSHd74F4VU5K5RTmPt2tRr1YwTH6opfxJ3EjCs065aC ITTanQTh23UjRG2lLuRwpqz7KK4RjzE= ARC-Authentication-Results: i=1; imf26.hostedemail.com; dkim=pass header.d=suse.de header.s=susede2_rsa header.b=TnzHIbUf; dkim=pass header.d=suse.de header.s=susede2_ed25519 header.b=bF+fGAUR; dkim=pass header.d=suse.de header.s=susede2_rsa header.b=TnzHIbUf; dkim=pass header.d=suse.de header.s=susede2_ed25519 header.b=bF+fGAUR; dmarc=pass (policy=none) header.from=suse.de; spf=pass (imf26.hostedemail.com: domain of osalvador@suse.de designates 195.135.223.131 as permitted sender) smtp.mailfrom=osalvador@suse.de ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1720067492; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=bJqkPqmkpON5Zxkai/QS1QclW4CGF+6WU8c6arXYcis=; b=SBnLaxmSXCvuBvBuV2F633NAnGTRNNTSG28G1JhhjKEGAZr+LVP1ZE6Y8aOmxe4tv4+fNS dCv2WMXcwfdEFhOVBhd2J9NTyZvj1auSiZZPHTGDowQQtTGo5RPXODHyXNiALUVlnmALnR uS9x8HcdIUVVQpmVPIPiHdhJmfAGulQ= Received: from imap1.dmz-prg2.suse.org (unknown [10.150.64.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id 97BAB1FCF7; Thu, 4 Jul 2024 04:31:50 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1720067510; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=bJqkPqmkpON5Zxkai/QS1QclW4CGF+6WU8c6arXYcis=; b=TnzHIbUfH5DwSZglMNw4Qi9u/goipKZZo3b5tCXWSNzXHcGWuTfIuyoCPpBaZ7EQWJZ4xP xc8s51VaseURhWTMdpAiOLKejN3kTYWph/K3arXExfBqJilkikxqtS1oEa7zsCdFYmuN0K Qf9du4CansKgid+LQOtm3331g59SMt8= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1720067510; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=bJqkPqmkpON5Zxkai/QS1QclW4CGF+6WU8c6arXYcis=; b=bF+fGAURxq2QVQDGIfnETCWk/28DcKQ+0DPeiRUpZJTGyNdki5AfAv3X8A2SjYr3HMCp0b 41nDiQe78mUbZ7Aw== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1720067510; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=bJqkPqmkpON5Zxkai/QS1QclW4CGF+6WU8c6arXYcis=; b=TnzHIbUfH5DwSZglMNw4Qi9u/goipKZZo3b5tCXWSNzXHcGWuTfIuyoCPpBaZ7EQWJZ4xP xc8s51VaseURhWTMdpAiOLKejN3kTYWph/K3arXExfBqJilkikxqtS1oEa7zsCdFYmuN0K Qf9du4CansKgid+LQOtm3331g59SMt8= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1720067510; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=bJqkPqmkpON5Zxkai/QS1QclW4CGF+6WU8c6arXYcis=; b=bF+fGAURxq2QVQDGIfnETCWk/28DcKQ+0DPeiRUpZJTGyNdki5AfAv3X8A2SjYr3HMCp0b 41nDiQe78mUbZ7Aw== Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 7C25713A82; Thu, 4 Jul 2024 04:31:49 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id YDqrGbUlhmbnDAAAD6G6ig (envelope-from ); Thu, 04 Jul 2024 04:31:49 +0000 From: Oscar Salvador To: Andrew Morton Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Peter Xu , Muchun Song , David Hildenbrand , SeongJae Park , Miaohe Lin , Michal Hocko , Matthew Wilcox , Christophe Leroy , Oscar Salvador Subject: [PATCH 01/45] arch/x86: Drop own definition of pgd,p4d_leaf Date: Thu, 4 Jul 2024 06:30:48 +0200 Message-ID: <20240704043132.28501-2-osalvador@suse.de> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240704043132.28501-1-osalvador@suse.de> References: <20240704043132.28501-1-osalvador@suse.de> MIME-Version: 1.0 X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: F170E140008 X-Stat-Signature: uw5t3syit76kz88fcwgrndxjxcdmrpyp X-Rspam-User: X-HE-Tag: 1720067511-638215 X-HE-Meta: U2FsdGVkX19cAlKlpxIgU8s5icOGpJ3r9PeOkA80yvVLL8I1IAp10poQfNbjMT10WkLKGapvSMZ+O3J7jKGfNWJ+RA6lPJLc8LsBh7fKNooLM2QU97ZXVwZfflNB1XkjrZN5eAZciFOs7XwrAeCsqo+zqC8/lFLklT5RpCFdBVTMK9Q85J6IxTkGT248OzWmudk7ELoH2Dovz+/kshVdDhehVZ+94O96xH4PUdssSoft2DycAfWDgnjUMBZWBcEUK6qnrTfA6fatwiRXjbDu1zr0JmHOLlPBGLagqdfDu+ld805amyfPora8C0xEf2ibUI2FEu4CKDWnmfe3edJHI2GEYxFpJ3EGJV8IUFHUW/PL+JgNpVjIManlHlkxrRUnDsqUqhj3CV7k2qlskh3gfd9McvVONVrls5IUQKAOCVImh3A26MyS2d2rZs43AFvncEqLmKpWySKQn3HDDmeQcaj/YCDn/vPVJ6X+kJpxYqKzStVcup6j1MIZp3IoDP4BhXPm5KBpF2k+72a448ugetxKJ3jL/N7SscFFHZ0CZPX9d0Uxhmd0TmjIMatsCGpx/91O6v3b6tMHPqYxHvyfhqh/LQmZJwiQS40LFBXXZxag69Ug1FEG+ZtnwV32uslUHR9FYrr0X3OasAXGMjSe8FARk9iZ6IAM3atmfruwyDfeuN9wP7jDKjaNV79UbnvfiAzdxRKAnju9NG3dLlsRIcE+hDbywD3px2tK2icPSiZSb3kb8R9+l4Ls+cfZUirSqLmOQsrlMwKm8hVYY/Fos8lsVdBq3XUUETOi+ONculHj3K6QPSzQnCqnfz7CI7iBMTPUslJggT0vrxMO2SGxT0Vp1gJl2LO591GU0P4Jo5s1S02y4sqNJ9r4v/upDdhsnR2FumcVHEqticBZmIo5vuX8KS+X/p5VUc7jBlbJ0c6e4X2jTdCDmZzSip24m6C/AK3Iwd/mEnkktQOWlf2 oKRJ2E9D 5EFn9n2FbR2spvmUO3pCsOioNAW/m89h/uXjSM6/NDdHuQEtcYI67Rjt7WBF0ob8pBcOvIdkxVQ7EL9XbssyhANnjVAt2IyF75yX7SrWbuWDmetFRNApEv00dhZ4LrtJcu1gjoOt5K3CxSdFoBq0V9Gy2klWzW4+Up5kX/wcrYG+ARYU7Qej6Pzt5ZfK1mNVxX9CkyzX1A5aFbXnKASlb8m50LaW0RhalDR2GRJzpESW9NIYTK1ptj3rlKS6uBWIYrmFa9dlU50+h7KFsWnk4wXCypQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: We provide generic definitions of pXd_leaf in pgtable.h when the arch do not define their own, where the generic pXd_leaf always return false. Although x86 defines {pgd,p4d}_leaf, they end up being a no-op, so drop them and make them fallback to the generic one. Signed-off-by: Oscar Salvador --- arch/x86/include/asm/pgtable.h | 10 ---------- 1 file changed, 10 deletions(-) diff --git a/arch/x86/include/asm/pgtable.h b/arch/x86/include/asm/pgtable.h index 65b8e5bb902c..772f778bac06 100644 --- a/arch/x86/include/asm/pgtable.h +++ b/arch/x86/include/asm/pgtable.h @@ -252,13 +252,6 @@ static inline unsigned long pgd_pfn(pgd_t pgd) return (pgd_val(pgd) & PTE_PFN_MASK) >> PAGE_SHIFT; } -#define p4d_leaf p4d_leaf -static inline bool p4d_leaf(p4d_t p4d) -{ - /* No 512 GiB pages yet */ - return 0; -} - #define pte_page(pte) pfn_to_page(pte_pfn(pte)) #define pmd_leaf pmd_leaf @@ -1396,9 +1389,6 @@ static inline bool pgdp_maps_userspace(void *__ptr) return (((ptr & ~PAGE_MASK) / sizeof(pgd_t)) < PGD_KERNEL_START); } -#define pgd_leaf pgd_leaf -static inline bool pgd_leaf(pgd_t pgd) { return false; } - #ifdef CONFIG_MITIGATION_PAGE_TABLE_ISOLATION /* * All top-level MITIGATION_PAGE_TABLE_ISOLATION page tables are order-1 pages From patchwork Thu Jul 4 04:30:49 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oscar Salvador X-Patchwork-Id: 13723180 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id E7FF9C3065C for ; Thu, 4 Jul 2024 04:31:57 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id ACB816B008A; Thu, 4 Jul 2024 00:31:55 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id A2A946B008C; Thu, 4 Jul 2024 00:31:55 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8A3A26B0093; Thu, 4 Jul 2024 00:31:55 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 6A1B96B008A for ; Thu, 4 Jul 2024 00:31:55 -0400 (EDT) Received: from smtpin13.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 1B5851C26DA for ; Thu, 4 Jul 2024 04:31:55 +0000 (UTC) X-FDA: 82300797390.13.98B42B5 Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.223.130]) by imf16.hostedemail.com (Postfix) with ESMTP id 2CD5518000F for ; Thu, 4 Jul 2024 04:31:52 +0000 (UTC) Authentication-Results: imf16.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=suse.de; spf=pass (imf16.hostedemail.com: domain of osalvador@suse.de designates 195.135.223.130 as permitted sender) smtp.mailfrom=osalvador@suse.de ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1720067485; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=nyBJF39GEbKS17ntB3QbFSYVgzYMJUJU6NaPzZoN+pI=; b=7VgxWpTsm9yATIwBdQeqU3+jTdWUa00IelptqQJ5kaav2bjwZy9OuRpnKEkdQKAQLyrCgv aZN+cUBDbn1CgtU7zAM3JBVzUv6tZn/m4TXSLC6m2hA7n0SCBVzCRIlNfDyAe/ypypmXmk qJUdnRZbifKFh9lWStdt91cg4lpSMog= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1720067485; a=rsa-sha256; cv=none; b=GSl5EeMtGEPPIAQGjSubMjXaHhMrncIYFDEJ9tb5JqPU8bm/uV2u5bGlg3/PAchVL7X2pd aGRxoPylknBlzS4PwxBBAKXQvi1CbQ6Qwah5kKuwdk0hOhItgDcSU1aDIw6mytNnJweNh3 7V2KfFGucpqi0+AIjt2HxJD8ZcRirn0= ARC-Authentication-Results: i=1; imf16.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=suse.de; spf=pass (imf16.hostedemail.com: domain of osalvador@suse.de designates 195.135.223.130 as permitted sender) smtp.mailfrom=osalvador@suse.de Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [IPv6:2a07:de40:b281:104:10:150:64:97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id D75C521C0D; Thu, 4 Jul 2024 04:31:51 +0000 (UTC) Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id B7C9513889; Thu, 4 Jul 2024 04:31:50 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id MGmJKLYlhmbnDAAAD6G6ig (envelope-from ); Thu, 04 Jul 2024 04:31:50 +0000 From: Oscar Salvador To: Andrew Morton Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Peter Xu , Muchun Song , David Hildenbrand , SeongJae Park , Miaohe Lin , Michal Hocko , Matthew Wilcox , Christophe Leroy , Oscar Salvador Subject: [PATCH 02/45] mm: Add {pmd,pud}_huge_lock helper Date: Thu, 4 Jul 2024 06:30:49 +0200 Message-ID: <20240704043132.28501-3-osalvador@suse.de> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240704043132.28501-1-osalvador@suse.de> References: <20240704043132.28501-1-osalvador@suse.de> MIME-Version: 1.0 X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Rspamd-Action: no action X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: 2CD5518000F X-Stat-Signature: fsacsuyfqiz19wx9z58a4bicimrsbxer X-Rspam-User: X-HE-Tag: 1720067512-164932 X-HE-Meta: U2FsdGVkX19lfpg+ECVF+kP4FN9Nw7K6fR9mfaL2dyx2TXB7nzm6lfiw9ncuMwIP88Mhn5HPtz1XOJ9NlhNGjiGAdQnDwrp9ulcdRcu4tCJmDLt+l5gmahxQYhplUI6HkdOYnTYijpGSi0cAkvi9epP64GM89TBdrcw/i5L/jCe2ScyfWsG//2YgGUfZfu8Tcdsp9Oox2S7Z4BfnSo3elpA36PBLlUeYTK3C+1ROX9aO/ZiyELAccAvE+Hm5rVpiwVMw/JOXCDKzP0Pe/Wv35Y/GJesix6bx1MWcIZEqvPeVghgf1fAjGflzAoM0DGT5lwGW8q1Vc0v28QBmWkQP9b/3BF6fIICCiNURdT/Ng3UnrQZ40onQv9HobitGRtRaHF+jcVUXkQuKbNfzGB+cZg20bN9gVbKDxnVOxcRy0x9IV3fF1uYAn2T/gYzXO3cgIW4tfNCyKdPeA+xYwKRQLDv54ksXrjGPG+rYSPxH2vHxFJSI4DdCKqQ98lm44H9wd4O5F2yzQw0R6zmRHLA6W9ErZkLDVt+6d0ipf6IK6oLCdsghpV5ClXB8Xj3OQjUVku0O4tQgln+ABBtbXTnx3uZctbskSfW1VSByjot6YD2uLqsPseOgnBGhrYyGddoHblPg93lyXvQgCyvKhHq7EnBBboB2XWj9KjUuf8WrSn50PjdPnKqygYqlh/8a6n5hDNVNPuiTIVqS5InK0vVsDDN7RPLOgFvaWLDvrmmVDFOpT0GLlzIMyETeOEYq5vBG5ZdPGEDbKRnpXbbvm20UyFh4QPJr/MZxYgshRRtjpPYNB3jyG1pQnRBFKKmnmBauwcYtQi7eRjja046/wp87uUPsmRSv/sUKBEZJQh8E0SyaZJAhgGWK40zAZFq2CyWwAbh4Spg8iRnX0X7NDUyFuwceDuCnzROC9Ql8Fxyr2e8SSRuwbLZRh8dvNukEgoCNwiV0C0kmAzWlrOd6MWr rnwvWQA3 NvWQrRGJ/8DPAOxt++9De+yZnKitj3TujEC/k+CGgo001rfAELB+MJh2MTiTzHcrvUepQH7YTlRdGZXZh/Fs209FBV8A0Wl/NhcLWLvo48XDCames0zV9GkshIaSi2QBqAqlXey3ZVWCB8B2NjNOnv+PxH9J2sbmdC2m9iJoEVZTpB+k= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Deep down hugetlb and thp use the same lock for pud and pmd. Create two helpers that can be directly used by both of them, as they will be used in the generic pagewalkers. Signed-off-by: Oscar Salvador --- include/linux/mm_inline.h | 27 +++++++++++++++++++++++++++ 1 file changed, 27 insertions(+) diff --git a/include/linux/mm_inline.h b/include/linux/mm_inline.h index f4fe593c1400..93e3eb86ef4e 100644 --- a/include/linux/mm_inline.h +++ b/include/linux/mm_inline.h @@ -9,6 +9,7 @@ #include #include #include +#include /** * folio_is_file_lru - Should the folio be on a file LRU or anon LRU? @@ -590,4 +591,30 @@ static inline bool vma_has_recency(struct vm_area_struct *vma) return true; } +static inline spinlock_t *pmd_huge_lock(pmd_t *pmd, struct vm_area_struct *vma) +{ + spinlock_t *ptl; + + if (pmd_leaf(*pmd)) { + ptl = pmd_lock(vma->vm_mm, pmd); + if (pmd_leaf(*pmd)) + return ptl; + spin_unlock(ptl); + } + return NULL; +} + +static inline spinlock_t *pud_huge_lock(pud_t *pud, struct vm_area_struct *vma) +{ + spinlock_t *ptl = pud_lock(vma->vm_mm, pud); + + if (pud_leaf(*pud)) { + ptl = pud_lock(vma->vm_mm, pud); + if (pud_leaf(*pud)) + return ptl; + spin_unlock(ptl); + } + return NULL; +} + #endif From patchwork Thu Jul 4 04:30:50 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oscar Salvador X-Patchwork-Id: 13723181 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id AE8CBC3065C for ; Thu, 4 Jul 2024 04:32:00 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 26C126B0093; Thu, 4 Jul 2024 00:31:57 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 17BE96B0095; Thu, 4 Jul 2024 00:31:57 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id F1EE06B0096; Thu, 4 Jul 2024 00:31:56 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id CD4F46B0093 for ; Thu, 4 Jul 2024 00:31:56 -0400 (EDT) Received: from smtpin14.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 73E9880A47 for ; Thu, 4 Jul 2024 04:31:56 +0000 (UTC) X-FDA: 82300797432.14.0306A0A Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.223.130]) by imf23.hostedemail.com (Postfix) with ESMTP id 701AE140005 for ; Thu, 4 Jul 2024 04:31:54 +0000 (UTC) Authentication-Results: imf23.hostedemail.com; dkim=none; spf=pass (imf23.hostedemail.com: domain of osalvador@suse.de designates 195.135.223.130 as permitted sender) smtp.mailfrom=osalvador@suse.de; dmarc=pass (policy=none) header.from=suse.de ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1720067490; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=P6aZ8d2FMHtbdVf956Be96e8ZvIEt8d4YS/FK2WFXU4=; b=jpDKj+PYGOSJ3V820V/j3dCpRIvHoJaBjLqiIZsYVsbbkwUnn1CyxX7r407l1CA+NRpDsJ DU1o0i+2K8iig6yyiB9/9+7dWmVSJkDjYIK3mhBqV5Tifa8wYknKPkF8ambt6U8rD9KKj8 OGbCA+fEM3OHWt+8clBZTejDOxUeR7U= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1720067490; a=rsa-sha256; cv=none; b=tpImOdrpdr5UY0gTP8puDn0jDTWyiq4PX9bENYDw5/p86HlYb6RhEOJ6zyMr81jx14EoCG 52Jlt0QR+UufNODfN4ZG/AzeQDV10XOz+24NtgM1sXpBHH1HFYrhzYvxyeLa3njYhYx53a 4nbf1Onz2bJ1qhDhrirvWWAyI7Tauog= ARC-Authentication-Results: i=1; imf23.hostedemail.com; dkim=none; spf=pass (imf23.hostedemail.com: domain of osalvador@suse.de designates 195.135.223.130 as permitted sender) smtp.mailfrom=osalvador@suse.de; dmarc=pass (policy=none) header.from=suse.de Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [IPv6:2a07:de40:b281:104:10:150:64:97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id 359EE21180; Thu, 4 Jul 2024 04:31:53 +0000 (UTC) Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 044A513889; Thu, 4 Jul 2024 04:31:51 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id 4D3mN7clhmbnDAAAD6G6ig (envelope-from ); Thu, 04 Jul 2024 04:31:51 +0000 From: Oscar Salvador To: Andrew Morton Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Peter Xu , Muchun Song , David Hildenbrand , SeongJae Park , Miaohe Lin , Michal Hocko , Matthew Wilcox , Christophe Leroy , Oscar Salvador Subject: [PATCH 03/45] mm/pagewalk: Move vma_pgtable_walk_begin and vma_pgtable_walk_end upfront Date: Thu, 4 Jul 2024 06:30:50 +0200 Message-ID: <20240704043132.28501-4-osalvador@suse.de> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240704043132.28501-1-osalvador@suse.de> References: <20240704043132.28501-1-osalvador@suse.de> MIME-Version: 1.0 X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Rspamd-Action: no action X-Rspam-User: X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 701AE140005 X-Stat-Signature: j78pj19c94fm5topmxcpmdte4tdzr887 X-HE-Tag: 1720067514-745561 X-HE-Meta: U2FsdGVkX1+PRHP0dLh9JszCZ9hjlJaksBdLh2HUMZObMo4X6bPbenz3hZPQrCA0jdV2hOxbvxE9JSSlKWs763tWm981vW21fDYcFgrfu8DCocKkaWSLGjxoeb9B5lG46kal3lkuD4BFC1Nl/T3sQvzFcGksX3cDp2kP8PNW3ANsPaAt97U4K3DXCCLm122VcYBRWICTzYc4f4Y46n8DLeS+boG/+Z8W1lhd03K/U8uTjpGOFIyV54qQb7bCYb8umJ4rGJ2YTTdPR8FD9D8Q4C5zo2WnqsNnSrVIqTiCNTcKSw1OipD8X9p/sb5M+k4LPxn8tLv2NfwzMwu5czayGhtSt8ZdaDydg3DCT7bCj+0VXNh2EooiOgqGm6GrH/8F2ZvzsbZ5xKhSO3S1qAJw+7Isb4yg2ejegKgv7+fW8fZTXFIvx4nTfW5HeMAnZvadBBRXx+FsRQT4hPVkGl34yxFvOYlJZrhI25Pt1gzZRyr4k5wBG9U90EF2hJ4tM92sfTrwcj3saDx+I8GtkMnhwmFAE07M3/yj9KB1N3chWxY+Mdyr3rI5vMK2klEF4gxSBiXYsNuKYCraA8taF6ZAgFtw4+I/Ar+VIlva9JVWFfYisS3BegjBkrKDvIYSux45HohVZI2Oa0Q7c1plp0Wx9AynMaxFfAPqiHD3kxLsR6aC6JRpjVtQlzNK2oGklCEf5O7mrJ2vRQtRnL5z/2ZrsWqUNMzH8GBU9vtF8ZvEaxbL+sYyzX9cQSBf5udbD5xPjJp4+UUsueYAi2baaeakjDlTk9cD3ZejWxFCLJF2ZOKUkQ2UVMaNxJjiRHmy3kwrx/ABVwtp4U3fwgdS61ktNZ3jpoaqO8ClBuf500TOa9ahSJnftHG9YP6vugAQdA4sEVh7AkAacdPMW4rDISwfyEd2tEizsZETJqMB8GFhSKHj8KadPB2IjdycRlOBMSg/6RnKcvBFJ1GuQyqVdOu 6adzMRF7 0QD3AKVO3pXvlK3qYi4SYCuJujXasBVtwkmtCKqAbcKLisQxl/1du+thy/oXR08XmUm2fptVn/uvBTxRbSmTa6eqcR/+e2OfU4xZTGz7PARMPv6P3ep+SwHJZFUSBjW+W4aNbSf1C5UyR8OQvNriFCju1sCwnwc2Av9Zh8ni2vpm9ZEk= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: In order to prepare walk_pgd_range for handling hugetlb pages, move the hugetlb vma locking into __walk_page_range. Signed-off-by: Oscar Salvador --- mm/pagewalk.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/mm/pagewalk.c b/mm/pagewalk.c index ae2f08ce991b..eba705def9a0 100644 --- a/mm/pagewalk.c +++ b/mm/pagewalk.c @@ -269,7 +269,6 @@ static int walk_hugetlb_range(unsigned long addr, unsigned long end, const struct mm_walk_ops *ops = walk->ops; int err = 0; - hugetlb_vma_lock_read(vma); do { next = hugetlb_entry_end(h, addr, end); pte = hugetlb_walk(vma, addr & hmask, sz); @@ -280,7 +279,6 @@ static int walk_hugetlb_range(unsigned long addr, unsigned long end, if (err) break; } while (addr = next, addr != end); - hugetlb_vma_unlock_read(vma); return err; } @@ -339,11 +337,13 @@ static int __walk_page_range(unsigned long start, unsigned long end, return err; } + vma_pgtable_walk_begin(vma); if (is_vm_hugetlb_page(vma)) { if (ops->hugetlb_entry) err = walk_hugetlb_range(start, end, walk); } else err = walk_pgd_range(start, end, walk); + vma_pgtable_walk_end(vma); if (ops->post_vma) ops->post_vma(walk); From patchwork Thu Jul 4 04:30:51 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oscar Salvador X-Patchwork-Id: 13723182 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 176A1C30653 for ; Thu, 4 Jul 2024 04:32:03 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 685D46B0095; Thu, 4 Jul 2024 00:31:58 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 60D446B0096; Thu, 4 Jul 2024 00:31:58 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 415C46B0098; Thu, 4 Jul 2024 00:31:58 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 142EA6B0095 for ; Thu, 4 Jul 2024 00:31:58 -0400 (EDT) Received: from smtpin24.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id BEE53A0FC9 for ; Thu, 4 Jul 2024 04:31:57 +0000 (UTC) X-FDA: 82300797474.24.0C67F49 Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.223.130]) by imf01.hostedemail.com (Postfix) with ESMTP id B76864001D for ; Thu, 4 Jul 2024 04:31:55 +0000 (UTC) Authentication-Results: imf01.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=suse.de; spf=pass (imf01.hostedemail.com: domain of osalvador@suse.de designates 195.135.223.130 as permitted sender) smtp.mailfrom=osalvador@suse.de ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1720067495; a=rsa-sha256; cv=none; b=EMy7/B0bLVPEusljspsE0FuXUaBnp2GN/S7y8NC37/eH+gRuqUY2572ReknCZ+4sXFxpiM R1FobHTW8gGgaCDgV6BCwvc+CDAWaPWHi3rmpXrx2vOOrg3FyVxLs/2Db3OKzCZJEogN/B h5j1PHQ+FVAXrcc31L/dfWx0WPXB3gk= ARC-Authentication-Results: i=1; imf01.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=suse.de; spf=pass (imf01.hostedemail.com: domain of osalvador@suse.de designates 195.135.223.130 as permitted sender) smtp.mailfrom=osalvador@suse.de ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1720067495; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=LPg1N4K/oKRWLAqwnTzV+x9t1Eetk6pBQBgkyAoigug=; b=3zQq1Xr4IlT60KlOzUgPsl60Zb4jEecgiLzPv9t8gZKT0H7E9kVP4P3qywgIE/XH6yx+Qj 7wdxmfZQrt6NYZXSbpBIiT4owtR2zyiyFL8iDF3Gw7seT/DieFe/Y+NLU/+xms+iX4euDA l5SOSu7DrVaWQelwRuc9+gIn/CYn5CA= Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [IPv6:2a07:de40:b281:104:10:150:64:97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id 8546721C10; Thu, 4 Jul 2024 04:31:54 +0000 (UTC) Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 5593A13889; Thu, 4 Jul 2024 04:31:53 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id SMuLELklhmbnDAAAD6G6ig (envelope-from ); Thu, 04 Jul 2024 04:31:53 +0000 From: Oscar Salvador To: Andrew Morton Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Peter Xu , Muchun Song , David Hildenbrand , SeongJae Park , Miaohe Lin , Michal Hocko , Matthew Wilcox , Christophe Leroy , Oscar Salvador Subject: [PATCH 04/45] mm/pagewalk: Only call pud_entry when we have a pud leaf Date: Thu, 4 Jul 2024 06:30:51 +0200 Message-ID: <20240704043132.28501-5-osalvador@suse.de> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240704043132.28501-1-osalvador@suse.de> References: <20240704043132.28501-1-osalvador@suse.de> MIME-Version: 1.0 X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Rspamd-Action: no action X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: B76864001D X-Stat-Signature: kmo9ysqa5rz4xts469pksfqeujzjgphu X-Rspam-User: X-HE-Tag: 1720067515-754563 X-HE-Meta: U2FsdGVkX19BJnsbZ+YJuR+sEKXiWiNMROcbSYHw7qppraSG+e0mjzhRK18KmFl/NC21xN7+B2VzFSmM/vQi1P+C+tjOeQ3v1xTT/C8nR7+eroeI/qmyQTU/MLSPOAHp216zhz7oKO8nYjYc3woDL2uMZtKYIZaHG9OBzruvQ316jPJlOrmy3aTWh+Bm2BCufWhubS2QI8w/KPqKiuwNlGMFzZ/uitHcRZZrtizdHAlDodb17CLC29dN/TXczCG8FPwyDF55R7Xir1H3g6vUuay/fcKxxrb8tocKeAoThWI+pqYY/J51wZfOBD69x38ax7v0CvOPg2VqQAH+C7CI3YAISi1weouuloVeg+kEHJ1kkqrSV66QVvNNaDXkHGmHEsin6yVACGTo9mBmmFcTrALrjsz3PY2rG8Xue90KRACUUpliqLx+MULn0vS5zrtDWYg5v7BRlVwo1QP8ncml6M4QqTyfGlBZ2OQ7+T/3uZsm1Y7XDHyew/LXuIwRY6bvDQhvMujdm9FcMMlTeAkph3eVpTZJgr+4lHBpAGqUY7Strt8D8InXwZuNOcgXVd/CJ1u/MtkFu2o3k2QokzmwlvITNcS8O54yeZHi45RqE++mWiBgW1Jc0hD88lKlpizTcN2F6MIxxHCkrgUqMAvampq7+8DSTgZNXAy/4qa5H6wEAkq2r1HwpgXGOokROe/dEK1npQImH6UBM/xgjSdkL0aJ8IanJXdPHU20+33/UorV6ORd0RRaIbk73De5EB+K6cfih5XlEcjs6O2R7yDLDE1GGcQTUWuo3+e7uMinUGyQIUjpbjFPD3/91+FPLVuc1bNbcB59DkB6tDB2nUWZdan1PS7SLS6IiHVJygl8peS+tAGCW3JfdQsa2Yd0k4epW6FWkk4w2Vf/80RrsARL7BOLFWql2WqTyIpYDF6mr+O6D5uAPa9yFeF0LvKsbJKUIWeI7rSpYD7mERHw0hP B5J0cfuX YdNdmWjL9WRo+6+1RWYKou615bVZVwCUj0Ypf1Bm3sROzgDkc0l2iRoYLxeB2O/aUGpys1W73bp23n9dYJUSueXugByRVAg4aPL+cZe4/DhA9qQ/lLIvt26myHBLGuL5eFAW8eJnYOLo+sW4hwaBgfPfEFZjKGjMtP93jwdNuUu7GL0k= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Check first whether the pud is leaf one before trying to call pud_entry. Signed-off-by: Oscar Salvador --- mm/pagewalk.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/mm/pagewalk.c b/mm/pagewalk.c index eba705def9a0..d93e77411482 100644 --- a/mm/pagewalk.c +++ b/mm/pagewalk.c @@ -155,7 +155,7 @@ static int walk_pud_range(p4d_t *p4d, unsigned long addr, unsigned long end, walk->action = ACTION_SUBTREE; - if (ops->pud_entry) + if (ops->pud_entry && pud_leaf(*pud)) err = ops->pud_entry(pud, addr, next, walk); if (err) break; From patchwork Thu Jul 4 04:30:52 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oscar Salvador X-Patchwork-Id: 13723183 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id E19F6C3065C for ; Thu, 4 Jul 2024 04:32:05 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 28B606B0098; Thu, 4 Jul 2024 00:32:00 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 17B526B0099; Thu, 4 Jul 2024 00:32:00 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E98466B009A; Thu, 4 Jul 2024 00:31:59 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id BEDFA6B0098 for ; Thu, 4 Jul 2024 00:31:59 -0400 (EDT) Received: from smtpin20.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 50DD2C0A37 for ; Thu, 4 Jul 2024 04:31:59 +0000 (UTC) X-FDA: 82300797558.20.607A16D Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.223.131]) by imf25.hostedemail.com (Postfix) with ESMTP id 283F1A000E for ; Thu, 4 Jul 2024 04:31:56 +0000 (UTC) Authentication-Results: imf25.hostedemail.com; dkim=pass header.d=suse.de header.s=susede2_rsa header.b=h+c3YldW; dkim=pass header.d=suse.de header.s=susede2_ed25519 header.b=frfAxxwF; dkim=pass header.d=suse.de header.s=susede2_rsa header.b=h+c3YldW; dkim=pass header.d=suse.de header.s=susede2_ed25519 header.b=frfAxxwF; spf=pass (imf25.hostedemail.com: domain of osalvador@suse.de designates 195.135.223.131 as permitted sender) smtp.mailfrom=osalvador@suse.de; dmarc=pass (policy=none) header.from=suse.de ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1720067493; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=hCzlv35dbgXloc5Cmstin2CCiJG29oVOef/hhtOkA+o=; b=8Q/G0ePPzsjb5bQjTGTWlikmjRvt10ULJL1cRF8w+beLsr2rJwZ/1dnsd8s8OFbhRUB0Uc qADVuA85OL+hvZIpxYNDOTQ6Fx7rKWOxbfGhXT91nHHrL9kOni0FfuMkBaZFf4DEKyXBhf Dhk6CZ/Cer4tU9kJhx9n1PgMVIBQCvs= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1720067493; a=rsa-sha256; cv=none; b=npYl3l5zRwF4weZvPC3gGZ+mYEHW0xpBGghraY4xRHazt6RzesYFHn7PErn+PhNg+m+2fp OjjjGqjWZCFUWITDV8N9RbhRPwrUp4W0+qq8FVOPUsJy3B7HxYUfUwRxLLbSepgA/tWCcM 5wDBLeFA6gyik3BSPHGObeK4X1LJbUU= ARC-Authentication-Results: i=1; imf25.hostedemail.com; dkim=pass header.d=suse.de header.s=susede2_rsa header.b=h+c3YldW; dkim=pass header.d=suse.de header.s=susede2_ed25519 header.b=frfAxxwF; dkim=pass header.d=suse.de header.s=susede2_rsa header.b=h+c3YldW; dkim=pass header.d=suse.de header.s=susede2_ed25519 header.b=frfAxxwF; spf=pass (imf25.hostedemail.com: domain of osalvador@suse.de designates 195.135.223.131 as permitted sender) smtp.mailfrom=osalvador@suse.de; dmarc=pass (policy=none) header.from=suse.de Received: from imap1.dmz-prg2.suse.org (unknown [10.150.64.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id BD80C1FCFB; Thu, 4 Jul 2024 04:31:55 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1720067515; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=hCzlv35dbgXloc5Cmstin2CCiJG29oVOef/hhtOkA+o=; b=h+c3YldWsDCWou0FH1uqAUtnGxvDkmkrw5MQS+eAbm6nc4xHjoC7SiMmO1KgT1ogxMucRW JsZAInX2H9tHjZJ29ry01mlBA/1iNay0rlLOoCB6DCVmON6lUqsmtsj4w0Z5P4bSRXrmdd E1ywbhdn4+EiM/LJbQhJoM39l9zXjNI= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1720067515; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=hCzlv35dbgXloc5Cmstin2CCiJG29oVOef/hhtOkA+o=; b=frfAxxwFBvqVUDaANiozJD91Zhfd1mCsF2hzZSdarkYr3nve6JYZaO/PFTJ73t684ex8JF l+vQvnp2Om9NkABQ== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1720067515; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=hCzlv35dbgXloc5Cmstin2CCiJG29oVOef/hhtOkA+o=; b=h+c3YldWsDCWou0FH1uqAUtnGxvDkmkrw5MQS+eAbm6nc4xHjoC7SiMmO1KgT1ogxMucRW JsZAInX2H9tHjZJ29ry01mlBA/1iNay0rlLOoCB6DCVmON6lUqsmtsj4w0Z5P4bSRXrmdd E1ywbhdn4+EiM/LJbQhJoM39l9zXjNI= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1720067515; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=hCzlv35dbgXloc5Cmstin2CCiJG29oVOef/hhtOkA+o=; b=frfAxxwFBvqVUDaANiozJD91Zhfd1mCsF2hzZSdarkYr3nve6JYZaO/PFTJ73t684ex8JF l+vQvnp2Om9NkABQ== Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id A7EBF13889; Thu, 4 Jul 2024 04:31:54 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id eHJfJLolhmbnDAAAD6G6ig (envelope-from ); Thu, 04 Jul 2024 04:31:54 +0000 From: Oscar Salvador To: Andrew Morton Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Peter Xu , Muchun Song , David Hildenbrand , SeongJae Park , Miaohe Lin , Michal Hocko , Matthew Wilcox , Christophe Leroy , Oscar Salvador Subject: [PATCH 05/45] mm/pagewalk: Enable walk_pmd_range to handle cont-pmds Date: Thu, 4 Jul 2024 06:30:52 +0200 Message-ID: <20240704043132.28501-6-osalvador@suse.de> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240704043132.28501-1-osalvador@suse.de> References: <20240704043132.28501-1-osalvador@suse.de> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 283F1A000E X-Stat-Signature: 6jqbpshurxugbata7f56yik4cwcjyphz X-HE-Tag: 1720067516-466164 X-HE-Meta: U2FsdGVkX1/2rtjZwwG3o7RWvZqBnfcf05T7oPTGfQ3jeNQflRznQP5WqDDTJN2ghw+4GpnQNads0AS5ViF90bmrH4AF7mcmBUL9WyniQcmhT7isobcjS078FqiAN2YH5yGhKcAw4Nn5PQukKdoiYF+Km6x9T2Ua23B8WiPmXtYMxHFn2T9Kosen15z1klGz/88wUC27knKoreG1a8oLlEw0cKuVgUT3waKfBuPOOwnxeC6Q1wVEsQ02/UalZV0NShtHPoR5sdbXIXsWUkeO2eKllIHNKGdqObHrsMsAmoKbO9pfy7z75eW6ls8HqRyOCDyZRbmg3cDfTo/TvvIreZSc5bHI9UcGxbWXTmGwDMm/nlIKIodezti55gL0DV9g+EUE/1AlDxKq8qvxB86H16YVSSU+FXhlWPoAts8Ei0LF9o6rB3OV8YIqo5MxJxff6CTXKxE6ql5Mog+pS+4lz2EMrKXMK9gpOX8OgtTF4rKl+ikxUS+foIccMpgj0cEk0NWxoBv2fgAdjyj3tmz7CSXatoj/Qkw8/cXlX8gvBre/XaOM5fy11rw0ZjOU2uASnymwqvvlyqrDxon23CwNjVE8V0P24DnUQCSAgr6qB9qXpmoyiGEd4ek2W90k5q+C4n6EJclgIb/2SXvPg3A+tCnnExo52xUU6xWRArwIC+fALqznnSu1JnFPcqE0hmc/OPZBS633k72aglWA37PiLgQbOcgTSjPjt/hKTnR5egg1ulRLA2nhdJrKXxPvUBBgfCnK1XLl69odCJLPT3baTQLbrncPjASNCix5jnxFKsXoc4NyIEhEvoXy7QXfP89RscWXzySvdiE6Tq9kldi78yQok7oicrUyFiCodZcuG0FK3ls4FK42mP0X8dKd4XA8jCGdUrp0MRcpYL23+oQRhPdnEey58u/eSTHZy/5rkxDapL2bygUYhJZddbWzpnHJHhtFHvwWwcZIFYFPHCx GWNO9eY/ La/1CRfPi3ooXQi71XdUZf/+pj+YF1nc8w82c04WyVQ0mm/WWEUeDNheT9qVzjXDLbvM3F4WpSp2vw5BTtgnizM7winEJ/3v6K2pKi7XWXd0xFJUI40lS1clJuK4QkVDa1AHvHiJoA1u2MXcIbB41kcGr6TRNRYNQdaseJnCe6PLz8zOSoFkeVtyLWjU+r82BmM8qGa6bFZUEfZVeOs4jpRnWKRsB+h2GWraGvH9b5wUizvkJYnBrwwL8GnlajOB6z62HJ+32ZMKwfLjPE4N5WebOYg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: HugeTLB pages can be cont-pmd mapped, so teach walk_pmd_range to handle those. This will save us some cycles as we do it in one-shot instead of calling in multiple times. Signed-off-by: Oscar Salvador --- include/linux/pgtable.h | 12 ++++++++++++ mm/pagewalk.c | 12 +++++++++--- 2 files changed, 21 insertions(+), 3 deletions(-) diff --git a/include/linux/pgtable.h b/include/linux/pgtable.h index 2a6a3cccfc36..3a7b8751747e 100644 --- a/include/linux/pgtable.h +++ b/include/linux/pgtable.h @@ -1914,6 +1914,18 @@ typedef unsigned int pgtbl_mod_mask; #define __pte_leaf_size(x,y) pte_leaf_size(y) #endif +#ifndef pmd_cont +#define pmd_cont(x) false +#endif + +#ifndef CONT_PMD_SIZE +#define CONT_PMD_SIZE 0 +#endif + +#ifndef CONT_PMDS +#define CONT_PMDS 0 +#endif + /* * We always define pmd_pfn for all archs as it's used in lots of generic * code. Now it happens too for pud_pfn (and can happen for larger diff --git a/mm/pagewalk.c b/mm/pagewalk.c index d93e77411482..a9c36f9e9820 100644 --- a/mm/pagewalk.c +++ b/mm/pagewalk.c @@ -81,11 +81,18 @@ static int walk_pmd_range(pud_t *pud, unsigned long addr, unsigned long end, const struct mm_walk_ops *ops = walk->ops; int err = 0; int depth = real_depth(3); + int cont_pmds; pmd = pmd_offset(pud, addr); do { again: - next = pmd_addr_end(addr, end); + if (pmd_cont(*pmd)) { + cont_pmds = CONT_PMDS; + next = pmd_cont_addr_end(addr, end); + } else { + cont_pmds = 1; + next = pmd_addr_end(addr, end); + } if (pmd_none(*pmd)) { if (ops->pte_hole) err = ops->pte_hole(addr, next, depth, walk); @@ -126,8 +133,7 @@ static int walk_pmd_range(pud_t *pud, unsigned long addr, unsigned long end, if (walk->action == ACTION_AGAIN) goto again; - - } while (pmd++, addr = next, addr != end); + } while (pmd += cont_pmds, addr = next, addr != end); return err; } From patchwork Thu Jul 4 04:30:53 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oscar Salvador X-Patchwork-Id: 13723184 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 79C93C30653 for ; Thu, 4 Jul 2024 04:32:08 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 090C76B0099; Thu, 4 Jul 2024 00:32:01 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id F35096B009A; Thu, 4 Jul 2024 00:32:00 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D3A586B009B; Thu, 4 Jul 2024 00:32:00 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id AAC2A6B0099 for ; Thu, 4 Jul 2024 00:32:00 -0400 (EDT) Received: from smtpin23.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 65DF11A0E8D for ; Thu, 4 Jul 2024 04:32:00 +0000 (UTC) X-FDA: 82300797600.23.B1F1ED1 Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.223.131]) by imf30.hostedemail.com (Postfix) with ESMTP id 451898001A for ; Thu, 4 Jul 2024 04:31:58 +0000 (UTC) Authentication-Results: imf30.hostedemail.com; dkim=pass header.d=suse.de header.s=susede2_rsa header.b="JZiVO/eg"; dkim=pass header.d=suse.de header.s=susede2_ed25519 header.b="yNbA1i/f"; dkim=pass header.d=suse.de header.s=susede2_rsa header.b="JZiVO/eg"; dkim=pass header.d=suse.de header.s=susede2_ed25519 header.b="yNbA1i/f"; spf=pass (imf30.hostedemail.com: domain of osalvador@suse.de designates 195.135.223.131 as permitted sender) smtp.mailfrom=osalvador@suse.de; dmarc=pass (policy=none) header.from=suse.de ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1720067495; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=R/rRNjZ7G0L51JRh0RrYbiNiKS2AE7CnLW4VygCa50U=; b=FN4F87yv05pFrewOxBssAiVmPKI3PFiRttKoKTTTV3f/MQR9q10IpRiov/dM7ehPn/YAkw 1n0/wjC4icMpPWBRCQM6nVQb8y76VA6PCTseXy9BHw3uhqH28Koqh5CUSrE0X9AYMJpRL4 QKSvOlLDeMcq6Z7PNOuBs3kNkopJXGM= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1720067495; a=rsa-sha256; cv=none; b=fEbpP6al6ukHJF2advlVuzqwHxTszzfsTZ1FwUYNPwnzSuIsmzNavQsFe1zsQM7lBtqImj uO5oUHPp5wITTmdtvaRq6oLxT+Klx7sUzWA8zCtS/0jyOjUCis5v12bAJR+pqRwg3IR/br gX+GzxF1UOUmcHVzFGzsnVN73J9KVCs= ARC-Authentication-Results: i=1; imf30.hostedemail.com; dkim=pass header.d=suse.de header.s=susede2_rsa header.b="JZiVO/eg"; dkim=pass header.d=suse.de header.s=susede2_ed25519 header.b="yNbA1i/f"; dkim=pass header.d=suse.de header.s=susede2_rsa header.b="JZiVO/eg"; dkim=pass header.d=suse.de header.s=susede2_ed25519 header.b="yNbA1i/f"; spf=pass (imf30.hostedemail.com: domain of osalvador@suse.de designates 195.135.223.131 as permitted sender) smtp.mailfrom=osalvador@suse.de; dmarc=pass (policy=none) header.from=suse.de Received: from imap1.dmz-prg2.suse.org (unknown [10.150.64.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id 0726C1FCF9; Thu, 4 Jul 2024 04:31:57 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1720067517; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=R/rRNjZ7G0L51JRh0RrYbiNiKS2AE7CnLW4VygCa50U=; b=JZiVO/egdO1NVZ9h89++ifx1zQVaAaRFVXTNxdvP7+Eqp6m0SFCnQDqtDBNPLeRwhOr/3k RIAR/120aKOEkaNyNgfpVNaziAWXWsQJTL2b+KTKfAdROuN3yyhKk9oEtfYU7aWFSE8j9d IVYd41OVz8gn1ZHBdLdnAJCiolhzEpA= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1720067517; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=R/rRNjZ7G0L51JRh0RrYbiNiKS2AE7CnLW4VygCa50U=; b=yNbA1i/f3g+Uk1+/g4XzzNXwTBw31YbefcoyfvVe9XSdiq9mU3Rgm3u5T2CICi+Xi0VqWq O+llDAuqdN+YiJDA== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1720067517; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=R/rRNjZ7G0L51JRh0RrYbiNiKS2AE7CnLW4VygCa50U=; b=JZiVO/egdO1NVZ9h89++ifx1zQVaAaRFVXTNxdvP7+Eqp6m0SFCnQDqtDBNPLeRwhOr/3k RIAR/120aKOEkaNyNgfpVNaziAWXWsQJTL2b+KTKfAdROuN3yyhKk9oEtfYU7aWFSE8j9d IVYd41OVz8gn1ZHBdLdnAJCiolhzEpA= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1720067517; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=R/rRNjZ7G0L51JRh0RrYbiNiKS2AE7CnLW4VygCa50U=; b=yNbA1i/f3g+Uk1+/g4XzzNXwTBw31YbefcoyfvVe9XSdiq9mU3Rgm3u5T2CICi+Xi0VqWq O+llDAuqdN+YiJDA== Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id DE2E513A82; Thu, 4 Jul 2024 04:31:55 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id WPQSMrslhmbnDAAAD6G6ig (envelope-from ); Thu, 04 Jul 2024 04:31:55 +0000 From: Oscar Salvador To: Andrew Morton Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Peter Xu , Muchun Song , David Hildenbrand , SeongJae Park , Miaohe Lin , Michal Hocko , Matthew Wilcox , Christophe Leroy , Oscar Salvador Subject: [PATCH 06/45] mm/pagewalk: Do not try to split non-thp pud or pmd leafs Date: Thu, 4 Jul 2024 06:30:53 +0200 Message-ID: <20240704043132.28501-7-osalvador@suse.de> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240704043132.28501-1-osalvador@suse.de> References: <20240704043132.28501-1-osalvador@suse.de> MIME-Version: 1.0 X-Rspamd-Queue-Id: 451898001A X-Stat-Signature: rgpjtwi5c6chznfi9gtbkx7mbw44ysry X-Rspamd-Server: rspam09 X-Rspam-User: X-HE-Tag: 1720067518-54119 X-HE-Meta: U2FsdGVkX19qCBpXt24/z67w5tC5skbTP1JFyQ/GyR4g5SQqlQv9dDRi+taPaA9weSQJ8ZAdJJdpEXNflQF6QGHIzRWLjlB11QPHhBiLdtdV/s0wU1LmyTMTTnSi8nFLuJdi7mxlIugX57kFDQP4Fk/lgrnidaTOn1vqxUJEwMD2hozfFKSG88qni/notGZ1G7xqk9NRIGApHjG3ri7NOf5XG/rPq7XgdZOORjGJbk16rNiGB/pmU4GQWYCn0V8G5zB+njln4mkz+3pPfffSw3kXZy1nVPkMayeUOxj+1XDoTifRJrgo/AiFsJSyq4soPhgyLq8AYPE5qjjGRu2gC5NxihJf7YS6u3Uf35xTdyF1ZgCoh0NY6khDp7viT2JpB9PlOev9KDzPJgGxWf1QlXLqX/gOdOBEqYBx0FgtDMxQR/W6BFw6g3SpVDaMEs1EzPdVI+2ufHqHACHYiqPts85Zw8Hf3y6U+0uaeP7mL5eNQQFzmGf02kt0t8Dv2xlMaZuRX8OZq8FzvtEl9d0B054S+6xXgrMk1XnCbrKJCD++GTTwR7pNBKWXV1aht5smIcYM6hbkcducqXscXgPH8Qar4E8440hNMWJI3pgKn64fkKEymXF/CcVnJV4Jx5R2vIbkOoPSMtcoWMM3MvorLfDP7xdF46c516W8gyCELT9so9WFpQwEmLNpiwx/NVzoDw8pbH/MwbXj2pqLp3bQkJRS8ZshI+HKWeYsjCV63j9gNNpfORMlmTd7YaojYKSqIrC2quuuy00wlrEig6NVZ3cR4uVHcGzayXEXVYNKsERDkB7scz/joa9sZc/2Y93o5sAD2p3zhTKjPrEzCUcdGWWRZIlRovQo+8wm1fhoBZE5A3FT9h+JJOFnHNzzSQkCbJszWi8Q4tRYBnuiiKEZdkaLyx5zbL3Ji6AsW2jA3RtWTP4ND39QlZqscepBMHiLH+g3+YlyCboykTOPj2z 9v9eMd3o mTDo26g1z6Zq1dxJQADkdU1W60OSe2oa8+J5lnzCKJscUqL8oCmwS2lAFe6uBvLK9DjxBQ7qArNeX3a7sAQRqOwl7GtLxJyWfGSkKDbmkJH/Sd/oGi24vqpnR+csJjGomIxq9C7ZAhF/WtesOBPghlJFYroH+leyzlm9NqDrJ3iYKQbZIJ1uIs14TgRqPK59CCEZwVjR4tKOPF7B2PvJlNgiK+wXVzCXbi0/T02qQhDHaNiI= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Hugetlb pages will be handled in the generic path, so do not try to split pud or pmd leafs if they are not thp. Signed-off-by: Oscar Salvador --- mm/pagewalk.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/mm/pagewalk.c b/mm/pagewalk.c index a9c36f9e9820..78d45f1450aa 100644 --- a/mm/pagewalk.c +++ b/mm/pagewalk.c @@ -119,7 +119,8 @@ static int walk_pmd_range(pud_t *pud, unsigned long addr, unsigned long end, * Check this here so we only break down trans_huge * pages when we _need_ to */ - if ((!walk->vma && (pmd_leaf(*pmd) || !pmd_present(*pmd))) || + if (((!walk->vma || is_vm_hugetlb_page(walk->vma)) && + (pmd_leaf(*pmd) || !pmd_present(*pmd))) || walk->action == ACTION_CONTINUE || !(ops->pte_entry)) continue; @@ -169,7 +170,8 @@ static int walk_pud_range(p4d_t *p4d, unsigned long addr, unsigned long end, if (walk->action == ACTION_AGAIN) goto again; - if ((!walk->vma && (pud_leaf(*pud) || !pud_present(*pud))) || + if (((!walk->vma || is_vm_hugetlb_page(walk->vma)) && + (pud_leaf(*pud) || !pud_present(*pud))) || walk->action == ACTION_CONTINUE || !(ops->pmd_entry || ops->pte_entry)) continue; From patchwork Thu Jul 4 04:30:54 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oscar Salvador X-Patchwork-Id: 13723185 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 50084C30653 for ; Thu, 4 Jul 2024 04:32:11 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 1A32F6B009A; Thu, 4 Jul 2024 00:32:02 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 12AEF6B009B; Thu, 4 Jul 2024 00:32:02 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id EC0056B009C; Thu, 4 Jul 2024 00:32:01 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id C454A6B009A for ; Thu, 4 Jul 2024 00:32:01 -0400 (EDT) Received: from smtpin25.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 6FA94C0ACF for ; Thu, 4 Jul 2024 04:32:01 +0000 (UTC) X-FDA: 82300797642.25.C8C13AD Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.223.130]) by imf17.hostedemail.com (Postfix) with ESMTP id 709FE40011 for ; Thu, 4 Jul 2024 04:31:59 +0000 (UTC) Authentication-Results: imf17.hostedemail.com; dkim=none; spf=pass (imf17.hostedemail.com: domain of osalvador@suse.de designates 195.135.223.130 as permitted sender) smtp.mailfrom=osalvador@suse.de; dmarc=pass (policy=none) header.from=suse.de ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1720067495; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=YieWm4jzRwGnc/AoSi0Kzmy4dIO/mgBRJsIk5x6bE30=; b=LjoCuj/zTlgGc0DgofVHDT94UiymvCrYiFMdbmT2GarYK2sabtpcosoX9yE81GaD4vvRAZ wWeIyPcfrP6l6YDyJNgGXNmiOXm7VySNMFryXqgsem+Yac5AsoUcqPOEHhRiJrfKLnXury j6fxUsltJKDc3QpB+bo+pmhBlvxvqmY= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1720067495; a=rsa-sha256; cv=none; b=r+3Zft3thp1iXXJBjweBsR2mGDyQWcUoCk8D5OrDkVty4WqHXPVcgSdZT4aB2cHhtanPic EU/H4eDueo915Vzeij5JJvOtigpsi1EY4h870SBsNEFuy50rXpwIFfZUaGzNcoYs/iun+g aoEqPf68oGyjN1Ntuy9RCDUsl9Jb/qI= ARC-Authentication-Results: i=1; imf17.hostedemail.com; dkim=none; spf=pass (imf17.hostedemail.com: domain of osalvador@suse.de designates 195.135.223.130 as permitted sender) smtp.mailfrom=osalvador@suse.de; dmarc=pass (policy=none) header.from=suse.de Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [IPv6:2a07:de40:b281:104:10:150:64:97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id 3F26421C12; Thu, 4 Jul 2024 04:31:58 +0000 (UTC) Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 270FD13889; Thu, 4 Jul 2024 04:31:57 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id 0IRfBb0lhmbnDAAAD6G6ig (envelope-from ); Thu, 04 Jul 2024 04:31:57 +0000 From: Oscar Salvador To: Andrew Morton Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Peter Xu , Muchun Song , David Hildenbrand , SeongJae Park , Miaohe Lin , Michal Hocko , Matthew Wilcox , Christophe Leroy , Oscar Salvador Subject: [PATCH 07/45] arch/s390: Enable __s390_enable_skey_pmd to handle hugetlb vmas Date: Thu, 4 Jul 2024 06:30:54 +0200 Message-ID: <20240704043132.28501-8-osalvador@suse.de> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240704043132.28501-1-osalvador@suse.de> References: <20240704043132.28501-1-osalvador@suse.de> MIME-Version: 1.0 X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Rspamd-Action: no action X-Rspam-User: X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 709FE40011 X-Stat-Signature: 9kkb5t7tco696gogww6ggqrywdfmp535 X-HE-Tag: 1720067519-726938 X-HE-Meta: U2FsdGVkX19/O/ukdlEPi3rIku9wEqyebtVlMu4EYIklKglNBBnc///gZP76XfzmKmMWHWx+gXhjbTa8jRNdk6JjtPkEd+2djPkU8gAdtrryLh+OucOi/H9BV+5Pgr1iyCG6tWyHvRerTnBo9rLsbP+c1hWYscWnxsCqqgco5ZixVtncpZvmM+M2YfFphxWsvHz7Oy61ud/MERc7TMbK5YL7ubg45uf1vmwDsO5jE3A+t2mYrB/M8ubsa7O3eTwmBRUq5VyI7wcSqkbNTLMrnzC0DHvtLr1N7K5BM16MJRhWCZPoTF9yDfg423E355kLEGZT5CmTrY3DMCjBkll50V+XkfcmjhwAqkphJfVL0aE7fA0tsRNuLiSrNyCdc7DjYtEbtD/3ehDvxIXB8HHMDRmyVLT4JHp7d9y2HkkwU8DZ82QwT0Yi4MT1TcrVcvyHGdKTLpzfAZ/t8Y9bEpDeuJCHnlxFEa0lrv/Qp6uv2LQvF2pJkOYA0Zk6FwpFKjpa6wRqOLhtfqQF5JumdBjvDWTtnEg3vzFCQqkvkBxYBkUmUExShmJveVPAsVNpc3ZkPRmozxtRhmXxUjWeDCbAlTUIi/pMXgEyae9MmqGR0zNhaIQQqfxJB7vQr72NccBRDDwNHQcnGgXSGFoCBUbWX5sueBwqphJKJYLz4UcS9qw6t83RSNpx7nqNCVYyWwkYKjtHhkjfzMQZjerg8uG9aJBynfDUZr8n657tF3GMAtg8X/dm6gjDJtdpp/S/cFcRF6JaaEAXZRyZMqxoYi4SWei+8pR1DUaDk/eEMw8yMEKeeQsYp8vohWDeWlLVk4cP50qNgAo8ImkG+adjYiiMDot2l70OUp0OcWgMwnTD8mNphqg5YwZwnEnpjzYi6X24z/x0OhONCUJRDY5OXWo2vGHSHBdbxqYwMdIXWipjzKMQe3i1nbJGo8nWjn8bHSgxjgWlIYkKMVRIpBaZw5K zW3SEXsm xbzXN7jZ2oZE4/FiBWsFfOjp8dpsGxvACgeZp5ORq89/K2Py0dDY8NI2ydGeVZNGk5KLMocArcG+hR/ieyLu6A0dwBFxRPJXUgHp0W4Y/ASa4PPU8Yf+7PPWfcoFB4NqOdN+Bqu9aJZLJ6HIN6uDMRwdqThZ8z6cxbTg/fjw9laQBK6Y= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: __s390_enable_skey_pmd does nothing for THP, but will handle pmd-leafs hugetlb vmas from now onwards, so teach it how to handle those. Signed-off-by: Oscar Salvador --- arch/s390/mm/gmap.c | 14 ++++++++++++++ 1 file changed, 14 insertions(+) diff --git a/arch/s390/mm/gmap.c b/arch/s390/mm/gmap.c index 474a25ca5c48..e1d098dc7f07 100644 --- a/arch/s390/mm/gmap.c +++ b/arch/s390/mm/gmap.c @@ -2723,6 +2723,20 @@ static int __s390_enable_skey_pte(pte_t *pte, unsigned long addr, static int __s390_enable_skey_pmd(pmd_t *pmd, unsigned long addr, unsigned long next, struct mm_walk *walk) { + if (pmd_leaf(*pmd) && is_vm_hugetlb_page(vma)) + unsigned long start, end; + struct page *page = pmd_page(*pmd); + + if (pmd_val(*pmd) & _SEGMENT_ENTRY_INVALID || + !(pmd_val(*pmd) & _SEGMENT_ENTRY_WRITE)) + return 0; + + start = pmd_val(*pmd) & HPAGE_MASK; + end = start + HPAGE_SIZE; + __storage_key_init_range(start, end); + set_bit(PG_arch_1, &page->flags); + } + cond_resched(); return 0; } From patchwork Thu Jul 4 04:30:55 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oscar Salvador X-Patchwork-Id: 13723186 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 18E8AC30653 for ; Thu, 4 Jul 2024 04:32:14 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 689486B009B; Thu, 4 Jul 2024 00:32:03 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 574266B009C; Thu, 4 Jul 2024 00:32:03 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 3C38B6B009D; Thu, 4 Jul 2024 00:32:03 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 15E4F6B009B for ; Thu, 4 Jul 2024 00:32:03 -0400 (EDT) Received: from smtpin09.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id C8CFCA0835 for ; Thu, 4 Jul 2024 04:32:02 +0000 (UTC) X-FDA: 82300797684.09.E90A489 Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.223.131]) by imf20.hostedemail.com (Postfix) with ESMTP id B6D841C0013 for ; Thu, 4 Jul 2024 04:32:00 +0000 (UTC) Authentication-Results: imf20.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=suse.de; spf=pass (imf20.hostedemail.com: domain of osalvador@suse.de designates 195.135.223.131 as permitted sender) smtp.mailfrom=osalvador@suse.de ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1720067501; a=rsa-sha256; cv=none; b=6c9tHDrLDVlxMS7ZzbqfgMzvONyRzosBpelOzzQFbGD4beutU9o3YPG5AB1XAaOt7OpRpM fu70nJoWcg3d1+Ohf4O2KI6GHLL5Jid0Y4K7YP1rHkCPB6TlK3MGemLnnOx3/hFBBMlk/l 4e1DrygaSHr6lECNDq33XcZ9WB5f+1Y= ARC-Authentication-Results: i=1; imf20.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=suse.de; spf=pass (imf20.hostedemail.com: domain of osalvador@suse.de designates 195.135.223.131 as permitted sender) smtp.mailfrom=osalvador@suse.de ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1720067501; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=3qKedk2fwycHSnQhhuoHW6DxwcyT9yK5ZC/WZBJG70E=; b=isI7webdiuwjJJaVR4117aEIG2WZvBOYKXR/P+wl+yrXDfBmuRhw6znMAaCQLxUIejGnyc 55Wds5JJvkCeAaZukZDpDbpUhLuZ3fmnR3aEiL4PtOvs+7FEfQ6ED59lkmewQK260VwLtR IowhV/f8Unx5F7+JX6OF/K843uZc9LE= Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [IPv6:2a07:de40:b281:104:10:150:64:97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id 824D71FCF7; Thu, 4 Jul 2024 04:31:59 +0000 (UTC) Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 5F5B113889; Thu, 4 Jul 2024 04:31:58 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id AKLwEr4lhmbnDAAAD6G6ig (envelope-from ); Thu, 04 Jul 2024 04:31:58 +0000 From: Oscar Salvador To: Andrew Morton Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Peter Xu , Muchun Song , David Hildenbrand , SeongJae Park , Miaohe Lin , Michal Hocko , Matthew Wilcox , Christophe Leroy , Oscar Salvador Subject: [PATCH 08/45] fs/proc: Enable smaps_pmd_entry to handle PMD-mapped hugetlb vmas Date: Thu, 4 Jul 2024 06:30:55 +0200 Message-ID: <20240704043132.28501-9-osalvador@suse.de> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240704043132.28501-1-osalvador@suse.de> References: <20240704043132.28501-1-osalvador@suse.de> MIME-Version: 1.0 X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Rspamd-Action: no action X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: B6D841C0013 X-Stat-Signature: 3ebizy9rjyjn1hm7oy6sh4o5asw6bdrz X-Rspam-User: X-HE-Tag: 1720067520-869317 X-HE-Meta: U2FsdGVkX1+/IgMzZcOgHHDL6W5VfPrNvxEvte7ACZeXZDXaPw5VYQScDVYpyse39hun+Jn0HrUJjyBxGQ1Tzpj1T1PZgqojAgYKf2PPCUeLyO1iM5cYhbo1/+kpCRGZGs8PsOoc3Aolt75WGle/lUA49w9PZtolbGUAVvIge2iCCmTqpraCR94CICwU1P/6VuzyIK5A3LdmaN3KLZCf3LhRRt/HFRDCr9Dgyu+FaBGyQl5za+kotlenErlthTZy1jufqwLAVTJt1zYhYpDloeYyNvGHNuwvZmR1Oho9ke2J29imzeid5lIJBjB+G8F3qkCIzZPEK03QSJ9bMcC+TA5xkJF0nnAUZpXbh8cfIEKdlqJHalJqCQkR8beLdssX+KcXneejDTXEIlTVuzNtJXDz/C7EQc1wJ1FCQgtm63nYJGF3X8cE6TnJgm4q1ff4hOuSFgJbtmEFI01/cwwLdmgTx6I1/FW2QWjrcQfWGJbF60tJOJYPDHol1Ey/L054TuyITQpbZqb9Z+nz5iPyy620DyNrX/cBV7bwx7haQWP7HMrAn8UY414LR01buYNm4OuKcRgDGMAjcvdSWWBHgwgNnUBFls582CuLprpB4EJJgumMzodzIu3P6wS16LoUmrrDtY6jyNQ6QG0d2u2en3cBHPV4tuiWxtV8NgCmmxBt+5X4+l7fMPU50gVzHtVVEGKyW31q+UCPCplV4qv6UNFNOI1+vLQI4Db41woZpi/NN9WkZo/KiPSUyWBUxBP2wO2WIS3+NWQO3wJFpT9cJRTXFzkC33k1yygVR1qFavcs+H/of7SW+bSEkdPfPEcWSKfzDaKWn7nfiI/pWv7ETkbz2tKrMdhCHRrL3Haqrn042Pbi2eKfxhF92WZy5lVYQiI2VYiq2kq/ss3dQL3jZsW07Wmg08QfXrqLBugkV6akRJr0xjyqWtnPCvTs3vBEtbXRgQsuVgExNyRaSKY /yDDuEAj hX24o0ol2eWyzwkBkh0SD8AN+5bUlNnW+ywh7m7r+9leodZor/WcqZmlfC8dAYubBx6XxFQMsm4G87uMQMIYdX4ZVD1W6IJQaCzgSBRKE9+O2XVwZ0TSJm9E5KAImTAUgQyWzeUR2IPD1FluCTKSWpemDb6ZdcejCjmrMUVlM2Kf8TPQ= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: PMD-mapped hugetlb vmas will also reach smaps_pmd_entry. Add the required code so it knows how to handle those there. Signed-off-by: Oscar Salvador --- fs/proc/task_mmu.c | 25 ++++++++++++++++++++++--- 1 file changed, 22 insertions(+), 3 deletions(-) diff --git a/fs/proc/task_mmu.c b/fs/proc/task_mmu.c index 775a2e8d600c..78d84d7e2353 100644 --- a/fs/proc/task_mmu.c +++ b/fs/proc/task_mmu.c @@ -804,6 +804,18 @@ static void smaps_pte_hole_lookup(unsigned long addr, struct mm_walk *walk) #endif } +#ifdef CONFIG_HUGETLB_PAGE +static void mss_hugetlb_update(struct mem_size_stats *mss, struct folio *folio, + struct vm_area_struct *vma, pte_t *pte) +{ + if (folio_likely_mapped_shared(folio) || + hugetlb_pmd_shared(pte)) + mss->shared_hugetlb += huge_page_size(hstate_vma(vma)); + else + mss->private_hugetlb += huge_page_size(hstate_vma(vma)); +} +#endif + static void smaps_pte_entry(pte_t *pte, unsigned long addr, struct mm_walk *walk) { @@ -851,12 +863,13 @@ static void smaps_pte_entry(pte_t *pte, unsigned long addr, smaps_account(mss, page, false, young, dirty, locked, present); } -#ifdef CONFIG_TRANSPARENT_HUGEPAGE +#ifdef CONFIG_PGTABLE_HAS_HUGE_LEAVES static void smaps_pmd_entry(pmd_t *pmd, unsigned long addr, struct mm_walk *walk) { struct mem_size_stats *mss = walk->private; struct vm_area_struct *vma = walk->vma; + bool hugetlb = is_vm_hugetlb_page(vma); bool locked = !!(vma->vm_flags & VM_LOCKED); struct page *page = NULL; bool present = false; @@ -865,7 +878,7 @@ static void smaps_pmd_entry(pmd_t *pmd, unsigned long addr, if (pmd_present(*pmd)) { page = vm_normal_page_pmd(vma, addr, *pmd); present = true; - } else if (unlikely(thp_migration_supported() && is_swap_pmd(*pmd))) { + } else if (is_swap_pmd(*pmd)) { swp_entry_t entry = pmd_to_swp_entry(*pmd); if (is_pfn_swap_entry(entry)) @@ -874,6 +887,12 @@ static void smaps_pmd_entry(pmd_t *pmd, unsigned long addr, if (IS_ERR_OR_NULL(page)) return; folio = page_folio(page); + + if (hugetlb) { + mss_hugetlb_update(mss, folio, vma, (pte_t *)pmd); + return; + } + if (folio_test_anon(folio)) mss->anonymous_thp += HPAGE_PMD_SIZE; else if (folio_test_swapbacked(folio)) @@ -900,7 +919,7 @@ static int smaps_pte_range(pmd_t *pmd, unsigned long addr, unsigned long end, pte_t *pte; spinlock_t *ptl; - ptl = pmd_trans_huge_lock(pmd, vma); + ptl = pmd_huge_lock(pmd, vma); if (ptl) { smaps_pmd_entry(pmd, addr, walk); spin_unlock(ptl); From patchwork Thu Jul 4 04:30:56 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oscar Salvador X-Patchwork-Id: 13723187 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id E4969C30653 for ; Thu, 4 Jul 2024 04:32:16 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 1570C6B009C; Thu, 4 Jul 2024 00:32:05 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 0DF096B009D; Thu, 4 Jul 2024 00:32:05 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id DFDE86B009E; Thu, 4 Jul 2024 00:32:04 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id B597C6B009C for ; Thu, 4 Jul 2024 00:32:04 -0400 (EDT) Received: from smtpin20.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 485ACA0FBD for ; Thu, 4 Jul 2024 04:32:04 +0000 (UTC) X-FDA: 82300797768.20.BECD8B8 Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.223.131]) by imf26.hostedemail.com (Postfix) with ESMTP id 288CF140010 for ; Thu, 4 Jul 2024 04:32:01 +0000 (UTC) Authentication-Results: imf26.hostedemail.com; dkim=pass header.d=suse.de header.s=susede2_rsa header.b=mvxA8c0w; dkim=pass header.d=suse.de header.s=susede2_ed25519 header.b=gtiiSyL8; dkim=pass header.d=suse.de header.s=susede2_rsa header.b=mvxA8c0w; dkim=pass header.d=suse.de header.s=susede2_ed25519 header.b=gtiiSyL8; spf=pass (imf26.hostedemail.com: domain of osalvador@suse.de designates 195.135.223.131 as permitted sender) smtp.mailfrom=osalvador@suse.de; dmarc=pass (policy=none) header.from=suse.de ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1720067498; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=759iJlxKnytdAlee8rRqvYsRUqK1GzIE0jiP12kfaog=; b=cl7GkMa0IimPtFB1v4U6nO5GSN+R0dn39tTHfxKr/S+9e3paPcTbHscYfp7tJatjBZYR+D gJ2G4Tz62eVXb/vHkuwacXvN+asX24/gvpkl538veGk5KUXlob/OPbAX6wRbQ+1ZK4DMAz zzV+3gGBXxdaK1/ZwUiSe7shkQvLsHs= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1720067498; a=rsa-sha256; cv=none; b=GIhc3Sau63rocnlDFUtN36CVN67QR1tUk4DveN24hiJ7UZRygGJcvNI7ub52l4EGkf95Gi tUux2OYtWTJ2dRRbvFSNtj1BHRR6WKmbsYAJAm5smcXoQ2Dhve/TpP0MpNWC0BwhggxTu7 wBGUk30JW51vpN4hoqqgosCThoQGAIg= ARC-Authentication-Results: i=1; imf26.hostedemail.com; dkim=pass header.d=suse.de header.s=susede2_rsa header.b=mvxA8c0w; dkim=pass header.d=suse.de header.s=susede2_ed25519 header.b=gtiiSyL8; dkim=pass header.d=suse.de header.s=susede2_rsa header.b=mvxA8c0w; dkim=pass header.d=suse.de header.s=susede2_ed25519 header.b=gtiiSyL8; spf=pass (imf26.hostedemail.com: domain of osalvador@suse.de designates 195.135.223.131 as permitted sender) smtp.mailfrom=osalvador@suse.de; dmarc=pass (policy=none) header.from=suse.de Received: from imap1.dmz-prg2.suse.org (unknown [10.150.64.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id D4A8C1FCF8; Thu, 4 Jul 2024 04:32:00 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1720067520; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=759iJlxKnytdAlee8rRqvYsRUqK1GzIE0jiP12kfaog=; b=mvxA8c0wNNkkavEyFQzOp2mf8378ze4FIH4m4ZfgOZRJ7zpzcoSPW950dQZsa3S/CK+ttv iZlMjg840j6JYRueSYvhI7YBduf5ZmQglN6zSGHvYGjJBa7Jy2q7URNYkz9x4o2JnliPCw zchyR+/MljAZfrr+nxrpTvg2ZDweQk4= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1720067520; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=759iJlxKnytdAlee8rRqvYsRUqK1GzIE0jiP12kfaog=; b=gtiiSyL8IoxKC27iK+u4Mo4MByRQvcwll4qDeKH5vMoKVQYazRAVgxQtYJlScwNErbS0UE MtZSHFG7y/+A7DCQ== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1720067520; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=759iJlxKnytdAlee8rRqvYsRUqK1GzIE0jiP12kfaog=; b=mvxA8c0wNNkkavEyFQzOp2mf8378ze4FIH4m4ZfgOZRJ7zpzcoSPW950dQZsa3S/CK+ttv iZlMjg840j6JYRueSYvhI7YBduf5ZmQglN6zSGHvYGjJBa7Jy2q7URNYkz9x4o2JnliPCw zchyR+/MljAZfrr+nxrpTvg2ZDweQk4= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1720067520; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=759iJlxKnytdAlee8rRqvYsRUqK1GzIE0jiP12kfaog=; b=gtiiSyL8IoxKC27iK+u4Mo4MByRQvcwll4qDeKH5vMoKVQYazRAVgxQtYJlScwNErbS0UE MtZSHFG7y/+A7DCQ== Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id A5A4213889; Thu, 4 Jul 2024 04:31:59 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id uIG7I78lhmbnDAAAD6G6ig (envelope-from ); Thu, 04 Jul 2024 04:31:59 +0000 From: Oscar Salvador To: Andrew Morton Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Peter Xu , Muchun Song , David Hildenbrand , SeongJae Park , Miaohe Lin , Michal Hocko , Matthew Wilcox , Christophe Leroy , Oscar Salvador Subject: [PATCH 09/45] mm: Implement pud-version functions for swap and vm_normal_page_pud Date: Thu, 4 Jul 2024 06:30:56 +0200 Message-ID: <20240704043132.28501-10-osalvador@suse.de> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240704043132.28501-1-osalvador@suse.de> References: <20240704043132.28501-1-osalvador@suse.de> MIME-Version: 1.0 X-Rspamd-Queue-Id: 288CF140010 X-Stat-Signature: diwuh6hz1711hdktud5q1kn77gp1fays X-Rspamd-Server: rspam09 X-Rspam-User: X-HE-Tag: 1720067521-97985 X-HE-Meta: U2FsdGVkX18UFur47wmtlqsUg3HtXm0B32XTp1QRL9lJmUavIFJsO/XeDikfx94pDpf5T1MTphsPen86j/YOec1Kohc/HI50uiCAyICUtPq9/JbEkmwEQCR72+grczzAm6bu/J2G0u1CbVrjb7lCm7RA0M2SLXZANDO33b5LNSdopRnX4Fkp58JCBPDF/kXX28jE/mCpNsGtSztBysEMitos6mnKWgkXLcp0s1Wb7pIPC9PUl4KI+xLudZBzmZV4m4PLrA3TQulhrCaHfURRn4HEB9AfaK9hLTIFQEqKwupFcjJzK49XUdC5e1/JFUyJ4ELuAPW/6kcEAdWX6MRkyzpv/QsTeLLAlyQD7A4CCVc8lcZet7rbDbZeMHAItASiiXjvtp4gi574GXayEmaQTMzLWem6pu99kMBnRWwvhYptPEUnJnIkCmSOF9sh8KdgHfpCZv3ehCcZ2o8ssySnVI+VWo3MydcLWaWH5BHwEdsGDAFVMMOC91vIo8tPw3IxNbAH/5SGnOHFQPSQ5Z/5YWgyQuRkAtqBF2B0UM/r08sYAGEFMc+yjyO9Cd+pDfcs/sc+o6GgFQG09t/No3hH1fFV6TeKVHfB5I1Tq0V9ht2PFLUULAFmiAtnK7M91HrBJi9J3gvsF7rUtouF5rmAvj6qfXb75IMauZSWXJ/JIxUKQbU14YQJNy9BQfex8J4KyhHmxL0rOnHjY4zBIVumZxEk245ic47UC1t5rhw6St8VKLiutpTU0s4gRtaGxs5pglMhg0XDXUTuP2MUHiGVSii7NO+JbcRQjFDhRShyZZGQ6i+D074ss4GH4LU4JNisIHPKcnXte6pxui5IOVs8czF1VWCTT0NRoGv4fTanD99P3QKdYqGP1WNJfnTU+2GwfR5uV+I9zdMAZKjICkmN6fLG//xie66E+yV8WsXWnDtOPKMQITcgNkhXyIaztBC6toOMgIITJQBvkRjIof1 xGA0jDJX R5A78YY1g6xMKtHLmgkRfJ0dvmhGz2jFXJicok3xubJlf5jVzpe5C8rAPSZ5D78JoAo8mLCviNX+ScrWZF/uZ3Nf+1p6PVR4R8BJnN6wlGodPxjD/2FFutnLgSED0ufjXV7LrUHTbD2XxDhMr+kVAzjVrqTFuwMBLf3rGQrOjj8hC28AkoaDGo0IxijHYn+jEg4SUOfrC62ajL++WDlQBohhxXbDJyLphIGs+BB++LqQ7CRIBDD8Y/Oxa+a60osCwwoOZ8FxMPSS8yAyXRRPwaD/WXumvHFR8Ic2a X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: HugeTLB pages will be handled on pud level as well, so we need to implement pud-version of vm_normal_page_pud and swp-pud functions. Signed-off-by: Oscar Salvador --- arch/powerpc/include/asm/book3s/64/pgtable.h | 1 + include/linux/mm.h | 4 ++ include/linux/pgtable.h | 6 +++ include/linux/swapops.h | 15 ++++++ mm/memory.c | 51 ++++++++++++++++++++ 5 files changed, 77 insertions(+) diff --git a/arch/powerpc/include/asm/book3s/64/pgtable.h b/arch/powerpc/include/asm/book3s/64/pgtable.h index 519b1743a0f4..fa4bb8d6356f 100644 --- a/arch/powerpc/include/asm/book3s/64/pgtable.h +++ b/arch/powerpc/include/asm/book3s/64/pgtable.h @@ -687,6 +687,7 @@ static inline pte_t pte_modify(pte_t pte, pgprot_t newprot) #define __pte_to_swp_entry(pte) ((swp_entry_t) { pte_val((pte)) & ~_PAGE_PTE }) #define __swp_entry_to_pte(x) __pte((x).val | _PAGE_PTE) #define __pmd_to_swp_entry(pmd) (__pte_to_swp_entry(pmd_pte(pmd))) +#define __pud_to_swp_entry(pud) (__pte_to_swp_entry(pud_pte(pud))) #define __swp_entry_to_pmd(x) (pte_pmd(__swp_entry_to_pte(x))) #ifdef CONFIG_MEM_SOFT_DIRTY diff --git a/include/linux/mm.h b/include/linux/mm.h index 5f1075d19600..baade06b159b 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -2371,6 +2371,10 @@ struct folio *vm_normal_folio_pmd(struct vm_area_struct *vma, unsigned long addr, pmd_t pmd); struct page *vm_normal_page_pmd(struct vm_area_struct *vma, unsigned long addr, pmd_t pmd); +struct folio *vm_normal_folio_pud(struct vm_area_struct *vma, + unsigned long addr, pud_t pud); +struct page *vm_normal_page_pud(struct vm_area_struct *vma, unsigned long addr, + pud_t pud); void zap_vma_ptes(struct vm_area_struct *vma, unsigned long address, unsigned long size); diff --git a/include/linux/pgtable.h b/include/linux/pgtable.h index 3a7b8751747e..a9edeb86b7fe 100644 --- a/include/linux/pgtable.h +++ b/include/linux/pgtable.h @@ -1990,4 +1990,10 @@ pgprot_t vm_get_page_prot(unsigned long vm_flags) \ } \ EXPORT_SYMBOL(vm_get_page_prot); +#ifdef CONFIG_HUGETLB_PAGE +#ifndef __pud_to_swp_entry +#define __pud_to_swp_entry(pud) ((swp_entry_t) { pud_val(pud) }) +#endif +#endif + #endif /* _LINUX_PGTABLE_H */ diff --git a/include/linux/swapops.h b/include/linux/swapops.h index cb468e418ea1..182957f0d013 100644 --- a/include/linux/swapops.h +++ b/include/linux/swapops.h @@ -126,6 +126,21 @@ static inline int is_swap_pte(pte_t pte) return !pte_none(pte) && !pte_present(pte); } +#ifdef CONFIG_HUGETLB_PAGE +static inline int is_swap_pud(pud_t pud) +{ + return !pud_none(pud) && !pud_present(pud); +} + +static inline swp_entry_t pud_to_swp_entry(pud_t pud) +{ + swp_entry_t arch_entry; + + arch_entry = __pud_to_swp_entry(pud); + return swp_entry(__swp_type(arch_entry), __swp_offset(arch_entry)); +} +#endif + /* * Convert the arch-dependent pte representation of a swp_entry_t into an * arch-independent swp_entry_t. diff --git a/mm/memory.c b/mm/memory.c index 0a769f34bbb2..90c5dfac35c6 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -718,6 +718,57 @@ struct folio *vm_normal_folio_pmd(struct vm_area_struct *vma, } #endif +#ifdef CONFIG_HUGETLB_PAGE +struct page *vm_normal_page_pud(struct vm_area_struct *vma, unsigned long addr, + pud_t pud) +{ + unsigned long pfn = pud_pfn(pud); + + /* + * There is no pmd_special() but there may be special pmds, e.g. + * in a direct-access (dax) mapping, so let's just replicate the + * !CONFIG_ARCH_HAS_PTE_SPECIAL case from vm_normal_page() here. + */ + if (unlikely(vma->vm_flags & (VM_PFNMAP|VM_MIXEDMAP))) { + if (vma->vm_flags & VM_MIXEDMAP) { + if (!pfn_valid(pfn)) + return NULL; + goto out; + } else { + unsigned long off; + + off = (addr - vma->vm_start) >> PAGE_SHIFT; + if (pfn == vma->vm_pgoff + off) + return NULL; + if (!is_cow_mapping(vma->vm_flags)) + return NULL; + } + } + + if (pud_devmap(pud)) + return NULL; + if (unlikely(pfn > highest_memmap_pfn)) + return NULL; + + /* + * NOTE! We still have PageReserved() pages in the page tables. + * eg. VDSO mappings can cause them to exist. + */ +out: + return pfn_to_page(pfn); +} + +struct folio *vm_normal_folio_pud(struct vm_area_struct *vma, + unsigned long addr, pud_t pud) +{ + struct page *page = vm_normal_page_pud(vma, addr, pud); + + if (page) + return page_folio(page); + return NULL; +} +#endif + static void restore_exclusive_pte(struct vm_area_struct *vma, struct page *page, unsigned long address, pte_t *ptep) From patchwork Thu Jul 4 04:30:57 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oscar Salvador X-Patchwork-Id: 13723188 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id D5EF4C30653 for ; Thu, 4 Jul 2024 04:32:19 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 1087B6B009D; Thu, 4 Jul 2024 00:32:06 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id EEA0B6B009E; Thu, 4 Jul 2024 00:32:05 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D10C96B009F; Thu, 4 Jul 2024 00:32:05 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id A03766B009D for ; Thu, 4 Jul 2024 00:32:05 -0400 (EDT) Received: from smtpin25.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 6DEBF1C264C for ; Thu, 4 Jul 2024 04:32:05 +0000 (UTC) X-FDA: 82300797810.25.6FF070F Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.223.130]) by imf19.hostedemail.com (Postfix) with ESMTP id 611CA1A0003 for ; Thu, 4 Jul 2024 04:32:03 +0000 (UTC) Authentication-Results: imf19.hostedemail.com; dkim=none; spf=pass (imf19.hostedemail.com: domain of osalvador@suse.de designates 195.135.223.130 as permitted sender) smtp.mailfrom=osalvador@suse.de; dmarc=pass (policy=none) header.from=suse.de ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1720067511; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=uuCDOs4UnzbN/DaULDdat4m597hHyp5dumV+Rvovzyo=; b=JZuh1Dn5tJ4tsC015XS24cfSdNh1ahFd4wkdZYd2gx9HU0jzr/kwwWtg9Qa1tK64mVJZ1K 7XxATyClSW93UdA16E9kK1nOKkY4+3Cy+gIpkaR2FQJo46RLCbopd1fX07298owTyGO+/t Ow0fCF4zNsUEpETA9cPPi3zFbm75BNs= ARC-Authentication-Results: i=1; imf19.hostedemail.com; dkim=none; spf=pass (imf19.hostedemail.com: domain of osalvador@suse.de designates 195.135.223.130 as permitted sender) smtp.mailfrom=osalvador@suse.de; dmarc=pass (policy=none) header.from=suse.de ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1720067511; a=rsa-sha256; cv=none; b=o+cm//Z0jg4TsSMXDWKp0a8btclsyay82ghNs2P78aWVStJFHT6k3WSpc1hKBiSBsHyoRc rR1TEpt+tSs1fMuKqebyCE2JaTKLh5HEi2UuEeth/xpJc0Zmh+gJDCKCstgoROrhFzlsW+ 9zPmyjvYAQyfiEc3k0lqGhjeVVS5vCw= Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [IPv6:2a07:de40:b281:104:10:150:64:97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id 26D7421C0D; Thu, 4 Jul 2024 04:32:02 +0000 (UTC) Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 01D7013889; Thu, 4 Jul 2024 04:32:00 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id MCZPN8AlhmbnDAAAD6G6ig (envelope-from ); Thu, 04 Jul 2024 04:32:00 +0000 From: Oscar Salvador To: Andrew Morton Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Peter Xu , Muchun Song , David Hildenbrand , SeongJae Park , Miaohe Lin , Michal Hocko , Matthew Wilcox , Christophe Leroy , Oscar Salvador Subject: [PATCH 10/45] fs/proc: Create smaps_pud_range to handle PUD-mapped hugetlb vmas Date: Thu, 4 Jul 2024 06:30:57 +0200 Message-ID: <20240704043132.28501-11-osalvador@suse.de> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240704043132.28501-1-osalvador@suse.de> References: <20240704043132.28501-1-osalvador@suse.de> MIME-Version: 1.0 X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Rspamd-Action: no action X-Rspamd-Server: rspam03 X-Rspam-User: X-Rspamd-Queue-Id: 611CA1A0003 X-Stat-Signature: pnupb695p6sm56opxk1u8dhdtm3ctfsx X-HE-Tag: 1720067523-249987 X-HE-Meta: U2FsdGVkX1+qp3yJWdWJiAmSutwzS/N2X5jNSAe+UZZx9BAYqShBXH4BETVx+KVKResbQ2kYI/sX30LBNfV7XG3plZV+mumK5SmbuhnSl4pYpv94xDvMbEm0YEm6KfHTA+OpIUSo7VR3J60Za8YFkH0vdvpgEQLNvU1OyIwYrfw8lXv1kRzwxB/6VCg0mbLktO/e+M5YVPkSvMfzLLi/aIb2ipcAxyfRiriQrnFJdjFskAqpr1MCsQpK+v/CrnPbgyiylfG7z1Ch/7I5a/l9WJCjedXbC9enoBfvxsHo6nEIHVINLENpL187jxiy10HaPG/c580VsGKneBMlcnvWCn7tNNQHsRtMAynxtCoap3u57aXlBmP+wRxX4gV+crKnJzy+RRDTQS3vyZAjTTHypMaXAAbywUCTXK8bPrCSMR3nfd0zNGM7Hz6aRmmAdMJcl8qF/x6/Bjzd5DhKy9wP8pwOkaydJsLxfIYJmmOUpu7VViHqnOOlge/1v/lO8pMkKbU1hOzu3vbdPqzpTNiWHdXUQ0dPH/2wJu670ktnyiZ6aFaxsKd41HRBju3hdllYs1HV1W/Uc3sq9DHWhUOdjQTrgSontrgUAAAzvzEYjQ4uENIoj9+AQxCGE5G+oLewIlN4Ix3vW3KqhNDJRyO9FneEhQbUcMzYNDTesT7YZRtE9ToKgz7+8rK6QtctCXm8f+j4zvwpJm/P9ZVWrE1y/HzYILVnhE1/1UnVMGnHWPMHR64Tcff+PkhvolmCitReFBrDZPvZHyHoPVUhk5s7khgU2xsOjgEQL7ekgnkGakfiWXG5xD+mCUVgzIP+ZnPRj+cYqII+9uwRN+FESGgVHFLA16mGUfq3v1bho9cYZOh+paeXitlmuOCPhshVBc56ci25yWHNGveQTCuT0eYFaIbhdOOcIJAnZeLArTIQ3F17T3+6xGM9LmbTipnnIlgoeTDuDcFUzugKiMjTka9 KHyLYMU5 96fLBTUXruv/650R27oVmwzmy9w7iChEPtOI49M08IHzf716YTnqPhaIDoHR+YfzTVns789x4/5LbvqfqjsmuOzGnqzXx4CtXzIHLR6cJ4pj80daA1ryF6oGYvrd7q0B4twPTljoWgRh6ff2YFid0rYYpFTjy8V9W/oqgHEXf/zxwOyc= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Normal THP cannot be PUD-mapped (besides devmap), but hugetlb can, so create smaps_pud_range in order to handle PUD-mapped hugetlb vmas. Signed-off-by: Oscar Salvador --- fs/proc/task_mmu.c | 36 ++++++++++++++++++++++++++++++++++++ 1 file changed, 36 insertions(+) diff --git a/fs/proc/task_mmu.c b/fs/proc/task_mmu.c index 78d84d7e2353..3f3460ff03b0 100644 --- a/fs/proc/task_mmu.c +++ b/fs/proc/task_mmu.c @@ -912,6 +912,40 @@ static void smaps_pmd_entry(pmd_t *pmd, unsigned long addr, } #endif +#ifdef CONFIG_HUGETLB_PAGE +static int smaps_pud_range(pud_t *pudp, unsigned long addr, unsigned long end, + struct mm_walk *walk) +{ + spinlock_t *ptl; + struct folio *folio; + struct vm_area_struct *vma = walk->vma; + struct mem_size_stats *mss = walk->private; + + ptl = pud_huge_lock(pudp, vma); + if (!ptl) + return 0; + + if (pud_present(*pudp)) { + folio = vm_normal_folio_pud(vma, addr, *pudp); + } else if (is_swap_pud(*pudp)) { + /* PUD-hugetlbs can have swap entries */ + swp_entry_t swpent = pud_to_swp_entry(*pudp); + + if (is_pfn_swap_entry(swpent)) + folio = pfn_swap_entry_folio(swpent); + } + + if (folio) + /* Only for now hugetlb pages can end up here */ + mss_hugetlb_update(mss, folio, vma, (pte_t *)pudp); + + spin_unlock(ptl); + return 0; +} +#else +#define smaps_pud_range NULL +#endif + static int smaps_pte_range(pmd_t *pmd, unsigned long addr, unsigned long end, struct mm_walk *walk) { @@ -1061,12 +1095,14 @@ static int smaps_hugetlb_range(pte_t *pte, unsigned long hmask, #endif /* HUGETLB_PAGE */ static const struct mm_walk_ops smaps_walk_ops = { + .pud_entry = smaps_pud_range, .pmd_entry = smaps_pte_range, .hugetlb_entry = smaps_hugetlb_range, .walk_lock = PGWALK_RDLOCK, }; static const struct mm_walk_ops smaps_shmem_walk_ops = { + .pud_entry = smaps_pud_range, .pmd_entry = smaps_pte_range, .hugetlb_entry = smaps_hugetlb_range, .pte_hole = smaps_pte_hole, From patchwork Thu Jul 4 04:30:58 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oscar Salvador X-Patchwork-Id: 13723189 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 40C1AC30653 for ; Thu, 4 Jul 2024 04:32:23 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 7476A6B009E; Thu, 4 Jul 2024 00:32:07 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 6A9426B009F; Thu, 4 Jul 2024 00:32:07 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 4FB3A6B00A0; Thu, 4 Jul 2024 00:32:07 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 271EF6B009E for ; Thu, 4 Jul 2024 00:32:07 -0400 (EDT) Received: from smtpin09.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id E235A140A92 for ; Thu, 4 Jul 2024 04:32:06 +0000 (UTC) X-FDA: 82300797852.09.82CB443 Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.223.131]) by imf27.hostedemail.com (Postfix) with ESMTP id BEBCA40004 for ; Thu, 4 Jul 2024 04:32:04 +0000 (UTC) Authentication-Results: imf27.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=suse.de; spf=pass (imf27.hostedemail.com: domain of osalvador@suse.de designates 195.135.223.131 as permitted sender) smtp.mailfrom=osalvador@suse.de ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1720067506; a=rsa-sha256; cv=none; b=qSiYCjrQwuHbz/FYuAStcGz+cVo/pthgA1qXHo4gLJhYA9GfA9Zj/0b1d6FFrRChy40n78 Jb16kb3WZu72AXvIMGbKj7AwB0dBWEko0jsPYKkS4jO9qVvMd5OQw0/WI1K/q+8otPo0Ge y3ScCE17XBljt3x4tunWY8XMBkmCEzw= ARC-Authentication-Results: i=1; imf27.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=suse.de; spf=pass (imf27.hostedemail.com: domain of osalvador@suse.de designates 195.135.223.131 as permitted sender) smtp.mailfrom=osalvador@suse.de ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1720067506; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=F9LfBkIVdTUBVSN3j07vFwBTlJlTYHksIUd3RDOpre8=; b=bM3rzVzkZj4bGsG6MH91BXMeJKLMxM6jm6CfHnMwK7bRh3gg5ex0pUN7dU5L++3B5HVHvx s+icyzYOKy5aNdFAXSZthVBvtNBMZPz63wU7eiJ9xKwaXvriiz19D9mRTkN46v28WpY9NK 9sfzsReedgsowDnSH6xCFhrAglbApCg= Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [IPv6:2a07:de40:b281:104:10:150:64:97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id 770A51FCFD; Thu, 4 Jul 2024 04:32:03 +0000 (UTC) Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 4D9D013889; Thu, 4 Jul 2024 04:32:02 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id IPAUDcIlhmbnDAAAD6G6ig (envelope-from ); Thu, 04 Jul 2024 04:32:02 +0000 From: Oscar Salvador To: Andrew Morton Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Peter Xu , Muchun Song , David Hildenbrand , SeongJae Park , Miaohe Lin , Michal Hocko , Matthew Wilcox , Christophe Leroy , Oscar Salvador Subject: [PATCH 11/45] fs/proc: Enable smaps_pte_entry to handle cont-pte mapped hugetlb vmas Date: Thu, 4 Jul 2024 06:30:58 +0200 Message-ID: <20240704043132.28501-12-osalvador@suse.de> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240704043132.28501-1-osalvador@suse.de> References: <20240704043132.28501-1-osalvador@suse.de> MIME-Version: 1.0 X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Rspamd-Action: no action X-Rspamd-Queue-Id: BEBCA40004 X-Rspam-User: X-Rspamd-Server: rspam05 X-Stat-Signature: j7bj9wuzomahe5c5ddmy3dysnm7rumhz X-HE-Tag: 1720067524-538987 X-HE-Meta: U2FsdGVkX18XemCUbwJbGnHWavJ59ny4OjS+60jStNMln1rG6g0d9exl5jMIYymWRz/jbax/ruRetAC2ICNhLwQvH5yk0a5hWWSJEQs42IlKyKHwfGrwBBIQIyhF+LrNaGQ+JAkNfeDmKWMz124pXo67d6i4O4+5NgQqeizBR/mYt4pTSCnn0iy/EYwXDiQ6rn9hJeVfbTIuakV+NLxEMBM/bv8QpF85l1RNxV0ttb2KrWcDmvLgKyh4fdDTvIA/n0+4m56bC9A9pme6tev/uYuUXhU3y5V0eZzAErDVZhxc+/qSgE8tMjhIHozfytiGO7JwQJLoHCgQkYWcWAoA2z+GcJcYajcVgtF3WNtdc+GjvQ2+C5tiH69YQXC8Om5qICAFlNMoR7KpwV5HTWu6mXEgrOyNNKA579v9jmyXBrtSQO3COFHD34FvHI/ZZ10uGFMYwLGvbslo9kTG1t+3F02Fkg6LJV6IzIVd7DkWIveYST2Otf5RCYhqwpy5M3yg+eh+IfqBLZQMu60SZu6GSsAr6FIS2LVvpXjfvDwn4HM+szDkXhTuRfNMLThSGiQbRZW4YNvpsvCX7a+cVHItZGnBlZDhFD7OSW8FtExp8rLalSMsi582u5E2GWJ6ee+YNw5Ki4LpYFS8U2w/ec7AFuQu/GDIScel2IQ3sSLDVFER1H3GVFWCaEBJSzpz85A6oFAw2mT4TbDaGEBlArmAHnzuyePfB14xJAYoqnT32Il541k2I6bgWvFckEOV5BlunFdPctzmjrrljXNJqXu4lk6tPk2qc5CFEO05LL1RgnkGcZzb5LYvcw2w2y0ZFQHpJFIU8M3xgLkBovQGcKNxs92t5ENEQanGwVwm72EyULz6gxSgQzQVvkxD2QbXfoDRUbUoAGZBfe9zY83v+7IVHeW/HWcLcjhGIIfwgRzcRhNW0CjRUQDEP4HHIjnQj5aPkNAO8FZR3KTjJ25gl1E xYvl6mNn nVpZQWe8j2UVeGnjmGkXSPTeMIuPSM8X1vnsskARtClDK5g5zpNzwCNTCo0lu0UvhqSxRAV1kQhxy+1ugxxF2+r8YQ0QVUbbJE0zAi4pF+hNzWHJiPdblD4OVCs/s8lSjkfteEK42DYUci0x023I/ltw1TLz1bpBgAivJfavjiwYw/0M= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: HugeTLB pages can be cont-pte mapped, so teach smaps_pte_entry to handle them. Signed-off-by: Oscar Salvador --- fs/proc/task_mmu.c | 19 +++++++++++++------ include/linux/pgtable.h | 12 ++++++++++++ 2 files changed, 25 insertions(+), 6 deletions(-) diff --git a/fs/proc/task_mmu.c b/fs/proc/task_mmu.c index 3f3460ff03b0..4d94b6ce58dd 100644 --- a/fs/proc/task_mmu.c +++ b/fs/proc/task_mmu.c @@ -825,6 +825,7 @@ static void smaps_pte_entry(pte_t *pte, unsigned long addr, struct page *page = NULL; bool present = false, young = false, dirty = false; pte_t ptent = ptep_get(pte); + unsigned long size = pte_cont(ptent) ? PAGE_SIZE * CONT_PTES : PAGE_SIZE; if (pte_present(ptent)) { page = vm_normal_page(vma, addr, ptent); @@ -834,18 +835,18 @@ static void smaps_pte_entry(pte_t *pte, unsigned long addr, } else if (is_swap_pte(ptent)) { swp_entry_t swpent = pte_to_swp_entry(ptent); - if (!non_swap_entry(swpent)) { + if (!is_vm_hugetlb_page(vma) && !non_swap_entry(swpent)) { int mapcount; - mss->swap += PAGE_SIZE; + mss->swap += size; mapcount = swp_swapcount(swpent); if (mapcount >= 2) { - u64 pss_delta = (u64)PAGE_SIZE << PSS_SHIFT; + u64 pss_delta = (u64)size << PSS_SHIFT; do_div(pss_delta, mapcount); mss->swap_pss += pss_delta; } else { - mss->swap_pss += (u64)PAGE_SIZE << PSS_SHIFT; + mss->swap_pss += (u64)size << PSS_SHIFT; } } else if (is_pfn_swap_entry(swpent)) { if (is_device_private_entry(swpent)) @@ -860,7 +861,10 @@ static void smaps_pte_entry(pte_t *pte, unsigned long addr, if (!page) return; - smaps_account(mss, page, false, young, dirty, locked, present); + if (is_vm_hugetlb_page(vma)) + mss_hugetlb_update(mss, page_folio(page), vma, pte); + else + smaps_account(mss, page, false, young, dirty, locked, present); } #ifdef CONFIG_PGTABLE_HAS_HUGE_LEAVES @@ -952,6 +956,7 @@ static int smaps_pte_range(pmd_t *pmd, unsigned long addr, unsigned long end, struct vm_area_struct *vma = walk->vma; pte_t *pte; spinlock_t *ptl; + unsigned long size, cont_ptes; ptl = pmd_huge_lock(pmd, vma); if (ptl) { @@ -965,7 +970,9 @@ static int smaps_pte_range(pmd_t *pmd, unsigned long addr, unsigned long end, walk->action = ACTION_AGAIN; return 0; } - for (; addr != end; pte++, addr += PAGE_SIZE) + size = pte_cont(ptep_get(pte)) ? PAGE_SIZE * CONT_PTES : PAGE_SIZE; + cont_ptes = pte_cont(ptep_get(pte)) ? CONT_PTES : 1; + for (; addr != end; pte += cont_ptes, addr += size) smaps_pte_entry(pte, addr, walk); pte_unmap_unlock(pte - 1, ptl); out: diff --git a/include/linux/pgtable.h b/include/linux/pgtable.h index a9edeb86b7fe..991137dab87e 100644 --- a/include/linux/pgtable.h +++ b/include/linux/pgtable.h @@ -1926,6 +1926,18 @@ typedef unsigned int pgtbl_mod_mask; #define CONT_PMDS 0 #endif +#ifndef pte_cont +#define pte_cont(x) false +#endif + +#ifndef CONT_PTE_SIZE +#define CONT_PTE_SIZE 0 +#endif + +#ifndef CONT_PTES +#define CONT_PTES 0 +#endif + /* * We always define pmd_pfn for all archs as it's used in lots of generic * code. Now it happens too for pud_pfn (and can happen for larger From patchwork Thu Jul 4 04:30:59 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oscar Salvador X-Patchwork-Id: 13723190 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 51FFAC30653 for ; Thu, 4 Jul 2024 04:32:26 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id E4F636B009F; Thu, 4 Jul 2024 00:32:08 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id D61FE6B00A0; Thu, 4 Jul 2024 00:32:08 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B63BC6B00A1; Thu, 4 Jul 2024 00:32:08 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 8E09E6B009F for ; Thu, 4 Jul 2024 00:32:08 -0400 (EDT) Received: from smtpin19.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 568D81A105C for ; Thu, 4 Jul 2024 04:32:08 +0000 (UTC) X-FDA: 82300797936.19.4E48F6D Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.223.131]) by imf15.hostedemail.com (Postfix) with ESMTP id 242C9A000C for ; Thu, 4 Jul 2024 04:32:05 +0000 (UTC) Authentication-Results: imf15.hostedemail.com; dkim=pass header.d=suse.de header.s=susede2_rsa header.b=RIxxEKxG; dkim=pass header.d=suse.de header.s=susede2_ed25519 header.b=qELKKw+j; dkim=pass header.d=suse.de header.s=susede2_rsa header.b=RIxxEKxG; dkim=pass header.d=suse.de header.s=susede2_ed25519 header.b=qELKKw+j; spf=pass (imf15.hostedemail.com: domain of osalvador@suse.de designates 195.135.223.131 as permitted sender) smtp.mailfrom=osalvador@suse.de; dmarc=pass (policy=none) header.from=suse.de ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1720067501; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=TzRTJDegzctm0o/zod38K6uM5eCrhph5+Nc4qITcL0E=; b=ZzeIgmsVEdFpPNlHjF41LmO6FGurxnQ4NW1pfhdlsDWnQKotVYl+kGFQrKcvp1tOB7keJG T7uDcIauYz+aS/sffFygiiCTzRD6aAnAbYbMcMe4nhSFWa64Lp8m3nbBIf3uy4VmWzBF6f 9iUOMuIlEQ9KIc1DCWvFkuo0njfDCWQ= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1720067501; a=rsa-sha256; cv=none; b=pKIu+fYS45FVn5THTDlm0qFhlVx8vPfoZybZudf1ABdXVnPO1oezfEzHLsglrUGGSfJgxg +R+7+h8y43Ljj7jNvv7SkX0vqfebC/BtqKehfhmBp2/Ufuv09kcmZWgoc2XcRrPnlXQfbq 450USWlaK6AD3gbEdQylqX0I3WlyMXM= ARC-Authentication-Results: i=1; imf15.hostedemail.com; dkim=pass header.d=suse.de header.s=susede2_rsa header.b=RIxxEKxG; dkim=pass header.d=suse.de header.s=susede2_ed25519 header.b=qELKKw+j; dkim=pass header.d=suse.de header.s=susede2_rsa header.b=RIxxEKxG; dkim=pass header.d=suse.de header.s=susede2_ed25519 header.b=qELKKw+j; spf=pass (imf15.hostedemail.com: domain of osalvador@suse.de designates 195.135.223.131 as permitted sender) smtp.mailfrom=osalvador@suse.de; dmarc=pass (policy=none) header.from=suse.de Received: from imap1.dmz-prg2.suse.org (unknown [10.150.64.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id C84641FCFC; Thu, 4 Jul 2024 04:32:04 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1720067524; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=TzRTJDegzctm0o/zod38K6uM5eCrhph5+Nc4qITcL0E=; b=RIxxEKxG5cR+JdvOXGvScHx2quxv+jhuEkgOckg4pFfPpwOj9VOZl1vh5qJmR4/JuTNxQ4 D/UknVe0GOar65hwsjkxTWr72/C44FARudhSihdPho6fBNUUoiPDaqE3fz2zwhDes4iDuP OItI5Fg3f+Kf/XMNa7u2ylOqzbGXysA= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1720067524; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=TzRTJDegzctm0o/zod38K6uM5eCrhph5+Nc4qITcL0E=; b=qELKKw+j6f1vh0ZBQrKxzDcNKdWTqCngnE0ElYT6ZMBZDJuWGRwB7g2w0Ut5FIQYasv9A8 JRnzseqZKYp7iOAQ== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1720067524; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=TzRTJDegzctm0o/zod38K6uM5eCrhph5+Nc4qITcL0E=; b=RIxxEKxG5cR+JdvOXGvScHx2quxv+jhuEkgOckg4pFfPpwOj9VOZl1vh5qJmR4/JuTNxQ4 D/UknVe0GOar65hwsjkxTWr72/C44FARudhSihdPho6fBNUUoiPDaqE3fz2zwhDes4iDuP OItI5Fg3f+Kf/XMNa7u2ylOqzbGXysA= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1720067524; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=TzRTJDegzctm0o/zod38K6uM5eCrhph5+Nc4qITcL0E=; b=qELKKw+j6f1vh0ZBQrKxzDcNKdWTqCngnE0ElYT6ZMBZDJuWGRwB7g2w0Ut5FIQYasv9A8 JRnzseqZKYp7iOAQ== Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 9BDC913889; Thu, 4 Jul 2024 04:32:03 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id SDs9IcMlhmbnDAAAD6G6ig (envelope-from ); Thu, 04 Jul 2024 04:32:03 +0000 From: Oscar Salvador To: Andrew Morton Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Peter Xu , Muchun Song , David Hildenbrand , SeongJae Park , Miaohe Lin , Michal Hocko , Matthew Wilcox , Christophe Leroy , Oscar Salvador Subject: [PATCH 12/45] fs/proc: Enable pagemap_pmd_range to handle hugetlb vmas Date: Thu, 4 Jul 2024 06:30:59 +0200 Message-ID: <20240704043132.28501-13-osalvador@suse.de> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240704043132.28501-1-osalvador@suse.de> References: <20240704043132.28501-1-osalvador@suse.de> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 242C9A000C X-Stat-Signature: fa8o94xzb7ogayjmj87db369r3o6yoij X-HE-Tag: 1720067525-876472 X-HE-Meta: U2FsdGVkX1/YvsLSGhXlEt5lCLXpHFW9cWO/Wj0Az0s5hnfINO6V1qilo94VK0pLxuQIb3KRQwm+hkAxPQrMyvkF243QaAilUYBwOaO1oFIwCX20m+l4YA+ek9PhaxKT2EDk4VZs+Vu1gqRJJA72YH8EcvZyU3M5aE6yuedYRF0JY3jduWPdEpGU8ObHYaCCncXoyhR8E45yL9UlACJQ5W5z3MSSANQ54WyX6jfxz8ZpJ/Anw7Q54LVzLWkjQS2hItmJI88mYT/4eJUMwb5IW2BkSh60mt+CAVl+tIw3NJx8QdCZfXeYOeUjTkNN13w4wdwTAsHYRdo4ej8xN6CfW/jDo4ez6GRRNHk9OkFVfJTnxTExZeAJ8tUcdejbClzMKXdHyubvYYCRXIbme5CAh0QhC9NR8HVZns4n8wnJYgaLL8+XW+tmLTT4FjPqftmWmsnhCXoodFdr9t1Tin8kj70xO/FHYfunywVSzNjZQpVklnZlXeZv9Lt8zZkzoqjqrQvgucat4zWka2RMKYqypXq9qVTOFiXDNQiVfaLDKs4LidkBbolbcQhhGfuATFsSAUHd2v/1DVtqYKRHcKfzrq01Vn4Frckuj7qr4c8FhHI7z3Nu3xmg0+ote2gXzAY+T4gCDptPsWrhftldBbb1pX8OxVFTjFstSJuVgO11FqUtJfqXwL9/GG4nI34S3qat+kV0EVgUU4uI5yNm20lmHDZUfC7FczK18I7zP9Kjw8I0NL3oTMIDQZN0+JhHHyfS4CVMOEiKVnWV09C/KHK+OPilTdLdVyHYdusV/NOXvESWVr1nqa90XwNzb4TDP3VamHaSQKxR3ldNsBs9pM9Wmn8WAHei0dPatToJ83vyu2pZ3aExb1Do+21/M0Ucpbc8n7l/SZzK/1a0nFYdNlGLX2gwKhNt3h2oNsjhCNQOZcQ5m6HLzyUfIJ9NZpb9gX1oC3XJ40A9BpaSMr4EAY6 3WAQg4Y3 kpbK54t4LsNfIIS5fGLfrXbaPcC9d4i33wJCV6ZuaVyRjL7UqpbjOOJBgCQyKeGOELfNawEQMvUHycbD0PskP93oQ3BBQIjCFn5n+661b3fENyKIxnHuZliHaEI3yEhA/WYwDFyNSFaqA8NlyEtBzi0mhgBaDX4D/DCbP0brhEdpD5FTgFMGw+H0YwPAmg1Bhe9tiPonauMHC+owpG8EqImh4uYvJDiaC+058AxZn6KAhdvHxo9BoZf//tGOwc+ldZER9X8L1nd1yD1G7G+Iclt/u4g== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: PMD-mapped hugetlb vmas will also reach pagemap_pmd_range. Add the required code so it knows how to handle those there. Signed-off-by: Oscar Salvador --- fs/proc/task_mmu.c | 11 +++++------ 1 file changed, 5 insertions(+), 6 deletions(-) diff --git a/fs/proc/task_mmu.c b/fs/proc/task_mmu.c index 4d94b6ce58dd..ec429d82b921 100644 --- a/fs/proc/task_mmu.c +++ b/fs/proc/task_mmu.c @@ -1824,9 +1824,9 @@ static int pagemap_pmd_range(pmd_t *pmdp, unsigned long addr, unsigned long end, spinlock_t *ptl; pte_t *pte, *orig_pte; int err = 0; -#ifdef CONFIG_TRANSPARENT_HUGEPAGE +#ifdef CONFIG_PGTABLE_HAS_HUGE_LEAVES - ptl = pmd_trans_huge_lock(pmdp, vma); + ptl = pmd_huge_lock(pmdp, vma); if (ptl) { unsigned int idx = (addr & ~PMD_MASK) >> PAGE_SHIFT; u64 flags = 0, frame = 0; @@ -1848,7 +1848,6 @@ static int pagemap_pmd_range(pmd_t *pmdp, unsigned long addr, unsigned long end, if (pm->show_pfn) frame = pmd_pfn(pmd) + idx; } -#ifdef CONFIG_ARCH_ENABLE_THP_MIGRATION else if (is_swap_pmd(pmd)) { swp_entry_t entry = pmd_to_swp_entry(pmd); unsigned long offset; @@ -1861,7 +1860,8 @@ static int pagemap_pmd_range(pmd_t *pmdp, unsigned long addr, unsigned long end, frame = swp_type(entry) | (offset << MAX_SWAPFILES_SHIFT); } - flags |= PM_SWAP; + if (!is_vm_hugetlb_page(vma)) + flags |= PM_SWAP; if (pmd_swp_soft_dirty(pmd)) flags |= PM_SOFT_DIRTY; if (pmd_swp_uffd_wp(pmd)) @@ -1869,7 +1869,6 @@ static int pagemap_pmd_range(pmd_t *pmdp, unsigned long addr, unsigned long end, VM_BUG_ON(!is_pmd_migration_entry(pmd)); page = pfn_swap_entry_to_page(entry); } -#endif if (page) { folio = page_folio(page); @@ -1899,7 +1898,7 @@ static int pagemap_pmd_range(pmd_t *pmdp, unsigned long addr, unsigned long end, spin_unlock(ptl); return err; } -#endif /* CONFIG_TRANSPARENT_HUGEPAGE */ +#endif /* CONFIG_PGTABLE_HAS_HUGE_LEAVES */ /* * We can assume that @vma always points to a valid one and @end never From patchwork Thu Jul 4 04:31:00 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oscar Salvador X-Patchwork-Id: 13723191 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6F318C3271F for ; Thu, 4 Jul 2024 04:32:29 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id EA4856B00A0; Thu, 4 Jul 2024 00:32:09 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id E22166B00A2; Thu, 4 Jul 2024 00:32:09 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C275A6B00A3; Thu, 4 Jul 2024 00:32:09 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id A18FE6B00A0 for ; Thu, 4 Jul 2024 00:32:09 -0400 (EDT) Received: from smtpin21.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 5ABC0140AE0 for ; Thu, 4 Jul 2024 04:32:09 +0000 (UTC) X-FDA: 82300797978.21.FFB02E1 Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.223.131]) by imf19.hostedemail.com (Postfix) with ESMTP id 51ACD1A000A for ; Thu, 4 Jul 2024 04:32:07 +0000 (UTC) Authentication-Results: imf19.hostedemail.com; dkim=none; spf=pass (imf19.hostedemail.com: domain of osalvador@suse.de designates 195.135.223.131 as permitted sender) smtp.mailfrom=osalvador@suse.de; dmarc=pass (policy=none) header.from=suse.de ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1720067515; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=jzvOqBRe34To4ICfaNP1mlZ4ayS8DrUPt1kh2NpZYUg=; b=gvPbhfttdhjaOAxb8qWQrvClTCS+GJBabjv2GEZxpvWPpi/xU5GVEbJMxvHz3Q7zzW0Yw/ f21PSzJAXx7ede4lg7keSrDhcEYkQ0UUnd0EVn8Rp52kyM3+QI/WoflSDig+RcuDHaVPez jbcPtSKtL5lprtdDggL9TJfcP4cXOyc= ARC-Authentication-Results: i=1; imf19.hostedemail.com; dkim=none; spf=pass (imf19.hostedemail.com: domain of osalvador@suse.de designates 195.135.223.131 as permitted sender) smtp.mailfrom=osalvador@suse.de; dmarc=pass (policy=none) header.from=suse.de ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1720067515; a=rsa-sha256; cv=none; b=Y5vs3/7vs2WqasENWxXlVHM9bwEvlEThjoHj3m49iT/gJbNryIgeklx5j6aUfiFIZGzn4j abF/7rQlw9LL3reBmhnF/8Gm0Z19XIVgBray4JdMKLixqRRsSRNE6E/TlzUbPM6S180MpD n0gfptWCLwxUboYb6LscInS6atqSjOY= Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [IPv6:2a07:de40:b281:104:10:150:64:97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id 2DFBF1FCF9; Thu, 4 Jul 2024 04:32:06 +0000 (UTC) Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id EC78E13889; Thu, 4 Jul 2024 04:32:04 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id MJ1iNMQlhmbnDAAAD6G6ig (envelope-from ); Thu, 04 Jul 2024 04:32:04 +0000 From: Oscar Salvador To: Andrew Morton Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Peter Xu , Muchun Song , David Hildenbrand , SeongJae Park , Miaohe Lin , Michal Hocko , Matthew Wilcox , Christophe Leroy , Oscar Salvador Subject: [PATCH 13/45] mm: Implement pud-version uffd functions Date: Thu, 4 Jul 2024 06:31:00 +0200 Message-ID: <20240704043132.28501-14-osalvador@suse.de> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240704043132.28501-1-osalvador@suse.de> References: <20240704043132.28501-1-osalvador@suse.de> MIME-Version: 1.0 X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Rspamd-Action: no action X-Rspamd-Server: rspam03 X-Rspam-User: X-Rspamd-Queue-Id: 51ACD1A000A X-Stat-Signature: aqp9jxrmzqmznwsni9hjazcasanqj33e X-HE-Tag: 1720067527-562181 X-HE-Meta: U2FsdGVkX19KdIAbe6juUsB55gcaZ/vw/220g60Ls6wSxarMNgagkbK3wSPyUh+ByqE2QvSDEXhrB9JHuy4hPIRDJDfBFty2nJMjybk4GjJrVbBD1b7uc8drPZb320VfSogHb24VbCcKw4GlFzbt3H5vkBNPecQzUEH4Gpe5QRnqQQDfvmAp27hlKTO0QZRVlMoNOO9RraT/yF3qUKy0oB4HRUlNDOe0lm9I8diS/uYd24T0Ynpxh0Wb2tiZLgHsfWJVZL10F5kD05O5SPngtB2ASljR2sY+OtbAkDpZD1Wv/xzF7j2ET114cHvlZdFeKlkIDMVABKY4nparu61KhuQyQgIFPGWhAud9qUXJWHXOJKtscMx/TLVlVgHsLLQOWSHQTdM/PLnRtNAfNt7Jn+nHdL3N5A0QPvfFSMeVmK2NayfUk42/boXeWFrD3DuJnGoKL3sdEqDDtb5rxR6IGlW8dWPaKeCPOdV4nnje85M4UFXAOU5twrat7UawFrJYUBm5xnkZ9AXJC7KcRTEigXRLg0X1Q8bLljmjSd3IzQPaePPgp+WjmFlCSVsxtrBS3p07vkX/3c+yPjUk+lOPiFn4ayEGH0T3xtIv7wYw1bjTHQDFl6OrhBSiYMUyqGLAR65hKp27Nlx/cSdPAwUCl2eANTvWM9cl5RYhMOsPjqTrG2MOJRpm4j6AJut8YuaGklxadkvnzi1RfYiHWdnlMWFlKXREDQmtl2rxjvy4HkVIL3iW9nG8dlELVs4CbJL1rv8+JbhBH91VoCO6uC75dVS4nVZUV9divsWVJgyWBlKq/t6Kg96CxSnnCUBMTaa21ii8dN70SFbRj8m2Sy5/JUlnutN4Hhd/Izuxy3ZIXMbNfU3ADlxqE1EPwzUtIALe4Q59JM60U1XxwTNFlQxbGlqiAxK4tWFoFxvTY0j3d+ei2lK+Nr9/ct8YbNXvG92OlR679eLvC6ie3eQQSCF 8xYw3MUR N1wDyY22RA+B0kFgIvHVrB5wyC0bjxwISIpctfpmPgK4v6xCuHiKwYItIWSoVmB4yTn5ee4zShI6R2bm4mTirqqg/qnv/KzvspQXTaNCWW4WT8RF6hZP4stZYH+buyDAbIqj+ X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: HugeTLB pages will be handled on pud level as well, so we need to implement pud-versions of uffd functions in order to properly handle it. Signed-off-by: Oscar Salvador --- arch/arm64/include/asm/pgtable.h | 7 ++ arch/x86/include/asm/pgtable.h | 158 +++++++++++++++++------------ include/asm-generic/pgtable_uffd.h | 30 ++++++ include/linux/pgtable.h | 5 + 4 files changed, 136 insertions(+), 64 deletions(-) diff --git a/arch/arm64/include/asm/pgtable.h b/arch/arm64/include/asm/pgtable.h index f8efbc128446..936ed3a915a3 100644 --- a/arch/arm64/include/asm/pgtable.h +++ b/arch/arm64/include/asm/pgtable.h @@ -556,6 +556,9 @@ static inline int pmd_trans_huge(pmd_t pmd) #define pmd_mkyoung(pmd) pte_pmd(pte_mkyoung(pmd_pte(pmd))) #define pmd_mkinvalid(pmd) pte_pmd(pte_mkinvalid(pmd_pte(pmd))) #ifdef CONFIG_HAVE_ARCH_USERFAULTFD_WP +#define pud_uffd_wp(pud) pte_uffd_wp(pud_pte(pud)) +#define pud_mkuffd_wp(pud) pte_pud(pte_mkuffd_wp(pud_pte(pud))) +#define pud_clear_uffd_wp(pud) pte_pud(pte_clear_uffd_wp(pud_pte(pud))) #define pmd_uffd_wp(pmd) pte_uffd_wp(pmd_pte(pmd)) #define pmd_mkuffd_wp(pmd) pte_pmd(pte_mkuffd_wp(pmd_pte(pmd))) #define pmd_clear_uffd_wp(pmd) pte_pmd(pte_clear_uffd_wp(pmd_pte(pmd))) @@ -563,6 +566,10 @@ static inline int pmd_trans_huge(pmd_t pmd) #define pmd_swp_mkuffd_wp(pmd) pte_pmd(pte_swp_mkuffd_wp(pmd_pte(pmd))) #define pmd_swp_clear_uffd_wp(pmd) \ pte_pmd(pte_swp_clear_uffd_wp(pmd_pte(pmd))) +#define pud_swp_uffd_wp(pud) pte_swp_uffd_wp(pud_pte(pud)) +#define pud_swp_mkuffd_wp(pud) pte_pud(pte_swp_mkuffd_wp(pud_pte(pud))) +#define pud_swp_clear_uffd_wp(pud) \ + pte_pud(pte_swp_clear_uffd_wp(pud_pte(pud))) #endif /* CONFIG_HAVE_ARCH_USERFAULTFD_WP */ #define pmd_write(pmd) pte_write(pmd_pte(pmd)) diff --git a/arch/x86/include/asm/pgtable.h b/arch/x86/include/asm/pgtable.h index 772f778bac06..640edc31962f 100644 --- a/arch/x86/include/asm/pgtable.h +++ b/arch/x86/include/asm/pgtable.h @@ -512,70 +512,6 @@ static inline pmd_t pmd_wrprotect(pmd_t pmd) return pmd_mksaveddirty(pmd); } -#ifdef CONFIG_HAVE_ARCH_USERFAULTFD_WP -static inline int pmd_uffd_wp(pmd_t pmd) -{ - return pmd_flags(pmd) & _PAGE_UFFD_WP; -} - -static inline pmd_t pmd_mkuffd_wp(pmd_t pmd) -{ - return pmd_wrprotect(pmd_set_flags(pmd, _PAGE_UFFD_WP)); -} - -static inline pmd_t pmd_clear_uffd_wp(pmd_t pmd) -{ - return pmd_clear_flags(pmd, _PAGE_UFFD_WP); -} -#endif /* CONFIG_HAVE_ARCH_USERFAULTFD_WP */ - -static inline pmd_t pmd_mkold(pmd_t pmd) -{ - return pmd_clear_flags(pmd, _PAGE_ACCESSED); -} - -static inline pmd_t pmd_mkclean(pmd_t pmd) -{ - return pmd_clear_flags(pmd, _PAGE_DIRTY_BITS); -} - -static inline pmd_t pmd_mkdirty(pmd_t pmd) -{ - pmd = pmd_set_flags(pmd, _PAGE_DIRTY | _PAGE_SOFT_DIRTY); - - return pmd_mksaveddirty(pmd); -} - -static inline pmd_t pmd_mkwrite_shstk(pmd_t pmd) -{ - pmd = pmd_clear_flags(pmd, _PAGE_RW); - - return pmd_set_flags(pmd, _PAGE_DIRTY); -} - -static inline pmd_t pmd_mkdevmap(pmd_t pmd) -{ - return pmd_set_flags(pmd, _PAGE_DEVMAP); -} - -static inline pmd_t pmd_mkhuge(pmd_t pmd) -{ - return pmd_set_flags(pmd, _PAGE_PSE); -} - -static inline pmd_t pmd_mkyoung(pmd_t pmd) -{ - return pmd_set_flags(pmd, _PAGE_ACCESSED); -} - -static inline pmd_t pmd_mkwrite_novma(pmd_t pmd) -{ - return pmd_set_flags(pmd, _PAGE_RW); -} - -pmd_t pmd_mkwrite(pmd_t pmd, struct vm_area_struct *vma); -#define pmd_mkwrite pmd_mkwrite - static inline pud_t pud_set_flags(pud_t pud, pudval_t set) { pudval_t v = native_pud_val(pud); @@ -659,6 +595,85 @@ static inline pud_t pud_mkwrite(pud_t pud) return pud_clear_saveddirty(pud); } +#ifdef CONFIG_HAVE_ARCH_USERFAULTFD_WP +static inline int pud_uffd_wp(pud_t pud) +{ + return pud_flags(pud) & _PAGE_UFFD_WP; +} + +static inline pud_t pud_mkuffd_wp(pud_t pud) +{ + return pud_wrprotect(pud_set_flags(pud, _PAGE_UFFD_WP)); +} + +static inline pud_t pud_clear_uffd_wp(pud_t pud) +{ + return pud_clear_flags(pud, _PAGE_UFFD_WP); +} + +static inline int pmd_uffd_wp(pmd_t pmd) +{ + return pmd_flags(pmd) & _PAGE_UFFD_WP; +} + +static inline pmd_t pmd_mkuffd_wp(pmd_t pmd) +{ + return pmd_wrprotect(pmd_set_flags(pmd, _PAGE_UFFD_WP)); +} + +static inline pmd_t pmd_clear_uffd_wp(pmd_t pmd) +{ + return pmd_clear_flags(pmd, _PAGE_UFFD_WP); +} +#endif /* CONFIG_HAVE_ARCH_USERFAULTFD_WP */ + +static inline pmd_t pmd_mkold(pmd_t pmd) +{ + return pmd_clear_flags(pmd, _PAGE_ACCESSED); +} + +static inline pmd_t pmd_mkclean(pmd_t pmd) +{ + return pmd_clear_flags(pmd, _PAGE_DIRTY_BITS); +} + +static inline pmd_t pmd_mkdirty(pmd_t pmd) +{ + pmd = pmd_set_flags(pmd, _PAGE_DIRTY | _PAGE_SOFT_DIRTY); + + return pmd_mksaveddirty(pmd); +} + +static inline pmd_t pmd_mkwrite_shstk(pmd_t pmd) +{ + pmd = pmd_clear_flags(pmd, _PAGE_RW); + + return pmd_set_flags(pmd, _PAGE_DIRTY); +} + +static inline pmd_t pmd_mkdevmap(pmd_t pmd) +{ + return pmd_set_flags(pmd, _PAGE_DEVMAP); +} + +static inline pmd_t pmd_mkhuge(pmd_t pmd) +{ + return pmd_set_flags(pmd, _PAGE_PSE); +} + +static inline pmd_t pmd_mkyoung(pmd_t pmd) +{ + return pmd_set_flags(pmd, _PAGE_ACCESSED); +} + +static inline pmd_t pmd_mkwrite_novma(pmd_t pmd) +{ + return pmd_set_flags(pmd, _PAGE_RW); +} + +pmd_t pmd_mkwrite(pmd_t pmd, struct vm_area_struct *vma); +#define pmd_mkwrite pmd_mkwrite + #ifdef CONFIG_HAVE_ARCH_SOFT_DIRTY static inline int pte_soft_dirty(pte_t pte) { @@ -1574,6 +1589,21 @@ static inline pmd_t pmd_swp_clear_uffd_wp(pmd_t pmd) { return pmd_clear_flags(pmd, _PAGE_SWP_UFFD_WP); } + +static inline pud_t pud_swp_mkuffd_wp(pud_t pud) +{ + return pud_set_flags(pud, _PAGE_SWP_UFFD_WP); +} + +static inline int pud_swp_uffd_wp(pud_t pud) +{ + return pud_flags(pud) & _PAGE_SWP_UFFD_WP; +} + +static inline pud_t pud_swp_clear_uffd_wp(pud_t pud) +{ + return pud_clear_flags(pud, _PAGE_SWP_UFFD_WP); +} #endif /* CONFIG_HAVE_ARCH_USERFAULTFD_WP */ static inline u16 pte_flags_pkey(unsigned long pte_flags) diff --git a/include/asm-generic/pgtable_uffd.h b/include/asm-generic/pgtable_uffd.h index 828966d4c281..453118eb2161 100644 --- a/include/asm-generic/pgtable_uffd.h +++ b/include/asm-generic/pgtable_uffd.h @@ -12,6 +12,11 @@ static __always_inline int pmd_uffd_wp(pmd_t pmd) return 0; } +static __always_inline int pud_uffd_wp(pud_t pud) +{ + return 0; +} + static __always_inline pte_t pte_mkuffd_wp(pte_t pte) { return pte; @@ -22,6 +27,11 @@ static __always_inline pmd_t pmd_mkuffd_wp(pmd_t pmd) return pmd; } +static __always_inline pud_t pud_mkuffd_wp(pud_t pud) +{ + return pmd; +} + static __always_inline pte_t pte_clear_uffd_wp(pte_t pte) { return pte; @@ -32,6 +42,11 @@ static __always_inline pmd_t pmd_clear_uffd_wp(pmd_t pmd) return pmd; } +static __always_inline pud_t pud_clear_uffd_wp(pud_t pud) +{ + return pmd; +} + static __always_inline pte_t pte_swp_mkuffd_wp(pte_t pte) { return pte; @@ -61,6 +76,21 @@ static inline pmd_t pmd_swp_clear_uffd_wp(pmd_t pmd) { return pmd; } + +static inline pud_t pud_swp_mkuffd_wp(pud_t pud) +{ + return pud; +} + +static inline int pud_swp_uffd_wp(pud_t pud) +{ + return 0; +} + +static inline pud_t pud_swp_clear_uffd_wp(pud_t pud) +{ + return pud; +} #endif /* CONFIG_HAVE_ARCH_USERFAULTFD_WP */ #endif /* _ASM_GENERIC_PGTABLE_UFFD_H */ diff --git a/include/linux/pgtable.h b/include/linux/pgtable.h index 991137dab87e..a2e2ebb93f21 100644 --- a/include/linux/pgtable.h +++ b/include/linux/pgtable.h @@ -1433,6 +1433,11 @@ static inline int pmd_soft_dirty(pmd_t pmd) return 0; } +static inline int pud_soft_dirty(pud_t pud) +{ + return 0; +} + static inline pte_t pte_mksoft_dirty(pte_t pte) { return pte; From patchwork Thu Jul 4 04:31:01 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oscar Salvador X-Patchwork-Id: 13723192 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9A206C30653 for ; Thu, 4 Jul 2024 04:32:32 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 6B6EA6B00A4; Thu, 4 Jul 2024 00:32:11 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 6328E6B00A3; Thu, 4 Jul 2024 00:32:11 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 4AF5C6B00A4; Thu, 4 Jul 2024 00:32:11 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 234626B00A2 for ; Thu, 4 Jul 2024 00:32:11 -0400 (EDT) Received: from smtpin18.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id C6A31160AEF for ; Thu, 4 Jul 2024 04:32:10 +0000 (UTC) X-FDA: 82300798020.18.BF8822C Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.223.131]) by imf17.hostedemail.com (Postfix) with ESMTP id B532B40009 for ; Thu, 4 Jul 2024 04:32:08 +0000 (UTC) Authentication-Results: imf17.hostedemail.com; dkim=none; spf=pass (imf17.hostedemail.com: domain of osalvador@suse.de designates 195.135.223.131 as permitted sender) smtp.mailfrom=osalvador@suse.de; dmarc=pass (policy=none) header.from=suse.de ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1720067516; a=rsa-sha256; cv=none; b=t7kPUBrDgHSc0NeSN42btMS38sEcfHIILjz5YHs0ak5GRQPcYz6biIAVMR/OHuRCzDeHL1 0/4+gZnppYf8cJBbm5x24WZNbA+LnlCDARWvLM0Xt/wXZ0C5YtUMM7+BoiYwM+vjx5Ulcr FvI5TKFv9SShMDPILpLKArMtb8wsUVo= ARC-Authentication-Results: i=1; imf17.hostedemail.com; dkim=none; spf=pass (imf17.hostedemail.com: domain of osalvador@suse.de designates 195.135.223.131 as permitted sender) smtp.mailfrom=osalvador@suse.de; dmarc=pass (policy=none) header.from=suse.de ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1720067516; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=YM4pkR47P5mKiI327bLPmyVEZ8VoXoGDDAHCQmx/AdI=; b=TtE6A/pZXaKnxYuoGhaTt+saufwo+Ua7K/JYgTQSgCvsOv0JFyCAH0vWO5/wV+wofhHeXN 9TfyMX9khYUtFi36Vx9JKfbBDM1c4mtVE46l7yGRw0yNlDrdF+siKSOCV5hrsnhS+SlMji jUyzsplAqG0KW/7RItR2703YC9OpDtY= Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [IPv6:2a07:de40:b281:104:10:150:64:97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id 820001FCF7; Thu, 4 Jul 2024 04:32:07 +0000 (UTC) Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 5043013889; Thu, 4 Jul 2024 04:32:06 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id cPLADsYlhmbnDAAAD6G6ig (envelope-from ); Thu, 04 Jul 2024 04:32:06 +0000 From: Oscar Salvador To: Andrew Morton Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Peter Xu , Muchun Song , David Hildenbrand , SeongJae Park , Miaohe Lin , Michal Hocko , Matthew Wilcox , Christophe Leroy , Oscar Salvador Subject: [PATCH 14/45] fs/proc: Create pagemap_pud_range to handle PUD-mapped hugetlb vmas Date: Thu, 4 Jul 2024 06:31:01 +0200 Message-ID: <20240704043132.28501-15-osalvador@suse.de> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240704043132.28501-1-osalvador@suse.de> References: <20240704043132.28501-1-osalvador@suse.de> MIME-Version: 1.0 X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Rspamd-Action: no action X-Stat-Signature: 47prs4dja1bjx8xou5976r6oftr7n8su X-Rspamd-Queue-Id: B532B40009 X-Rspam-User: X-Rspamd-Server: rspam10 X-HE-Tag: 1720067528-21556 X-HE-Meta: U2FsdGVkX18QOIcoMAF6nEPZAHqdAYVhkkcl/DXDJ+9ngjQ/RP0naXjbl3NfHRiLe66mhCH92eNTMcNQLarN7kIsFIi/pbSEUKb00g/A0/VMEeqRaoxOdp5EaO9H8uE1XljajIuCvQuCBfMYAeI+uWrGOrUu81ewmcKlFHPeeGV7jnNu++tC8gRtigKsHM0Ci2R4Qsbo4+vacqHvk2mZG7ifu5/a9rscjXrXLghp1iP08wOqOTDJhNuTsynGAuwV9tI48aJfBVOTwCmZTBjb8+iH9nH71e+FpFeIiQHeVvi1qcyykp+pFDTE58HdMWwjdYarDyjL1z02dcn9KLU1wI5jtGbBW/Md8Pv3FDRilJfDGeRhq/O3k8m/Mttpc0/JAJ3vC5a5zhtE1ZW7lO5lToLG3pRtLx5JDKrWtpGPtuZLPCPIcS5JfYBDpYq+naFk/QqSEVRW9uFgGYfVpQZDXy6l0qf7VOsqoylvk1y8Oa4Zp6z5AJc//DjRUS+B1ZhNci58lxGFz6x4FPUw4QLBO1s73vV5XG9NDZpI/1yxqxlwnDSTkJFfiW6A4/XF5omYT7Oz2q3oEFa3MOUGWAHjL8BJnfRTPgmFSeNWb7JQOtUVoAVyQjNHYwx2J+9xor2ldK2uLELPYwNbdltG4AjNvX14DSL0ttzXSGiLZT6/Y4Coay0CPKTPiP98s8rVPFC0+2elXLxBgA1NUv5UG5Xb/6A2JPK4jRXBiYi0+CNdZ+52+wqRBLf0VmnSCLRrr+qTbq78aoSp3o6kwI+VVc7fMKpGnNMvD7NccWkPCdlBxEtxcSHh/z85POZIP5/vklcPvN3qCk840Sqshv1a/5a+GeI/ZdSbffIsSiytbPCdu24l6lUttS9yDmwhuK8mOr54HWaxRw+EtwOYlfF5ctWuLTs6zLXCU7AxXaL1ArJ4+84KVWJQfri8HscJt5Xa1WBkNFi0BVunvOfqhABv220 uCbQenDq A4WJnf9nEjvMjim4j7CmaQ9ri9sJzZZhy4EA97sWJPcdU8nniVma4Fff44Ft9BFOF5hBUOzArnzbmMKCrxvGXyj1MRzBpfe7UPHpytVEpXamDOf81LGhOibvMq9bWQaNNTQdalhHU/KJmIiwK6sswuP7q36lyyi9/1Jl+9t7arFgjARs= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Normal THP cannot be PUD-mapped (besides devmap) but hugetlb can, so create pagemap_pud_range in order to handle PUD-mapped hugetlb vmas Signed-off-by: Oscar Salvador --- fs/proc/task_mmu.c | 60 ++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 60 insertions(+) diff --git a/fs/proc/task_mmu.c b/fs/proc/task_mmu.c index ec429d82b921..5965a074467e 100644 --- a/fs/proc/task_mmu.c +++ b/fs/proc/task_mmu.c @@ -1924,6 +1924,65 @@ static int pagemap_pmd_range(pmd_t *pmdp, unsigned long addr, unsigned long end, return err; } +#ifdef CONFIG_HUGETLB_PAGE +static int pagemap_pud_range(pud_t *pudp, unsigned long addr, unsigned long end, + struct mm_walk *walk) +{ + pud_t pud; + int err = 0; + spinlock_t *ptl; + u64 flags = 0, frame = 0; + struct pagemapread *pm = walk->private; + struct vm_area_struct *vma = walk->vma; + + ptl = pud_huge_lock(pudp, vma); + if (!ptl) + return err; + + pud = *pudp; + + if (vma->vm_flags & VM_SOFTDIRTY) + flags |= PM_SOFT_DIRTY; + if (pud_present(pud)) { + struct folio *folio = page_folio(pud_page(pud)); + + flags |= PM_PRESENT; + if (!folio_test_anon(folio)) + flags |= PM_FILE; + + if (!folio_likely_mapped_shared(folio)) + flags |= PM_MMAP_EXCLUSIVE; + + if (pud_soft_dirty(pud)) + flags |= PM_SOFT_DIRTY; + if (pud_uffd_wp(pud)) + flags |= PM_UFFD_WP; + if (pm->show_pfn) + frame = pud_pfn(pud) + + ((addr & ~PUD_MASK) >> PAGE_SHIFT); + } else if (pud_swp_uffd_wp(pud)) { + /* Only hugetlb can have swap entries at PUD level*/ + flags |= PM_UFFD_WP; + } + + for (; addr != end; addr += PAGE_SIZE) { + pagemap_entry_t pme = make_pme(frame, flags); + + err = add_to_pagemap(&pme, pm); + if (err) + return err; + if (pm->show_pfn && (flags & PM_PRESENT)) + frame++; + } + spin_unlock(ptl); + + cond_resched(); + return err; +} +#else +#define pagemap_pud_range NULL +#endif + #ifdef CONFIG_HUGETLB_PAGE /* This function walks within one hugetlb entry in the single call */ static int pagemap_hugetlb_range(pte_t *ptep, unsigned long hmask, @@ -1980,6 +2039,7 @@ static int pagemap_hugetlb_range(pte_t *ptep, unsigned long hmask, #endif /* HUGETLB_PAGE */ static const struct mm_walk_ops pagemap_ops = { + .pud_entry = pagemap_pud_range, .pmd_entry = pagemap_pmd_range, .pte_hole = pagemap_pte_hole, .hugetlb_entry = pagemap_hugetlb_range, From patchwork Thu Jul 4 04:31:02 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oscar Salvador X-Patchwork-Id: 13723193 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 44968C3065C for ; Thu, 4 Jul 2024 04:32:35 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 091A56B00A3; Thu, 4 Jul 2024 00:32:13 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 01B6A6B00A5; Thu, 4 Jul 2024 00:32:12 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D12826B00A6; Thu, 4 Jul 2024 00:32:12 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id A9CBA6B00A3 for ; Thu, 4 Jul 2024 00:32:12 -0400 (EDT) Received: from smtpin10.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 693EA40A22 for ; Thu, 4 Jul 2024 04:32:12 +0000 (UTC) X-FDA: 82300798104.10.5661E34 Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.223.131]) by imf08.hostedemail.com (Postfix) with ESMTP id 44477160013 for ; Thu, 4 Jul 2024 04:32:09 +0000 (UTC) Authentication-Results: imf08.hostedemail.com; dkim=pass header.d=suse.de header.s=susede2_rsa header.b=reCsEU+x; dkim=pass header.d=suse.de header.s=susede2_ed25519 header.b="cWSSs/Dz"; dkim=pass header.d=suse.de header.s=susede2_rsa header.b=reCsEU+x; dkim=pass header.d=suse.de header.s=susede2_ed25519 header.b="cWSSs/Dz"; spf=pass (imf08.hostedemail.com: domain of osalvador@suse.de designates 195.135.223.131 as permitted sender) smtp.mailfrom=osalvador@suse.de; dmarc=pass (policy=none) header.from=suse.de ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1720067518; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=fM4emcZ2cxXRGcFoZk72IXVFIB/7cc45xsWWyiF4u+4=; b=uoz+eQiJF5IPTlnUlbxXTwex1JIfQdStIl/7pPa48aPa9gJ65oJYHBPlpf4mwsrLn4eSHf xtmXiJHif0iQlQNrqzJU5ykCCS9kz8qIHoZ9V4tSPilRFThIOTMKc8gUyi2TeESQ9S2Wgc 3RPQ612Gc6sqRwODvDTo+82yocM9K4A= ARC-Authentication-Results: i=1; imf08.hostedemail.com; dkim=pass header.d=suse.de header.s=susede2_rsa header.b=reCsEU+x; dkim=pass header.d=suse.de header.s=susede2_ed25519 header.b="cWSSs/Dz"; dkim=pass header.d=suse.de header.s=susede2_rsa header.b=reCsEU+x; dkim=pass header.d=suse.de header.s=susede2_ed25519 header.b="cWSSs/Dz"; spf=pass (imf08.hostedemail.com: domain of osalvador@suse.de designates 195.135.223.131 as permitted sender) smtp.mailfrom=osalvador@suse.de; dmarc=pass (policy=none) header.from=suse.de ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1720067518; a=rsa-sha256; cv=none; b=e+LIgyj2U9aswb5xx5VuHpn/QcMErOSRebmok94ipudUP+D8PlnJwj1yptrZAcNosPUQcJ UqtmTKAFTN+8IphRxgb1U+AfrWg2x3LBrQfvb5eduheP5jLZtYo6GwcEAHoS77z92hNmOK +p2r/j2RKkfH/tCoJAwNP7/GgC85Z28= Received: from imap1.dmz-prg2.suse.org (unknown [10.150.64.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id E2CC31FCFB; Thu, 4 Jul 2024 04:32:08 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1720067528; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=fM4emcZ2cxXRGcFoZk72IXVFIB/7cc45xsWWyiF4u+4=; b=reCsEU+xOf0IC135atdv1qtTW7GwE3KuG5U1k4v+SluCIY6H3D2FUi8FAWIyXYCZIx84yW 2P3PP6oHEwHguqWCpYJ2Wi1pGYR41wpaVnU4D4lDnskSWrRP5eI4/8eoAU1cZhaAgG1ovS /ddxekibmWnrHYkaVCjv2+cJtRQFDJs= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1720067528; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=fM4emcZ2cxXRGcFoZk72IXVFIB/7cc45xsWWyiF4u+4=; b=cWSSs/Dz6GKtXty5+7bttcX9oK91QygswkqdEf4rRjRzvfkLm/fnE40h99EAYCwuc44cs1 Y9XuXXYLWDeZrCBA== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1720067528; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=fM4emcZ2cxXRGcFoZk72IXVFIB/7cc45xsWWyiF4u+4=; b=reCsEU+xOf0IC135atdv1qtTW7GwE3KuG5U1k4v+SluCIY6H3D2FUi8FAWIyXYCZIx84yW 2P3PP6oHEwHguqWCpYJ2Wi1pGYR41wpaVnU4D4lDnskSWrRP5eI4/8eoAU1cZhaAgG1ovS /ddxekibmWnrHYkaVCjv2+cJtRQFDJs= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1720067528; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=fM4emcZ2cxXRGcFoZk72IXVFIB/7cc45xsWWyiF4u+4=; b=cWSSs/Dz6GKtXty5+7bttcX9oK91QygswkqdEf4rRjRzvfkLm/fnE40h99EAYCwuc44cs1 Y9XuXXYLWDeZrCBA== Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id A7B8713889; Thu, 4 Jul 2024 04:32:07 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id cGPCI8clhmbnDAAAD6G6ig (envelope-from ); Thu, 04 Jul 2024 04:32:07 +0000 From: Oscar Salvador To: Andrew Morton Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Peter Xu , Muchun Song , David Hildenbrand , SeongJae Park , Miaohe Lin , Michal Hocko , Matthew Wilcox , Christophe Leroy , Oscar Salvador Subject: [PATCH 15/45] fs/proc: Adjust pte_to_pagemap_entry for hugetlb vmas Date: Thu, 4 Jul 2024 06:31:02 +0200 Message-ID: <20240704043132.28501-16-osalvador@suse.de> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240704043132.28501-1-osalvador@suse.de> References: <20240704043132.28501-1-osalvador@suse.de> MIME-Version: 1.0 X-Rspamd-Server: rspam03 X-Rspam-User: X-Rspamd-Queue-Id: 44477160013 X-Stat-Signature: x76wj8p44bsx1h9iuthrrpurigmtyqwy X-HE-Tag: 1720067529-114597 X-HE-Meta: U2FsdGVkX19K/e1uOP3rdObeQF22mlqokmvfoyxenzN+Rev2AuYx0MF2HBSw7pTkay33xFFHdRHtAqkFLJizFzxJKj445T1DHTUC871Xia4razTJu7VbSNaFdYRzznOeCEX/ZQCO1TNFLFB+5o7TsMdqrpXnTwSmvSpykI5EnshKzW/bVzu2As074ql0er1Vuknp1zIo0mTcKnsDvHwu2IzeDpMNeNaGOv1ZMmV8n9ELggckXoUsG1VcAoO0hsAgyuRsW789wm4lSRJE/abytkF/cVTQE8i3E6cvlB/m8E6cVVPQNT+AgKE3C6EpJ5La1I/Y60ZaQBoeYoCSh0WeH+l2LDoSBYjhHpWGGARbUCKT2wGRZtl7ZDbNmCHF+Igmh8GN6N31FnInsgjzenX2EXT9gp7PVVgQf6yErQJiWcn/XhiJpJDhl7FyWc7E9Aq+6+QhYMJv/eiWCXs5zCPnx5QFDKnApyP/W3Ixof46MY2B4/RVlyHPkBmhXH8N50PbywHHAZErrmzBmBOwDokpg/6QSJYMQj5zJxEREAh0Z2SMHxxiEPRWjUhhWWxKB+Hfq4FwYZSLQNHGvM9p6dm1tLk+sa/GKEZMn76BEA/Oz1GcCsbdwE3zkO5a85jEnvfFUg9p6rbofXlO7s+e85GIvrVdKuVgg8MPC6r3i36/FIqjhX0voES8B2X1w/YXycgcwoFYl7YtaTg89YA+rxl4CNCJ4HgcKSn33xJQ8BEpz/Iw+lnVLevZrQ6v6gpQINB7oFlV4FmdfIftNUQ3C7MC82Or3oSCFONHlDZX9KcILsbyby4raaG7sk+Xv4cI5IS1CIQSs4NgbfbdwJLUScgN59UluxksW+Ntd98XlY7nYzxArdBHiKy4MypjNHJDlIbwOAMDuXCG6r8dnfjluZ7b9Gh7RL690x2l/rpLcnJ752oBnJWhV8Uie+BWTgncBfeUcbJiyKTTYKgVi/Xw+F7 nZp4lUiU Y9kS/o8ZIJ0XOhcXKTrVwLrvmsLe4SIAOVzd7QA6oj/XJWQ4jujp6MnOyD1Waqjk7d65cjM3D17PUaLWMRoHvoecs/zYHTtkvOXBzbPH+AsMLvNRPro69ISyQSHAlhbb9g3AoUG0ubN2mgszoQxezLjyhSMnu5nkr2Oc1MYjpfAKfxZNmWaQpxDgN3LeGHSY5MrAf6Y8AjtPO2mvBU0ERbZ3IGPDDJOBbtWFoyapKgfzqtiR7y+ndONr7XCC/PeCduEgno4wGL+m7+YlZpcK94/hPonu7BvnDgWtm X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: HugeTLB pages cannot be swapped , so do not mark them as that. Signed-off-by: Oscar Salvador --- fs/proc/task_mmu.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/fs/proc/task_mmu.c b/fs/proc/task_mmu.c index 5965a074467e..22200018371d 100644 --- a/fs/proc/task_mmu.c +++ b/fs/proc/task_mmu.c @@ -1795,7 +1795,8 @@ static pagemap_entry_t pte_to_pagemap_entry(struct pagemapread *pm, frame = swp_type(entry) | (offset << MAX_SWAPFILES_SHIFT); } - flags |= PM_SWAP; + if (!is_vm_hugetlb_page(vma)) + flags |= PM_SWAP; if (is_pfn_swap_entry(entry)) page = pfn_swap_entry_to_page(entry); if (pte_marker_entry_uffd_wp(entry)) From patchwork Thu Jul 4 04:31:03 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oscar Salvador X-Patchwork-Id: 13723194 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id BB894C30653 for ; Thu, 4 Jul 2024 04:32:37 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 7984A6B00A5; Thu, 4 Jul 2024 00:32:14 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 7205B6B00A6; Thu, 4 Jul 2024 00:32:14 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 54C736B00A7; Thu, 4 Jul 2024 00:32:14 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 330FC6B00A5 for ; Thu, 4 Jul 2024 00:32:14 -0400 (EDT) Received: from smtpin15.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id A9F3F140A71 for ; Thu, 4 Jul 2024 04:32:13 +0000 (UTC) X-FDA: 82300798146.15.6217C5B Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.223.130]) by imf21.hostedemail.com (Postfix) with ESMTP id 6D6221C000C for ; Thu, 4 Jul 2024 04:32:11 +0000 (UTC) Authentication-Results: imf21.hostedemail.com; dkim=pass header.d=suse.de header.s=susede2_rsa header.b=GTzwuWvk; dkim=pass header.d=suse.de header.s=susede2_ed25519 header.b=jvDVW3J6; dkim=pass header.d=suse.de header.s=susede2_rsa header.b=GTzwuWvk; dkim=pass header.d=suse.de header.s=susede2_ed25519 header.b=jvDVW3J6; spf=pass (imf21.hostedemail.com: domain of osalvador@suse.de designates 195.135.223.130 as permitted sender) smtp.mailfrom=osalvador@suse.de; dmarc=pass (policy=none) header.from=suse.de ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1720067519; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=Ywx5qYincblCns1hUo2a7V32kYl58scbVMBY9LF3T58=; b=Dg2oLOevl27hAR4ImsoQ/QPqlfjp1yhJ07wODJQn1viAYPxLIMJQt+ejg78npLxp3obmev Z4SBe+yWl0xtsZxSE6kWNNsADr0DMXHOPnH+6TgSWwi8MM+clTPt2Fdi8RwYSbdKjo2pxi OOqKlRZb0jn0kR0wfAGhMviEpCF1BEg= ARC-Authentication-Results: i=1; imf21.hostedemail.com; dkim=pass header.d=suse.de header.s=susede2_rsa header.b=GTzwuWvk; dkim=pass header.d=suse.de header.s=susede2_ed25519 header.b=jvDVW3J6; dkim=pass header.d=suse.de header.s=susede2_rsa header.b=GTzwuWvk; dkim=pass header.d=suse.de header.s=susede2_ed25519 header.b=jvDVW3J6; spf=pass (imf21.hostedemail.com: domain of osalvador@suse.de designates 195.135.223.130 as permitted sender) smtp.mailfrom=osalvador@suse.de; dmarc=pass (policy=none) header.from=suse.de ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1720067519; a=rsa-sha256; cv=none; b=xBLi0uvZTjIIDryoBB1GGiEf2VzSttGk+OYodOlmFEWUNl1R+5t3RvKcAYyx3PLiUVTWvu YmK4hH9LC1HViwpibuu5jkYBCg8czr19ITOXsSGSTpSoI7Qn5Db02JQhlLzR8QG0Vb7VAm SRwpCS4sEj1oAb07bTTzDkQF5YGh1ns= Received: from imap1.dmz-prg2.suse.org (unknown [10.150.64.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id 2A18B21180; Thu, 4 Jul 2024 04:32:10 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1720067530; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Ywx5qYincblCns1hUo2a7V32kYl58scbVMBY9LF3T58=; b=GTzwuWvkW4352FKSC60QDraIO7eJOmAixIJz5RFRrudJmWfKJkpiuKZHVlx8nO0wQbcFzt qLNYDqW+shNAbNnPmhFgaxr/xkRqxsQJ9oLNgCTLnZXKEUR5ZknhTrQbnFUU6RkjXXQiol jNr+qp+D+BH1yGYXIesILMJvxw273IE= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1720067530; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Ywx5qYincblCns1hUo2a7V32kYl58scbVMBY9LF3T58=; b=jvDVW3J69xcP45jkzhmWbnnV1lDl4uY2pXNQ4DMnh1no9iNhknQuhbQaJO2boD0G3/hFKq b91zyh46S6CvurBg== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1720067530; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Ywx5qYincblCns1hUo2a7V32kYl58scbVMBY9LF3T58=; b=GTzwuWvkW4352FKSC60QDraIO7eJOmAixIJz5RFRrudJmWfKJkpiuKZHVlx8nO0wQbcFzt qLNYDqW+shNAbNnPmhFgaxr/xkRqxsQJ9oLNgCTLnZXKEUR5ZknhTrQbnFUU6RkjXXQiol jNr+qp+D+BH1yGYXIesILMJvxw273IE= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1720067530; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Ywx5qYincblCns1hUo2a7V32kYl58scbVMBY9LF3T58=; b=jvDVW3J69xcP45jkzhmWbnnV1lDl4uY2pXNQ4DMnh1no9iNhknQuhbQaJO2boD0G3/hFKq b91zyh46S6CvurBg== Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 1083C13889; Thu, 4 Jul 2024 04:32:08 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id QIUEO8glhmbnDAAAD6G6ig (envelope-from ); Thu, 04 Jul 2024 04:32:08 +0000 From: Oscar Salvador To: Andrew Morton Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Peter Xu , Muchun Song , David Hildenbrand , SeongJae Park , Miaohe Lin , Michal Hocko , Matthew Wilcox , Christophe Leroy , Oscar Salvador Subject: [PATCH 16/45] fs/proc: Enable pagemap_scan_pmd_entry to handle hugetlb vmas Date: Thu, 4 Jul 2024 06:31:03 +0200 Message-ID: <20240704043132.28501-17-osalvador@suse.de> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240704043132.28501-1-osalvador@suse.de> References: <20240704043132.28501-1-osalvador@suse.de> MIME-Version: 1.0 X-Rspamd-Server: rspam03 X-Rspam-User: X-Rspamd-Queue-Id: 6D6221C000C X-Stat-Signature: pztweqmgo1ej415e6fqnz1c6p5x7s58r X-HE-Tag: 1720067531-648812 X-HE-Meta: U2FsdGVkX19YwOjO936A6+lmViBMjyN3SNvXwn2o1Ncw8BssqMQRzl4mQsNJ8Dr/QkzUlbZBW6T0/EHm0mfNduwhYDEnhhae4hM8mZPB9JW03m6KJrLEaL9djNZjOZ5TKgbf4lPzzvSl1waXTQLCZX2AvzBD12wr6J/+UfoBsx6A8ot289WhVKpNKoRA0VhBkPIs1TpJzBptawQviV1j5VFuwGHl4SmFTUbFWr/9aRFA4MZ0BaY5oWUx4D18qdEW/xMp7D52cCSq8y/O9I7jQiH6ar1fgcgpVQPWjeDfpdhieQI8ilGuORZGAWcf/iR3G3BetnuszuxuqfYK1wHfp3KDhsEOSLB2W5/M7WRda1NY/mZIZu2TM9xJqqjsO1Lb+YN8AOBDCzzPZnM8kNCr1ODDxhvT9VVU/SWsVLTuhYODfBxQfRXV0IKdRbFUBLT4Iq8P+2nwGXT3ykWfB33XBxgHW7Ikq6rufj7fOvmvXhnsTQfpt5m5L9nh13/Nignje+prBClygNGQWW9Im9zT85Chuq4BRDChDRBba30/6GBel8E4bLer10FQZkocLMvxJqTulUv88aBP7lq/ECQgALnDm/z6S+j0c7DaoCqPNMff7fZD3s0arMxArln0UhuV3csUhCRG1AGt9Z8P+8JghyrB4cbGfHOyniDJ/w554Spp9sWAfY/YmcxS/leHtFUZDpjNxIN0m8Kody+5wPwXP4Gum+oG3UNJeLRT6Ddc4lPkBz1sAOR0S9XSZFx7gzpUrNPx07iWX5wGYfDjW4F/Q1W5Stz0Y8s2bBvdvqC4WAjX899iNb22IG6jz7T8FZWMkEUzqpARgVpkBcSWqeD/d5RpkAviYYbMZoRY+EYwNoGQgPIJOcnEGJIDQF+vjR70Qx8tXpgcgnPymY7CdV3An86TNTcG6yFQT1eJyGDE/3OTDABEHJdn9+dPMw67ZSkm3M3VaipJOtr//udGiwy bkHfomYt a7GXqPg4kqYVhXEJfRZkx/ChurZ81G3j/HJwFUPYRkOTd5o0w30L7J2rS/J4eAJttlKUNemw8PMIie6Ja9jgDl5oyGhzSOeOkKEjClLWkFYakvP1hWzwymMAgTa3aS4tgp42swoatxf4C1l6rB9SX0J+ogEW5khx45GADzNDSqnC4H/dT3OMYM2fACMjNY03gqxf9oLz1TeATVEbyycrjxBgosQ31fVx2cohSFvf+gFDclevnbvKKR385vrrjOoP1Kf/d4UwaDgu6mKntBMDxJTziZ1cdi55Amv8VihLUOulfvQo4gVP8fbQK5R0P+Io7rJjF2FhDpHreNpw= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: PMD-mapped hugetlb vmas will also reach pagemap_scan_pmd_entry. Add the required code so it knows how to handle those there. Signed-off-by: Oscar Salvador --- fs/proc/task_mmu.c | 41 ++++++++++++++++++++++++++--------------- 1 file changed, 26 insertions(+), 15 deletions(-) diff --git a/fs/proc/task_mmu.c b/fs/proc/task_mmu.c index 22200018371d..df649f69ea2c 100644 --- a/fs/proc/task_mmu.c +++ b/fs/proc/task_mmu.c @@ -2263,8 +2263,8 @@ static void make_uffd_wp_pte(struct vm_area_struct *vma, } } -#ifdef CONFIG_TRANSPARENT_HUGEPAGE -static unsigned long pagemap_thp_category(struct pagemap_scan_private *p, +#ifdef CONFIG_PGTABLE_HAS_HUGE_LEAVES +static unsigned long pagemap_pmd_category(struct pagemap_scan_private *p, struct vm_area_struct *vma, unsigned long addr, pmd_t pmd) { @@ -2296,7 +2296,8 @@ static unsigned long pagemap_thp_category(struct pagemap_scan_private *p, if (pmd_swp_soft_dirty(pmd)) categories |= PAGE_IS_SOFT_DIRTY; - if (p->masks_of_interest & PAGE_IS_FILE) { + if ((p->masks_of_interest & PAGE_IS_FILE) && + !is_vm_hugetlb_page(vma)) { swp = pmd_to_swp_entry(pmd); if (is_pfn_swap_entry(swp) && !folio_test_anon(pfn_swap_entry_folio(swp))) @@ -2321,7 +2322,7 @@ static void make_uffd_wp_pmd(struct vm_area_struct *vma, set_pmd_at(vma->vm_mm, addr, pmdp, pmd); } } -#endif /* CONFIG_TRANSPARENT_HUGEPAGE */ +#endif /* CONFIG_PGTABLE_HAS_HUGE_LEAVES */ #ifdef CONFIG_HUGETLB_PAGE static unsigned long pagemap_hugetlb_category(pte_t pte) @@ -2522,22 +2523,22 @@ static int pagemap_scan_output(unsigned long categories, return ret; } -static int pagemap_scan_thp_entry(pmd_t *pmd, unsigned long start, +static int pagemap_scan_huge_entry(pmd_t *pmd, unsigned long start, unsigned long end, struct mm_walk *walk) { -#ifdef CONFIG_TRANSPARENT_HUGEPAGE +#ifdef CONFIG_PGTABLE_HAS_HUGE_LEAVES struct pagemap_scan_private *p = walk->private; struct vm_area_struct *vma = walk->vma; unsigned long categories; spinlock_t *ptl; int ret = 0; - ptl = pmd_trans_huge_lock(pmd, vma); + ptl = pmd_huge_lock(pmd, vma); if (!ptl) return -ENOENT; categories = p->cur_vma_category | - pagemap_thp_category(p, vma, start, *pmd); + pagemap_pmd_category(p, vma, start, *pmd); if (!pagemap_scan_is_interesting_page(categories, p)) goto out_unlock; @@ -2556,19 +2557,29 @@ static int pagemap_scan_thp_entry(pmd_t *pmd, unsigned long start, * needs to be performed on a portion of the huge page. */ if (end != start + HPAGE_SIZE) { - spin_unlock(ptl); - split_huge_pmd(vma, pmd, start); pagemap_scan_backout_range(p, start, end); - /* Report as if there was no THP */ - return -ENOENT; + if (!is_vm_hugetlb_page(vma)) { + /* Report as if there was no THP */ + spin_unlock(ptl); + split_huge_pmd(vma, pmd, start); + ret = -ENOENT; + goto out; + } + ret = 0; + p->arg.walk_end = start; + goto out_unlock; } make_uffd_wp_pmd(vma, start, pmd); - flush_tlb_range(vma, start, end); + if (is_vm_hugetlb_page(vma)) + flush_hugetlb_tlb_range(vma, start, end); + else + flush_tlb_range(vma, start, end); out_unlock: spin_unlock(ptl); +out: return ret; -#else /* !CONFIG_TRANSPARENT_HUGEPAGE */ +#else /* CONFIG_PGTABLE_HAS_HUGE_LEAVES */ return -ENOENT; #endif } @@ -2585,7 +2596,7 @@ static int pagemap_scan_pmd_entry(pmd_t *pmd, unsigned long start, arch_enter_lazy_mmu_mode(); - ret = pagemap_scan_thp_entry(pmd, start, end, walk); + ret = pagemap_scan_huge_entry(pmd, start, end, walk); if (ret != -ENOENT) { arch_leave_lazy_mmu_mode(); return ret; From patchwork Thu Jul 4 04:31:04 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oscar Salvador X-Patchwork-Id: 13723195 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9B314C3065C for ; Thu, 4 Jul 2024 04:32:40 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 881F56B00A6; Thu, 4 Jul 2024 00:32:15 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 808A86B00A7; Thu, 4 Jul 2024 00:32:15 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 6604A6B00A8; Thu, 4 Jul 2024 00:32:15 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 3EAAE6B00A6 for ; Thu, 4 Jul 2024 00:32:15 -0400 (EDT) Received: from smtpin12.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id ECB80160FC2 for ; Thu, 4 Jul 2024 04:32:14 +0000 (UTC) X-FDA: 82300798188.12.E74BE91 Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.223.131]) by imf23.hostedemail.com (Postfix) with ESMTP id B6117140005 for ; Thu, 4 Jul 2024 04:32:12 +0000 (UTC) Authentication-Results: imf23.hostedemail.com; dkim=pass header.d=suse.de header.s=susede2_rsa header.b=rGzyLD7L; dkim=pass header.d=suse.de header.s=susede2_ed25519 header.b=8nHTN6MV; dkim=pass header.d=suse.de header.s=susede2_rsa header.b=rGzyLD7L; dkim=pass header.d=suse.de header.s=susede2_ed25519 header.b=8nHTN6MV; spf=pass (imf23.hostedemail.com: domain of osalvador@suse.de designates 195.135.223.131 as permitted sender) smtp.mailfrom=osalvador@suse.de; dmarc=pass (policy=none) header.from=suse.de ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1720067508; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=fKLC6fxewOrLf0i98Rd25YFQam44kkf2M+DOutpQYdQ=; b=Nfskna/yHITUE/HjyApQr/HZFHRjCiX+m+iKGk3JbxuzHzdFm5nXIbkHhyRjMrCHxAUsK1 8VJjQ3ilP1Ru4wqUhYomyTojE5w9gkBnHchOtrUeKyS3WDj2uY2IOpOr8gIcGUg1JNtR0k V0CchcVx2Bo+Go2NTPd/Bh3ft7Recws= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1720067508; a=rsa-sha256; cv=none; b=Nk9fVQM3lE49FbPPHs+iVuKg81LWTyJq5dRDheUNszGoqQXvDyeAmE/F3W9s9NZz8E+Hh4 Wd198GuOm3Pdk2zMt+RG7Dsc7MV9yePiRdQZMeP8nWBc3szq2v0D03MVWAZHpAK8jbivdX QhHqiaGi7ivpEKoySeGq+n+/PEzrnME= ARC-Authentication-Results: i=1; imf23.hostedemail.com; dkim=pass header.d=suse.de header.s=susede2_rsa header.b=rGzyLD7L; dkim=pass header.d=suse.de header.s=susede2_ed25519 header.b=8nHTN6MV; dkim=pass header.d=suse.de header.s=susede2_rsa header.b=rGzyLD7L; dkim=pass header.d=suse.de header.s=susede2_ed25519 header.b=8nHTN6MV; spf=pass (imf23.hostedemail.com: domain of osalvador@suse.de designates 195.135.223.131 as permitted sender) smtp.mailfrom=osalvador@suse.de; dmarc=pass (policy=none) header.from=suse.de Received: from imap1.dmz-prg2.suse.org (unknown [10.150.64.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id 717E31FCF8; Thu, 4 Jul 2024 04:32:11 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1720067531; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=fKLC6fxewOrLf0i98Rd25YFQam44kkf2M+DOutpQYdQ=; b=rGzyLD7LkYyMAvmIw4xFUDH+buX9d/II8oeFi+WRfNzUSU6mBH1Xd+8r3gYOEPGL9dYDCy IlLqdjWxRy5mraTCwA+0ut4zZnng0xAJIvDVvopn+3dMXHd6APSdUZmCaNvfWzCqwkjiNn u/St7F29ojB/6wl0e44bDnr1D1/7TB8= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1720067531; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=fKLC6fxewOrLf0i98Rd25YFQam44kkf2M+DOutpQYdQ=; b=8nHTN6MVi8gMxqJWpwXDGmQUc1P5i9EMQ+Rwyw40Sbfz/xQvdeEiZOiulE9osEkcCpeY4k 3YaniTexv02tM6Cg== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1720067531; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=fKLC6fxewOrLf0i98Rd25YFQam44kkf2M+DOutpQYdQ=; b=rGzyLD7LkYyMAvmIw4xFUDH+buX9d/II8oeFi+WRfNzUSU6mBH1Xd+8r3gYOEPGL9dYDCy IlLqdjWxRy5mraTCwA+0ut4zZnng0xAJIvDVvopn+3dMXHd6APSdUZmCaNvfWzCqwkjiNn u/St7F29ojB/6wl0e44bDnr1D1/7TB8= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1720067531; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=fKLC6fxewOrLf0i98Rd25YFQam44kkf2M+DOutpQYdQ=; b=8nHTN6MVi8gMxqJWpwXDGmQUc1P5i9EMQ+Rwyw40Sbfz/xQvdeEiZOiulE9osEkcCpeY4k 3YaniTexv02tM6Cg== Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 4E2E313889; Thu, 4 Jul 2024 04:32:10 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id UP40DsolhmbnDAAAD6G6ig (envelope-from ); Thu, 04 Jul 2024 04:32:10 +0000 From: Oscar Salvador To: Andrew Morton Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Peter Xu , Muchun Song , David Hildenbrand , SeongJae Park , Miaohe Lin , Michal Hocko , Matthew Wilcox , Christophe Leroy , Oscar Salvador Subject: [PATCH 17/45] mm: Implement pud-version for pud_mkinvalid and pudp_establish Date: Thu, 4 Jul 2024 06:31:04 +0200 Message-ID: <20240704043132.28501-18-osalvador@suse.de> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240704043132.28501-1-osalvador@suse.de> References: <20240704043132.28501-1-osalvador@suse.de> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: B6117140005 X-Stat-Signature: eqgfwtg7fyytzs1x86ihni4kyh4fypkq X-HE-Tag: 1720067532-236303 X-HE-Meta: U2FsdGVkX1888D+iyzwkpV90QtXXNjTlF9/WmQO0tV0S2SWTQftH4I3TG8VoCKOMxPqN+WCnMAxD4wJaMgJIJqlTIB/JILzVIdNpu0KHHb5vLB8hPrbSG5ENkmB/o8quzwWsorvUsxb+KTTIruyUhgQPMGxH5vKY9l8w+YlqpNHF6Xu4iWWD3tpK3usf/QITGp51q8mhwSNr92ExMhJzHjIjJfF+c9KI9vaJP0e/6LZwONWVH1XiChcixYiijH+wWNoRjOBNNks/go60IReKNEbzGwFt+tVeJONCsKxGyY7CCcq6LsNeCwnXv6FKkoFuPGGHD0TcO/sv2CFmtHDYQW2KoTyhl/S/RJuVzIYNnTXlf6CtFo5Kcn1Av9Qe4fvJm5VDy0JJ7YvCjIGRMHxCpzBsotiudXFGDMBrTV7sl966qk406FYg6xSX49KADllIf0BdDkFbdBR19Lkuh3Vh5w5g8nUQqHaf52fjmWne+R6wPDHvnugwe/i9ALwkMez0kh3VZWoWfAG14oZnQVyAwyXPZXYY0qHAQSbhwc+RlpBHUVy/QTyuKEwOuQ2RfC4NusPoTUB2uRrGm0dyzN5Eh3xOeU3oBz+HPuPuJtarDnCwDp8+Fau5Ka51lQLQiAZmdKIOW5plQ2dVnyYXHhvv8vVvaOMSG3kato+ieLbUfW6mW3Z3RyrI5jyPkIe8L9EQSg9Bylc2NN7xUPXhVLDbsPNQyfATWWBnXp4gwzVVNbvKNbviJhjcmfV6DyFAt0/9XM53ZIT/naBuUULsy+1DX3gMAgA+HpqNN8BKkW7BagxdHKEo0fw7UMndbGA2A7rLJAoEYAftbN7Wt4Csrlmy4d0hBrIE1qwaDgyqmHw4U59X/hduQN5NFHXT6QAySOWChm95iYXKMoaZ44q/45Tp/H5moVOz0pWzfrgudxFIjwoR0vPQyOL9fwzLuBfftJB9DHc1UvmxRah4Q5Q2dXR CTDV27uj WvTpU1wQVYBrMJfTJntg7OyAwiZlO63TVzbUxCTW3EbjHWtAvZF4HSZryp30lL8yph3sb9xmt5QoUSZhql3sCBcJAS84psORzgGbhCw6bHf4aoymecLviKkRmuZtNo2GWOamtbemNnFQEOOdg1LRIJ9h/IBgiT7POEQTVXinooQmJL67N9P4Xv7DW0270U70Nj11VPqhELCyQkuuQyGr4lxrg+dtg+rcCM1f0ahj94a2YV8/8Mbnk1NM3BAZvvYfVJ38HyhdX2RsNYb8QHpmNsAoMxFcnusLHTXaR X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: HugeTLB pages will be handled on pud level as well, so we need to implement pud-versions of pud_mkinvalid and pudp_establish. Signed-off-by: Oscar Salvador --- arch/arm64/include/asm/pgtable.h | 11 ++++++ arch/loongarch/include/asm/pgtable.h | 8 ++++ arch/mips/include/asm/pgtable.h | 7 ++++ arch/powerpc/include/asm/book3s/64/pgtable.h | 7 +++- arch/powerpc/mm/book3s64/pgtable.c | 15 ++++++- arch/riscv/include/asm/pgtable.h | 15 +++++++ arch/x86/include/asm/pgtable.h | 31 ++++++++++++++- include/linux/pgtable.h | 41 +++++++++++++++++++- mm/pgtable-generic.c | 21 ++++++++++ 9 files changed, 150 insertions(+), 6 deletions(-) diff --git a/arch/arm64/include/asm/pgtable.h b/arch/arm64/include/asm/pgtable.h index 936ed3a915a3..5e26e63b1012 100644 --- a/arch/arm64/include/asm/pgtable.h +++ b/arch/arm64/include/asm/pgtable.h @@ -595,6 +595,7 @@ static inline pmd_t pmd_mkdevmap(pmd_t pmd) #define pud_write(pud) pte_write(pud_pte(pud)) #define pud_mkhuge(pud) (__pud(pud_val(pud) & ~PUD_TABLE_BIT)) +#define pud_mkinvalid(pud) pte_pud(pte_mkinvalid(pud_pte(pud))) #define __pud_to_phys(pud) __pte_to_phys(pud_pte(pud)) #define __phys_to_pud_val(phys) __phys_to_pte_val(phys) @@ -1344,6 +1345,16 @@ static inline pmd_t pmdp_establish(struct vm_area_struct *vma, } #endif +#ifdef CONFIG_HUGETLB_PAGE +#define pudp_establish pudp_establish +static inline pud_t pudp_establish(struct vm_area_struct *vma, + unsigned long address, pud_t *pudp, pud_t pud) +{ + page_table_check_pud_set(vma->vm_mm, pudp, pud); + return __pud(xchg_relaxed(&pud_val(*pudp), pud_val(pud))); +} +#endif + /* * Encode and decode a swap entry: * bits 0-1: present (must be zero) diff --git a/arch/loongarch/include/asm/pgtable.h b/arch/loongarch/include/asm/pgtable.h index 161dd6e10479..cf73c2f2da2c 100644 --- a/arch/loongarch/include/asm/pgtable.h +++ b/arch/loongarch/include/asm/pgtable.h @@ -581,6 +581,14 @@ static inline pmd_t pmd_mkinvalid(pmd_t pmd) return pmd; } +static inline pud_t pud_mkinvalid(pud_t pud) +{ + pud_val(pud) |= _PAGE_PRESENT_INVALID; + pud_val(pud) &= ~(_PAGE_PRESENT | _PAGE_VALID | _PAGE_DIRTY | _PAGE_PROTNONE); + + return pud; +} + /* * The generic version pmdp_huge_get_and_clear uses a version of pmd_clear() with a * different prototype. diff --git a/arch/mips/include/asm/pgtable.h b/arch/mips/include/asm/pgtable.h index c29a551eb0ca..390a2f022147 100644 --- a/arch/mips/include/asm/pgtable.h +++ b/arch/mips/include/asm/pgtable.h @@ -736,6 +736,13 @@ static inline pmd_t pmd_mkinvalid(pmd_t pmd) return pmd; } +static inline pud_t pud_mkinvalid(pud_t pud) +{ + pud_val(pud) &= ~(_PAGE_PRESENT | _PAGE_VALID | _PAGE_DIRTY); + + return pud; +} + /* * The generic version pmdp_huge_get_and_clear uses a version of pmd_clear() with a * different prototype. diff --git a/arch/powerpc/include/asm/book3s/64/pgtable.h b/arch/powerpc/include/asm/book3s/64/pgtable.h index fa4bb8d6356f..f95ac2a87548 100644 --- a/arch/powerpc/include/asm/book3s/64/pgtable.h +++ b/arch/powerpc/include/asm/book3s/64/pgtable.h @@ -1085,7 +1085,8 @@ static inline pte_t *pmdp_ptep(pmd_t *pmd) #define pmd_mksoft_dirty(pmd) pte_pmd(pte_mksoft_dirty(pmd_pte(pmd))) #define pmd_clear_soft_dirty(pmd) pte_pmd(pte_clear_soft_dirty(pmd_pte(pmd))) -#ifdef CONFIG_ARCH_ENABLE_THP_MIGRATION +#if defined(CONFIG_ARCH_ENABLE_THP_MIGRATION) || defined(CONFIG_HUGETLB_PAGE) +#define pud_swp_soft_dirty(pud) pte_swp_soft_dirty(pud_pte(pud)) #define pmd_swp_mksoft_dirty(pmd) pte_pmd(pte_swp_mksoft_dirty(pmd_pte(pmd))) #define pmd_swp_soft_dirty(pmd) pte_swp_soft_dirty(pmd_pte(pmd)) #define pmd_swp_clear_soft_dirty(pmd) pte_pmd(pte_swp_clear_soft_dirty(pmd_pte(pmd))) @@ -1386,6 +1387,10 @@ static inline pgtable_t pgtable_trans_huge_withdraw(struct mm_struct *mm, extern pmd_t pmdp_invalidate(struct vm_area_struct *vma, unsigned long address, pmd_t *pmdp); +#define __HAVE_ARCH_PUDP_INVALIDATE +extern pud_t pudp_invalidate(struct vm_area_struct *vma, unsigned long address, + pud_t *pudp); + #define pmd_move_must_withdraw pmd_move_must_withdraw struct spinlock; extern int pmd_move_must_withdraw(struct spinlock *new_pmd_ptl, diff --git a/arch/powerpc/mm/book3s64/pgtable.c b/arch/powerpc/mm/book3s64/pgtable.c index f4d8d3c40e5c..1b6ae7898f99 100644 --- a/arch/powerpc/mm/book3s64/pgtable.c +++ b/arch/powerpc/mm/book3s64/pgtable.c @@ -37,7 +37,7 @@ EXPORT_SYMBOL(__pmd_frag_nr); unsigned long __pmd_frag_size_shift; EXPORT_SYMBOL(__pmd_frag_size_shift); -#ifdef CONFIG_TRANSPARENT_HUGEPAGE +#ifdef CONFIG_PGTABLE_HAS_HUGE_LEAVES /* * This is called when relaxing access to a hugepage. It's also called in the page * fault path when we don't hit any of the major fault cases, ie, a minor @@ -259,7 +259,18 @@ pmd_t pmd_modify(pmd_t pmd, pgprot_t newprot) pmdv &= _HPAGE_CHG_MASK; return pmd_set_protbits(__pmd(pmdv), newprot); } -#endif /* CONFIG_TRANSPARENT_HUGEPAGE */ + +pud_t pudp_invalidate(struct vm_area_struct *vma, unsigned long address, + pud_t *pudp) +{ + unsigned long old_pud; + + VM_WARN_ON_ONCE(!pud_present(*pudp)); + old_pud = pud_hugepage_update(vma->vm_mm, address, pudp, _PAGE_PRESENT, _PAGE_INVALID); + flush_pud_tlb_range(vma, address, address + HPAGE_PUD_SIZE); + return __pud(old_pud); +} +#endif /* CONFIG_PGTABLE_HAS_HUGE_LEAVES */ /* For use by kexec, called with MMU off */ notrace void mmu_cleanup_all(void) diff --git a/arch/riscv/include/asm/pgtable.h b/arch/riscv/include/asm/pgtable.h index ebfe8faafb79..51600afa203c 100644 --- a/arch/riscv/include/asm/pgtable.h +++ b/arch/riscv/include/asm/pgtable.h @@ -657,6 +657,11 @@ static inline unsigned long pud_pfn(pud_t pud) return ((__pud_to_phys(pud) & PUD_MASK) >> PAGE_SHIFT); } +static inline pud_t pud_mkinvalid(pud_t pud) +{ + return __pud(pud_val(pud) & ~(_PAGE_PRESENT|_PAGE_PROT_NONE)); +} + static inline pmd_t pmd_modify(pmd_t pmd, pgprot_t newprot) { return pte_pmd(pte_modify(pmd_pte(pmd), newprot)); @@ -804,6 +809,16 @@ extern pmd_t pmdp_collapse_flush(struct vm_area_struct *vma, unsigned long address, pmd_t *pmdp); #endif /* CONFIG_TRANSPARENT_HUGEPAGE */ +#ifdef CONFIG_HUGETLB_PAGE +#define pudp_establish pudp_establish +static inline pud_t pudp_establish(struct vm_area_struct *vma, + unsigned long address, pud_t *pudp, pud_t pud) +{ + page_table_check_pud_set(vma->vm_mm, pudp, pud); + return __pud(atomic_long_xchg((atomic_long_t *)pudp, pud_val(pud))); +} +#endif + /* * Encode/decode swap entries and swap PTEs. Swap PTEs are all PTEs that * are !pte_none() && !pte_present(). diff --git a/arch/x86/include/asm/pgtable.h b/arch/x86/include/asm/pgtable.h index 640edc31962f..572458a106e9 100644 --- a/arch/x86/include/asm/pgtable.h +++ b/arch/x86/include/asm/pgtable.h @@ -783,6 +783,12 @@ static inline pmd_t pmd_mkinvalid(pmd_t pmd) __pgprot(pmd_flags(pmd) & ~(_PAGE_PRESENT|_PAGE_PROTNONE))); } +static inline pud_t pud_mkinvalid(pud_t pud) +{ + return pfn_pud(pud_pfn(pud), + __pgprot(pud_flags(pud) & ~(_PAGE_PRESENT|_PAGE_PROTNONE))); +} + static inline u64 flip_protnone_guard(u64 oldval, u64 val, u64 mask); static inline pte_t pte_modify(pte_t pte, pgprot_t newprot) @@ -1353,6 +1359,23 @@ static inline pud_t pudp_huge_get_and_clear(struct mm_struct *mm, return pud; } +#ifndef pudp_establish +#define pudp_establish pudp_establish +static inline pud_t pudp_establish(struct vm_area_struct *vma, + unsigned long address, pud_t *pudp, pud_t pud) +{ + page_table_check_pud_set(vma->vm_mm, pudp, pud); + if (IS_ENABLED(CONFIG_SMP)) { + return xchg(pudp, pud); + } else { + pud_t old = *pudp; + + WRITE_ONCE(*pudp, pud); + return old; + } +} +#endif + #define __HAVE_ARCH_PMDP_SET_WRPROTECT static inline void pmdp_set_wrprotect(struct mm_struct *mm, unsigned long addr, pmd_t *pmdp) @@ -1389,7 +1412,6 @@ static inline pmd_t pmdp_establish(struct vm_area_struct *vma, #define __HAVE_ARCH_PMDP_INVALIDATE_AD extern pmd_t pmdp_invalidate_ad(struct vm_area_struct *vma, unsigned long address, pmd_t *pmdp); - /* * Page table pages are page-aligned. The lower half of the top * level is used for userspace and the top half for the kernel. @@ -1541,7 +1563,12 @@ static inline pte_t pte_swp_clear_soft_dirty(pte_t pte) return pte_clear_flags(pte, _PAGE_SWP_SOFT_DIRTY); } -#ifdef CONFIG_ARCH_ENABLE_THP_MIGRATION +#if defined(CONFIG_ARCH_ENABLE_THP_MIGRATION) || defined(CONFIG_HUGETLB_PAGE) +static inline int pud_swp_soft_dirty(pud_t pud) +{ + return pud_flags(pud) & _PAGE_SWP_SOFT_DIRTY; +} + static inline pmd_t pmd_swp_mksoft_dirty(pmd_t pmd) { return pmd_set_flags(pmd, _PAGE_SWP_SOFT_DIRTY); diff --git a/include/linux/pgtable.h b/include/linux/pgtable.h index a2e2ebb93f21..458e3cbc96b2 100644 --- a/include/linux/pgtable.h +++ b/include/linux/pgtable.h @@ -956,6 +956,11 @@ extern pmd_t pmdp_invalidate(struct vm_area_struct *vma, unsigned long address, pmd_t *pmdp); #endif +#ifndef __HAVE_ARCH_PUDP_INVALIDATE +extern pud_t pudp_invalidate(struct vm_area_struct *vma, unsigned long address, + pud_t *pudp); +#endif + #ifndef __HAVE_ARCH_PMDP_INVALIDATE_AD /* @@ -976,6 +981,26 @@ extern pmd_t pmdp_invalidate_ad(struct vm_area_struct *vma, unsigned long address, pmd_t *pmdp); #endif +#ifndef __HAVE_ARCH_PUDP_INVALIDATE_AD + +/* + * pudp_invalidate_ad() invalidates the PMD while changing a hugetlb mapping in + * the page tables. This function is similar to pudp_invalidate(), but should + * only be used if the access and dirty bits would not be cleared by the software + * in the new PUD value. The function ensures that hardware changes of the access + * and dirty bits updates would not be lost. + * + * Doing so can allow in certain architectures to avoid a TLB flush in most + * cases. Yet, another TLB flush might be necessary later if the PUD update + * itself requires such flush (e.g., if protection was set to be stricter). Yet, + * even when a TLB flush is needed because of the update, the caller may be able + * to batch these TLB flushing operations, so fewer TLB flush operations are + * needed. + */ +extern pud_t pudp_invalidate_ad(struct vm_area_struct *vma, + unsigned long address, pud_t *pudp); +#endif + #ifndef __HAVE_ARCH_PTE_SAME static inline int pte_same(pte_t pte_a, pte_t pte_b) { @@ -1406,7 +1431,16 @@ static inline pgprot_t pgprot_modify(pgprot_t oldprot, pgprot_t newprot) #endif #ifdef CONFIG_HAVE_ARCH_SOFT_DIRTY -#ifndef CONFIG_ARCH_ENABLE_THP_MIGRATION +static inline int pud_soft_dirty(pud_t pud) +{ + return 0; +} +#if !defined(CONFIG_ARCH_ENABLE_THP_MIGRATION) && !defined(CONFIG_HUGETLB_PAGE) +static inline int pud_swp_soft_dirty(pud_t pud) +{ + return 0; +} + static inline pmd_t pmd_swp_mksoft_dirty(pmd_t pmd) { return pmd; @@ -1487,6 +1521,11 @@ static inline pmd_t pmd_swp_clear_soft_dirty(pmd_t pmd) { return pmd; } + +static inline int pud_swp_soft_dirty(pud_t pud) +{ + return 0; +} #endif #ifndef __HAVE_PFNMAP_TRACKING diff --git a/mm/pgtable-generic.c b/mm/pgtable-generic.c index a78a4adf711a..e11ad8663903 100644 --- a/mm/pgtable-generic.c +++ b/mm/pgtable-generic.c @@ -194,6 +194,27 @@ pgtable_t pgtable_trans_huge_withdraw(struct mm_struct *mm, pmd_t *pmdp) } #endif +#ifndef __HAVE_ARCH_PUDP_INVALIDATE +pud_t pudp_invalidate(struct vm_area_struct *vma, unsigned long address, + pud_t *pudp) +{ + VM_WARN_ON_ONCE(!pud_present(*pudp)); + pud_t old = pudp_establish(vma, address, pudp, pud_mkinvalid(*pudp)); + + flush_pud_tlb_range(vma, address, address + PUD_SIZE); + return old; +} +#endif + +#ifndef __HAVE_ARCH_PUDP_INVALIDATE_AD +pud_t pudp_invalidate_ad(struct vm_area_struct *vma, unsigned long address, + pud_t *pudp) +{ + VM_WARN_ON_ONCE(!pud_present(*pudp)); + return pudp_invalidate(vma, address, pudp); +} +#endif + #ifndef __HAVE_ARCH_PMDP_INVALIDATE pmd_t pmdp_invalidate(struct vm_area_struct *vma, unsigned long address, pmd_t *pmdp) From patchwork Thu Jul 4 04:31:05 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oscar Salvador X-Patchwork-Id: 13723196 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 38458C30653 for ; Thu, 4 Jul 2024 04:32:44 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 96B846B00A7; Thu, 4 Jul 2024 00:32:16 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 8F4A56B00A8; Thu, 4 Jul 2024 00:32:16 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 748026B00A9; Thu, 4 Jul 2024 00:32:16 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 52B926B00A7 for ; Thu, 4 Jul 2024 00:32:16 -0400 (EDT) Received: from smtpin20.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 153A21C2667 for ; Thu, 4 Jul 2024 04:32:16 +0000 (UTC) X-FDA: 82300798272.20.C3D4381 Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.223.131]) by imf16.hostedemail.com (Postfix) with ESMTP id 02EF218001C for ; Thu, 4 Jul 2024 04:32:13 +0000 (UTC) Authentication-Results: imf16.hostedemail.com; dkim=none; spf=pass (imf16.hostedemail.com: domain of osalvador@suse.de designates 195.135.223.131 as permitted sender) smtp.mailfrom=osalvador@suse.de; dmarc=pass (policy=none) header.from=suse.de ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1720067515; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=jf9TaAMeyGiTtzlVo2uGSB7dEGDehBnqIYrJsOaHQk8=; b=ZEiMaGVpzi0n+v2m5Nz3YQcv6CEVBpJsAM4VLFJHU/y/o0YMoQCSjyqr3L/hhlfd0tw/f6 y5wkA12JMHQ6NbNHmFm5QjpniNeXjd/JJPXUk25sDVgW/T8LNoFM8Of+YUtPytkNqJagmm aOBCqSSbLyYS3zErFWI0DTPql4nowh0= ARC-Authentication-Results: i=1; imf16.hostedemail.com; dkim=none; spf=pass (imf16.hostedemail.com: domain of osalvador@suse.de designates 195.135.223.131 as permitted sender) smtp.mailfrom=osalvador@suse.de; dmarc=pass (policy=none) header.from=suse.de ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1720067515; a=rsa-sha256; cv=none; b=uhQH0O2qslzvn0j6GNnbTaxYoyCyOvIc23j6PtepOTBsT8QpmFCmkFFQIqY+2rsGszEiQx 2sY608GrCqQMzYdXxzxv7BV0IB//7Zq8NDCU5J0oo+Uclv6nngK/OiwoRSTnQ/g0W97/ld ls0WMg3/m4IIaoWejjB87XvPknCAD10= Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [IPv6:2a07:de40:b281:104:10:150:64:97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id BB4731FCFA; Thu, 4 Jul 2024 04:32:12 +0000 (UTC) Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 9738313889; Thu, 4 Jul 2024 04:32:11 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id gC+BH8slhmbnDAAAD6G6ig (envelope-from ); Thu, 04 Jul 2024 04:32:11 +0000 From: Oscar Salvador To: Andrew Morton Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Peter Xu , Muchun Song , David Hildenbrand , SeongJae Park , Miaohe Lin , Michal Hocko , Matthew Wilcox , Christophe Leroy , Oscar Salvador Subject: [PATCH 18/45] fs/proc: Create pagemap_scan_pud_entry to handle PUD-mapped hugetlb vmas Date: Thu, 4 Jul 2024 06:31:05 +0200 Message-ID: <20240704043132.28501-19-osalvador@suse.de> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240704043132.28501-1-osalvador@suse.de> References: <20240704043132.28501-1-osalvador@suse.de> MIME-Version: 1.0 X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Rspamd-Action: no action X-Stat-Signature: ejybwg1t6hghzdri5toja6w9uay5r6bd X-Rspam-User: X-Rspamd-Queue-Id: 02EF218001C X-Rspamd-Server: rspam02 X-HE-Tag: 1720067533-271832 X-HE-Meta: U2FsdGVkX18XncctQ9mWTHbCtWV87+UwxSoQD4NBY18xMwD8oyDMK5dIciddqFOOcfAtn2dHTQu/2lmdBPEq9mwHNMqjXbCT6ruUdLGQvhY9VcKW2LmDfvAlHrD/NOidLOPkBu2hBriAKtSnagYEU1ox26SkbRfYutTOUHASMTYndQI1UjZEnB/FEQirW7ftmIkltt/QLvA6IwfPKKB/iPd5NkCKmeJRcmiClOcS5zPrU5VqJHQ1/r31yyUES2wfpPMkYgmJZcavcBo4kThs2UCGlhlV8LtxJqLa6FIRKPHlhOfs8DjC2xY0Uztg9FhuqzipOhuGg1AL13KZhrXV832+2htpz3GaUWIaRfRmYHbcg3MPvYYSRpr4hW3J1akMMja4r02W3Dsf2cpKEEIHg/6EUGacB8xjf8LYKx12ynXz0hhIkeo34rJKRNBcd4w0W0vBJ+bgHE4d0m3U3uJJ9ydH6JsLmqTRioeTBZ/i16Hh1T0IN4o9asBBq9zaHUF+vbjzqnbRDS3RULasPDt2QDMFrqoq3aeYHkiO3E2kqBOvtY+xEWLDVUhb9PPzNV2sXNBL3wIA+HudfI6uukDlpPs0CVT+NaDLE6Xf4N4usmZnGzQefHDQZ5LOW/LYxPOo58lst0LhefxBrvVV+/E+aI/udcyipLnALJop4upSG2NgUMm0wHVQlwdl9/guYaMK8ALqmp392W28LidB5I4lJ8PbpgQBJabGaRmlmiBDR3nRenZEdxkEsM0GchZwOQo+U6cI+911m8gMRH/biSdzVcw8Uk3o5GHPo+ukO35vkMdOjgXrHmtniJ8G5TG8wiN5MWHFvot8PtA/gh2AoDHZiAxbtkNdNgwh/39i2El4gL9ORaQdDLAAXM9ounaAvkuw180mq9LL7tgZlr8IQd0FLpAzMq6Bgcg/ZyYgvkYXy/4TYmzshwG0cSWrEv9WUs+n+HbBY5vPS316Oe1r83F DFJZnq9i dMOzQlYrgJ5ly4wYbkvDz+9/AyowVL2B7bFzusTIS9Za7FT1maan447Uxgs895/BWZ5ZQqvkr7GGRaLc69FuLqR4rpU3rjoK2Is5KTYvvSw+j0aqaah3S3P/yZCweD4S99Pv/J6YsxiUghn6R1pLGxi6oomb3q6dmSbM4V4s/9Etay9E= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Normal THP cannot be PUD-mapped (besides devmap) but hugetlb can, so create pagemap_scan_pud_entry in order to handle PUD-mapped hugetlb vmas. Signed-off-by: Oscar Salvador --- fs/proc/task_mmu.c | 104 ++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 103 insertions(+), 1 deletion(-) diff --git a/fs/proc/task_mmu.c b/fs/proc/task_mmu.c index df649f69ea2c..3785a44b97fa 100644 --- a/fs/proc/task_mmu.c +++ b/fs/proc/task_mmu.c @@ -1925,7 +1925,7 @@ static int pagemap_pmd_range(pmd_t *pmdp, unsigned long addr, unsigned long end, return err; } -#ifdef CONFIG_HUGETLB_PAGE +#ifdef CONFIG_PGTABLE_HAS_HUGE_LEAVES static int pagemap_pud_range(pud_t *pudp, unsigned long addr, unsigned long end, struct mm_walk *walk) { @@ -2324,6 +2324,59 @@ static void make_uffd_wp_pmd(struct vm_area_struct *vma, } #endif /* CONFIG_PGTABLE_HAS_HUGE_LEAVES */ +#ifdef CONFIG_PGTABLE_HAS_HUGE_LEAVES +static unsigned long pagemap_pud_category(struct pagemap_scan_private *p, + struct vm_area_struct *vma, + unsigned long addr, pud_t pud) +{ + unsigned long categories = PAGE_IS_HUGE; + + if (pud_present(pud)) { + struct page *page; + + categories |= PAGE_IS_PRESENT; + if (!pud_uffd_wp(pud)) + categories |= PAGE_IS_WRITTEN; + + if (p->masks_of_interest & PAGE_IS_FILE) { + page = vm_normal_page_pud(vma, addr, pud); + if (page && !PageAnon(page)) + categories |= PAGE_IS_FILE; + } + + if (is_zero_pfn(pud_pfn(pud))) + categories |= PAGE_IS_PFNZERO; + if (pud_soft_dirty(pud)) + categories |= PAGE_IS_SOFT_DIRTY; + } else if (is_swap_pud(pud)) { + swp_entry_t swp; + + categories |= PAGE_IS_SWAPPED; + if (!pud_swp_uffd_wp(pud)) + categories |= PAGE_IS_WRITTEN; + if (pud_swp_soft_dirty(pud)) + categories |= PAGE_IS_SOFT_DIRTY; + } + + return categories; +} + +static void make_uffd_wp_pud(struct vm_area_struct *vma, + unsigned long addr, pud_t *pudp) +{ + pud_t old, pud = *pudp; + + if (pud_present(pud)) { + old = pudp_invalidate_ad(vma, addr, pudp); + pud = pud_mkuffd_wp(old); + set_pud_at(vma->vm_mm, addr, pudp, pud); + } else if (is_migration_entry(pud_to_swp_entry(pud))) { + pud = pud_swp_mkuffd_wp(pud); + set_pud_at(vma->vm_mm, addr, pudp, pud); + } +} +#endif /* CONFIG_PGTABLE_HAS_HUGE_LEAVES */ + #ifdef CONFIG_HUGETLB_PAGE static unsigned long pagemap_hugetlb_category(pte_t pte) { @@ -2685,6 +2738,54 @@ static int pagemap_scan_pmd_entry(pmd_t *pmd, unsigned long start, return ret; } +#ifdef CONFIG_HUGETLB_PAGE +static int pagemap_scan_pud_entry(pud_t *pud, unsigned long start, + unsigned long end, struct mm_walk *walk) +{ + int ret = 0; + spinlock_t *ptl; + unsigned long categories; + struct vm_area_struct *vma = walk->vma; + struct pagemap_scan_private *p = walk->private; + + /* Only PUD-mapped hugetlb can reach here at this moment */ + ptl = pud_huge_lock(pud, vma); + if (!ptl) + return 0; + + categories = p->cur_vma_category | + pagemap_pud_category(p, vma, start, *pud); + + if (!pagemap_scan_is_interesting_page(categories, p)) + goto out_unlock; + + ret = pagemap_scan_output(categories, p, start, &end); + if (start == end) + goto out_unlock; + + if (~p->arg.flags & PM_SCAN_WP_MATCHING) + goto out_unlock; + if (~categories & PAGE_IS_WRITTEN) + goto out_unlock; + + if (end != start + PUD_SIZE) { + ret = 0; + pagemap_scan_backout_range(p, start, end); + p->arg.walk_end = start; + goto out_unlock; + } + + make_uffd_wp_pud(vma, start, pud); + flush_hugetlb_tlb_range(vma, start, end); + +out_unlock: + spin_unlock(ptl); + return ret; +} +#else +#define pagemap_scan_pud_entry NULL +#endif + #ifdef CONFIG_HUGETLB_PAGE static int pagemap_scan_hugetlb_entry(pte_t *ptep, unsigned long hmask, unsigned long start, unsigned long end, @@ -2772,6 +2873,7 @@ static int pagemap_scan_pte_hole(unsigned long addr, unsigned long end, static const struct mm_walk_ops pagemap_scan_ops = { .test_walk = pagemap_scan_test_walk, + .pud_entry = pagemap_scan_pud_entry, .pmd_entry = pagemap_scan_pmd_entry, .pte_hole = pagemap_scan_pte_hole, .hugetlb_entry = pagemap_scan_hugetlb_entry, From patchwork Thu Jul 4 04:31:06 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oscar Salvador X-Patchwork-Id: 13723197 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 10858C3271F for ; Thu, 4 Jul 2024 04:32:47 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 0DF536B00A8; Thu, 4 Jul 2024 00:32:18 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 044326B00AA; Thu, 4 Jul 2024 00:32:17 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D61446B00AB; Thu, 4 Jul 2024 00:32:17 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id AD7716B00A8 for ; Thu, 4 Jul 2024 00:32:17 -0400 (EDT) Received: from smtpin26.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 677BC80B20 for ; Thu, 4 Jul 2024 04:32:17 +0000 (UTC) X-FDA: 82300798314.26.555F185 Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.223.131]) by imf25.hostedemail.com (Postfix) with ESMTP id 47084A0002 for ; Thu, 4 Jul 2024 04:32:15 +0000 (UTC) Authentication-Results: imf25.hostedemail.com; dkim=pass header.d=suse.de header.s=susede2_rsa header.b=j2uGj66F; dkim=pass header.d=suse.de header.s=susede2_ed25519 header.b=aF1AIMEQ; dkim=pass header.d=suse.de header.s=susede2_rsa header.b=j2uGj66F; dkim=pass header.d=suse.de header.s=susede2_ed25519 header.b=aF1AIMEQ; spf=pass (imf25.hostedemail.com: domain of osalvador@suse.de designates 195.135.223.131 as permitted sender) smtp.mailfrom=osalvador@suse.de; dmarc=pass (policy=none) header.from=suse.de ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1720067511; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=wUAXkDKulvUPvsNZqyUfVRpX873YTDqHqTEgZWFzyt4=; b=GcDTbsZWALaTRmMC40WIDM5kvGWj6+XeHBnFe+5lvHW+uuOHpYD+YWRHorTp3RZ/AEokq/ q5pSo7F5mGDpxwIeg2a9PXLne4aZ42sIj3bJ0sND0n/2us+SFyolOpeanRv2dXtq52u5Uv aDEpIxyqpklUmucL0NWGp7CyMX2weIU= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1720067511; a=rsa-sha256; cv=none; b=XZ5wYpJzBWBR4OVNvRN8lfPQ6cO2/xFPiOYBD1qIFug+UhPMFQqpPPIkzmjd4LDUXfq1AA fWLf+a2mRCVCD4YmClMYoaRuI1noSDkUUQWYRUHAnwbqx7aIAIEEcpn2LPhrpy7ZgWs7Nd O7sObQhHXxYTiWDGF2fwXzQi3W0RmRM= ARC-Authentication-Results: i=1; imf25.hostedemail.com; dkim=pass header.d=suse.de header.s=susede2_rsa header.b=j2uGj66F; dkim=pass header.d=suse.de header.s=susede2_ed25519 header.b=aF1AIMEQ; dkim=pass header.d=suse.de header.s=susede2_rsa header.b=j2uGj66F; dkim=pass header.d=suse.de header.s=susede2_ed25519 header.b=aF1AIMEQ; spf=pass (imf25.hostedemail.com: domain of osalvador@suse.de designates 195.135.223.131 as permitted sender) smtp.mailfrom=osalvador@suse.de; dmarc=pass (policy=none) header.from=suse.de Received: from imap1.dmz-prg2.suse.org (unknown [10.150.64.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id 053031FCFC; Thu, 4 Jul 2024 04:32:14 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1720067534; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=wUAXkDKulvUPvsNZqyUfVRpX873YTDqHqTEgZWFzyt4=; b=j2uGj66FrtGxFBiwFctbDQ4OTG+w9M73dH62olOmsLGZIhnPfy8i6BSi1M1uU7Z995dm0M rDy3fOSDZiLRMBRFvYSwMC3MWiBIZt60dh4cyu70ITZlu4RYz92tsp3lGRkhC2d9EZ2fgM G1/oiZiEwG0JggCZhXJlACHwMP0ZVFI= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1720067534; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=wUAXkDKulvUPvsNZqyUfVRpX873YTDqHqTEgZWFzyt4=; b=aF1AIMEQj6zRqtFUbpMf5dtjXnB1Crkf8BYEYuKhtpIYT+dX14+NYfft0LEejATVkvxiN5 /oB961ArpbntQiAw== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1720067534; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=wUAXkDKulvUPvsNZqyUfVRpX873YTDqHqTEgZWFzyt4=; b=j2uGj66FrtGxFBiwFctbDQ4OTG+w9M73dH62olOmsLGZIhnPfy8i6BSi1M1uU7Z995dm0M rDy3fOSDZiLRMBRFvYSwMC3MWiBIZt60dh4cyu70ITZlu4RYz92tsp3lGRkhC2d9EZ2fgM G1/oiZiEwG0JggCZhXJlACHwMP0ZVFI= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1720067534; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=wUAXkDKulvUPvsNZqyUfVRpX873YTDqHqTEgZWFzyt4=; b=aF1AIMEQj6zRqtFUbpMf5dtjXnB1Crkf8BYEYuKhtpIYT+dX14+NYfft0LEejATVkvxiN5 /oB961ArpbntQiAw== Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id DD98813889; Thu, 4 Jul 2024 04:32:12 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id WFxbMcwlhmbnDAAAD6G6ig (envelope-from ); Thu, 04 Jul 2024 04:32:12 +0000 From: Oscar Salvador To: Andrew Morton Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Peter Xu , Muchun Song , David Hildenbrand , SeongJae Park , Miaohe Lin , Michal Hocko , Matthew Wilcox , Christophe Leroy , Oscar Salvador Subject: [PATCH 19/45] fs/proc: Enable gather_pte_stats to handle hugetlb vmas Date: Thu, 4 Jul 2024 06:31:06 +0200 Message-ID: <20240704043132.28501-20-osalvador@suse.de> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240704043132.28501-1-osalvador@suse.de> References: <20240704043132.28501-1-osalvador@suse.de> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 47084A0002 X-Stat-Signature: ae4n74oxouqpjmg9y18by9qk7gzjf3su X-HE-Tag: 1720067535-194124 X-HE-Meta: U2FsdGVkX1/v0M43mP/cWPqyRnuXv3/GVkxVZRU+nQeHEu/bngpUkOFQ1ZMnWmFaCnSY2zWWv1CShnT1NY0rdRLUaiou9jdb4tidFBQG5APw9lR+5BF8+Qc9fD8PoNaqeVlEFLaKacndDtrcNHMZlUCur3Py9PrWgKR29BTBWdWULiQNrBg0AyDvUoVPmAwqFJGZPvf90DGSqckpxG/9Sle2EU8xWvFl7eExDODy273ujuasi6tIO9IVqRSFNyLnIeTSBRyCKCCMSY+NmkUAjMGkEJPqxCEX83ufEUNjGD4FDfPsJkTmIjGtaDoUhwOD4sOZeAlhNEKi044TMBZt0wK2Stnw32QRB/kgU0arTXYkEBJvkuemj1qHsHX2vY1SuXG/sJsUgU7KApCUX5vKm5xWl3HDNgmBm33X1J4ARbcRjV/r1ouj3Du4fG2pgSbGFzRQHG4HOpXLV5vo9PDkO55QP0pwPsSSVX+cV+F6NC9b9KROXVLnTunch5S/FJWQspjoWOfYzYQWYnPsuH7C7KIEHwmGB7Yd9L/E+f2/SPi3xUIakJC0I2XNr9yIlz60Wt1YkxQh8wLdRy/ZCMADYcV5FN76ot/1A+2sFNSnUlP/FBRKZK8XtAT/6wnxpEVTAIDDFbI1TJz9xE5haKw6yA9VfBZE00BKg0J5/+qBINtT1/fPxZAAtpIhosHJ9/YLfLlDfwaOXkJWc1uptJhCxzPaB6aF8PFPYNVMXEZJoTbzeGmjo3vEdR2DSjpDkFK0zhyrxExb+RuF8cSD/FDqy8bQKUfqrIYPXdlWScBHijHll93MPMNl7njEYCbehpM3IjhCSUQxfkwSjPNe7Bh09kJx2LQHy09bEP6U6rGkRR7fYvlXb7erOxMVuHemxH/eHTreHaVC8GHj1eDyVtwYmznD+BxWmEnTMzYll/KdNQrUpa/bwZDm6rdhw3VvhIWynvgazjiMDWsUcDJhXwq qEdphoTr ldLchgpUrygS7onxrWgcOislpX1cajZbk5dotL/LFIREXVGJPdlSKkhHQWhTvs/SFGhj3j/GITRcjyErVhZO8yWG1UZm8rdTm1E7Pc2f69YdxL1WCIlyYb5dhGYCowzPKG4lmGaMxVvXUBRrweAFz7UjRQ6YJ27icsC8dzw+sJ5+BScNPp5kIjzIYdAzxbPELCtEVbRLRVEYej5F/nufTpsMqMnJkifRUCrlym9iHuIgzGGvT/YU9d+rcxp5fhaUbozZCIqGd4ulPRzP55fykSed4hHJziP4iEcw5 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: PMD-mapped hugetlb vmas will also reach gather_pte_stats. Add the required code so it knows how to handle those there. Signed-off-by: Oscar Salvador --- fs/proc/task_mmu.c | 14 ++++++++++---- 1 file changed, 10 insertions(+), 4 deletions(-) diff --git a/fs/proc/task_mmu.c b/fs/proc/task_mmu.c index 3785a44b97fa..e13754d3246e 100644 --- a/fs/proc/task_mmu.c +++ b/fs/proc/task_mmu.c @@ -3141,7 +3141,7 @@ static struct page *can_gather_numa_stats(pte_t pte, struct vm_area_struct *vma, return page; } -#ifdef CONFIG_TRANSPARENT_HUGEPAGE +#ifdef CONFIG_PGTABLE_HAS_HUGE_LEAVES static struct page *can_gather_numa_stats_pmd(pmd_t pmd, struct vm_area_struct *vma, unsigned long addr) @@ -3176,15 +3176,21 @@ static int gather_pte_stats(pmd_t *pmd, unsigned long addr, pte_t *orig_pte; pte_t *pte; -#ifdef CONFIG_TRANSPARENT_HUGEPAGE - ptl = pmd_trans_huge_lock(pmd, vma); +#ifdef CONFIG_PGTABLE_HAS_HUGE_LEAVES + ptl = pmd_huge_lock(pmd, vma); if (ptl) { + unsigned long nr_pages; struct page *page; + if (is_vm_hugetlb_page(vma)) + nr_pages = 1; + else + nr_pages = HPAGE_PMD_SIZE / PAGE_SIZE; + page = can_gather_numa_stats_pmd(*pmd, vma, addr); if (page) gather_stats(page, md, pmd_dirty(*pmd), - HPAGE_PMD_SIZE/PAGE_SIZE); + nr_pages); spin_unlock(ptl); return 0; } From patchwork Thu Jul 4 04:31:07 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oscar Salvador X-Patchwork-Id: 13723198 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id A34E7C30653 for ; Thu, 4 Jul 2024 04:32:49 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 379116B00AB; Thu, 4 Jul 2024 00:32:19 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 2D8276B00AC; Thu, 4 Jul 2024 00:32:19 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 0DE266B00AD; Thu, 4 Jul 2024 00:32:19 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id DD1706B00AB for ; Thu, 4 Jul 2024 00:32:18 -0400 (EDT) Received: from smtpin20.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id A30EE140A71 for ; Thu, 4 Jul 2024 04:32:18 +0000 (UTC) X-FDA: 82300798356.20.1E984C7 Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.223.131]) by imf10.hostedemail.com (Postfix) with ESMTP id 90496C000F for ; Thu, 4 Jul 2024 04:32:16 +0000 (UTC) Authentication-Results: imf10.hostedemail.com; dkim=pass header.d=suse.de header.s=susede2_rsa header.b=Sp75Awte; dkim=pass header.d=suse.de header.s=susede2_ed25519 header.b=Mj+n3kpv; dkim=pass header.d=suse.de header.s=susede2_rsa header.b=Sp75Awte; dkim=pass header.d=suse.de header.s=susede2_ed25519 header.b=Mj+n3kpv; dmarc=pass (policy=none) header.from=suse.de; spf=pass (imf10.hostedemail.com: domain of osalvador@suse.de designates 195.135.223.131 as permitted sender) smtp.mailfrom=osalvador@suse.de ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1720067516; a=rsa-sha256; cv=none; b=bvFGBIcBDrlu7G08HVb8rF7h+i3KWIwgbw21BJaFAMWkaNQfi6VxsMaCKPlY2apPFdP81n U/yR5rDAgOfMUSsvToQn7zwupwLDJ0tDG0otYK6JhEr8iDFTf0NySfRmvKAimKQ52LJ/JP 7fpb2CmUm9FKsWio/lX4yHffz8mAYt4= ARC-Authentication-Results: i=1; imf10.hostedemail.com; dkim=pass header.d=suse.de header.s=susede2_rsa header.b=Sp75Awte; dkim=pass header.d=suse.de header.s=susede2_ed25519 header.b=Mj+n3kpv; dkim=pass header.d=suse.de header.s=susede2_rsa header.b=Sp75Awte; dkim=pass header.d=suse.de header.s=susede2_ed25519 header.b=Mj+n3kpv; dmarc=pass (policy=none) header.from=suse.de; spf=pass (imf10.hostedemail.com: domain of osalvador@suse.de designates 195.135.223.131 as permitted sender) smtp.mailfrom=osalvador@suse.de ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1720067516; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=vSKmwPU7BkbGtKq84iiMlBZLjbm/8YgbxNgKPRpjw3o=; b=07icD0LJMZX6ePBY551PhoueZwaNBMWoDwdHCDtNsHC1u71m5in6eJUvYl13ITbSzVyflr YSikTZNij3WuRD5fFTJzrv6pETuATuReKlayWLx3eqyHOiEWZ9nvIwgKcDoSTy9bbLnQbq Z1GVKye4iGc18vPzobJDuF87zBMO0/k= Received: from imap1.dmz-prg2.suse.org (unknown [10.150.64.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id 487F31FCF9; Thu, 4 Jul 2024 04:32:15 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1720067535; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=vSKmwPU7BkbGtKq84iiMlBZLjbm/8YgbxNgKPRpjw3o=; b=Sp75Awtev0kxk3LPqK9XxrdT8mVKHhhRuRIBMOFdY8fEHNVHNNZ06aTT5CglZTRF8npqNz 2Yy3uyOucP4OPr8Nv3JT4zfkut2bNZa/kPSEfP3yjJrNw6aJH7PjRQ35FXbRRBUM/Hdhb5 qpJvjTmjFlI704iRP7Ya2VX/BQVd3/k= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1720067535; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=vSKmwPU7BkbGtKq84iiMlBZLjbm/8YgbxNgKPRpjw3o=; b=Mj+n3kpvgdn1kni7UbDFyJvbmlmLCXttzKSC9JGDxhxjtdRvE8XYhxKLUr9QMKGLWWztko A652jlHgiyBk0NDg== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1720067535; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=vSKmwPU7BkbGtKq84iiMlBZLjbm/8YgbxNgKPRpjw3o=; b=Sp75Awtev0kxk3LPqK9XxrdT8mVKHhhRuRIBMOFdY8fEHNVHNNZ06aTT5CglZTRF8npqNz 2Yy3uyOucP4OPr8Nv3JT4zfkut2bNZa/kPSEfP3yjJrNw6aJH7PjRQ35FXbRRBUM/Hdhb5 qpJvjTmjFlI704iRP7Ya2VX/BQVd3/k= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1720067535; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=vSKmwPU7BkbGtKq84iiMlBZLjbm/8YgbxNgKPRpjw3o=; b=Mj+n3kpvgdn1kni7UbDFyJvbmlmLCXttzKSC9JGDxhxjtdRvE8XYhxKLUr9QMKGLWWztko A652jlHgiyBk0NDg== Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 28FA913889; Thu, 4 Jul 2024 04:32:14 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id uClaBc4lhmbnDAAAD6G6ig (envelope-from ); Thu, 04 Jul 2024 04:32:14 +0000 From: Oscar Salvador To: Andrew Morton Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Peter Xu , Muchun Song , David Hildenbrand , SeongJae Park , Miaohe Lin , Michal Hocko , Matthew Wilcox , Christophe Leroy , Oscar Salvador Subject: [PATCH 20/45] fs/proc: Enable gather_pte_stats to handle cont-pte mapped hugetlb vmas Date: Thu, 4 Jul 2024 06:31:07 +0200 Message-ID: <20240704043132.28501-21-osalvador@suse.de> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240704043132.28501-1-osalvador@suse.de> References: <20240704043132.28501-1-osalvador@suse.de> MIME-Version: 1.0 X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: 90496C000F X-Stat-Signature: bux67qmmut66eud4zxayky4mhsjsuncp X-Rspam-User: X-HE-Tag: 1720067536-633005 X-HE-Meta: U2FsdGVkX1+EyOe0yLeoKqjbiFfkkCXMKRAGaF9ESn8p4rqfD1cpny2c9r4YGfNrk9Xr0Ho5dGZ195REIKy0oy8NK7AxuDipCs/NhoH3wOh5zbTwKLqJHya6OFOmxy8dDTYEIR8EYbCq9g0X7rEAgM4sfVAkB/pTOLFT5i1y7Xfq6h0NQVgHdhLXAYSDBfw8x+0B73tY90f8vScsWrhwyF1rKwPI98rQl3o9twhpnraF6GUskbtLUZj/KVuMZAOzRrBN67YtHm1XK8ez6YmNy3dkGEqjzYzoKfXtenM7X9fc4jNbcVMKtu08d2Q1Qup1xXs9a7Lgd0yQIC4kNC0hgQUuS1UIcenlfRqk/7ibgrzskwLxKdguf6WHV3B5ZZQr9PCspSswCN1OrCf3rF6hy/mj/QFjsNIXt9RbOP73CuFiQXzFSelj+Ob2rLu5z0TYZfmH9VCJxao+JRRv3y2ZDXwxBVlt+/zQzaI5StL3IIZbJCwPRfpzs89tNyPG9WmRNTLJCfS+L7udeP2f3L85CCxzO5ScNBQH5QrK9nI7ebF7eR7J+tl5/nhYAmtiiwmC/kWB57PbQU6SpM7llK6a4Zp0pRfWiDtDkL+8BXFgfswkwoYGFgP9g5dGyL2FZCudA0dzX6fFOMkBdnMWjjQJkFanhI98KQMomxPO3M5aa7mpq69v6QeefNliBkLyUBtHLcKR1NJV1Fl0RCHATbYBbhhtk7W8MF6I4Qfb9b1lBAYU3hw6Y+H/Dhjv5BJWS9bDLa52eZ0qTOLWfNeaTuwpVrQWH7xJM2ePIDbq4N1U1rQpcyJ4XZIg+FEuB5QVbaT6CfSJONWUI+Opl8gKCXPSSRPBOrbJ34TzqgXaDYJu2DztCIUQaTVlPm/N6SZMzTXoI6t846acXeJwv3KiVC59XGGRG1s69GiqgXizBFIeVJX9H/ke4YabVmALIYihXIqAPa7QxyQSl4pK7RhYs5O L/HH93wN yvS40dcgRvJ+NkToFo6pqvbkcwPnki0pswIWrgjFUeHae4OL+Jx0cbnGaDXTC/rok5h/mjEhdlCahP8DJ1z/9EoeS+IvVXKTotwUlYl8RWB3MTORl9ceXyWiXinLRNwLbRAWbHHVGghhRDGu8eG1Mek6sRTty3BNggyK06DrgPEO/AflqJ2UVh1RXv2EZ+wZifTkyOsiXrPMl9WEnv0U2A0TYO4Ti5craFbf2ouI5Vv0+z8fkw22YC+vMXTayP3+4VnLPAuRHVdefupOzwdTRwpj6Yylz57B9KmQB X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: HugeTLB pages can be cont-pte mapped, so teach smaps_pte_entry to handle them. Signed-off-by: Oscar Salvador --- fs/proc/task_mmu.c | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/fs/proc/task_mmu.c b/fs/proc/task_mmu.c index e13754d3246e..98dd03c26e68 100644 --- a/fs/proc/task_mmu.c +++ b/fs/proc/task_mmu.c @@ -3175,6 +3175,7 @@ static int gather_pte_stats(pmd_t *pmd, unsigned long addr, spinlock_t *ptl; pte_t *orig_pte; pte_t *pte; + unsigned long size = PAGE_SIZE, cont_ptes = 1; #ifdef CONFIG_PGTABLE_HAS_HUGE_LEAVES ptl = pmd_huge_lock(pmd, vma); @@ -3200,6 +3201,10 @@ static int gather_pte_stats(pmd_t *pmd, unsigned long addr, walk->action = ACTION_AGAIN; return 0; } + if (pte_cont(ptep_get(pte))) { + size = PAGE_SIZE * CONT_PTES; + cont_ptes = CONT_PTES; + } do { pte_t ptent = ptep_get(pte); struct page *page = can_gather_numa_stats(ptent, vma, addr); @@ -3207,7 +3212,7 @@ static int gather_pte_stats(pmd_t *pmd, unsigned long addr, continue; gather_stats(page, md, pte_dirty(ptent), 1); - } while (pte++, addr += PAGE_SIZE, addr != end); + } while (pte += cont_ptes, addr += size, addr != end); pte_unmap_unlock(orig_pte, ptl); cond_resched(); return 0; From patchwork Thu Jul 4 04:31:08 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oscar Salvador X-Patchwork-Id: 13723199 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5814CC3065C for ; Thu, 4 Jul 2024 04:32:52 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id B68056B00AC; Thu, 4 Jul 2024 00:32:20 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id AEFFB6B00AD; Thu, 4 Jul 2024 00:32:20 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 968F16B00AE; Thu, 4 Jul 2024 00:32:20 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 7011C6B00AC for ; Thu, 4 Jul 2024 00:32:20 -0400 (EDT) Received: from smtpin05.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 0B024120C93 for ; Thu, 4 Jul 2024 04:32:20 +0000 (UTC) X-FDA: 82300798440.05.E12A6A3 Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.223.131]) by imf05.hostedemail.com (Postfix) with ESMTP id DA645100018 for ; Thu, 4 Jul 2024 04:32:17 +0000 (UTC) Authentication-Results: imf05.hostedemail.com; dkim=none; spf=pass (imf05.hostedemail.com: domain of osalvador@suse.de designates 195.135.223.131 as permitted sender) smtp.mailfrom=osalvador@suse.de; dmarc=pass (policy=none) header.from=suse.de ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1720067519; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=fCuliDJCz3L6Q0kfsxcXAf+XqJA59bLxL2NmRDKv3LA=; b=lzoZy+QIC/exmStIDrlX9HSCPZig9xAFD9XNfpaAvyzT8Qr5/6KolKSCLYPfPZESKY9lbl y1fJ2hXoX3nTx4GttO6BzQ0G28Ba5Ur2BN/86/Ri82o6o8mRRUTr6TWEFFhruApym6pt4T Y3pw77iZi08rTjPQCwGGxt6YDjtSpok= ARC-Authentication-Results: i=1; imf05.hostedemail.com; dkim=none; spf=pass (imf05.hostedemail.com: domain of osalvador@suse.de designates 195.135.223.131 as permitted sender) smtp.mailfrom=osalvador@suse.de; dmarc=pass (policy=none) header.from=suse.de ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1720067519; a=rsa-sha256; cv=none; b=K/mHDZPxopnasayEieBMKcVzAELiUzq5PHW5i7i08nDhFi4OWcvC/qsQwdhpoZsuVQkOn2 Cvix1sDGDzhMnPmeducNoIvye7B53YYfDtLeZY3v+SzTbITnfhNHgqbDxdJpsjRijy16GS ogP2/7k6H+HKRFNdCUnIhH0m1EsLVVk= Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [IPv6:2a07:de40:b281:104:10:150:64:97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id A8ED61FCF7; Thu, 4 Jul 2024 04:32:16 +0000 (UTC) Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 6D23B13889; Thu, 4 Jul 2024 04:32:15 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id 2KYBFs8lhmbnDAAAD6G6ig (envelope-from ); Thu, 04 Jul 2024 04:32:15 +0000 From: Oscar Salvador To: Andrew Morton Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Peter Xu , Muchun Song , David Hildenbrand , SeongJae Park , Miaohe Lin , Michal Hocko , Matthew Wilcox , Christophe Leroy , Oscar Salvador Subject: [PATCH 21/45] fs/proc: Create gather_pud_stats to handle PUD-mapped hugetlb pages Date: Thu, 4 Jul 2024 06:31:08 +0200 Message-ID: <20240704043132.28501-22-osalvador@suse.de> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240704043132.28501-1-osalvador@suse.de> References: <20240704043132.28501-1-osalvador@suse.de> MIME-Version: 1.0 X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Rspamd-Action: no action X-Stat-Signature: ci9n6ggn85bgc87dtg1akhibw5ejgzka X-Rspam-User: X-Rspamd-Queue-Id: DA645100018 X-Rspamd-Server: rspam02 X-HE-Tag: 1720067537-498064 X-HE-Meta: U2FsdGVkX19/97kAf3cNHFbd3KbAoTo9nMmMPTeZrkHsi9m1giekCWlMkRLXwugHsHLO5OBvWtOfSKFtN3BmgbJx/C8sdqADnyRSQ1cd9I3bBP22FEmx6P0Hlq8kIEk7C0GpZULv7kuuSza6gUPj7kBmYK6RSW+LFgI0qpp6EEbPP+sfNHjYXPA2Tk0nmAvyuf7BQ1WzM7xixuXCCuvEEZgDa99zU6jnswy6XkqoEpG5s6G8qnF8g4taaEb7UYY38Wk2cyxX9UxDEIZr24K0kcO2KHjXGkcU1n8e/70Hpyq7jbcZ/QtiavibJF+Q2YmGq7xuMALlRijcGdrxTd6uAcdFQ92IWPpbFkuEItwkF9LIdbd0R/oZIS+XmH316tEAWwXqImLYvmAYlwYNgVieUFITAp4PweavtWxDOLxWQQY/80PKsHCNSSqSMLHKk7bTweSLmitV1uWc/21PySamAidw2BrWUqc6dP4eaVLFkwCFP+F7n9U7rCLTkw51hx3/L9x+vjdzYBkbESMjeCky/+8hs6J6QtqXxJQbr4vF2/F6sJAvAxDvIO/TJEnNa+HVP0t5E8GmrNiwkgkCxcrjxT0AwWxz9NIKgWLQWlQlBn6qqDf2dC2JsGMIKctzaBNxNEmr1O4tD6uz8JtEbNBRdV7wuRd5QooIPTUuZufJ5s+Ocd8fIuAcZyJqKZFAQHXeM0mAxeT7kUJFMLoL07B3eJHQx5XpL2QzSPJut58amu+t3F7uAVrbwGzeG0NcYw3qjma5rPngPHZnXVltpharq5PvUgdm6D46wn0YaWi3bqXRCp2pGkTIqP+q7vu3+pMSde9o06be6HKkP6VAexLnJkxTx+YIyD3YJ8xj09bAKCSBv0jXZzXo1YItcftZOkBmDMBbduUohuhNvwDN+NQKVLeHBIfyKCQIy+O+9LhKprAGuIzTmYDC2VNcfoS+5REqe1MxHn05gdzVLuPJNfc 4Qvn4IIY +6AfVBPQWPow22NuERBPi1Xjidx3Z9CpbyPO2nT/A+8pcJu6lpUvBoRo7Xf4X1ZKITGu+9XNr+HYp8aV3yKLAA5ef357eb7aMW1fuTj9r5+RHlqFobTnZLLKT8Ggod/jgT17AsNaRltzGPUOhmK3WN0RJzrDSMC8Gp9wQAt3PgRHVHpA= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Normal THP cannot be PUD-mapped (besides devmap), but hugetlb can, so create gather_pud_stats in order to handle PUD-mapped hugetlb vmas. Also implement can_gather_numa_stats_pud which is the pud version of can_gather_numa_stats_pmd. Signed-off-by: Oscar Salvador --- arch/arm64/include/asm/pgtable.h | 1 + fs/proc/task_mmu.c | 56 ++++++++++++++++++++++++++++++++ 2 files changed, 57 insertions(+) diff --git a/arch/arm64/include/asm/pgtable.h b/arch/arm64/include/asm/pgtable.h index 5e26e63b1012..1a6b8be2f0d0 100644 --- a/arch/arm64/include/asm/pgtable.h +++ b/arch/arm64/include/asm/pgtable.h @@ -590,6 +590,7 @@ static inline pmd_t pmd_mkdevmap(pmd_t pmd) #define pfn_pmd(pfn,prot) __pmd(__phys_to_pmd_val((phys_addr_t)(pfn) << PAGE_SHIFT) | pgprot_val(prot)) #define mk_pmd(page,prot) pfn_pmd(page_to_pfn(page),prot) +#define pud_dirty(pud) pte_dirty(pud_pte(pud)) #define pud_young(pud) pte_young(pud_pte(pud)) #define pud_mkyoung(pud) pte_pud(pte_mkyoung(pud_pte(pud))) #define pud_write(pud) pte_write(pud_pte(pud)) diff --git a/fs/proc/task_mmu.c b/fs/proc/task_mmu.c index 98dd03c26e68..5df17b7cfe6c 100644 --- a/fs/proc/task_mmu.c +++ b/fs/proc/task_mmu.c @@ -3141,6 +3141,61 @@ static struct page *can_gather_numa_stats(pte_t pte, struct vm_area_struct *vma, return page; } +#ifdef CONFIG_HUGETLB_PAGE +static struct page *can_gather_numa_stats_pud(pud_t pud, + struct vm_area_struct *vma, + unsigned long addr) +{ + struct page *page; + int nid; + + if (!pud_present(pud)) + return NULL; + + page = pud_page(pud); + if (!page) + return NULL; + + if (PageReserved(page)) + return NULL; + + nid = page_to_nid(page); + if (!node_isset(nid, node_states[N_MEMORY])) + return NULL; + + return page; +} + +static int gather_pud_stats(pud_t *pud, unsigned long addr, + unsigned long end, struct mm_walk *walk) +{ + spinlock_t *ptl; + struct page *page; + unsigned long nr_pages; + struct numa_maps *md = walk->private; + struct vm_area_struct *vma = walk->vma; + + ptl = pud_huge_lock(pud, vma); + if (!ptl) + return 0; + + if (is_vm_hugetlb_page(vma)) + nr_pages = 1; + else + nr_pages = HPAGE_PUD_SIZE / PAGE_SIZE; + + page = can_gather_numa_stats_pud(*pud, vma, addr); + if (page) + gather_stats(page, md, pud_dirty(*pud), + nr_pages); + + spin_unlock(ptl); + return 0; +} +#else +#define gather_pud_stats NULL +#endif + #ifdef CONFIG_PGTABLE_HAS_HUGE_LEAVES static struct page *can_gather_numa_stats_pmd(pmd_t pmd, struct vm_area_struct *vma, @@ -3245,6 +3300,7 @@ static int gather_hugetlb_stats(pte_t *pte, unsigned long hmask, static const struct mm_walk_ops show_numa_ops = { .hugetlb_entry = gather_hugetlb_stats, + .pud_entry = gather_pud_stats, .pmd_entry = gather_pte_stats, .walk_lock = PGWALK_RDLOCK, }; From patchwork Thu Jul 4 04:31:09 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oscar Salvador X-Patchwork-Id: 13723200 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id CC58CC30653 for ; Thu, 4 Jul 2024 04:32:54 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id E28046B00AD; Thu, 4 Jul 2024 00:32:21 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id D86206B00AE; Thu, 4 Jul 2024 00:32:21 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id AEEED6B00AF; Thu, 4 Jul 2024 00:32:21 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 853DE6B00AD for ; Thu, 4 Jul 2024 00:32:21 -0400 (EDT) Received: from smtpin04.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 3EB6DA0FC5 for ; Thu, 4 Jul 2024 04:32:21 +0000 (UTC) X-FDA: 82300798482.04.571AD1E Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.223.131]) by imf29.hostedemail.com (Postfix) with ESMTP id 2447C120014 for ; Thu, 4 Jul 2024 04:32:18 +0000 (UTC) Authentication-Results: imf29.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=suse.de; spf=pass (imf29.hostedemail.com: domain of osalvador@suse.de designates 195.135.223.131 as permitted sender) smtp.mailfrom=osalvador@suse.de ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1720067519; a=rsa-sha256; cv=none; b=2b6fZyml7NswzHHh5GSfO3/VKsHBVrDHjcTBjdr1WodnpJzlbAlEYc8/bkke95xY++kDyH 1PQ2Q91MhdzJY163ow6ngYVmZc929u4mLdhQY0GW0k8tsqrYh0x6XmMgGmJvNb5ZNDb6db nAIT0o2zbvn7kZUBi7qyo2T8gBazCE4= ARC-Authentication-Results: i=1; imf29.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=suse.de; spf=pass (imf29.hostedemail.com: domain of osalvador@suse.de designates 195.135.223.131 as permitted sender) smtp.mailfrom=osalvador@suse.de ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1720067519; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Uzs8V2oOmleXqErej25OEJn8fB6VvJqW1r2F1UQZ8IY=; b=So6ICzJITOkOEqeevkNWXDwF86lelWEsHa5Io/WpqtiraFpHtRJB9oKhiLTvxOJn39z444 Z7DRhnYHVMtahQNYyM278uGbqbBHiOzzv1+HL2cmLwMX5eCrDnmq76EIcpDL7SpjZSf9x7 rFifE7q/gGPSOCTE6YE9IgohS8OdX1o= Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [IPv6:2a07:de40:b281:104:10:150:64:97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id DB2C41FCFD; Thu, 4 Jul 2024 04:32:17 +0000 (UTC) Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id C2D9D13889; Thu, 4 Jul 2024 04:32:16 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id iBXOKtAlhmbnDAAAD6G6ig (envelope-from ); Thu, 04 Jul 2024 04:32:16 +0000 From: Oscar Salvador To: Andrew Morton Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Peter Xu , Muchun Song , David Hildenbrand , SeongJae Park , Miaohe Lin , Michal Hocko , Matthew Wilcox , Christophe Leroy , Oscar Salvador Subject: [PATCH 22/45] mm/mempolicy: Enable queue_folios_pmd to handle hugetlb vmas Date: Thu, 4 Jul 2024 06:31:09 +0200 Message-ID: <20240704043132.28501-23-osalvador@suse.de> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240704043132.28501-1-osalvador@suse.de> References: <20240704043132.28501-1-osalvador@suse.de> MIME-Version: 1.0 X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Rspamd-Action: no action X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: 2447C120014 X-Stat-Signature: rb65t377f4owhkck5d9kdsgdg3wgxujz X-Rspam-User: X-HE-Tag: 1720067538-231927 X-HE-Meta: U2FsdGVkX192cj9jR4RLgEnekYX1y1NIgnxbhCMXl0ZTWtahW1B4tQDv2KLJ9HTuiUmkr9xxxIuhDQ0JtbkK0UVfwck/I7H07edkGwT1Kx52xEG0rOZyMBgpHC4QS0qjI/344jnb9tXyBVoiVG5t2x98yLJcdGoPXLcnp8VGJiuf+ben/Zsr4mlunjG5rFVUvGl7qI28yHF6QjwsICwdrcnCJ7TXNRqwgz1vBlYEtiGe9yYdkzJ6t5Z2ipx4IGV7qzbfvq5HgR4mvMCsH/3BGTMuH+/qD0+NkVVNu9sROpm8CrbMhJsLsK00NHtAtIBDsUoE4GRl+NGwAyEmx/n9oUKq5bFR7UtQuMIe8dnfy3UPFtd1Z45CxqVoi4/s1SVnoeW1MSXFdfUqEr9eZenmgg/77AxywHtqepNVOOSYxlTwtdYTzuMTwQadLrJmP1yVExmx85WxV0w03bJ2s9CcjaK1qhdeBvPilqSZRZ82KUmr60WqfxhCpdHWWvUPcTO19YcdDWDyhi4pivvuKIi2fAe/m5YHSTizaHGgNcdT4z7h0WQ3iaqQh6NOuRsdrEGdeqzvulC2I4KS2evQsQ5EMwNblk6o41AfAid0tQNOaMEdNN/oQZMO++ZSMyaZSRsvca0ck+0T3+i6kaXwxd905OnBOj2js4sb7rpCgMV52NFwkf76+UYFrsxptTx7+cDNuOB9qDAsm7m8xWOKZaw2XgjIfwf6fEfMXc9Uh+/utrDgG8XNnzeNz3els9hZSxPyX++KOjmhL7TsOBe0H7kqqfjZ0prhkhIYRDWBwen77ZUlblKL5vX4T1r18Zw5WJwJpKQnbR8oYCUKEij2fXf58txKwIVoPRtE+kxqpPZZdr6WO57esHqdFTpZMIxvwud5NkOOhnnMFwpnlKmnlNlQR7DcQJ8akVgrXqVpK2cthBSmXpBKEl8kAmcI+6X7ZYNQHx1bsrRO8Pvssf+mtdb Feydtx0d e62Yh2n2JY94pwL1zl5/P1mdlfqnI/fFceimtXJbEP4YxUqxQVv4NMNdIT9M85kQsoEN0gdINNI2G4KJliDS1KIcjW1UFV1l/wsacqJqTZsvFQ5XJJlTG+1IcmaB6pmfMXB+S/XlQnAUkeibDMdoTQLgD+PFi7kKOF98ZdP7APmr5tiM= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: PMD-mapped hugetlb vmas will also reach smaps_pmd_entry. Add the required code so it knows how to handle those there. Signed-off-by: Oscar Salvador --- include/linux/mm_inline.h | 7 +++++++ mm/mempolicy.c | 42 ++++++++++++++++++++++++--------------- 2 files changed, 33 insertions(+), 16 deletions(-) diff --git a/include/linux/mm_inline.h b/include/linux/mm_inline.h index 93e3eb86ef4e..521a001429d2 100644 --- a/include/linux/mm_inline.h +++ b/include/linux/mm_inline.h @@ -591,6 +591,13 @@ static inline bool vma_has_recency(struct vm_area_struct *vma) return true; } +static inline bool is_shared_pmd(pmd_t *pmd, struct vm_area_struct *vma) +{ + if (!is_vm_hugetlb_page(vma)) + return false; + return hugetlb_pmd_shared((pte_t *)pmd); +} + static inline spinlock_t *pmd_huge_lock(pmd_t *pmd, struct vm_area_struct *vma) { spinlock_t *ptl; diff --git a/mm/mempolicy.c b/mm/mempolicy.c index f8703feb68b7..5baf29da198c 100644 --- a/mm/mempolicy.c +++ b/mm/mempolicy.c @@ -455,7 +455,8 @@ static const struct mempolicy_operations mpol_ops[MPOL_MAX] = { }; static bool migrate_folio_add(struct folio *folio, struct list_head *foliolist, - unsigned long flags); + unsigned long flags, struct vm_area_struct *vma, + bool shared); static nodemask_t *policy_nodemask(gfp_t gfp, struct mempolicy *pol, pgoff_t ilx, int *nid); @@ -518,7 +519,8 @@ static void queue_folios_pmd(pmd_t *pmd, struct mm_walk *walk) return; if (!(qp->flags & (MPOL_MF_MOVE | MPOL_MF_MOVE_ALL)) || !vma_migratable(walk->vma) || - !migrate_folio_add(folio, qp->pagelist, qp->flags)) + !migrate_folio_add(folio, qp->pagelist, qp->flags, walk->vma, + is_shared_pmd(pmd, walk->vma))) qp->nr_failed++; } @@ -543,7 +545,7 @@ static int queue_folios_pte_range(pmd_t *pmd, unsigned long addr, pte_t ptent; spinlock_t *ptl; - ptl = pmd_trans_huge_lock(pmd, vma); + ptl = pmd_huge_lock(pmd, vma); if (ptl) { queue_folios_pmd(pmd, walk); spin_unlock(ptl); @@ -598,7 +600,7 @@ static int queue_folios_pte_range(pmd_t *pmd, unsigned long addr, } if (!(flags & (MPOL_MF_MOVE | MPOL_MF_MOVE_ALL)) || !vma_migratable(vma) || - !migrate_folio_add(folio, qp->pagelist, flags)) { + !migrate_folio_add(folio, qp->pagelist, flags, vma, false)) { qp->nr_failed++; if (strictly_unmovable(flags)) break; @@ -1025,8 +1027,11 @@ static long do_get_mempolicy(int *policy, nodemask_t *nmask, #ifdef CONFIG_MIGRATION static bool migrate_folio_add(struct folio *folio, struct list_head *foliolist, - unsigned long flags) + unsigned long flags, struct vm_area_struct *vma, + bool shared) { + bool ret = true; + bool is_hugetlb = is_vm_hugetlb_page(vma); /* * Unless MPOL_MF_MOVE_ALL, we try to avoid migrating a shared folio. * Choosing not to migrate a shared folio is not counted as a failure. @@ -1034,23 +1039,27 @@ static bool migrate_folio_add(struct folio *folio, struct list_head *foliolist, * See folio_likely_mapped_shared() on possible imprecision when we * cannot easily detect if a folio is shared. */ - if ((flags & MPOL_MF_MOVE_ALL) || !folio_likely_mapped_shared(folio)) { - if (folio_isolate_lru(folio)) { - list_add_tail(&folio->lru, foliolist); - node_stat_mod_folio(folio, - NR_ISOLATED_ANON + folio_is_file_lru(folio), - folio_nr_pages(folio)); - } else { + if ((flags & MPOL_MF_MOVE_ALL) || + (!folio_likely_mapped_shared(folio) && !shared)) { + if (is_hugetlb) + return isolate_hugetlb(folio, foliolist); + + ret = folio_isolate_lru(folio); + if (!ret) /* * Non-movable folio may reach here. And, there may be * temporary off LRU folios or non-LRU movable folios. * Treat them as unmovable folios since they can't be * isolated, so they can't be moved at the moment. */ - return false; - } + return ret; + + list_add_tail(&folio->lru, foliolist); + node_stat_mod_folio(folio, + NR_ISOLATED_ANON + folio_is_file_lru(folio), + folio_nr_pages(folio)); } - return true; + return ret; } /* @@ -1239,7 +1248,8 @@ static struct folio *alloc_migration_target_by_mpol(struct folio *src, #else static bool migrate_folio_add(struct folio *folio, struct list_head *foliolist, - unsigned long flags) + unsigned long flags, struct vm_area_struct *vma, + bool shared) { return false; } From patchwork Thu Jul 4 04:31:10 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oscar Salvador X-Patchwork-Id: 13723201 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 40CBBC3271F for ; Thu, 4 Jul 2024 04:32:57 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 1086D6B00AE; Thu, 4 Jul 2024 00:32:23 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 0905C6B00AF; Thu, 4 Jul 2024 00:32:23 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id DFB326B00B0; Thu, 4 Jul 2024 00:32:22 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id B9D906B00AE for ; Thu, 4 Jul 2024 00:32:22 -0400 (EDT) Received: from smtpin10.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 79B231A08BE for ; Thu, 4 Jul 2024 04:32:22 +0000 (UTC) X-FDA: 82300798524.10.878171E Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.223.130]) by imf04.hostedemail.com (Postfix) with ESMTP id 65B3740004 for ; Thu, 4 Jul 2024 04:32:20 +0000 (UTC) Authentication-Results: imf04.hostedemail.com; dkim=none; spf=pass (imf04.hostedemail.com: domain of osalvador@suse.de designates 195.135.223.130 as permitted sender) smtp.mailfrom=osalvador@suse.de; dmarc=pass (policy=none) header.from=suse.de ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1720067527; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=zjLSe21IECWIlLwNB5/Nlo0ijeI0XlWwGvDlsWxpGWs=; b=L5OBqppfVmf8/jkUh1q84F0+/WitSpZrDPa581ITMbZTG4SXWiQQAU1MFd64etp5MLlwLK koR0ls/jgzC+QNKsb5GYZJ7Fl6zQQdL+vRtePFE7ZvD/RxBrD+GgM2YSbyKqu/8a3UG9vH q19tjuKImSuiP6fygKSmNAs+b87hEnY= ARC-Authentication-Results: i=1; imf04.hostedemail.com; dkim=none; spf=pass (imf04.hostedemail.com: domain of osalvador@suse.de designates 195.135.223.130 as permitted sender) smtp.mailfrom=osalvador@suse.de; dmarc=pass (policy=none) header.from=suse.de ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1720067527; a=rsa-sha256; cv=none; b=ABxVfvghmEx5IyO4lbor/Yat6ZUus/bFIUMHbPo21+j0ORWIJVAuXRaWKDhhRDKgF9YtZz 9VJlx9i/mTTWqoGLsDSM5TVegUYyoo2kDAgox6E0oy5nSFjp6iPNjLSsxZ8ysrBBgrmTXf fn5nZ+IP9OkQiUBmx92D+4i98KJt7io= Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [IPv6:2a07:de40:b281:104:10:150:64:97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id 2B07D21C0D; Thu, 4 Jul 2024 04:32:19 +0000 (UTC) Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 06B6F13889; Thu, 4 Jul 2024 04:32:17 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id cMqbONElhmbnDAAAD6G6ig (envelope-from ); Thu, 04 Jul 2024 04:32:17 +0000 From: Oscar Salvador To: Andrew Morton Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Peter Xu , Muchun Song , David Hildenbrand , SeongJae Park , Miaohe Lin , Michal Hocko , Matthew Wilcox , Christophe Leroy , Oscar Salvador Subject: [PATCH 23/45] mm/mempolicy: Create queue_folios_pud to handle PUD-mapped hugetlb vmas Date: Thu, 4 Jul 2024 06:31:10 +0200 Message-ID: <20240704043132.28501-24-osalvador@suse.de> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240704043132.28501-1-osalvador@suse.de> References: <20240704043132.28501-1-osalvador@suse.de> MIME-Version: 1.0 X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Rspamd-Action: no action X-Rspamd-Server: rspam11 X-Rspamd-Queue-Id: 65B3740004 X-Stat-Signature: e5g1snwt8m5ijza1hu66y5rwwt6sqe5y X-Rspam-User: X-HE-Tag: 1720067540-971529 X-HE-Meta: U2FsdGVkX1+8ZHZ/XQo6ChzJ01BIfzSo7IPFWOBif9hbhYj5pWLM9dP8O/H8jruzTgphxvBw3p+vUsQORPsle+O8wtU3LP8cVSkQqG+Ckr3T9cqC1E62AiQWp8N3otyoSoFGGBUmaTYxX2tAQF0dk1QrOTe1/kRHnC1p0zNi6WKkv3XhnHMPDhSVECSqLHBlF+CeanDwicftANZerYi8bCLGyOrzjx9zuxJyHulnEmg3De1N9Yiqr1vlaDY/d30cQItUcMA58Gfe8SdC+b72ahM/t+yirEhPNul5srskgl5y0mqBHtQZ9IoaqkqRig47nSN1zTTr03FScUUw6BCD8kzD/1KBJG1CrOrbVgRiiZm6XcOaHam+PSu0knP8iCf7qjAjsM9+tn0PBdpoHJSoaTEltbs+gASSpUcokdkRma2ZPP5C6qQUNXCo+0FImn2cp+AzTpUdIkKKTIk9A6bPyrWgfRk4qk8ACeKwZeID8XapcjiBhyMJKp5UtxbKZGrfqchwttyUCvI5OAUCLPAg2n2UgF2CEwqRZgklt7N2o50ZHd3lDUqWXLDgrV77tmLkN9aSYMYMHwdblJuEBbwfn2wUMxjgJtD1Hh2dbqu6wx2rA+mpeepVt0UqCM74TdUtF2OOq60pH5I0dr+zYxhUtqaApLwR7B5WlPy+tSIQyQnejVHHwpVrZ2DcUgHt0wj26j2RIs1t5KcYu/zuHOk0adglx/c5G8CluMrEUyalau6ShSYyR+o0u34queLKO34F1vqw7DV5u3UznGpd7assDrsI64NJ5BeJFxLDYgHnADolJl9rspGAZ9cktJJ32t5Cu2GGz7DdsnHJUhU7UBt3efw2dKFLTkQUfpNpaqx+VkCAn8+oxm8VF8tSDUAQseiS9dNS1AZyC9Zfjed8FYCJ4gt9/cNXcptbJ9bulApVtLfEOWu+EX5iuHiBuIRw9b5enySKrS6UKBOLV6dgh8o 05HcuwBc 0IpUPtTMUgswyCxMzZ16oWUGO/7v4e2x4qR3W3o7HpDqYtmz9z0JfnhqK6A8NEVOXF80BDIBLkU+L33m+uWTGXF8ZcdC0Tm92fe7VpHpZmLu5t80UcLHxYF37+a/g6Eug9IdchxR2j25i5Xt4NNOb9NISyqq63iQ1sUkTyzHPJ/1zwBk= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Normal THP cannot be PUD-mapped (besides devmap), but hugetlb can, so create queue_folios_pud in order to handle PUD-mapped hugetlb vmas. Also implement is_pud_migration_entry and pud_folio, as they will be used in this patch. Signed-off-by: Oscar Salvador --- include/linux/pgtable.h | 1 + include/linux/swapops.h | 12 ++++++++++++ mm/mempolicy.c | 32 ++++++++++++++++++++++++++++++++ 3 files changed, 45 insertions(+) diff --git a/include/linux/pgtable.h b/include/linux/pgtable.h index 458e3cbc96b2..23d51fec81ac 100644 --- a/include/linux/pgtable.h +++ b/include/linux/pgtable.h @@ -51,6 +51,7 @@ #endif #define pmd_folio(pmd) page_folio(pmd_page(pmd)) +#define pud_folio(pud) page_folio(pud_page(pud)) /* * A page table page can be thought of an array like this: pXd_t[PTRS_PER_PxD] diff --git a/include/linux/swapops.h b/include/linux/swapops.h index 182957f0d013..a23900961d11 100644 --- a/include/linux/swapops.h +++ b/include/linux/swapops.h @@ -542,6 +542,18 @@ static inline bool is_pfn_swap_entry(swp_entry_t entry) struct page_vma_mapped_walk; +#ifdef CONFIG_HUGETLB_PAGE +static inline int is_pud_migration_entry(pud_t pud) +{ + return is_swap_pud(pud) && is_migration_entry(pud_to_swp_entry(pud)); +} +#else +static inline int is_pud_migration_entry(pud_t pud) +{ + return 0; +} +#endif + #ifdef CONFIG_ARCH_ENABLE_THP_MIGRATION extern int set_pmd_migration_entry(struct page_vma_mapped_walk *pvmw, struct page *page); diff --git a/mm/mempolicy.c b/mm/mempolicy.c index 5baf29da198c..93b14090d484 100644 --- a/mm/mempolicy.c +++ b/mm/mempolicy.c @@ -501,6 +501,37 @@ static inline bool queue_folio_required(struct folio *folio, return node_isset(nid, *qp->nmask) == !(flags & MPOL_MF_INVERT); } +static int queue_folios_pud(pud_t *pud, unsigned long addr, unsigned long end, + struct mm_walk *walk) +{ + spinlock_t *ptl; + struct folio *folio; + struct vm_area_struct *vma = walk->vma; + struct queue_pages *qp = walk->private; + + ptl = pud_huge_lock(pud, vma); + if (!ptl) + return 0; + + if (unlikely(is_pud_migration_entry(*pud))) { + qp->nr_failed++; + goto out; + } + folio = pud_folio(*pud); + if (!queue_folio_required(folio, qp)) + goto out; + if (!(qp->flags & (MPOL_MF_MOVE | MPOL_MF_MOVE_ALL)) || + !vma_migratable(walk->vma) || + !migrate_folio_add(folio, qp->pagelist, qp->flags, walk->vma, false)) + qp->nr_failed++; + + spin_unlock(ptl); +out: + if (qp->nr_failed && strictly_unmovable(qp->flags)) + return -EIO; + return 0; +} + static void queue_folios_pmd(pmd_t *pmd, struct mm_walk *walk) { struct folio *folio; @@ -730,6 +761,7 @@ static int queue_pages_test_walk(unsigned long start, unsigned long end, static const struct mm_walk_ops queue_pages_walk_ops = { .hugetlb_entry = queue_folios_hugetlb, + .pud_entry = queue_folios_pud, .pmd_entry = queue_folios_pte_range, .test_walk = queue_pages_test_walk, .walk_lock = PGWALK_RDLOCK, From patchwork Thu Jul 4 04:31:11 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oscar Salvador X-Patchwork-Id: 13723202 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0FE63C30653 for ; Thu, 4 Jul 2024 04:33:00 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 59AFC6B00AF; Thu, 4 Jul 2024 00:32:24 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 522456B00B0; Thu, 4 Jul 2024 00:32:24 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 325F76B00B1; Thu, 4 Jul 2024 00:32:24 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 0EB876B00AF for ; Thu, 4 Jul 2024 00:32:24 -0400 (EDT) Received: from smtpin06.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id C0CA840A8F for ; Thu, 4 Jul 2024 04:32:23 +0000 (UTC) X-FDA: 82300798566.06.5BC6BDC Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.223.131]) by imf07.hostedemail.com (Postfix) with ESMTP id A7C6440010 for ; Thu, 4 Jul 2024 04:32:21 +0000 (UTC) Authentication-Results: imf07.hostedemail.com; dkim=pass header.d=suse.de header.s=susede2_rsa header.b=ZZlPRFX8; dkim=pass header.d=suse.de header.s=susede2_ed25519 header.b=DZydrxp6; dkim=pass header.d=suse.de header.s=susede2_rsa header.b=ZZlPRFX8; dkim=pass header.d=suse.de header.s=susede2_ed25519 header.b=DZydrxp6; spf=pass (imf07.hostedemail.com: domain of osalvador@suse.de designates 195.135.223.131 as permitted sender) smtp.mailfrom=osalvador@suse.de; dmarc=pass (policy=none) header.from=suse.de ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1720067509; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=HbU3NixA+ORa4pbwAalk6/qaoqPhUeSqtYDCOzcSsu0=; b=JyWN/bcGlBwJUvN20Evz4/UunomWz0jIJeJnL/gt9TEGdqu3ue+lTXdxcBaU3YAoxH4rzH EfBSi9OER2q9rN//Hw16t2/jmOjGDSs5l7AZXZjstLcr9KH9qnoYA1Vd7Pps0JYiJ41od+ eqq8hT5CP9sI1r1U5Vr316daV257k+4= ARC-Authentication-Results: i=1; imf07.hostedemail.com; dkim=pass header.d=suse.de header.s=susede2_rsa header.b=ZZlPRFX8; dkim=pass header.d=suse.de header.s=susede2_ed25519 header.b=DZydrxp6; dkim=pass header.d=suse.de header.s=susede2_rsa header.b=ZZlPRFX8; dkim=pass header.d=suse.de header.s=susede2_ed25519 header.b=DZydrxp6; spf=pass (imf07.hostedemail.com: domain of osalvador@suse.de designates 195.135.223.131 as permitted sender) smtp.mailfrom=osalvador@suse.de; dmarc=pass (policy=none) header.from=suse.de ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1720067509; a=rsa-sha256; cv=none; b=TCyxgl3HApjzwF5vPiagjkXoqSY6u7SAhvOzbESFmSr8b4VqHMkKa//ROquiyLl8cE3rbE F2YHaPcT349nMWnNKC6skySnonkTrgxqGHC3giJGqDAlrgdidKpuE2bZfKB/G9SPXB4wU/ Ww3Nwiam2/FF/1Q4N9akNRPWupo7NHs= Received: from imap1.dmz-prg2.suse.org (unknown [10.150.64.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id 6CA131FCFE; Thu, 4 Jul 2024 04:32:20 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1720067540; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=HbU3NixA+ORa4pbwAalk6/qaoqPhUeSqtYDCOzcSsu0=; b=ZZlPRFX8HPVeyLyltxoPjntrB3vyrMGorhnSIvawEhprDCtQiJ8kvY1aBNeBJMEnFNoka7 zEhX5cg2k/xkhsjTOrqeSfEe1lZe8CG3PQEh3+8UdhDAt9ff/QXS9LBS82y+sKrbT8fyYn t7/3YM8I8df36X7okK3RGPJBJuuuMoQ= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1720067540; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=HbU3NixA+ORa4pbwAalk6/qaoqPhUeSqtYDCOzcSsu0=; b=DZydrxp6It5HlKS3mNh//5MUdSN05oCnIEplA26v4hntyU+xxNFeMDdkBpvi66dQmJHnJM /gkrpKtXKANlwKBg== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1720067540; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=HbU3NixA+ORa4pbwAalk6/qaoqPhUeSqtYDCOzcSsu0=; b=ZZlPRFX8HPVeyLyltxoPjntrB3vyrMGorhnSIvawEhprDCtQiJ8kvY1aBNeBJMEnFNoka7 zEhX5cg2k/xkhsjTOrqeSfEe1lZe8CG3PQEh3+8UdhDAt9ff/QXS9LBS82y+sKrbT8fyYn t7/3YM8I8df36X7okK3RGPJBJuuuMoQ= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1720067540; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=HbU3NixA+ORa4pbwAalk6/qaoqPhUeSqtYDCOzcSsu0=; b=DZydrxp6It5HlKS3mNh//5MUdSN05oCnIEplA26v4hntyU+xxNFeMDdkBpvi66dQmJHnJM /gkrpKtXKANlwKBg== Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 5045F13889; Thu, 4 Jul 2024 04:32:19 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id +B4rDtMlhmbnDAAAD6G6ig (envelope-from ); Thu, 04 Jul 2024 04:32:19 +0000 From: Oscar Salvador To: Andrew Morton Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Peter Xu , Muchun Song , David Hildenbrand , SeongJae Park , Miaohe Lin , Michal Hocko , Matthew Wilcox , Christophe Leroy , Oscar Salvador Subject: [PATCH 24/45] mm/memory_failure: Enable check_hwpoisoned_pmd_entry to handle hugetlb vmas Date: Thu, 4 Jul 2024 06:31:11 +0200 Message-ID: <20240704043132.28501-25-osalvador@suse.de> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240704043132.28501-1-osalvador@suse.de> References: <20240704043132.28501-1-osalvador@suse.de> MIME-Version: 1.0 X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: A7C6440010 X-Stat-Signature: i3kqx19qc4ysjh5ck3zaawyx9g3cadk5 X-Rspam-User: X-HE-Tag: 1720067541-977300 X-HE-Meta: U2FsdGVkX198UwwAZUJn56VLdleLXzeF5wi8reAF5zTKda/eZFgRTPsY/qamj0hNYVazKjO3inNICksiW052hYkoCWVmeVzKhn9kFEMFnbJqXSZH4H11F9XtZ8OgGvd3DO8ESqWS1coBq68KTSJpkFjcn04h5lBqrgqAb50t/pqN6uDeTWBd9xCgEYOxhxudk3Rg7mGBHc/t8dwTmQ3V6JTJ2CKpak7lzKcq7QR+/wv53XXbxOQs7WPSHfClKEI3lMOlk+t/BHxTjPFKZcohiqUnK+Z2X9xvGX+cPLmZOutVQgYmj6uz8c3Jvl3wxjdv/On4NyYvRykkkEMldEOFZytKGgkmoHFBsyEWrj9AZbov9e8VUKtxF64ZmE0tEe0TCi7o1YJDqHZPGSgvT3zg02P8MEh091qsZBR8B47K6Q9wO6Ku9vV9VARqSRug9AOfYiOl6yuK05mSBHceNocXHvWYCI7cEc9uoe4jHP0ajxmzc1zv0X6IOCyrj85YFyNoypjgHs3rwYMLpJz63KA+ks6CrP3gcVBdMFOuSc+8l8ewb5AwzmSG2PEOoq07qXTvpbljBj75pQqt0HuTO4KjB84a2dL8KKaIuGb8nt8k0RL0u+37y+ruyHbjsyDnLiLKFdBlVQ+o9sRYf4QdhlWquWS3KFf0IlQmr1LAVc2+L585QZQAdQj375tfYGCmIFHd/v26eqiOdGoYIXr/CxGgmkjJ7V8RNj9ov7oJLTjKMm8QIdb/8sfp8F86i9WWogKMPTHOo+hyhhcaahhg09/DFsJmbEaKe6Mi5Urh4+06n3v2rEi5yXYbGAFWgHBWqVsrXT+96yzeePSfrJmfB1VkR6A4PDoB3wbzJ6kt052WJWDc8xP5mcI4LHzeWtmELKbLIcPNHNxubGqC8T2HXTDdfa8TJ9ELZP/BV7puioVS8kjjx1jvc+KzvdbGkccw5Y571JV3vL/pNJO6QMV/QAk uUo9GhUf e7Y2syJ+dW0s0D2Hmi5R0tsD8c7fJ0hhpDPcNDM/QGtHE18YvPtBd0ds1IpecPVSyBuBUvVKAN3c01oeEIm5XPGd6MBk0PYa8z9hOSG6OHLYwUJ20u4CoqSqanA4QeNuj8cWNNoH25/M4RR4CttMlDYbyJCq47ZWSurVbcfmTOxs4t328PhWWOA/vBj+5RIiG8bxmUnrXn6AZwjJXAhmqNW5ShjQd6bEdA/Q3K7Oaq7N+HiomeCVLgwQS//WN27RiQVGpUpVtl8xijoZYZLNv8SsRng== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: PMD-mapped hugetlb vmas will also check_hwpoisoned_pmd_entry. Add the required code so it knows how to handle those there. Signed-off-by: Oscar Salvador --- mm/memory-failure.c | 44 ++++++++++++++++++++++++++++++-------------- 1 file changed, 30 insertions(+), 14 deletions(-) diff --git a/mm/memory-failure.c b/mm/memory-failure.c index 0cb1b7bea9a5..8cae95e36365 100644 --- a/mm/memory-failure.c +++ b/mm/memory-failure.c @@ -771,27 +771,43 @@ static int check_hwpoisoned_entry(pte_t pte, unsigned long addr, short shift, return 1; } -#ifdef CONFIG_TRANSPARENT_HUGEPAGE +#ifdef CONFIG_PGTABLE_HAS_HUGE_LEAVES static int check_hwpoisoned_pmd_entry(pmd_t *pmdp, unsigned long addr, - struct hwpoison_walk *hwp) + struct hwpoison_walk *hwp, + struct vm_area_struct *vma) { pmd_t pmd = *pmdp; unsigned long pfn; - unsigned long hwpoison_vaddr; + unsigned short shift; + unsigned long hwpoison_vaddr = addr; - if (!pmd_present(pmd)) - return 0; - pfn = pmd_pfn(pmd); - if (pfn <= hwp->pfn && hwp->pfn < pfn + HPAGE_PMD_NR) { - hwpoison_vaddr = addr + ((hwp->pfn - pfn) << PAGE_SHIFT); - set_to_kill(&hwp->tk, hwpoison_vaddr, PAGE_SHIFT); - return 1; + if (pmd_present(pmd)) { + pfn = pmd_pfn(pmd); + } else { + swp_entry_t swp = pmd_to_swp_entry(pmd); + + if (!is_hwpoison_entry(swp)) + return 0; + pfn = swp_offset_pfn(swp); } - return 0; + + shift = is_vm_hugetlb_page(vma) ? huge_page_shift(hstate_vma(vma)) + : PAGE_SHIFT; + + if (pfn > hwp->pfn || hwp->pfn > pfn + HPAGE_PMD_NR) + return 0; + + if (!is_vm_hugetlb_page(vma)) + hwpoison_vaddr += (hwp->pfn - pfn) << PAGE_SHIFT; + + set_to_kill(&hwp->tk, hwpoison_vaddr, shift); + + return 1; } #else static int check_hwpoisoned_pmd_entry(pmd_t *pmdp, unsigned long addr, - struct hwpoison_walk *hwp) + struct hwpoison_walk *hwp, + struct vm_area_struct *vma) { return 0; } @@ -805,9 +821,9 @@ static int hwpoison_pte_range(pmd_t *pmdp, unsigned long addr, pte_t *ptep, *mapped_pte; spinlock_t *ptl; - ptl = pmd_trans_huge_lock(pmdp, walk->vma); + ptl = pmd_huge_lock(pmdp, walk->vma); if (ptl) { - ret = check_hwpoisoned_pmd_entry(pmdp, addr, hwp); + ret = check_hwpoisoned_pmd_entry(pmdp, addr, hwp, walk->vma); spin_unlock(ptl); goto out; } From patchwork Thu Jul 4 04:31:12 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oscar Salvador X-Patchwork-Id: 13723203 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 165B5C30653 for ; Thu, 4 Jul 2024 04:33:03 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 720846B00B2; Thu, 4 Jul 2024 00:32:25 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 6829D6B00B3; Thu, 4 Jul 2024 00:32:25 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 410B86B00B4; Thu, 4 Jul 2024 00:32:25 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 1EBA46B00B2 for ; Thu, 4 Jul 2024 00:32:25 -0400 (EDT) Received: from smtpin01.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id CFDC9A07E3 for ; Thu, 4 Jul 2024 04:32:24 +0000 (UTC) X-FDA: 82300798608.01.E47B49D Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.223.131]) by imf14.hostedemail.com (Postfix) with ESMTP id DFEBA100017 for ; Thu, 4 Jul 2024 04:32:22 +0000 (UTC) Authentication-Results: imf14.hostedemail.com; dkim=none; spf=pass (imf14.hostedemail.com: domain of osalvador@suse.de designates 195.135.223.131 as permitted sender) smtp.mailfrom=osalvador@suse.de; dmarc=pass (policy=none) header.from=suse.de ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1720067531; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=8g3OGkXPEHlFEZuyFqxzuyVJtkyZFU8vkNp30PpsX/A=; b=8J7UVMblHbKEErsYMTORp7z84Iasw2UhOTeFhf+S3pN2XDPkbw/6YtON+adknwgFj9yi8O JRT8pwOXbrU+CE5zVwO1jdGEtWzAMWxDUqu7oWuP7vTG//6AQuI0e+JQaPNgGc/Pa8RBnb Cr55f35rmo/zne4m5wv0O01OTkLyVmQ= ARC-Authentication-Results: i=1; imf14.hostedemail.com; dkim=none; spf=pass (imf14.hostedemail.com: domain of osalvador@suse.de designates 195.135.223.131 as permitted sender) smtp.mailfrom=osalvador@suse.de; dmarc=pass (policy=none) header.from=suse.de ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1720067531; a=rsa-sha256; cv=none; b=Mi046oYXVU8niqQMidJlXOBLod/yvh7peT1m6c1EQudhIC4TGp7nfp/bSsQlvRIYKcTfn3 MdRbHKS3zZMeID297Tns5DzYb/lK9ciF8slSBlsVe236H7QABVcl4yo3t0FrpTF8KKDt+2 9+/BfHG9xjZeP/mtaGYQPrF4+La5D08= Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [IPv6:2a07:de40:b281:104:10:150:64:97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id ABF721FD00; Thu, 4 Jul 2024 04:32:21 +0000 (UTC) Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 8D48513889; Thu, 4 Jul 2024 04:32:20 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id sKv0HdQlhmbnDAAAD6G6ig (envelope-from ); Thu, 04 Jul 2024 04:32:20 +0000 From: Oscar Salvador To: Andrew Morton Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Peter Xu , Muchun Song , David Hildenbrand , SeongJae Park , Miaohe Lin , Michal Hocko , Matthew Wilcox , Christophe Leroy , Oscar Salvador Subject: [PATCH 25/45] mm/memory-failure: Create check_hwpoisoned_pud_entry to handle PUD-mapped hugetlb vmas Date: Thu, 4 Jul 2024 06:31:12 +0200 Message-ID: <20240704043132.28501-26-osalvador@suse.de> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240704043132.28501-1-osalvador@suse.de> References: <20240704043132.28501-1-osalvador@suse.de> MIME-Version: 1.0 X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Rspamd-Action: no action X-Rspamd-Server: rspam03 X-Rspam-User: X-Rspamd-Queue-Id: DFEBA100017 X-Stat-Signature: foade67mkfacsftubgbkrrihxc4bnffg X-HE-Tag: 1720067542-74229 X-HE-Meta: U2FsdGVkX1+BbSkEFsLVVUbmXX+vQSlkhZhnx708YNtT1PZXnbWyLIG5jyUEW4wamRU1cJ40DxAVuuj9jLUMoTVro8df40x2XWqGRFRIHHOCA3npjbgH5+9podlFHcrcGhD8km42rCZwn0MoWrmXEAIQEps+3u6/1cvE5AQVdPqnlViYkt00JQaLW0uAxjdw0I8SOhKv1tODCvm+gR5j9vvp6EEuJ9/ptxspjQIQwF12upWki4B2A+nOynFfXc/eVuIQGDKrEC8bSX7MNGw/kebusfTYvFqbvfSNRg+NYvsTkr8eCyUx4dZEeSftwbGmpX0EpuwYJQv5EyU+3jqEnnHWWSP0o3rgFbVd2IdeYRFdQ33p8aHy2y3WzjjFGuR7dlMZNwTS8aCHjbRQCWna2o9gxQVa8b1ecojBPA01GOTBFOxUu3cOMu1FWQ/0ZojOhoy2OGhaJJcUWxS9Z861yQNn1m8qf+EkIeonhb3Gvwv9aWjBHgfyWo5wM1wlanzLCrxiFoXu9zcT/lVltLezkbqLi/ZWeikU164il4t7cawOBA7Mz9N2Nos+sWjEZYdJ7RcrqHCCxHS4/HuFNIUjEO2PMyP1n4ROOb10tc5W+FJkHqf172hjBdDCh5eZT9OvP6YAVi0AoHHEVByQnjqifdKo1lPLtUtvBFPsTQl+rllatNjtj3rGnYEoPpdVQ9mnx7ab0ZZYpj6hD/vtS4xbaqGn9yR01LAvEHfn2MM9KPS061FjIrMitzz43smwg7rOW5s1vspiJLFgpAe+tJQZlHzvVoGXtmbF9PSe45HQ3X1JGddqqMZdFCZfadtgkwYtmY2RlPUpW6Y7UtJl0YxSsfyeWT/EMeO8efxOl2O/EBH7dC9Az2faHyalJobyS7C1LupAMkMtiTM3dVP1+SicPi8okteloRJpqHxff2G6Zszw+akFIrAv/a3+VJW/rN2XNFY1vYnoEtogbc/v69q TGSCZ7AB iRWa0CtC+5H+lGgBMJmTP9a7Qa8BNa/T1kS0GFx4SsBBoYSbmf36TRGIBv56KbkXustKGcErpqjTdKuJD+acjRkOxep9Mh+jQWxX7Q2wgevXv6Bu2oPRubX4gIkGpzLADDqiLVVfeJW4E8DhIbWjeVeHo7GUrrsNGHmXXseKRv/x8DIM= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Normal THP cannot be PUD-mapped (besides devmap), but hugetlb can, so create check_hwpoisoned_pud_entry in order to handle PUD-mapped hugetlb vmas. Signed-off-by: Oscar Salvador --- mm/memory-failure.c | 38 ++++++++++++++++++++++++++++++++++++++ 1 file changed, 38 insertions(+) diff --git a/mm/memory-failure.c b/mm/memory-failure.c index 8cae95e36365..622862c4c300 100644 --- a/mm/memory-failure.c +++ b/mm/memory-failure.c @@ -771,6 +771,43 @@ static int check_hwpoisoned_entry(pte_t pte, unsigned long addr, short shift, return 1; } +#ifdef CONFIG_PGTABLE_HAS_HUGE_LEAVES +static int hwpoison_pud_range(pud_t *pudp, unsigned long addr, + unsigned long end, struct mm_walk *walk) +{ + int ret = 0; + spinlock_t *ptl; + pud_t pud = *pudp; + unsigned long pfn; + struct hwpoison_walk *hwp = walk->private; + + ptl = pud_huge_lock(pudp, walk->vma); + if (!ptl) + return 0; + + if (pud_present(pud)) { + pfn = pud_pfn(pud); + } else { + swp_entry_t swp = pud_to_swp_entry(pud); + + if (!is_hwpoison_entry(swp)) + goto out_unlock; + pfn = swp_offset_pfn(swp); + } + + if (!pfn || pfn != hwp->pfn) + goto out_unlock; + + set_to_kill(&hwp->tk, addr, huge_page_shift(hstate_vma(walk->vma))); + ret = 1; +out_unlock: + spin_unlock(ptl); + return ret; +} +#else +hwpoison_pud_range NULL +#endif + #ifdef CONFIG_PGTABLE_HAS_HUGE_LEAVES static int check_hwpoisoned_pmd_entry(pmd_t *pmdp, unsigned long addr, struct hwpoison_walk *hwp, @@ -862,6 +899,7 @@ static int hwpoison_hugetlb_range(pte_t *ptep, unsigned long hmask, #endif static const struct mm_walk_ops hwpoison_walk_ops = { + .pud_entry = hwpoison_pud_range, .pmd_entry = hwpoison_pte_range, .hugetlb_entry = hwpoison_hugetlb_range, .walk_lock = PGWALK_RDLOCK, From patchwork Thu Jul 4 04:31:13 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oscar Salvador X-Patchwork-Id: 13723204 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id CF2DBC3065C for ; Thu, 4 Jul 2024 04:33:05 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D68926B00B3; Thu, 4 Jul 2024 00:32:26 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id CECD76B00B4; Thu, 4 Jul 2024 00:32:26 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B15556B00B5; Thu, 4 Jul 2024 00:32:26 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 85A776B00B3 for ; Thu, 4 Jul 2024 00:32:26 -0400 (EDT) Received: from smtpin27.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 43DC8120F9D for ; Thu, 4 Jul 2024 04:32:26 +0000 (UTC) X-FDA: 82300798692.27.EB7ECE0 Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.223.130]) by imf19.hostedemail.com (Postfix) with ESMTP id 39EA71A0012 for ; Thu, 4 Jul 2024 04:32:23 +0000 (UTC) Authentication-Results: imf19.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=suse.de; spf=pass (imf19.hostedemail.com: domain of osalvador@suse.de designates 195.135.223.130 as permitted sender) smtp.mailfrom=osalvador@suse.de ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1720067524; a=rsa-sha256; cv=none; b=xr6eacKwf94RI04/LfLT52jntjjawC6lU/mfG1ULNonU6D/60GwS8CFj/goECKiMn6c+n4 epQ3PocC8AfRKoDpnsWKD6M0oBEz8hEDELouBSHjDuV7zjIVpTrZ2kDy/8n03xKW+Lzu7V E4H8bzyAdBZpTjvdYVrismR6O32P5sg= ARC-Authentication-Results: i=1; imf19.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=suse.de; spf=pass (imf19.hostedemail.com: domain of osalvador@suse.de designates 195.135.223.130 as permitted sender) smtp.mailfrom=osalvador@suse.de ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1720067524; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=tDyxny4TknAwVCp4MGOFy70tk6Ey0sbLgqgCHwZ91s4=; b=UzXeMvcLXy/EYGDg2YB3SaqrHZV/2vrzJ9/4RR+L7ZslTMGO3hwWx58qAvIjTt5kr9J8tg BX/L21y7y0LRsuNZazmlHxQXO64Hw4GUmKeC7u5Pk5qqe0shDcRH88hhkA1Dm7mot8XAhb eyd2EBGJ7qg30YeTc8VdSyCEV3gVKTI= Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [IPv6:2a07:de40:b281:104:10:150:64:97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id 09ECD21180; Thu, 4 Jul 2024 04:32:23 +0000 (UTC) Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id CD1EE13889; Thu, 4 Jul 2024 04:32:21 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id UADILdUlhmbnDAAAD6G6ig (envelope-from ); Thu, 04 Jul 2024 04:32:21 +0000 From: Oscar Salvador To: Andrew Morton Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Peter Xu , Muchun Song , David Hildenbrand , SeongJae Park , Miaohe Lin , Michal Hocko , Matthew Wilcox , Christophe Leroy , Oscar Salvador Subject: [PATCH 26/45] mm/damon: Enable damon_young_pmd_entry to handle hugetlb vmas Date: Thu, 4 Jul 2024 06:31:13 +0200 Message-ID: <20240704043132.28501-27-osalvador@suse.de> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240704043132.28501-1-osalvador@suse.de> References: <20240704043132.28501-1-osalvador@suse.de> MIME-Version: 1.0 X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Rspamd-Action: no action X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: 39EA71A0012 X-Stat-Signature: 5t93gg5ii793omi8mtkypay6eq7mboq6 X-Rspam-User: X-HE-Tag: 1720067543-717720 X-HE-Meta: U2FsdGVkX1+5XcqRjTWriFiiAeP3vSjGZWzJeE3QImevPIGYlRtsTaxwT/3WKHQqlbIe1/AXKB4cGjH029fYxHt5ZLhH1UDy2Lg8aAHPgpTPFiytQgljIlVPmTREujJK9L9qWpPr0hS7arNgADImYM6swoDUibsnnY10gjOCTAtOhShZsZ/hWIelllCqYTp+tW/acOE+RuJgxeidoc87O74QUHWKXBwOjAYMSWae9BoIFRsBYEl6sW+nIiXBDrE2Q1hPjGVjWO1E9vrnVI8OTuEpsL9eIdCFZzGibYkamiS+fiKu9xatHDKSD9umGinVFUzVzg769g+bPKaOOKP4uIT8URZibN3AThQ6pc1JOSIu078AQpqp3e30kqgM9mfZGr+/WT+LSsklfytNUdhZZdQRPVyllIPE6kitI3CmBFX9dnoyO5SxW26Vf5Rh7UBDmiQcCxD6XKciKh6wKA5mNqu/nkWLlwCv1TRR+w/rtm7zKNO5Nzrh1DiIqSVJt3CQCRGxfd8TzcuWmEuqrWyP17J541hcTdhRKadtu4A/nTlF/fZ1WYJIiMd3/vUlvU+m5ZGwTrGa2J/yhj4UDMX/xyaDt/137XLFb94S+7RNo9p8fdN5IeZAsyRQC8UP3dzm4JoegBcWyB/dpmODpHJFbzPC3woIlCrnxzFqir/OkwLGekGExiFQj58oOhgnYMjVhgFYfrM5r2G9xaXoLFpHT4gEZPLfjXSla9KzfoWQVHZcmZuNjPZj8dmv+9Z/nqGvlqyuXxsLYqfb1QDd+thCa7XbeS04M78j9eP0MpnNqyVres0DwsJw+/mCwuKKqMzPlZcJwRKKPG4AOaMzQ1KX68cofI53sr1svgbXgXBp3Hht+EgUnQOEbDwOjYgsesS7t6W/3XWGlF6ULnJcmZgWN10NTDi9JWcLx8F49vaTyrLNqaAn93fORnHLY3e982eNC/GnIKnuQuyxO8LrauD ONgCcMmf MKfZbwFKBJzbk0/5ucO906LD7swPhzBJB3PfEp3A5T6eSe1S/X8MmhWx7qU2GMTquHiec4m13VqXghsHI3Yo2QFGAudL8GBxXRPRzQx2t2vI22oc9606t9Al6FMZdtk9kQ2Y4yQPxPW7ciJLB0t9KY0S1KS3Rax79/aZq7uwjug0AV7A= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: PMD-mapped hugetlb vmas will also reach damon_young_pmd_entry. Add the required code so it knows how to handle those there. Signed-off-by: Oscar Salvador --- mm/damon/vaddr.c | 31 ++++++++++++++++++------------- 1 file changed, 18 insertions(+), 13 deletions(-) diff --git a/mm/damon/vaddr.c b/mm/damon/vaddr.c index 58829baf8b5d..00d32beffe38 100644 --- a/mm/damon/vaddr.c +++ b/mm/damon/vaddr.c @@ -443,30 +443,35 @@ static int damon_young_pmd_entry(pmd_t *pmd, unsigned long addr, struct folio *folio; struct damon_young_walk_private *priv = walk->private; -#ifdef CONFIG_TRANSPARENT_HUGEPAGE - if (pmd_trans_huge(pmdp_get(pmd))) { - pmd_t pmde; - - ptl = pmd_lock(walk->mm, pmd); - pmde = pmdp_get(pmd); +#ifdef CONFIG_PGTABLE_HAS_HUGE_LEAVES + ptl = pmd_huge_lock(vma, pmd); + if (ptl) { + unsigned long pfn; - if (!pmd_present(pmde)) { + if (!pmd_present(*pmd)) { spin_unlock(ptl); return 0; } - if (!pmd_trans_huge(pmde)) { - spin_unlock(ptl); - goto regular_page; + pfn = pmd_pfn(*pmd); + if (is_vm_hugetlb_page(walk->vma)) { + folio = pfn_folio(pfn); + if (folio) + folio_get(folio); + } else { + folio = damon_get_folio(pfn); } - folio = damon_get_folio(pmd_pfn(pmde)); if (!folio) goto huge_out; if (pmd_young(pmde) || !folio_test_idle(folio) || mmu_notifier_test_young(walk->mm, addr)) priv->young = true; - *priv->folio_sz = HPAGE_PMD_SIZE; + + if (is_vm_hugetlb_page(walk->vma)) + *priv->folio_sz = huge_page_size(h); + else + *priv->folio_sz = HPAGE_PMD_SIZE; folio_put(folio); huge_out: spin_unlock(ptl); @@ -474,7 +479,7 @@ static int damon_young_pmd_entry(pmd_t *pmd, unsigned long addr, } regular_page: -#endif /* CONFIG_TRANSPARENT_HUGEPAGE */ +#endif /* CONFIG_PGTABLE_HAS_HUGE_LEAVES */ pte = pte_offset_map_lock(walk->mm, pmd, addr, &ptl); if (!pte) { From patchwork Thu Jul 4 04:31:14 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oscar Salvador X-Patchwork-Id: 13723205 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 923B6C3271F for ; Thu, 4 Jul 2024 04:33:08 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 12CEF6B00B4; Thu, 4 Jul 2024 00:32:28 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 0B7226B00B5; Thu, 4 Jul 2024 00:32:28 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E49956B00B6; Thu, 4 Jul 2024 00:32:27 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id B5DDB6B00B4 for ; Thu, 4 Jul 2024 00:32:27 -0400 (EDT) Received: from smtpin15.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 72064A3D40 for ; Thu, 4 Jul 2024 04:32:27 +0000 (UTC) X-FDA: 82300798734.15.5325CA1 Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.223.131]) by imf19.hostedemail.com (Postfix) with ESMTP id 79B451A0002 for ; Thu, 4 Jul 2024 04:32:25 +0000 (UTC) Authentication-Results: imf19.hostedemail.com; dkim=none; spf=pass (imf19.hostedemail.com: domain of osalvador@suse.de designates 195.135.223.131 as permitted sender) smtp.mailfrom=osalvador@suse.de; dmarc=pass (policy=none) header.from=suse.de ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1720067533; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=/cqrVYiZZcs66W9WXK22WJ3CXHLy1FyvZXdL+GqDNXA=; b=0YJsr5T9Eu0rM0T9AhxRc0OUJ0h1Fcjg+nGUgILIoYXDsZ5ph5YsZoxUrYN/vgcknCm+jv 2X8eIQQ34r84dNZHTYgyE/hJrVkRiBQK4uGJAZbe7zsS4wNe+LenRa9LZfepAt4W77mHpW O4nKPlrMO6bMMIet6x0HsoE6oWfXx/4= ARC-Authentication-Results: i=1; imf19.hostedemail.com; dkim=none; spf=pass (imf19.hostedemail.com: domain of osalvador@suse.de designates 195.135.223.131 as permitted sender) smtp.mailfrom=osalvador@suse.de; dmarc=pass (policy=none) header.from=suse.de ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1720067533; a=rsa-sha256; cv=none; b=nSrbp9G+33Fjz4UCmdJ3UYy0l53MVXP95scL9O5rm1vYJDOryORFkfYhwJ/7VOIaislpSj M1opaCEYMcmTpriPVVnyST0Dlh84V5WL0qpsxnxs/DmKP2t2uMQfT+cuk6GLlIJgqXEkzD liJIlU4Q/2yeFLnTfhsdHad6vN+sBR4= Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [IPv6:2a07:de40:b281:104:10:150:64:97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id 41E2C1FCF8; Thu, 4 Jul 2024 04:32:24 +0000 (UTC) Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 22FA613889; Thu, 4 Jul 2024 04:32:23 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id YJuNA9clhmbnDAAAD6G6ig (envelope-from ); Thu, 04 Jul 2024 04:32:23 +0000 From: Oscar Salvador To: Andrew Morton Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Peter Xu , Muchun Song , David Hildenbrand , SeongJae Park , Miaohe Lin , Michal Hocko , Matthew Wilcox , Christophe Leroy , Oscar Salvador Subject: [PATCH 27/45] mm/damon: Create damon_young_pud_entry to handle PUD-mapped hugetlb vmas Date: Thu, 4 Jul 2024 06:31:14 +0200 Message-ID: <20240704043132.28501-28-osalvador@suse.de> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240704043132.28501-1-osalvador@suse.de> References: <20240704043132.28501-1-osalvador@suse.de> MIME-Version: 1.0 X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Rspamd-Action: no action X-Rspamd-Server: rspam03 X-Rspam-User: X-Rspamd-Queue-Id: 79B451A0002 X-Stat-Signature: a57sswx9g6pmap5adyfn3odkec1czgom X-HE-Tag: 1720067545-78103 X-HE-Meta: U2FsdGVkX1+uqxSPRs7ZUz6VbjA1hMSWW80074gXXL7IxZ1QLk604FTUmtiwFZ+DTwrhw5d/KCphlWrnDSMuWfFprLwLHcFra/Zvmu0cqYhOhLCxLrFNSrmeH9ECfgQNYGX+fqFxSKW8PTCwFaOU1asyBkJN/xBDh2NIa+N7lVjC+VMKVTc+NAl34zxmPN/kklpeylclRY0uBqWFkzmRKrB8NI9AwUKOnMZvbfEDF2OrZt3tt9ubAzbfB2EwX/Bm/3XnunjRc0TkopDPtetN9V43t5+3NrIeYHC3S77+PeTzx8xib1aAVOx6v4Kda0qODn48IVSLGPTq+ISpMZzAfkETCIIaYHnIvvkgq8rUdYQpIijhhGAomsG7DrjR96/JbeuK6mdph4EQIxkZ8UI7ckzuQmVyfWrerPGn54VEvHZBr1BjNBXF48KAfN7AepmCbtzAbZYgFIuRZVJjtM3g6sr84El/niMmQXz7+4JK9KL4gVdF+JRpmX5PAdXxB2URH65SbwNNvuCPjxTPEBMlFfvP8yem1+TqpROFkr59h+vresLRy5WLGu+79RL4O2veojunaNygpAfTL5DMrRv7YzTFq7mgPM7ODpWSCGHQ9JDh3CuQEaVZJFuBkWc+7r9FneE7mHKNbuF0MGxfZZdty0cRv5UiPyWsDKfxKqtsPEBGG6OyxBlKfFRnkuNL/1liPEKwLg3PtS6gdArl2I6+0drgfOnTW4gBSuDwMMMmwHDdpo6OHdNPpn7l/yuWymVU1KzFU4zsx5Yf00fkj3WdVFauEikBLSS2uZuHjyqJQEowmFAeDaiaPCCuslnAX45zEwEdg313xe+PSJ5c31R/b/PK3SC9EUSAPS1jqhRBMgn5//6McgRbNYqqewPB3zIHIubgsJ8cHClM9rbl3nPKLZy2p2zMrWixUB7nroCNrcDJgpJhYO1jkX6kLHN0UB2KbtJodYPcbwKH5OfK53Y /MurvwYV xY0zMiXKsq5e7beAy+pyfZpTtQCuZLA+GODpjLKdrqzBtXpbn9QfFaK6sy90ly2LhE+q7fVqb4j/2UWtlF0e6ENKWV2PRiXCcOIjWdZ2Wbo66JkjDrhNR+p9FP7gAUpz+09NzFbPfkPm/ZEey16+IH0LevTMgGN0+IG/jfhQsBt/4f/Y= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Normal THP cannot be PUD-mapped (besides devmap), but hugetlb can, so create damon_young_pud_entry in order to handle PUD-mapped hugetlb vmas. Signed-off-by: Oscar Salvador --- mm/damon/vaddr.c | 34 ++++++++++++++++++++++++++++++++++ 1 file changed, 34 insertions(+) diff --git a/mm/damon/vaddr.c b/mm/damon/vaddr.c index 00d32beffe38..2d5ad47b9dae 100644 --- a/mm/damon/vaddr.c +++ b/mm/damon/vaddr.c @@ -434,6 +434,39 @@ struct damon_young_walk_private { bool young; }; +static int damon_young_pud_entry(pmd_t *pud, unsigned long addr, + unsigned long next, struct mm_walk *walk) +{ +#ifdef CONFIG_PGTABLE_HAS_HUGE_LEAVES + spinlock_t *ptl; + struct folio *folio; + struct damon_young_walk_private *priv = walk->private; + + ptl = pud_huge_lock(vma, pud); + if (!ptl) + return 0; + + if (!pud_present(*pud)) + goto out; + + folio = pfn_folio(pud_pfn(*pud)); + if (!folio) + goto out; + folio_get(folio); + + if (pud_young(pmde) || !folio_test_idle(folio) || + mmu_notifier_test_young(walk->mm, addr)) + priv->young = true; + + *priv->folio_sz = huge_page_size(h); + folio_put(folio); +out: + spin_unlock(ptl); +#endif + return 0; +} + + static int damon_young_pmd_entry(pmd_t *pmd, unsigned long addr, unsigned long next, struct mm_walk *walk) { @@ -537,6 +570,7 @@ static int damon_young_hugetlb_entry(pte_t *pte, unsigned long hmask, #endif /* CONFIG_HUGETLB_PAGE */ static const struct mm_walk_ops damon_young_ops = { + .pud_entry = damon_young_pud_entry, .pmd_entry = damon_young_pmd_entry, .hugetlb_entry = damon_young_hugetlb_entry, .walk_lock = PGWALK_RDLOCK, From patchwork Thu Jul 4 04:31:15 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oscar Salvador X-Patchwork-Id: 13723206 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7C715C3065C for ; Thu, 4 Jul 2024 04:33:11 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 4FDEE6B00B5; Thu, 4 Jul 2024 00:32:29 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 486866B00B7; Thu, 4 Jul 2024 00:32:29 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 2B30E6B00B8; Thu, 4 Jul 2024 00:32:29 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 07A5F6B00B5 for ; Thu, 4 Jul 2024 00:32:29 -0400 (EDT) Received: from smtpin08.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id C46EE1A1051 for ; Thu, 4 Jul 2024 04:32:28 +0000 (UTC) X-FDA: 82300798776.08.481CF2C Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.223.130]) by imf13.hostedemail.com (Postfix) with ESMTP id C534720006 for ; Thu, 4 Jul 2024 04:32:26 +0000 (UTC) Authentication-Results: imf13.hostedemail.com; dkim=none; spf=pass (imf13.hostedemail.com: domain of osalvador@suse.de designates 195.135.223.130 as permitted sender) smtp.mailfrom=osalvador@suse.de; dmarc=pass (policy=none) header.from=suse.de ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1720067522; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=9NTITUgNzypalmbjIIhE1wxYVO8aTaBxoagSewHtJpo=; b=b8S7FhzpZPVRcAOsDKNrDgGxYAETTg8vxCpKAYxoJk9T8ZeL0L6cH7pwt9VKlsJ+y9YXWf URcgSnvwwA7Vq9BD4n/62nxxObA4vgUo1jguTs89XGShmd2fBHJZFMKOuxRvUb94lC93MQ g4AE2+BIDBvl4fzcmQf4zOOoFCMf3SU= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1720067522; a=rsa-sha256; cv=none; b=ux89J8JKpx6ShaJhJxArbGL7Ut+Bd8OPVsGMkpz75DDOpGl+IEU2uUPk1Eq8BXn9gqbU2Z nieJyP1YfTMlpse0p0nPXwXfgCgoYsqbqdC+wZhf01FkboNHRlKtSrTOTf6pRJZB6yAdVI gUn3GpI2IsBj3QyY/MR1sBcnfVxtoII= ARC-Authentication-Results: i=1; imf13.hostedemail.com; dkim=none; spf=pass (imf13.hostedemail.com: domain of osalvador@suse.de designates 195.135.223.130 as permitted sender) smtp.mailfrom=osalvador@suse.de; dmarc=pass (policy=none) header.from=suse.de Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [IPv6:2a07:de40:b281:104:10:150:64:97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id 94F3A21C10; Thu, 4 Jul 2024 04:32:25 +0000 (UTC) Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 6893D13889; Thu, 4 Jul 2024 04:32:24 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id EK8YFNglhmbnDAAAD6G6ig (envelope-from ); Thu, 04 Jul 2024 04:32:24 +0000 From: Oscar Salvador To: Andrew Morton Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Peter Xu , Muchun Song , David Hildenbrand , SeongJae Park , Miaohe Lin , Michal Hocko , Matthew Wilcox , Christophe Leroy , Oscar Salvador Subject: [PATCH 28/45] mm/damon: Enable damon_mkold_pmd_entry to handle hugetlb vmas Date: Thu, 4 Jul 2024 06:31:15 +0200 Message-ID: <20240704043132.28501-29-osalvador@suse.de> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240704043132.28501-1-osalvador@suse.de> References: <20240704043132.28501-1-osalvador@suse.de> MIME-Version: 1.0 X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Rspamd-Action: no action X-Rspam-User: X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: C534720006 X-Stat-Signature: tsgdmcpgghbucpuupp784nczngue7f74 X-HE-Tag: 1720067546-266168 X-HE-Meta: U2FsdGVkX185KMx6LkKwJk4xhanSrZ6syaiH2+sDyT+ffiMIoae7nxNnxhRudwhqGKJyXe9v/VhxkyPoaspyQa3lXrJXjK5dB+J+KCZ04s9pkmFc/3ceam9YUvLV2YZrXsYkstRt6iYb9oESHWAVv+vUAZu3swaY4/rSmi//taKxcQi1sE3PJ+kG7/baQDfflLdVc8B8wSGMc8/nKEBdqk9NFFAB4FKl5Hk9cZZtM0FhKxOmITdLegtpRQs4OvZTSC1pjCe4OohBLLaKDYUKYmNZmmrtnA3MsJ21i6l7pmfD14HySzYh+kO7RHJSq5l3d/QdyRY3kpFeioLCf6UbrCI2HFiELEEKBxpEWdDhPH1Dme4ZkEjgZddhNMqtRntl22dAu15NwTX50vN95G8sWEHayncm0Jscu0yjQtXUIeLHasxYDPfqyipvgeapf9zOs6FVdkTHtGsg95HvH8M6eYoCpJtcconePKIRLWPJQ+iLkUJsLovfLQs4HFNT/YRoJUDwjoicjiflL0K7i4oZPs8rg7MyO4TylpZK8hWJcDpgB557ZFcuSwAMBgJOyKTYaGiKZIQFJUkkFXPgjn6giRJiJJgFs+vJH5tKOxQT6hxfVb7P9vBWHXOOtBgGFiRTzVxEZNU/NEoEse4uqWp0FmsQcpCHJzXtU7bpSF9YosFKgtl79aB9y7PBfjMe8zl9BspEOyVLHKqo0T7119SDp6ORwW45YVSqGJQLDuEZawd7plIrour7r+fIm1wUiItSIKLb/qYjnz3IwwsTk+FooD57WHEEBM8/jvmpku9oxHUSh28bFeWcRmb0h9yM4YZ89P8UdrY6pgV3f65tUYASJm8Wk3iGdoTC1aY40w02I/IahKYwy+13ohR9eX4hUJz0V1uCCmYxRFiHQjKY78zJ9uqJNAkyS77C93/2D0zi0AppG/ax5H5zNKLIZWxJZ6dM6cavBCaA/bHMVHLKhxl guSsn2xw bYM+2XhzCJ6QrcNXiXizGAPTLbO2R2vGbSKQBmRJ+RSGJukNmY32zPXivohRzP2BrjlrsrZlBHIRYkdYv86N4FTz2F+waY2JZ65oUBN5ZJMaDlVsBwWGs1uIEALm4TW/+nNSpCQsovv3cF2+xiRFPPD9vgkSU9j9hGdS137KpP0Eshyo= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: PMD-mapped hugetlb vmas will also reach damon_mkold_pmd_entry. Add the required code so it knows how to handle those there. Signed-off-by: Oscar Salvador --- mm/damon/ops-common.c | 21 ++++++++++++++++----- mm/damon/vaddr.c | 15 +++++---------- 2 files changed, 21 insertions(+), 15 deletions(-) diff --git a/mm/damon/ops-common.c b/mm/damon/ops-common.c index d25d99cb5f2b..6727658a3ef5 100644 --- a/mm/damon/ops-common.c +++ b/mm/damon/ops-common.c @@ -53,18 +53,29 @@ void damon_ptep_mkold(pte_t *pte, struct vm_area_struct *vma, unsigned long addr void damon_pmdp_mkold(pmd_t *pmd, struct vm_area_struct *vma, unsigned long addr) { -#ifdef CONFIG_TRANSPARENT_HUGEPAGE - struct folio *folio = damon_get_folio(pmd_pfn(pmdp_get(pmd))); +#ifdef CONFIG_PGTABLE_HAS_HUGE_LEAVES + struct folio *folio; + unsigned long size; + + if (is_vm_hugetlb_page(vma)) { + folio = pfn_folio(pdm_pfn(*pmd)) + folio_get(folio); + size = huge_page_size(hstate_vma(vma)); + } else { + folio = damon_get_folio(pmd_pfn(*pmd)); + size = PMD_SIZE; + } if (!folio) - return; + return 0; - if (pmdp_clear_young_notify(vma, addr, pmd)) + if (pmdp_test_and_clear_young(vma, addr, pmd) || + mmu_notifier_clear_young(mm, addr, addr + size)) folio_set_young(folio); folio_set_idle(folio); folio_put(folio); -#endif /* CONFIG_TRANSPARENT_HUGEPAGE */ +#endif /*CONFIG_PGTABLE_HAS_HUGE_LEAVES */ } #define DAMON_MAX_SUBSCORE (100) diff --git a/mm/damon/vaddr.c b/mm/damon/vaddr.c index 2d5ad47b9dae..47c84cdda32c 100644 --- a/mm/damon/vaddr.c +++ b/mm/damon/vaddr.c @@ -304,21 +304,16 @@ static int damon_mkold_pmd_entry(pmd_t *pmd, unsigned long addr, pmd_t pmde; spinlock_t *ptl; - if (pmd_trans_huge(pmdp_get(pmd))) { - ptl = pmd_lock(walk->mm, pmd); - pmde = pmdp_get(pmd); - - if (!pmd_present(pmde)) { + ptl = pmd_huge_lock(walk->vma, pmd); + if (ptl) { + if (!pmd_present(*pmd)) { spin_unlock(ptl); return 0; } - if (pmd_trans_huge(pmde)) { - damon_pmdp_mkold(pmd, walk->vma, addr); - spin_unlock(ptl); - return 0; - } + damon_pmdp_mkold(pmd, walk->vma, addr); spin_unlock(ptl); + return 0; } pte = pte_offset_map_lock(walk->mm, pmd, addr, &ptl); From patchwork Thu Jul 4 04:31:16 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oscar Salvador X-Patchwork-Id: 13723207 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7F592C30653 for ; Thu, 4 Jul 2024 04:33:14 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id C05C66B00B8; Thu, 4 Jul 2024 00:32:30 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id B8F456B00B9; Thu, 4 Jul 2024 00:32:30 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 98F786B00BA; Thu, 4 Jul 2024 00:32:30 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 68ADF6B00B8 for ; Thu, 4 Jul 2024 00:32:30 -0400 (EDT) Received: from smtpin30.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 32EF2A0A7B for ; Thu, 4 Jul 2024 04:32:30 +0000 (UTC) X-FDA: 82300798860.30.D57A8F5 Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.223.130]) by imf11.hostedemail.com (Postfix) with ESMTP id 1A05E4000C for ; Thu, 4 Jul 2024 04:32:27 +0000 (UTC) Authentication-Results: imf11.hostedemail.com; dkim=pass header.d=suse.de header.s=susede2_rsa header.b=vi5KLgdE; dkim=pass header.d=suse.de header.s=susede2_ed25519 header.b=PNL9w7HV; dkim=pass header.d=suse.de header.s=susede2_rsa header.b=vi5KLgdE; dkim=pass header.d=suse.de header.s=susede2_ed25519 header.b=PNL9w7HV; dmarc=pass (policy=none) header.from=suse.de; spf=pass (imf11.hostedemail.com: domain of osalvador@suse.de designates 195.135.223.130 as permitted sender) smtp.mailfrom=osalvador@suse.de ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1720067528; a=rsa-sha256; cv=none; b=VqWKP72FHp5U75vLJeTB8BZ2lSunngd+Oq6IeBby0/FRn1YZRVFigFNgl8XxXagemG2P6X HwF/n+TNlBa5h6bUZFOP4Mh86oD9thWNEitm0+6Rq/M+cboqH7kw6Q4W69+jLk3pApBBtM G0+EqCcuqKRGhcLjISDKDJ8NdzfZVZA= ARC-Authentication-Results: i=1; imf11.hostedemail.com; dkim=pass header.d=suse.de header.s=susede2_rsa header.b=vi5KLgdE; dkim=pass header.d=suse.de header.s=susede2_ed25519 header.b=PNL9w7HV; dkim=pass header.d=suse.de header.s=susede2_rsa header.b=vi5KLgdE; dkim=pass header.d=suse.de header.s=susede2_ed25519 header.b=PNL9w7HV; dmarc=pass (policy=none) header.from=suse.de; spf=pass (imf11.hostedemail.com: domain of osalvador@suse.de designates 195.135.223.130 as permitted sender) smtp.mailfrom=osalvador@suse.de ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1720067528; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=+Eaq4YS+I2JMNj2PlLYmITmw1dUtdxAYyfPBt6d+VGg=; b=Zrws0IoF4wuPiyPj+cBctTgd9vExu0taV3niSpZ6ys96G+iNQBLjcjuk03uwN1EpgsurNS nDF7tlwtsRvRp/OzAEt9XBaMUFH5F5Bajo++PKtA1BRlGZEkHAT8OqBgXTOLQ88vNXKB+c V7eHpx1DdFUKN4e/6VcNUkoM3dt6HXs= Received: from imap1.dmz-prg2.suse.org (unknown [10.150.64.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id BBF8521C0D; Thu, 4 Jul 2024 04:32:26 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1720067546; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=+Eaq4YS+I2JMNj2PlLYmITmw1dUtdxAYyfPBt6d+VGg=; b=vi5KLgdENdkrnyeebmRYr1GTUMnsDQhLSyx0si8djR9aeKoZAFXDbChioD9oLU5n2WEp19 sZ/1YXnUx4zsqmQaqBABY+2OXBM8pE0ZclE7Xl4ENMKN82fsQA0jjs9APgzard6WS/Yweg oOQsVALT38rdOWZXcSnlqZBgxaDhVAo= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1720067546; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=+Eaq4YS+I2JMNj2PlLYmITmw1dUtdxAYyfPBt6d+VGg=; b=PNL9w7HV1zFYjASnMOA/fooIqae95dSIt4v71VfIDGS0e9g8E4lsal0N9CyLMaXb/wifCJ M4zsefGx9x+meuCg== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1720067546; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=+Eaq4YS+I2JMNj2PlLYmITmw1dUtdxAYyfPBt6d+VGg=; b=vi5KLgdENdkrnyeebmRYr1GTUMnsDQhLSyx0si8djR9aeKoZAFXDbChioD9oLU5n2WEp19 sZ/1YXnUx4zsqmQaqBABY+2OXBM8pE0ZclE7Xl4ENMKN82fsQA0jjs9APgzard6WS/Yweg oOQsVALT38rdOWZXcSnlqZBgxaDhVAo= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1720067546; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=+Eaq4YS+I2JMNj2PlLYmITmw1dUtdxAYyfPBt6d+VGg=; b=PNL9w7HV1zFYjASnMOA/fooIqae95dSIt4v71VfIDGS0e9g8E4lsal0N9CyLMaXb/wifCJ M4zsefGx9x+meuCg== Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id ACDBD13889; Thu, 4 Jul 2024 04:32:25 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id WC9sJdklhmbnDAAAD6G6ig (envelope-from ); Thu, 04 Jul 2024 04:32:25 +0000 From: Oscar Salvador To: Andrew Morton Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Peter Xu , Muchun Song , David Hildenbrand , SeongJae Park , Miaohe Lin , Michal Hocko , Matthew Wilcox , Christophe Leroy , Oscar Salvador Subject: [PATCH 29/45] mm/damon: Create damon_mkold_pud_entry to handle PUD-mapped hugetlb vmas Date: Thu, 4 Jul 2024 06:31:16 +0200 Message-ID: <20240704043132.28501-30-osalvador@suse.de> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240704043132.28501-1-osalvador@suse.de> References: <20240704043132.28501-1-osalvador@suse.de> MIME-Version: 1.0 X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: 1A05E4000C X-Stat-Signature: wijm9dpp7zjwhfrw3hucgnsmwcri7qn5 X-Rspam-User: X-HE-Tag: 1720067547-703186 X-HE-Meta: U2FsdGVkX1+0zuMA39t4dj7dTKfd8BuTfmElllZay1uf8Wnu6LNz9O3lIrNjwfATlJdc9TYPXLbwTeqHjJ/qvEngVaWSy8BH2wuIKOt13bqIRcRKgyO0oiTR360BULhUtP17WLY3f2KjoAdJ5I5RYd0J0LMKacDsjn3qrrmywrhtlM6aFeX55ciHWwx3AG+rXglY3KwQQ996Upjkj6HPNEUgqhF4ugCoEhH/oHcQBTKGaom/8Cphkh+vL53WPIbF3Bp3VtrCrrisQ4aASVf9Gbd2nPKAbyAYyQqJBuAwhcbWvOJzmB5A3zJkrwf7rWndqg8mvVhDWob44Nhi32nA2kNJPmItiHr+gv6y2jMnl+LVmS9l1BLDrpDtDHPHCdp+8nEDQKGxCzEH5bGXUHkG4wWPc4D7Gwz3b1eg4hOexgqfgUl3fU97Bn9+/vOQCWvL5AGggGcdXH2QMypZeoYA7hexboOuYPDR+YL+0ZIka0cGFUnj3gUHAghvEF7q4WNC0WAK/D1qvnk3vMtdJpNrn6hgoVJWwElgUYkiSwKqZE60I3dcREMlNiRpUU38WJK/DcCpP6shG1DGP/h8mcmROWWz38puO9e71wwryrP3f9LHXW7kB5O3BHuxpHMPqSVfK79wdzCFb6Rybg2HlTokxRRIhQ7mUyJ/zynxfGnvQhuxHUJ3/BuI7lGXQL8QJtHhZunF2ECmzdV5mQAd00eGufH1TOAPfBoktwxmws46EjbVs0CJ3ORRVoDp78mtEubQ2QTHeaxj51ebNf0SpEXF0PwddN7L6Kelci1+EOS8LRQgGIu+pHcYeYBvcBurUqSU9Nr0YOYxBor/oLHFdo8OYzRxOtHZ8ThW+lGaUWsIrw3JGdVWn/yQc71ZgIPUv7sSLxYcHmYkC7GoOzpLE/I/TkSQ5bFUuzOMfrEN+7JMXCjN+TsRi8OIQtHeAOm7mvPWbNSy9SnZFSUaYiHxw5M qRwtmL/x RRj/YevbQCd706ioshidR4dtdFUGapz0qxv8j0hBlocSacY1y18nZs5RNQQzyTaS9Vms1uC6ehBDZX1VUpfp0FyugyWPmWymYcH8Y70X+GjLPCiQtF+6OUurKbEDSAW/b+y7a3D/p0vH1Z0dxjl+19nREP5hHUk1xSbz1CutEYHsQXiHytd2SKBOU/uy82P6jwuJG5MQi8y9WipUHfpK/jN/aL1rcH4sLk1LVK09+5B1DBHXq/n75USOlYanntU7RueN17v4u7fEjyliLkGHvSUUAA5ABjH285KmG X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Normal THP cannot be PUD-mapped (besides devmap), but hugetlb can, so create damon_mkold_pud_entry in order to handle PUD-mapped hugetlb vmas. Signed-off-by: Oscar Salvador --- mm/damon/vaddr.c | 32 ++++++++++++++++++++++++++++++++ 1 file changed, 32 insertions(+) diff --git a/mm/damon/vaddr.c b/mm/damon/vaddr.c index 47c84cdda32c..6a383ce5a775 100644 --- a/mm/damon/vaddr.c +++ b/mm/damon/vaddr.c @@ -329,6 +329,37 @@ static int damon_mkold_pmd_entry(pmd_t *pmd, unsigned long addr, return 0; } +static int damon_mkold_pud_entry(pmd_t *pud, unsigned long addr, + unsigned long next, struct mm_walk *walk) +{ +#ifdef CONFIG_PGTABLE_HAS_HUGE_LEAVES + spinlock_t *ptl; + struct folio *folio; + struct vm_area_struct *vma = walk->vma; + unsigned long size = huge_page_size(hstate_vma(vma)); + + ptl = pud_huge_lock(vma, pud); + if (!ptl) + return 0; + + if (!pud_present(*pud)) + goto out; + + folio = pfn_folio(pud_pfn(*pud)); + folio_get(folio); + + if (pudp_test_and_clear_young(vma, addr, pud) || + mmu_notifier_clear_young(mm, addr, addr + size)) + folio_set_young(folio); + + folio_set_idle(folio); + folio_put(folio); +out: + spin_unlock(ptl); +#endif + return 0; +} + #ifdef CONFIG_HUGETLB_PAGE static void damon_hugetlb_mkold(pte_t *pte, struct mm_struct *mm, struct vm_area_struct *vma, unsigned long addr) @@ -383,6 +414,7 @@ static int damon_mkold_hugetlb_entry(pte_t *pte, unsigned long hmask, #endif /* CONFIG_HUGETLB_PAGE */ static const struct mm_walk_ops damon_mkold_ops = { + .pud_entry = damon_mkold_pud_entry, .pmd_entry = damon_mkold_pmd_entry, .hugetlb_entry = damon_mkold_hugetlb_entry, .walk_lock = PGWALK_RDLOCK, From patchwork Thu Jul 4 04:31:17 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oscar Salvador X-Patchwork-Id: 13723208 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 70776C30653 for ; Thu, 4 Jul 2024 04:33:17 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id C79BA6B00BA; Thu, 4 Jul 2024 00:32:31 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id C0C6E6B00BB; Thu, 4 Jul 2024 00:32:31 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A05276B00BC; Thu, 4 Jul 2024 00:32:31 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 71EAB6B00BA for ; Thu, 4 Jul 2024 00:32:31 -0400 (EDT) Received: from smtpin16.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 311521A08BE for ; Thu, 4 Jul 2024 04:32:31 +0000 (UTC) X-FDA: 82300798902.16.C1FD1C4 Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.223.130]) by imf16.hostedemail.com (Postfix) with ESMTP id 420B0180008 for ; Thu, 4 Jul 2024 04:32:29 +0000 (UTC) Authentication-Results: imf16.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=suse.de; spf=pass (imf16.hostedemail.com: domain of osalvador@suse.de designates 195.135.223.130 as permitted sender) smtp.mailfrom=osalvador@suse.de ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1720067522; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ydHGdWE13xUNLo1e2WTJKQjsSMTrUxDPdYXIAbTptmg=; b=D9F+Td5Ac8Cvy82+HjJNK6L4wU6rn6DW2WBlk3Ay8Q6FkOZ9EyRjSIxbekobuEPANR77T/ ICbJt2h+4TNv9UJix7Z9J8aI0IPzJ+tgAiex2oJJQO7uPhOQ0NUqCzPMfw5tkI6B1rDXu5 GmQyIcR87zDA31syAA+WxaLeY1zzU4M= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1720067522; a=rsa-sha256; cv=none; b=qlEMWdd4c4j42mKUR7q/4NycUcziOIZlMMZhPeQwLyk6Yz1qiJBwo/AViFEAyLzJI8nBFT UUWAdrRxPcCqZ6NrKqHIVhoQ6PX9GpGGtXQ/tzdbaKUv/ASDNj7nztMCRkVi/S1Eu3b/+Y e/ltIk798LeGRKmVA9om+Kl2hvIoIb4= ARC-Authentication-Results: i=1; imf16.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=suse.de; spf=pass (imf16.hostedemail.com: domain of osalvador@suse.de designates 195.135.223.130 as permitted sender) smtp.mailfrom=osalvador@suse.de Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [IPv6:2a07:de40:b281:104:10:150:64:97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id 0FE1121C11; Thu, 4 Jul 2024 04:32:28 +0000 (UTC) Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id DDC9613889; Thu, 4 Jul 2024 04:32:26 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id qGv2MdolhmbnDAAAD6G6ig (envelope-from ); Thu, 04 Jul 2024 04:32:26 +0000 From: Oscar Salvador To: Andrew Morton Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Peter Xu , Muchun Song , David Hildenbrand , SeongJae Park , Miaohe Lin , Michal Hocko , Matthew Wilcox , Christophe Leroy , Oscar Salvador Subject: [PATCH 30/45] mm,mincore: Enable mincore_pte_range to handle hugetlb vmas Date: Thu, 4 Jul 2024 06:31:17 +0200 Message-ID: <20240704043132.28501-31-osalvador@suse.de> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240704043132.28501-1-osalvador@suse.de> References: <20240704043132.28501-1-osalvador@suse.de> MIME-Version: 1.0 X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Rspamd-Action: no action X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: 420B0180008 X-Stat-Signature: utjoqty51ab1n4jafpo7exekid5n8oze X-Rspam-User: X-HE-Tag: 1720067549-920787 X-HE-Meta: U2FsdGVkX19qaERgVt/NAh0rbfqHGmtf8nJq4Hx0ppdYowqf4/0N43IC9Pcr7He9qWQnHLZhpofGagN6ELEtq6GOLu0XWIri/XA5GSoASM+O4tSCYtQG2vNLXIUarqEs3KNyttPuqjnbMCkNWdDNuDjSi3aDY835tZuCFA5kGbJTKet+PNHm8Lyc1s0Uxpja3fSdcmUOFjOB0NbK17FlyZzJTEdolK3Qseod6JmQ6g+Lsl95VkNyENmL7pRE24RYzeSt3zJPchpGFwIYJd/2wai2Bl3EwljfNm7PTmZMPsEYUbJ95erSPVemZpXztwjRWKjehhVGt583w6//SP2PwUjD11yF13aVv+I/QkwWhBs+t1CrWTyXRi1cXbxB2B13C3kWOpbCMLUzHIifveCFW5F6BNt/84E+xbTLJ36ssqxlieA7aei2YWPQS+thzYDi60ye3aCRpVLXOibE1wcr/+9vJ4Il2xe8BQfcPtJ8yVI/fdSLQbHoFWRnMide3N3HaJEMe9bX/5hEzuo3VBadY1Mc7QceKzWBiyWz0CgyJPgxlwpT8gdq6oG/IqZlNUGxtYsIYR1J7NSQ68s59vupJ6kxuOrXQk66magYSCvFFCnbtVRLfmXBoO7apsxKXhY3sG23ckVaZyLneKLdnSDCz3eecZqyJIM425kLng1o1Y9usORGhIkwd9MJrfBGCCIlocMJOFXy5UOM5JhUVwy6JAnoyPBVnWzXH919MXOwyONqSOv/FZHGfbIUnN3ZEajblBvLjRyCsYGWpHz5GJ98UnYAanX9tIMGJ0uJbbVLZY92Gfc2vk432aaumLhxLB99MWEKZTFr04eFgF4LETRtYuVx00NOPQCMGYE+y6mPvVlzqj5m2nLcwxd2zxpwejFuOt2yePYlFmBOeo5p0dYNbJdviEQ/XlE8gN7fatbRLP5Zr64TEdxgF6hSZLBW+k4GInmmdON3MseSQxwLFYx 4rDJ0Ci/ 4BOoDAxrpuWWujdX30N1VuYsghKLNXrmQyOrgvyvMjvaO+ZXYkXY6SseHhojUS5UiNlU2ZeqpVF/9zMLk5LYZxHEAk82/6E49g8DGCDrhE3uPdIGw3YyRTUl2XsceLYVTeLIVDVtkvC+yyahdQSKkAnTNPBlgNpapJOHzg7PPNmr7MvA= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: PMD-mapped hugetlb vmas will also mincore_pte_range. Add the required code so it knows how to handle those there. Signed-off-by: Oscar Salvador --- mm/mincore.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/mm/mincore.c b/mm/mincore.c index d6bd19e520fc..5154bc705f60 100644 --- a/mm/mincore.c +++ b/mm/mincore.c @@ -18,6 +18,7 @@ #include #include #include +#include #include #include "swap.h" @@ -106,8 +107,9 @@ static int mincore_pte_range(pmd_t *pmd, unsigned long addr, unsigned long end, unsigned char *vec = walk->private; int nr = (end - addr) >> PAGE_SHIFT; - ptl = pmd_trans_huge_lock(pmd, vma); + ptl = pmd_huge_lock(pmd, vma); if (ptl) { + /* Better handling of hugetlb is required (pte marker etc.) */ memset(vec, 1, nr); spin_unlock(ptl); goto out; From patchwork Thu Jul 4 04:31:18 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oscar Salvador X-Patchwork-Id: 13723209 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 253F1C30653 for ; Thu, 4 Jul 2024 04:33:20 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 4D8496B00BC; Thu, 4 Jul 2024 00:32:33 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 45F1D6B00BD; Thu, 4 Jul 2024 00:32:33 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 1F0816B00BE; Thu, 4 Jul 2024 00:32:33 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id EEE3B6B00BC for ; Thu, 4 Jul 2024 00:32:32 -0400 (EDT) Received: from smtpin11.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id AD0121C00F1 for ; Thu, 4 Jul 2024 04:32:32 +0000 (UTC) X-FDA: 82300798944.11.AEEAB1B Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.223.130]) by imf07.hostedemail.com (Postfix) with ESMTP id 9A2F74000B for ; Thu, 4 Jul 2024 04:32:30 +0000 (UTC) Authentication-Results: imf07.hostedemail.com; dkim=pass header.d=suse.de header.s=susede2_rsa header.b=QpIyFU+S; dkim=pass header.d=suse.de header.s=susede2_ed25519 header.b=udDCKM8V; dkim=pass header.d=suse.de header.s=susede2_rsa header.b=QpIyFU+S; dkim=pass header.d=suse.de header.s=susede2_ed25519 header.b=udDCKM8V; spf=pass (imf07.hostedemail.com: domain of osalvador@suse.de designates 195.135.223.130 as permitted sender) smtp.mailfrom=osalvador@suse.de; dmarc=pass (policy=none) header.from=suse.de ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1720067538; a=rsa-sha256; cv=none; b=4w3HPfwBfjJPLmvalCsVEpXMlIkazDj849V2R6vvCE9l+9PznO3hSQvw7t3ro1PMGuoxV4 fYsZfittrdn+cTFmz1uWyDUxCIRATrFyJYdzGppsIqfR+14sbJ2qTHmgavjNRP6sPdj1TC Pi+XS51dXs4rC1BHBhWEbr27yBqc5J4= ARC-Authentication-Results: i=1; imf07.hostedemail.com; dkim=pass header.d=suse.de header.s=susede2_rsa header.b=QpIyFU+S; dkim=pass header.d=suse.de header.s=susede2_ed25519 header.b=udDCKM8V; dkim=pass header.d=suse.de header.s=susede2_rsa header.b=QpIyFU+S; dkim=pass header.d=suse.de header.s=susede2_ed25519 header.b=udDCKM8V; spf=pass (imf07.hostedemail.com: domain of osalvador@suse.de designates 195.135.223.130 as permitted sender) smtp.mailfrom=osalvador@suse.de; dmarc=pass (policy=none) header.from=suse.de ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1720067538; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=Hlyc02hswOFBPpFjPMKLH2SPKevIA4btQnEnfz8PzX8=; b=w5QpefvEgLA6dlVlMhyptCuVs5dnImgDF+zX0bGSks8N/BH4nFRqYNfV2wqAn1wm/aDpdB cS3/DQHJRCSIUjPyh+O7cF7EgQKSORvYG0LEu88zvVPjTxym5QBxvBofR9LtsyJPh60LJT xRVeAB1ZvCvV520kUHtYkWQXHTLCRnw= Received: from imap1.dmz-prg2.suse.org (unknown [10.150.64.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id 5024D21C12; Thu, 4 Jul 2024 04:32:29 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1720067549; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Hlyc02hswOFBPpFjPMKLH2SPKevIA4btQnEnfz8PzX8=; b=QpIyFU+SNEDWpoiZMvudIRFmIbkNPlPQAdHO8szaEcTZHDv7mQap7dxr0MNsVTThx2cHcx jhq/PHrBd4eT+7TOlVKBNgQLFqsLoSzcymO1iD3cWeXTiS0+9wXL7guAJDapk88hwYPbsN UXGfJEukOZwgFWGHbc1yAGUFbMmlXQI= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1720067549; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Hlyc02hswOFBPpFjPMKLH2SPKevIA4btQnEnfz8PzX8=; b=udDCKM8VSh5fm40PS9NUBV30hL3Hc21Fph1pBda42Ie9OAKZOIyY5YZKKy4Vasv/sNi9T4 b+SyyOfTIIs5xDDg== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1720067549; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Hlyc02hswOFBPpFjPMKLH2SPKevIA4btQnEnfz8PzX8=; b=QpIyFU+SNEDWpoiZMvudIRFmIbkNPlPQAdHO8szaEcTZHDv7mQap7dxr0MNsVTThx2cHcx jhq/PHrBd4eT+7TOlVKBNgQLFqsLoSzcymO1iD3cWeXTiS0+9wXL7guAJDapk88hwYPbsN UXGfJEukOZwgFWGHbc1yAGUFbMmlXQI= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1720067549; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Hlyc02hswOFBPpFjPMKLH2SPKevIA4btQnEnfz8PzX8=; b=udDCKM8VSh5fm40PS9NUBV30hL3Hc21Fph1pBda42Ie9OAKZOIyY5YZKKy4Vasv/sNi9T4 b+SyyOfTIIs5xDDg== Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 3048813889; Thu, 4 Jul 2024 04:32:28 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id kBsMB9wlhmbnDAAAD6G6ig (envelope-from ); Thu, 04 Jul 2024 04:32:28 +0000 From: Oscar Salvador To: Andrew Morton Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Peter Xu , Muchun Song , David Hildenbrand , SeongJae Park , Miaohe Lin , Michal Hocko , Matthew Wilcox , Christophe Leroy , Oscar Salvador Subject: [PATCH 31/45] mm/mincore: Create mincore_pud_range to handle PUD-mapped hugetlb vmas Date: Thu, 4 Jul 2024 06:31:18 +0200 Message-ID: <20240704043132.28501-32-osalvador@suse.de> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240704043132.28501-1-osalvador@suse.de> References: <20240704043132.28501-1-osalvador@suse.de> MIME-Version: 1.0 X-Stat-Signature: 5ewfbjqef45zrw6moyy5tps84pcs76j9 X-Rspamd-Queue-Id: 9A2F74000B X-Rspam-User: X-Rspamd-Server: rspam10 X-HE-Tag: 1720067550-62382 X-HE-Meta: U2FsdGVkX1/sTr7fyZi4zlOYDqE6Kp8kZ1r4D5dceCCTddYJpJ6/dP0X15sWJ890vb+nQ2UBlxZvQDNvzmjl85rZd/a522YGJotin3DH6Tu4V9DTsP2yguPW0Ezbyi5BBLtmDcu0O0m0KiK7pxpgeK+DIu4iL/2+IXtrHs7RLf6GDQbybGujwEG8/sRU3GhtcTQuiCIUmDhcxbOXgWX9ozzS7Fu7+8tr26Ypq96qXKu4FFKtvMsKY55Q2udHiDjzxW3yVCFRhbTXZ80QKpv+3J2QPfcs8G+ncTy9i0jBtWfrhylf9/CLiQrm1X/qsWMEC42ik5AXaQfSxyAaN0mlDQZ+LVxB8Ky/pmD3B6cGqXDqJbx5dKVAqk/eTIlpKBbDjEqe2YfFj0j5VUVfX0Xx6a35t/7GCICoNVy3blLghKrIE3y2z7yETsEHewqJL+pz+7uMcnxwuK6xA4FzVpwkFPNMCHHrKQHl/FItHTaauoWtuSfVuJGx9K7NvoZdax72k7qpfFLedtsAnqLnHtfH4nYWOO4uoDoFv8uyGVqKTWkdPZHOLK+BGHI6cg8kZjp7dvqQSwpYq51u+ylLo4e4l3OzvMYoh8n+xhDWdSeFi2cuSt0w0mQ+KQWSurB8oQLiJsB9+7N4/GlpZuNzjdcPPbSxtjri6b7OkPCd6+SVeHNJcog6mRhSs2lV/G/bfC48wmS7UnzGPX1RCctynH+SpjytjsoKy0LIpafpHYo5Iej5RNx+Jau04s//6T6XG+L4bqdAMK6sPm1yVVtejlqukMLyha3tUQ3XIGur/LqOV1bzcyr42dUUeel9SOgkg3bMIN9xpEeOFEQqBnNvmmQ12PPZUQJBEfi3DYTC2eh9xAewJkkoJuIKwAwZB7eaCpN9+/9chtJkhJPtZxIXq8pG9MRjok7l3Te57DxIwE+pXrL9p9YEOoUgdFq7IKRDFeP0yn5OuYmM+XKGSuC4g0l DWsBnH6W 34mG110U7vZkWqPYzKaH+oHNal2BsVJOkPS64fQmIzgzcVJ1X/qLbt+C94AYz2Q/fdkymHFP7V25oCuo/twHnzDlyt5b7YO5WyPvKzwcLIa4o64PJTTSmowXP4Sj5Uhm/YtD1aOYtgFDL1pXoGpPNu6ObZSLAiTR6w0dIZ85g0HuPGLL2DsKMMoS5+cvscEyvpW8CXQA7UElc/eLUVBrb6YiYrfScdJ4Ce68+tOTY8w6K5Ahhdre0TRIn1wGyofJW+x5jC08+OOde9DNO/td6OlQyGA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Normal THP cannot be PUD-mapped (besides devmap), but hugetlb can, so create mincore_pud_range in order to handle PUD-mapped hugetlb vmas. Signed-off-by: Oscar Salvador --- mm/mincore.c | 20 ++++++++++++++++++++ 1 file changed, 20 insertions(+) diff --git a/mm/mincore.c b/mm/mincore.c index 5154bc705f60..786df7246899 100644 --- a/mm/mincore.c +++ b/mm/mincore.c @@ -98,6 +98,25 @@ static int mincore_unmapped_range(unsigned long addr, unsigned long end, return 0; } +static int mincore_pud_range(pud_t *pud, unsigned long addr, unsigned long end, + struct mm_walk *walk) +{ +#ifdef CONFIG_PGTABLE_HAS_HUGE_LEAVES + spinlock_t *ptl; + unsigned char *vec = walk->private; + int nr = (end - addr) >> PAGE_SHIFT; + struct vm_area_struct *vma = walk->vma; + + ptl = pud_huge_lock(pud, vma); + if (!ptl) + return 0; + + memset(vec, 1, nr); + spin_unlock(ptl); +#endif + return 0; +} + static int mincore_pte_range(pmd_t *pmd, unsigned long addr, unsigned long end, struct mm_walk *walk) { @@ -175,6 +194,7 @@ static inline bool can_do_mincore(struct vm_area_struct *vma) } static const struct mm_walk_ops mincore_walk_ops = { + .pud_entry = mincore_pud_range, .pmd_entry = mincore_pte_range, .pte_hole = mincore_unmapped_range, .hugetlb_entry = mincore_hugetlb, From patchwork Thu Jul 4 04:31:19 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oscar Salvador X-Patchwork-Id: 13723210 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id D210FC30653 for ; Thu, 4 Jul 2024 04:33:22 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 687EF6B00BD; Thu, 4 Jul 2024 00:32:34 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 611556B00C0; Thu, 4 Jul 2024 00:32:34 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 4430A6B00BF; Thu, 4 Jul 2024 00:32:34 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 1EC2F6B00BD for ; Thu, 4 Jul 2024 00:32:34 -0400 (EDT) Received: from smtpin21.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id C9982140A71 for ; Thu, 4 Jul 2024 04:32:33 +0000 (UTC) X-FDA: 82300798986.21.18BC276 Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.223.130]) by imf25.hostedemail.com (Postfix) with ESMTP id B3699A0002 for ; Thu, 4 Jul 2024 04:32:31 +0000 (UTC) Authentication-Results: imf25.hostedemail.com; dkim=none; spf=pass (imf25.hostedemail.com: domain of osalvador@suse.de designates 195.135.223.130 as permitted sender) smtp.mailfrom=osalvador@suse.de; dmarc=pass (policy=none) header.from=suse.de ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1720067533; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Famt0o018CzjK/D0oDe+3Tk0JBUDuAsygB5SDTmdZI4=; b=hRJ3YG1v/ijxjIcrxjDt4jkYHkWCTMalbMwWBhfdDGcP1fzVRNt6XASopPZamxE8vomOqc 618hN4zkUWyx/CPr7FN/J1GvRxLVIxUFe3g5Pj7Bi3IR9snbCpbxfhZPmFsP19xftN0GTE ahHtE2p0SQaxTVbsupzzG+p/b9CtKi0= ARC-Authentication-Results: i=1; imf25.hostedemail.com; dkim=none; spf=pass (imf25.hostedemail.com: domain of osalvador@suse.de designates 195.135.223.130 as permitted sender) smtp.mailfrom=osalvador@suse.de; dmarc=pass (policy=none) header.from=suse.de ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1720067533; a=rsa-sha256; cv=none; b=vqn1CDdNKCORh+gGPfWwq8Un4cFQfCpw+9S2duhiv0OvGpAzve2I27Jn5FOlqdo7Qg9DhO cLRfYdm8e75DLBy42zT0G27mBIGkdrcK8ubYBdca+lgmqrwTFOY4HJtOUsUFE4yzn/4muX xpCxfIZ4MtbaudcKS3DNGJO9Oa80IHw= Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [IPv6:2a07:de40:b281:104:10:150:64:97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id 8B3D621180; Thu, 4 Jul 2024 04:32:30 +0000 (UTC) Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 7235D13889; Thu, 4 Jul 2024 04:32:29 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id 0HweF90lhmbnDAAAD6G6ig (envelope-from ); Thu, 04 Jul 2024 04:32:29 +0000 From: Oscar Salvador To: Andrew Morton Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Peter Xu , Muchun Song , David Hildenbrand , SeongJae Park , Miaohe Lin , Michal Hocko , Matthew Wilcox , Christophe Leroy , Oscar Salvador Subject: [PATCH 32/45] mm/hmm: Enable hmm_vma_walk_pmd, to handle hugetlb vmas Date: Thu, 4 Jul 2024 06:31:19 +0200 Message-ID: <20240704043132.28501-33-osalvador@suse.de> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240704043132.28501-1-osalvador@suse.de> References: <20240704043132.28501-1-osalvador@suse.de> MIME-Version: 1.0 X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Rspamd-Action: no action X-Stat-Signature: c5a5ws7g19jbfaefk5nbpbxypr664nm3 X-Rspam-User: X-Rspamd-Queue-Id: B3699A0002 X-Rspamd-Server: rspam02 X-HE-Tag: 1720067551-831683 X-HE-Meta: U2FsdGVkX18V6YhZ9V7ACBsI/0dBA7+yEqFQTo9vkN5LbIplJUPvyKoDTLqTwYAlLbqBjM1vUG+/kCODNjmVSgS+IubOYE2CsugrsFXZuVusws7/H58YaTn2fMGscF9qv9DYwLaCaH57r3CnKu6RnQbOEEwyZTLC/LbFAkPK1IAEjFAfV5SrB+pbdx6MNPjNZvNmTt5TQmdcBS29/uTcsOdDNMyNgQitK4E5ItrihSo269ySYR7huAGGaqsnwNC4axqLQ0nNjkPcxvOoO2zbAbLogwUAVf/U5/AgFkvDz61ihASzQ+azUVbJGURQ3G8mbQ8MOplAMcsnnJZgOsfBnCacA0dfsxmJ2iuoX3Qz1CrjDOC+I4pnFVzLVqhowvKRXXDBh2YNkJr1ZK6bn7NkCX/Y/odRNUycRS4xsRQ0ZZ5rKehZJqgZR7XWb/0tsTATozW3DElCFOc2B2Mh972slitBAXYm07cpgu01jtN9V6FtIHmwcO10GqjRcCTN6exVR0dytVqg8f5B5fVpWPIGxu4E21jCtEM+Dq9kgFEmUO0NDWNLUR1Y1pF2/S9BEpOkhPNAbl1a/GiRkzgEAH8OH1ZaZntpqKEkhWAXN0VSSOnTjqh5RStah1r316eFj9cezni170H1TOkkWGGjeE/OidgWyrOY2a6Hj9+0lJD+igFRTUVLKUDgoPKKwSEGmcX1KwUIMPmSVq4+xj0Z+RlNioIaMbupK+ZsaFkhGq/RBdaarYGj2IlJZJN1hxUAH7OV5c498T7qluj9ExXLvepRvlEttXPre8rvfirW3NMUHdTmIon31f8n4tnAafZ0UGAsQFjOZeSMWRSbHBm8WOo0sV3EYyJ28DfxPbpDriL/NP38odIN/BIIbOhs2b2Q9gb3IA7tRvpz6JsJBvPUtf+0uEf/Qynh0nhCVHWAiHY3g+9V1WQjnHq6FaeOk5Gj7+b2qeA3wci+WlIS058cUVs D3veyEVa d/5i5ehi91r5et+CnrNuNww+jpJUxLv23pcfC3b4hQ2l6Wv955bgdbTbmGjOPQjK76AHJ2snHjIiCAo83ALhYytu3zx97NqJ9cfv2INddHZHBFX0SkKftcyfBDdmMJExnLyFjvj5w5RJUOxrpyD7qI3XUuiV9UDaCFjcxhEvZmZOXrbc= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: PMD-mapped hugetlb vmas will also reach hmm_vma_walk_pmd. Add the required code so it knows how to handle those there. Signed-off-by: Oscar Salvador --- mm/hmm.c | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/mm/hmm.c b/mm/hmm.c index 7e0229ae4a5a..fbee08973544 100644 --- a/mm/hmm.c +++ b/mm/hmm.c @@ -183,7 +183,7 @@ static inline unsigned long pmd_to_hmm_pfn_flags(struct hmm_range *range, hmm_pfn_flags_order(PMD_SHIFT - PAGE_SHIFT); } -#ifdef CONFIG_TRANSPARENT_HUGEPAGE +#ifdef CONFIG_PGTABLE_HAS_HUGE_LEAVES static int hmm_vma_handle_pmd(struct mm_walk *walk, unsigned long addr, unsigned long end, unsigned long hmm_pfns[], pmd_t pmd) @@ -206,11 +206,11 @@ static int hmm_vma_handle_pmd(struct mm_walk *walk, unsigned long addr, hmm_pfns[i] = pfn | cpu_flags; return 0; } -#else /* CONFIG_TRANSPARENT_HUGEPAGE */ +#else /* CONFIG_PGTABLE_HAS_HUGE_LEAVES */ /* stub to allow the code below to compile */ int hmm_vma_handle_pmd(struct mm_walk *walk, unsigned long addr, unsigned long end, unsigned long hmm_pfns[], pmd_t pmd); -#endif /* CONFIG_TRANSPARENT_HUGEPAGE */ +#endif /* CONFIG_PGTABLE_HAS_HUGE_LEAVES */ static inline unsigned long pte_to_hmm_pfn_flags(struct hmm_range *range, pte_t pte) @@ -336,7 +336,7 @@ static int hmm_vma_walk_pmd(pmd_t *pmdp, if (pmd_none(pmd)) return hmm_vma_walk_hole(start, end, -1, walk); - if (thp_migration_supported() && is_pmd_migration_entry(pmd)) { + if (is_pmd_migration_entry(pmd)) { if (hmm_range_need_fault(hmm_vma_walk, hmm_pfns, npages, 0)) { hmm_vma_walk->last = addr; pmd_migration_entry_wait(walk->mm, pmdp); @@ -351,7 +351,7 @@ static int hmm_vma_walk_pmd(pmd_t *pmdp, return hmm_pfns_fill(start, end, range, HMM_PFN_ERROR); } - if (pmd_devmap(pmd) || pmd_trans_huge(pmd)) { + if (pmd_devmap(pmd) || pmd_leaf(pmd)) { /* * No need to take pmd_lock here, even if some other thread * is splitting the huge pmd we will get that event through From patchwork Thu Jul 4 04:31:20 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oscar Salvador X-Patchwork-Id: 13723211 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 93CB5C30653 for ; Thu, 4 Jul 2024 04:33:25 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id B402C6B00C0; Thu, 4 Jul 2024 00:32:35 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id AA25C6B00C1; Thu, 4 Jul 2024 00:32:35 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8574B6B00C2; Thu, 4 Jul 2024 00:32:35 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 60B2D6B00C0 for ; Thu, 4 Jul 2024 00:32:35 -0400 (EDT) Received: from smtpin02.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 18B38120A39 for ; Thu, 4 Jul 2024 04:32:35 +0000 (UTC) X-FDA: 82300799070.02.F0D36B0 Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.223.131]) by imf24.hostedemail.com (Postfix) with ESMTP id 19625180016 for ; Thu, 4 Jul 2024 04:32:32 +0000 (UTC) Authentication-Results: imf24.hostedemail.com; dkim=none; spf=pass (imf24.hostedemail.com: domain of osalvador@suse.de designates 195.135.223.131 as permitted sender) smtp.mailfrom=osalvador@suse.de; dmarc=pass (policy=none) header.from=suse.de ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1720067528; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=wxCRzy9udt8uFnD1otT2eiNuqHqdgC2Y99Too233j8w=; b=zjLW7bmg25nys+CZscLbK+N3ulo24JQ0J/ID0yVlXu9hZgq5dXqrYsBK902UD1BGnPf1R8 5vsIa6SWPDtlMmO0L/Myuah3AZ99APHw7cUQebch/i1ooLhk7SxNdT1tIxODXzdBkpNOWh fPJP+kqSNFpzd4Aw313xxbt0zrrdOyE= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1720067528; a=rsa-sha256; cv=none; b=Azy/HqDoHPPitSGjg06Sj8DqZ4klTkeqDqbLFCSGgG+sQp/3HyqwBvrj2v5opwPEA97PH7 4wWywyU4JIgusaU8gUVj6ID0p1VyD58ybYkUgzlse5LdBmpynJ5Zjg8dL5kmQ3lgV9GM6v F1T5laZfcsnjJS/Muy+ZpG0nqwcYoBQ= ARC-Authentication-Results: i=1; imf24.hostedemail.com; dkim=none; spf=pass (imf24.hostedemail.com: domain of osalvador@suse.de designates 195.135.223.131 as permitted sender) smtp.mailfrom=osalvador@suse.de; dmarc=pass (policy=none) header.from=suse.de Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [IPv6:2a07:de40:b281:104:10:150:64:97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id D6F111FCF7; Thu, 4 Jul 2024 04:32:31 +0000 (UTC) Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id B441C13889; Thu, 4 Jul 2024 04:32:30 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id 8OBhJ94lhmbnDAAAD6G6ig (envelope-from ); Thu, 04 Jul 2024 04:32:30 +0000 From: Oscar Salvador To: Andrew Morton Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Peter Xu , Muchun Song , David Hildenbrand , SeongJae Park , Miaohe Lin , Michal Hocko , Matthew Wilcox , Christophe Leroy , Oscar Salvador Subject: [PATCH 33/45] mm/hmm: Enable hmm_vma_walk_pud to handle PUD-mapped hugetlb vmas Date: Thu, 4 Jul 2024 06:31:20 +0200 Message-ID: <20240704043132.28501-34-osalvador@suse.de> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240704043132.28501-1-osalvador@suse.de> References: <20240704043132.28501-1-osalvador@suse.de> MIME-Version: 1.0 X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Rspamd-Action: no action X-Stat-Signature: w5ng35pi5uy5bnw4enwriix71ftfkekb X-Rspamd-Queue-Id: 19625180016 X-Rspam-User: X-Rspamd-Server: rspam08 X-HE-Tag: 1720067552-851387 X-HE-Meta: U2FsdGVkX1/OaNUpaXMgraQzZWrPYacg1u173uTJ0aI+ter7o5UNRHz8rWu15FpkErKJ3BawTdIiiSt+Db7SboFojgP9F541SuyNmRavvm0pq9fEQmZAU/X4+tLyMcTRSKYIao+7QjrXIc66yOfq+hwk7FKtppIASKVbfQf3O0LJ8ZubyRVPxRblrkzouBaAbqlnhZv9kFDL4BPYLxXxUGX4zOqlrnR0zCT1QGzrG3HDpNjH56poJG6+VV5Tv8GGg0lHSNRj/w2+lwqx7UW8Zq6f9D7wZ3L81appYU5je7wCV1dWIVUk5TPZ0DgN/f4toJFsU7s9Zot3q2gXtOS0iS/UXDaiyBosXq82WxvXm1zIsRI0cMKeeWu98q3hPiXJc1gO7fds5TQJHFoe0qcEr0vJKl2jDKNrvn3Qrp/9Ee2GBrm3HeCKA31qFqtqOCvd6UbL54t/bJkwFS0aVusGxiNHvYS119oyMkrURl2EJah/tLb93PQrZ8P6yLl4sh/0l+KOiajMuLeK3wwHoa+vTJ5jej09QQpDGk9qWuzaafkucUUcesvO/tAfaEExDW6PWHWkPIs8ozr+QewOfwvLHvAtgORUlv4qe9BDBlbMFv8YySke0zVJ8Y4x6XD8ClM9pC5RXFyZXiRXMXXHv7dSLoI73uj5WSXOqF1sh+ONhXNWyTHN5xYaZ7aDDGvO64W6natjSuuvxGea1dmHAMVtvBtSHcfk1WzAE7sqrrCBxe1sA5j/CG4eAlssgdpS+enakAgsobsZTPE3Lb/0nyyHBYDkzexjJBkJCd9eHTCDnH0IwB8XtrP8NLVEZfcU/k+Fd7iISAdaPeK+yDs30uA9mwCSlQQP8g7pRTwtLgce46h4wjmR9L5e2wXoqfgQSrpotYu3Un9m2tgPIKWp1d0wFMeM7BvT0OGeEClCiCq20srXgmCzcaHve+FPWrHy9h3njqMGIzCvYGYK9k47AQU a82u7Sv9 r2uc8QPETCr8RHO7irZ++nm4+KtYg5O14SVkYjGPsDy26loNd8tYTgvQXMTpinamk2q4QLoh11yTByCDriTWIr6RXSUnY+bf4rKrEiUU1LbLmK4HeUAJihXmsjHiFe+1HqV06BYEGU5BX7ZHAt5NPu2X0bUfP5AiCgvVvG/Xvl/TXL3g= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Normal THP cannot be PUD-mapped (besides devmap), but hugetlb can, so enable hmm_vma_walk_pud to handle PUD-mapped hugetlb vmas. Signed-off-by: Oscar Salvador --- mm/hmm.c | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/mm/hmm.c b/mm/hmm.c index fbee08973544..2b752f703b6d 100644 --- a/mm/hmm.c +++ b/mm/hmm.c @@ -396,8 +396,7 @@ static int hmm_vma_walk_pmd(pmd_t *pmdp, return 0; } -#if defined(CONFIG_ARCH_HAS_PTE_DEVMAP) && \ - defined(CONFIG_HAVE_ARCH_TRANSPARENT_HUGEPAGE_PUD) +#if (defined(CONFIG_ARCH_HAS_PTE_DEVMAP) || defined (CONFIG_PGTABLE_HAS_HUGE_LEAVES)) static inline unsigned long pud_to_hmm_pfn_flags(struct hmm_range *range, pud_t pud) { @@ -429,7 +428,7 @@ static int hmm_vma_walk_pud(pud_t *pudp, unsigned long start, unsigned long end, return hmm_vma_walk_hole(start, end, -1, walk); } - if (pud_leaf(pud) && pud_devmap(pud)) { + if (pud_leaf(pud)) { unsigned long i, npages, pfn; unsigned int required_fault; unsigned long *hmm_pfns; From patchwork Thu Jul 4 04:31:21 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oscar Salvador X-Patchwork-Id: 13723213 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id DB848C30653 for ; Thu, 4 Jul 2024 04:33:30 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 7225E6B00C2; Thu, 4 Jul 2024 00:32:38 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 6ACC36B00C4; Thu, 4 Jul 2024 00:32:38 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 4B3EE6B00C6; Thu, 4 Jul 2024 00:32:38 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 0EA296B00C2 for ; Thu, 4 Jul 2024 00:32:38 -0400 (EDT) Received: from smtpin12.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id C0506140A71 for ; Thu, 4 Jul 2024 04:32:37 +0000 (UTC) X-FDA: 82300799154.12.953F31A Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.223.131]) by imf06.hostedemail.com (Postfix) with ESMTP id 9AC3D180015 for ; Thu, 4 Jul 2024 04:32:35 +0000 (UTC) Authentication-Results: imf06.hostedemail.com; dkim=pass header.d=suse.de header.s=susede2_rsa header.b=sTaFfZGs; dkim=pass header.d=suse.de header.s=susede2_ed25519 header.b=CWK1uG79; dkim=pass header.d=suse.de header.s=susede2_rsa header.b=bqsFGOa2; dkim=pass header.d=suse.de header.s=susede2_ed25519 header.b=B6iVvmiy; dmarc=pass (policy=none) header.from=suse.de; spf=pass (imf06.hostedemail.com: domain of osalvador@suse.de designates 195.135.223.131 as permitted sender) smtp.mailfrom=osalvador@suse.de ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1720067528; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=sh4f2O4+wRTS3lNqvGwH972LquLk6Sxiod9/OijqLHw=; b=R3Pujyjxs5eGYyDC+3yOM5947JDfnqN96xrjjFexmKhT7tEojIO81VLzA0MPbHaKnCGI9E 9Xw+YMUiYWEGnFluYY07EuwxduWstwNLpb5KAT3QWNuaLO8aZkR61mCd6xLtdPSd5k9yBk bx1Y7iUE3gFYNhvw1xF7k5qzsCoIRmY= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1720067528; a=rsa-sha256; cv=none; b=zPtae2ObxnL6SJ2w0d5MYjS6CevXdBIP4g6kPbfvVfkpjU+3XE/Tp9dcQQ9YjQ8HL8AliA R/IFN7ssockhAE8EsTGivcl1ZXIUg75VuCy0mTItv0RKv2feKq1H6Z2CpwLHaoFhios3b2 NLAzkyDf+U5/EqSSB37M+Vp6qY5A/PA= ARC-Authentication-Results: i=1; imf06.hostedemail.com; dkim=pass header.d=suse.de header.s=susede2_rsa header.b=sTaFfZGs; dkim=pass header.d=suse.de header.s=susede2_ed25519 header.b=CWK1uG79; dkim=pass header.d=suse.de header.s=susede2_rsa header.b=bqsFGOa2; dkim=pass header.d=suse.de header.s=susede2_ed25519 header.b=B6iVvmiy; dmarc=pass (policy=none) header.from=suse.de; spf=pass (imf06.hostedemail.com: domain of osalvador@suse.de designates 195.135.223.131 as permitted sender) smtp.mailfrom=osalvador@suse.de Received: from imap1.dmz-prg2.suse.org (unknown [10.150.64.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id 215491FCFB; Thu, 4 Jul 2024 04:32:33 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1720067554; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=sh4f2O4+wRTS3lNqvGwH972LquLk6Sxiod9/OijqLHw=; b=sTaFfZGsh0QN0KmSZGZupmvc9+i8aT6yh6wqcxWRjw+XHu1zEisZrGN4NNNyCZDSvyJGEU PFjK4MlwkHT8GncglykUIdk+8ISfhp13f+z8nTbaYj1t2I52ze+OrMapZLKDq37mK/EAY8 up94b5BLuEXKnEEwf9Dw0eVYsTwPEig= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1720067554; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=sh4f2O4+wRTS3lNqvGwH972LquLk6Sxiod9/OijqLHw=; b=CWK1uG79vcWta/RwK11oIoeDmAvTnhOeCFGg5+Xlvmv5A/bL50Ok8Y/E9WOG3clC5ufnFC Qi1EyHmHHAYPxtAw== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1720067553; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=sh4f2O4+wRTS3lNqvGwH972LquLk6Sxiod9/OijqLHw=; b=bqsFGOa2yGJrOu6R6Wz97cBOkJkF3/uec0u9bnCDleCJ/vApP0GE8p/TCajg1w1b3lZVJQ Wb9dzSwUbpFaSXKXl4Q6eXjlUVVBl4JNvifi+kag3LCsXlCHFKonxim9J8IEGBEX9C5QV0 GiEpgBU7SWbf0a+5Jsh7OpzPXIYc/BU= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1720067553; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=sh4f2O4+wRTS3lNqvGwH972LquLk6Sxiod9/OijqLHw=; b=B6iVvmiyIQfFPvvBsrtHLZj8Lm+q7MF8LvHMT8cMfsKfCmslqOG4WdKgUVsuxWbCfOn648 Wa2GOQPr90SaqfAw== Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id F0D7D13889; Thu, 4 Jul 2024 04:32:31 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id OPOFNd8lhmbnDAAAD6G6ig (envelope-from ); Thu, 04 Jul 2024 04:32:31 +0000 From: Oscar Salvador To: Andrew Morton Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Peter Xu , Muchun Song , David Hildenbrand , SeongJae Park , Miaohe Lin , Michal Hocko , Matthew Wilcox , Christophe Leroy , Oscar Salvador Subject: [PATCH 34/45] arch/powerpc: Skip hugetlb vmas in subpage_mark_vma_nohuge Date: Thu, 4 Jul 2024 06:31:21 +0200 Message-ID: <20240704043132.28501-35-osalvador@suse.de> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240704043132.28501-1-osalvador@suse.de> References: <20240704043132.28501-1-osalvador@suse.de> MIME-Version: 1.0 X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: 9AC3D180015 X-Stat-Signature: ezjq3pumns7yqbznm5891rpz8b7b73rp X-Rspam-User: X-HE-Tag: 1720067555-305210 X-HE-Meta: U2FsdGVkX1/V9qqz/3LUkUUF6RCGzdynsVWLe1ViCOUIO3bsce+V46C6xuFDl/P55fNpU7ebS7O9YMGGEQVJ5ryC6QFu3yRX0YoiHmQs6ijhBk9l49GsT+sEE+jFQCCpBQ+OLyxusnV/JplCDw/DGiHAS2Q/89CkDK/a31mI4b9lY2Q7TA4jqi+xzUODML9ikJ/TU7Z9itMPMoXkk0w/57ExRCn+TFdQj1hwXNai/CIBsE86aJUkP1Su8bzklB+xia4cGrL1VKHUBTFinwOmzgzwyASt5xweM/vyQAhR+ZZdXqFicE7EhrAAXB7xJx1hUjq18akz+bH9cpyKnXwfaYF00XW9XWprj4J1EVbffLenvOGaEu6rmyR0RPSgpQ6i2ImT1+XPYmBippeO7mIfoje6VQ24pBxDNsRSWB0yEuYZkoo4tddTfAqC92A4wMJS6mzZ/czdjzscW2PVokAY9/ZkrpMWMTu8pIxy6Jt5hnuc6wZ/bHhE1FQmlNcI+7NE84w+PEsW/yoNn5DDY9SSdVEmvjkKAwLXYnfSV44jGsSTs1twQhD9EsYSpVuVOn3shZQrhuQ2Berdnqqcd2yLREQVoAvBCz25M9HB21dOgU9qLlYamLN/Bjeji2LKT1Wpw6A4n2ChVpiycKcv/jsz1BUh/OVL06B57N7rnO28KQsrV/1ldCE8fUnip/GLIJiEDgI2xDr0QPVBtR9IiMAO08HNjXeyH8xm/0E+rYQlJLgf1jMwO1//kGccCiG09Nw2lyisciEs5PVgUJCcHYGe8/8TmGAcrh3lRtvFiMl6HoMSbQfyxvISlxw1dMXWQovaxdnykS/VGOYEs+hYwNdjrRqEmloQbwYy302FvRa6vQadX+/j7uGH9LQDPifw+CiGKJhlKl8V4arzqhlO+lZOl5Nw3QqaX5xThDyzPnVLOmohBLJFDjOhpRJQ7x6z2/0l1rzVXgdKPXk4ENN/jIT KF94HKSl hJzO6fMq+UXZj3EFT/yWFd/hNCJGeVEKpmaPyhv//nEdT+3Lh/eyPY+oWG//nabAR6saGczc52D/i75KCaQHEEa13D0FaIZUTz0RqK1ylm13XUGLWjeegTH0sfh96LZ/1zVTDUVmX5zyA7sM+yHSlAmpoDmD//OcLAtUxIFzeXLuBFoDBdGIN/gc2l7IHP9MyOyWWu4YgHPEZAJQvKJIhcfrgmHiHImIky1j+TlT97X/Y/bekjj0c0qN/xi3aSg/VrH/dXl/ZG2FR+grnyzeITVtFmg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Skip hugetlb vmas as we are not interested in those. Signed-off-by: Oscar Salvador --- arch/powerpc/mm/book3s64/subpage_prot.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/arch/powerpc/mm/book3s64/subpage_prot.c b/arch/powerpc/mm/book3s64/subpage_prot.c index ec98e526167e..dd529adda87f 100644 --- a/arch/powerpc/mm/book3s64/subpage_prot.c +++ b/arch/powerpc/mm/book3s64/subpage_prot.c @@ -159,6 +159,8 @@ static void subpage_mark_vma_nohuge(struct mm_struct *mm, unsigned long addr, * VM_NOHUGEPAGE and split them. */ for_each_vma_range(vmi, vma, addr + len) { + if (is_vm_hugetlb_page(vma)) + continue; vm_flags_set(vma, VM_NOHUGEPAGE); walk_page_vma(vma, &subpage_walk_ops, NULL); } From patchwork Thu Jul 4 04:31:22 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oscar Salvador X-Patchwork-Id: 13723212 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6EA8EC30653 for ; Thu, 4 Jul 2024 04:33:28 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 2DCD16B00C3; Thu, 4 Jul 2024 00:32:38 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 265926B00C5; Thu, 4 Jul 2024 00:32:38 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id F0E9D6B00C4; Thu, 4 Jul 2024 00:32:37 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id CFECC6B00C2 for ; Thu, 4 Jul 2024 00:32:37 -0400 (EDT) Received: from smtpin27.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 9427B140AC0 for ; Thu, 4 Jul 2024 04:32:37 +0000 (UTC) X-FDA: 82300799154.27.A01FCCB Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.223.131]) by imf14.hostedemail.com (Postfix) with ESMTP id A2FF510000F for ; Thu, 4 Jul 2024 04:32:35 +0000 (UTC) Authentication-Results: imf14.hostedemail.com; dkim=none; spf=pass (imf14.hostedemail.com: domain of osalvador@suse.de designates 195.135.223.131 as permitted sender) smtp.mailfrom=osalvador@suse.de; dmarc=pass (policy=none) header.from=suse.de ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1720067543; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=lSf4BKmGPnBihL3uAt2bMjOwktH+wXyYKX1193FAJw8=; b=pGay7ft7uhkin3fsIWPIF3cbcMRKfONulmZPnf77vpyZN/5kCAROMMsvzElHv36jthA2j1 6SG/05j/J5d9iWdj+dhp48I2leQk+AbhJfXqPbklNzwJrt9bZnKD+7z2UqeEAPuhKWwdbp mof3/VbG+C0eOtLNT2LV2UPGp8eiTNs= ARC-Authentication-Results: i=1; imf14.hostedemail.com; dkim=none; spf=pass (imf14.hostedemail.com: domain of osalvador@suse.de designates 195.135.223.131 as permitted sender) smtp.mailfrom=osalvador@suse.de; dmarc=pass (policy=none) header.from=suse.de ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1720067543; a=rsa-sha256; cv=none; b=uMFRSp8JZeRP07UpSIOqsZCvyB458KVoGcpwPo2Jby6qEn/ReKAc78ccrdrkCsxN3C+ZAH 44ZJwOd/M+NBeJvCUq55NYJcek/i5U3DWlvFupwxSvePX+MzdOojfFVEbHz2sMEX4q6DZT r1J6Gb8AZFO5ZCqYYA+S+fP5LKLy8FQ= Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [IPv6:2a07:de40:b281:104:10:150:64:97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id 6F06E1FCFA; Thu, 4 Jul 2024 04:32:34 +0000 (UTC) Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 3F79513A82; Thu, 4 Jul 2024 04:32:33 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id eMdVC+ElhmbnDAAAD6G6ig (envelope-from ); Thu, 04 Jul 2024 04:32:33 +0000 From: Oscar Salvador To: Andrew Morton Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Peter Xu , Muchun Song , David Hildenbrand , SeongJae Park , Miaohe Lin , Michal Hocko , Matthew Wilcox , Christophe Leroy , Oscar Salvador Subject: [PATCH 35/45] arch/s390: Skip hugetlb vmas in thp_split_mm Date: Thu, 4 Jul 2024 06:31:22 +0200 Message-ID: <20240704043132.28501-36-osalvador@suse.de> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240704043132.28501-1-osalvador@suse.de> References: <20240704043132.28501-1-osalvador@suse.de> MIME-Version: 1.0 X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Rspamd-Action: no action X-Rspamd-Server: rspam03 X-Rspam-User: X-Rspamd-Queue-Id: A2FF510000F X-Stat-Signature: whjsgb9wakax5138c4j6t6iqzqidnkgt X-HE-Tag: 1720067555-99674 X-HE-Meta: U2FsdGVkX1/IERPA0FhJComCaRf8yNfGvZGYUPIhMIOTegroAHQnxLJ1ZkGjqD5t35kpzOBjQW2VvxgAGmPc83wHgfsLOzB4bkacFjDSPQUYOrNtL/N3Kl09QMqLA3SJ9yrkZAaiwiXrEMo/T/DOhw1Mu/aguE5PQosuRcdMKM50CPtj9G9KJ4gk8TCwwPNpeGwzgO4fU3foRlH8GgJKbHSanrf6FOw+SrRQurFC0//okTFcaMf8ndnL4Z9QpXl5DPuQvMlqJVjaLknjnOtaVoX0owVLq4s2vcvkoAdqMi9yGvP5Ip3T6V2hJb/WZN/y+r7hTNxsTQHOPNzaCM9t26Y04UmrHK5V1IUfP1a1XAJAc2ArvFcJiboxOdgGyUy+T+2CWi7+k45FrybuWLpUZtmVtMkEe9hOb0zdXFHcxekLkYiB4PIz0vyNYNOpMZt5MlMVehd0c0kSEDs1Rh/GGIlmTTpr+jmI3Xqiioa5liYg2yRPteVEKX1PadhJ9a72qv7V7J3NmfHE/cfr6NVKtWi3fhQQYGWuSLllTUx6Un+27u3H6+hE6Dw5Ri/SaipweKf54b6ih5YRNGEbjAC0M/Zak/ki8T97UdO1SHr208beiKtvsqKfCmqwhhQiwmTZ5ozuAw8Kqt+lpEAUK3J7r8Xy4zT+RMnqgZvLozf/BktPMLt3XTc8voNV0lTTC/CL8MZGX/fsqu8GHo3Dyvw24Cbkem666sfLnF6IeEzWz0I4ieaRZZgk6XQ+JneBwAUZNm1oFFyT8KpupIycBXSMG9P1v/M+KWttZWUytzc9xc//XmLw3kakonnLPWvQRgHuRa2wSTrWLfR0FblY7iAGs/y3/FNafJTedBUCckr0aHVUF54cuRX9Pvl76sUhKyW1b49dcqAmdrooWvIFynpeHGRt+n9U42iAk4bzIMecTaKlT6WLkAqZSQLJ7REAJC27byJuMYewV08MDhjwQcE CiFU6dGz FrBMa+DtN0iOGfQ91EE6dWsPoX7bwEcGXHVae9uQ8EEDWd4qtn53/Uhav3eeBErGugjFHkRaS0hYC4SRd5b9BZfRnk6FTnjL3lqAt5j0ACoa/oLdrJh7DGSPMJ5VQ/eN9Jso7Tjqq6iXMYL/+GYRCbAdsHcAhbrPsUQ+pYSbz9TL6e6g= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Skip hugetlb vmas as we are not interested in those. Signed-off-by: Oscar Salvador --- arch/s390/mm/gmap.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/arch/s390/mm/gmap.c b/arch/s390/mm/gmap.c index e1d098dc7f07..580e4ab6f018 100644 --- a/arch/s390/mm/gmap.c +++ b/arch/s390/mm/gmap.c @@ -2538,6 +2538,8 @@ static inline void thp_split_mm(struct mm_struct *mm) VMA_ITERATOR(vmi, mm, 0); for_each_vma(vmi, vma) { + if (is_vm_hugetlb_page(vma)) + continue; vm_flags_mod(vma, VM_NOHUGEPAGE, VM_HUGEPAGE); walk_page_vma(vma, &thp_split_walk_ops, NULL); } From patchwork Thu Jul 4 04:31:23 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oscar Salvador X-Patchwork-Id: 13723214 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4C2F0C30653 for ; Thu, 4 Jul 2024 04:33:33 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id ACAC26B00C4; Thu, 4 Jul 2024 00:32:39 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id A544D6B00C5; Thu, 4 Jul 2024 00:32:39 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 859416B00C6; Thu, 4 Jul 2024 00:32:39 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 5BCDD6B00C4 for ; Thu, 4 Jul 2024 00:32:39 -0400 (EDT) Received: from smtpin10.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 1909AA3D40 for ; Thu, 4 Jul 2024 04:32:39 +0000 (UTC) X-FDA: 82300799238.10.D2A6D64 Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.223.130]) by imf26.hostedemail.com (Postfix) with ESMTP id ED5E3140003 for ; Thu, 4 Jul 2024 04:32:36 +0000 (UTC) Authentication-Results: imf26.hostedemail.com; dkim=pass header.d=suse.de header.s=susede2_rsa header.b=MBgo3NSc; dkim=pass header.d=suse.de header.s=susede2_ed25519 header.b=sCT0CMUz; dkim=pass header.d=suse.de header.s=susede2_rsa header.b=MBgo3NSc; dkim=pass header.d=suse.de header.s=susede2_ed25519 header.b=sCT0CMUz; dmarc=pass (policy=none) header.from=suse.de; spf=pass (imf26.hostedemail.com: domain of osalvador@suse.de designates 195.135.223.130 as permitted sender) smtp.mailfrom=osalvador@suse.de ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1720067537; a=rsa-sha256; cv=none; b=l0DXrH1giEnJxc8UkrF+L8TD9DX0g4iwjjcRs8Vbe3k2ornWRTtofXBqOdiVWeHw/3X4zw 3gbJdVkCxicMuHH1F66y2KsigN864KfUgdvXvX37qyr+7MxdkTeg8OWlC2PCiwAA58YnEd 5xOmPM6kG5TB7cYkl+TYpcOezbD4rJ0= ARC-Authentication-Results: i=1; imf26.hostedemail.com; dkim=pass header.d=suse.de header.s=susede2_rsa header.b=MBgo3NSc; dkim=pass header.d=suse.de header.s=susede2_ed25519 header.b=sCT0CMUz; dkim=pass header.d=suse.de header.s=susede2_rsa header.b=MBgo3NSc; dkim=pass header.d=suse.de header.s=susede2_ed25519 header.b=sCT0CMUz; dmarc=pass (policy=none) header.from=suse.de; spf=pass (imf26.hostedemail.com: domain of osalvador@suse.de designates 195.135.223.130 as permitted sender) smtp.mailfrom=osalvador@suse.de ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1720067537; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=GKNX/My9OkoTrzpvFpT5zCUWuebklZgLLB1xxXIoHYo=; b=I9QTK02igYyoGNdnB/6OrOXkvTJuoo2gM+briiuVHZFe/DuI3xxd3RvMzfhD7i9NJIn/SV gBK6AGOC3nswR+vnB9ptod2HNYs44sQbiM2cSBYNsLE/ykcctgN9UQxZqvDCVu7/vDZxFc H1sgoQSwUre9WLXmnIsBW8PDYwuFA5Y= Received: from imap1.dmz-prg2.suse.org (unknown [10.150.64.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id B4E2A21180; Thu, 4 Jul 2024 04:32:35 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1720067555; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=GKNX/My9OkoTrzpvFpT5zCUWuebklZgLLB1xxXIoHYo=; b=MBgo3NSczt8U7EEAXcD2SQA8wQ51zLbu6/4GLxv3/acFo/O9aeAjJinYHrahN+wHh6WN6E nJWgfUf/qMJFBilRyvUHpRCrHv3QH2RC9j0K0L3OAEZimxeISUardGz2TZ7zET0RJVlsf9 vPaxOvkBTgVVPFz9W64mTDrSkfrzns0= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1720067555; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=GKNX/My9OkoTrzpvFpT5zCUWuebklZgLLB1xxXIoHYo=; b=sCT0CMUzMRt8XjozOQiz7eH0jI8RX27HURNG1GIXpzDYx1fhHI5xooLjdVheihL1p8vQfG 66YxnUMPC011JyDw== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1720067555; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=GKNX/My9OkoTrzpvFpT5zCUWuebklZgLLB1xxXIoHYo=; b=MBgo3NSczt8U7EEAXcD2SQA8wQ51zLbu6/4GLxv3/acFo/O9aeAjJinYHrahN+wHh6WN6E nJWgfUf/qMJFBilRyvUHpRCrHv3QH2RC9j0K0L3OAEZimxeISUardGz2TZ7zET0RJVlsf9 vPaxOvkBTgVVPFz9W64mTDrSkfrzns0= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1720067555; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=GKNX/My9OkoTrzpvFpT5zCUWuebklZgLLB1xxXIoHYo=; b=sCT0CMUzMRt8XjozOQiz7eH0jI8RX27HURNG1GIXpzDYx1fhHI5xooLjdVheihL1p8vQfG 66YxnUMPC011JyDw== Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 93F1113889; Thu, 4 Jul 2024 04:32:34 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id QADdHuIlhmbnDAAAD6G6ig (envelope-from ); Thu, 04 Jul 2024 04:32:34 +0000 From: Oscar Salvador To: Andrew Morton Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Peter Xu , Muchun Song , David Hildenbrand , SeongJae Park , Miaohe Lin , Michal Hocko , Matthew Wilcox , Christophe Leroy , Oscar Salvador Subject: [PATCH 36/45] fs/proc: Make clear_refs_test_walk skip hugetlb vmas Date: Thu, 4 Jul 2024 06:31:23 +0200 Message-ID: <20240704043132.28501-37-osalvador@suse.de> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240704043132.28501-1-osalvador@suse.de> References: <20240704043132.28501-1-osalvador@suse.de> MIME-Version: 1.0 X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: ED5E3140003 X-Stat-Signature: 7jhxn9tmqcwn45b8pqqigqc8kutmea1g X-Rspam-User: X-HE-Tag: 1720067556-659318 X-HE-Meta: U2FsdGVkX191FinxiCaX14jLBjEoC3fzI2Lfqhh8k+O1+pHgQHOShDjVkNMDdMIltiz6hj++jrmrtJiRLdHiCPQUVDqosZQmY6Hwr+pOWw6Zu/tZIaL7eSZyz9cIenpNpUqufYn7CxqI5LeeLmgRMyGSWcV0znwG7zzcG1ZAyY29eUbhKZQQAuRHvwG7LBUzjLN1Cs9Zi7Wvc1+CvHDz4TUZkJiqPahE2mcJdTY6cAz1Ox3xvzDLSjXvjRW+SHJwhx7D/15xu0HUJwWKSPO96RFWbHzkvjJyhcj72MI5jMSDTafKfTbXIJDfMqbUdzlhOMvXS59zfekj7fYAUsx9mS6rTtlvtwwUXijnCC1OPvRb4MckMu1tFtSiJv5YvKnqo+arnHMFz8vC6am+enj/+NSnqSd5It+TvJ4g3wal8uY/wvQu8XjvvXNn/p+zwNw8mKnhqoWCQnVlp6zbnuyJ9hT65GAX5NC5QgyMDRneD4ecacM3L0Vk1W/0vqZIjhssU0zuWD9vVmfv433laT8g7YXJb5gIdmG9fhyzuh8Y2vGCYWoUIzMUxF8kUipak/7MBwyVwEM5FUfuYq6a42rQ/NjqdZuDzcnHUKZc4XGZfIT1Fjjo/9/tCpLG0XhSiVYh/1F1nto03mzgcPH5tRX1NQHTCXMNYNcZ0c+wsoUMIrFMEutvw75Fl1KjzjSr96ShKV20z7v65k2GjufAkNItFv1Ze321gwi4E53iOwPUTI6abkQHJOkoTG4sj253UColOslcJmRtvt0pVNABlUwolhQ1OuR+2b6hMK7KqiFbFz7mhTuz4qYDKOD8ozELVzseq7ssl0OWcM/8Yem2frREbww3pZQ+LJgrnwMx7lg2LE5LT7+ko5P9dnUcCEV6KAJqbzrgfq+g820TbZR2g62YnMmX5gkqHdDjFttj7ZQeDqd6tkZE/p5vSap9CjLUn0op4pNa79t0QwVn1w0Jso+ Y9kW2ZGf aACQJbX3aXEO+m8o2OdCxdFB87/+wCTmewSXHpKW3j5/m+axayvBQK/QUJRKH5OKXla6Q7RYFqLct2mcPTb0J+51xFTMRI+PyxAQ1xaESTnUZdsLvSe6H+5gZu9pUo606Cee8yuehQMXzXuFmUHEcN6CmXMlUDlL6fC0n2GWa7ZoXOPVT8hvb9IKAY6nnhqu4U+0xmSwMbi5j+hCRmBiIvqk2HjXO7PCjwgF84tEo8SArkDlNIoTIoqWPDnZGicfNG8kxETeYFnaC3EODv7eNaOgoVweNDM6LYzHt X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Skip hugetlb vmas as we are not interested in those. Signed-off-by: Oscar Salvador --- fs/proc/task_mmu.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/fs/proc/task_mmu.c b/fs/proc/task_mmu.c index 5df17b7cfe6c..df94f2093588 100644 --- a/fs/proc/task_mmu.c +++ b/fs/proc/task_mmu.c @@ -1579,6 +1579,9 @@ static int clear_refs_test_walk(unsigned long start, unsigned long end, struct clear_refs_private *cp = walk->private; struct vm_area_struct *vma = walk->vma; + if (is_vm_hugetlb_page(vma)) + return 1; + if (vma->vm_flags & VM_PFNMAP) return 1; From patchwork Thu Jul 4 04:31:24 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oscar Salvador X-Patchwork-Id: 13723215 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id A7291C3065C for ; Thu, 4 Jul 2024 04:33:35 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 12E366B00C6; Thu, 4 Jul 2024 00:32:41 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 0B6726B00C7; Thu, 4 Jul 2024 00:32:41 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id DFC516B00C8; Thu, 4 Jul 2024 00:32:40 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id B8FBB6B00C6 for ; Thu, 4 Jul 2024 00:32:40 -0400 (EDT) Received: from smtpin19.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 77314A2DCB for ; Thu, 4 Jul 2024 04:32:40 +0000 (UTC) X-FDA: 82300799280.19.03C639A Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.223.130]) by imf22.hostedemail.com (Postfix) with ESMTP id 5FBC1C0011 for ; Thu, 4 Jul 2024 04:32:38 +0000 (UTC) Authentication-Results: imf22.hostedemail.com; dkim=pass header.d=suse.de header.s=susede2_rsa header.b=FLLFrGlR; dkim=pass header.d=suse.de header.s=susede2_ed25519 header.b=Xn+qmriL; dkim=pass header.d=suse.de header.s=susede2_rsa header.b=FLLFrGlR; dkim=pass header.d=suse.de header.s=susede2_ed25519 header.b=Xn+qmriL; spf=pass (imf22.hostedemail.com: domain of osalvador@suse.de designates 195.135.223.130 as permitted sender) smtp.mailfrom=osalvador@suse.de; dmarc=pass (policy=none) header.from=suse.de ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1720067535; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=2Nyp1GchWN9j9DCiswMlchcq+0OOi/ydZqtxsOmg3fI=; b=MvAGKtg3mp53EjrR/vQSj4yp5DORtXaJRUuB876MsraO4jN3ABsEt5G4L1PdgLUe8KxAey jNvRdk/OquNn6hGdvW3P0ivTCEXM5MfWILdOOOa7YIM+h8S8q9ymFGIJRH6xyWKcH6OR71 96ua5X7ehUOvRZiyU/WFEKEiFOASkNw= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1720067535; a=rsa-sha256; cv=none; b=jKMSuBx1ZsNZsFMh5jKBax+vh6zfq9XbZ7RY9h1RFcYbx5fg1xHVcwU4BWCzlp5TUJM+Tm nWn9bT1CSC3YaQtN+C2BBchDdQ6vFppwHwKwF0j+uiPv2LjbxMqC1H/0o/2DCGQmVHXLVR Qnl26f9jjXI+jahlXjwWitRbRAcYgtM= ARC-Authentication-Results: i=1; imf22.hostedemail.com; dkim=pass header.d=suse.de header.s=susede2_rsa header.b=FLLFrGlR; dkim=pass header.d=suse.de header.s=susede2_ed25519 header.b=Xn+qmriL; dkim=pass header.d=suse.de header.s=susede2_rsa header.b=FLLFrGlR; dkim=pass header.d=suse.de header.s=susede2_ed25519 header.b=Xn+qmriL; spf=pass (imf22.hostedemail.com: domain of osalvador@suse.de designates 195.135.223.130 as permitted sender) smtp.mailfrom=osalvador@suse.de; dmarc=pass (policy=none) header.from=suse.de Received: from imap1.dmz-prg2.suse.org (unknown [10.150.64.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id 166C121C0D; Thu, 4 Jul 2024 04:32:37 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1720067557; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=2Nyp1GchWN9j9DCiswMlchcq+0OOi/ydZqtxsOmg3fI=; b=FLLFrGlRYQvTCYxVLiVDlOvcqdI03sK7tsIHp0UaVD2y3rWv0sfPku+5j1Sb+TGxATnbIj 2SvrEeBnu35COLNGzRhuWQiuv+d1fMu1ZxIzeBld3MHcaPV5255bX31dpcaYpS7sxPB+yj Wh3sG0PCXGnABuZy8kGWiY/fHIKMyn0= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1720067557; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=2Nyp1GchWN9j9DCiswMlchcq+0OOi/ydZqtxsOmg3fI=; b=Xn+qmriLdnNRRO+exHfJ7IQkKp73CpsQ6YRsMlGxYx0y8gyUntE4TOjbftU2SpDj9j99UM 4IbKv/L2PFMFs+CA== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1720067557; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=2Nyp1GchWN9j9DCiswMlchcq+0OOi/ydZqtxsOmg3fI=; b=FLLFrGlRYQvTCYxVLiVDlOvcqdI03sK7tsIHp0UaVD2y3rWv0sfPku+5j1Sb+TGxATnbIj 2SvrEeBnu35COLNGzRhuWQiuv+d1fMu1ZxIzeBld3MHcaPV5255bX31dpcaYpS7sxPB+yj Wh3sG0PCXGnABuZy8kGWiY/fHIKMyn0= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1720067557; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=2Nyp1GchWN9j9DCiswMlchcq+0OOi/ydZqtxsOmg3fI=; b=Xn+qmriLdnNRRO+exHfJ7IQkKp73CpsQ6YRsMlGxYx0y8gyUntE4TOjbftU2SpDj9j99UM 4IbKv/L2PFMFs+CA== Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id DDB1513889; Thu, 4 Jul 2024 04:32:35 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id QI6DMOMlhmbnDAAAD6G6ig (envelope-from ); Thu, 04 Jul 2024 04:32:35 +0000 From: Oscar Salvador To: Andrew Morton Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Peter Xu , Muchun Song , David Hildenbrand , SeongJae Park , Miaohe Lin , Michal Hocko , Matthew Wilcox , Christophe Leroy , Oscar Salvador Subject: [PATCH 37/45] mm/lock: Make mlock_test_walk skip hugetlb vmas Date: Thu, 4 Jul 2024 06:31:24 +0200 Message-ID: <20240704043132.28501-38-osalvador@suse.de> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240704043132.28501-1-osalvador@suse.de> References: <20240704043132.28501-1-osalvador@suse.de> MIME-Version: 1.0 X-Rspamd-Queue-Id: 5FBC1C0011 X-Stat-Signature: sj8paas3fysfr3kcfmmbtea5ffey75dn X-Rspamd-Server: rspam09 X-Rspam-User: X-HE-Tag: 1720067558-68467 X-HE-Meta: U2FsdGVkX19KKlH3Z8RT5NBIkcv/nWUUjm6XEIdjz8sc1CWLx+VwrihnNsFEOPQxG5vVlO1iYNUWsbIoP7MczowgmexZdPH0umrAEAJNybqHPcWO+4Au4gN4Un5vB6AtfGFWauBaIRBoZV0S1e6CV6V+ad4HvUAJQlIipW20Coh4AC8bbWy9Dq2+Kwwc1gOwJPf0WDNva4LqJtsfKoFlWn4jKuQAeF5lkH8kBQpmiofh/ca6pkDP8qLZ+o/l8LVHG2s82i2ZBQHPGsdZNTQU+CjstCf2Tj6KE0/4JrWslBNchEnC3v519ipwP/erIeHYdTwEhObEg9wGYfKzDALxw47nXuN6VsDFCc8E3GJ6doCLDMD6sSJ+deiaO4EA9xGrLjdV0Dkte6Pit3+ybwkt5SimnPgwKnHGpuohQQC1NLGPVzYWRwUCnv7HUe2PJXsNWLHjC/E8V+dalRoDOZkEwxqDTRr2HbOiYx7l2plEr1zCGwzIYTNV9w0ec+A17HJhA5Ht796lfUZSixAwGGGIeOhvZoOorOE8mNUixUuqvbP/iqoz8LRLC4rNK1Wm4WXHc887lCqJ+hKh2FiPlKax8Krmu8EIZDfnPwqZnMDxokkqlauqOCNpY3r7xwuh1ntxL78fT5edbp9pWQ/zcsXQZTatgxFAZJe4InsZ3+pniWpWnCV6gAfYFivUFumX7nM/gnLaw4UoFx14XcuNJOX/bJCHFqKwN46pNImLQVG0UvbekhXKfojjEb4i4fXy5cUqMMfnFnxLwzAST2QlLWuhnAVbTKcjxt/AhCN0ZH8EtC6bWWmBTKuSrc8jDSUDwk+ysCto2ezr1etm4WdrH/F0HICf3WrSrlErdovjq+7FFaJZKIt9Md7uK1FZIDOTg6PbhJ5MK5GsvXdnIZTRm5U3ovSdqQUjNcf/jl/JGvO44+knBRfFFc02O0TrhguJgnmOy0n1M4XDZE2NYu3SHOS SD1FyG4s PHWPXyAUqbdbGN4NmZlp6l7Mb4ZZgMQy4AJdYuWGXEqBlprduYgHIFYUeQrul3LB1MEavO9eAmXikfZnl4bg3GnOoWQ9RyDzI/G0wgYkP6dMB50BC1KoZBjXaPSqAijfQRfsl252KwfYNhwTw9La1y8NQti1lifiM891mL1JREo0Ha5Wy38TNTi86bwflc4Swf2JsnUKdODqCYH7GoEt6ieKj8ZmpnPDRqAO5+9CRzQuHm29Ehxa9Y/WPM52vnNdvod+NmLsxVj/iJzzo7mkWtIWvUg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Skip hugetlb vmas as we are not interested in those. Signed-off-by: Oscar Salvador --- mm/mlock.c | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/mm/mlock.c b/mm/mlock.c index 52d6e401ad67..b37079b3505f 100644 --- a/mm/mlock.c +++ b/mm/mlock.c @@ -409,6 +409,17 @@ static int mlock_pte_range(pmd_t *pmd, unsigned long addr, return 0; } +static int mlock_test_walk(unsigned long start, unsigned long end, + struct mm_walk *walk) +{ + struct vm_area_struct *vma = walk->vma; + + if (is_vm_hugetlb_page(vma)) + return 1; + + return 0; +} + /* * mlock_vma_pages_range() - mlock any pages already in the range, * or munlock all pages in the range. @@ -425,6 +436,7 @@ static void mlock_vma_pages_range(struct vm_area_struct *vma, { static const struct mm_walk_ops mlock_walk_ops = { .pmd_entry = mlock_pte_range, + .test_walk = mlock_test_walk, .walk_lock = PGWALK_WRLOCK_VERIFY, }; From patchwork Thu Jul 4 04:31:25 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oscar Salvador X-Patchwork-Id: 13723216 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id A3B09C30653 for ; Thu, 4 Jul 2024 04:33:38 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 3784D6B00C7; Thu, 4 Jul 2024 00:32:42 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 3022E6B00C8; Thu, 4 Jul 2024 00:32:42 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 0DE976B00C9; Thu, 4 Jul 2024 00:32:42 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id DFB1B6B00C7 for ; Thu, 4 Jul 2024 00:32:41 -0400 (EDT) Received: from smtpin06.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 9B405120A39 for ; Thu, 4 Jul 2024 04:32:41 +0000 (UTC) X-FDA: 82300799322.06.DB36416 Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.223.131]) by imf30.hostedemail.com (Postfix) with ESMTP id 874C880022 for ; Thu, 4 Jul 2024 04:32:39 +0000 (UTC) Authentication-Results: imf30.hostedemail.com; dkim=pass header.d=suse.de header.s=susede2_rsa header.b=wu38pd2U; dkim=pass header.d=suse.de header.s=susede2_ed25519 header.b=3vLpc8uy; dkim=pass header.d=suse.de header.s=susede2_rsa header.b=wu38pd2U; dkim=pass header.d=suse.de header.s=susede2_ed25519 header.b=3vLpc8uy; dmarc=pass (policy=none) header.from=suse.de; spf=pass (imf30.hostedemail.com: domain of osalvador@suse.de designates 195.135.223.131 as permitted sender) smtp.mailfrom=osalvador@suse.de ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1720067541; a=rsa-sha256; cv=none; b=5FOxk7kAXuF6UavtL8zZ4u2Pd9zC/qkOQu0GO27eN6rTgkGLfHOJsimFGqm9Cq8BrUTEHo hEomZ9ci39zGGCeXQFwZ3UOL8ub4l+lVgGNP7H6LsZMNiZez5CvFlf7B/TTxGV/Lzho09f RYN/238UqeFuP0wnu2M2GngPnKhrEo8= ARC-Authentication-Results: i=1; imf30.hostedemail.com; dkim=pass header.d=suse.de header.s=susede2_rsa header.b=wu38pd2U; dkim=pass header.d=suse.de header.s=susede2_ed25519 header.b=3vLpc8uy; dkim=pass header.d=suse.de header.s=susede2_rsa header.b=wu38pd2U; dkim=pass header.d=suse.de header.s=susede2_ed25519 header.b=3vLpc8uy; dmarc=pass (policy=none) header.from=suse.de; spf=pass (imf30.hostedemail.com: domain of osalvador@suse.de designates 195.135.223.131 as permitted sender) smtp.mailfrom=osalvador@suse.de ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1720067541; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=Lrt/KcsLV9mhSUC002LaYTg+7UaG25OX7K0/IX8flZ0=; b=DOtu9wuigE/JLOJzMoSO+UzlgqG2fQ1v9ogr2QAk5rXlLGWS7zfoCyiI+cFD3nWG0yHuva r6srObnM+grO8uTRj+1BDxxAl3bbgrzWC3B2ugV+pleVKmjoaIzW+A3JvP99Sx0kKugTgA bQssrbahfP73yd0uLcgBluSITx46M9E= Received: from imap1.dmz-prg2.suse.org (unknown [10.150.64.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id 4E4831FCF7; Thu, 4 Jul 2024 04:32:38 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1720067558; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Lrt/KcsLV9mhSUC002LaYTg+7UaG25OX7K0/IX8flZ0=; b=wu38pd2Ur04WQggmlopkDml4wHinDJSRhnDrkpQt+PtRYgYO79418P+xobCSITpMNuUGJr N+r3hCiv4ga0bzDDG6Hf5y+011wgZlCP9rDrAInPoWk4nEgL/L2Ryqm/Fs6j9r+hmIMepL jQdzslJTtqTqyHqPiP2tBLeURFlt62k= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1720067558; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Lrt/KcsLV9mhSUC002LaYTg+7UaG25OX7K0/IX8flZ0=; b=3vLpc8uy3DMDbdMeN7hpDSFZj9ThV9qGHZSesmupuyJqz3PIjoXBK+9NW6fTTC/h3zsQFl 1ol0a9rc5zXXd3CQ== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1720067558; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Lrt/KcsLV9mhSUC002LaYTg+7UaG25OX7K0/IX8flZ0=; b=wu38pd2Ur04WQggmlopkDml4wHinDJSRhnDrkpQt+PtRYgYO79418P+xobCSITpMNuUGJr N+r3hCiv4ga0bzDDG6Hf5y+011wgZlCP9rDrAInPoWk4nEgL/L2Ryqm/Fs6j9r+hmIMepL jQdzslJTtqTqyHqPiP2tBLeURFlt62k= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1720067558; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Lrt/KcsLV9mhSUC002LaYTg+7UaG25OX7K0/IX8flZ0=; b=3vLpc8uy3DMDbdMeN7hpDSFZj9ThV9qGHZSesmupuyJqz3PIjoXBK+9NW6fTTC/h3zsQFl 1ol0a9rc5zXXd3CQ== Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 370FC13889; Thu, 4 Jul 2024 04:32:37 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id 4F+tCOUlhmbnDAAAD6G6ig (envelope-from ); Thu, 04 Jul 2024 04:32:37 +0000 From: Oscar Salvador To: Andrew Morton Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Peter Xu , Muchun Song , David Hildenbrand , SeongJae Park , Miaohe Lin , Michal Hocko , Matthew Wilcox , Christophe Leroy , Oscar Salvador Subject: [PATCH 38/45] mm/madvise: Make swapin_test_walk skip hugetlb vmas Date: Thu, 4 Jul 2024 06:31:25 +0200 Message-ID: <20240704043132.28501-39-osalvador@suse.de> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240704043132.28501-1-osalvador@suse.de> References: <20240704043132.28501-1-osalvador@suse.de> MIME-Version: 1.0 X-Rspamd-Queue-Id: 874C880022 X-Rspam-User: X-Rspamd-Server: rspam05 X-Stat-Signature: ebrzutst861q7p59xiqeqi6ckxb7nyaq X-HE-Tag: 1720067559-117566 X-HE-Meta: U2FsdGVkX19OoUfpJWg94CrmYpxnbTv2in05FRhzFoqkZaQiB2w61HsYiHIY2q8GacpHle7tYn+L4YhkEQGBzTkEIKuBEQ9i+HtHyPbq/Rr5fhl/fwbVQu12z4aIvHMrng5dirxlgT+Y+T7Jv/1EB23mbt41g59WLCbxaJIBm1cja2XbyCL6kNVijRkPlAekrhcyK8shiO8Rt/ZN1UROTNb6Pe2+xsPyB/iXYeOmh8MHnwDmuwz7VFtzgciscQEBURZAD5nOBZb4AG7qniQuhqBP+jEzICfySo7o4E9k8NSWg4/nqxaRGQrGMZBTZVS5N9eEjFeI766q8Xs77WTa4etmJmYxUFjaWmDN18Y/p8OZeTTrOROVk0pV8ihJJOlGtq6WVvh0c+lXIBHlg1H12HvvPSkD3KVpT/OTgmhwzB6MnkoPzdDqisiXN/9IRGKwx/a1aN1Rc0Jjg6tZ4NQdzAuvDgerM5CbDcjbLxRKJ1c+DoPhDyg514SZKnENY7tSDnha9kXb/29Fr1xaimF4jd0rw1TQusrWhbaoyxAv5SMDn6/eGYfFwmkyuRXkz5p74xUic3w/g2DaBbXVzkMzjw+jqNoG1HtH9+dSJvNCWQf1KZho2c+S/yrxlb4eVRe4OVQzmuuRf20ndxos32IQwqzIeT2AjJ31P7403zBO6+DnhRXIVR/xlSlhLZXKiD9BpV/iCMfjk9weiBFxeTIagbELkQMR14oJyCPGqNdJKBya71W/rZ8Nsf2wh+2FsFqorjlFXzVpYM6gvQ0TvX8WlyX297EwrsJx4JSBFdCL93h14pq/MgwXiNGzKml97MuRf6KzXUePVz5Vhbmk4YQ2JKouGxeUM0UOHTREmGiV9fgqezROdm6xYBrzlA91LFipZd1TFOONOc7tPEc4v8iIz5ddtCGXV0W/Cvp7VBo2UgdZC79jxA50iYeNgUc9u56sGpN9hSBZ8HPxvqx2eUX JK9W23yr lxcVgsQJCJH38Ip18AocwtOrcSJg94DxtVrMzULHTwyS2yUShf8xyoNZgN8wTiHhAeyE5s/o2n7ybpizw6UC3JeX4kYwpcfBnXavRG4ZW6quMtQ5iVTmgT7vHtGj20IfIMhcF3kLboyONkmF0hPUlV25duO4Gy7XGe7w+MP+PNHRS2ceFD4JGw4T3Ec/4O9jNJgOm1i33fA0pMJGTOBnNZ3dr/3ShC6E/b+/CSKq3rdPqbp2YQjOuAaQrZ9cfM5zPJyZ7c1TwBRnWlJqYchKBcrcWvQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Skip hugetlb vmas as we are not interested in those. Signed-off-by: Oscar Salvador --- mm/madvise.c | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/mm/madvise.c b/mm/madvise.c index 96c026fe0c99..4c7c409e8b4a 100644 --- a/mm/madvise.c +++ b/mm/madvise.c @@ -212,8 +212,20 @@ static int swapin_walk_pmd_entry(pmd_t *pmd, unsigned long start, return 0; } +static int swapin_test_walk(unsigned long start, unsigned long end, + struct mm_walk *walk) +{ + struct vm_area_struct *vma = walk->vma; + + if (is_vm_hugetlb_page(vma)) + return 1; + + return 0; +} + static const struct mm_walk_ops swapin_walk_ops = { .pmd_entry = swapin_walk_pmd_entry, + .test_walk = swapin_test_walk, .walk_lock = PGWALK_RDLOCK, }; From patchwork Thu Jul 4 04:31:26 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oscar Salvador X-Patchwork-Id: 13723217 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3EA14C30653 for ; Thu, 4 Jul 2024 04:33:41 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 9BE6E6B00C9; Thu, 4 Jul 2024 00:32:43 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 8F4CC6B00CA; Thu, 4 Jul 2024 00:32:43 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 6F8E06B00CB; Thu, 4 Jul 2024 00:32:43 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 43F546B00C9 for ; Thu, 4 Jul 2024 00:32:43 -0400 (EDT) Received: from smtpin05.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id ED1C880A47 for ; Thu, 4 Jul 2024 04:32:42 +0000 (UTC) X-FDA: 82300799364.05.7A1E45E Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.223.130]) by imf06.hostedemail.com (Postfix) with ESMTP id C6DAE180017 for ; Thu, 4 Jul 2024 04:32:40 +0000 (UTC) Authentication-Results: imf06.hostedemail.com; dkim=pass header.d=suse.de header.s=susede2_rsa header.b=hwqDJNVK; dkim=pass header.d=suse.de header.s=susede2_ed25519 header.b=LSnnQJXR; dkim=pass header.d=suse.de header.s=susede2_rsa header.b=hwqDJNVK; dkim=pass header.d=suse.de header.s=susede2_ed25519 header.b=LSnnQJXR; spf=pass (imf06.hostedemail.com: domain of osalvador@suse.de designates 195.135.223.130 as permitted sender) smtp.mailfrom=osalvador@suse.de; dmarc=pass (policy=none) header.from=suse.de ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1720067536; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=OwRMEygtt9hYpWc9+hDgMaQcaULG8MqCO3fZe7sMmEA=; b=jen2RjZGpVmoxD59/tJDFAKQ21V+PBoeHi2mz4c6wPIan9k40vkBslHSZhyg4ffC1Dx0Eq oVylsoWE86WnBitQ4UBjh14ns11hSpXSfEGSk9NvzNli6LR5l0ilUpE/+o95rZbARsYO2m F4m0O0Ipve7L8aCOehX7FdWUjALQpf8= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1720067536; a=rsa-sha256; cv=none; b=Ak3jlmRtvMHWifo7OqlkSDCK/ICIAMsC4LOaB8wZc9111ATD7kw3cWxwJBs51QPPVMQN/M pvego9+QbG7Ptn279e9qlNe/5B71MqBttA0pouHhixo0GjO2YX4kiggFFtdirNocH/l5a3 c0We20Euhn6Jw02Bx4OJFf86lBbxmJI= ARC-Authentication-Results: i=1; imf06.hostedemail.com; dkim=pass header.d=suse.de header.s=susede2_rsa header.b=hwqDJNVK; dkim=pass header.d=suse.de header.s=susede2_ed25519 header.b=LSnnQJXR; dkim=pass header.d=suse.de header.s=susede2_rsa header.b=hwqDJNVK; dkim=pass header.d=suse.de header.s=susede2_ed25519 header.b=LSnnQJXR; spf=pass (imf06.hostedemail.com: domain of osalvador@suse.de designates 195.135.223.130 as permitted sender) smtp.mailfrom=osalvador@suse.de; dmarc=pass (policy=none) header.from=suse.de Received: from imap1.dmz-prg2.suse.org (unknown [10.150.64.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id 9876421C10; Thu, 4 Jul 2024 04:32:39 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1720067559; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=OwRMEygtt9hYpWc9+hDgMaQcaULG8MqCO3fZe7sMmEA=; b=hwqDJNVK/CCOylYSLTwWXUVICQyx83G/knpDrCzGjHxZsYrpDDVMSCB321NNI98Hf7c5Fe fliGr2rF4bdqNm4A5diMwERQGXRj6hZTmNR2Bkz1zFSjM3T+0O/7G2l+AXTUcqMyvnhuqU fHH0ZeRfPHMYS1U4b1g+F9jlmYWgAqw= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1720067559; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=OwRMEygtt9hYpWc9+hDgMaQcaULG8MqCO3fZe7sMmEA=; b=LSnnQJXRLmy1Y92nnMR+gbafnAKL4iJ1SAKvsbEP0jdZUQgWseUiTP/YO4XvsimDx5/+i6 QWKNbAFvKiCbi8AA== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1720067559; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=OwRMEygtt9hYpWc9+hDgMaQcaULG8MqCO3fZe7sMmEA=; b=hwqDJNVK/CCOylYSLTwWXUVICQyx83G/knpDrCzGjHxZsYrpDDVMSCB321NNI98Hf7c5Fe fliGr2rF4bdqNm4A5diMwERQGXRj6hZTmNR2Bkz1zFSjM3T+0O/7G2l+AXTUcqMyvnhuqU fHH0ZeRfPHMYS1U4b1g+F9jlmYWgAqw= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1720067559; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=OwRMEygtt9hYpWc9+hDgMaQcaULG8MqCO3fZe7sMmEA=; b=LSnnQJXRLmy1Y92nnMR+gbafnAKL4iJ1SAKvsbEP0jdZUQgWseUiTP/YO4XvsimDx5/+i6 QWKNbAFvKiCbi8AA== Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 6F2E913889; Thu, 4 Jul 2024 04:32:38 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id SEcAFuYlhmbnDAAAD6G6ig (envelope-from ); Thu, 04 Jul 2024 04:32:38 +0000 From: Oscar Salvador To: Andrew Morton Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Peter Xu , Muchun Song , David Hildenbrand , SeongJae Park , Miaohe Lin , Michal Hocko , Matthew Wilcox , Christophe Leroy , Oscar Salvador Subject: [PATCH 39/45] mm/madvise: Make madvise_cold_test_walk skip hugetlb vmas Date: Thu, 4 Jul 2024 06:31:26 +0200 Message-ID: <20240704043132.28501-40-osalvador@suse.de> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240704043132.28501-1-osalvador@suse.de> References: <20240704043132.28501-1-osalvador@suse.de> MIME-Version: 1.0 X-Stat-Signature: ykwucmpm4pbo8s6tsuborqd5yb9ufyfr X-Rspamd-Queue-Id: C6DAE180017 X-Rspam-User: X-Rspamd-Server: rspam08 X-HE-Tag: 1720067560-296674 X-HE-Meta: U2FsdGVkX1+qhtQ3CcPdM4z5Vh1OOeUDgtwXLNpPbeZXSi3qI7EThmOTfk5+6DjdUkgGEHp+QwALe5KvScIBo3kgwtQNchp6S0xyaipSxo+RnVO5YiEiwOoWcmgG+7k6OpCvpbQsITH9Bx1tedWSo/00yYVMNRdHJE4qYJRYtmS1g6vX9YJ/tKYYicX7qEuJiTfkapCPZliVt+r0L3DVgb+v9vmpDG8uvOHEPNgKYAhIGkMwmGV8Nk5OjAXufFsJKuk7SQuF33eyIPS2BR5LZB2KDfFaRjm5NUI0erXd4tYEHBBcvM8mGvpIYmD1eKCK+6H0U9m3w2EouCW/4YwOM5dU7ps8rbV0YwFPh0i2OS34O1QBARQpI2vNZIPvqi/tGdMiz1eE5NCeAFB80OR9WjHba0bYZjS0kxpFMRgaM81L0kgS9GK8bppqbocMK/djgjhBOQlmzYb3FAhXSp54urr2DfVwe84OQUAn2v/IoU94E9UGIIUobWF3tYaDL4EthFLG14wKebQsYhhzR1SxPj0QkqFc6g7SBwVFJpyifU5Bt3XQoCohNnKeSPKJ2Q7vf3oQqjx2JpT8qcxEmwMiBE4FGkkdGI2W2BkbLTpR8+UJrUWq4OeS/xwbmZyBQ8FTYe21hDFE1ec8RbU/a6QjBZvnmTLK9LwKJOSjpYp2q85lVFzFZ8G8FtxwmK523m0CTirICHzhMKJgz6buTfoHQ1pFh/Ssgy+awjm1fVZsPaZI+Rw6VtL1Q4tcp0E8t3AS/pQhiWIvejBXifMuoh+jQArALZWFPEW0ktMQsFaXyLiY89dGWNHJho3SvvO+1Csy7HfnOtWF85ZHLC+SDtlBcEGDzemM/v6znp+7dafQVz3rZPy9Q0eXmqxIuDMtQlj42Fb9Pzterk8G6vCOVxZVg3DajoyVWOqN0URSG7NuneQkHjiICFYJxikpus923D06dK9cOkQN1Do8k9cDxU1 Y7ecChdl QPZTT0xKBQmb5Sylmo9Hp9FUbZY9+Pn5rKbQaJeaTgucyIxrM8ra5Zwx0blfiwtaeZRPZV4NtCDggrD/Kg1LTzmtxELzrkKZw6XGHLhT3fy2cEMetg/NDoyjeyVx+mhE4aSNTrnLdAQacmS6YOlH9IIv7e/gqilzJiEKV7AaW3+EPF/QNF95L6Kks4hFyVDXQABxQnPJWhBptvtYvFDd+m1emdEAZiRRHxsbz6EvlMXrp9n1U9Z8/w4YBTKPhw1YBUIb5KV+0egOsr+CgxSDKeWYYBA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Skip hugetlb vmas as we are not interested in those. Signed-off-by: Oscar Salvador --- mm/madvise.c | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/mm/madvise.c b/mm/madvise.c index 4c7c409e8b4a..e60311636c4c 100644 --- a/mm/madvise.c +++ b/mm/madvise.c @@ -565,8 +565,20 @@ static int madvise_cold_or_pageout_pte_range(pmd_t *pmd, return 0; } +static int madvise_cold_test_walk(unsigned long start, unsigned long end, + struct mm_walk *walk) +{ + struct vm_area_struct *vma = walk->vma; + + if (is_vm_hugetlb_page(vma)) + return 1; + + return 0; +} + static const struct mm_walk_ops cold_walk_ops = { .pmd_entry = madvise_cold_or_pageout_pte_range, + .test_walk = madvise_cold_test_walk, .walk_lock = PGWALK_RDLOCK, }; From patchwork Thu Jul 4 04:31:27 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oscar Salvador X-Patchwork-Id: 13723218 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id D2CECC30653 for ; Thu, 4 Jul 2024 04:33:45 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D3AA46B00CA; Thu, 4 Jul 2024 00:32:44 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id CC3F46B00CB; Thu, 4 Jul 2024 00:32:44 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id AEFDA6B00CC; Thu, 4 Jul 2024 00:32:44 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 87C126B00CA for ; Thu, 4 Jul 2024 00:32:44 -0400 (EDT) Received: from smtpin06.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 415D1140AC0 for ; Thu, 4 Jul 2024 04:32:44 +0000 (UTC) X-FDA: 82300799448.06.6AB03DB Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.223.130]) by imf29.hostedemail.com (Postfix) with ESMTP id 1B6CB120008 for ; Thu, 4 Jul 2024 04:32:41 +0000 (UTC) Authentication-Results: imf29.hostedemail.com; dkim=pass header.d=suse.de header.s=susede2_rsa header.b=j6iEK8Pq; dkim=pass header.d=suse.de header.s=susede2_ed25519 header.b=vQ4JL4C6; dkim=pass header.d=suse.de header.s=susede2_rsa header.b=j6iEK8Pq; dkim=pass header.d=suse.de header.s=susede2_ed25519 header.b=vQ4JL4C6; spf=pass (imf29.hostedemail.com: domain of osalvador@suse.de designates 195.135.223.130 as permitted sender) smtp.mailfrom=osalvador@suse.de; dmarc=pass (policy=none) header.from=suse.de ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1720067543; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=GvSHNTt8bLluhGl3df5sQFwppsiw5zo4fh71urj/PzI=; b=LWQTN2Tb03ka0/G0YUFfsdQ/A1ZtfpxvKbY6haqqcuPdKpJt92tOa11HN84DZYVChnukAs Ls0x4lN8/Hv7M++07pSBuIqht2QqLxuJ01b3plp5hjFRyJ7Dspk5QRCEf4/Tcb9qvrRtFy /yJSXTe1xx1dHq+bRVp5kLH343M2vYE= ARC-Authentication-Results: i=1; imf29.hostedemail.com; dkim=pass header.d=suse.de header.s=susede2_rsa header.b=j6iEK8Pq; dkim=pass header.d=suse.de header.s=susede2_ed25519 header.b=vQ4JL4C6; dkim=pass header.d=suse.de header.s=susede2_rsa header.b=j6iEK8Pq; dkim=pass header.d=suse.de header.s=susede2_ed25519 header.b=vQ4JL4C6; spf=pass (imf29.hostedemail.com: domain of osalvador@suse.de designates 195.135.223.130 as permitted sender) smtp.mailfrom=osalvador@suse.de; dmarc=pass (policy=none) header.from=suse.de ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1720067543; a=rsa-sha256; cv=none; b=bOSN1SMpOiksfq3i/rTqEmzFAWv0jAvX3DawPxye861yUvMEpxLE1Ipk9WrCpamMYwQJQh 9WiPXcL2y1XaZC8mPwq/c9BE/JH/YLTJg6E4o3Rnvwal2xoldXBop8V9oD7z9R0F9sUwd6 sO4m4lRPCXifJ2KRHA5GBiMkrXnlBTg= Received: from imap1.dmz-prg2.suse.org (unknown [10.150.64.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id D03B721C11; Thu, 4 Jul 2024 04:32:40 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1720067560; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=GvSHNTt8bLluhGl3df5sQFwppsiw5zo4fh71urj/PzI=; b=j6iEK8PqgI9QRFImbu/UVsJysu4KND3ZF8Hf9UZxL94XKAaxTl/7gm9F4NIZVKQ4NnS873 TQNyiXjMCFq8XNcKsBsPNmyqfm2qouwQCrbcFcFAGdwMNdfwK24xrwVgkLwP2jbuyhvLUH BPd9xM4HZ98MSq7QMtepwwtCtWt/SOs= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1720067560; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=GvSHNTt8bLluhGl3df5sQFwppsiw5zo4fh71urj/PzI=; b=vQ4JL4C6UPipOlC2cZ+CsrSqGZzo+7etIj0YNYzIdJOh1eq9XU9yhO8jX+7cuSFjbBdfjf W0zSPHZ7LidSpVCg== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1720067560; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=GvSHNTt8bLluhGl3df5sQFwppsiw5zo4fh71urj/PzI=; b=j6iEK8PqgI9QRFImbu/UVsJysu4KND3ZF8Hf9UZxL94XKAaxTl/7gm9F4NIZVKQ4NnS873 TQNyiXjMCFq8XNcKsBsPNmyqfm2qouwQCrbcFcFAGdwMNdfwK24xrwVgkLwP2jbuyhvLUH BPd9xM4HZ98MSq7QMtepwwtCtWt/SOs= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1720067560; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=GvSHNTt8bLluhGl3df5sQFwppsiw5zo4fh71urj/PzI=; b=vQ4JL4C6UPipOlC2cZ+CsrSqGZzo+7etIj0YNYzIdJOh1eq9XU9yhO8jX+7cuSFjbBdfjf W0zSPHZ7LidSpVCg== Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id AF3B513A82; Thu, 4 Jul 2024 04:32:39 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id 2CQKJuclhmbnDAAAD6G6ig (envelope-from ); Thu, 04 Jul 2024 04:32:39 +0000 From: Oscar Salvador To: Andrew Morton Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Peter Xu , Muchun Song , David Hildenbrand , SeongJae Park , Miaohe Lin , Michal Hocko , Matthew Wilcox , Christophe Leroy , Oscar Salvador Subject: [PATCH 40/45] mm/madvise: Make madvise_free_test_walk skip hugetlb vmas Date: Thu, 4 Jul 2024 06:31:27 +0200 Message-ID: <20240704043132.28501-41-osalvador@suse.de> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240704043132.28501-1-osalvador@suse.de> References: <20240704043132.28501-1-osalvador@suse.de> MIME-Version: 1.0 X-Stat-Signature: b89c6mumattagpck34d3jnzempwiknif X-Rspam-User: X-Rspamd-Queue-Id: 1B6CB120008 X-Rspamd-Server: rspam02 X-HE-Tag: 1720067561-848173 X-HE-Meta: U2FsdGVkX1/Ur3nlfz4bFvPDuc6qWWyvQnz1+B4rS5/tbzkSji0WDspjtviQ5EAOj5/857aqBT9quiQPMQDRV5Sj2y80n25lNpBW6HW+K+LyZ+PfVEdcTaIY1MK5CkUue1b3+zJ6ApK3zK6KQNCZsapIgbiGXCn8vMC/lX0HpjtpgDovPqbnrv4Hky9fToe3EACFQ3dza6vmbtTvoDsWUMPdFDwif8+eiRW05+uNgV2A5d11POM/uic8d3SrthfIjTf5JRSe+4X/rd7jNOX/uW2goGTIWQmXO4e8EDzvS1aPUF6o1SZHnZqxEIAPcJgi6fTjebBcFz8aDdj9T9Rqd4HUb8BXjk0dobV29d6zRObas8a1qzfcuwwRowFGLfiidLJCaO9J9W9Tfw3lSqDmFFuB+z0ozIhe408yN2r+h825RhGSCFUp15s/nAd+NAukG7KK4ZcvpohRyYfQFThdbnG2iphajgOUyDA4rN53nZcE6OMWbRlAQj/uF++gj+ISotEQ08QIaC04kUrjufvbjc3i2nTWmUi9oJtK+TI+sWmnaHtvyRde2USvPfFGIoSirnZakSckAZToJ4NDFUR006sJ3WwV+/aKdIBIRi8LbMjB/uZfRwyVih22NO8vYte+6blCah5JWz7FkksiheFtgeSuk7ymmYOVY3j0grsWeqRHBW6XDCVmtK4m+1PhkHio/yr4N7P44yVE69gxYQp6ucs/KwuHyKITudmO5YHgSIE05M+RD/0Ui9BQOOjIt1jh/LTcNW3Pk5wEwMNYhUcEDj4cUOHbpsvEu03PXxeMYowfogrG8ulG5DWNOhEiiHB4AT9J9961yuVvkc3kQ6vHdx9VfUlQDoUYURkDZda5ogLnVCgybKVveNHp6ga1d+wf/h+uplvryS9YhPjKleTlaXy2oU2JBO+NJW17Mn/ocYzA8PZ7iIxKogm3ZMe7aksRAbrS4hho5Huze5TYUQt Vfo479fK MzYA81zjrIR8hXZeu1QW0vdRVqAd+d/vDOEkO2TAqOzCTjJmsTkO0Ws3wStGm+3L02uW47M1ldNUMWc4KWrUChT0pKRPbhvWaR808Ic27367ioeOU2KWGWxqK/+HSETCVt62l8P5Klv4hx/Y4f4/yxNv9O3uFNEaPzMs2Q8sx+4UfXjZ+XYd0NFRa8xS92hJqEIXCsaGOgsfnS+DY/UChNgWdIdt5MKI2ZwGW4x73Gg9XcNTMJrH2cxQKSycSA/kHFTKZDl2N0PZRKEjFmuwXbeQVKpvrYKiXQLK7 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Skip hugetlb vmas as we are not interested in those. Signed-off-by: Oscar Salvador --- mm/madvise.c | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/mm/madvise.c b/mm/madvise.c index e60311636c4c..08f72622913f 100644 --- a/mm/madvise.c +++ b/mm/madvise.c @@ -805,8 +805,20 @@ static int madvise_free_pte_range(pmd_t *pmd, unsigned long addr, return 0; } +static int madvise_free_test_walk(unsigned long start, unsigned long end, + struct mm_walk *walk) +{ + struct vm_area_struct *vma = walk->vma; + + if (is_vm_hugetlb_page(vma)) + return 1; + + return 0; +} + static const struct mm_walk_ops madvise_free_walk_ops = { .pmd_entry = madvise_free_pte_range, + .test_walk = madvise_free_test_walk, .walk_lock = PGWALK_RDLOCK, }; From patchwork Thu Jul 4 04:31:28 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oscar Salvador X-Patchwork-Id: 13723219 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 82E77C30653 for ; Thu, 4 Jul 2024 04:33:48 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id E74276B00CB; Thu, 4 Jul 2024 00:32:45 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id DFE426B00CC; Thu, 4 Jul 2024 00:32:45 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C03356B00CD; Thu, 4 Jul 2024 00:32:45 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 990F56B00CB for ; Thu, 4 Jul 2024 00:32:45 -0400 (EDT) Received: from smtpin13.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 5D9AB80F90 for ; Thu, 4 Jul 2024 04:32:45 +0000 (UTC) X-FDA: 82300799490.13.A278F0C Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.223.131]) by imf14.hostedemail.com (Postfix) with ESMTP id 5BDD5100012 for ; Thu, 4 Jul 2024 04:32:43 +0000 (UTC) Authentication-Results: imf14.hostedemail.com; dkim=none; spf=pass (imf14.hostedemail.com: domain of osalvador@suse.de designates 195.135.223.131 as permitted sender) smtp.mailfrom=osalvador@suse.de; dmarc=pass (policy=none) header.from=suse.de ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1720067550; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=NqVElTeXwGL2JOvfvH1ngb1/8anfrNsjiXSKP+knOAE=; b=hSaVrq35s1z5Zd4MkH9i/Q8rwl10ySA32Yh8JUkwutDF/ZwvKHeaRZHDoxwzy8N9ciDsiJ bK0otIh/FGIAdgrMEY3Fe+d4eMk9fUNAtqDVq8bc+iUBhEhc8RW4ohbC9SpRGdYYiwwCls h4IdxbMcLd9y+49hpMigHKEJzSew3GQ= ARC-Authentication-Results: i=1; imf14.hostedemail.com; dkim=none; spf=pass (imf14.hostedemail.com: domain of osalvador@suse.de designates 195.135.223.131 as permitted sender) smtp.mailfrom=osalvador@suse.de; dmarc=pass (policy=none) header.from=suse.de ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1720067550; a=rsa-sha256; cv=none; b=HBGDs/8EkfS4RUS831E/THL9lXxR479ljprar17H+OKIbJxLrG7mSAmSmN+2rmSUnm4aiM 4u0nPYfIcgNkbXdKribqriICycFlDhdVCP4PVIR//yYMGcNSo5x6mZSw9OKnHHdV288XN8 qO8I3AD+yY5H10b2aBfXwQKyiRK8Ep8= Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [IPv6:2a07:de40:b281:104:10:150:64:97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id 27A961FCF8; Thu, 4 Jul 2024 04:32:42 +0000 (UTC) Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 0274313889; Thu, 4 Jul 2024 04:32:40 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id SAXCNuglhmbnDAAAD6G6ig (envelope-from ); Thu, 04 Jul 2024 04:32:40 +0000 From: Oscar Salvador To: Andrew Morton Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Peter Xu , Muchun Song , David Hildenbrand , SeongJae Park , Miaohe Lin , Michal Hocko , Matthew Wilcox , Christophe Leroy , Oscar Salvador Subject: [PATCH 41/45] mm/migrate_device: Make migrate_vma_test_walk skip hugetlb vmas Date: Thu, 4 Jul 2024 06:31:28 +0200 Message-ID: <20240704043132.28501-42-osalvador@suse.de> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240704043132.28501-1-osalvador@suse.de> References: <20240704043132.28501-1-osalvador@suse.de> MIME-Version: 1.0 X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Rspamd-Action: no action X-Rspamd-Server: rspam11 X-Rspamd-Queue-Id: 5BDD5100012 X-Stat-Signature: qfr6n3k77f9o7enekwtgewjht8ns7gqu X-Rspam-User: X-HE-Tag: 1720067563-821916 X-HE-Meta: U2FsdGVkX1/1ERdgz1VNNKvx7Srv9fbX4vpfL6qL6cbjdnc0tZZxaMnd98rGpQbql/+NFKelpcnR3j8KrE4MUBXMLexF5jDh/1y4/rD5nkPxdu984PKr4a1up7T47/55QGOL2nywaYlKs9yWhs9Am9L6OontuE59NbxMn1hSPqBskb896pwNjaHqfigLH5JtiPk5YG1fvgJrXvpDHApHqzIVkVY1YmBiV/xxHqg7+5kfbGFnxSWmlmeMNv3H4wyH9QWOqTE/AP4Ajt2jqalTXykqaPOZwUYKXA8/X2CTAnUlrRB+KRyFnlmUZlAkMctBvPrN8KufCPhlY7lfqL6nHiBBcE5SslZxT33Yyf+JpEcNfSKzP1RkchmtMhPviDREXsCKsdDTSCLVN/M0CvLd6/t8dq3S3mac9cX0QV/wwRPXK/fJxqV3beeSrrTfDgR98x7ZY7o9K6hX8YJ9ar/z9pQCRExQjDuB6GiJarXG+kbWtQhCxdoNaDp6KmRmuA0R8Zhf4rSI7B+JD92O6ADzRQI0sn5vlQPjsnzB/o3eWx/mtrNnkBdyV+ErlGi7sKF5wFftr2Ws3sn6vN+No6Z1z+VNUg5YLen9+kEvU8UXOzfaqeF9kCMMizkw1AyTPlXz+PbsgJqAekPsuUPF8U5sn+kslYdDQwImVXmc8UFTrpLxqhxPFkTteLlfqe+3MB7GZRzHi5AQWAF18mmvzt0/lFnVKa8dNC8+45EGMs/x77bpbFyJYpIN5zFhjf5plK/RgvS77Hr3aWNQ9U4uZYSUn54vF0jawHBp5+hm5h8G1vup9aO+tj3hgrG9d6RdhQKSCDO5MCZvTtVSBIN+Ly7sztxNZryFh+seYaZapNK9shatrpwzyFrg2NXmhLkbe0bbRFdFeBpvNNXGb+nJvTpgql0tH8Ac0aNfAXUSzANktvD/SmltJIe80TLRKTXr/rIO2m6zIKC8rlPuU9IdPVT fQA0n9fa kvzsDqOV7YbTThbk4ewEoFJyyJJjhPlLmi2Nq5LtN9Y+16qTWamI8/pgEkNpbUJ1jeiP4M4LfOCQwbwyj/4daYb3vY5puZvrSR0xCR0TyBG1beyTgNYICO3zU8OaLoZUrEpmlZmBbtkqph4C1wKfLZZqWOlDw+2bv3gG3CJ01SKKOgeA= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Skip hugetlb vmas as we are not interested in those. Signed-off-by: Oscar Salvador --- mm/migrate_device.c | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/mm/migrate_device.c b/mm/migrate_device.c index 6d66dc1c6ffa..c44ac45b207d 100644 --- a/mm/migrate_device.c +++ b/mm/migrate_device.c @@ -279,8 +279,20 @@ static int migrate_vma_collect_pmd(pmd_t *pmdp, return 0; } +static int migrate_vma_test_walk(unsigned long start, unsigned long end, + struct mm_walk *walk) +{ + struct vm_area_struct *vma = walk->vma; + + if (is_vm_hugetlb_page(vma)) + return 1; + + return 0; +} + static const struct mm_walk_ops migrate_vma_walk_ops = { .pmd_entry = migrate_vma_collect_pmd, + .test_walk = migrate_vma_test_walk, .pte_hole = migrate_vma_collect_hole, .walk_lock = PGWALK_RDLOCK, }; From patchwork Thu Jul 4 04:31:29 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oscar Salvador X-Patchwork-Id: 13723220 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4B17BC3271F for ; Thu, 4 Jul 2024 04:33:50 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 65F656B00CD; Thu, 4 Jul 2024 00:32:47 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 5E6386B00CE; Thu, 4 Jul 2024 00:32:47 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 3C7406B00CF; Thu, 4 Jul 2024 00:32:47 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 133B36B00CD for ; Thu, 4 Jul 2024 00:32:47 -0400 (EDT) Received: from smtpin04.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id C83E680F90 for ; Thu, 4 Jul 2024 04:32:46 +0000 (UTC) X-FDA: 82300799532.04.0C1C955 Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.223.131]) by imf04.hostedemail.com (Postfix) with ESMTP id AE37840004 for ; Thu, 4 Jul 2024 04:32:44 +0000 (UTC) Authentication-Results: imf04.hostedemail.com; dkim=pass header.d=suse.de header.s=susede2_rsa header.b=f6TMrMH1; dkim=pass header.d=suse.de header.s=susede2_ed25519 header.b=7wTQx6i+; dkim=pass header.d=suse.de header.s=susede2_rsa header.b=f6TMrMH1; dkim=pass header.d=suse.de header.s=susede2_ed25519 header.b=7wTQx6i+; dmarc=pass (policy=none) header.from=suse.de; spf=pass (imf04.hostedemail.com: domain of osalvador@suse.de designates 195.135.223.131 as permitted sender) smtp.mailfrom=osalvador@suse.de ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1720067544; a=rsa-sha256; cv=none; b=weBfKwotGKKLhfABJvTqRjfyMAYfkxQA20ruAuT1KJqeeIz71NU6ET9pLIIxMjaXZE+9FM Si+fFs5yac1N47FAZiE4F3Ac7o3C/lwpS2SPFbc7nZsvnGhMuXRB/W2ITWdJWA5UTzPaEG qs6w/sgUMImTsdYwaRgMsMQuFu4RDPo= ARC-Authentication-Results: i=1; imf04.hostedemail.com; dkim=pass header.d=suse.de header.s=susede2_rsa header.b=f6TMrMH1; dkim=pass header.d=suse.de header.s=susede2_ed25519 header.b=7wTQx6i+; dkim=pass header.d=suse.de header.s=susede2_rsa header.b=f6TMrMH1; dkim=pass header.d=suse.de header.s=susede2_ed25519 header.b=7wTQx6i+; dmarc=pass (policy=none) header.from=suse.de; spf=pass (imf04.hostedemail.com: domain of osalvador@suse.de designates 195.135.223.131 as permitted sender) smtp.mailfrom=osalvador@suse.de ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1720067544; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=OvwrckhnGQ1ia89v5nuOKOB6N7ZG52TVWWVE7z3ysd0=; b=zSP9Y6b/GNKyzHwsM0C7q+Az7yqDpj+CHWeh3soSvRmalHM9WCy0O+AgtO/QH8EUZ0ldT8 elLCQaFgA8pL8BhC4vzjCl1YykUCqItOjHCkezc3U/Kq5QCkPXGd4qqQeoqmjaP3dCAHSl +DECQMkeliGhoIHa5LYhaaD5kTdu1XU= Received: from imap1.dmz-prg2.suse.org (unknown [10.150.64.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id 69E391FCFA; Thu, 4 Jul 2024 04:32:43 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1720067563; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=OvwrckhnGQ1ia89v5nuOKOB6N7ZG52TVWWVE7z3ysd0=; b=f6TMrMH1l0DF/S8bfNE80GfcQ/4KU9eJ27sxLCy3LzyHrHWU4yVZdFy7kYhwfc7mVnZVdu xsRX3Nb1px3A9NnzgBOX5ai81npXNqEkbHs8Yk6Z2158wI9WwdgLuclp+kv3Leuji+b1kc vOC5FehQO0GHOMoxqJ/B9oV+I3yygyc= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1720067563; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=OvwrckhnGQ1ia89v5nuOKOB6N7ZG52TVWWVE7z3ysd0=; b=7wTQx6i+oEK8SbN8wor1yxW+lVrtevHmwonbwTOp+kRSZvWPS1cw79vzxMRpqgGlJBhIHR U7Fz/q4+qA9HUzCQ== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1720067563; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=OvwrckhnGQ1ia89v5nuOKOB6N7ZG52TVWWVE7z3ysd0=; b=f6TMrMH1l0DF/S8bfNE80GfcQ/4KU9eJ27sxLCy3LzyHrHWU4yVZdFy7kYhwfc7mVnZVdu xsRX3Nb1px3A9NnzgBOX5ai81npXNqEkbHs8Yk6Z2158wI9WwdgLuclp+kv3Leuji+b1kc vOC5FehQO0GHOMoxqJ/B9oV+I3yygyc= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1720067563; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=OvwrckhnGQ1ia89v5nuOKOB6N7ZG52TVWWVE7z3ysd0=; b=7wTQx6i+oEK8SbN8wor1yxW+lVrtevHmwonbwTOp+kRSZvWPS1cw79vzxMRpqgGlJBhIHR U7Fz/q4+qA9HUzCQ== Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 4B28813889; Thu, 4 Jul 2024 04:32:42 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id WEYNDuolhmbnDAAAD6G6ig (envelope-from ); Thu, 04 Jul 2024 04:32:42 +0000 From: Oscar Salvador To: Andrew Morton Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Peter Xu , Muchun Song , David Hildenbrand , SeongJae Park , Miaohe Lin , Michal Hocko , Matthew Wilcox , Christophe Leroy , Oscar Salvador Subject: [PATCH 42/45] mm/memcontrol: Make mem_cgroup_move_test_walk skip hugetlb vmas Date: Thu, 4 Jul 2024 06:31:29 +0200 Message-ID: <20240704043132.28501-43-osalvador@suse.de> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240704043132.28501-1-osalvador@suse.de> References: <20240704043132.28501-1-osalvador@suse.de> MIME-Version: 1.0 X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: AE37840004 X-Stat-Signature: jo1965gcqguqtur3q7d5psjnh9o3u1cr X-Rspam-User: X-HE-Tag: 1720067564-394671 X-HE-Meta: U2FsdGVkX19X7FgGCN3kea+EKheqy0HERMhuuswVGIcc64W1N2nS/oR/qT7O0OZRPx+tgOluXvUoG9y5j80Fgr1bV33xy5SxbCuKeWtXmEuxOtZXD2V+rNy4B7E6XRnDwuvgD/iVqARUpD4s5OCEnzGpESO2SXE0FkVs+c7y34/TtpOYwcn+78zXZ8T7pxUV8ROUrGbUvGMt5vaGzcpSS6jCdnUe1tLjTQ5vuC0+rZHPaXt+u1H+nIzt1sUaOFWtXIyY10rB8vsSebBsp+ggCb3x8ky+/oOlrSBV0pYjsvK/2YJgEBPyKv8CCme/3/dEo9CF2RhtEzaZxYL/iNU+V3Mrn60nn6vg1Up0nZuCACUCCWZBGgKLMakmrLX0r3zM1VYVrbch/a/iK9vvQF+4SaMlUcG7/AvdrPPPbYZSMM+Vsfhrr4MoKInLoKEHbgKurCfU8nzAvNqT4rszfjcr+Zza+ZRaU1wY6p835nlmfQzRb5kinRnoq8kOcg7GC4X55nvmCVOYgMgxCDh1GHOnmFkoPpzgoz6JE1BvcGUGuZbFLv/MqGsdIPZeC9/hkqHpqYKf5zybet2II4rYSkE6xZF6zMZYGuFjMDnaQZMk7DWvOr7wtEF5d6mEc9oy6QJGgXPR+Qb/dbMKRYdZgIAfE/r3VoyCo2jmtphY1eQS8jVEOwJaMqfjrIL5Nw2bms6wyJCcxJbkc6oSmYOLNSPGWpK9d3zI55ibTFoX2EIPhZhoPKV04e+m69arMry5vuueo0Eg2k4V6MpfFujsgz89Eiphi01Vd4SrwG/SRl13qZfafv3yCxHjvRJpmSsBVHg3v7qGCfS1+LvjyiHlTRW1EYi5bFWwJsTRGTQq+eGoPPWrSHSEL8Uw4vZWXozpX63Fgyu446KNEXfZJIZSAqoAS2lcyu07VfjmXM6lKhTZC7B5Yzzg/XEPxcZnTNE+4R3SnyB/3kSMXcRDOdZzhby mAKRRG9w NqOV3x7mT43viu8yH0JwqdtXDhpyCRvSf1cnTVNdXX2AlUUUxtYKmvG1aFt6XDo4tqcUw0UpJMeZTLaTLN7Cl5GGTjlPTDGWSTkiqmFJ0yOkGY+OuCsjbBshznLyVJz8Db0KIZMO4yj3EU/t0Wb4Wc2NXRM/3Uechw0Odj2wpCZfDDa1iga22IjSzxcqHTZPbMlz9YeydBWXXbTPrnNvGwhdjXhhVe2U5HdsGJAWUp2S6JOyh12SchxoJvUqcDp16ELgzmbAzKM+pqFo1cX2UJCdyf2HgjQlbI0dz X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Skip hugetlb vmas as we are not interested in those. Signed-off-by: Oscar Salvador --- mm/memcontrol-v1.c | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/mm/memcontrol-v1.c b/mm/memcontrol-v1.c index 99cc9501eec1..542922562cf9 100644 --- a/mm/memcontrol-v1.c +++ b/mm/memcontrol-v1.c @@ -1319,8 +1319,20 @@ static int mem_cgroup_move_charge_pte_range(pmd_t *pmd, return ret; } +static int mem_cgroup_move_test_walk(unsigned long start, unsigned long end, + struct mm_walk *walk) +{ + struct vm_area_struct *vma = walk->vma; + + if (is_vm_hugetlb_page(vma)) + return 1; + + return 0; +} + static const struct mm_walk_ops charge_walk_ops = { .pmd_entry = mem_cgroup_move_charge_pte_range, + .test_walk = mem_cgroup_move_test_walk, .walk_lock = PGWALK_RDLOCK, }; From patchwork Thu Jul 4 04:31:30 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oscar Salvador X-Patchwork-Id: 13723221 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id A2CECC3065C for ; Thu, 4 Jul 2024 04:33:53 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 7543E6B00CE; Thu, 4 Jul 2024 00:32:48 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 6D1946B00CF; Thu, 4 Jul 2024 00:32:48 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 522846B00D0; Thu, 4 Jul 2024 00:32:48 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 2D6BC6B00CE for ; Thu, 4 Jul 2024 00:32:48 -0400 (EDT) Received: from smtpin14.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id DF868A3D40 for ; Thu, 4 Jul 2024 04:32:47 +0000 (UTC) X-FDA: 82300799574.14.64DA18F Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.223.131]) by imf02.hostedemail.com (Postfix) with ESMTP id E81B28001E for ; Thu, 4 Jul 2024 04:32:45 +0000 (UTC) Authentication-Results: imf02.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=suse.de; spf=pass (imf02.hostedemail.com: domain of osalvador@suse.de designates 195.135.223.131 as permitted sender) smtp.mailfrom=osalvador@suse.de ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1720067546; a=rsa-sha256; cv=none; b=wExYjLbbWYkKwv5UQC/197AHGMxZ9k+9+qRVXl66BzMdK2m/jRJ75/ayvMiVIUBj6i5ojT WKVnWJHUb2t4lxDc8Uva12cGQqlcFdwLvdZtULacOiQcB7m0qgnuEns4Yf7vuXRgUdUkif 8KxJOw32cQzBU1e4d+9pL/UaQkalXxE= ARC-Authentication-Results: i=1; imf02.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=suse.de; spf=pass (imf02.hostedemail.com: domain of osalvador@suse.de designates 195.135.223.131 as permitted sender) smtp.mailfrom=osalvador@suse.de ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1720067546; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=kK41s/CyBLAGFygZDMCjq8VuA+rrMM/fx7eQHYvrz/w=; b=72jjobuYfCxxjHuETXefPlCAmMiZ0W7YEs5bE4bgFLx/Gt3DTUc790PXYcJcgniGBrtUdD tEZv6eo08SbNDCggOe+ZtBbNn/02k6NO6Isuw59RLHBOFn7wVvMj74O1z0F3FfF6EryuCs FXI4Fvgmi4qGNIfTePfpofwba+CGuDo= Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [IPv6:2a07:de40:b281:104:10:150:64:97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id A41351FCFB; Thu, 4 Jul 2024 04:32:44 +0000 (UTC) Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 8D0C513889; Thu, 4 Jul 2024 04:32:43 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id 2IqsHeslhmbnDAAAD6G6ig (envelope-from ); Thu, 04 Jul 2024 04:32:43 +0000 From: Oscar Salvador To: Andrew Morton Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Peter Xu , Muchun Song , David Hildenbrand , SeongJae Park , Miaohe Lin , Michal Hocko , Matthew Wilcox , Christophe Leroy , Oscar Salvador Subject: [PATCH 43/45] mm/memcontrol: Make mem_cgroup_count_test_walk skip hugetlb vmas Date: Thu, 4 Jul 2024 06:31:30 +0200 Message-ID: <20240704043132.28501-44-osalvador@suse.de> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240704043132.28501-1-osalvador@suse.de> References: <20240704043132.28501-1-osalvador@suse.de> MIME-Version: 1.0 X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Rspamd-Action: no action X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: E81B28001E X-Stat-Signature: pjtxduede9a5id9znkgg5y1r7nb6na6u X-Rspam-User: X-HE-Tag: 1720067565-913941 X-HE-Meta: U2FsdGVkX1/nNmEMW7LiY+281YvkKOoo8tOMDCh8XX9HnlZ1/gmiqWfa8eWj6UPbUttHV/FnUfRDAxkT/t+NiwiBTI36cuYpv4DsQhqTXPH/wPhxJLW7Y5loSBlY0oKRySrMiB9TKyBgGXyuOCA0QYZv7VMrE6TGSsqJGXSTa/ibfWfimvq74XFTLXJoiTsVv1KEAfZW6kanI7xfGi2FaU6V7UDe1mqRC3C3aBvuQVJxnbtDLR79loVcbMMQm/zan+Pguh+/wCEyhnTtMtFmY+WUmtCCJf+6hFXykgWYm0EivwG6WXXrAH01k0uRxHSVGyD5WoKhFcCbml3QbdHw75qu/u5uvQl8CoXlbw1femn1GV0xF2VNzaAk7qHTktZbYMKb92OO7QU5MjklS3i6V9o9clVajrddvwMUpSiwxsd7749Hl+uQxm3CB2+fOJI0fJxLmyizMx4ocXpZEgSqdnM9/OpZvomYQv1wWDEHcE1728VtcBj3DpSk5GVntKml3EjQu16rvlWYvh2qpdwEki3nCjZ5wiyANBDtKmoZ4b2KkDNGCsqZY1b9NR/e12V8PIx36mcLJZoOZKO/ayIInNeRurhtGZjZrpNSXWoNSVghm92qceZmlYGkNo739xGkWndAJjzEE+reL2CkhSeoUlUHookQfLtx/UndYdaMsI6nbBAvWB1LKnvKywmA3b4xbQO/+bcbx4TQS1b/GS8I/oS0PPeUHs4+mmClJBv70AUSRjkoSHu96dp5rHa4JdzWdVp+VVMPQ3iHUr/gKDcyylekqvUzU9oeoPCfC7Y+PA7z/tNLIRzQ2yMi/nqHlTHqtkG16Hy2ovsfNUPWRnbqpvEb8jIgSY2buhTSCTMQe7NdrGOZ9gRZXUUv9lKSgaigLnEsmmriFZJ+lU+4PJVZQkd3Q83ENfDwhrdhppr4leUcEe47kWzYbe2sjo7LU0Z++gdgDXG45aZtgeMV0oz hnf7udIu BNvOZW7Ha10pAE9qtMjjfvcnrEoYYSnz/8rkQ97VORZ+jwVZG475YYLKfTEUMDq8gvAEsOGDIhYV/R1ZiiTLVG4uCfkZmx+UEPFHWUFd5tEQUSKC40EQBhCtRXwIwPzSyIjlutj3tVMj7ex/FKlIDdK+OumW/g7nyW8SMbKNjdHwZYi8= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Skip hugetlb vmas as we are not interested in those. Signed-off-by: Oscar Salvador --- mm/memcontrol-v1.c | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/mm/memcontrol-v1.c b/mm/memcontrol-v1.c index 542922562cf9..cd8ad1a7f170 100644 --- a/mm/memcontrol-v1.c +++ b/mm/memcontrol-v1.c @@ -1039,8 +1039,20 @@ static int mem_cgroup_count_precharge_pte_range(pmd_t *pmd, return 0; } +static int mem_cgroup_count_test_walk(unsigned long start, unsigned long end, + struct mm_walk *walk) +{ + struct vm_area_struct *vma = walk->vma; + + if (is_vm_hugetlb_page(vma)) + return 1; + + return 0; +} + static const struct mm_walk_ops precharge_walk_ops = { .pmd_entry = mem_cgroup_count_precharge_pte_range, + .test_walk = mem_cgroup_count_test_walk, .walk_lock = PGWALK_RDLOCK, }; From patchwork Thu Jul 4 04:31:31 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oscar Salvador X-Patchwork-Id: 13723222 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4A73FC30653 for ; Thu, 4 Jul 2024 04:33:57 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id DDB9F6B00D0; Thu, 4 Jul 2024 00:32:49 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id CEB2F6B00D1; Thu, 4 Jul 2024 00:32:49 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id AA38E6B00D2; Thu, 4 Jul 2024 00:32:49 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 7EA066B00D0 for ; Thu, 4 Jul 2024 00:32:49 -0400 (EDT) Received: from smtpin19.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 384FFA0FC8 for ; Thu, 4 Jul 2024 04:32:49 +0000 (UTC) X-FDA: 82300799658.19.A3314A0 Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.223.130]) by imf13.hostedemail.com (Postfix) with ESMTP id 223A32000B for ; Thu, 4 Jul 2024 04:32:46 +0000 (UTC) Authentication-Results: imf13.hostedemail.com; dkim=pass header.d=suse.de header.s=susede2_rsa header.b=qWQLNf4E; dkim=pass header.d=suse.de header.s=susede2_ed25519 header.b=9PS3LOnT; dkim=pass header.d=suse.de header.s=susede2_rsa header.b=qWQLNf4E; dkim=pass header.d=suse.de header.s=susede2_ed25519 header.b=9PS3LOnT; spf=pass (imf13.hostedemail.com: domain of osalvador@suse.de designates 195.135.223.130 as permitted sender) smtp.mailfrom=osalvador@suse.de; dmarc=pass (policy=none) header.from=suse.de ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1720067555; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=9eR5Nhbb0ANiKXQprN8VC6tLG959+ERIV2i0QqHsYD4=; b=f8EEUNLqJkFay9Ci+PMRRS5oSpiOEbUIM1kr3IFNkAWt6vN3nUGsYer04LDrqSBkytYsFg WEfV7PVppDM68KgRPYUGokJK+DoscVqFdw7mD0UEWLBahx2CYd6zBi0pAntxZ/7x4ia6/l 4Bu8iE+Oqpn/Sjy5d99wwtzimoYD6z8= ARC-Authentication-Results: i=1; imf13.hostedemail.com; dkim=pass header.d=suse.de header.s=susede2_rsa header.b=qWQLNf4E; dkim=pass header.d=suse.de header.s=susede2_ed25519 header.b=9PS3LOnT; dkim=pass header.d=suse.de header.s=susede2_rsa header.b=qWQLNf4E; dkim=pass header.d=suse.de header.s=susede2_ed25519 header.b=9PS3LOnT; spf=pass (imf13.hostedemail.com: domain of osalvador@suse.de designates 195.135.223.130 as permitted sender) smtp.mailfrom=osalvador@suse.de; dmarc=pass (policy=none) header.from=suse.de ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1720067555; a=rsa-sha256; cv=none; b=xnsWbRd+LUckjp0Ub127YdHGQUVRmbm7E6hSF1Z8voggXjHZlsaxDuFdmj8PMqeNQAh21V fASrF64K8dLKDfHlPxI3MHRokOj9afI1TzlZzcFMm9pXZc4tZ34aX7s5jRTf2csA9+kOzu 5JLNSyW80XPJIKddES72heidhd2QpBU= Received: from imap1.dmz-prg2.suse.org (unknown [10.150.64.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id D89E621180; Thu, 4 Jul 2024 04:32:45 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1720067565; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=9eR5Nhbb0ANiKXQprN8VC6tLG959+ERIV2i0QqHsYD4=; b=qWQLNf4EDHAPFOcSiw8o9yIVaDWy3wN7EvJ2sANX/QaSm9/dzdpfyxkS+tGCPT67FTj67/ qxSBubrEfCFMg2meOU9ONT/982yfM0wORqljjkCK0CIkPk/MmyWkBuIbujyg+alQu42Nox +3A8UPaXHPIjQsi6zuoCjBr/Vxx+ZLY= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1720067565; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=9eR5Nhbb0ANiKXQprN8VC6tLG959+ERIV2i0QqHsYD4=; b=9PS3LOnTWX4tc9eF256ZXVJczKFHze/Ne7gNltkr5CcTePNcTuvg5Xdekx74hNiZwcr4HX nEMGuKsw4KpydzBA== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1720067565; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=9eR5Nhbb0ANiKXQprN8VC6tLG959+ERIV2i0QqHsYD4=; b=qWQLNf4EDHAPFOcSiw8o9yIVaDWy3wN7EvJ2sANX/QaSm9/dzdpfyxkS+tGCPT67FTj67/ qxSBubrEfCFMg2meOU9ONT/982yfM0wORqljjkCK0CIkPk/MmyWkBuIbujyg+alQu42Nox +3A8UPaXHPIjQsi6zuoCjBr/Vxx+ZLY= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1720067565; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=9eR5Nhbb0ANiKXQprN8VC6tLG959+ERIV2i0QqHsYD4=; b=9PS3LOnTWX4tc9eF256ZXVJczKFHze/Ne7gNltkr5CcTePNcTuvg5Xdekx74hNiZwcr4HX nEMGuKsw4KpydzBA== Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id C103213889; Thu, 4 Jul 2024 04:32:44 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id +JDnKuwlhmbnDAAAD6G6ig (envelope-from ); Thu, 04 Jul 2024 04:32:44 +0000 From: Oscar Salvador To: Andrew Morton Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Peter Xu , Muchun Song , David Hildenbrand , SeongJae Park , Miaohe Lin , Michal Hocko , Matthew Wilcox , Christophe Leroy , Oscar Salvador Subject: [PATCH 44/45] mm/hugetlb_vmemmap: Make vmemmap_test_walk skip hugetlb vmas Date: Thu, 4 Jul 2024 06:31:31 +0200 Message-ID: <20240704043132.28501-45-osalvador@suse.de> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240704043132.28501-1-osalvador@suse.de> References: <20240704043132.28501-1-osalvador@suse.de> MIME-Version: 1.0 X-Rspamd-Server: rspam03 X-Rspam-User: X-Rspamd-Queue-Id: 223A32000B X-Stat-Signature: pranmx6oobo99ug3gzgciw37zqf9hosc X-HE-Tag: 1720067566-590369 X-HE-Meta: U2FsdGVkX1+mlMxdGfGSPv/IoWGAbclN11TQVhBl/1G/cmAdp+RfAiY8mPVQKsQ7jZjEuoXv9DWfUK9QEn8X5a9Zpll5dRmsJpt6IEukazms2rAq3UvrhD7mBySnD9NqNQdzE6PFRaLMzDn/H1SfDy+opKRDGqrcAvnDngdXN37DQrBG9KQ1lNVXyZLLU7ls+UyHMIBbAF2SKXMKEc8pQ9XA/40OFwv0lYt10ZDd+3Khikcmhn9hFOS9CH+ak1Fzs/9IbTMMAsvMJOvK8L6B/Oa8wyqjL8ViZRo4axVbvxBQpiyX53cfCmowz9Y4fZh14fbu/lVmSLXXavDI+aZxvcFlOqQRHeOwTMP+gaM6lAb4euudrff3jS+DhPQ134pyDPkydRQFBLC65exyjUltK03s1nEIBzgZX3AgpHaqasz/UwuOpYGVPKHv/MB+rrERdJ3XbP1sKa3pEbO6BazrP3a0DD7RQD1kX3w7OE/JEpuB2Wcsi6iwzyx8xHYNs4TTApUL8rULOIHp26LreJwz/tX/vNsXgZXSAYBARwlUzEr+ZaQk0UybW4bF8zVkZv1fhcts+sTIUWBazJxhhyrPjmJYET/OJuGz5K+woVL9JBaHCCyeQuyYToW6vuVORVDypjcVxfdehexdbmmRg+XDF0i9sfTMeAH3Pt0XDLZN3i+3OvfIryAW1W8BQgeDNdCOs23bLi9kbjMtAk9lz0R8lX+BeZPJVSecOuO+3Nfh/QS2/lwvxwYElbEk1SH1gsvjCipzjEWe+iJcj/6m0yg/NJ3q2TXmXBXpUTSVoIDFe+/XC4Kjy+M4vDmkbOGsAjQpwr+zwGwP1mfSaL0giiqvIi1Cv6W/hgsO6olEon3FCfiq/LwcAZXCxeL3Becrl9Z6TchwG+ni0FhkruB55+gD3KlwhETrG50zCeZCGRH3IHuva55C/o9sZLjZdrG90q6qfBTe4QgOF5mBYVIu8Wg TTwiXeIt ZVlgqiwiHn6Y75BEPdwMvdIs7GqOPLZ6JKdLkBdv6w90H58qiw1TNqdNPNHvL4mXxp/slgI5s0naJsa02GaIJm/g07ciGPR1nLLEjRIOmpOHjRGU0XQtWXbHcRg45JbBoMLgOSTVZYss84m/naGi9crm7KdVj1Do/d6TRxdDP1SqDhZ0Y/A0pYh27WaPQpDKpui9xMJMUn5j1seFrFHxph4v78mnGJgFSGRK2s+wE/nkDzEA9CBok2GhEmJdnrmXtcLbUVL8DUktpAqbr40ZuzcF9FZ/PYzaVhYzE X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Skip hugetlb vmas as we are not interested in those. Signed-off-by: Oscar Salvador --- mm/hugetlb_vmemmap.c | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/mm/hugetlb_vmemmap.c b/mm/hugetlb_vmemmap.c index 829112b0a914..3e6fd5ae27bd 100644 --- a/mm/hugetlb_vmemmap.c +++ b/mm/hugetlb_vmemmap.c @@ -151,9 +151,21 @@ static int vmemmap_pte_entry(pte_t *pte, unsigned long addr, return 0; } +static int vmemmap_test_walk(unsigned long start, unsigned long end, + struct mm_walk *walk) +{ + struct vm_area_struct *vma = walk->vma; + + if (is_vm_hugetlb_page(vma)) + return 1; + + return 0; +} + static const struct mm_walk_ops vmemmap_remap_ops = { .pmd_entry = vmemmap_pmd_entry, .pte_entry = vmemmap_pte_entry, + .test_walk = vmemmap_test_walk, }; static int vmemmap_remap_range(unsigned long start, unsigned long end, From patchwork Thu Jul 4 04:31:32 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oscar Salvador X-Patchwork-Id: 13723223 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9A503C30653 for ; Thu, 4 Jul 2024 04:34:59 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 01D046B00D1; Thu, 4 Jul 2024 00:32:51 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id EE8006B00D2; Thu, 4 Jul 2024 00:32:50 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C77406B00D3; Thu, 4 Jul 2024 00:32:50 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id A04126B00D1 for ; Thu, 4 Jul 2024 00:32:50 -0400 (EDT) Received: from smtpin13.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 559B240F4B for ; Thu, 4 Jul 2024 04:32:50 +0000 (UTC) X-FDA: 82300799700.13.C01F67A Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.223.131]) by imf20.hostedemail.com (Postfix) with ESMTP id 42F711C0015 for ; Thu, 4 Jul 2024 04:32:48 +0000 (UTC) Authentication-Results: imf20.hostedemail.com; dkim=none; spf=pass (imf20.hostedemail.com: domain of osalvador@suse.de designates 195.135.223.131 as permitted sender) smtp.mailfrom=osalvador@suse.de; dmarc=pass (policy=none) header.from=suse.de ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1720067555; a=rsa-sha256; cv=none; b=lgbsD+qvP/KC8ZX/UH0m6WJTHtOwwSZBYJL8TrazNe0UUgr3GSwbDI62rdRqSteEh+865S v/An0I4poz6PW/vzLgij6st7Af9A3uq+P4UVFQIPU8zGFkCfIwCnoxpjE+1JKjtgmsFOh1 FjxicyMRhgXp9v6b0cPQgUTIlzSEt90= ARC-Authentication-Results: i=1; imf20.hostedemail.com; dkim=none; spf=pass (imf20.hostedemail.com: domain of osalvador@suse.de designates 195.135.223.131 as permitted sender) smtp.mailfrom=osalvador@suse.de; dmarc=pass (policy=none) header.from=suse.de ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1720067555; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Y5QFJXv81b4111n8jLgilOPk2qT5XCixzAqKwvrmWNc=; b=zLpFyRVkDS6TNXmL3Y0yDdDB9DGujREtIweaUzKylIuF9diFDGW2lVt4RYtrCi0E/ccUC9 N6EavPgXmqD0PbKnrQ6rPDsmSZr33GRHnrMbjiSSBhOW3DP6SdBX420PI7ZWeZw37B0U1P m9eC4ZGjUvJOajqAofqK5GTRAsOcLlM= Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [IPv6:2a07:de40:b281:104:10:150:64:97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id 1983A1FCF7; Thu, 4 Jul 2024 04:32:47 +0000 (UTC) Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 06B7013889; Thu, 4 Jul 2024 04:32:45 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id 0LqbOO0lhmbnDAAAD6G6ig (envelope-from ); Thu, 04 Jul 2024 04:32:45 +0000 From: Oscar Salvador To: Andrew Morton Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Peter Xu , Muchun Song , David Hildenbrand , SeongJae Park , Miaohe Lin , Michal Hocko , Matthew Wilcox , Christophe Leroy , Oscar Salvador Subject: [PATCH 45/45] mm: Delete all hugetlb_entry entries Date: Thu, 4 Jul 2024 06:31:32 +0200 Message-ID: <20240704043132.28501-46-osalvador@suse.de> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240704043132.28501-1-osalvador@suse.de> References: <20240704043132.28501-1-osalvador@suse.de> MIME-Version: 1.0 X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Rspamd-Action: no action X-Stat-Signature: uqnpcfwhfx3u7zbhiqqhicki1n188x6x X-Rspamd-Queue-Id: 42F711C0015 X-Rspam-User: X-Rspamd-Server: rspam10 X-HE-Tag: 1720067568-471122 X-HE-Meta: U2FsdGVkX1+65MWaZgQrAixLBzcK5YU262j/7gNuus2yJMiss1229vGGlv656EuqRlA5qZ2v0LO84YYSFSM7A3DAqUa9+Axh7nOPxsIFj0+CDWKLfpu2tVJWBY+AYH0QzE/qtV3IUnBpCTw+mAfTGcOSwEKY39rYvJYgYsS5RjeTnPxNxk0r+HiH0KxObmQ/aZUgQZsz3rk5A0scrE44J8ZFwmrz1WG78fXaqhM86MU5ylrLr9qTzjkDyft2+oRnnCKQtmqnCle+TYjnzqh86zq+Ld+TIzrMJdLh9N6pxi+cHNVbEQ2XWTLC6w/2KuXYNnIppaQI1vhARzcWcDGvNx7McaekZB31PsUsTCMA0kn6fXJAVnFqQ09blGxZP1dAJU9tGYl3JU4OSNnf/6p5MPs9F0sm0UwxY1ruG47FZcIdKVxvTTuB/Pg5ctOeHYW7oNlz/LWwL2hHTiHgPJwwlPkGDoB2NDt/t2f/hI6CU4Mp+x6uMI/nSuCAmZqDUw+CTmtshqCvk6xq7d7QL/L7QjoT1yYDBewBbDQC9r+2jNquRl78ovAbX9Xc4nGlbyAqGFjjDAdEmLhtoVbozLNloDFOXB/thU8RAjzcWHDBsJ3smOA6irbasODPBSUF1LenR3eE5FxnT83mrsuXbcIISOCSMluCv7XrXFUzYo1BoIzZS9iUy5JFTAJrpB+xI3mLvBgFZC7rTVrBGHvPhnfY2FKsWQmM1sD0916wrE80hR8qn5Hj72Emvcwfaeb6UNujSzRRQYCFPIrDzLyLkDGnsH98cMi0Fy9SdbNJsPI4LPoQM3aHkDIu5+gNgk1iNto8XzWRGKhzyZKdhqab76QtFRGmBq8b4JMGLV3NHY0rT6rKY70k0HcCISV9eGYWbw3aGWwBOLJ+/hPvIqNhCo5nMIwoFl5yFGmXSaSwDh+gEfIaaAELv5qGDmMSpIhtnAsNF7ipEa+F9dEuyxImz4T R2fevKtC WMmPHnsegdVDA/60+8T4XJszmddI+aASPsTr+MRGidkDR3YUDP8IiIXElaA+xRE6vaBfnyJMmbBqCX4Ee7FWO+Q/7952xScywq4hshXyafLt5DWFWqosZiybuGwyFKquWMtq+qJu+QABWKtH6f1z2rBoJBujxtWW/3Vfs9n85f8GB698= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Generic pagewalker can now deal with hugetlb pages as well, so there is no need for specific hugetlb_entry functions. Drop them all. Signed-off-by: Oscar Salvador --- arch/s390/mm/gmap.c | 27 ------ fs/proc/task_mmu.c | 181 --------------------------------------- include/linux/pagewalk.h | 10 --- mm/damon/vaddr.c | 89 ------------------- mm/hmm.c | 54 ------------ mm/memory-failure.c | 17 ---- mm/mempolicy.c | 47 ---------- mm/mincore.c | 22 ----- mm/mprotect.c | 10 --- mm/pagewalk.c | 49 +---------- 10 files changed, 1 insertion(+), 505 deletions(-) diff --git a/arch/s390/mm/gmap.c b/arch/s390/mm/gmap.c index 580e4ab6f018..3307f0ec505c 100644 --- a/arch/s390/mm/gmap.c +++ b/arch/s390/mm/gmap.c @@ -2743,34 +2743,7 @@ static int __s390_enable_skey_pmd(pmd_t *pmd, unsigned long addr, return 0; } -static int __s390_enable_skey_hugetlb(pte_t *pte, unsigned long addr, - unsigned long hmask, unsigned long next, - struct mm_walk *walk) -{ - pmd_t *pmd = (pmd_t *)pte; - unsigned long start, end; - struct page *page = pmd_page(*pmd); - - /* - * The write check makes sure we do not set a key on shared - * memory. This is needed as the walker does not differentiate - * between actual guest memory and the process executable or - * shared libraries. - */ - if (pmd_val(*pmd) & _SEGMENT_ENTRY_INVALID || - !(pmd_val(*pmd) & _SEGMENT_ENTRY_WRITE)) - return 0; - - start = pmd_val(*pmd) & HPAGE_MASK; - end = start + HPAGE_SIZE; - __storage_key_init_range(start, end); - set_bit(PG_arch_1, &page->flags); - cond_resched(); - return 0; -} - static const struct mm_walk_ops enable_skey_walk_ops = { - .hugetlb_entry = __s390_enable_skey_hugetlb, .pte_entry = __s390_enable_skey_pte, .pmd_entry = __s390_enable_skey_pmd, .walk_lock = PGWALK_WRLOCK, diff --git a/fs/proc/task_mmu.c b/fs/proc/task_mmu.c index df94f2093588..52fa82336825 100644 --- a/fs/proc/task_mmu.c +++ b/fs/proc/task_mmu.c @@ -1066,52 +1066,15 @@ static void show_smap_vma_flags(struct seq_file *m, struct vm_area_struct *vma) seq_putc(m, '\n'); } -#ifdef CONFIG_HUGETLB_PAGE -static int smaps_hugetlb_range(pte_t *pte, unsigned long hmask, - unsigned long addr, unsigned long end, - struct mm_walk *walk) -{ - struct mem_size_stats *mss = walk->private; - struct vm_area_struct *vma = walk->vma; - pte_t ptent = huge_ptep_get(walk->mm, addr, pte); - struct folio *folio = NULL; - bool present = false; - - if (pte_present(ptent)) { - folio = page_folio(pte_page(ptent)); - present = true; - } else if (is_swap_pte(ptent)) { - swp_entry_t swpent = pte_to_swp_entry(ptent); - - if (is_pfn_swap_entry(swpent)) - folio = pfn_swap_entry_folio(swpent); - } - - if (folio) { - /* We treat non-present entries as "maybe shared". */ - if (!present || folio_likely_mapped_shared(folio) || - hugetlb_pmd_shared(pte)) - mss->shared_hugetlb += huge_page_size(hstate_vma(vma)); - else - mss->private_hugetlb += huge_page_size(hstate_vma(vma)); - } - return 0; -} -#else -#define smaps_hugetlb_range NULL -#endif /* HUGETLB_PAGE */ - static const struct mm_walk_ops smaps_walk_ops = { .pud_entry = smaps_pud_range, .pmd_entry = smaps_pte_range, - .hugetlb_entry = smaps_hugetlb_range, .walk_lock = PGWALK_RDLOCK, }; static const struct mm_walk_ops smaps_shmem_walk_ops = { .pud_entry = smaps_pud_range, .pmd_entry = smaps_pte_range, - .hugetlb_entry = smaps_hugetlb_range, .pte_hole = smaps_pte_hole, .walk_lock = PGWALK_RDLOCK, }; @@ -1987,66 +1950,10 @@ static int pagemap_pud_range(pud_t *pudp, unsigned long addr, unsigned long end, #define pagemap_pud_range NULL #endif -#ifdef CONFIG_HUGETLB_PAGE -/* This function walks within one hugetlb entry in the single call */ -static int pagemap_hugetlb_range(pte_t *ptep, unsigned long hmask, - unsigned long addr, unsigned long end, - struct mm_walk *walk) -{ - struct pagemapread *pm = walk->private; - struct vm_area_struct *vma = walk->vma; - u64 flags = 0, frame = 0; - int err = 0; - pte_t pte; - - if (vma->vm_flags & VM_SOFTDIRTY) - flags |= PM_SOFT_DIRTY; - - pte = huge_ptep_get(walk->mm, addr, ptep); - if (pte_present(pte)) { - struct folio *folio = page_folio(pte_page(pte)); - - if (!folio_test_anon(folio)) - flags |= PM_FILE; - - if (!folio_likely_mapped_shared(folio) && - !hugetlb_pmd_shared(ptep)) - flags |= PM_MMAP_EXCLUSIVE; - - if (huge_pte_uffd_wp(pte)) - flags |= PM_UFFD_WP; - - flags |= PM_PRESENT; - if (pm->show_pfn) - frame = pte_pfn(pte) + - ((addr & ~hmask) >> PAGE_SHIFT); - } else if (pte_swp_uffd_wp_any(pte)) { - flags |= PM_UFFD_WP; - } - - for (; addr != end; addr += PAGE_SIZE) { - pagemap_entry_t pme = make_pme(frame, flags); - - err = add_to_pagemap(&pme, pm); - if (err) - return err; - if (pm->show_pfn && (flags & PM_PRESENT)) - frame++; - } - - cond_resched(); - - return err; -} -#else -#define pagemap_hugetlb_range NULL -#endif /* HUGETLB_PAGE */ - static const struct mm_walk_ops pagemap_ops = { .pud_entry = pagemap_pud_range, .pmd_entry = pagemap_pmd_range, .pte_hole = pagemap_pte_hole, - .hugetlb_entry = pagemap_hugetlb_range, .walk_lock = PGWALK_RDLOCK, }; @@ -2789,67 +2696,6 @@ static int pagemap_scan_pud_entry(pud_t *pud, unsigned long start, #define pagemap_scan_pud_entry NULL #endif -#ifdef CONFIG_HUGETLB_PAGE -static int pagemap_scan_hugetlb_entry(pte_t *ptep, unsigned long hmask, - unsigned long start, unsigned long end, - struct mm_walk *walk) -{ - struct pagemap_scan_private *p = walk->private; - struct vm_area_struct *vma = walk->vma; - unsigned long categories; - spinlock_t *ptl; - int ret = 0; - pte_t pte; - - if (~p->arg.flags & PM_SCAN_WP_MATCHING) { - /* Go the short route when not write-protecting pages. */ - - pte = huge_ptep_get(walk->mm, start, ptep); - categories = p->cur_vma_category | pagemap_hugetlb_category(pte); - - if (!pagemap_scan_is_interesting_page(categories, p)) - return 0; - - return pagemap_scan_output(categories, p, start, &end); - } - - i_mmap_lock_write(vma->vm_file->f_mapping); - ptl = huge_pte_lock(hstate_vma(vma), vma->vm_mm, ptep); - - pte = huge_ptep_get(walk->mm, start, ptep); - categories = p->cur_vma_category | pagemap_hugetlb_category(pte); - - if (!pagemap_scan_is_interesting_page(categories, p)) - goto out_unlock; - - ret = pagemap_scan_output(categories, p, start, &end); - if (start == end) - goto out_unlock; - - if (~categories & PAGE_IS_WRITTEN) - goto out_unlock; - - if (end != start + HPAGE_SIZE) { - /* Partial HugeTLB page WP isn't possible. */ - pagemap_scan_backout_range(p, start, end); - p->arg.walk_end = start; - ret = 0; - goto out_unlock; - } - - make_uffd_wp_huge_pte(vma, start, ptep, pte); - flush_hugetlb_tlb_range(vma, start, end); - -out_unlock: - spin_unlock(ptl); - i_mmap_unlock_write(vma->vm_file->f_mapping); - - return ret; -} -#else -#define pagemap_scan_hugetlb_entry NULL -#endif - static int pagemap_scan_pte_hole(unsigned long addr, unsigned long end, int depth, struct mm_walk *walk) { @@ -2879,7 +2725,6 @@ static const struct mm_walk_ops pagemap_scan_ops = { .pud_entry = pagemap_scan_pud_entry, .pmd_entry = pagemap_scan_pmd_entry, .pte_hole = pagemap_scan_pte_hole, - .hugetlb_entry = pagemap_scan_hugetlb_entry, }; static int pagemap_scan_get_args(struct pm_scan_arg *arg, @@ -3275,34 +3120,8 @@ static int gather_pte_stats(pmd_t *pmd, unsigned long addr, cond_resched(); return 0; } -#ifdef CONFIG_HUGETLB_PAGE -static int gather_hugetlb_stats(pte_t *pte, unsigned long hmask, - unsigned long addr, unsigned long end, struct mm_walk *walk) -{ - pte_t huge_pte = huge_ptep_get(walk->mm, addr, pte); - struct numa_maps *md; - struct page *page; - - if (!pte_present(huge_pte)) - return 0; - - page = pte_page(huge_pte); - - md = walk->private; - gather_stats(page, md, pte_dirty(huge_pte), 1); - return 0; -} - -#else -static int gather_hugetlb_stats(pte_t *pte, unsigned long hmask, - unsigned long addr, unsigned long end, struct mm_walk *walk) -{ - return 0; -} -#endif static const struct mm_walk_ops show_numa_ops = { - .hugetlb_entry = gather_hugetlb_stats, .pud_entry = gather_pud_stats, .pmd_entry = gather_pte_stats, .walk_lock = PGWALK_RDLOCK, diff --git a/include/linux/pagewalk.h b/include/linux/pagewalk.h index 27cd1e59ccf7..6df0726eecb6 100644 --- a/include/linux/pagewalk.h +++ b/include/linux/pagewalk.h @@ -31,16 +31,6 @@ enum page_walk_lock { * depth is -1 if not known, 0:PGD, 1:P4D, 2:PUD, 3:PMD. * Any folded depths (where PTRS_PER_P?D is equal to 1) * are skipped. - * @hugetlb_entry: if set, called for each hugetlb entry. This hook - * function is called with the vma lock held, in order to - * protect against a concurrent freeing of the pte_t* or - * the ptl. In some cases, the hook function needs to drop - * and retake the vma lock in order to avoid deadlocks - * while calling other functions. In such cases the hook - * function must either refrain from accessing the pte or - * ptl after dropping the vma lock, or else revalidate - * those items after re-acquiring the vma lock and before - * accessing them. * @test_walk: caller specific callback function to determine whether * we walk over the current vma or not. Returning 0 means * "do page table walk over the current vma", returning diff --git a/mm/damon/vaddr.c b/mm/damon/vaddr.c index 6a383ce5a775..82a8d3146f05 100644 --- a/mm/damon/vaddr.c +++ b/mm/damon/vaddr.c @@ -360,63 +360,9 @@ static int damon_mkold_pud_entry(pmd_t *pud, unsigned long addr, return 0; } -#ifdef CONFIG_HUGETLB_PAGE -static void damon_hugetlb_mkold(pte_t *pte, struct mm_struct *mm, - struct vm_area_struct *vma, unsigned long addr) -{ - bool referenced = false; - pte_t entry = huge_ptep_get(mm, addr, pte); - struct folio *folio = pfn_folio(pte_pfn(entry)); - unsigned long psize = huge_page_size(hstate_vma(vma)); - - folio_get(folio); - - if (pte_young(entry)) { - referenced = true; - entry = pte_mkold(entry); - set_huge_pte_at(mm, addr, pte, entry, psize); - } - -#ifdef CONFIG_MMU_NOTIFIER - if (mmu_notifier_clear_young(mm, addr, - addr + huge_page_size(hstate_vma(vma)))) - referenced = true; -#endif /* CONFIG_MMU_NOTIFIER */ - - if (referenced) - folio_set_young(folio); - - folio_set_idle(folio); - folio_put(folio); -} - -static int damon_mkold_hugetlb_entry(pte_t *pte, unsigned long hmask, - unsigned long addr, unsigned long end, - struct mm_walk *walk) -{ - struct hstate *h = hstate_vma(walk->vma); - spinlock_t *ptl; - pte_t entry; - - ptl = huge_pte_lock(h, walk->mm, pte); - entry = huge_ptep_get(walk->mm, addr, pte); - if (!pte_present(entry)) - goto out; - - damon_hugetlb_mkold(pte, walk->mm, walk->vma, addr); - -out: - spin_unlock(ptl); - return 0; -} -#else -#define damon_mkold_hugetlb_entry NULL -#endif /* CONFIG_HUGETLB_PAGE */ - static const struct mm_walk_ops damon_mkold_ops = { .pud_entry = damon_mkold_pud_entry, .pmd_entry = damon_mkold_pmd_entry, - .hugetlb_entry = damon_mkold_hugetlb_entry, .walk_lock = PGWALK_RDLOCK, }; @@ -562,44 +508,9 @@ static int damon_young_pmd_entry(pmd_t *pmd, unsigned long addr, return 0; } -#ifdef CONFIG_HUGETLB_PAGE -static int damon_young_hugetlb_entry(pte_t *pte, unsigned long hmask, - unsigned long addr, unsigned long end, - struct mm_walk *walk) -{ - struct damon_young_walk_private *priv = walk->private; - struct hstate *h = hstate_vma(walk->vma); - struct folio *folio; - spinlock_t *ptl; - pte_t entry; - - ptl = huge_pte_lock(h, walk->mm, pte); - entry = huge_ptep_get(walk->mm, addr, pte); - if (!pte_present(entry)) - goto out; - - folio = pfn_folio(pte_pfn(entry)); - folio_get(folio); - - if (pte_young(entry) || !folio_test_idle(folio) || - mmu_notifier_test_young(walk->mm, addr)) - priv->young = true; - *priv->folio_sz = huge_page_size(h); - - folio_put(folio); - -out: - spin_unlock(ptl); - return 0; -} -#else -#define damon_young_hugetlb_entry NULL -#endif /* CONFIG_HUGETLB_PAGE */ - static const struct mm_walk_ops damon_young_ops = { .pud_entry = damon_young_pud_entry, .pmd_entry = damon_young_pmd_entry, - .hugetlb_entry = damon_young_hugetlb_entry, .walk_lock = PGWALK_RDLOCK, }; diff --git a/mm/hmm.c b/mm/hmm.c index 2b752f703b6d..fccde5dae818 100644 --- a/mm/hmm.c +++ b/mm/hmm.c @@ -463,59 +463,6 @@ static int hmm_vma_walk_pud(pud_t *pudp, unsigned long start, unsigned long end, #define hmm_vma_walk_pud NULL #endif -#ifdef CONFIG_HUGETLB_PAGE -static int hmm_vma_walk_hugetlb_entry(pte_t *pte, unsigned long hmask, - unsigned long start, unsigned long end, - struct mm_walk *walk) -{ - unsigned long addr = start, i, pfn; - struct hmm_vma_walk *hmm_vma_walk = walk->private; - struct hmm_range *range = hmm_vma_walk->range; - struct vm_area_struct *vma = walk->vma; - unsigned int required_fault; - unsigned long pfn_req_flags; - unsigned long cpu_flags; - spinlock_t *ptl; - pte_t entry; - - ptl = huge_pte_lock(hstate_vma(vma), walk->mm, pte); - entry = huge_ptep_get(walk->mm, addr, pte); - - i = (start - range->start) >> PAGE_SHIFT; - pfn_req_flags = range->hmm_pfns[i]; - cpu_flags = pte_to_hmm_pfn_flags(range, entry) | - hmm_pfn_flags_order(huge_page_order(hstate_vma(vma))); - required_fault = - hmm_pte_need_fault(hmm_vma_walk, pfn_req_flags, cpu_flags); - if (required_fault) { - int ret; - - spin_unlock(ptl); - hugetlb_vma_unlock_read(vma); - /* - * Avoid deadlock: drop the vma lock before calling - * hmm_vma_fault(), which will itself potentially take and - * drop the vma lock. This is also correct from a - * protection point of view, because there is no further - * use here of either pte or ptl after dropping the vma - * lock. - */ - ret = hmm_vma_fault(addr, end, required_fault, walk); - hugetlb_vma_lock_read(vma); - return ret; - } - - pfn = pte_pfn(entry) + ((start & ~hmask) >> PAGE_SHIFT); - for (; addr < end; addr += PAGE_SIZE, i++, pfn++) - range->hmm_pfns[i] = pfn | cpu_flags; - - spin_unlock(ptl); - return 0; -} -#else -#define hmm_vma_walk_hugetlb_entry NULL -#endif /* CONFIG_HUGETLB_PAGE */ - static int hmm_vma_walk_test(unsigned long start, unsigned long end, struct mm_walk *walk) { @@ -554,7 +501,6 @@ static const struct mm_walk_ops hmm_walk_ops = { .pud_entry = hmm_vma_walk_pud, .pmd_entry = hmm_vma_walk_pmd, .pte_hole = hmm_vma_walk_hole, - .hugetlb_entry = hmm_vma_walk_hugetlb_entry, .test_walk = hmm_vma_walk_test, .walk_lock = PGWALK_RDLOCK, }; diff --git a/mm/memory-failure.c b/mm/memory-failure.c index 622862c4c300..c4ce4cf16651 100644 --- a/mm/memory-failure.c +++ b/mm/memory-failure.c @@ -882,26 +882,9 @@ static int hwpoison_pte_range(pmd_t *pmdp, unsigned long addr, return ret; } -#ifdef CONFIG_HUGETLB_PAGE -static int hwpoison_hugetlb_range(pte_t *ptep, unsigned long hmask, - unsigned long addr, unsigned long end, - struct mm_walk *walk) -{ - struct hwpoison_walk *hwp = walk->private; - pte_t pte = huge_ptep_get(walk->mm, addr, ptep); - struct hstate *h = hstate_vma(walk->vma); - - return check_hwpoisoned_entry(pte, addr, huge_page_shift(h), - hwp->pfn, &hwp->tk); -} -#else -#define hwpoison_hugetlb_range NULL -#endif - static const struct mm_walk_ops hwpoison_walk_ops = { .pud_entry = hwpoison_pud_range, .pmd_entry = hwpoison_pte_range, - .hugetlb_entry = hwpoison_hugetlb_range, .walk_lock = PGWALK_RDLOCK, }; diff --git a/mm/mempolicy.c b/mm/mempolicy.c index 93b14090d484..8b5ca719193c 100644 --- a/mm/mempolicy.c +++ b/mm/mempolicy.c @@ -645,51 +645,6 @@ static int queue_folios_pte_range(pmd_t *pmd, unsigned long addr, return 0; } -static int queue_folios_hugetlb(pte_t *pte, unsigned long hmask, - unsigned long addr, unsigned long end, - struct mm_walk *walk) -{ -#ifdef CONFIG_HUGETLB_PAGE - struct queue_pages *qp = walk->private; - unsigned long flags = qp->flags; - struct folio *folio; - spinlock_t *ptl; - pte_t entry; - - ptl = huge_pte_lock(hstate_vma(walk->vma), walk->mm, pte); - entry = huge_ptep_get(walk->mm, addr, pte); - if (!pte_present(entry)) { - if (unlikely(is_hugetlb_entry_migration(entry))) - qp->nr_failed++; - goto unlock; - } - folio = pfn_folio(pte_pfn(entry)); - if (!queue_folio_required(folio, qp)) - goto unlock; - if (!(flags & (MPOL_MF_MOVE | MPOL_MF_MOVE_ALL)) || - !vma_migratable(walk->vma)) { - qp->nr_failed++; - goto unlock; - } - /* - * Unless MPOL_MF_MOVE_ALL, we try to avoid migrating a shared folio. - * Choosing not to migrate a shared folio is not counted as a failure. - * - * See folio_likely_mapped_shared() on possible imprecision when we - * cannot easily detect if a folio is shared. - */ - if ((flags & MPOL_MF_MOVE_ALL) || - (!folio_likely_mapped_shared(folio) && !hugetlb_pmd_shared(pte))) - if (!isolate_hugetlb(folio, qp->pagelist)) - qp->nr_failed++; -unlock: - spin_unlock(ptl); - if (qp->nr_failed && strictly_unmovable(flags)) - return -EIO; -#endif - return 0; -} - #ifdef CONFIG_NUMA_BALANCING /* * This is used to mark a range of virtual addresses to be inaccessible. @@ -760,7 +715,6 @@ static int queue_pages_test_walk(unsigned long start, unsigned long end, } static const struct mm_walk_ops queue_pages_walk_ops = { - .hugetlb_entry = queue_folios_hugetlb, .pud_entry = queue_folios_pud, .pmd_entry = queue_folios_pte_range, .test_walk = queue_pages_test_walk, @@ -768,7 +722,6 @@ static const struct mm_walk_ops queue_pages_walk_ops = { }; static const struct mm_walk_ops queue_pages_lock_vma_walk_ops = { - .hugetlb_entry = queue_folios_hugetlb, .pmd_entry = queue_folios_pte_range, .test_walk = queue_pages_test_walk, .walk_lock = PGWALK_WRLOCK, diff --git a/mm/mincore.c b/mm/mincore.c index 786df7246899..26f699a47371 100644 --- a/mm/mincore.c +++ b/mm/mincore.c @@ -23,27 +23,6 @@ #include #include "swap.h" -static int mincore_hugetlb(pte_t *pte, unsigned long hmask, unsigned long addr, - unsigned long end, struct mm_walk *walk) -{ -#ifdef CONFIG_HUGETLB_PAGE - unsigned char present; - unsigned char *vec = walk->private; - - /* - * Hugepages under user process are always in RAM and never - * swapped out, but theoretically it needs to be checked. - */ - present = pte && !huge_pte_none_mostly(huge_ptep_get(walk->mm, addr, pte)); - for (; addr != end; vec++, addr += PAGE_SIZE) - *vec = present; - walk->private = vec; -#else - BUG(); -#endif - return 0; -} - /* * Later we can get more picky about what "in core" means precisely. * For now, simply check to see if the page is in the page cache, @@ -197,7 +176,6 @@ static const struct mm_walk_ops mincore_walk_ops = { .pud_entry = mincore_pud_range, .pmd_entry = mincore_pte_range, .pte_hole = mincore_unmapped_range, - .hugetlb_entry = mincore_hugetlb, .walk_lock = PGWALK_RDLOCK, }; diff --git a/mm/mprotect.c b/mm/mprotect.c index 222ab434da54..ca1962d5cb95 100644 --- a/mm/mprotect.c +++ b/mm/mprotect.c @@ -555,15 +555,6 @@ static int prot_none_pte_entry(pte_t *pte, unsigned long addr, 0 : -EACCES; } -static int prot_none_hugetlb_entry(pte_t *pte, unsigned long hmask, - unsigned long addr, unsigned long next, - struct mm_walk *walk) -{ - return pfn_modify_allowed(pte_pfn(ptep_get(pte)), - *(pgprot_t *)(walk->private)) ? - 0 : -EACCES; -} - static int prot_none_test(unsigned long addr, unsigned long next, struct mm_walk *walk) { @@ -572,7 +563,6 @@ static int prot_none_test(unsigned long addr, unsigned long next, static const struct mm_walk_ops prot_none_walk_ops = { .pte_entry = prot_none_pte_entry, - .hugetlb_entry = prot_none_hugetlb_entry, .test_walk = prot_none_test, .walk_lock = PGWALK_WRLOCK, }; diff --git a/mm/pagewalk.c b/mm/pagewalk.c index 78d45f1450aa..7e2721f49e68 100644 --- a/mm/pagewalk.c +++ b/mm/pagewalk.c @@ -257,49 +257,6 @@ static int walk_pgd_range(unsigned long addr, unsigned long end, return err; } -#ifdef CONFIG_HUGETLB_PAGE -static unsigned long hugetlb_entry_end(struct hstate *h, unsigned long addr, - unsigned long end) -{ - unsigned long boundary = (addr & huge_page_mask(h)) + huge_page_size(h); - return boundary < end ? boundary : end; -} - -static int walk_hugetlb_range(unsigned long addr, unsigned long end, - struct mm_walk *walk) -{ - struct vm_area_struct *vma = walk->vma; - struct hstate *h = hstate_vma(vma); - unsigned long next; - unsigned long hmask = huge_page_mask(h); - unsigned long sz = huge_page_size(h); - pte_t *pte; - const struct mm_walk_ops *ops = walk->ops; - int err = 0; - - do { - next = hugetlb_entry_end(h, addr, end); - pte = hugetlb_walk(vma, addr & hmask, sz); - if (pte) - err = ops->hugetlb_entry(pte, hmask, addr, next, walk); - else if (ops->pte_hole) - err = ops->pte_hole(addr, next, -1, walk); - if (err) - break; - } while (addr = next, addr != end); - - return err; -} - -#else /* CONFIG_HUGETLB_PAGE */ -static int walk_hugetlb_range(unsigned long addr, unsigned long end, - struct mm_walk *walk) -{ - return 0; -} - -#endif /* CONFIG_HUGETLB_PAGE */ - /* * Decide whether we really walk over the current vma on [@start, @end) * or skip it via the returned value. Return 0 if we do walk over the @@ -346,11 +303,7 @@ static int __walk_page_range(unsigned long start, unsigned long end, } vma_pgtable_walk_begin(vma); - if (is_vm_hugetlb_page(vma)) { - if (ops->hugetlb_entry) - err = walk_hugetlb_range(start, end, walk); - } else - err = walk_pgd_range(start, end, walk); + err = walk_pgd_range(start, end, walk); vma_pgtable_walk_end(vma); if (ops->post_vma)