From patchwork Wed Jul 17 22:02:13 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Xu X-Patchwork-Id: 13735848 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 79A0CC3DA62 for ; Wed, 17 Jul 2024 22:02:38 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 84B4E6B0095; Wed, 17 Jul 2024 18:02:35 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 7FB706B00B9; Wed, 17 Jul 2024 18:02:35 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 5B2436B00BA; Wed, 17 Jul 2024 18:02:35 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 3A6296B0095 for ; Wed, 17 Jul 2024 18:02:35 -0400 (EDT) Received: from smtpin24.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id B0624140CC7 for ; Wed, 17 Jul 2024 22:02:34 +0000 (UTC) X-FDA: 82350619428.24.A306B0A Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by imf23.hostedemail.com (Postfix) with ESMTP id E97F4140011 for ; Wed, 17 Jul 2024 22:02:30 +0000 (UTC) Authentication-Results: imf23.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=hu3Jkpkv; spf=pass (imf23.hostedemail.com: domain of peterx@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=peterx@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1721253733; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding:in-reply-to: references:dkim-signature; bh=22URfq8NjvGndByOojJP9eQog6fnp2xGbHVHJb+hEIw=; b=gpMb24AuX2UmzHZIN+oI8dG/DHqbXFlldMxF7+uHFzjQ1U/Rodg6VkYFQlrgT+IB6rZYHS yCAJ89W8gaRX6mg6oiXRXrCFzMC6dqPnxWV6CbiGAvX/oMZ3ZlN/yfOjzxLMzRR9sQ8yQp RcR9R9xtDwkbnwtpDsQPj+2MDengBYU= ARC-Authentication-Results: i=1; imf23.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=hu3Jkpkv; spf=pass (imf23.hostedemail.com: domain of peterx@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=peterx@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1721253733; a=rsa-sha256; cv=none; b=XG1BPM426kWTerYfal75EJ09wYU+FyeMT7+w6Zi6eKiu8I2Z7ZMFcd19rvYRi037nZE+HV HYKMoKUTPgsdNxb43Eey7eB0Pfmzn5wrfQFGsATl9rYpKQoYrTeoFmKKliiwlGeCUjOp32 nNfOruSQZ2RfGCJnzK7tjlCkDqQGG5Q= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1721253750; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=22URfq8NjvGndByOojJP9eQog6fnp2xGbHVHJb+hEIw=; b=hu3Jkpkv859IsBgXtYdMvzzjcMoMvi+lnXECKTeSpq4ZvZwYQzdZTeV8wv972WA+eIsiTb vi2XgjM7oW3MyQJMjhpjbShk0KUwAqijXDRleeBKz7TqfUS93YFaF6Ad21/iwZVG7zUDHd 7S2AA/J4cpKWUg2CR+Es64rhqoAZ6P8= Received: from mail-qv1-f69.google.com (mail-qv1-f69.google.com [209.85.219.69]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-45-T6iRdDjDPgqjuNe1o5Llmw-1; Wed, 17 Jul 2024 18:02:23 -0400 X-MC-Unique: T6iRdDjDPgqjuNe1o5Llmw-1 Received: by mail-qv1-f69.google.com with SMTP id 6a1803df08f44-6b7740643fbso589976d6.1 for ; Wed, 17 Jul 2024 15:02:23 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1721253743; x=1721858543; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=22URfq8NjvGndByOojJP9eQog6fnp2xGbHVHJb+hEIw=; b=xLy8tDbAFelpLXhphuSdT6FfCkfu0kTQ+Dq6BITCOJBxj7KLyAWp1A9RRepgTj1kzH BlBuQEHVkOoQdT2OE8tOVYAu6E/yFkDi4DcVtM0407n4VQz2QQ5MMJNkIyo738n8vbMI 9N4iYzr4FO3cRT6r08Cdixc1WpyVyChjrr4aQJJZgyieHWCsGL3N3ySOpOZxAn/0AYD+ Z3YGClPlniCXUscA57PLgKk+wca6j5dFkHK/A1g1nr1RF9ABlErksHW/57Kkt75R/h57 Re7OWy7iOKtMkDHpvUXrx53X0CRdzkAoQsr+ZAWyLwomFx/e2w58Pg8hKcEiAmTvaSjO 0Kqw== X-Forwarded-Encrypted: i=1; AJvYcCWjs7qCw72gbkuV6ZlHNB6GmojoZ30GfPXUif5Aw6Z2pDGlrE0xTw1enNSLcoLuJjzo3d2Bdt4Cuw==@kvack.org X-Gm-Message-State: AOJu0YxBKuty+ckXfSZHciMSh1AXc/jltcUI0AdaOJHk3KvnTS16I9Nt 2tWB7B7+kJgDeodcT5Ok3b3kC0cIGfn23lFW81QNccxI2tkOaOOJwfJAV+SZVR3Ooli8OC/MlVG tg+uOOFwCdWiEYZpHUKulw7HQnC+DhhjbnNuErsaY0KSPEIef X-Received: by 2002:ac8:5e4e:0:b0:44f:89e3:e8d2 with SMTP id d75a77b69052e-44f89e3ec09mr15923371cf.12.1721253742630; Wed, 17 Jul 2024 15:02:22 -0700 (PDT) X-Google-Smtp-Source: AGHT+IH030U/0PuH20CJyPZJMzLVzXnymjVCbugSgXnKR5JUMtRiVtVl+1d1cBUhmI+Iq8e7T2bLTA== X-Received: by 2002:ac8:5e4e:0:b0:44f:89e3:e8d2 with SMTP id d75a77b69052e-44f89e3ec09mr15923181cf.12.1721253742192; Wed, 17 Jul 2024 15:02:22 -0700 (PDT) Received: from x1n.redhat.com (pool-99-254-121-117.cpe.net.cable.rogers.com. [99.254.121.117]) by smtp.gmail.com with ESMTPSA id d75a77b69052e-44f5b83f632sm53071651cf.85.2024.07.17.15.02.20 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 17 Jul 2024 15:02:21 -0700 (PDT) From: Peter Xu To: linux-kernel@vger.kernel.org, linux-mm@kvack.org Cc: Vlastimil Babka , peterx@redhat.com, David Hildenbrand , Oscar Salvador , linux-s390@vger.kernel.org, Andrew Morton , Matthew Wilcox , Dan Williams , Michal Hocko , linux-riscv@lists.infradead.org, sparclinux@vger.kernel.org, Alex Williamson , Jason Gunthorpe , x86@kernel.org, Alistair Popple , linuxppc-dev@lists.ozlabs.org, linux-arm-kernel@lists.infradead.org, Ryan Roberts , Hugh Dickins , Axel Rasmussen Subject: [PATCH RFC 0/6] mm: THP-agnostic refactor on huge mappings Date: Wed, 17 Jul 2024 18:02:13 -0400 Message-ID: <20240717220219.3743374-1-peterx@redhat.com> X-Mailer: git-send-email 2.45.0 MIME-Version: 1.0 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-Rspam-User: X-Rspamd-Server: rspam01 X-Rspamd-Queue-Id: E97F4140011 X-Stat-Signature: 8137sntqz15msgrp9py3gnaepsktxgt3 X-HE-Tag: 1721253750-59655 X-HE-Meta: U2FsdGVkX1/JlGxpxwfLIIkR5Z9WwUpO8y6pX/WwiAV+aYgyVICeW3yxAW5wN3PzuqN9tKEPOwgk70BEXQ+89YIZNWda4Ee0dPLT9++8Qn2J26cyYBIsmZ6s10PZdyEt/OCeyC3hODndqnZzFWiyFsBzBl0/XC6bpQNcdM8TqR3JRmF4BXmN51F6kPFI+diICC1cRvzcGNfL0Yb2oSrwzEpvq13KOk07soRFwy9V/oNc1G6bcQu4YGoMk6UGJCPL9WahD4SnBQEj1hZXfZoqZwM+ncUsYzbaHPBr/htgm3nd6if4vEEythLkt8My+LVOc0uAqyba+c8mK/KFQScjEYcwO8PMkV8Pn7p60+6oaxV4dkMQDf3lzi92aN1U75C1oNjuJtu7mbbEIdrDfWjjoweqCbp7Th0MPhi12wpIjnWhBxWb+yrFQY12KtDhDSBRwGCHCRCrwNQPYnFUdvQLDfYgfhXkAyhWQI8C75/PI/+t+nE5DnWNdVTvM2linQKUDeXgwoms9HgsAagZlAFjToWlMy0WFKffsHR+6dAUbvxGWQetllu4IszDzTcfg2KAnrFmZJbUJADSpUcdHhfkZ1bCLDP05CxMhif8iGHYsCaF0R4oib4lqxvv/DZF793BXn+7DD/+EjKeiN5AjIOvBV99QsquNnSyTzhiuriwA7PjLWNMWTkJrt3WrVtn4dEHyvNl1sckchLI3a9Lt0oMHTGauMQ+ie18tgYefIm3ssHCe5BNKHSUIHSCI2AtUFcpnlemKvcxsyKyEZ2S6DGozZLqhVy2U62zSqNW1S1ISAn11J1R0zTz2GSN4rbXNBEsHxWeeSMc8fLw8BwzpNXsYn8KcYtpS3wdo/hPL7+c+Qq6G6Du0thI/aYXuih5Rj84bqY2kTxxkA88eeJKWcI8j3+mGLVrgIkjnkwoQZrFcmVPZ5EuQ91VZgxcu3r8Ql5L+fCpaD5xmU1qqSSQWrX S5IiUowH oXLHAMzRp003gDvU0ogY65uKxfufxs9gKrBHE8D/qq6qPyosA83VJxiznVqHod+VUb63oDBaihI/iDlBe4eNz1ipm4iYv6rn3seRF+sBDm9tlroSOuzIYq7OgE7tFuAd77Ebya713PQPBN1S6J+mu0PgP8TUP7UgUTAi7cW8ftAQSKGolSrHwsFEsOYsoUisku6EWDCAOAsRsntn66ScsQoR9gzdG9fG10rtHMUHYUSXqqcX/PB/opasVBKBV4IbBAG6cyiWXNNPzpFt5wHxe+2KvNCGOKetDkL1lT3d8o53GCKYoJGM0b6bmi0sPk/yjdWkzUsTsELkHHfJ3GnCHuk9d9dpDSbeVzjos X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: This is an RFC series, so not yet for merging. Please don't be scared by the code changes: most of them are code movements only. This series is based on the dax mprotect fix series here (while that one is based on mm-unstable): [PATCH v3 0/8] mm/mprotect: Fix dax puds https://lore.kernel.org/r/20240715192142.3241557-1-peterx@redhat.com Overview ======== This series doesn't provide any feature change. The only goal of this series is to start decoupling two ideas: "THP" and "huge mapping". We already started with having PGTABLE_HAS_HUGE_LEAVES config option, and this one extends that idea into the code. The issue is that we have so many functions that only compile with CONFIG_THP=on, even though they're about huge mappings, and huge mapping is a pretty common concept, which can apply to many things besides THPs nowadays. The major THP file is mm/huge_memory.c as of now. The first example of such huge mapping users will be hugetlb. We lived until now with no problem simply because Linux almost duplicated all the logics there in the "THP" files into hugetlb APIs. If we want to get rid of hugetlb specific APIs and paths, this _might_ be the first thing we want to do, because we want to be able to e.g., zapping a hugetlb pmd entry even if !CONFIG_THP. Then consider other things like dax / pfnmaps. Dax can depend on THP, then it'll naturally be able to use pmd/pud helpers, that's okay. However is it a must? Do we also want to have every new pmd/pud mappings in the future to depend on THP (like PFNMAP)? My answer is no, but I'm open to opinions. If anyone agrees with me that "huge mapping" (aka, PMD/PUD mappings that are larger than PAGE_SIZE) is a more generic concept than THP, then I think at some point we need to move the generic code out of THP code into a common code base. This is what this series does as a start. In general, this series tries to move many THP things (mostly resides in huge_memory.c right now) into two new files: huge_mapping_{pmd|pud}.c. When I move them out, I also put them separately into different files for different layers. Then if an arch supports e.g. only PMD, it can avoid compile the PUD helpers, with things like: CONFIG_PGTABLE_HAS_PUD_LEAVES=n obj-$(CONFIG_PGTABLE_HAS_PUD_LEAVES) += huge_mapping_pud.o Note that there're a few tree-wide changes into arch/, but that's not a lot, to make this not disturbing too much people, I only copied the open lists of each arch not yet the arch maintainers. Tests ===== My normal 19-archs cross-compilation tests pass with it, and smoke tested on x86_64 with a local config of mine. Comments welcomed, thanks. Peter Xu (6): mm/treewide: Remove pgd_devmap() mm: PGTABLE_HAS_P[MU]D_LEAVES config options mm/treewide: Make pgtable-generic.c THP agnostic mm: Move huge mapping declarations from internal.h to huge_mm.h mm/huge_mapping: Create huge_mapping_pxx.c mm: Convert "*_trans_huge() || *_devmap()" to use *_leaf() arch/arm64/include/asm/pgtable.h | 11 +- arch/powerpc/include/asm/book3s/64/pgtable.h | 7 +- arch/powerpc/mm/book3s64/pgtable.c | 2 +- arch/riscv/include/asm/pgtable.h | 4 +- arch/s390/include/asm/pgtable.h | 2 +- arch/s390/mm/pgtable.c | 4 +- arch/sparc/mm/tlb.c | 2 +- arch/x86/include/asm/pgtable.h | 5 - arch/x86/mm/pgtable.c | 15 +- include/linux/huge_mm.h | 332 ++++-- include/linux/mm.h | 18 + include/linux/mm_types.h | 2 +- include/linux/pgtable.h | 61 +- include/trace/events/huge_mapping.h | 41 + include/trace/events/thp.h | 28 - mm/Kconfig | 6 + mm/Makefile | 2 + mm/gup.c | 2 - mm/hmm.c | 4 +- mm/huge_mapping_pmd.c | 976 +++++++++++++++ mm/huge_mapping_pud.c | 235 ++++ mm/huge_memory.c | 1125 +----------------- mm/internal.h | 33 - mm/mapping_dirty_helpers.c | 4 +- mm/memory.c | 16 +- mm/migrate_device.c | 2 +- mm/mprotect.c | 4 +- mm/mremap.c | 5 +- mm/page_vma_mapped.c | 5 +- mm/pgtable-generic.c | 37 +- 30 files changed, 1595 insertions(+), 1395 deletions(-) create mode 100644 include/trace/events/huge_mapping.h create mode 100644 mm/huge_mapping_pmd.c create mode 100644 mm/huge_mapping_pud.c