From patchwork Mon Aug 19 15:15:06 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Suren Baghdasaryan X-Patchwork-Id: 13768487 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3EEFCC52D6F for ; Mon, 19 Aug 2024 15:15:19 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id BD9DC6B0085; Mon, 19 Aug 2024 11:15:18 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id B8BFF6B0088; Mon, 19 Aug 2024 11:15:18 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A520B6B0089; Mon, 19 Aug 2024 11:15:18 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 83AC86B0085 for ; Mon, 19 Aug 2024 11:15:18 -0400 (EDT) Received: from smtpin08.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id EE68A121317 for ; Mon, 19 Aug 2024 15:15:17 +0000 (UTC) X-FDA: 82469343474.08.522D782 Received: from mail-yw1-f202.google.com (mail-yw1-f202.google.com [209.85.128.202]) by imf06.hostedemail.com (Postfix) with ESMTP id 17C12180003 for ; Mon, 19 Aug 2024 15:15:15 +0000 (UTC) Authentication-Results: imf06.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=JwVsMzfZ; spf=pass (imf06.hostedemail.com: domain of 3gmHDZgYKCI8BDAx6uz77z4x.v75416DG-553Etv3.7Az@flex--surenb.bounces.google.com designates 209.85.128.202 as permitted sender) smtp.mailfrom=3gmHDZgYKCI8BDAx6uz77z4x.v75416DG-553Etv3.7Az@flex--surenb.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1724080438; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding:in-reply-to: references:dkim-signature; bh=A78JLOnxz0DbZH9XkqfQj68kUgx61DwizYim3aGCBzo=; b=2SYrHL+y45wlPoaEgZVwDNHUybbTYPC8e/gbMJZewK7DWsKHTll/m2y3yHb/qnKCaCaZyj Sh5KFVISluzwtQM2xbPnPo96VZVs8MtDDCwf4L6K0CVMFNo26rRsdAueFzeg3SD9/VJVQN pIzXU6biehhIs1HvfhYkafMQYPZZY1Y= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1724080438; a=rsa-sha256; cv=none; b=lfCqzT0ctP/AqQ9redXRzq0/+rMEL0okWasLkzaMFk5mrnhFPR+V5MXdelWAB0mT8kD78a fUXLi7Xrm2lJp1JA3z5pC+r/kwaY9M9KrHSK612d7Ki8QthNkSVC0JHILLONrzfKzVvDm2 oG/c8jYo4YOUibXqXlm/CJc4AxoIzRk= ARC-Authentication-Results: i=1; imf06.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=JwVsMzfZ; spf=pass (imf06.hostedemail.com: domain of 3gmHDZgYKCI8BDAx6uz77z4x.v75416DG-553Etv3.7Az@flex--surenb.bounces.google.com designates 209.85.128.202 as permitted sender) smtp.mailfrom=3gmHDZgYKCI8BDAx6uz77z4x.v75416DG-553Etv3.7Az@flex--surenb.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com Received: by mail-yw1-f202.google.com with SMTP id 00721157ae682-6b052c5939cso38876567b3.1 for ; Mon, 19 Aug 2024 08:15:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1724080515; x=1724685315; darn=kvack.org; h=cc:to:from:subject:message-id:mime-version:date:from:to:cc:subject :date:message-id:reply-to; bh=A78JLOnxz0DbZH9XkqfQj68kUgx61DwizYim3aGCBzo=; b=JwVsMzfZ4g4QmFpU7nScLmgD2RI+vtjuTAP1Riytgt10qpIhhfvpAREPC7lCb4zLaH zdPCPicyn5wvw7sxApsCmPsgKUkdHozqMuWLBky9gjvOqE5p4AooOgzdBtgS7KLG7ZB1 ike+EiZmz6vZxsal9+zIsEMIBiAbicwjuttGE4MZLz52IfCNc1M64W9GIXf5IretLGAa cGI0fUAuybVJv57Qi9uiLKxSVOzPM3nJtXPmvWd4Mg3TT4qioCtMfl+oQLd7mdMrGXZs g8YL3brvyPIWI/AaZOq04lKfCVpU118yphFZI+9IBZ2Ild9Ik5ObM8u+VPgfOTPx+vff 4zcQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1724080515; x=1724685315; h=cc:to:from:subject:message-id:mime-version:date:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=A78JLOnxz0DbZH9XkqfQj68kUgx61DwizYim3aGCBzo=; b=CuTIPXdh2saPjkb6o4fHVZWpPPP/VM6YhaqKctBFqZbbDv0VZPiPuvc1v0KMYCp+u8 Mvb+1txS1V5Xxev04BHFdMK7SiAvWcZJuxgrH0s4cG9q0RiPJwKzzvDGAkswylwL8N3E gdfWDg3/T9pYqnSXrZKqzKmGQOS7phk8raC8gnxEos5v+hQdm50FHkF/9FPVY9Ttxelo KL/PCzhL5uPpdQS4UGCPSHhQd5vYhONkeFX0lh2SHuX+tgzl4n0m0iVQAgxqvvNqm/rI aEsuIBpHaz3XKRCozkTTd8fTtQ7Ulw8Us8LklO3LMHggXy3W9C1uPVmq18qxRHwLVERx luMw== X-Forwarded-Encrypted: i=1; AJvYcCU/ejgCfgsiBoOemn3NCytNEqQyc6vS4MUT8Iat12sf7rqtKCDrP0AGJFKd3bkezilmfSncTumuLw==@kvack.org X-Gm-Message-State: AOJu0YzPUiTefNbrKoKYFGN7LDD5mBmNqXKKVIbMneZAhlHdIq74D+ay vP1+1nt4tStmh0+mTMZ1Rngzbadakf9rtvFTzxq4iCm1o/Xlo7hGiWqpNBxJ0DZJS1tzfIHgiRz eGw== X-Google-Smtp-Source: AGHT+IGn4wFVR9Ch9/vmxxSH0rhO9DsLWlVWUxLzfz71jRdOXd7qjKZt0JLceuItsIaKthQC3GCSSl3ZiT0= X-Received: from surenb-desktop.mtv.corp.google.com ([2620:15c:211:201:5aea:cf26:50f4:76db]) (user=surenb job=sendgmr) by 2002:a05:690c:2b06:b0:6b2:e9cd:ede9 with SMTP id 00721157ae682-6b2e9cdef05mr783597b3.3.1724080514873; Mon, 19 Aug 2024 08:15:14 -0700 (PDT) Date: Mon, 19 Aug 2024 08:15:06 -0700 Mime-Version: 1.0 X-Mailer: git-send-email 2.46.0.184.g6999bdac58-goog Message-ID: <20240819151512.2363698-1-surenb@google.com> Subject: [PATCH 0/5] page allocation tag compression From: Suren Baghdasaryan To: akpm@linux-foundation.org Cc: kent.overstreet@linux.dev, corbet@lwn.net, arnd@arndb.de, mcgrof@kernel.org, rppt@kernel.org, paulmck@kernel.org, thuth@redhat.com, tglx@linutronix.de, bp@alien8.de, xiongwei.song@windriver.com, ardb@kernel.org, david@redhat.com, vbabka@suse.cz, mhocko@suse.com, hannes@cmpxchg.org, roman.gushchin@linux.dev, dave@stgolabs.net, willy@infradead.org, liam.howlett@oracle.com, pasha.tatashin@soleen.com, souravpanda@google.com, keescook@chromium.org, dennis@kernel.org, jhubbard@nvidia.com, yuzhao@google.com, vvvvvv@google.com, rostedt@goodmis.org, iamjoonsoo.kim@lge.com, rientjes@google.com, minchan@google.com, kaleshsingh@google.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arch@vger.kernel.org, linux-mm@kvack.org, linux-modules@vger.kernel.org, kernel-team@android.com, surenb@google.com X-Rspam-User: X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 17C12180003 X-Stat-Signature: cewguiztt3qubqrbipticahz1yg8a471 X-HE-Tag: 1724080515-140958 X-HE-Meta: U2FsdGVkX193nzKlUN6qcjH66C2dBSjq3GPAi4TpJX960MheWl1C28p3yPCb7zc34B/5du0SENsdvy2GYLhsKuABTAUQFlfBw84lcacbceKa6cSPfroJjOvRrO4vRVriSuBJ/5Sv8JrPQj8xtS+c52sd0n2TSuMZtZpkSD1KraGeX4VYOIz+CWoJQkB/vTr9NEfUEp2VANvnc2Odm3UOJdDwBgVo6FreFv2mB1EOt0TBTFiBfZqHjageRXu/0Ly9g6TkuxMiLWK9bBo+GpfbFa9jqbqY9E2kTuDtKjDQvRKnvggzqjhzWNCZT0b8zmg0FIu8cwnxuS3ZBm8bQWa3fM10A4jmnU/n/LkepAbiEUaO8H6mf5STucQZqcyvo5MYJlh4M4ossccCYYGf7Q8IwW699Y1hvfD9RxXejuayGTjj62XdngzCYQZH6+pGtYN/hf+RYZThf8Fo2+06l065IrdxwtYfpQp22nxqPjyJBGgcWr0k1Ngfwzklc2HbECosjT/dXQtZAjIvB0JpY8Qk8fd8n+P6tzdcfQW/VFoURxyDFHsAQ0BNXrwkLJxuvodnqZ5aOox0e39miGa6CxhuJCy2niGoNm7wgIRJGvHW1aEFvj/bdPuRyqTvdpBfxKo2xyXe/B6dzHQJ0Dn4og6GajcySs+VM655IdYk1X3IsRmYTwCdNpTZmNWLrqUJuYLrC3YBuVz15xw/W1yvvi7bTv/h4LRL2y1hoe9r6KEDKvBQzlTocsu68DaoxqXcg5CR5XMSSOUguIMIGEk4G2TJ4OZQxFiWfz35OL+kXR4XSQR/VF2N0fyiHhzG7rywdCWIlPm7ado3PQ+9vGaOAF7ZgSWPdhofQQ4XyzbrCD3xEO7YRXtdK9SK2v6MHVoJQ/WjFa6baMWnUKbe/WQa8f98VuPM+wimZDHLDWaeJX4y/Zp+6075HPJ8QJ7qogMfeoh07eWezMvGvZFonQ5l1zF uHSpDzxA sccwlmcrz91H9M1qW33zUMDTwhwdmQ1CaPqDvrVCsH9UXuIiwRvJkH5uD21hXoYVtgynJMuD87XElVCOHDD1yQNTo3EF/EGRUmiLMKvfUnSKzIYo02ENZGFPQzmbNNB3uUX/bvBDZAXPtWb3RguYUU3K3gOQ5ggj3YPRbcQiVwYpZUbMEdfx1TChVnWu8Jo0iKjTmzi0gMOGtHjZ0seASppc1Tpg4Lms0opr5sutLFoLYApFWSnlxy+jml5rrfoSJyvVcAjnGje/eM2OpZGUhOhYnd6Q1bI0i7OH2NXGM0bBhIwqIuMC5hzQm4H2xxK/Ti8iA686xwMMPfUiS8O3jX+vl1lmkxL/wzyhZlOiBbaD2niLls3HhZomy+t3SNSLc2o3jR6A31rfbVWWwUjVb6UNtpgmUMHIir3KJM7P5ad81E9ODupzF/nV7YMtACTj8LmB2KKgb/GAKICZspudN9m4wcDOR5BQHSJn19E382124Eg7kbmbBZ1nqV1yBQgiOWsHpDuQB5rzwTSulow7z9RZFRw== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: This patchset implements several improvements: 1. Gracefully handles module unloading while there are used allocations allocated from that module; 2. Provides an option to reduce memory overhead from storing page allocation references by indexing allocation tags; 3. Provides an option to store page allocation tag references in the page flags, removing dependency on page extensions and eliminating the memory overhead from storing page allocation references (~0.2% of total system memory). 4. Improves page allocation performance when CONFIG_MEM_ALLOC_PROFILING is enabled by eliminating page extension lookup. Page allocation performance overhead is reduced from 14% to 5.5%. Patch #1 copies module tags into virtually contiguous memory which serves two purposes: - Lets us deal with the situation when module is unloaded while there are still live allocations from that module. Since we are using a copy version of the tags we can safely unload the module. Space and gaps in this contiguous memory are managed using a maple tree. - Enables simple indexing of the tags in the later patches. Preallocated virtually contiguous memory size can be configured using max_module_alloc_tags kernel parameter. Patch #2 is a code cleanup to simplify later changes. Patch #3 abstracts page allocation tag reference to simplify later changes. Patch #4 lets us control page allocation tag reference sizes and introduces tag indexing. Patch #5 adds a config to store page allocation tag references inside page flags if they fit. Patchset applies to mm-unstable. Suren Baghdasaryan (5): alloc_tag: load module tags into separate continuous memory alloc_tag: eliminate alloc_tag_ref_set alloc_tag: introduce pgalloc_tag_ref to abstract page tag references alloc_tag: make page allocation tag reference size configurable alloc_tag: config to store page allocation tag refs in page flags .../admin-guide/kernel-parameters.txt | 4 + include/asm-generic/codetag.lds.h | 19 ++ include/linux/alloc_tag.h | 46 ++- include/linux/codetag.h | 38 ++- include/linux/mmzone.h | 3 + include/linux/page-flags-layout.h | 10 +- include/linux/pgalloc_tag.h | 257 ++++++++++++--- kernel/module/main.c | 67 ++-- lib/Kconfig.debug | 36 ++- lib/alloc_tag.c | 300 ++++++++++++++++-- lib/codetag.c | 105 +++++- mm/mm_init.c | 1 + mm/page_ext.c | 2 +- scripts/module.lds.S | 5 +- 14 files changed, 759 insertions(+), 134 deletions(-) base-commit: 651c8c1d735983040bec4f71d0e2e690f3c0fc2d