From patchwork Tue Aug 20 12:22:10 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Hildenbrand X-Patchwork-Id: 13770053 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 74C3AC3DA4A for ; Tue, 20 Aug 2024 12:22:25 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id C8EC06B007B; Tue, 20 Aug 2024 08:22:24 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id C17946B0082; Tue, 20 Aug 2024 08:22:24 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id AB8656B0083; Tue, 20 Aug 2024 08:22:24 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 8ACD36B007B for ; Tue, 20 Aug 2024 08:22:24 -0400 (EDT) Received: from smtpin10.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 18CD6C03E1 for ; Tue, 20 Aug 2024 12:22:24 +0000 (UTC) X-FDA: 82472536608.10.C8FE3B8 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by imf20.hostedemail.com (Postfix) with ESMTP id 49DBE1C002A for ; Tue, 20 Aug 2024 12:22:21 +0000 (UTC) Authentication-Results: imf20.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=DeC70bNf; spf=pass (imf20.hostedemail.com: domain of david@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=david@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1724156525; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:references:dkim-signature; bh=VjujtiGU7dij70tk/CDjYPdobVrS/ZRJontXr8mA6JU=; b=0O+sdbtFS4QrVnkB0vmGhbmcCdK0pXEgVuyKY/eUiNMUAtTEYYQ/mWQIzqTd74bDYkTYe4 ThpONhDM1cFIXhy8+ddBcG3k7RTA1qW3lJ2odxSjhzBaFyoR2h60oMs5kRjEmZOVnPVgy7 2neKNjHwCBj97nD1fOEL5zoiu9mvCuE= ARC-Authentication-Results: i=1; imf20.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=DeC70bNf; spf=pass (imf20.hostedemail.com: domain of david@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=david@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1724156525; a=rsa-sha256; cv=none; b=OCULog3IWQXuesPv2KnfuKghHjW7gK5WOeY17Ym3Jc5zeHHgM4B6idCNmveSH44OaPYVAs 9fdaxoPzjUv31r2GpFuQ8Nux76wwoyI7EKNfxXaMKmFiN7m3l0zynwJlHPDdYXT9EE/5hD LMxKRy5nib4E3Z0VetEfPLXHGf5xRCM= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1724156540; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding; bh=VjujtiGU7dij70tk/CDjYPdobVrS/ZRJontXr8mA6JU=; b=DeC70bNfvBSKW9pTSgZyzrifvbSflMEr7GqTPQq+NG4MFTucZe2DaBTi/K7BXjaA7E5uqJ 2Yv0YtEZV4wBtlw9K2Zo96/Q5dNdAna49khufKF4+Ds7k29Zn4Cacb8Y9KfVTgx+GjCqJF vZwnvGCtFr36sEPpxPbhKOmmFGQ+9Ec= Received: from mx-prod-mc-04.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-617-2RViYE6yP1qSqPNUHQwTbA-1; Tue, 20 Aug 2024 08:22:17 -0400 X-MC-Unique: 2RViYE6yP1qSqPNUHQwTbA-1 Received: from mx-prod-int-05.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-05.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.17]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-04.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id A703E19560BF; Tue, 20 Aug 2024 12:22:15 +0000 (UTC) Received: from t14s.redhat.com (unknown [10.22.16.200]) by mx-prod-int-05.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 8D9CD1956053; Tue, 20 Aug 2024 12:22:12 +0000 (UTC) From: David Hildenbrand To: linux-kernel@vger.kernel.org Cc: linux-mm@kvack.org, David Hildenbrand , kernel test robot , Andrew Morton , Peter Xu Subject: [PATCH v1] mm: always inline _compound_head() with CONFIG_HUGETLB_PAGE_OPTIMIZE_VMEMMAP=y Date: Tue, 20 Aug 2024 14:22:10 +0200 Message-ID: <20240820122210.660140-1-david@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.0 on 10.30.177.17 X-Rspam-User: X-Stat-Signature: hb6ehpqwq887jci6n85skpj7sd7iuk66 X-Rspamd-Queue-Id: 49DBE1C002A X-Rspamd-Server: rspam11 X-HE-Tag: 1724156541-632732 X-HE-Meta: U2FsdGVkX19DHJmTUTQhMXr1rrAJXTXnkD9P9/Zq1rF8JCOtLc4nly3xP4tQoVm8z4fh63zmF/scD9e+Z+mmwib9qPDE9X6gYXHc2IoEH6G/zlnYnCwdVQjbVVlmFwM+yb+Gm6V1JgbNfuD/mRBNmDgCg37p283uA/55b4W8m++S733zcFwhTXE1DoDcPsNzAXlLbps0gBLhRQtvbTyEZ10OH5PP3y/pLCk+6h1GFKBDxWCv6hJXi/tEddkFig8WImpnCVq/q+7dFDJQ1AtWsrrrtiRSNKVMj2iyey2Q/No/43qRMhdSM+KFBYz5P5T+HDmeHeXiwoAEHNacXSiiMWoJVtB+6NxGG/Acp/VKRQosI1aO5VXKG3CsdJofS0mfLzOmCsQzCINlH5KZ+1qQPAkLril3evAj2TAU1vp1zvjTwRQq7AabufO+MLaqwtRJ+L9VLnFwZanKrwWcubmAzVZolWOao9RkLghaaA95Fbxmv59oVeSgulnhElUfENZ462h54EgeZqyZXcy2ev/wskdPLQ1peMWUObICy1W9OEW767HpXI3k3Dt/Rad/GzWiuvAh1AK8yXPBNFO1Vhd30n/UkvW8TMZNiEiUUjo4E6y0ucBZTcAONR3Mn/u9KmrN5HJdbpOQzp0tfnrGZEWVZf9MoRcft/A9MbWNn6U4iIp6kduWAIAvXiRs8h36579rZSA2Lm7oZthKiBydDIzzx0McFBXTOPbLzCfVXw+gQwk+T8xwTfPfX3nJKSJZUDA0DwudFVUpk9KqfQaidLKU4E+Fc5bn//SUb8gz+6kgJ4kgli3h8EliUVBbiijC1tDk+Ns4EI54lJ4BRfyYE0x3P2KXulCh2IhCkJWEbksytQfT+AC8ayAG6heryaGAtABrive2DCTpPLQiuQX9HoxUHTyADrZBqHO1KVMyKpb+OhDLKvLsjyXUIjvG1lkSdifOEfZnpFZ3ueXhXptuuQY vnCUMtKU FkwNL0R9FVTOGk/8UwxlFyoPEPIHJvXfuyqaDqvVNlwetUlAR7mYFB1QHtkf9kYTNxZRa7Ks8/+2UfevhhRLu2qFzSODsVHk3vEdnNv4b+tDl5mjYL7kJg0mNYKj4oaKvnmh9sdSPgfj5ACt3+G6cDzTeNBZaSZ1Iqw5n30vnlBipClxHcE2ts12CFan+fNvXCRPd8b/b0AnIQte24aXaCEdmVso6AEnG2TdLgiv/QBaZRi28Ef92F15ujuA9yy1qdkeYvyI6wvPcYoH1I9YtYX4AOfxk9tfZnoPNuIU/xd1jlrPGs4Dw4VbyeCl0vaf/IsSGwkU1/qwRTNV1scQ9sOsp+2LT7Z/hSGOh7ny2ZXq8maKZwkvewlOPSUZ2NdEW3U9Oz4m9R9FOwmwHkvF1S98p4fBuSD2Sl5OzmJXxkhzgA2GAUb8LXk+EKa7gjQosk5JF96pM/m65x4a50QGO/VsyMwdxNwA6f/MEMdcdFM7EM20= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: We already force-inline page_fixed_fake_head(), page_is_fake_head() and PageTail(), however the compiler might decide that _compound_head() is not worthy to be inlined, because of page_fixed_fake_head(). The result is that, for example, PageAnonExclusive() now might involve a function call when checking PageHuge(), which performs a page_folio()->_compound_head() call. This can lead to a slight regression of the stress-ng.clone benchmark. This is not super-urgent to fix, but always inlining _compound_head() seems like the obvious thing to do for this primitive, similar to the other ones. This change restores the slight regression and a compilation with CONFIG_HUGETLB_PAGE_OPTIMIZE_VMEMMAP=y shows no relevant bloat [2]: add/remove: 15/14 grow/shrink: 79/87 up/down: 12836/-13917 (-1081) ... Total: Before=32786363, After=32785282, chg -0.00% [1] https://lkml.kernel.org/r/817150f2-abf7-430f-9973-540bd6cdd26f@intel.com [2] https://lore.kernel.org/all/116e117c-2821-401d-8e62-b85cdec37f4a@redhat.com/ Fixes: c0bff412e67b ("mm: allow anon exclusive check over hugetlb tail pages") Reported-by: kernel test robot Closes: https://lore.kernel.org/oe-lkp/202407301049.5051dc19-oliver.sang@intel.com Cc: Andrew Morton Cc: Peter Xu Signed-off-by: David Hildenbrand Tested-by: Yin Fengwei --- include/linux/page-flags.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/include/linux/page-flags.h b/include/linux/page-flags.h index b753d158762fc..af58b2ad854c3 100644 --- a/include/linux/page-flags.h +++ b/include/linux/page-flags.h @@ -233,7 +233,7 @@ static __always_inline int page_is_fake_head(const struct page *page) return page_fixed_fake_head(page) != page; } -static inline unsigned long _compound_head(const struct page *page) +static __always_inline unsigned long _compound_head(const struct page *page) { unsigned long head = READ_ONCE(page->compound_head);