From patchwork Mon Aug 19 07:02:04 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yongqiang Liu X-Patchwork-Id: 13767976 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9E72FC52D7C for ; Mon, 19 Aug 2024 07:02:14 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 2FD486B0083; Mon, 19 Aug 2024 03:02:14 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 284CC6B0085; Mon, 19 Aug 2024 03:02:14 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 174DD6B0088; Mon, 19 Aug 2024 03:02:14 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id EF9996B0083 for ; Mon, 19 Aug 2024 03:02:13 -0400 (EDT) Received: from smtpin13.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 7AC401A1040 for ; Mon, 19 Aug 2024 07:02:13 +0000 (UTC) X-FDA: 82468100946.13.3C7B468 Received: from szxga02-in.huawei.com (szxga02-in.huawei.com [45.249.212.188]) by imf15.hostedemail.com (Postfix) with ESMTP id 16E59A002B for ; Mon, 19 Aug 2024 07:02:10 +0000 (UTC) Authentication-Results: imf15.hostedemail.com; dkim=none; dmarc=pass (policy=quarantine) header.from=huawei.com; spf=pass (imf15.hostedemail.com: domain of liuyongqiang13@huawei.com designates 45.249.212.188 as permitted sender) smtp.mailfrom=liuyongqiang13@huawei.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1724050916; a=rsa-sha256; cv=none; b=sxhL3iaCZVIalW9W2umZqf0bmyo2EqTYDQ6O6V6RW87O+rlWwJpHI2x9JwLRQZYN0HLshE Ss7CWsxsF32pWXc78ql5U5zorhrym1SdSlQwX7SIQ8jMX8w+sQ2jGw+woyi3GKfcy1J1RP V1ltrGprV0V7b2Y3AF7inQqvW9iG6+Y= ARC-Authentication-Results: i=1; imf15.hostedemail.com; dkim=none; dmarc=pass (policy=quarantine) header.from=huawei.com; spf=pass (imf15.hostedemail.com: domain of liuyongqiang13@huawei.com designates 45.249.212.188 as permitted sender) smtp.mailfrom=liuyongqiang13@huawei.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1724050916; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding:in-reply-to: references; bh=EE7QzaAnWtFb7e4ewzv/C+t0axz+6cBJ+7C0xWWupPo=; b=gOut5Cy5xvBfZr9A/n1481fg1a2P2ekP6Za8CtJ8Itv1F1cJk+kaXcM3h5Kj1vsUX1/XaX H2OrTzNQ2NbH9ErDv2q0TJiw39Xnsen5oEHrV/0cD/negTViuA4YB+OR5o/9CmwkKytbgq gK+YIP7NaxRzOA+344i6wY8l2FX02yU= Received: from mail.maildlp.com (unknown [172.19.163.174]) by szxga02-in.huawei.com (SkyGuard) with ESMTP id 4WnNlB05f3zfbYH; Mon, 19 Aug 2024 15:00:06 +0800 (CST) Received: from dggpeml500005.china.huawei.com (unknown [7.185.36.59]) by mail.maildlp.com (Postfix) with ESMTPS id 8ADFC1400CD; Mon, 19 Aug 2024 15:02:05 +0800 (CST) Received: from huawei.com (10.175.112.125) by dggpeml500005.china.huawei.com (7.185.36.59) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.2507.39; Mon, 19 Aug 2024 15:02:04 +0800 From: Yongqiang Liu To: CC: , , , , , , , , , , <42.hyeyoo@gmail.com> Subject: [PATCH] mm, slub: prefetch freelist in ___slab_alloc() Date: Mon, 19 Aug 2024 15:02:04 +0800 Message-ID: <20240819070204.753179-1-liuyongqiang13@huawei.com> X-Mailer: git-send-email 2.25.1 MIME-Version: 1.0 X-Originating-IP: [10.175.112.125] X-ClientProxiedBy: dggems706-chm.china.huawei.com (10.3.19.183) To dggpeml500005.china.huawei.com (7.185.36.59) X-Rspam-User: X-Rspamd-Queue-Id: 16E59A002B X-Rspamd-Server: rspam01 X-Stat-Signature: rxjo9xjg54j3psjihmzhg37ib3re8zyx X-HE-Tag: 1724050930-997617 X-HE-Meta: U2FsdGVkX1/hT0TVtvTloegS+nH1INqyVudQAcARXwD+CAPW/XaCxKtmUBU6jnnB1C4FdoO8SqV5W9CXkbFnWlrhnuxfhu70PBixZidUAShz23r0WAMvyjuixlbVEskky18hyfeiOvbQnfKbWNrG4sk8x/anAcOJ+LMdZo5pab8wxFeglkWnAQ9Eg/JxW4wEpfKQj833PwvzPou06qgM3kGwJw2EzqNO4Kp50r6qPnOF5HlLINylXR9I6abL0hNf/t9N1OWMgsCnSGli3zUQH+lSjH9NXxT/TJExO/EGjHrxp9MOt5r8kCMh6bkt0yy8KCiE3DJdJ23gg2LsGg2cykoN8Ore+M8kqSfYVn55Se+Zi92DsQhV7D5kGA/L5XpG+CvCr5lHfXFirCclDBzvYkGXL5cnK0+Uex0souqsGw0eJzVsu9rKAqpkfbIZYP1GwATO0JoRcnHrCVs0wiJN8wUoDEJhKJEmEWbngnjwBqIBS1Ocr83cTf/S+sd8BaFsxbCggVzYqYv/4McTMUI4FuEdDjEUCcUKID3zd85C47jMVNO7Q65vlJyJqDVmnZ6GM5NV4pW89zVyMsdE1iPpHg9lR7el2QQBVpYCWyFmQoc3076HA/ej5A0XCHf5cfvwp5+eDsLHubRCICPUdtGCOLaZjc8VmiAL6d4FuuVxVw/KEbOSAAlVrhg9SkVyq/HnxGV0TsPMTysojYDcCyEq6oqMI0xvC5S0w+NRQ7BwkAdGeHzN7y60wQ2TMvqHPdE9uvEtAQOo+gz3yTlDjrMLz4PxjFotdmgsjlAUmL9yPLQCU6PcjtRezl7Pubv/woLwM0g1UrFSTMS1euLlHYgCn8eYJzWB2Xl9L5hOj+H9uYtrG6BX1Tsnn0pyDlZJaY6mQuhje96JdcRcEW58ptzGAVHZNeyURqpZS7XlDKI1rO8R+VwT30O3XGSEKk/qLE0jZTBILVbORzltPOHEJmi PfG2rP3O gy7QTMrhshRXSi76XhdywmU5UJsmYre+CIZ0uQJuTvjPVwEl7bful2iJ2JlUrXs6jP8Yc6HP2oKiY41ZGgtDKDJ0sMBChodJ+NGEqiCy/moymQ77KdCID6PIx2UXF7l/ARXOgOqJcWg3zr3PDQjVosU1gXjQIR611b2p7KF94Nu7dgQA/pFzESlfmxluVyknBtMBIxILitVHfSI5h82Ctcib2PkgL0zI28mTCKktxAPS/P5djFWdgg59qE6u4EUB3zityf7dLYxWS6bg= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: commit 0ad9500e16fe ("slub: prefetch next freelist pointer in slab_alloc()") introduced prefetch_freepointer() for fastpath allocation. Use it at the freelist firt load could have a bit improvement in some workloads. Here is hackbench results at arm64 machine(about 3.8%): Before: average time cost of 'hackbench -g 100 -l 1000': 17.068 Afther: average time cost of 'hackbench -g 100 -l 1000': 16.416 There is also having about 5% improvement at x86_64 machine for hackbench. Signed-off-by: Yongqiang Liu --- mm/slub.c | 1 + 1 file changed, 1 insertion(+) diff --git a/mm/slub.c b/mm/slub.c index c9d8a2497fd6..f9daaff10c6a 100644 --- a/mm/slub.c +++ b/mm/slub.c @@ -3630,6 +3630,7 @@ static void *___slab_alloc(struct kmem_cache *s, gfp_t gfpflags, int node, VM_BUG_ON(!c->slab->frozen); c->freelist = get_freepointer(s, freelist); c->tid = next_tid(c->tid); + prefetch_freepointer(s, c->freelist); local_unlock_irqrestore(&s->cpu_slab->lock, flags); return freelist;