From patchwork Fri Mar 8 15:15:36 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Donet Tom X-Patchwork-Id: 13586937 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 61F72C5475B for ; Fri, 8 Mar 2024 15:17:12 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id DD17E6B039F; Fri, 8 Mar 2024 10:17:11 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id D81B36B03A1; Fri, 8 Mar 2024 10:17:11 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C22616B03A2; Fri, 8 Mar 2024 10:17:11 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id AFEC46B039F for ; Fri, 8 Mar 2024 10:17:11 -0500 (EST) Received: from smtpin17.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 8656D1A0766 for ; Fri, 8 Mar 2024 15:17:11 +0000 (UTC) X-FDA: 81874225062.17.746C63D Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) by imf14.hostedemail.com (Postfix) with ESMTP id 54E81100012 for ; Fri, 8 Mar 2024 15:17:08 +0000 (UTC) Authentication-Results: imf14.hostedemail.com; dkim=pass header.d=ibm.com header.s=pp1 header.b="sM/8bn7l"; dmarc=pass (policy=none) header.from=ibm.com; spf=pass (imf14.hostedemail.com: domain of donettom@linux.ibm.com designates 148.163.156.1 as permitted sender) smtp.mailfrom=donettom@linux.ibm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1709911028; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:references:dkim-signature; bh=MvrDX0dkLUQjrwJI3GByS9ZHPaZz9JeJvCJGU/DolGg=; b=YcfyfcrG6c7TGVOzekHRCoORYEKcw+3jUK07MvjUu5AfsKrD75M8kLNOVZGsVJ/KYQK/+C rjK0wzo1wAvKlTNfMTk+YphIF433uwOa3GvAJosSWCNBbwLGXBwkgQLzr97q0Uvtv8iEg3 BhUvi6Prja05reGjr1JMbcPwMOq4XdU= ARC-Authentication-Results: i=1; imf14.hostedemail.com; dkim=pass header.d=ibm.com header.s=pp1 header.b="sM/8bn7l"; dmarc=pass (policy=none) header.from=ibm.com; spf=pass (imf14.hostedemail.com: domain of donettom@linux.ibm.com designates 148.163.156.1 as permitted sender) smtp.mailfrom=donettom@linux.ibm.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1709911028; a=rsa-sha256; cv=none; b=yNMsP57JHPRiQQHfRbCS6sQs+2ibAKJmtA5KaOuhgHBll0IXMJdKJw4Lh10ra3GvI55sxR cBO0GJERCJIpPAtefZwMXws/hJu0wwKUPryzjCSaj3lklYlFlfOwMLUXYGd4D+SjAGemVN g7SDbANrwhT8yMQVKoFsjKpDA2mcRR0= Received: from pps.filterd (m0353726.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 428F7FGC004940; Fri, 8 Mar 2024 15:16:28 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : cc : subject : date : message-id : content-transfer-encoding : mime-version; s=pp1; bh=MvrDX0dkLUQjrwJI3GByS9ZHPaZz9JeJvCJGU/DolGg=; b=sM/8bn7lpoSoG0xPk/0QiQjjGjQYvSQCenytI5Ne821ELZ+E9P5krqvGhPwigqHCPado UYh7aNw7o91bhWNu376dpV5flNjsAO9mey20stSK+jI76K+suRRMqX4I7rPHkUzUEUb2 ThCAVOS1kYY71Rb3r/MBRDdPDa6XUACnUmcHSg7qhCJcpOEEPcy6gXr5mzmDpAks97L+ /nJydbjf4m85VlwdiuwKIQEajmpQFqUe0ZKNYtJWdQOFLcNKcl81K779/nkGqkF9ipOF KUjPbQPvRCUXp++nioW0yH1cOLZb6rpg/MWLhIwbgB4NGx7OB5U+OVy/4/iCvuaj0we0 5g== Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 3wr51jr4rj-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 08 Mar 2024 15:16:27 +0000 Received: from m0353726.ppops.net (m0353726.ppops.net [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 428F8e2E010141; Fri, 8 Mar 2024 15:16:27 GMT Received: from ppma22.wdc07v.mail.ibm.com (5c.69.3da9.ip4.static.sl-reverse.com [169.61.105.92]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 3wr51jr4r2-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 08 Mar 2024 15:16:27 +0000 Received: from pps.filterd (ppma22.wdc07v.mail.ibm.com [127.0.0.1]) by ppma22.wdc07v.mail.ibm.com (8.17.1.19/8.17.1.19) with ESMTP id 428EFrnH025387; Fri, 8 Mar 2024 15:16:25 GMT Received: from smtprelay02.fra02v.mail.ibm.com ([9.218.2.226]) by ppma22.wdc07v.mail.ibm.com (PPS) with ESMTPS id 3wmeu059qm-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 08 Mar 2024 15:16:25 +0000 Received: from smtpav02.fra02v.mail.ibm.com (smtpav02.fra02v.mail.ibm.com [10.20.54.101]) by smtprelay02.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 428FGLpH34734492 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 8 Mar 2024 15:16:23 GMT Received: from smtpav02.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 8CEAE20043; Fri, 8 Mar 2024 15:16:21 +0000 (GMT) Received: from smtpav02.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 95A182004E; Fri, 8 Mar 2024 15:16:18 +0000 (GMT) Received: from ltczz402-lp1.aus.stglabs.ibm.com (unknown [9.53.171.174]) by smtpav02.fra02v.mail.ibm.com (Postfix) with ESMTP; Fri, 8 Mar 2024 15:16:18 +0000 (GMT) From: Donet Tom To: Andrew Morton , linux-mm@kvack.org, linux-kernel@vger.kernel.org Cc: Aneesh Kumar , Huang Ying , Michal Hocko , Dave Hansen , Mel Gorman , Feng Tang , Andrea Arcangeli , Peter Zijlstra , Ingo Molnar , Rik van Riel , Johannes Weiner , Matthew Wilcox , Vlastimil Babka , Dan Williams , Hugh Dickins , Kefeng Wang , Suren Baghdasaryan , Donet Tom Subject: [PATCH v2 0/2] Allow migrate on protnone reference with MPOL_PREFERRED_MANY policy Date: Fri, 8 Mar 2024 09:15:36 -0600 Message-Id: X-Mailer: git-send-email 2.39.3 X-TM-AS-GCONF: 00 X-Proofpoint-ORIG-GUID: dz6WQicQ7yDFSFge9YVM14OTooKmwlro X-Proofpoint-GUID: UNkepK4yDNo7wby3-J_PaFMZ85cfzzXn X-Proofpoint-UnRewURL: 0 URL was un-rewritten MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.1011,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2024-03-08_08,2024-03-06_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 clxscore=1011 bulkscore=0 spamscore=0 priorityscore=1501 malwarescore=0 mlxlogscore=999 mlxscore=0 lowpriorityscore=0 phishscore=0 suspectscore=0 impostorscore=0 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2311290000 definitions=main-2403080123 X-Rspam-User: X-Stat-Signature: uroxbnsf8qy1jsi8wgx97g43pf4wewhn X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: 54E81100012 X-HE-Tag: 1709911028-449133 X-HE-Meta: U2FsdGVkX188v5esPXM+VjJUHmmIeEiFFFWoHUG0LSIm/NckXKOZIIGTmA7AF1B/QshDec4I9Jz6NMc4e5BImw+NOeHJ2K9o/nBRrexoE7+P+pO9EAoSeMRYQt3QBH5BJRwEN+1hSMFm8vUoTEbQFSLX7cNjFXwM6KJE/nmS4hldC0Q+H8f9UhM37XSGrvvJ3u7I8YChhE3UDL4/tVV4BsR4mhDbCURQ4O42DCa+uIWD1F2RzTOSj596UWuiZbEPxoEnKj8nkvwMPmFD9egCUb6iytNYySv1/ahbav+SQV7kueJJATbcw5nPrrI4BZzaD8rfTMQAJC/8TfelFDXh7bVMYumn+AQ2WOt9No7vZFOUzwe/J4TC1rlCpb8JVH5Vulz2q7E2geaVZlaYI49Lz+epEw3fWEpjwD5uKTp4LLSd581pbmTG993mj6nydliCtJSjjEAIBmcwIy5FtXnwcCK11iYZwfNFqN+ZVVAEtk3+RJtjaOakHjQDxivgiq5E9pbktGPGomRq/JJdBMirpSEsIdZY4ONkVW0A/cgGfp5IDGdS3OvGeqPPMoTTS+CmHhq3xyv03Ng00TNT0jHI1plPwmGwcdSRtEpX3HhRq6bRDyZSMONYv48dHtFiwan7WguH7vXou4CaoUjywpIojYFyfXVWHiCAer4SRP6wOdESvRcw2bO0jX3VE0tIGUOlMUiVg6vDJTJZzcV1EGvWMBFbiUh51kCc5CjxYviYYi9wU+O0Xt98a/v0uhCKqdlC0rBFt4IdZmMEmtFAJjlZ3hblBfjRtmy3gcJCP8YtQ0jtFeRVAcFOOp2mCkxq2sBWkbFFR7dTAcMXjE6boIYJAoxFhxUjw/RgGScg/8m0kg1ml+oYjPQ2roC0Uxg5WY0kiXbD1tE20VQfED03O37k3kT64d9a+gnAEGHri2shZAI= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: This patchset is to optimize the cross-socket memory access with MPOL_PREFERRED_MANY policy. To test this patch we ran the following test on a 3 node system. Node 0 - 2GB - Tier 1 Node 1 - 11GB - Tier 1 Node 6 - 10GB - Tier 2 Below changes are made to memcached to set the memory policy, It select Node0 and Node1 as preferred nodes. #include #include unsigned long nodemask; int ret; nodemask = 0x03; ret = set_mempolicy(MPOL_PREFERRED_MANY | MPOL_F_NUMA_BALANCING, &nodemask, 10); /* If MPOL_F_NUMA_BALANCING isn't supported, * fall back to MPOL_PREFERRED_MANY */ if (ret < 0 && errno == EINVAL){ printf("set mem policy normal\n"); ret = set_mempolicy(MPOL_PREFERRED_MANY, &nodemask, 10); } if (ret < 0) { perror("Failed to call set_mempolicy"); exit(-1); } Test Procedure: =============== 1. Make sure memory tiring and demotion are enabled. 2. Start memcached. # ./memcached -b 100000 -m 204800 -u root -c 1000000 -t 7 -d -s "/tmp/memcached.sock" 3. Run memtier_benchmark to store 3200000 keys. #./memtier_benchmark -S "/tmp/memcached.sock" --protocol=memcache_binary --threads=1 --pipeline=1 --ratio=1:0 --key-pattern=S:S --key-minimum=1 --key-maximum=3200000 -n allkeys -c 1 -R -x 1 -d 1024 4. Start a memory eater on node 0 and 1. This will demote all memcached pages to node 6. 5. Make sure all the memcached pages got demoted to lower tier by reading /proc//numa_maps. # cat /proc/2771/numa_maps --- default anon=1009 dirty=1009 active=0 N6=1009 kernelpagesize_kB=64 default anon=1009 dirty=1009 active=0 N6=1009 kernelpagesize_kB=64 --- 6. Kill memory eater. 7. Read the pgpromote_success counter. 8. Start reading the keys by running memtier_benchmark. #./memtier_benchmark -S "/tmp/memcached.sock" --protocol=memcache_binary --pipeline=1 --distinct-client-seed --ratio=0:3 --key-pattern=R:R --key-minimum=1 --key-maximum=3200000 -n allkeys --threads=64 -c 1 -R -x 6 9. Read the pgpromote_success counter. Test Results: ============= Without Patch ------------------ 1. pgpromote_success before test Node 0: pgpromote_success 11 Node 1: pgpromote_success 140974 pgpromote_success after test Node 0: pgpromote_success 11 Node 1: pgpromote_success 140974 2. Memtier-benchmark result. AGGREGATED AVERAGE RESULTS (6 runs) ================================================================== Type Ops/sec Hits/sec Misses/sec Avg. Latency p50 Latency ------------------------------------------------------------------ Sets 0.00 --- --- --- --- Gets 305792.03 305791.93 0.10 0.18949 0.16700 Waits 0.00 --- --- --- --- Totals 305792.03 305791.93 0.10 0.18949 0.16700 ====================================== p99 Latency p99.9 Latency KB/sec ------------------------------------- --- --- 0.00 0.44700 1.71100 11542.69 --- --- --- 0.44700 1.71100 11542.69 With Patch --------------- 1. pgpromote_success before test Node 0: pgpromote_success 5 Node 1: pgpromote_success 89386 pgpromote_success after test Node 0: pgpromote_success 57895 Node 1: pgpromote_success 141463 2. Memtier-benchmark result. AGGREGATED AVERAGE RESULTS (6 runs) ==================================================================== Type Ops/sec Hits/sec Misses/sec Avg. Latency p50 Latency -------------------------------------------------------------------- Sets 0.00 --- --- --- --- Gets 521942.24 521942.07 0.17 0.11459 0.10300 Waits 0.00 --- --- --- --- Totals 521942.24 521942.07 0.17 0.11459 0.10300 ======================================= p99 Latency p99.9 Latency KB/sec --------------------------------------- --- --- 0.00 0.23100 0.31900 19701.68 --- --- --- 0.23100 0.31900 19701.68 Test Result Analysis: ===================== 1. With patch we could observe pages are getting promoted. 2. Memtier-benchmark results shows that, with the patch, performance has increased more than 50%. Ops/sec without fix - 305792.03 Ops/sec with fix - 521942.24 Changes: v2: - Rebased on latest upstream (v6.8-rc7) - Used 'numa_node_id()' to get the current execution node ID, Added 'lockdep_assert_held' to make sure that the 'mpol_misplaced()' is called with ptl held. - The migration condition has been updated; now, migration will only occur if the execution node is present in the policy nodemask. -v1: https://lore.kernel.org/linux-mm/9c3f7b743477560d1c5b12b8c111a584a2cc92ee.1708097962.git.donettom@linux.ibm.com/#t Donet Tom (2): mm/mempolicy: Use numa_node_id() instead of cpu_to_node() mm/numa_balancing:Allow migrate on protnone reference with MPOL_PREFERRED_MANY policy include/linux/mempolicy.h | 5 +++-- mm/huge_memory.c | 2 +- mm/internal.h | 2 +- mm/memory.c | 8 +++++--- mm/mempolicy.c | 34 ++++++++++++++++++++++++++-------- 5 files changed, 36 insertions(+), 15 deletions(-)