From patchwork Fri Mar 7 19:30:47 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mathieu Desnoyers X-Patchwork-Id: 14007060 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9B12BC28B24 for ; Fri, 7 Mar 2025 19:30:55 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 2A0586B0083; Fri, 7 Mar 2025 14:30:54 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 251206B0085; Fri, 7 Mar 2025 14:30:54 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 119016B0088; Fri, 7 Mar 2025 14:30:54 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id E73166B0083 for ; Fri, 7 Mar 2025 14:30:53 -0500 (EST) Received: from smtpin21.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 76466160FC0 for ; Fri, 7 Mar 2025 19:30:54 +0000 (UTC) X-FDA: 83195747628.21.5B90C63 Received: from smtpout.efficios.com (smtpout.efficios.com [158.69.130.18]) by imf30.hostedemail.com (Postfix) with ESMTP id 86BD280010 for ; Fri, 7 Mar 2025 19:30:52 +0000 (UTC) Authentication-Results: imf30.hostedemail.com; dkim=pass header.d=efficios.com header.s=smtpout1 header.b=SFFHlPiQ; spf=pass (imf30.hostedemail.com: domain of mathieu.desnoyers@efficios.com designates 158.69.130.18 as permitted sender) smtp.mailfrom=mathieu.desnoyers@efficios.com; dmarc=pass (policy=none) header.from=efficios.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1741375852; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:references:dkim-signature; bh=Xa1RkV03nO8Hb1jv+twQRWoBs7gZ7YOoJHRb8Ry+S04=; b=aMDnplt7A4IZk2+hCN+G1E4ymxleY4/Ab6uGqJOYvasWHHZ2Mg3gczjGo8C8XxgDnWVBMt 9X+W2JGgAExHcTNCzwoZZxwFS/cWFS0Wcpo+IJkV3ZNuNxV095Vs2v4lBA0SpKynD4KA8z ceegkIN12m2J5yO9T79k44Ltmv3aOoo= ARC-Authentication-Results: i=1; imf30.hostedemail.com; dkim=pass header.d=efficios.com header.s=smtpout1 header.b=SFFHlPiQ; spf=pass (imf30.hostedemail.com: domain of mathieu.desnoyers@efficios.com designates 158.69.130.18 as permitted sender) smtp.mailfrom=mathieu.desnoyers@efficios.com; dmarc=pass (policy=none) header.from=efficios.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1741375852; a=rsa-sha256; cv=none; b=Kdk0O0iQGjE+hx0XIETgH0M/onEbCA5015VIk6U367HFVxe6MNQowitxsY7guxPPhF/PuR gYyMoS20EbUQd5nZMqOovimLj8tMAPyob6WjfbfFYLCm/ez8W9HcGWprlUe7hmmxf5yo9/ U8hIiiTldwufMTlSLf3DNCEbsgYTYzk= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=efficios.com; s=smtpout1; t=1741375851; bh=I7qUEBsf9z9DvFVeTUwEL2QF/mwYSpxIB5OX23QFtcU=; h=From:To:Cc:Subject:Date:From; b=SFFHlPiQ1dVRlkA83oEwuD3d/a54xPfyT7af1B56LbjBkK/qQx8EPBooVir5Xs/PP YxGzct4ODrmkgHTHkZwnmo0ATuRRc/KHU8rW3KKL6C4Imra6Y4zOIDMFvysDE/jOfb a0oimfFSYWEs5uXP9CkxbuL5QXmK6LtMlYVpaoabLXi1NzV7LLLRm6g/mYyAFiivtO 0prauRBELPV8Pqk0Yohajy7xlCeMI2qQHPAEYV4TlMwTZkIaytCPrRj32k7Y4zLiAC CUxskCVt+Bm9uC4wlzCauOiIw2ew0yIQRtXv00qPOUJAcGlSjwslPr/VYrOprNy4r8 zbwl56EPgfkrA== Received: from localhost.localdomain (96-127-217-162.qc.cable.ebox.net [96.127.217.162]) by smtpout.efficios.com (Postfix) with ESMTPSA id 4Z8bx71SLyz1gPP; Fri, 7 Mar 2025 14:30:51 -0500 (EST) From: Mathieu Desnoyers To: Andrew Morton Cc: linux-kernel@vger.kernel.org, Mathieu Desnoyers , Lorenzo Stoakes , Matthew Wilcox , Alan Stern , Andrea Parri , Will Deacon , Peter Zijlstra , Boqun Feng , Nicholas Piggin , David Howells , Jade Alglave , Luc Maranget , "Paul E. McKenney" , linux-mm@kvack.org Subject: [RFC PATCH] mm: Add missing release barrier on PGDAT_RECLAIM_LOCKED unlock Date: Fri, 7 Mar 2025 14:30:47 -0500 Message-Id: <20250307193047.66079-1-mathieu.desnoyers@efficios.com> X-Mailer: git-send-email 2.25.1 MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Queue-Id: 86BD280010 X-Rspamd-Server: rspam03 X-Stat-Signature: rn6qkrrmakjqqex3k93ansubk3e16tx1 X-HE-Tag: 1741375852-715209 X-HE-Meta: U2FsdGVkX1+7iUqriFzDs/zt+ngRfCvsnsZXF12EfABsG+je5fQrFiCT5VEwtVRTNnms2aJmv2Ud0huv83mR5fhZen5NdQvpGqG6ltVSJXEJ+xQbi+cvrOsSE1Wig/9d/wecDQLwuVXNz6vrDWTJ6GdwRze0xfbZtzZuGATEagRa32nkNguXxPOQIOTg2GIuiK8aEP/EOxB+QlT/8zXy8vAVJj0HkCOV1HekZ2tUEFjlZ/1Gpjew7LgQcoRmRYDuseGTR+k5AfOIte1ZABGqdfDzgqLqEXfqJDJgJ+yW6GhMf0zC2gcMAmkt/5HxVoAKACLTAKqUc4b9X9LDrbJqpGb3udhztDA9MBjFNRmtwuSxwGd7KzqC/+N+bfK5DTiA/qrl/Gs+pAlaAwWGMsEPr6HZxEM7hHLFOilvV6umgYTpS430ESqLiUjMAKdCyXi8dk6/kR0nPiEIs0ROoCUgjdiHqAnC4Mzd7DmuXODvh+OnpfV/2jz0uR8Xg/udx5cAzzwXet1ZYaZZz23o/gRK+jllf1USzc8QeYq3ZV+T8Y92C7SoBOa2LfUpBN6RGSzAj4W3OVkEc8Pk6FXGRa7k4ltXJyRySnWIhAR0zt7u9mNQCvtYKGvQOYs1XqSbEfSJioDyhPLJawe0eHxvpFrqbSyIxLP5xL7lKOWtSBPTIx2hDhMfo9rUhkRu8Gqk6XhLG2saifOKdhjo08iJ3OQh12f/6kEr43rGYLqDKaJYebpluadkJ9fosdEcCb5Ve9qIB6hGZJxQKQOqNOL2GQjaqoYnlJ7KjmChW76M4JjfVBamEuut2XMDbHYNVQYyw36DjvcCpbIGq0hRtlOJnB0oSqECxzTAy7rgAwtkGqkcIBmn/cddk/LEuu6OeJ4qZ6StqpnAnLqg8sRXZaAZmDwLnXBz2R5eqdvgt+H22DAU8l2CcozmQBXm7Ktulk+AjOdjEvpaHcIICd+niPu+szn w0b9KB2/ 5jG/mSdq08khhhd/vAbk4uAkLoUmqw37HeuJ+iNDxY7FZSbCFDUh+my+fOsWRody85BLmtPTJBMURtJd8r82UJJBfAlZq2wYqLbTBkLSxTUDqJAYKdi91Jn+45N1a7ztQt9oKPTb9rP1B05c67osJqysjXVDBYe0lW+OkiNQNlXH6wK3AUUvKdzq51iiRrZL48r8FpP/DGMqwZcbVb+/CXBkIHjDbPZEFphkAjSeoEhlcnLECNE1516z6LkIyN84MKO7guP+6PV82S+5K74vwaAQLmRM84bLBZaiyZ8wmktwVIXTnDVdXFNU148K34avvEMQl39JnaQy+ysUTqieWab0fNu/C4i/m64Pv0UZwfJrKGBNtKnHQnUvDbwdEluZg5iB4Sr4JRXWgkQ0+N9PADe+EimMdxrO3RWerxbr7B7g2F9amQcreRF9YAXPmJxuNPNBUp6TcImwiujj4gwhlOK7tZqw+oSCoCHLIGtpp3VieYZMgO/wHUmm9F/KygtoyfX2nHljfoP/AlB71WCWH9iLwDa+JuQ6jB0Sp5/xrOatr+7AGLrGpTSLMHrUZP4jarhBEcPMVegRnV6luFm2TOCeMK8iOWFlJZ7dMwfimFz8EHKS3YL57LJ3eVBLQPI074g1blQ3fuRv+4OEFXPlP3ESXIrPJcbQZWkuIQ1KRIblhUGIzi1QOtuyYPN92JnvZFe/lfrHI/6H302K7tQBPhlMQXg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: The PGDAT_RECLAIM_LOCKED bit is used to provide mutual exclusion of node reclaim for struct pglist_data using a single bit. It is "locked" with a test_and_set_bit (similarly to a try lock) which provides full ordering with respect to loads and stores done within __node_reclaim(). It is "unlocked" with clear_bit(), which does not provide any ordering with respect to loads and stores done before clearing the bit. The lack of clear_bit() memory ordering with respect to stores within __node_reclaim() can cause a subsequent CPU to fail to observe stores from a prior node reclaim. This is not an issue in practice on TSO (e.g. x86), but it is an issue on weakly-ordered architectures (e.g. arm64). Fix this with following changes: A) Use clear_bit_unlock rather than clear_bit to clear PGDAT_RECLAIM_LOCKED with a release memory ordering semantic. This provides stronger memory ordering (release rather than relaxed). B) Use test_and_set_bit_lock rather than test_and_set_bit to test-and-set PGDAT_RECLAIM_LOCKED with an acquire memory ordering semantic. This changes the "lock" acquisition from a full barrier to an acquire memory ordering, which is weaker. The acquire semi-permeable barrier paired with the release on unlock is sufficient for this mutual exclusion use-case. Signed-off-by: Mathieu Desnoyers Cc: Andrew Morton Cc: Lorenzo Stoakes Cc: Matthew Wilcox Cc: Alan Stern Cc: Andrea Parri Cc: Will Deacon Cc: Peter Zijlstra Cc: Boqun Feng Cc: Nicholas Piggin Cc: David Howells Cc: Jade Alglave Cc: Luc Maranget Cc: "Paul E. McKenney" Cc: linux-mm@kvack.org --- mm/vmscan.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/mm/vmscan.c b/mm/vmscan.c index c22175120f5d..021b25bdba91 100644 --- a/mm/vmscan.c +++ b/mm/vmscan.c @@ -7567,11 +7567,11 @@ int node_reclaim(struct pglist_data *pgdat, gfp_t gfp_mask, unsigned int order) if (node_state(pgdat->node_id, N_CPU) && pgdat->node_id != numa_node_id()) return NODE_RECLAIM_NOSCAN; - if (test_and_set_bit(PGDAT_RECLAIM_LOCKED, &pgdat->flags)) + if (test_and_set_bit_lock(PGDAT_RECLAIM_LOCKED, &pgdat->flags)) return NODE_RECLAIM_NOSCAN; ret = __node_reclaim(pgdat, gfp_mask, order); - clear_bit(PGDAT_RECLAIM_LOCKED, &pgdat->flags); + clear_bit_unlock(PGDAT_RECLAIM_LOCKED, &pgdat->flags); if (ret) count_vm_event(PGSCAN_ZONE_RECLAIM_SUCCESS);