From patchwork Thu Dec 12 13:54:47 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Usama Arif X-Patchwork-Id: 13905230 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id B8DC8E77180 for ; Thu, 12 Dec 2024 13:54:55 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 10A796B0085; Thu, 12 Dec 2024 08:54:55 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 0BB6D6B0089; Thu, 12 Dec 2024 08:54:55 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id EC4496B008C; Thu, 12 Dec 2024 08:54:54 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id CF2FE6B0085 for ; Thu, 12 Dec 2024 08:54:54 -0500 (EST) Received: from smtpin03.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 437E6B0B7F for ; Thu, 12 Dec 2024 13:54:54 +0000 (UTC) X-FDA: 82886451774.03.7731D96 Received: from mail-qk1-f172.google.com (mail-qk1-f172.google.com [209.85.222.172]) by imf05.hostedemail.com (Postfix) with ESMTP id 00A9710000E for ; Thu, 12 Dec 2024 13:54:05 +0000 (UTC) Authentication-Results: imf05.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=BhGzqk+0; spf=pass (imf05.hostedemail.com: domain of usamaarif642@gmail.com designates 209.85.222.172 as permitted sender) smtp.mailfrom=usamaarif642@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1734011669; a=rsa-sha256; cv=none; b=p3BR77LqgleTv/VM1l+FZZUc4SkQI+q8tWXNY/7jhu93OhaiaV/bYqhtC9mTYfy2ocDG8e mOmobk5Ra9m/cpkGpx1t1FhkHR8nMH5jzBlpHp9Rv5z20YsDEgJxLBTNlP2G/sZYZr8052 4ISR3AvVZuZoDrSjNL6It0I6F4CBhBI= ARC-Authentication-Results: i=1; imf05.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=BhGzqk+0; spf=pass (imf05.hostedemail.com: domain of usamaarif642@gmail.com designates 209.85.222.172 as permitted sender) smtp.mailfrom=usamaarif642@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1734011669; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:references:dkim-signature; bh=j9RP6+zMY8aaxIMFUve3LW6blFGvDtrBs3qVoCstadg=; b=e9sLkTcMMqM5cFHJQUYtWU/K5W0hTkMyd87qLzvPPhOf0il7ILPyvEr9/qVEq7kpebbEgp rSXdTC6gV/vGvfttPiKcAE56fInqCXXhQa0KAawnaC8CTqOR79E9L01zBlb3GPh2rvNUU+ 2Dtg1oeGW971dKzeYvv25fdm+s+KGnA= Received: by mail-qk1-f172.google.com with SMTP id af79cd13be357-7b6e5c74cb7so48887685a.2 for ; Thu, 12 Dec 2024 05:54:52 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1734011691; x=1734616491; darn=kvack.org; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:from:to:cc:subject:date:message-id:reply-to; bh=j9RP6+zMY8aaxIMFUve3LW6blFGvDtrBs3qVoCstadg=; b=BhGzqk+0eqnmBK6v2Ajt36+E0VOWpt6kliB2kdU6kIgQr499xVqlAxqAMp3AQG0waM mzi0mCOgShwN3UAzqjpa/9jv3tcraeTZ6/CsR1xU5EQpqgpT0+HlzV54K2iVO4Gi7UfQ +e8nMCyyAD6lwQqCWEX1Lrny06hgyz7f/oqSFA6xPV4KwR/DFLc0AY3NGQsdg12QlB4L kp509s3GWxGu8HI+gx4r+piYSOswv+JwooU0A70Ys6WY/1Ed7w9IlVaLrYOnkgRHMbRM 1czOIyVaH0Zbj7JPD6t25lDsp8zRgQpaV1W390PrVcVRqiTSpBH+qSVLQe1VF/vfr0Sy cK/A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1734011691; x=1734616491; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=j9RP6+zMY8aaxIMFUve3LW6blFGvDtrBs3qVoCstadg=; b=akDrP8izTkB0bSPkdWwO/PQMsyfCpnx4/UiDBtFtl/0yUxfmumlOTZh3my7cHTjg0V Lw8POnpTMpPACKIwONfSngs2Lx3XoXyg4fB7I6M6OTxjUWHt1kUQV6zb/YfKZ2FJjisT 5swBH+Vbvu24HzRC8MOS8ktXAigWX3DNYrd3IbfZTxOEazVv7ZrTw9gf0Qd8W9RbNwPw dyG1vsccscZtJ9xZUqddhY0RJ5a4kyqEvdTzwj47LFkLGc4prEZR4QppA1vZUt6Pvqem lcx6Fj6lEFgQrAf3e60dy4vLkywA4fECtHD33TvzlNkweC5QrZrMCqEEmQ7Uy6Uo12Ly FlDg== X-Forwarded-Encrypted: i=1; AJvYcCV6VreXvmJlNzJvbDIjEeTpRIqnVRswUM9ma6V2nl9PnMcY4TZ/4Ivsaz3dZudUxJrTURRFXvmVMQ==@kvack.org X-Gm-Message-State: AOJu0Yz42fbxABYGZUPedotb8kgNdEq0OIRAC2Z1fvUuN40qU4GJuCFV A4vugbAtMLuqv+Ixa458BaGYTFkAT+b3WzA2asD5dadyzBk4XGip X-Gm-Gg: ASbGnct6/PpBqhmvtDyCCfaQjLQ9hpsbbNz8d3iuDRy9RjF7onTBoLs23Dnc7J5u+hh zU7jDNo66i+Ya7HVaD/3KWthzeuUWJnWW8va8nbRQVtR+5uxmJySepjiWWIf/qdnwCq8O/HNnDs 575nkHrddtZxo6pkSSh4TW4DBvCSR337Cx9nh716A5510Rc/uGHRi3nnN0Vk4tmytJBdLe+ECFX asDrrxGRvVNxVXpz6mfDfGGUscpaS1NHmzjoD/JV1UW+PtqGDHy5A== X-Google-Smtp-Source: AGHT+IFnwr5pOaX9RAL7iibiXd0fE91LuA/C2ZlY+4FYQT+m+IouRTrDzR2WX7o9HmUFOBZ0IUG5og== X-Received: by 2002:a05:620a:6005:b0:7b6:91a4:845b with SMTP id af79cd13be357-7b6f88e7889mr93164685a.15.1734011691567; Thu, 12 Dec 2024 05:54:51 -0800 (PST) Received: from localhost ([2a03:2880:20ff:2::]) by smtp.gmail.com with ESMTPSA id af79cd13be357-7b6b5a8441asm741138985a.84.2024.12.12.05.54.49 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 12 Dec 2024 05:54:50 -0800 (PST) From: Usama Arif To: david@redhat.com, akpm@linux-foundation.org, linux-mm@kvack.org Cc: hannes@cmpxchg.org, riel@surriel.com, shakeel.butt@linux.dev, roman.gushchin@linux.dev, yuzhao@google.com, npache@redhat.com, baohua@kernel.org, ryan.roberts@arm.com, rppt@kernel.org, willy@infradead.org, cerasuolodomenico@gmail.com, ryncsn@gmail.com, corbet@lwn.net, linux-kernel@vger.kernel.org, kernel-team@meta.com, Usama Arif Subject: [PATCH] mm: convert partially_mapped set/clear operations to be atomic Date: Thu, 12 Dec 2024 13:54:47 +0000 Message-ID: <20241212135447.3530047-1-usamaarif642@gmail.com> X-Mailer: git-send-email 2.43.5 MIME-Version: 1.0 X-Rspamd-Queue-Id: 00A9710000E X-Stat-Signature: gqzckf88r77gjnk666csrb395zeoxyni X-Rspam-User: X-Rspamd-Server: rspam09 X-HE-Tag: 1734011645-989369 X-HE-Meta: U2FsdGVkX19xz/8ADp5Vmh16QO6RAf/gCucNhQUK9alT4tYvbWA/H5J64ZYjLmX9ZcXeEiHU05Va0kxz3dngauyo/S4wE2GwvB94ogdmbxqAKYoCg2qslkp4PTOnuYGFRs4geQHUMdwfcdkv8JXLUeAEcUnejzmM/CZJNEHsvUkrcVpkgc+a5wpDrTfqn2Bgrl7GhHvrhRQ8s8VWgZ2MOPcDR6nfoe9ItntNp+J+yXcDxaU+1pzoYOMuU+jMGAuqs8MYiXE57uGWDjPyYsTk3rIGRZZTfvjE8NkC06F+/vYGaWFm6ci5+tRddvVm1cPQ1UfqwIt8YkooABzgtVYzlu4CcIAIEZACbhxWRydvFd8yILHVszLS0jYDuuFcG6oZHrHCh1k9YlxgwHkxen3OvOl6WCOx6aMFx+kA4aZo1zUhCSRDq4GVb/r481gW/osC6C4vwCIOgLDR29RO69Gw8Hahv9zc9a5JuPekHNiwt7t4gRmvYBnZhkPpdXxJp3UiDwE8+NJ4gcL/MiWpvH2LcYQmTE3D4IPWGMX6mntz637BxoajhOAV8jpQQ0npdLYw/lwb4hFARY3w7O2c3dXDGGrzolyDd4jGQtHwE8303d/25aWW/u2acrYV+lVmKp3hK7Z6oTyw/6xCg2CBNJZLk4Yl5VIpC4MS9SEziRFa5P9scTpd8VnlXXuWeaPW1+Qj64R3sTgd7tE/AwvJiUrqQAgMF7uCEP4kuUTQ0qtYbkejvHOwxin5SI3oH847ZOmRkFtWwIcuiCbvsFdDh6jP/R9J5m8DpPt2s5yZoDD998yGpvusHEB0WrkUmi/yDSghJ3qddUDTTZdwVOjBTVOuUSWZu3AUYe9zL386QoZy/ZIQ0LfWXGBAR1neN3ZbEknm2pOOMwLQ5GxccCOziN3SW2aAkG/cCdCuPrmEM+cMWoCHS2AAisoYrqv9dPKsYEFzHGcFKjm9xRUF8otkKuF HZC5xyhj 24kq9AwcOWt5dHQPhZCvV8xr4Eb6MJM/cfXjkNYkjSdxY2OgOliu+qeaHqBhUU/xVuBiR2xBXATUFFz58OWITxg2yD/TTbfJQSU0BEMRk+CT7ifkmb4SW9bHWQS6RMa1KDj/8JEZurSy3YPmpFfAxHtMJ+ILGUYJ6OdYbh8qPUWaL66edv25UQRRTuf/uZltUpXcB3HVnyfY6PQpYmGwGnsRalQrDgYPuiHfOj16vqeG9U99OWY3CtMwHa0wymVM/ptKC9+NmBGs9G1kfTEvl6oEomvO34zytnaieENZM2nnfZvtjAf6r4TxIsoqFnUhn0K+4LRnLGmHTSUf3up1LenkQWt/C6sqA4lZSBGWFymqpPNnvvzRI0ELXcu6uW98QsohyVmXEMwD/MYgEJ5G606K0I6sKBOcz0cvZe1nFF81bYMOPpckcd3Ki9R7OKteBit60sCa//MNprcqUFkR5prCPZ6jx6XtzCCVy X-Bogosity: Ham, tests=bogofilter, spamicity=0.048373, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Other page flags in the 2nd page, like PG_hwpoison and PG_anon_exclusive can get modified concurrently. Hence, partially_mapped flags need to be changed atomically. Fixes: 8422acdc97ed ("mm: introduce a pageflag for partially mapped folios") Reported-by: David Hildenbrand Signed-off-by: Usama Arif Acked-by: David Hildenbrand Acked-by: Johannes Weiner Acked-by: Roman Gushchin --- include/linux/page-flags.h | 12 ++---------- mm/huge_memory.c | 8 ++++---- 2 files changed, 6 insertions(+), 14 deletions(-) diff --git a/include/linux/page-flags.h b/include/linux/page-flags.h index cf46ac720802..691506bdf2c5 100644 --- a/include/linux/page-flags.h +++ b/include/linux/page-flags.h @@ -862,18 +862,10 @@ static inline void ClearPageCompound(struct page *page) ClearPageHead(page); } FOLIO_FLAG(large_rmappable, FOLIO_SECOND_PAGE) -FOLIO_TEST_FLAG(partially_mapped, FOLIO_SECOND_PAGE) -/* - * PG_partially_mapped is protected by deferred_split split_queue_lock, - * so its safe to use non-atomic set/clear. - */ -__FOLIO_SET_FLAG(partially_mapped, FOLIO_SECOND_PAGE) -__FOLIO_CLEAR_FLAG(partially_mapped, FOLIO_SECOND_PAGE) +FOLIO_FLAG(partially_mapped, FOLIO_SECOND_PAGE) #else FOLIO_FLAG_FALSE(large_rmappable) -FOLIO_TEST_FLAG_FALSE(partially_mapped) -__FOLIO_SET_FLAG_NOOP(partially_mapped) -__FOLIO_CLEAR_FLAG_NOOP(partially_mapped) +FOLIO_FLAG_FALSE(partially_mapped) #endif #define PG_head_mask ((1UL << PG_head)) diff --git a/mm/huge_memory.c b/mm/huge_memory.c index 2da5520bfe24..120cd2cdc614 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -3583,7 +3583,7 @@ int split_huge_page_to_list_to_order(struct page *page, struct list_head *list, !list_empty(&folio->_deferred_list)) { ds_queue->split_queue_len--; if (folio_test_partially_mapped(folio)) { - __folio_clear_partially_mapped(folio); + folio_clear_partially_mapped(folio); mod_mthp_stat(folio_order(folio), MTHP_STAT_NR_ANON_PARTIALLY_MAPPED, -1); } @@ -3695,7 +3695,7 @@ bool __folio_unqueue_deferred_split(struct folio *folio) if (!list_empty(&folio->_deferred_list)) { ds_queue->split_queue_len--; if (folio_test_partially_mapped(folio)) { - __folio_clear_partially_mapped(folio); + folio_clear_partially_mapped(folio); mod_mthp_stat(folio_order(folio), MTHP_STAT_NR_ANON_PARTIALLY_MAPPED, -1); } @@ -3739,7 +3739,7 @@ void deferred_split_folio(struct folio *folio, bool partially_mapped) spin_lock_irqsave(&ds_queue->split_queue_lock, flags); if (partially_mapped) { if (!folio_test_partially_mapped(folio)) { - __folio_set_partially_mapped(folio); + folio_set_partially_mapped(folio); if (folio_test_pmd_mappable(folio)) count_vm_event(THP_DEFERRED_SPLIT_PAGE); count_mthp_stat(folio_order(folio), MTHP_STAT_SPLIT_DEFERRED); @@ -3832,7 +3832,7 @@ static unsigned long deferred_split_scan(struct shrinker *shrink, } else { /* We lost race with folio_put() */ if (folio_test_partially_mapped(folio)) { - __folio_clear_partially_mapped(folio); + folio_clear_partially_mapped(folio); mod_mthp_stat(folio_order(folio), MTHP_STAT_NR_ANON_PARTIALLY_MAPPED, -1); }