From patchwork Tue Feb 11 13:00:23 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Brendan Jackman X-Patchwork-Id: 13969550 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1D007C0219D for ; Tue, 11 Feb 2025 13:00:39 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 8E8AC6B0089; Tue, 11 Feb 2025 08:00:38 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 8718A6B008A; Tue, 11 Feb 2025 08:00:38 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 6ECEB280001; Tue, 11 Feb 2025 08:00:38 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 4B47C6B0089 for ; Tue, 11 Feb 2025 08:00:38 -0500 (EST) Received: from smtpin25.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id F40D51416AC for ; Tue, 11 Feb 2025 13:00:37 +0000 (UTC) X-FDA: 83107672914.25.CA9F778 Received: from mail-wm1-f74.google.com (mail-wm1-f74.google.com [209.85.128.74]) by imf10.hostedemail.com (Postfix) with ESMTP id B89DCC000D for ; Tue, 11 Feb 2025 13:00:35 +0000 (UTC) Authentication-Results: imf10.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=KyfH8cen; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf10.hostedemail.com: domain of 38kmrZwgKCHceVXfhViWbjjbgZ.Xjhgdips-hhfqVXf.jmb@flex--jackmanb.bounces.google.com designates 209.85.128.74 as permitted sender) smtp.mailfrom=38kmrZwgKCHceVXfhViWbjjbgZ.Xjhgdips-hhfqVXf.jmb@flex--jackmanb.bounces.google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1739278835; a=rsa-sha256; cv=none; b=XMFp1yzEVpelefkuGs2QkQNQZ0h0W4Gyjwg8KzHZinVhQ0XaK+YYgWTD0YvmSduGuSRbnk o3JIMMZ+q8rsom0X0g5+0HDo4oa56+IICL/J3VPfJfjupwiP69f6Ojz6Fi6CfUDc0wTYGe Auz+HQz0zxW69p1ebMGdhPZ9ZapP/S0= ARC-Authentication-Results: i=1; imf10.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=KyfH8cen; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf10.hostedemail.com: domain of 38kmrZwgKCHceVXfhViWbjjbgZ.Xjhgdips-hhfqVXf.jmb@flex--jackmanb.bounces.google.com designates 209.85.128.74 as permitted sender) smtp.mailfrom=38kmrZwgKCHceVXfhViWbjjbgZ.Xjhgdips-hhfqVXf.jmb@flex--jackmanb.bounces.google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1739278835; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding:in-reply-to: references:dkim-signature; bh=W7gYg59+sDZQNRnefhKDnqvbzcvbYlsLc2LHTucHdDo=; b=QX06G1SmVfvjIPAqW3/385v2wlS1mv6rPvpmZzTC9nUgYxe18JsGk3/FnER7ngldryVe3g lbyEEV0bckjUxoQxM8vP0ZbxjyCAVjuKbaq8DudNrkUY5voBgH3DqKr4g6aqrEgDTSE4iA rojTjpAikfcXLFU35CshtGylrOJbhDY= Received: by mail-wm1-f74.google.com with SMTP id 5b1f17b1804b1-4392fc6bd21so13519555e9.1 for ; Tue, 11 Feb 2025 05:00:35 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1739278834; x=1739883634; darn=kvack.org; h=cc:to:from:subject:message-id:mime-version:date:from:to:cc:subject :date:message-id:reply-to; bh=W7gYg59+sDZQNRnefhKDnqvbzcvbYlsLc2LHTucHdDo=; b=KyfH8cenE6XXIyE0KdDUT//pLVfbc52Co5eLHgd74B6TLgz8wUa2R1LtVUPJoSiYva Q2I9H3m6Rd9fXPBRYy6yd4M0xBGhVVBYxX55R/A3NwMsN6gDFpjCsDIrJYuqJrOT2ApK Q2o2KYYCeDbuG1WxezvdTr22ncIUf6a1DzH63oiPZ2jYTeG7LJoelaOPR2c5J/r60vqk WL6Mg9ff1PPa5BKSYvuPTRESbesVuiZI1ZI4zXiA2GbTgA/db+QQNQVlezolIiCtCQgA 3oPCY7drxWrdCUbEHQr9Q15t08RO2xIjJ4FzTTa0+Ole0EueWoMjjpy9ZMCFw4nSnoKN wSQQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1739278834; x=1739883634; h=cc:to:from:subject:message-id:mime-version:date:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=W7gYg59+sDZQNRnefhKDnqvbzcvbYlsLc2LHTucHdDo=; b=pFdIt0/9jGX7/2oZqqYqcnxDmADWtesyfRoXE3PlX349KGPw/La+Moc89eKPeGsQNF 5Bt9tgmIlxmdakpJoph4exNcxTbx9D1ED7cFOeOmMvPOR1/v6X+T2qEHWOozfpS2+6UQ U7nIiqiXVdmqmEeOifYk1DTzExGWgTo+0W10Po6ISlHoc9FksI7uEgjhj4vHXF7ZLA0L 2udj+darRvauA7DeGY5NKYVzdh1wNDA8veOwroQbEI++nVU4XVRK4eYBxhIUQ6dxyMGa JzFHBphfFhhF+slpgXpYEYoCuAtPx5kwf+R5VxiDpPnUsMAYPR9VHCBeDDn0WlO3PTwH 4hMA== X-Forwarded-Encrypted: i=1; AJvYcCX+pNekxdypvnzQQ/9+CrbjKIAomNHKWhThrIPSpaEpn7ys7LyiywwFMGs/taEe5L+4agriv+o0GQ==@kvack.org X-Gm-Message-State: AOJu0Yw3wYuB8110RNvFt59E7En549mk5ffBADXvYfK8y30kyyaiiCwx +jqxMLGz20nPE/8x7VlARFVG4mr7xbf3sWtouAvbdMsqLlX/kVFzbeJm1Z825+ErLJCStPVyZAl B69TSiqVR+Q== X-Google-Smtp-Source: AGHT+IHwr3r+Gl9rsjjNjLPgMI6GwOLR7Cj0VDO2gN5+9hWA4l0IXZ1OmyM9sLdTtfqcOymt28f8pEh85VJfxw== X-Received: from wmsp2.prod.google.com ([2002:a05:600c:1d82:b0:439:4a82:6746]) (user=jackmanb job=prod-delivery.src-stubby-dispatcher) by 2002:a05:600c:1f09:b0:439:4d37:7f49 with SMTP id 5b1f17b1804b1-4394d37819emr25173465e9.28.1739278834302; Tue, 11 Feb 2025 05:00:34 -0800 (PST) Date: Tue, 11 Feb 2025 13:00:23 +0000 Mime-Version: 1.0 X-B4-Tracking: v=1; b=H4sIAOdJq2cC/x3MQQqAIBBA0avErBPUEqKrRAuZJp2FGmoRRHdPW r7F/w8UykwF5u6BTBcXTrFB9R2gt9GR4K0ZtNRGaqVECKez1VMWmEKgWMWA1uyjmqQhhNYdmXa +/+eyvu8H7kVt+GMAAAA= X-Change-Id: 20250211-mmugather-comment-3ca5f41805ec X-Mailer: b4 0.15-dev Message-ID: <20250211-mmugather-comment-v1-1-1ac1e0c765d2@google.com> Subject: [PATCH] mm/mmu_gather: Update comment on RCU freeing From: Brendan Jackman To: Will Deacon , "Aneesh Kumar K.V" , Andrew Morton , Nick Piggin , Peter Zijlstra Cc: Rik van Riel , linux-arch@vger.kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Brendan Jackman X-Rspamd-Server: rspam02 X-Rspamd-Queue-Id: B89DCC000D X-Stat-Signature: 9d9ssjxhkb8xkzuwnmn9efubt6rxgn1i X-Rspam-User: X-HE-Tag: 1739278835-638986 X-HE-Meta: U2FsdGVkX19IbsS9/TGglTFp4nzbCcmBzRBpibi8bWfIpwWPPfulXWOe3pofKmMly48GthPRPdS/EWUfwX4lEyPccD25LDImvNRGOaufnG+46CKlNRf512d9r0qdpPmGCsfIorQEqi3h/nwaD/oW53jW77oqmhnjBZYzxCtei+aAWfqrPn72qpFtnjbD/kPogs5Sd4jiwUFc7tmQIkUWsw3W7ca8BMzvKJfl5tdhZ0ACHS1gQUTQli7iHuMatTWovH9A0Bom50rOkG9jgtyW5ZlcqGb37gGhWDDdoDnHCl9ruyk+snv+ieARPEKG42imcfXzodFMAkQFM08E5RO98gBKhP4a/YUSmw4IHGEHoBkURWHpGP2bhEpQBOHosGM0xr+cT1447v3JzVJCiGdasyIicYV8X3HtMDeuk2G2BldY2WXrdUMRG9owQSKXb20kD8isqHtJv/GhC3mSq4Wq8HJiVIfkUimRvwQIcmPTXDJfcGY7/Gg7G9K8M0ebAo449uc6IxaMca2a3gHRh0MHTVJY5YpF5E3dTz7WWirul3yLf/4ozP4pspA+4CcnPJu+HkBWAPTsurhjWKZkzOU01xZ6ywvR38Ic5J3M+AbjqCp7WC5u9BWJm95a0Ci7I4qmi/mZHn02yNRBeckNuhg2L+D2YCuGWHwj9f4J5aVlSNM6TZQp6W6eO9VAqcmgsHX3lFjdW3kLbwBk4FdNGYymj5tP4lLNFGL9l0gnhFcxmhffSfee1cY4yMWV/o/WOpFkofwDQoP6oKb9hzGRDgn5O2XCpGRYbfSZfkbQCAevdjAAz1zunW63oUmTR5ENnqrBEQxosgMjP2YuqeSkspzdQro19ZP5TGY3CYrk4t6oTR0sAJghgeHgARiC+LVKY5tgT2Bu+CRfLddp+RbqDen6Xw5UHUtbIkK4cWtJ1q4uOxh8lKM3P42hTk6931oNK5iClJN7dEYRZxf0ZZG45yb kM4KMB+f Cm5dQ59Z/WFNIqJdjRYyAy+H4vCqiJhscBXCkKX6dzXzOe39lB+bvqEK1rN9ooLJwKE+qvrF9MZEsT9Lui7KkR6bbhVHymO/+7LaTtWjv/JEh7ehU2JAiEey05ejYspOiHRpCfLtdVNrw3uI82g8zyhE/9KRkkpeHi6mYdGo6YiK03G2MTNlUdiYRkXB9xLY/pz07J6LfJkQDdwiZfYIUFf+tnlBHVGy8JeP96oypX8iFSm6MiE8G/8E8SW5RPDZ/QKNPTXD3gvehYkmzA29RzJ4jduiqH5l85C+3/8X29L4jIhfWs575gvOtTQPGHXPc6QtADEGPJKTh3IK0z4SQpo3hvdek32/pXdkTK6Kd3yfr6q6Si6ukdqdj/hc2D1MAMiKtpYAdM1eMxGpYZyaQSrBtTPVFP8xrhrGgjmSUthAYLdVNUuW3r/JdSZApVLO01K5y1cr/V8fahvmlPZhhlEj7GDDbPQ2rGa6c8XbGN+P6KczLs5gJ0fX/CPoBJ+ZOJd76cuzS6fdt02bxPK3aLbD++MsMQ86uXKKSZyBqVaeCfwLRMyK3a8kKX8ai14HEX0MSYEec4FyxRjKXL3RVCVPcuQHRD7pHBqq9MKAJWpEB9sXLUQGNJ6/7XmPm6kAzpqIpkYmVlPr6EJdA4egPOakxGKAyxAjoIKlhe+g/tpR6qhmqwpjcBA0zEjteizL8mpKGCKVnXpV5RNdOzk5qBmkJOUYl2WQV0i8nWmAzqijD0Go1a11j26eW0w== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Some recent discussion on LMKL [0] brought up some interesting and useful additional context on RCU-freeing for pagetables. Note down some extra info in here, in particular a) be concrete about the reason why an arch might not have an IPI and b) add the interesting paravirt details. [0] https://lore.kernel.org/linux-kernel/20250206044346.3810242-2-riel@surriel.com/ --- Note the Lore link in here is referring to the base of the thread. The mail I wanted to actually refer to is not yet on Lore as it's not currently updating. Here's what I have in my mailbox: On Tue, 11 Feb 2025 at 12:07, Peter Zijlstra wrote: > > It would be nice to update the CONFIG_MMU_GATHER_RCU_TABLE_FREE > > comment in mm/mmu_gather.c to mention INVLPG alongside "Architectures > > that do not have this (PPC)" > > Why? This is just one more architecture that does broadcast. Hmm yeah, I didn't really make the leap from "doesn't do an IPI" to "that just means it uses broadcast TLB flush". In that light I can see how it's "just another architecture". I do think it would make sense to be more explicit about that, even though it seems obvious now you point it out. But it's not really relevant to this patchset. > - and while that's being updated it would > > also be useful to note down the paravirt thing you explained above, > > IMO it's pretty helpful to have more examples of the concrete usecases > > for this logic. > > Look at kvm_flush_tlb_multi() if you're interested. The notable detail > is that is avoids flushing TLB for vCPUs that are preempted, and instead > lets the vCPU resume code do the invalidate. Oh that actually looks like a slightly different case from what Rik mentioned? > some paravirt TLB flush implementations > handle the TLB flush in the hypervisor, and will do the flush > even when the target CPU has interrupts disabled. Do we have a) Systems where the flush gets completely pushed into the hypervisor - xen_flush_tlb_multi()? b) Systems where the guest coordinates with the hypervisor to avoid IPI-ing preempted vCPUs? Maybe I can send a separate patch to note this in the commentary, it's interesting and useful to know. Signed-off-by: Brendan Jackman --- mm/mmu_gather.c | 12 ++++++++++-- 1 file changed, 10 insertions(+), 2 deletions(-) --- base-commit: 266a5a879d40554630c7e485cb5576227759c7a0 change-id: 20250211-mmugather-comment-3ca5f41805ec Best regards, diff --git a/mm/mmu_gather.c b/mm/mmu_gather.c index 7aa6f18c500b2d292621ec308f575ed4ddbdcd3e..db7ba4a725d6ad445eb7f35f0b34e0d4364eb693 100644 --- a/mm/mmu_gather.c +++ b/mm/mmu_gather.c @@ -246,8 +246,16 @@ static void __tlb_remove_table_free(struct mmu_table_batch *batch) * IRQs delays the completion of the TLB flush we can never observe an already * freed page. * - * Architectures that do not have this (PPC) need to delay the freeing by some - * other means, this is that means. + * Not all systems IPI every CPU for this purpose: + * + * - Some architectures have HW support for cross-CPU synchronisation of TLB + * flushes, so there's no IPI at all. + * + * - Paravirt guests can do this TLB flushing in the hypervisor, or coordinate + * with the hypervisor to defer flushing on preempted vCPUs. + * + * Such systems need to delay the freeing by some other means, this is that + * means. * * What we do is batch the freed directory pages (tables) and RCU free them. * We use the sched RCU variant, as that guarantees that IRQ/preempt disabling