From patchwork Tue Feb 18 22:22:00 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alex Williamson X-Patchwork-Id: 13980970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id C6F73C021AA for ; Tue, 18 Feb 2025 22:22:35 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id DE2A22801A1; Tue, 18 Feb 2025 17:22:34 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id D946828019B; Tue, 18 Feb 2025 17:22:34 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C0ADB2801A1; Tue, 18 Feb 2025 17:22:34 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id A3A3528019B for ; Tue, 18 Feb 2025 17:22:34 -0500 (EST) Received: from smtpin29.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 6DB76A0BA3 for ; Tue, 18 Feb 2025 22:22:34 +0000 (UTC) X-FDA: 83134490628.29.EC89838 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by imf18.hostedemail.com (Postfix) with ESMTP id AC3FA1C0008 for ; Tue, 18 Feb 2025 22:22:32 +0000 (UTC) Authentication-Results: imf18.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=EdXFq+hc; spf=pass (imf18.hostedemail.com: domain of alex.williamson@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=alex.williamson@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1739917352; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:references:dkim-signature; bh=efAXFa+L6vXYh488YwyV1z4UrGp5zEEawYWAKbu8b9w=; b=WWQhLQR3jm+ZRWwaFY/vE9+TQKqUs6Kjbjt6d/s+ei+QJHzTqg9GFD2CfA6uQoyYfMLSpr F2GEJ046QjnE4klWj88imWK1U438aecWCL5qS1u8wAOPZOnpK0g2FUxKuQECHUN+F/RknC G6Y2HaBYIblnX96dAxiMrF5fn36sxrw= ARC-Authentication-Results: i=1; imf18.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=EdXFq+hc; spf=pass (imf18.hostedemail.com: domain of alex.williamson@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=alex.williamson@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1739917352; a=rsa-sha256; cv=none; b=HxrQuZ4KYApBMx88YZI8gD4mjIRYkUU56XVPrP8o48OBEIwN+K1fzo/L1VKeGzriUX+AE0 rgQCY/LxxKpsYTo6DXwmGlNaJznmmxdZeAQWsysLFRy0MwVoIEPoBCEK5OOPpuraOiITEH Q6qj/ecyR8iSJP9Iitd3DkDDces6JcQ= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1739917352; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding; bh=efAXFa+L6vXYh488YwyV1z4UrGp5zEEawYWAKbu8b9w=; b=EdXFq+hcOFDK7p6TgejkRWySGKG40Sw2zmP797n/zZyG/mkgvltpsdnmuuOvWAJvjfW13Q YWlRO5UQiqZlQe8NdP9od3T6420O6dt63zmpGqAKcoa08uTxnjjIC89/qODhmNlMOtaJQ8 YV8vFkAMaC64u0bd6g9ccHrTYq/4jSs= Received: from mx-prod-mc-08.mail-002.prod.us-west-2.aws.redhat.com (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-528-PBxebS9QNbq5q6NhHJoBKQ-1; Tue, 18 Feb 2025 17:22:26 -0500 X-MC-Unique: PBxebS9QNbq5q6NhHJoBKQ-1 X-Mimecast-MFC-AGG-ID: PBxebS9QNbq5q6NhHJoBKQ_1739917345 Received: from mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-08.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id D8476180087D; Tue, 18 Feb 2025 22:22:24 +0000 (UTC) Received: from omen.home.shazbot.org (unknown [10.22.88.77]) by mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 3C482300019F; Tue, 18 Feb 2025 22:22:15 +0000 (UTC) From: Alex Williamson To: alex.williamson@redhat.com Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, peterx@redhat.com, mitchell.augustin@canonical.com, clg@redhat.com, jgg@nvidia.com, akpm@linux-foundation.org, linux-mm@kvack.org, david@redhat.com, willy@infradead.org Subject: [PATCH v2 0/6] vfio: Improve DMA mapping performance for huge pfnmaps Date: Tue, 18 Feb 2025 15:22:00 -0700 Message-ID: <20250218222209.1382449-1-alex.williamson@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.4 X-Rspamd-Queue-Id: AC3FA1C0008 X-Stat-Signature: 6nanga33krdd5mutqoc8bhgjpmfqg3wi X-Rspam-User: X-Rspamd-Server: rspam01 X-HE-Tag: 1739917352-342407 X-HE-Meta: U2FsdGVkX1+bA/CWUX2OApYcEMNDRI64CRF/mzZY5dVvk2k2m+Q2qCmacwK98RS8cJWbkxld/b4cflPl5M0LAm8y8XI7M1aQr03K/mMsdV0jvDNFGkle6MQ2YJEcibWgxzm2uhtwtz+GvuDW7/j7xRvO671e2c6Fo0wjZXamUESgTosbpUGp8lom2bV60phlr2bI3L0/ostaWqcbeLCDqmapDyyXlS19VEw6oIrYU1Lw51eeeHq/BagwB8HKUXku9CbfFDH3QuFZa3avde4OMNzjRHy3rI1lhVmH11E0LdFFv6w2u3KxhJXvInSjiC9ESqVtYjsRGczSvBLPSgndyoTn+lEk7RpO9ASpJWSZAu/8PKWhY0HR1RNWvc8Aw/LlSNPe74YTL9wZNUxWpgqPGE0Iu1r/n3f5T1C4P53p3S5nbbOh5NLdSaQty1lkZcTxrihR2MYeLmnsMNQ5+80OeGOe/zJxKiJcbdtHnuYAoIAaT/KN0PoxNes4kkM5Ed5lCebj9dqNrErOl34gTbYV1/G0aYzcakt8sgiJx4LwG+8YEaGZjN7Dl2A83pmSeD5SaggiJcnaDexJMsKQjHz/AGn0nxrVze4lbJqglfKXl1pl4d2XIuoziexfRWw1sX3ce0vTciQu6lB0DbXXEG0oYJdp55mh/jDA7tVfqAbioXvrS10uCET4QLu21a2inV0ZQxtE10Pf2cE9mYZJFZA2n5xFo4Bz1O92zkHZ+R+ydg8sAMYHsuCsNHD6Vwf1lbiIiaNiaFAHWnkivtb/sm+C9xOWB2zJah/KxKIcj6eZbrlxzk8MVjmP07gYiZ2iDkdgg9OYwDYapeoQ1PIjYszm4tQ9Jl+GdW1NqPpPPnzQjWS4Q/lI5PXJnmSNpYWmAOnFHoFXGoz9qfp4ay8RPHFUHt/Q9JIH8q7/ogOaYakFFDH0KK+Fmv0BFpgBaSdVX1hNbsMPQ8fZX+/jzfyogKh dxa0vkCH 6vE1nXgkygTfUaVELKuCC6WSX+bhF1+tClBoSsGSd9U5kZEcEOdshTOwxLc/4XDFwfCpns3gSRaLuv9lmHY5rDj+VB5tWJq9g823TBdb1iVyACGKc9VGpoaesAgmznK//vVROGCgCLtfBGjthl7qoiBzyo0MMjzumTY/ZMO5mj5X2E0KtdilB1qV5fcgkwPq0jNF8wwdBX0XhjwpvWrhL4hbu4fhLOSKXblCYx/AxOZ6KHXizTktLeJfrj6LPFPuFjl+j/ci/Lt5ZLMWlV+iFR6Ca1X2iPmqjny6yHAYx9HR10lcAoOSGoB4L1RPffSOW5ICTaZUrol4VC9FKHE1Y2dPGtbdWYHC0yVg2 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000004, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: v2: - Rewrapped comment block in 3/6 - Added 4/6 to use consistent types (Jason) - Renamed s/pgmask/addr_mask/ (David) - Updated 6/6 with proposed epfn algorithm (Jason) - Applied and retained sign-offs for all but 6/6 where the epfn calculation changed v1: https://lore.kernel.org/all/20250205231728.2527186-1-alex.williamson@redhat.com/ As GPU BAR sizes increase, the overhead of DMA mapping pfnmap ranges has become a significant overhead for VMs making use of device assignment. Not only does each mapping require upwards of a few seconds, but BARs are mapped in and out of the VM address space multiple times during guest boot. Also factor in that multi-GPU configurations are increasingly commonplace and BAR sizes are continuing to increase. Configurations today can already be delayed minutes during guest boot. We've taken steps to make Linux a better guest by batching PCI BAR sizing operations[1], but it only provides and incremental improvement. This series attempts to fully address the issue by leveraging the huge pfnmap support added in v6.12. When we insert pfnmaps using pud and pmd mappings, we can later take advantage of the knowledge of the mapping level page mask to iterate on the relevant mapping stride. In the commonly achieved optimal case, this results in a reduction of pfn lookups by a factor of 256k. For a local test system, an overhead of ~1s for DMA mapping a 32GB PCI BAR is reduced to sub-millisecond (8M page sized operations reduced to 32 pud sized operations). Please review, test, and provide feedback. I hope that mm folks can ack the trivial follow_pfnmap_args update to provide the mapping level page mask. Naming is hard, so any preference other than pgmask is welcome. Thanks, Alex [1]https://lore.kernel.org/all/20250120182202.1878581-1-alex.williamson@redhat.com/ Alex Williamson (6): vfio/type1: Catch zero from pin_user_pages_remote() vfio/type1: Convert all vaddr_get_pfns() callers to use vfio_batch vfio/type1: Use vfio_batch for vaddr_get_pfns() vfio/type1: Use consistent types for page counts mm: Provide address mask in struct follow_pfnmap_args vfio/type1: Use mapping page mask for pfnmaps drivers/vfio/vfio_iommu_type1.c | 123 ++++++++++++++++++++------------ include/linux/mm.h | 2 + mm/memory.c | 1 + 3 files changed, 80 insertions(+), 46 deletions(-) Reviewed-by: "Mitchell Augustin" Tested-by: "Mitchell Augustin"