From patchwork Sat Feb 24 15:05:46 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ankit Agrawal X-Patchwork-Id: 13570517 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 06EB5C5478C for ; Sat, 24 Feb 2024 15:07:14 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 8FD016B0093; Sat, 24 Feb 2024 10:07:13 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 8ADFC6B0095; Sat, 24 Feb 2024 10:07:13 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 727136B0096; Sat, 24 Feb 2024 10:07:13 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 5CEC66B0093 for ; Sat, 24 Feb 2024 10:07:13 -0500 (EST) Received: from smtpin19.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 3A5A3C0369 for ; Sat, 24 Feb 2024 15:07:13 +0000 (UTC) X-FDA: 81827025546.19.19B5E59 Received: from NAM02-DM3-obe.outbound.protection.outlook.com (mail-dm3nam02on2065.outbound.protection.outlook.com [40.107.95.65]) by imf02.hostedemail.com (Postfix) with ESMTP id 52F3880008 for ; Sat, 24 Feb 2024 15:07:10 +0000 (UTC) Authentication-Results: imf02.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b=k8Z5dSS1; spf=pass (imf02.hostedemail.com: domain of ankita@nvidia.com designates 40.107.95.65 as permitted sender) smtp.mailfrom=ankita@nvidia.com; dmarc=pass (policy=reject) header.from=nvidia.com; arc=pass ("microsoft.com:s=arcselector9901:i=1") ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1708787230; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=vrVed4bH4ZNo5+WAlGpRDDR5GnNw8bBIVXunyFMfGm8=; b=f2VFi9G/GZR/lnTER1jauSX/E2hDfAIc3ylt31KqWOKhEkEoWZahfUh63cJrkaRzyjwkLa yI98fzOmrfp0EjlZAtv8sQIsWPG7toyS5SpQj2ZDEPfRbF2Mrr9AQCxvFyOs0+P6NRsVaN D6biyoQzEPZ2c0w8X2Z4EzcA+64BR9k= ARC-Seal: i=2; s=arc-20220608; d=hostedemail.com; t=1708787230; a=rsa-sha256; cv=pass; b=w2ULBj3OuTJl5/2Ce77Zd8RpQoCcDl8Bqzyu4GCzUFhA4Cki5bioG08aOlFbXbFyfbiiyy ItS8yd1Ifb8v4u1sutvhgrPp6c0fUstTbzWO8IFAj47u2MvWnWTdjpM/dSvZ9vpry5vn17 R2k4e2VbuLGJOteQrxLFWuQ2NqC/9H0= ARC-Authentication-Results: i=2; imf02.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b=k8Z5dSS1; spf=pass (imf02.hostedemail.com: domain of ankita@nvidia.com designates 40.107.95.65 as permitted sender) smtp.mailfrom=ankita@nvidia.com; dmarc=pass (policy=reject) header.from=nvidia.com; arc=pass ("microsoft.com:s=arcselector9901:i=1") ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=OcnfSWyOFQggSiPTsES65PhBheOmwfvj3zgWuTC9GWHqsJ2j2mjY4ZwNU4SJUwhc9x53l1C6biFXWtziQw+WxYDOpWWnrbUjItzDBD1qLLs1+qt5I/FYjojRZmqkBbWNeSP3GebSLtthUIt6fMEGQuv2XuoHLDYl5bztSPdR0p07cciaTEaMvlsIcqUGPRzqf9SHjgZOqDG2ph5IJvqINHVfwYwLG+zMpWW1pu/8nPX8wqe6X0Mc3mUkMJSOVIcBd41zIycC4Dp410PSG9loxEx08tWvvgIOhDb9wk09hEqj0Y9OTmv+0Pda7VgDDmLjZmEu/1Tr4+NVc0swjTgRwQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=vrVed4bH4ZNo5+WAlGpRDDR5GnNw8bBIVXunyFMfGm8=; b=imUB0lDCeJbrMEpP74471V1uwXeC6zSjztnlULNvXgyu0dv3ZBz5AKFRnmvIbvzmYKO4kokxcRkX3vtJY9M1eLn54YsvQ3HNr4qUfKV8zwsMG9SlFd3LUqhokzJ4HmZpc4cpI83shk39unNjyXFjFFD4OygZJKHv3+ji6S2n9qKnYeK0QoNx/xZrTnsP4tCURd1GJ39WkVciDPEP2CNlSb5DeRLOl1fxoO9CNaUl1W6mw1kM/Qq7AAvLdD+JMmEcJ6R9AelNZhpNc0E4tQyRrjqt8tynH7EmxphT0QJE3U7oFi8P1uXlzlViBXb2MlaCjwFm66aktIEcKpyznax8CA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.160) smtp.rcpttodomain=kernel.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=vrVed4bH4ZNo5+WAlGpRDDR5GnNw8bBIVXunyFMfGm8=; b=k8Z5dSS1+XZWFfeyb7JtGwpF6xBbsOykuKYdSiI4lG1DjI2pLyD//987TA9VAmaEbOW4OqO7/nuY/0+wLeYfUe9mH7PqW6E9Ndo4XEI5Ycv4naE+HCJyn26NiadmBQZYr0cehi8skRdt2O9W4E+IhZp3sgMmCgu2qjOy748G/qGnEg4iOjTRiF9Z+F5kUUjUqB7oJG95Ow6JasVtqYxlKXRwyL946X4dBBBFPgHl8b30BS+1ZPCRo5MA3NmcgdHB4PmuPJ6eee0pUWyL/7zBuXeGpakWYlI7dtfN8z+lVw2CbkK7sKGkd2p+Eit51cjIaxuyKXh3cI232QcVzrtlDw== Received: from DM5PR08CA0052.namprd08.prod.outlook.com (2603:10b6:4:60::41) by BY5PR12MB5511.namprd12.prod.outlook.com (2603:10b6:a03:1d8::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7316.33; Sat, 24 Feb 2024 15:07:04 +0000 Received: from DS3PEPF000099D3.namprd04.prod.outlook.com (2603:10b6:4:60:cafe::71) by DM5PR08CA0052.outlook.office365.com (2603:10b6:4:60::41) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7292.48 via Frontend Transport; Sat, 24 Feb 2024 15:07:04 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.160) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.160 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.160; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.160) by DS3PEPF000099D3.mail.protection.outlook.com (10.167.17.4) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7292.25 via Frontend Transport; Sat, 24 Feb 2024 15:07:04 +0000 Received: from rnnvmail201.nvidia.com (10.129.68.8) by mail.nvidia.com (10.129.200.66) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.41; Sat, 24 Feb 2024 07:06:52 -0800 Received: from rnnvmail203.nvidia.com (10.129.68.9) by rnnvmail201.nvidia.com (10.129.68.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1258.12; Sat, 24 Feb 2024 07:06:50 -0800 Received: from sgarnayak-dt.nvidia.com (10.127.8.9) by mail.nvidia.com (10.129.68.9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1258.12 via Frontend Transport; Sat, 24 Feb 2024 07:06:38 -0800 From: To: , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , CC: , , , , , , , , , , , , , , Subject: [PATCH v9 4/4] vfio: Convey kvm that the vfio-pci device is wc safe Date: Sat, 24 Feb 2024 20:35:46 +0530 Message-ID: <20240224150546.368-5-ankita@nvidia.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20240224150546.368-1-ankita@nvidia.com> References: <20240224150546.368-1-ankita@nvidia.com> MIME-Version: 1.0 X-NV-OnPremToCloud: ExternallySecured X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DS3PEPF000099D3:EE_|BY5PR12MB5511:EE_ X-MS-Office365-Filtering-Correlation-Id: a1b01687-e8ad-4423-fd11-08dc354a4307 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: GxjN7Bwq1Hwv/A/mtDSKWWm4FjbCfLkQvpJEyuKbzX9Y6HdylNrePnfdtgf55bu9XhhbP6bLnOFqWF9i6xaIvdW7BvhuCWgdvoxhyq43q5a7orbA/0bSt95gkrspg2sPSGRrQuzL5mCEwyzmBYEgtTHmsSEm2yH4jU+N+H+Pjfb9EgWy/wHIHVRrvCxzQDccTcqD4WfTgWGZ7b5g38jcfd/asoc8kboPe1C/BMJTk/n8DCAwHf0xjRhCAD09b1KrNMA0zc9kmK5PnNvMmIOa7Vqbbad+RwMFuZDbDYjMg1c2bmomp7ZacCqQquXLOuinEn3dLlssscL0qF6deYGQOwyTIjNFDuib/i7Ld1xpwPEtIczzLZ3ab/KFz3/o8xb5C+5SU7ttaTD93P/ggwca9IGPRjSxe9eFv8JpCZVyzPI6KOoFc5N7KwTJvGunhtl/5QjGRPfMMENd6K4s+dYtIUfPo1ynL2fcxG+v4lMwFRaO/82ttdnhFvM/1JBG+wnMc9JxzTtJqSTtZU+fjzEzIxiOB0jrZEB3lE1XS28DR/fhn/1+MAN4k9ZRLRWUKaQ2yvtvU0QoU/J4u7xe9HnxmV2Q95zJgJbXPfbSbi/7bdKFH6Tlwwc9kS+nhkfk/mmdsaS8elePZESo6uhoynwlMKv7pHgt6TVbfCObTclF/JA= X-Forefront-Antispam-Report: CIP:216.228.117.160;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc6edge1.nvidia.com;CAT:NONE;SFS:(13230031)(36860700004)(40470700004)(46966006)(921011);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 24 Feb 2024 15:07:04.6499 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: a1b01687-e8ad-4423-fd11-08dc354a4307 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.117.160];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: DS3PEPF000099D3.namprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: BY5PR12MB5511 X-Rspamd-Queue-Id: 52F3880008 X-Rspam-User: X-Rspamd-Server: rspam11 X-Stat-Signature: h5ohrzkqa19bz8e41p7khm657uc8h9iu X-HE-Tag: 1708787230-580362 X-HE-Meta: U2FsdGVkX1/4e8ANIjS/8qfUaksJObdPD3C/ro1c+2ja9mv3T9emeDuvwdpyYYrrbAKKkzUZz9f990rScxV6aQ30iMnMnyFxzOLUsNqO/9JhTeHorofROK7qd9Yul/hM9LADR7TAGXHIIjQFTVY0I7lAOCab89dXB9EZkhI1iV36/POP592JAthjMTOWiChoxPHWPKrkWJQ/lgKzjovlU7Oj/Krv+CAd2APrw6niva5YoOEKWRF9P+eli5hXxvAb0w4h76/f2x+6N3J1dqBilBFNBSetTu5r+uoOxOV6Ad+hvbhGGVnLaI3qj625uRmvXdnQv4CjCaCj+32Y3mFQcpq0H9zsdK0RVxc1WHw4YeKfVdQqyAG3WrGLJ/A7C6MZDNaRId5JtktikzRFk6h3akOaOsLuu0qfJeCy2/KNFqFyopt3CohYLWU3ZRIp2T5E59Cl1PKv+BPYpOqShHWg7HaVBT+OnMMx83PESD1pvKDm4wuVQmXEhOgBpdm1PP3dPxCTLNGhHhQgYUEXhe3eu3tKsJ43wS8NGqqRT8RG2j+cW5VMTcgLAo2np75Zh+Ydrd7FXErAoL/4q/sNYNQVe06YspDXH0NVLlPKUeYG7PATuLJQ0JesKh8mk7C+rqc6uC+kiFdwLOJjovKgjOToJ1Fu/Z2dMu4LYenzV2n3QxBYknS68qr0g7ByU/aEYwlkysvPF8pcY/0qKwJoBpjXvxecsavdVkfBaf5S8sxed9k99KoBObaq7JZ2eJfKOHl4FGWvi6qUoe4a/SYASkCfgygSXm8F38Wh7iZjXvI1lZ/+4Q4xOunSUd+ToTHfqxPnNMdf8O1MXEv+RUX+Gx19qF8VqZUaX7ejgyzYegQvsiNn1QeBdcVNgvCH0bgXrhgK8K/8sD2FaPDnyt/Vs5yD6gEdV+B70l6C8BQsX7cm1dy+DHlIIFTdJJDgdZNs9pv/1LdsiTrrtImi4E/UsbG 2FjU/RhM +f3SdyBhkUkBoJgi0zYaYOdfwAgSqwHd7djMRNcNNrSOalpAIqsGtVZ+LW3I8wsXCWpsOA6M9Oi1eodXp42AhmGK+4Ht9Mm2JwpGqreWy0ArurBLIbfdf7WvLACHXVuxaiomziOZvTrnjjYrCuK8xJPT0Ps68rJ647bnx6pvzAGxiZgpSmSxta5AMWtONV5E32FZesTGI0gqVYMu+cRYY+GraUDLBjmt5nKogv/0Oy1kQ/6U9c7ZqY1IgtDJOQoyw/RTQpSHQQ5N1pM8lXVWRZJ1p1etq/GIxU9UGecB98ocSv1mf8CidPuWG3mXia6zo6zQGaBlXiTInS3ikvO1iQB+Tk+iGDZvmUtv2q34D1CMUGeQ= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Ankit Agrawal The VM_ALLOW_ANY_UNCACHED flag is implemented for ARM64, allowing KVM stage 2 device mapping attributes to use Normal-NC rather than DEVICE_nGnRE, which allows guest mappings supporting write-combining attributes (WC). ARM does not architecturally guarantee this is safe, and indeed some MMIO regions like the GICv2 VCPU interface can trigger uncontained faults if Normal-NC is used. To safely use VFIO in KVM the platform must guarantee full safety in the guest where no action taken against a MMIO mapping can trigger an uncontained failure. The expectation is that most VFIO PCI platforms support this for both mapping types, at least in common flows, based on some expectations of how PCI IP is integrated. So make vfio-pci set the VM_ALLOW_ANY_UNCACHED flag. Suggested-by: Catalin Marinas Acked-by: Jason Gunthorpe Acked-by: Catalin Marinas Acked-by: Alex Williamson Reviewed-by: David Hildenbrand Reviewed-by: Marc Zyngier Signed-off-by: Ankit Agrawal --- drivers/vfio/pci/vfio_pci_core.c | 19 ++++++++++++++++++- 1 file changed, 18 insertions(+), 1 deletion(-) diff --git a/drivers/vfio/pci/vfio_pci_core.c b/drivers/vfio/pci/vfio_pci_core.c index 1cbc990d42e0..df6f99bdf70d 100644 --- a/drivers/vfio/pci/vfio_pci_core.c +++ b/drivers/vfio/pci/vfio_pci_core.c @@ -1862,8 +1862,25 @@ int vfio_pci_core_mmap(struct vfio_device *core_vdev, struct vm_area_struct *vma /* * See remap_pfn_range(), called from vfio_pci_fault() but we can't * change vm_flags within the fault handler. Set them now. + * + * VM_ALLOW_ANY_UNCACHED: The VMA flag is implemented for ARM64, + * allowing KVM stage 2 device mapping attributes to use Normal-NC + * rather than DEVICE_nGnRE, which allows guest mappings + * supporting write-combining attributes (WC). ARM does not + * architecturally guarantee this is safe, and indeed some MMIO + * regions like the GICv2 VCPU interface can trigger uncontained + * faults if Normal-NC is used. + * + * To safely use VFIO in KVM the platform must guarantee full + * safety in the guest where no action taken against a MMIO + * mapping can trigger an uncontained failure. The assumption is + * that most VFIO PCI platforms support this for both mapping types, + * at least in common flows, based on some expectations of how + * PCI IP is integrated. Hence VM_ALLOW_ANY_UNCACHED is set in + * the VMA flags. */ - vm_flags_set(vma, VM_IO | VM_PFNMAP | VM_DONTEXPAND | VM_DONTDUMP); + vm_flags_set(vma, VM_ALLOW_ANY_UNCACHED | VM_IO | VM_PFNMAP | + VM_DONTEXPAND | VM_DONTDUMP); vma->vm_ops = &vfio_pci_mmap_ops; return 0;