From patchwork Mon Dec 18 09:07:17 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ankit Agrawal X-Patchwork-Id: 13496432 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id B434EC35274 for ; Mon, 18 Dec 2023 09:08:02 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 4246D6B0082; Mon, 18 Dec 2023 04:08:02 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 3D1AE6B0083; Mon, 18 Dec 2023 04:08:02 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 272EE6B0085; Mon, 18 Dec 2023 04:08:02 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 12DD26B0082 for ; Mon, 18 Dec 2023 04:08:02 -0500 (EST) Received: from smtpin05.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id DD9481C1362 for ; Mon, 18 Dec 2023 09:08:01 +0000 (UTC) X-FDA: 81579361962.05.B2D7F8E Received: from NAM12-DM6-obe.outbound.protection.outlook.com (mail-dm6nam12on2078.outbound.protection.outlook.com [40.107.243.78]) by imf14.hostedemail.com (Postfix) with ESMTP id E560E100012 for ; Mon, 18 Dec 2023 09:07:58 +0000 (UTC) Authentication-Results: imf14.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b=a+jMdB4z; spf=pass (imf14.hostedemail.com: domain of ankita@nvidia.com designates 40.107.243.78 as permitted sender) smtp.mailfrom=ankita@nvidia.com; arc=pass ("microsoft.com:s=arcselector9901:i=1"); dmarc=pass (policy=reject) header.from=nvidia.com ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1702890479; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=9Q6+cppTzXQ6+dca78fLTIGwICeCojPHJ5xk8VX6mVg=; b=dlXxG2YvT3k+M6jV2NWF2lODxPFxL5AeZ7iy/qqD5/bPZuNkcUgXfZ4/39EnpK4707VtTe wGVTRUnQBxVmk857NPjtgKNX4tNqqFqD0tWl4SpwCggRR0MfWZBwBTeonDIg52UEyHW49r W/TE/yOz5fCRjnUV/0yPZCzZ/euwhZU= ARC-Authentication-Results: i=2; imf14.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b=a+jMdB4z; spf=pass (imf14.hostedemail.com: domain of ankita@nvidia.com designates 40.107.243.78 as permitted sender) smtp.mailfrom=ankita@nvidia.com; arc=pass ("microsoft.com:s=arcselector9901:i=1"); dmarc=pass (policy=reject) header.from=nvidia.com ARC-Seal: i=2; s=arc-20220608; d=hostedemail.com; t=1702890479; a=rsa-sha256; cv=pass; b=TWn92o8bwtqfkzuRnzKYCzAOJ3j4/ybqMldgkU90m2KPGb7TxOjR0cQ65V+KzIG3m0+IJR HIMKQiV876eb39ZX0x2A+NQH64CR1z9czqTQkqdjxEVrJjfCGjhWtRvPh0OYzbTWDU3AZP 5BffrqjUL+Bv9JcpBMvBsyXxPvMC2Ik= ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Ap3lA2dKh9Gstrfm9m5Mr0w6PPzafbsLzKhnPCjjIzdbIr1gcMjj5GI+OBYF6rNe10vkFfoNjutdJ310s3aADVG96Pmhp1bM0a0p1zrYtgt4TrXWhDzD7Lkqn3pZVEnsUA6Fpf+1K2rGg/3DGC0beyVi34tFCn+upZVPgxPT2reo6k1APSplzL8qQp0bJaugRFieXLzWukXIEf82D5PX7jhBKavG7vTVnzhbcrk9gsDYcguXGd13AZ8KwKZmd8UWgKkNQVOMuJ3q9ZGMU0v97DQdwkFHB7n456PWYjaj0EqFYe4FBpSz0+dvGZhGeQhqu04c1EdPd4XiAuPTdMxKUQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=9Q6+cppTzXQ6+dca78fLTIGwICeCojPHJ5xk8VX6mVg=; b=JsAgiWQuVJrI6aKPTjKxdGg5/ULbUSJvdkR3mNCan3dmdgrSerjl++/vsqbZzh8ro1r77M9A+e0dJWHggfSg1Zo0rY4iTv8V14cRTNTzjpWRosrHpoHhz8OHEtIAUBpMFq6ZyDW723GMxGnL5x9y2n3HKcKSewwqUDymy9zSwfC7BfcqWFtlCrb9uDbxlEyEDmOPDGQ4HC/5C+/+mOd1l38Zt/F/9aFrIhTJpb7EQpC0Y5QWvyv2h8Ft+u7P59etOmaIEjRDp/7RRusjL7WGquOFAlfuWShEiGfqp7Mzytujhfu99t1OdJPSjWb6ZPHYGjwryL5LSsceiu39mdN7hg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.160) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=9Q6+cppTzXQ6+dca78fLTIGwICeCojPHJ5xk8VX6mVg=; b=a+jMdB4zsdaHs9QPkhK147SxRh1NlSSkqngXpvWjHX9ZMzE+ERQQMT5bBVNmIMmhu+Ycea2xGOErMOthnixRioJAT9Vlcg7A/LItqS+xOdKRAMjVW/cX9uhn/p/uVIm4alCuUVOFsoEEOGMT48munnfV+cHo6cbe/qmCNMVTHSIfdYfhi90fvQK5aP/odkkjDjmpp4EoBIaIjef7CzBDjCgz78wsPDpZP60eUaYcRKkFe/l9UxHxDG1LlpsjEqyT69YTe/kaUDx4yLNocEsJCtRCSJ5GDbwI5viZAP9xLsAMH7prUgCnil6mwRaigipCuJCKIL9yVO91LPac8wBkdw== Received: from SN1PR12CA0097.namprd12.prod.outlook.com (2603:10b6:802:21::32) by MW3PR12MB4410.namprd12.prod.outlook.com (2603:10b6:303:5b::24) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7091.37; Mon, 18 Dec 2023 09:07:54 +0000 Received: from SN1PEPF0002BA50.namprd03.prod.outlook.com (2603:10b6:802:21:cafe::31) by SN1PR12CA0097.outlook.office365.com (2603:10b6:802:21::32) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7091.38 via Frontend Transport; Mon, 18 Dec 2023 09:07:54 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.160) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.160 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.160; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.160) by SN1PEPF0002BA50.mail.protection.outlook.com (10.167.242.73) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7113.14 via Frontend Transport; Mon, 18 Dec 2023 09:07:54 +0000 Received: from rnnvmail204.nvidia.com (10.129.68.6) by mail.nvidia.com (10.129.200.66) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.41; Mon, 18 Dec 2023 01:07:41 -0800 Received: from rnnvmail202.nvidia.com (10.129.68.7) by rnnvmail204.nvidia.com (10.129.68.6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.41; Mon, 18 Dec 2023 01:07:39 -0800 Received: from sgarnayak-dt.nvidia.com (10.127.8.9) by mail.nvidia.com (10.129.68.7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.41 via Frontend Transport; Mon, 18 Dec 2023 01:07:30 -0800 From: To: , , , , , , , , , , , , , , , CC: , , , , , , , , , , , , , Subject: [PATCH v4 1/3] kvm: arm64: introduce new flag for non-cacheable IO memory Date: Mon, 18 Dec 2023 14:37:17 +0530 Message-ID: <20231218090719.22250-2-ankita@nvidia.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20231218090719.22250-1-ankita@nvidia.com> References: <20231218090719.22250-1-ankita@nvidia.com> MIME-Version: 1.0 X-NV-OnPremToCloud: ExternallySecured X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: SN1PEPF0002BA50:EE_|MW3PR12MB4410:EE_ X-MS-Office365-Filtering-Correlation-Id: 2426bba0-971e-46d6-1851-08dbffa8d1ef X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: yuoZ6c0t4VOS62FPu8+vbtDD9HoBy5nPdB82kr0WVHCWEI1JysWAFhcJ1Hl+CTfxk7Qtnxq6cqBy4Li7qlQVomGKAC4zzM/zpuXJlxM22jSQE8U5BtJmycebFxBF9oMXAtq00rLoCD8ff+f1vsn4xm5yZb74BhSBvdn/thZQW9TfC8X63av7DppXGr6uTbqJ3HmsDNaLNW+ZUUSDsQQ5HnkJvt1l4uNOsRZJ3HQDCslJd8TGN736c1yEUL+s+IAdIyN/4+Wphnuc9G132PVIJhLXpwWJW/C45XxvWHDKum3x1IpuaZYsM7cEjMpgkQE70cc2vmIM6T5FBBB60zanWQ6nRyjp6v8j0RrnlxpzRc081kBp3F+dDcfuSzELyVtJoY1vn+B/5WFsvNSEwksmP6eVslhFWbc6sNEjihfljPMHLJRWdL1HUwLHF+DUbp9F8aBajU47C+etQVXWMo32s50RYrrMOmSk2Od7DSREbvJntPUqUPOyk1x6mfGdpI0SjZPmyww62J9syMR7z8jLhas2v/9htcA1PYKG1o28AKnD3K5/0AXie+0A/SPlxmwcFPsu1suSHusHofYzGQIuzBOufTfzp+BjiDVRU58zqfq6aMlQV2LDNRU74DFY2ZIccObtjssDKbGKI18+l0rzcaQVE0CGN7qcZcLM5YxmorB5Gy02/IOxhRr91tyoTnXuSsDBb0PYbyh5ccyDLt/ZxPK1nFsqHQI9UgI+lUilWI6z7NCeSdLH+uZyFlp/JNewjz+EnxooLm51qaksxumOsSjtDsjBeK9f8WLgStiTkUMyYlwtaxkLioCml19+gAm8 X-Forefront-Antispam-Report: CIP:216.228.117.160;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc6edge1.nvidia.com;CAT:NONE;SFS:(13230031)(4636009)(346002)(376002)(39860400002)(136003)(396003)(230922051799003)(451199024)(1800799012)(82310400011)(64100799003)(186009)(40470700004)(46966006)(36840700001)(336012)(1076003)(2616005)(26005)(19627235002)(6666004)(426003)(7696005)(36860700001)(83380400001)(47076005)(5660300002)(478600001)(2876002)(2906002)(7416002)(8676002)(8936002)(4326008)(110136005)(41300700001)(316002)(70206006)(54906003)(70586007)(7636003)(356005)(82740400003)(921008)(86362001)(36756003)(40480700001)(40460700003)(2101003)(83996005);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 18 Dec 2023 09:07:54.3346 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 2426bba0-971e-46d6-1851-08dbffa8d1ef X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.117.160];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: SN1PEPF0002BA50.namprd03.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: MW3PR12MB4410 X-Rspamd-Queue-Id: E560E100012 X-Rspam-User: X-Stat-Signature: 1iq13pxkdfjrrm5ihhrpmy34sh9emi84 X-Rspamd-Server: rspam01 X-HE-Tag: 1702890478-831295 X-HE-Meta: U2FsdGVkX19Dj/bqeUH+HZo2ZQrCrHalMMi6Qvg8qPAamYafukbq221P5AdNfOi/27TCnbPvny2DNJ9oLm314tTbSbRAPWpyEjaOs3z/RFnXhQQ3ZAvhe9Y1fMz81Etatwxr8L40/7tTYKAIqkmSvXlGbYW6Ju8XXK9WvEwwI1VlEzExEowO2xqKiFvbg2qhu0V7RSaL8wTsGHKok2PzyDrI0LPmk4/pY0s39Ujn56T6KscxsvdTRA9jsmWfkawa1VOzG1LJJt4PC6LNen2osbEL9E1DUXJ/G3l1putLc5z8quAEevoD4or/+a+Sy/hqzHhCNoOfQQmiRhL0CWAiFxVdXlvJ8jjQzmKzrc9DsNYy7LwDYYyGSQRqnMJXC0z5F+bX6dm1xjD4Wn8IEpSveLY9XX+PwR062udLTmEJ0FSSNBSio9FPr+SAYFZWshzCETtx+51A243X+buGuuVJtJG6DJaZT23onrOJl93rue/G63g+iflC8T/yYyI1roldAXkoOjnclev/ONREKzIpLF/LQF26LFYni/OcHa942Y668dupaBUmaZH1c2oUt/dsigjLqQnFkZVdbF8zKHWGCA4LhOUiomFDPCmkI9+mG42e6o2zDSsNEBBr4sdZ93M3D0YQm3scvBlvH033jCs8RpaMJ621QZDEsbxUFLvo52Irjbhbh6RtlEEnbhFErIcBIkQa/xiX8tYdWIiTd5pofYNmMFICmDjEDo+MGPSsgUy9h4pCbiFFrzeq5bYqLReQfHwqgMPtWWlnOkUtfyeFkGXQupJI2Uv6rOjltzkVUb5L4Fx/+kMZrpt4Z1l04jVEaBsq6SKrV3Hyre80qrHHBzD0AO9PVxtjqYmjB8yeBsTnR5UE9I42j2/2JAlXzBFD/kClcPwfayfCVytwNObVdQuwpJI0lMS9d1kd76DK/3ksa2Rb37LH5nLVX+GOskCwAVUiqRbBsE2gNO/3TNH NhXroTWw +MG8lWVZcfrzIc6R4ll0fTg2f9H2Gntmwg4wMYXy0jiYRPk2ZGrUG9OGbazsZzm8qSMtGPIDtG9RYur/NiDXRUyF7Wac3XYJ6q2oBKhZaBs5nsxVD/iUTtgKtmt/hndC3t+h0IBECDQWwbvjP/Do+BkBWoOeEHWUoG418vDkONkSbcTJdGD36IV7v6+wsga6QfUgFdzU3ohEG7Tsb16pgrHq0wsJgRKI0kxupYc16UBQR5KbhJVJymA/8OCku2cVTkc6bETCSgyRNzVcjBWMsrsrLrLNYd2bhyIcE X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Ankit Agrawal For various reasons described in the cover letter, and primarily to allow VM get IO memory with NORMALNC properties, it is desired to relax the KVM stage 2 device memory attributes from DEVICE_nGnRE to NormalNC. So set S2 PTE for IO memory as NORMAL_NC. A Normal-NC flag is not present today. So add a new kvm_pgtable_prot (KVM_PGTABLE_PROT_NORMAL_NC) flag for it, along with its corresponding PTE value 0x5 (0b101) determined from [1]. Lastly, adapt the stage2 PTE property setter function (stage2_set_prot_attr) to handle the NormalNC attribute. [1] section D8.5.5 of DDI0487J_a_a-profile_architecture_reference_manual.pdf Signed-off-by: Ankit Agrawal Suggested-by: Jason Gunthorpe Acked-by: Catalin Marinas Tested-by: Ankit Agrawal --- arch/arm64/include/asm/kvm_pgtable.h | 2 ++ arch/arm64/include/asm/memory.h | 2 ++ arch/arm64/kvm/hyp/pgtable.c | 13 +++++++++++-- 3 files changed, 15 insertions(+), 2 deletions(-) diff --git a/arch/arm64/include/asm/kvm_pgtable.h b/arch/arm64/include/asm/kvm_pgtable.h index cfdf40f734b1..19278dfe7978 100644 --- a/arch/arm64/include/asm/kvm_pgtable.h +++ b/arch/arm64/include/asm/kvm_pgtable.h @@ -197,6 +197,7 @@ enum kvm_pgtable_stage2_flags { * @KVM_PGTABLE_PROT_W: Write permission. * @KVM_PGTABLE_PROT_R: Read permission. * @KVM_PGTABLE_PROT_DEVICE: Device attributes. + * @KVM_PGTABLE_PROT_NORMAL_NC: Normal noncacheable attributes. * @KVM_PGTABLE_PROT_SW0: Software bit 0. * @KVM_PGTABLE_PROT_SW1: Software bit 1. * @KVM_PGTABLE_PROT_SW2: Software bit 2. @@ -208,6 +209,7 @@ enum kvm_pgtable_prot { KVM_PGTABLE_PROT_R = BIT(2), KVM_PGTABLE_PROT_DEVICE = BIT(3), + KVM_PGTABLE_PROT_NORMAL_NC = BIT(4), KVM_PGTABLE_PROT_SW0 = BIT(55), KVM_PGTABLE_PROT_SW1 = BIT(56), diff --git a/arch/arm64/include/asm/memory.h b/arch/arm64/include/asm/memory.h index fde4186cc387..c247e5f29d5a 100644 --- a/arch/arm64/include/asm/memory.h +++ b/arch/arm64/include/asm/memory.h @@ -147,6 +147,7 @@ * Memory types for Stage-2 translation */ #define MT_S2_NORMAL 0xf +#define MT_S2_NORMAL_NC 0x5 #define MT_S2_DEVICE_nGnRE 0x1 /* @@ -154,6 +155,7 @@ * Stage-2 enforces Normal-WB and Device-nGnRE */ #define MT_S2_FWB_NORMAL 6 +#define MT_S2_FWB_NORMAL_NC 5 #define MT_S2_FWB_DEVICE_nGnRE 1 #ifdef CONFIG_ARM64_4K_PAGES diff --git a/arch/arm64/kvm/hyp/pgtable.c b/arch/arm64/kvm/hyp/pgtable.c index c651df904fe3..0fff079a0ef3 100644 --- a/arch/arm64/kvm/hyp/pgtable.c +++ b/arch/arm64/kvm/hyp/pgtable.c @@ -718,10 +718,19 @@ static int stage2_set_prot_attr(struct kvm_pgtable *pgt, enum kvm_pgtable_prot p kvm_pte_t *ptep) { bool device = prot & KVM_PGTABLE_PROT_DEVICE; - kvm_pte_t attr = device ? KVM_S2_MEMATTR(pgt, DEVICE_nGnRE) : - KVM_S2_MEMATTR(pgt, NORMAL); + bool normal_nc = prot & KVM_PGTABLE_PROT_NORMAL_NC; + kvm_pte_t attr; u32 sh = KVM_PTE_LEAF_ATTR_LO_S2_SH_IS; + WARN_ON_ONCE(device && normal_nc); + + if (device) + attr = KVM_S2_MEMATTR(pgt, DEVICE_nGnRE); + else if (normal_nc) + attr = KVM_S2_MEMATTR(pgt, NORMAL_NC); + else + attr = KVM_S2_MEMATTR(pgt, NORMAL); + if (!(prot & KVM_PGTABLE_PROT_X)) attr |= KVM_PTE_LEAF_ATTR_HI_S2_XN; else if (device) From patchwork Mon Dec 18 09:07:18 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ankit Agrawal X-Patchwork-Id: 13496433 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1187CC35274 for ; Mon, 18 Dec 2023 09:08:12 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id A63346B0085; Mon, 18 Dec 2023 04:08:11 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id A13AF6B0087; Mon, 18 Dec 2023 04:08:11 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 88CBF6B0088; Mon, 18 Dec 2023 04:08:11 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 778446B0085 for ; Mon, 18 Dec 2023 04:08:11 -0500 (EST) Received: from smtpin14.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 42578C045B for ; Mon, 18 Dec 2023 09:08:11 +0000 (UTC) X-FDA: 81579362382.14.7557EF7 Received: from NAM10-DM6-obe.outbound.protection.outlook.com (mail-dm6nam10on2070.outbound.protection.outlook.com [40.107.93.70]) by imf18.hostedemail.com (Postfix) with ESMTP id 3B4A31C0015 for ; Mon, 18 Dec 2023 09:08:08 +0000 (UTC) Authentication-Results: imf18.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b=KXs8NoKI; spf=pass (imf18.hostedemail.com: domain of ankita@nvidia.com designates 40.107.93.70 as permitted sender) smtp.mailfrom=ankita@nvidia.com; arc=pass ("microsoft.com:s=arcselector9901:i=1"); dmarc=pass (policy=reject) header.from=nvidia.com ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1702890488; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=K4I6QhWVt2wYpDE//XpKCme7DDvvwiXK2/rXSlFvPwU=; b=IMDS9Z6NO2pVRc/eKtGFK7/IC8m6/ED4cCpHqVRuTkIYch5vvNKPjMbmdIZZ4tX59Wam1O zX7J5WgCz9SDm5co6ud1VA+tfKKgVlysZkOlV3iknMBHNjTEfjcD99DVDEVg9i083Jdf8k P3dTovO1P2LMerxDIvX8mgeueDSOLq8= ARC-Authentication-Results: i=2; imf18.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b=KXs8NoKI; spf=pass (imf18.hostedemail.com: domain of ankita@nvidia.com designates 40.107.93.70 as permitted sender) smtp.mailfrom=ankita@nvidia.com; arc=pass ("microsoft.com:s=arcselector9901:i=1"); dmarc=pass (policy=reject) header.from=nvidia.com ARC-Seal: i=2; s=arc-20220608; d=hostedemail.com; t=1702890488; a=rsa-sha256; cv=pass; b=Pnw70kT3Z/7wUGpioP0Wio8GxUAFvHZA7ldro/rhvqiR2YNBP3kkdzaTBI8dYn4u5qv95V Svh4IXFLpWtWLQrleT86j/y36ddRcPPH6RmXcUHpHZhCXggJTUUQLp6OUYU/2V5NZrhmeB CVgKRSMdLfkXfOB8ur5kqkVv5CBQNNI= ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=mdXh5pdtD00NHKxZNaCt+2h6wL+9mnq/NLW8aPR1cDi+U61VjMUhUJvb3GfvtHhwQYDyCw6snG8R4qCKvp9NJv1viByqoejQ/TKaDk46PCgQA3Qf66U4BYC0TY7TBionTQiYg0aXQB91/uJiMyoVFKPNwuPhlzGI1A7qwivbF7F8mU+yGZnqr2AEhR/qY9rzRmk843fwPHrG+SahoLh5Z0KETHgESk2FbiQYpk6VY+EVDkus7Csp2jIbBxSdUQCzNuu7QTEsBr5JiP4TSp/mRceQBhfVVRfEcRbbIqMHvtCdCbGsQXRidS5mpDjVfN0rXcG14pklA9Uc1D/2gj5+Bg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=K4I6QhWVt2wYpDE//XpKCme7DDvvwiXK2/rXSlFvPwU=; b=g6UUywMKZ0OWxzT3p6yB0aiR4QNEzHTG66MfO2S3Z4spNZtnWeehDOTXYwEP8qu7VjMcSMsdw9Lu+mecsWPZkbjsDy9wSDaCsBqqzg7hzVg8ROEeiVuIoNVMWtxKECwo6Gv2sxTmowoNsJdPQpd2nUlMZvQqprGMQXcc481v9Z1adXEQrEWE8/y/ZMGUMb0yknPn3dihm62bMw+cilcTFeZhitkKl5GXI61uVOHVFxuJTpvr+tjNYc2y7KwuDrf7vFrTB+k+hvApOiVBeFbXug4Ko59jorIKOKNxco74ZATd0yd+qtGna4lkg4tF3vuPPOoZfuteJKMmnYpCIUs5kw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.161) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=K4I6QhWVt2wYpDE//XpKCme7DDvvwiXK2/rXSlFvPwU=; b=KXs8NoKIIBd/K2vLbfEOkr/+ODw6MMFTWi/Y9/WKaDyw2ikHLiWUtxf5Ucy5heCxhM/kmgNFslOQfasQ8Ps9SvNSQfIwEA6D9sp5JbzwRkIVulft85U9CkTu3RuaS1sjIesu/j6bEKP4UGeGgo1BcX9Wyax5L+cOCu9/unySyyd5y0WUr5pXxrj8a0f2JNzh/kWSDKg8p3S97HPzjs33UF5/wtqnxmAKPGiL1g/zRHbqTDHC2v9uibZyOdCsPxG1J2HybaHdCzild/4jjxpLwhw/kqJ7ImoC0I2KJqsx7upeUvYBU2vIazf9mlEJvK1bQ4RUaD040UonbI6s7Ptyew== Received: from SA1PR05CA0011.namprd05.prod.outlook.com (2603:10b6:806:2d2::13) by DM6PR12MB4404.namprd12.prod.outlook.com (2603:10b6:5:2a7::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7091.37; Mon, 18 Dec 2023 09:08:03 +0000 Received: from SN1PEPF0002529D.namprd05.prod.outlook.com (2603:10b6:806:2d2:cafe::c8) by SA1PR05CA0011.outlook.office365.com (2603:10b6:806:2d2::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7113.15 via Frontend Transport; Mon, 18 Dec 2023 09:08:03 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.161) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.161 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.161; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.161) by SN1PEPF0002529D.mail.protection.outlook.com (10.167.242.4) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7113.14 via Frontend Transport; Mon, 18 Dec 2023 09:08:02 +0000 Received: from rnnvmail203.nvidia.com (10.129.68.9) by mail.nvidia.com (10.129.200.67) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.41; Mon, 18 Dec 2023 01:07:52 -0800 Received: from rnnvmail202.nvidia.com (10.129.68.7) by rnnvmail203.nvidia.com (10.129.68.9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.41; Mon, 18 Dec 2023 01:07:50 -0800 Received: from sgarnayak-dt.nvidia.com (10.127.8.9) by mail.nvidia.com (10.129.68.7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.41 via Frontend Transport; Mon, 18 Dec 2023 01:07:42 -0800 From: To: , , , , , , , , , , , , , , , CC: , , , , , , , , , , , , , Subject: [PATCH v4 2/3] kvm: arm64: set io memory s2 pte as normalnc for vfio pci devices Date: Mon, 18 Dec 2023 14:37:18 +0530 Message-ID: <20231218090719.22250-3-ankita@nvidia.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20231218090719.22250-1-ankita@nvidia.com> References: <20231218090719.22250-1-ankita@nvidia.com> MIME-Version: 1.0 X-NV-OnPremToCloud: ExternallySecured X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: SN1PEPF0002529D:EE_|DM6PR12MB4404:EE_ X-MS-Office365-Filtering-Correlation-Id: f51a83f0-31c9-4550-d9f6-08dbffa8d718 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: B9aaZeGx5d2XH8GrYze4BxLj38qvOTxggsMSVQSDe7PFAVLAg1NrvO5jvmfu8rMOrgXrpvRKKTA2K8RGT07+FpB61NhqGbDW38t2DFwB3kx8uLt95OWNcmvDJTDLdRIfUmFbJknbbmrvu0cvrDZ41M2YglmILQNcuJphrGOzau9zi+Cg6JTrsPNZjYNBVWiAyGrFtI2YL3qXWLBaO83LdnhHhl1e3TG5NyztdTpKR1ill7FZPAt3td38kOMwg8H/jmC1B38PnME++e5cL98A8KlEOdqeKKHfsbWjgKS5RWOj+wEnlH1bkd8ONgG0kEKg0MwAmlNamMTBeGc+D0otDO9lLJoslbh721a8R8lrzyjcLDW0Le616xOWAIpjLE1VfisbWoewMXrNOMfYl1mi5muGGhwj7dsys35msHysLhfq+4McVB8fALhgaYERObWR33bcZnzy8Af0/e5ZhRC9xt9zsPgEMvtZrpl2AjfBbQgxv5+OZhMWdiqtdhPhgzfza53fT9HP64AoNBLKyYe5KTz/OZo6KwN0mJjY/fXGEG52sRDs5B3ox5JHGh0scqWrJOtrbUEDWssmDRwGRPt5CDcHv9QqqB/9cGc80pqoFgkv2i2KCM++HvIQPW1qi4w3NNWeE7E6M5etl822rizxwjHGP3sCaa3VwQ5YfxOWsf4r9lSbMe4hwnZxgUInTRlXURrAELFmCtkfQqiiNS2oDcq+VYRxE9iN1kanEyfeQ7ZAlhKTLUwwU9pJUb44gWvIqtoqfw9TWqJEStXQA1uU8CEclrN49OyTFNyHwhuf8lKWraPVw76SHfJyg/PvOBrzD6ZU5nJnSOwxlQmj1DIPc5J7p2WeTyISdplSBQEUJ/w= X-Forefront-Antispam-Report: CIP:216.228.117.161;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc6edge2.nvidia.com;CAT:NONE;SFS:(13230031)(4636009)(39860400002)(346002)(396003)(376002)(136003)(230173577357003)(230273577357003)(230922051799003)(82310400011)(1800799012)(186009)(64100799003)(451199024)(40470700004)(46966006)(36840700001)(2616005)(40480700001)(7696005)(26005)(7416002)(5660300002)(478600001)(6666004)(83380400001)(426003)(336012)(1076003)(70206006)(54906003)(70586007)(316002)(110136005)(40460700003)(4326008)(47076005)(8936002)(8676002)(921008)(2876002)(2906002)(86362001)(41300700001)(36860700001)(82740400003)(356005)(7636003)(36756003)(83996005)(2101003);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 18 Dec 2023 09:08:02.9969 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: f51a83f0-31c9-4550-d9f6-08dbffa8d718 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.117.161];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: SN1PEPF0002529D.namprd05.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR12MB4404 X-Rspamd-Queue-Id: 3B4A31C0015 X-Rspam-User: X-Stat-Signature: pc1uu56p8tet6d535wc3zthrojamburi X-Rspamd-Server: rspam01 X-HE-Tag: 1702890488-638079 X-HE-Meta: U2FsdGVkX1+CUNAe1Jy9CWx8aABXROTbh0FDt83bjeer3CRSG3yYJTSAtqcXf+7LDzSWWTbsq//3/cZhiqIHW6CPak4sBwcZ4ymbCNYIHPYLeKjR6SOCF/TS53RlGQnEdtPtLGBaSzZ7RbM88ILQV9g1abV7skuXCXl12mINPIHTEfenMrEWHxnQGKBFYX4mrn7cmvn+1flKUu4JhH7n8zfEsNEYIWcMtGGut6yI5hEQD7WqCMvGop3V8QPlZxwWqr5i9MF1RA638dJceYtk49vkHrrAzd+v4xB0uVtNmKMv7CJB2gVLfEA+XZcuVxQ0aQ6GUoVNKhNFbwi+pwkX89ZAwBUEmrLpfcuaE8EfEdD+I2/W9mSCNdFlaOFQkdrnpn5NYqGepdeA/IXiEGvk9Pe9HafBTAqITIk8Km5+DjD1dCAscL9U0EEMht6ZwczE9R/d+5APrKbXw/WAtRJy/zDHxkC2aXEhVPEKFFXSqpsdEZS1Zh9bmSyLnxhG1rTEy89feH2E8sSbdvQ66uhpRAiFtnToOVUXshSwWiE+yKuLPZw4OGKFsJbwxxoqpSBmhU8U2pyncwg/H3ufMX+x0S44DJBfrF2coNLvBJA33kd0Xkpz/cf3YY1PzbRJhp5RT87He9+Kw8PV1RyDtEbWgRjmpaR+VS0fZhJkzlBg1JlJ7oUxk/7y5jQ6zv1KG0pYAtTt+QyuLTOxGSLP87xzPBWRZ9s/CtlGVX+i7lzkTQWiJkBJQk2sPPsUrQ57ZnippgUBUXtzVmwbHBgd60kUKFA5XWC86snOI/WXjbNhRzTfGNXNSpDyllVbCo/u8nhmsIZJvMGbjDITar83X6vTMV4dG+sCdplmtlB1YtLHQewk5vsywRQiSMUJdivW7yT9pS8Upp5rhsEs3QZMOM82MSqwJiz7MKtGyewhh9SLWv0rOv5x3m8LpMoUm7kczwnud0WrrHkTKMDc57G0Cfc +9MWH8+R XjGZd2T5w5LuE5kC5clVv6jOAlFIYLjTpOTmH0VEtGicyNq8mHXC1VG2MBaqi5KMnNfivIWsJNEQYhSFsG9VkZnAXT64+XcQANgMUE+iSIAvZfjW2sqseNCxDcKJGInFi8Vtl1nutmhh4bZ7Ijt6x8+I1GDMB7hgn2xQp0XdzUTslyv2BAPUbzsj6cY1u2oRWI0plKOKhIFT5Hl15nmqTNC5iZOFW1glVGxBTfFcPylOpbaoIxNcUQa88GUFQWqXilAnEY7Fh/R2NLHD6W0SzPa0ePd5QUIT9YdP/ X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Ankit Agrawal To provide VM with the ability to get device IO memory with NormalNC property, map device MMIO in KVM for ARM64 at stage2 as NormalNC. Having NormalNC S2 default puts guests in control (based on [1], "Combining stage 1 and stage 2 memory type attributes") of device MMIO regions memory mappings. The rules are summarized below: ([(S1) - stage1], [(S2) - stage 2]) S1 | S2 | Result NORMAL-WB | NORMAL-NC | NORMAL-NC NORMAL-WT | NORMAL-NC | NORMAL-NC NORMAL-NC | NORMAL-NC | NORMAL-NC DEVICE | NORMAL-NC | DEVICE Generalizing this to non PCI devices may be problematic. E.g. GICv2 vCPU interface, which is effectively a shared peripheral, can allow a guest to affect another guest's interrupt distribution. The issue may be solved by limiting the relaxation to mappings that have a user VMA. Still There is insufficient information and uncertainity in the behavior of non PCI driver. Hence caution is maintained and the change is restricted to the VFIO-PCI devices. PCIe on the other hand is safe because the PCI bridge does not generate errors, and thus do not cause uncontained failures. A new flag VM_VFIO_ALLOW_WC to indicate KVM that the device is WC capable. KVM use this flag to activate the code. This could be extended to other devices in the future once that is deemed safe. [1] section D8.5.5 of DDI0487J_a_a-profile_architecture_reference_manual.pdf Signed-off-by: Ankit Agrawal Suggested-by: Catalin Marinas Acked-by: Jason Gunthorpe Tested-by: Ankit Agrawal --- arch/arm64/kvm/mmu.c | 18 ++++++++++++++---- include/linux/mm.h | 13 +++++++++++++ 2 files changed, 27 insertions(+), 4 deletions(-) diff --git a/arch/arm64/kvm/mmu.c b/arch/arm64/kvm/mmu.c index d14504821b79..e1e6847a793b 100644 --- a/arch/arm64/kvm/mmu.c +++ b/arch/arm64/kvm/mmu.c @@ -1381,7 +1381,7 @@ static int user_mem_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa, int ret = 0; bool write_fault, writable, force_pte = false; bool exec_fault, mte_allowed; - bool device = false; + bool device = false, vfio_allow_wc = false; unsigned long mmu_seq; struct kvm *kvm = vcpu->kvm; struct kvm_mmu_memory_cache *memcache = &vcpu->arch.mmu_page_cache; @@ -1472,6 +1472,8 @@ static int user_mem_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa, gfn = fault_ipa >> PAGE_SHIFT; mte_allowed = kvm_vma_mte_allowed(vma); + vfio_allow_wc = (vma->vm_flags & VM_VFIO_ALLOW_WC); + /* Don't use the VMA after the unlock -- it may have vanished */ vma = NULL; @@ -1557,10 +1559,18 @@ static int user_mem_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa, if (exec_fault) prot |= KVM_PGTABLE_PROT_X; - if (device) - prot |= KVM_PGTABLE_PROT_DEVICE; - else if (cpus_have_final_cap(ARM64_HAS_CACHE_DIC)) + if (device) { + /* + * To provide VM with the ability to get device IO memory + * with NormalNC property, map device MMIO as NormalNC in S2. + */ + if (vfio_allow_wc) + prot |= KVM_PGTABLE_PROT_NORMAL_NC; + else + prot |= KVM_PGTABLE_PROT_DEVICE; + } else if (cpus_have_final_cap(ARM64_HAS_CACHE_DIC)) { prot |= KVM_PGTABLE_PROT_X; + } /* * Under the premise of getting a FSC_PERM fault, we just need to relax diff --git a/include/linux/mm.h b/include/linux/mm.h index 2bea89dc0bdf..d2f0f969875c 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -391,6 +391,19 @@ extern unsigned int kobjsize(const void *objp); # define VM_UFFD_MINOR VM_NONE #endif /* CONFIG_HAVE_ARCH_USERFAULTFD_MINOR */ +/* This flag is used to connect VFIO to arch specific KVM code. It + * indicates that the memory under this VMA is safe for use with any + * non-cachable memory type inside KVM. Some VFIO devices, on some + * platforms, are thought to be unsafe and can cause machine crashes if + * KVM does not lock down the memory type. + */ +#ifdef CONFIG_64BIT +#define VM_VFIO_ALLOW_WC_BIT 39 +#define VM_VFIO_ALLOW_WC BIT(VM_VFIO_ALLOW_WC_BIT) +#else +#define VM_VFIO_ALLOW_WC VM_NONE +#endif + /* Bits set in the VMA until the stack is in its final location */ #define VM_STACK_INCOMPLETE_SETUP (VM_RAND_READ | VM_SEQ_READ | VM_STACK_EARLY) From patchwork Mon Dec 18 09:07:19 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ankit Agrawal X-Patchwork-Id: 13496434 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 859B6C35274 for ; Mon, 18 Dec 2023 09:08:23 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 229616B0088; Mon, 18 Dec 2023 04:08:23 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 1D6396B0089; Mon, 18 Dec 2023 04:08:23 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 0783E6B008A; Mon, 18 Dec 2023 04:08:23 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id ECE3C6B0088 for ; Mon, 18 Dec 2023 04:08:22 -0500 (EST) Received: from smtpin18.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id C88AD40A46 for ; Mon, 18 Dec 2023 09:08:22 +0000 (UTC) X-FDA: 81579362844.18.0AB4BE0 Received: from NAM04-MW2-obe.outbound.protection.outlook.com (mail-mw2nam04on2054.outbound.protection.outlook.com [40.107.101.54]) by imf20.hostedemail.com (Postfix) with ESMTP id C48DB1C001B for ; Mon, 18 Dec 2023 09:08:19 +0000 (UTC) Authentication-Results: imf20.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b=gs4vFhTA; dmarc=pass (policy=reject) header.from=nvidia.com; spf=pass (imf20.hostedemail.com: domain of ankita@nvidia.com designates 40.107.101.54 as permitted sender) smtp.mailfrom=ankita@nvidia.com; arc=pass ("microsoft.com:s=arcselector9901:i=1") ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1702890499; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=bIgOQYSUcDqbQrALPAOUM80vmPREds8DjkfHG1eIAJU=; b=k+T1fFjO1yy5yKmpcp6YTjdetpKnh+1ZsyC1+/QD0xhXxJe3tbd5EpvC/Fke1EmDpbYLZm LFwwuKjrxM/NA0LiRImTMXrpi8mvDw41Evi3otr3zwBU7LZZ1zAAwliOeOBQFm6lyzoZii kMjr4NrOfYuKwXxdrHWL702kOdLf0Fg= ARC-Authentication-Results: i=2; imf20.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b=gs4vFhTA; dmarc=pass (policy=reject) header.from=nvidia.com; spf=pass (imf20.hostedemail.com: domain of ankita@nvidia.com designates 40.107.101.54 as permitted sender) smtp.mailfrom=ankita@nvidia.com; arc=pass ("microsoft.com:s=arcselector9901:i=1") ARC-Seal: i=2; s=arc-20220608; d=hostedemail.com; t=1702890499; a=rsa-sha256; cv=pass; b=ehzKL5lDzPns2UVtioq7iDBCqdcKxGXQUIkT1u5X/HHRNwjoX5bPdqvbafrhu1pCnoNNi7 AY+9qLX6t9B0XComxtvqWM5G/4Y00LtYaWwn/Vwu0ygw5HL+Gltiiwt4QVnCCk1rYMtc9q 463BOJwpRV4n0uEK6gBvXlFQyj6vc3g= ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=MN/P/0336/kDzC5OhncxeCdivouBz0O12PisQmWLhFjMotA0fmM6qG+ezr9WTNBc3TSq5qizgJ0QHTapRfRW41XWjPVoEMQdKwABd4K2cRM+InC39AyxxEKdKTa2U02w10ywUY5F5lZRcHF2Y8xWln3dqtYhTpR0cVdaky06U3YvsAcIzm004dnRgDTUlyFWuN66VE8pITnuH3aZu6Ovv+IX2p5nSU0tN5eZverv5UT43XAgou/kddvFx8R0tRwg3YgRLzUgjETG41tnkbdjcjnXkvtVtlYnlf05kI5jNawiVCQ2mahkgeBEIAm40j7KrGo//qQ4PqN1mDU8LhdFbw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=bIgOQYSUcDqbQrALPAOUM80vmPREds8DjkfHG1eIAJU=; b=h9CK5Hn3me0MhfXcE8dMGWkMxJjCMdjjBPXCxAaCWnuWPPVHjJ4Azjy5VQJYlH6UK259ZKcfYF0XKM8+1Jq4JUGscSxAI2ElNP1Tzl4uypK5aXFXVTtVkhUGU6p7timNB8Lg62xLgJc/NocXRRIQpwn0H7D/pceZh+Td6aBbohGSFgBjzpWkyyVGo1N2DQWKVT8ka1NGWv0CCZSF6xiwhrrTZbPyrXVOR8AITesMUDH9gP5WOpMnTIpfqFR5SXSvFCDZYSE+5vdv5HKifBDXCUDYqaD9JZpbrslL3jyaL8+phOAaM3h0RyoC0dOINF/oiQMn1iCg4NyFOgbl5L4BKw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.160) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=bIgOQYSUcDqbQrALPAOUM80vmPREds8DjkfHG1eIAJU=; b=gs4vFhTAouIbhD6YvNojDBEN5C8QhMQtPF58OJP9V4TzSDl90YkPPNSGFV9bQU0u/4CJ2V8N4gE9BzsmVq7+6rvkEqat7H0mzfxk/c0eE9RqHcJfZK7gyJDITfhCW+9k36XSAHcne6urLl+0uytFm3cbI76R/z5UK9xjJeosKtrTsz6f5ptVOF4jkIg6679UCncs1UuI4c4GkWQ3RCGOIt9yYwUtmSjWrwmXAqhIu4ZkNqzRfXM2+iKZF/UttOVPZ7tNMyOTDXmzwzLUw1fgzLlRkhy6jWhoMiiGfTBKnKtCTkjEj+ojiMxvfPIkyb0PXJQXEwwKJuSRv/WFdf6DDg== Received: from SA0PR11CA0101.namprd11.prod.outlook.com (2603:10b6:806:d1::16) by BL1PR12MB5303.namprd12.prod.outlook.com (2603:10b6:208:317::14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7091.38; Mon, 18 Dec 2023 09:08:15 +0000 Received: from SN1PEPF0002BA4F.namprd03.prod.outlook.com (2603:10b6:806:d1:cafe::d4) by SA0PR11CA0101.outlook.office365.com (2603:10b6:806:d1::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7091.38 via Frontend Transport; Mon, 18 Dec 2023 09:08:15 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.160) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.160 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.160; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.160) by SN1PEPF0002BA4F.mail.protection.outlook.com (10.167.242.72) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7113.14 via Frontend Transport; Mon, 18 Dec 2023 09:08:14 +0000 Received: from rnnvmail204.nvidia.com (10.129.68.6) by mail.nvidia.com (10.129.200.66) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.41; Mon, 18 Dec 2023 01:08:01 -0800 Received: from rnnvmail202.nvidia.com (10.129.68.7) by rnnvmail204.nvidia.com (10.129.68.6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.41; Mon, 18 Dec 2023 01:08:00 -0800 Received: from sgarnayak-dt.nvidia.com (10.127.8.9) by mail.nvidia.com (10.129.68.7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.41 via Frontend Transport; Mon, 18 Dec 2023 01:07:52 -0800 From: To: , , , , , , , , , , , , , , , CC: , , , , , , , , , , , , , Subject: [PATCH v4 3/3] vfio: convey kvm that the vfio-pci device is wc safe Date: Mon, 18 Dec 2023 14:37:19 +0530 Message-ID: <20231218090719.22250-4-ankita@nvidia.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20231218090719.22250-1-ankita@nvidia.com> References: <20231218090719.22250-1-ankita@nvidia.com> MIME-Version: 1.0 X-NV-OnPremToCloud: ExternallySecured X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: SN1PEPF0002BA4F:EE_|BL1PR12MB5303:EE_ X-MS-Office365-Filtering-Correlation-Id: a427e793-120b-45ce-226a-08dbffa8de3b X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 3lOi9508GZA6Cl4x71zeBQ/Zdg93uI09PNS8og8qUjNgPuk0ss8x8YnWg/8ezTa0BTvYzDDNa7zoMUQfyn8dSBPztxJ0CnBO5xUgJRfMOQ6r7kvHXWyxFiVPzGpzv+/iQp/SHci4yQxxW/uJ176//+fYi7idCpn1hA5azxcflo7dqrygZvqFgTgLECw9l+1mc7YtL9tSU8/K/u1DpGiq9JYNvJT2iItUCXMG4j2FQEMTWVpMo6MhYTCie5F7myCI8GAfFPPE3Xy9bTwrEEqD5wETUifPNjs6HkPQ5tTQa9NLdTcp35LPjOwBOT+6IqFtb3rnmUNN1AoXYAFgxdA6uqS+ZmkjxJwAgtnpKUnXu2bOeNND7etgKnacf9dn7eXgLWZhIzxBpz5BYWVa9syMwH07AhgKhpB3bfwO9HTbEccnnDLQMAD7r5pdLlTc53/woWsLm6OJeazoF91fWuLzW7Jj5iG0ppmVK9jes+m6OpmqXD7OT0e0E+jSpnWdXCKvuhd4Bj4JRx05stzGrpz3Lzt9jzisbrLsoCdlUroRXvjwykuxpTsLKt8DP5vfmWR2IGheExGeUMfDmmx7GnN+NOmJKLhcu/tn85XhaCzx6kbsKG5lzHMIx+BFmhplHJDGdyB4xrYX9uLCjCt71r7U3iTqWl+vygOpH3FCBgisqJAHYmc6tL4jS4UF/nLIhyrKdZWsB0Nnq1NyzeL0EjpDxQPxWH3GN2o+GmWrRg22pVRhDY0cU9QWDMHdZKOlfqoy0HiLR/sM65UU8qCjtGHZzXijx5vbgRt68ZeN+6cWzCBMQgM9V1NqPrQBNEz0uwIL X-Forefront-Antispam-Report: CIP:216.228.117.160;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc6edge1.nvidia.com;CAT:NONE;SFS:(13230031)(4636009)(136003)(376002)(346002)(39860400002)(396003)(230922051799003)(451199024)(186009)(1800799012)(64100799003)(82310400011)(46966006)(40470700004)(36840700001)(40460700003)(1076003)(26005)(336012)(2616005)(426003)(7696005)(6666004)(36860700001)(83380400001)(47076005)(5660300002)(7416002)(2876002)(2906002)(41300700001)(478600001)(316002)(54906003)(8676002)(8936002)(4326008)(70206006)(70586007)(110136005)(356005)(82740400003)(36756003)(86362001)(7636003)(921008)(40480700001)(2101003)(83996005);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 18 Dec 2023 09:08:14.9656 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: a427e793-120b-45ce-226a-08dbffa8de3b X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.117.160];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: SN1PEPF0002BA4F.namprd03.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: BL1PR12MB5303 X-Rspamd-Queue-Id: C48DB1C001B X-Rspam-User: X-Rspamd-Server: rspam04 X-Stat-Signature: tu7j9ajozfb6suzk4xcjkeasyjcno9us X-HE-Tag: 1702890499-748048 X-HE-Meta: U2FsdGVkX183+f8PLuZAdo6lnwijceBLrX1Fm60f8FX2YaqLGAE0HrFgoDh8Jt/CK3G+hF7eeSJs2KF3ANVIaDACxPAippl6BQnCzgItKApgJec5P9pk04LX6lV0QbJA160tA6FJZLqCjSJhQO4Hhj+3UFWJ8O4a+5jYpPA5yLosq5A/sH9IRUqE64YYauNcU9IEtVS41UpJkWjphGygVpYzX9Vx4U4d2GEoBUkLLvWkUdkN/cHh2PpfHpJKLbji4CA9fhS9jS2VV7IjupC4KGnFe7cb7aHUVmaJ0gd+BJ6ohyfB+vzPyN3VAxtR2FodnBzM1gJzF2QIITFp3VZd/R+yx6UJph9zwzYNi34Fbi+AY7SU6zg9h4WEbeSnQJgo2BNP+7fUaYVuqr7Anktc75+pEXawN+tQ2857crRn7lEn4DjBLMruLttoAg5+9dAIbN8RxfKm5Fwh7q4yM9O2gKGEo9UUMAZlUnvpX2UOTo8nOfpFs8L6tQOLAn9B/bFHc8JQip0ze0LJf5OaPn+Z0mcPYWlf4fUwWB5ODcOUlQ58xEJ1pCCM9FqBqJgh/euuTpGvNFwDCHdiuMyN6A+UfciZudp4BATeWz6Ow7BauO1JmyOxRP/yBcaOq5WPa//guSq19xIQYfaDQEHIgLe1DgjaBpYn0ZCwgnhJ4e5vjQXTsJUMdiAlgdQTFBUznMJFBkLFQcjZwNeuGBFDgAqJEcd8HpqOMOF0wx0C/ZnEl/a040to/vTc7T8A2I2sOO/XuWWuOkj4HvcVZnV3c+tO+qVdyCA+IHvQsn0e2/FuBh3Ql5mK6BilGHMnTLbiSPLRcWPF+5Tizmi0ayWwY2cWIyEn+EE01PFso8jVacJQRw8pi+GBmfAh1xbQmgUI0XA5yp3Vsqg4AGdB93Yp+beQF7IfIiui7C7sc+UrnRoEP0+WBSae2tPIEfB3bA0Y7yD6hctI/647U0Rcg8X7AAi /PL2/L0w alMLUBE4zNPzpUbV0m7qS1HEqCPxGafls+gTeu2fghJeJnNHw0YrpYAY+jL+WTl9rGSgfNNFjNyy6bd2GGQdKGdrbN3HGiSIFIzVXlbxTpQXfR0Z20IulRrs/D4upHBu6VkdSHg5ow67S+krj1LjvgyzePzgDtuh3L/cdt04Co7hRbbf205fOd645DGGH04sBgBsn8VW+TmmuHSoFQL8fdszA31e9a05/Ae73eFXaa+DimzYyGG1Slo2Ndji8NY0KrJFOa7ttMjO23a8s50VLPpkVKQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Ankit Agrawal The code to map the MMIO in S2 as NormalNC is enabled when conveyed that the device is WC safe using a new flag VM_VFIO_ALLOW_WC. Make vfio-pci set the VM_VFIO_ALLOW_WC flag. Signed-off-by: Ankit Agrawal Suggested-by: Catalin Marinas Acked-by: Jason Gunthorpe Tested-by: Ankit Agrawal --- drivers/vfio/pci/vfio_pci_core.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/drivers/vfio/pci/vfio_pci_core.c b/drivers/vfio/pci/vfio_pci_core.c index 1cbc990d42e0..c3f95ec7fc3a 100644 --- a/drivers/vfio/pci/vfio_pci_core.c +++ b/drivers/vfio/pci/vfio_pci_core.c @@ -1863,7 +1863,8 @@ int vfio_pci_core_mmap(struct vfio_device *core_vdev, struct vm_area_struct *vma * See remap_pfn_range(), called from vfio_pci_fault() but we can't * change vm_flags within the fault handler. Set them now. */ - vm_flags_set(vma, VM_IO | VM_PFNMAP | VM_DONTEXPAND | VM_DONTDUMP); + vm_flags_set(vma, VM_VFIO_ALLOW_WC | VM_IO | VM_PFNMAP | + VM_DONTEXPAND | VM_DONTDUMP); vma->vm_ops = &vfio_pci_mmap_ops; return 0;