From patchwork Mon Dec 18 09:07:17 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ankit Agrawal X-Patchwork-Id: 13496432 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id B434EC35274 for ; Mon, 18 Dec 2023 09:08:02 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 4246D6B0082; Mon, 18 Dec 2023 04:08:02 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 3D1AE6B0083; Mon, 18 Dec 2023 04:08:02 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 272EE6B0085; Mon, 18 Dec 2023 04:08:02 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 12DD26B0082 for ; Mon, 18 Dec 2023 04:08:02 -0500 (EST) Received: from smtpin05.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id DD9481C1362 for ; Mon, 18 Dec 2023 09:08:01 +0000 (UTC) X-FDA: 81579361962.05.B2D7F8E Received: from NAM12-DM6-obe.outbound.protection.outlook.com (mail-dm6nam12on2078.outbound.protection.outlook.com [40.107.243.78]) by imf14.hostedemail.com (Postfix) with ESMTP id E560E100012 for ; Mon, 18 Dec 2023 09:07:58 +0000 (UTC) Authentication-Results: imf14.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b=a+jMdB4z; spf=pass (imf14.hostedemail.com: domain of ankita@nvidia.com designates 40.107.243.78 as permitted sender) smtp.mailfrom=ankita@nvidia.com; arc=pass ("microsoft.com:s=arcselector9901:i=1"); dmarc=pass (policy=reject) header.from=nvidia.com ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1702890479; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=9Q6+cppTzXQ6+dca78fLTIGwICeCojPHJ5xk8VX6mVg=; b=dlXxG2YvT3k+M6jV2NWF2lODxPFxL5AeZ7iy/qqD5/bPZuNkcUgXfZ4/39EnpK4707VtTe wGVTRUnQBxVmk857NPjtgKNX4tNqqFqD0tWl4SpwCggRR0MfWZBwBTeonDIg52UEyHW49r W/TE/yOz5fCRjnUV/0yPZCzZ/euwhZU= ARC-Authentication-Results: i=2; imf14.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b=a+jMdB4z; spf=pass (imf14.hostedemail.com: domain of ankita@nvidia.com designates 40.107.243.78 as permitted sender) smtp.mailfrom=ankita@nvidia.com; arc=pass ("microsoft.com:s=arcselector9901:i=1"); dmarc=pass (policy=reject) header.from=nvidia.com ARC-Seal: i=2; s=arc-20220608; d=hostedemail.com; t=1702890479; a=rsa-sha256; cv=pass; b=TWn92o8bwtqfkzuRnzKYCzAOJ3j4/ybqMldgkU90m2KPGb7TxOjR0cQ65V+KzIG3m0+IJR HIMKQiV876eb39ZX0x2A+NQH64CR1z9czqTQkqdjxEVrJjfCGjhWtRvPh0OYzbTWDU3AZP 5BffrqjUL+Bv9JcpBMvBsyXxPvMC2Ik= ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Ap3lA2dKh9Gstrfm9m5Mr0w6PPzafbsLzKhnPCjjIzdbIr1gcMjj5GI+OBYF6rNe10vkFfoNjutdJ310s3aADVG96Pmhp1bM0a0p1zrYtgt4TrXWhDzD7Lkqn3pZVEnsUA6Fpf+1K2rGg/3DGC0beyVi34tFCn+upZVPgxPT2reo6k1APSplzL8qQp0bJaugRFieXLzWukXIEf82D5PX7jhBKavG7vTVnzhbcrk9gsDYcguXGd13AZ8KwKZmd8UWgKkNQVOMuJ3q9ZGMU0v97DQdwkFHB7n456PWYjaj0EqFYe4FBpSz0+dvGZhGeQhqu04c1EdPd4XiAuPTdMxKUQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=9Q6+cppTzXQ6+dca78fLTIGwICeCojPHJ5xk8VX6mVg=; b=JsAgiWQuVJrI6aKPTjKxdGg5/ULbUSJvdkR3mNCan3dmdgrSerjl++/vsqbZzh8ro1r77M9A+e0dJWHggfSg1Zo0rY4iTv8V14cRTNTzjpWRosrHpoHhz8OHEtIAUBpMFq6ZyDW723GMxGnL5x9y2n3HKcKSewwqUDymy9zSwfC7BfcqWFtlCrb9uDbxlEyEDmOPDGQ4HC/5C+/+mOd1l38Zt/F/9aFrIhTJpb7EQpC0Y5QWvyv2h8Ft+u7P59etOmaIEjRDp/7RRusjL7WGquOFAlfuWShEiGfqp7Mzytujhfu99t1OdJPSjWb6ZPHYGjwryL5LSsceiu39mdN7hg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.160) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=9Q6+cppTzXQ6+dca78fLTIGwICeCojPHJ5xk8VX6mVg=; b=a+jMdB4zsdaHs9QPkhK147SxRh1NlSSkqngXpvWjHX9ZMzE+ERQQMT5bBVNmIMmhu+Ycea2xGOErMOthnixRioJAT9Vlcg7A/LItqS+xOdKRAMjVW/cX9uhn/p/uVIm4alCuUVOFsoEEOGMT48munnfV+cHo6cbe/qmCNMVTHSIfdYfhi90fvQK5aP/odkkjDjmpp4EoBIaIjef7CzBDjCgz78wsPDpZP60eUaYcRKkFe/l9UxHxDG1LlpsjEqyT69YTe/kaUDx4yLNocEsJCtRCSJ5GDbwI5viZAP9xLsAMH7prUgCnil6mwRaigipCuJCKIL9yVO91LPac8wBkdw== Received: from SN1PR12CA0097.namprd12.prod.outlook.com (2603:10b6:802:21::32) by MW3PR12MB4410.namprd12.prod.outlook.com (2603:10b6:303:5b::24) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7091.37; Mon, 18 Dec 2023 09:07:54 +0000 Received: from SN1PEPF0002BA50.namprd03.prod.outlook.com (2603:10b6:802:21:cafe::31) by SN1PR12CA0097.outlook.office365.com (2603:10b6:802:21::32) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7091.38 via Frontend Transport; Mon, 18 Dec 2023 09:07:54 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.160) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.160 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.160; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.160) by SN1PEPF0002BA50.mail.protection.outlook.com (10.167.242.73) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7113.14 via Frontend Transport; Mon, 18 Dec 2023 09:07:54 +0000 Received: from rnnvmail204.nvidia.com (10.129.68.6) by mail.nvidia.com (10.129.200.66) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.41; Mon, 18 Dec 2023 01:07:41 -0800 Received: from rnnvmail202.nvidia.com (10.129.68.7) by rnnvmail204.nvidia.com (10.129.68.6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.41; Mon, 18 Dec 2023 01:07:39 -0800 Received: from sgarnayak-dt.nvidia.com (10.127.8.9) by mail.nvidia.com (10.129.68.7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.41 via Frontend Transport; Mon, 18 Dec 2023 01:07:30 -0800 From: To: , , , , , , , , , , , , , , , CC: , , , , , , , , , , , , , Subject: [PATCH v4 1/3] kvm: arm64: introduce new flag for non-cacheable IO memory Date: Mon, 18 Dec 2023 14:37:17 +0530 Message-ID: <20231218090719.22250-2-ankita@nvidia.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20231218090719.22250-1-ankita@nvidia.com> References: <20231218090719.22250-1-ankita@nvidia.com> MIME-Version: 1.0 X-NV-OnPremToCloud: ExternallySecured X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: SN1PEPF0002BA50:EE_|MW3PR12MB4410:EE_ X-MS-Office365-Filtering-Correlation-Id: 2426bba0-971e-46d6-1851-08dbffa8d1ef X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: yuoZ6c0t4VOS62FPu8+vbtDD9HoBy5nPdB82kr0WVHCWEI1JysWAFhcJ1Hl+CTfxk7Qtnxq6cqBy4Li7qlQVomGKAC4zzM/zpuXJlxM22jSQE8U5BtJmycebFxBF9oMXAtq00rLoCD8ff+f1vsn4xm5yZb74BhSBvdn/thZQW9TfC8X63av7DppXGr6uTbqJ3HmsDNaLNW+ZUUSDsQQ5HnkJvt1l4uNOsRZJ3HQDCslJd8TGN736c1yEUL+s+IAdIyN/4+Wphnuc9G132PVIJhLXpwWJW/C45XxvWHDKum3x1IpuaZYsM7cEjMpgkQE70cc2vmIM6T5FBBB60zanWQ6nRyjp6v8j0RrnlxpzRc081kBp3F+dDcfuSzELyVtJoY1vn+B/5WFsvNSEwksmP6eVslhFWbc6sNEjihfljPMHLJRWdL1HUwLHF+DUbp9F8aBajU47C+etQVXWMo32s50RYrrMOmSk2Od7DSREbvJntPUqUPOyk1x6mfGdpI0SjZPmyww62J9syMR7z8jLhas2v/9htcA1PYKG1o28AKnD3K5/0AXie+0A/SPlxmwcFPsu1suSHusHofYzGQIuzBOufTfzp+BjiDVRU58zqfq6aMlQV2LDNRU74DFY2ZIccObtjssDKbGKI18+l0rzcaQVE0CGN7qcZcLM5YxmorB5Gy02/IOxhRr91tyoTnXuSsDBb0PYbyh5ccyDLt/ZxPK1nFsqHQI9UgI+lUilWI6z7NCeSdLH+uZyFlp/JNewjz+EnxooLm51qaksxumOsSjtDsjBeK9f8WLgStiTkUMyYlwtaxkLioCml19+gAm8 X-Forefront-Antispam-Report: CIP:216.228.117.160;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc6edge1.nvidia.com;CAT:NONE;SFS:(13230031)(4636009)(346002)(376002)(39860400002)(136003)(396003)(230922051799003)(451199024)(1800799012)(82310400011)(64100799003)(186009)(40470700004)(46966006)(36840700001)(336012)(1076003)(2616005)(26005)(19627235002)(6666004)(426003)(7696005)(36860700001)(83380400001)(47076005)(5660300002)(478600001)(2876002)(2906002)(7416002)(8676002)(8936002)(4326008)(110136005)(41300700001)(316002)(70206006)(54906003)(70586007)(7636003)(356005)(82740400003)(921008)(86362001)(36756003)(40480700001)(40460700003)(2101003)(83996005);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 18 Dec 2023 09:07:54.3346 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 2426bba0-971e-46d6-1851-08dbffa8d1ef X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.117.160];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: SN1PEPF0002BA50.namprd03.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: MW3PR12MB4410 X-Rspamd-Queue-Id: E560E100012 X-Rspam-User: X-Stat-Signature: 1iq13pxkdfjrrm5ihhrpmy34sh9emi84 X-Rspamd-Server: rspam01 X-HE-Tag: 1702890478-831295 X-HE-Meta: U2FsdGVkX19Dj/bqeUH+HZo2ZQrCrHalMMi6Qvg8qPAamYafukbq221P5AdNfOi/27TCnbPvny2DNJ9oLm314tTbSbRAPWpyEjaOs3z/RFnXhQQ3ZAvhe9Y1fMz81Etatwxr8L40/7tTYKAIqkmSvXlGbYW6Ju8XXK9WvEwwI1VlEzExEowO2xqKiFvbg2qhu0V7RSaL8wTsGHKok2PzyDrI0LPmk4/pY0s39Ujn56T6KscxsvdTRA9jsmWfkawa1VOzG1LJJt4PC6LNen2osbEL9E1DUXJ/G3l1putLc5z8quAEevoD4or/+a+Sy/hqzHhCNoOfQQmiRhL0CWAiFxVdXlvJ8jjQzmKzrc9DsNYy7LwDYYyGSQRqnMJXC0z5F+bX6dm1xjD4Wn8IEpSveLY9XX+PwR062udLTmEJ0FSSNBSio9FPr+SAYFZWshzCETtx+51A243X+buGuuVJtJG6DJaZT23onrOJl93rue/G63g+iflC8T/yYyI1roldAXkoOjnclev/ONREKzIpLF/LQF26LFYni/OcHa942Y668dupaBUmaZH1c2oUt/dsigjLqQnFkZVdbF8zKHWGCA4LhOUiomFDPCmkI9+mG42e6o2zDSsNEBBr4sdZ93M3D0YQm3scvBlvH033jCs8RpaMJ621QZDEsbxUFLvo52Irjbhbh6RtlEEnbhFErIcBIkQa/xiX8tYdWIiTd5pofYNmMFICmDjEDo+MGPSsgUy9h4pCbiFFrzeq5bYqLReQfHwqgMPtWWlnOkUtfyeFkGXQupJI2Uv6rOjltzkVUb5L4Fx/+kMZrpt4Z1l04jVEaBsq6SKrV3Hyre80qrHHBzD0AO9PVxtjqYmjB8yeBsTnR5UE9I42j2/2JAlXzBFD/kClcPwfayfCVytwNObVdQuwpJI0lMS9d1kd76DK/3ksa2Rb37LH5nLVX+GOskCwAVUiqRbBsE2gNO/3TNH NhXroTWw +MG8lWVZcfrzIc6R4ll0fTg2f9H2Gntmwg4wMYXy0jiYRPk2ZGrUG9OGbazsZzm8qSMtGPIDtG9RYur/NiDXRUyF7Wac3XYJ6q2oBKhZaBs5nsxVD/iUTtgKtmt/hndC3t+h0IBECDQWwbvjP/Do+BkBWoOeEHWUoG418vDkONkSbcTJdGD36IV7v6+wsga6QfUgFdzU3ohEG7Tsb16pgrHq0wsJgRKI0kxupYc16UBQR5KbhJVJymA/8OCku2cVTkc6bETCSgyRNzVcjBWMsrsrLrLNYd2bhyIcE X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Ankit Agrawal For various reasons described in the cover letter, and primarily to allow VM get IO memory with NORMALNC properties, it is desired to relax the KVM stage 2 device memory attributes from DEVICE_nGnRE to NormalNC. So set S2 PTE for IO memory as NORMAL_NC. A Normal-NC flag is not present today. So add a new kvm_pgtable_prot (KVM_PGTABLE_PROT_NORMAL_NC) flag for it, along with its corresponding PTE value 0x5 (0b101) determined from [1]. Lastly, adapt the stage2 PTE property setter function (stage2_set_prot_attr) to handle the NormalNC attribute. [1] section D8.5.5 of DDI0487J_a_a-profile_architecture_reference_manual.pdf Signed-off-by: Ankit Agrawal Suggested-by: Jason Gunthorpe Acked-by: Catalin Marinas Tested-by: Ankit Agrawal --- arch/arm64/include/asm/kvm_pgtable.h | 2 ++ arch/arm64/include/asm/memory.h | 2 ++ arch/arm64/kvm/hyp/pgtable.c | 13 +++++++++++-- 3 files changed, 15 insertions(+), 2 deletions(-) diff --git a/arch/arm64/include/asm/kvm_pgtable.h b/arch/arm64/include/asm/kvm_pgtable.h index cfdf40f734b1..19278dfe7978 100644 --- a/arch/arm64/include/asm/kvm_pgtable.h +++ b/arch/arm64/include/asm/kvm_pgtable.h @@ -197,6 +197,7 @@ enum kvm_pgtable_stage2_flags { * @KVM_PGTABLE_PROT_W: Write permission. * @KVM_PGTABLE_PROT_R: Read permission. * @KVM_PGTABLE_PROT_DEVICE: Device attributes. + * @KVM_PGTABLE_PROT_NORMAL_NC: Normal noncacheable attributes. * @KVM_PGTABLE_PROT_SW0: Software bit 0. * @KVM_PGTABLE_PROT_SW1: Software bit 1. * @KVM_PGTABLE_PROT_SW2: Software bit 2. @@ -208,6 +209,7 @@ enum kvm_pgtable_prot { KVM_PGTABLE_PROT_R = BIT(2), KVM_PGTABLE_PROT_DEVICE = BIT(3), + KVM_PGTABLE_PROT_NORMAL_NC = BIT(4), KVM_PGTABLE_PROT_SW0 = BIT(55), KVM_PGTABLE_PROT_SW1 = BIT(56), diff --git a/arch/arm64/include/asm/memory.h b/arch/arm64/include/asm/memory.h index fde4186cc387..c247e5f29d5a 100644 --- a/arch/arm64/include/asm/memory.h +++ b/arch/arm64/include/asm/memory.h @@ -147,6 +147,7 @@ * Memory types for Stage-2 translation */ #define MT_S2_NORMAL 0xf +#define MT_S2_NORMAL_NC 0x5 #define MT_S2_DEVICE_nGnRE 0x1 /* @@ -154,6 +155,7 @@ * Stage-2 enforces Normal-WB and Device-nGnRE */ #define MT_S2_FWB_NORMAL 6 +#define MT_S2_FWB_NORMAL_NC 5 #define MT_S2_FWB_DEVICE_nGnRE 1 #ifdef CONFIG_ARM64_4K_PAGES diff --git a/arch/arm64/kvm/hyp/pgtable.c b/arch/arm64/kvm/hyp/pgtable.c index c651df904fe3..0fff079a0ef3 100644 --- a/arch/arm64/kvm/hyp/pgtable.c +++ b/arch/arm64/kvm/hyp/pgtable.c @@ -718,10 +718,19 @@ static int stage2_set_prot_attr(struct kvm_pgtable *pgt, enum kvm_pgtable_prot p kvm_pte_t *ptep) { bool device = prot & KVM_PGTABLE_PROT_DEVICE; - kvm_pte_t attr = device ? KVM_S2_MEMATTR(pgt, DEVICE_nGnRE) : - KVM_S2_MEMATTR(pgt, NORMAL); + bool normal_nc = prot & KVM_PGTABLE_PROT_NORMAL_NC; + kvm_pte_t attr; u32 sh = KVM_PTE_LEAF_ATTR_LO_S2_SH_IS; + WARN_ON_ONCE(device && normal_nc); + + if (device) + attr = KVM_S2_MEMATTR(pgt, DEVICE_nGnRE); + else if (normal_nc) + attr = KVM_S2_MEMATTR(pgt, NORMAL_NC); + else + attr = KVM_S2_MEMATTR(pgt, NORMAL); + if (!(prot & KVM_PGTABLE_PROT_X)) attr |= KVM_PTE_LEAF_ATTR_HI_S2_XN; else if (device)