From patchwork Mon Apr 25 08:42:19 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Jan Beulich X-Patchwork-Id: 12825421 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id ECC58C433F5 for ; Mon, 25 Apr 2022 08:52:04 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.312614.529951 (Exim 4.92) (envelope-from ) id 1niuRi-0001wS-Os; Mon, 25 Apr 2022 08:51:50 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 312614.529951; Mon, 25 Apr 2022 08:51:50 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1niuRi-0001wL-Ld; Mon, 25 Apr 2022 08:51:50 +0000 Received: by outflank-mailman (input) for mailman id 312614; Mon, 25 Apr 2022 08:51:49 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1niuIX-0003e7-Mc for xen-devel@lists.xenproject.org; Mon, 25 Apr 2022 08:42:21 +0000 Received: from de-smtp-delivery-102.mimecast.com (de-smtp-delivery-102.mimecast.com [194.104.109.102]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id 9ed68a8e-c473-11ec-a405-831a346695d4; Mon, 25 Apr 2022 10:42:20 +0200 (CEST) Received: from EUR04-DB3-obe.outbound.protection.outlook.com (mail-db3eur04lp2052.outbound.protection.outlook.com [104.47.12.52]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id de-mta-29-0CaPfVyuOMuCq0QkgcbLoA-1; Mon, 25 Apr 2022 10:42:19 +0200 Received: from DU2PR04MB8616.eurprd04.prod.outlook.com (2603:10a6:10:2db::16) by DB8PR04MB6891.eurprd04.prod.outlook.com (2603:10a6:10:112::7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5186.21; Mon, 25 Apr 2022 08:42:18 +0000 Received: from DU2PR04MB8616.eurprd04.prod.outlook.com ([fe80::5cb0:5195:4203:7c2f]) by DU2PR04MB8616.eurprd04.prod.outlook.com ([fe80::5cb0:5195:4203:7c2f%9]) with mapi id 15.20.5186.021; Mon, 25 Apr 2022 08:42:18 +0000 X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 9ed68a8e-c473-11ec-a405-831a346695d4 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619; t=1650876140; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=mS9j+KCKEN6hv0oR924WDhW3Lh9+MjR+9IlswbKwkL8=; b=I76/w8DGOZpgW35SScLZK8Mfh3oq3E5u84hjo3ndSjfDTWyL3P4rbqlbB/25bWPxIsEIyK +wtbtZZJE6qemVuJtRFhAhzUwfo31vQjAsxcwBVAwI3bN6/bkzkIZoerO4uNbKAcTxw/G9 EGzw/qpATgJ/6PBlcdpqTy2Nl4czJpY= X-MC-Unique: 0CaPfVyuOMuCq0QkgcbLoA-1 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=SRUjh57B/BFkSVqND4gx+M5oyLYLIjTb2ioq0JI2nOKfFKrDlC2fcypIHjbzwpogWZFibaVtm0Aj2VMyKG0kYReO4PqjH4xIpXS1qAJReuqlnU1NuJlhHb3XxPXMyk/7u1UcTBE8nv7y14Ei9KUnHRnM+FgVm6cguFicahET4e68zmW4ylxmbNN9mSiToqBNCImRsQidj0DoU4nrH7nHGfeUJI1aPam+fZaghqrfwV4mHAaWBmMXGEM3n61O0CeZM/KI/svS6hFyjj91+d96Z+ryr6i9fw0gRPd/UpY67xNbVtI7ZzXzQk+hsjBwO88hhQYocsjBMgcjN0JFnX8eRg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=mS9j+KCKEN6hv0oR924WDhW3Lh9+MjR+9IlswbKwkL8=; b=HS8FiBEfGvy5bXFfl5ZZJw0pwmK+B1s2/2a781d+5SFuBRlK/KfPFlx9nlRms3xVIYYpb6dR/dUnjivS3JcXy0IaAn2OJXIIRZr8kd2jJgBwxmJBdj6T+1VVXknqpZYLuqBHPgZ09qvOKQTKEmKQTjOWAO466TJk35T9e+33Dt8vDvi1I2iAdu1TKO1rA8WoKMQleikUZeMatUuTdHTZiMBX+qoShmclds+MiJcNmRBodWgiDlcumpv3t6sGA+hq+at2V3AdpLihoG0t40UCrPhJ4OgijXwX9Bcw2iVzZ54HZFFB0rD/4pU3eybiJqWBWPESWJomArl0nzg7tHS8Nw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com; dkim=pass header.d=suse.com; arc=none Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=suse.com; Message-ID: <16f5b398-56f6-a70c-9ce0-0ad72eab5058@suse.com> Date: Mon, 25 Apr 2022 10:42:19 +0200 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:91.0) Gecko/20100101 Thunderbird/91.8.1 Subject: [PATCH v4 15/21] AMD/IOMMU: free all-empty page tables Content-Language: en-US From: Jan Beulich To: "xen-devel@lists.xenproject.org" Cc: Andrew Cooper , Paul Durrant , =?utf-8?q?Roger_Pau_Monn=C3=A9?= References: In-Reply-To: X-ClientProxiedBy: AM5PR1001CA0046.EURPRD10.PROD.OUTLOOK.COM (2603:10a6:206:15::23) To DU2PR04MB8616.eurprd04.prod.outlook.com (2603:10a6:10:2db::16) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: f3b9e395-2f89-4334-3add-08da26978160 X-MS-TrafficTypeDiagnostic: DB8PR04MB6891:EE_ X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: I0gBddgZfz9sq6a4pIpAEnA7XSRVz2ByjpF0e5hz/z57IZ6KtRO6xJsoSySDSDhotybWUy9goPbPUQfJfCl47A7oFtyR84Z/FnouxFNDF/jR72MgR/D1JewmLds6CR8iFx6anCp/UXUi5GztMbOBZTVfd9fa6FMdof44zIPpZCsZ+xgG4nml0AZstfxrFKZfpnf6xqYpgPPTTa87+8KZd0CUaFQaNYkaEbqE/tdirqb8+k2FsH9Xce7YJw6l4nWwaKeejLWW0PKDm4Pf83lhXP0lwcoFniHxJk4yY8HwUXrvHp7/a1B9Zzhhj53n+PCAusdRj0YZZA2eYv1xUU6wScm0VoFh3ViR7S11cbXlwURFQ8hd75Mzdo+rspWljYqz/DINMEuvV+Tr5sMkWC0qxq4QipBBaYSdON2zdLdXNJH18xIEaQgyiiDbTA7zdYBdthslmblMqBnvyXxI/3UHW2Bx4rb4puYRtAtuUkG5PVOTD1zoyuPBFto1NT8dUNBtlBhnPy2umwpq5ScL3DoNr9T4oFnt7zxyXGCJkCgP5zT7li9PhOgGp9GovU+4L+dq+GuHt6zsuWbGkS6ZlVsxwC5OpilNKODOHzgMLyWaqtwGc0GgrNVDZLYYouPxFuX3SbO2s2GJthEJSIqMG9vvWoTmiR3RILNQ/bb7y6VVrb8+jELrA0TFCB0RO8KdHw/vrZUVC3bMEzAfjlJ29c1SqA+whRKM9GRZrf1WLZXa00g= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DU2PR04MB8616.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230001)(366004)(66556008)(26005)(66946007)(6512007)(38100700002)(508600001)(66476007)(316002)(6506007)(83380400001)(54906003)(6486002)(186003)(6916009)(86362001)(31696002)(2616005)(2906002)(5660300002)(31686004)(8936002)(36756003)(4326008)(8676002)(45980500001)(43740500002);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?q?a9Rnz0ncCNhXFC3PIFDW6aSwr7Au?= =?utf-8?q?JLUgOcFOd2zXPRyNGSrM1bLfGfYfS2NpeGKL0qQkU1x8swrzCnc9ZmncvN9eK9OhK?= =?utf-8?q?J0oLIr3+v38/dnzcBALvSIpfWwB07P1+saLbRorvlUrdX4KOEYQxpAN1idqL3n8f/?= =?utf-8?q?DS+DAckYDnuO2KTdt6nlOW9m0Z8om0LOMeV4USL/B5UKHRvIeOGsJVG85Ig7OxjNN?= =?utf-8?q?4HlsUNObjqUhV2YWkhTVEbibreYnojK6OZfi6VsF/6R2274i/2ljCivEjR5SO5X8d?= =?utf-8?q?tC1Eha6LNO20nPEhPfL+96B5/DN2FM1fl0hQ2FFn0CfQZbDE0bmUpXkX141AxdOee?= =?utf-8?q?tWnFE8Bn1GOI0E9IYjVUSmvIzFbQg5yObF2y53I1nS2QxqYTbGl2o6R8rtGVn227+?= =?utf-8?q?A5OyNoiZ6vYqq97zclWnl4Ln4ebQU1kJdLcvJoQvEx0SsS+DplJS49XTGZmbhDANH?= =?utf-8?q?zundFCpsLeCOsudYr0R8SjacehKgurRX2+I97359xBdQN2jzEXkBie0tn0Cyu2PlZ?= =?utf-8?q?jgnwVTxlBMvKwt9LElMQ7EEJnh+7nuP2FqLEh4SGcqh0XM9HdWVCVqTffvxQt6hVp?= =?utf-8?q?mqwJnpvRdniI8/ApRP/VTzivTP2XnqqSJ6C6n5O+gHJrpx2D11rifcFANCx/YsAKR?= =?utf-8?q?/ou2ZoCtmBtzm51HS8ZODnnZUYLBCB3mFqF6iQLFKMRdU8Q8iFvhIPczFz7X/uVUf?= =?utf-8?q?bXyDpp+ngONAYstA3OZAx0fW3Hj4Nli00O5utif9yXwAFLYUmMBqFVWlnpZBn8WfI?= =?utf-8?q?HzPOyt26KRejmLzDDzbHMltPplrc2z3Q/aDeLMATPWnncnTJjv/SQeE3fBe+oyzq/?= =?utf-8?q?8fAHtigckLur0gAG+wYCnvIBOMfswaamaHeQBqOvg+dBB0FO+kK7FYFdgMx/3yEZj?= =?utf-8?q?Ot58PpX5kr2IiY79L2Nfhr7rDvFf/rxOY+A+2j91RCfjMP2nv9gSn1rUcjjEs0WBI?= =?utf-8?q?lx6d1eXAL4X959OB6d2rkDnrw1bCFzXDOzvXHTYtuC0XDmAypRTD3rJRog0TEEObj?= =?utf-8?q?uWVbz5woqZZ88i/OHDKkTbGU8STr6z5HKrkmUZX6Qicv7ScqgGyt94aRkJAiCTj/a?= =?utf-8?q?IGJjwyCsGHcktge3Zj0yUfLwWl3SlGNkFK6X8V7/t9Jvq2TqyED5SzcOOOqRX5UUY?= =?utf-8?q?+grj5ryW3NTC1VHfWwMnJ/wefNX2ABAnXDzAjGekns7FiNPL7YCJusfLydTlO2r24?= =?utf-8?q?pG2214pVWoCl4jPmvfmkxiBX883LJwiSdkJ90Cud9h8AW1omUzyXAyJowcY9KKAPU?= =?utf-8?q?0d/xPupN05UPB50ahJWxcVpxm+9/TvrxHPF3nHmBr0mm26LvaS2V+IL9LwUh6kMT9?= =?utf-8?q?FI+CRBYonV+mto3WzK6TBghvvDJkEjVjiOyBhI+GuU3LVPvQT9y/tPpkls+mljzYN?= =?utf-8?q?SSoi4ZNGEO/EzZVMlb1c/FgYBpqXQNDRu5rpyd0cm4rneVEIGtxzI0rz8o8Tx+QNl?= =?utf-8?q?4sGgunBYEupf7nB9foWsKpKL8iH0B/kqbqNOLMzb2HNpHvQAO2t53hSzs4bWW/kEM?= =?utf-8?q?ny9mFynfOKMd8tX0Qkp0CFn0a9p7w2qt6xkbJhwHsRPgpcOQm0Iz4lCbf953ZNj1w?= =?utf-8?q?168knHb+qfVVNd3t3tbXbK08YM0lj6YPkOZMXlTeekJ8n4wu5VIYtcdbNTllUD+wH?= =?utf-8?q?F5A1r2gnKCchAJfJU43CI1PGa2O5j8WQ=3D=3D?= X-OriginatorOrg: suse.com X-MS-Exchange-CrossTenant-Network-Message-Id: f3b9e395-2f89-4334-3add-08da26978160 X-MS-Exchange-CrossTenant-AuthSource: DU2PR04MB8616.eurprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 25 Apr 2022 08:42:17.9624 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: Ckv8qioaFNZ4ZHa0ocqeNtfYSlenojKy/sQjR16M66MjYgxEcIqLVoMJWYNO4mVpbqd2BZSFVp1uDjSRcmY3iQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB8PR04MB6891 When a page table ends up with no present entries left, it can be replaced by a non-present entry at the next higher level. The page table itself can then be scheduled for freeing. Note that while its output isn't used there yet, pt_update_contig_markers() right away needs to be called in all places where entries get updated, not just the one where entries get cleared. Signed-off-by: Jan Beulich Reviewed-by: Roger Pau Monné --- v4: Re-base over changes earlier in the series. v3: Re-base over changes earlier in the series. v2: New. --- a/xen/drivers/passthrough/amd/iommu_map.c +++ b/xen/drivers/passthrough/amd/iommu_map.c @@ -21,6 +21,9 @@ #include "iommu.h" +#define CONTIG_MASK IOMMU_PTE_CONTIG_MASK +#include + /* Given pfn and page table level, return pde index */ static unsigned int pfn_to_pde_idx(unsigned long pfn, unsigned int level) { @@ -33,16 +36,20 @@ static unsigned int pfn_to_pde_idx(unsig static union amd_iommu_pte clear_iommu_pte_present(unsigned long l1_mfn, unsigned long dfn, - unsigned int level) + unsigned int level, + bool *free) { union amd_iommu_pte *table, *pte, old; + unsigned int idx = pfn_to_pde_idx(dfn, level); table = map_domain_page(_mfn(l1_mfn)); - pte = &table[pfn_to_pde_idx(dfn, level)]; + pte = &table[idx]; old = *pte; write_atomic(&pte->raw, 0); + *free = pt_update_contig_markers(&table->raw, idx, level, PTE_kind_null); + unmap_domain_page(table); return old; @@ -85,7 +92,11 @@ static union amd_iommu_pte set_iommu_pte if ( !old.pr || old.next_level || old.mfn != next_mfn || old.iw != iw || old.ir != ir ) + { set_iommu_pde_present(pde, next_mfn, 0, iw, ir); + pt_update_contig_markers(&table->raw, pfn_to_pde_idx(dfn, level), + level, PTE_kind_leaf); + } else old.pr = false; /* signal "no change" to the caller */ @@ -322,6 +333,9 @@ static int iommu_pde_from_dfn(struct dom smp_wmb(); set_iommu_pde_present(pde, next_table_mfn, next_level, true, true); + pt_update_contig_markers(&next_table_vaddr->raw, + pfn_to_pde_idx(dfn, level), + level, PTE_kind_table); *flush_flags |= IOMMU_FLUSHF_modified; } @@ -347,6 +361,9 @@ static int iommu_pde_from_dfn(struct dom next_table_mfn = mfn_x(page_to_mfn(table)); set_iommu_pde_present(pde, next_table_mfn, next_level, true, true); + pt_update_contig_markers(&next_table_vaddr->raw, + pfn_to_pde_idx(dfn, level), + level, PTE_kind_table); } else /* should never reach here */ { @@ -474,8 +491,24 @@ int cf_check amd_iommu_unmap_page( if ( pt_mfn ) { + bool free; + /* Mark PTE as 'page not present'. */ - old = clear_iommu_pte_present(pt_mfn, dfn_x(dfn), level); + old = clear_iommu_pte_present(pt_mfn, dfn_x(dfn), level, &free); + + while ( unlikely(free) && ++level < hd->arch.amd.paging_mode ) + { + struct page_info *pg = mfn_to_page(_mfn(pt_mfn)); + + if ( iommu_pde_from_dfn(d, dfn_x(dfn), level, &pt_mfn, + flush_flags, false) ) + BUG(); + BUG_ON(!pt_mfn); + + clear_iommu_pte_present(pt_mfn, dfn_x(dfn), level, &free); + *flush_flags |= IOMMU_FLUSHF_all; + iommu_queue_free_pgtable(hd, pg); + } } spin_unlock(&hd->arch.mapping_lock);