From patchwork Tue Jul 5 12:44:26 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Jan Beulich X-Patchwork-Id: 12906549 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 7D8E7C433EF for ; Tue, 5 Jul 2022 12:44:45 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.361139.590475 (Exim 4.92) (envelope-from ) id 1o8huq-0002s2-Vj; Tue, 05 Jul 2022 12:44:32 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 361139.590475; Tue, 05 Jul 2022 12:44:32 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1o8huq-0002rt-St; Tue, 05 Jul 2022 12:44:32 +0000 Received: by outflank-mailman (input) for mailman id 361139; Tue, 05 Jul 2022 12:44:30 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1o8huo-0002H7-LV for xen-devel@lists.xenproject.org; Tue, 05 Jul 2022 12:44:30 +0000 Received: from EUR01-VE1-obe.outbound.protection.outlook.com (mail-eopbgr140058.outbound.protection.outlook.com [40.107.14.58]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id 35fa6309-fc60-11ec-a8e4-439420d8e422; Tue, 05 Jul 2022 14:44:29 +0200 (CEST) Received: from VE1PR04MB6560.eurprd04.prod.outlook.com (2603:10a6:803:122::25) by DB6PR0401MB2613.eurprd04.prod.outlook.com (2603:10a6:4:37::23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5395.21; Tue, 5 Jul 2022 12:44:28 +0000 Received: from VE1PR04MB6560.eurprd04.prod.outlook.com ([fe80::dfa:a64a:432f:e26b]) by VE1PR04MB6560.eurprd04.prod.outlook.com ([fe80::dfa:a64a:432f:e26b%7]) with mapi id 15.20.5395.021; Tue, 5 Jul 2022 12:44:28 +0000 X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 35fa6309-fc60-11ec-a8e4-439420d8e422 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=kL8LBiaAEBvf7qLD6agVUNSxaTLmrvXLZaej+LJWW3qFbNMkgF9h36ISRbMYC4iif0KhHR6B8pESs39SbVa7du3+EAAgpA+devYBut8AfLQzhVmkh15NijePUi7dPLzYjs5EZApaGrlkb/J9x7Hf4SBL3WEmlIT1LyHv+HtTywPkI4AqPyKk19X0yy0v6HyIGYwVEXjw7NTW4F6KswyAreCIoaZoKgRUArupkdT/s91xqNm+9iHs+sM1wBT15NbIA+qjBVVJHMp6W+BaklMXc/Q0IclL8f9RRe9uubm1ZfI8cm/RBMe1CfNkNmH5uaJniPDM7Lnu/255Uk2ktI+LRA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=tz6ozk9S/AhujlkpGh0jwCZXiL+OVP402DUlNiNYn5k=; b=RtJiwQGL9MCjvyeyUHPnBI6SCug4Vvpc6W0yvPQWdG4oIjgjkzxLsFf22Lu7v517BFqAPi0I1LOy+EbazHfkX6YvsAzBudz8Em4MQZpHgaiA5kbr5I7cac4Di9R52jGy+6S3S/966rOvpRPHd4kPuH+xn0x7hcFJR2OL0QuQBpMoQECHAW37TMp62CGway32WsRIvO9d8+ThX3goe7cS4t/xJjz0jTfnd6sibhKopAPjWmdAs/DTJsRNUg5oM3jY2qiSbFJvDZ9zzvp1pBmNRwSAmWlApBAYGwUuqmZ6b87enQeBlw5TC07BZiXo0tQifS9KPVGkv71qKa0VvUXS3A== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com; dkim=pass header.d=suse.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=tz6ozk9S/AhujlkpGh0jwCZXiL+OVP402DUlNiNYn5k=; b=FeIksqwS0FJlZsZx6jYdT7bMGr/qfRUWBYrjmngzaUI5d9Sx3McvBbVqh/wF2Km1imrisK8fCKYcIQrOGXOajUzJ+wk6JTtF/W841tbeQud+R2TYpeDhSgSoEHGdGowl1YQNYEFPFmePQ5sBxsBKR+YLqSRIaxGUBnv9vIAORCzYpExK9Ewzz/y7hUdaG7yZyP0wSFp8JTnbobImr8p3qV71uCn2wea924xGVltovkM9SNmNmUvFwIcCt5kBqwl/7P5WXfneD2h8Q+L8H3hFXO9z2OQkkK0ncAuKqQKCrIdVdkHY78bhCNN+nZ6pl3wx4p1vQGSq2jkmcXP9kkvC4Q== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=suse.com; Message-ID: Date: Tue, 5 Jul 2022 14:44:26 +0200 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:91.0) Gecko/20100101 Thunderbird/91.11.0 Subject: [PATCH v7 03/14] IOMMU/x86: support freeing of pagetables Content-Language: en-US From: Jan Beulich To: "xen-devel@lists.xenproject.org" Cc: Andrew Cooper , Paul Durrant , =?utf-8?q?Roger_Pau_Monn=C3=A9?= References: <4c9d4b32-d4db-3a4f-fe92-6022e63c3f82@suse.com> In-Reply-To: <4c9d4b32-d4db-3a4f-fe92-6022e63c3f82@suse.com> X-ClientProxiedBy: AS8PR07CA0058.eurprd07.prod.outlook.com (2603:10a6:20b:459::15) To VE1PR04MB6560.eurprd04.prod.outlook.com (2603:10a6:803:122::25) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 45afa46d-ca31-40b4-164e-08da5e841954 X-MS-TrafficTypeDiagnostic: DB6PR0401MB2613:EE_ X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: YBgGshiM5C7vrHVxGj9SyG2zYY46XA2p++bdqO5mwSmRLOgTJUOhm3BVCxOv2O+45ezR/2kNGwkQuWyp40Z3LogBx3JdS8MxkRWL+FHRS6BxIdcMxZKnBwkq7jVW7hctPKh43dCmkF1vDymp/p93pRXZpZBXJQ4rMRXBiheM4IWiT2ZqVxEh4VkmVFXB58ywAAXioFt+xM5c88YRT/i9dpiyDaqu3e7jxOyF3rJ+KkNOBtnGd+nka15V/BQ+E8xu41gsQBqvWiTcouWwbiryXVCA93fWelaMVENUOA27y8rESqCg9QurSD5gcgIOQchT+6rAZJ7xv7HnhOSSGXJ11w+ryIzfoy7fUAgs/ejXHxUG6nxqkS8eQ7W9GmNznmTSVmHFcnPpnRLAAdPPaclqHeaP7duv51KzS+IKg7wKUJppd4p9GbhII8w76LMjvBmZ9aPP0GGU0q9el44jwfHH9A0yjOu5TaOLdGb5guUJbj1ZTlm3YytSzEqNj5l6/tCTdUndHMPc5B8kIJBnQHFnNMyXPTRPEd+ljDWIsppx0FlXwh/DpXc5FLO5O4fOKdwfMQ3rYTo+lsN0eeHRQiARbErZ/4xI67i/lierYdOGRC67tyaGWZiEs6OM0rE0qRpqwrA/lWaLsyIILw/a6+0KThwQGFsX1wfvxDGPu2Cqv0F3tYtf2exNHGlkdgUM1ilq/9CxZ/K8khbcV3rtpfuKmpH/yCriJk8osif/YW6YPHPbscNtFPIN/lXfIFOu54htv/ATwWMm4a69CVddt5JPQVr7weyRks302g01fdhvbsMaBdyjsJXDFfgl38MrzueiQ+jjcPB3MU+lH/uIBfF/rdpZXyAj7ych1mMFauHPwrEDTSGNWaPFnoWTpp5HsLnp X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VE1PR04MB6560.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230016)(136003)(376002)(396003)(366004)(39860400002)(346002)(8936002)(31696002)(38100700002)(6486002)(478600001)(86362001)(66946007)(4326008)(6512007)(8676002)(66476007)(66556008)(5660300002)(2616005)(26005)(186003)(36756003)(316002)(6916009)(31686004)(54906003)(6506007)(41300700001)(83380400001)(2906002)(45980500001)(43740500002);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?q?8GzijYfJLGHKCmhElvGtp9Tf/TGb?= =?utf-8?q?HM0ft3MfGBSh9NLOrgPdhwbeJJnWaFgHDU6u5i/4XKKAnMtFLMUvzS9AVlpSbSrdQ?= =?utf-8?q?Gs9lfbhkT693Mwmsl5/yFqbYGQLCYg3rXK6cQB0zppIdmQd3kmjj3fYraLew0UeBv?= =?utf-8?q?JRTWM82OxDvo9iG1m0+B0cRQC2P6urb/a+0cXygcJAbqaGDMHOKyHdwV7Pg1zzhRp?= =?utf-8?q?UAiUDczsm+ZAxVI4Rd8qW5+kheyQj0GqB2/w+KXhnyW7q1MZV6GMBJwCJQ8O0/NY6?= =?utf-8?q?RzdCLZxdsMc5dwBrRQDZNSZBCBj20wm+t4uCWmmsdHo6ZyDKTByjK30uv0OIykWRw?= =?utf-8?q?0viu3oEhYdAbqCk0V2nZWjCqkMoAyLMbM/N7NdEEv0nfUhGra0MZJTo/a/N3XlZG/?= =?utf-8?q?EHyC9A0EzhzVrUMqRV9rUj/ip2hDqqcmnPBAlo1liRYRJeeelm9egj+i0Bue/5CVv?= =?utf-8?q?kDbxZKdd/0bRe1cor2Qg1fe84gzxKGk/aT9pFhhIwR3gplKIucpYfKWM9kcXJ0Oz1?= =?utf-8?q?ML8QHzx7rLbsdFLf5p6VCypt977fzQmmbVttCQiOqtxMEUkymi2KRCt6oBzVstXWK?= =?utf-8?q?AQp9a6u6pehCGV7C/jI0o25zkJY+QmHZLZmzpnTjzCnSqMcdk0PjEfNoU22bSqBwn?= =?utf-8?q?OJ6Ip1ZPxbmWgTdkyJMW8C5iq4XzOcOAHhKJCHBGeC8UL6raKUfuTwpFCxT6hPvvs?= =?utf-8?q?XK9TKQNjdW40OVyOGSEhAO1zgjGFTlq465if/gPNuhfsTLMKjn52y7duyptVuGa1Z?= =?utf-8?q?VbD2Ullpa/f2O8ZkiX4hMyFe8RuJiMLLFQVAHK+wWB8KA40VeNgFOce/iFlWSO2oR?= =?utf-8?q?jc9s0IisinfvLfidtjRJS67wwigAyIyFNzQGz3ZcYixXoNjEVkIXwsahcEoGMD6qu?= =?utf-8?q?OwWZXTVayarEwU7mOyeUHJl4HDg4szSuZ1CWgwl6dVC3NCImBSRkurF16Iw8GcyNW?= =?utf-8?q?Jbhs5jHSPVFjkqJLNggnGfv5ynLYFN5ElOmKTirTNb6NI9CJKrumokkrFZtSuZcq7?= =?utf-8?q?7kmiUFP4qooI/v1OG5aoDi9KJysMeD8DefydUaInls/BYQyUwlOl8XtooPyt+iciw?= =?utf-8?q?s2l2y2HKB7FUHSozVZQcWjopP8cTUrhN9JaBSPPyqo8nYzfxNmWMxO78uqGIdSsed?= =?utf-8?q?hZH//ai78hin2sYPA2I8zwajMKdhfbSN8lcOui8wusfOYHrB3ry2oCX/YsCep6Ql6?= =?utf-8?q?jjBovc6BNAwmkGpSRiri8Q2V5pwMlSBlV1wyCRXzC+hS8M4Bqhsj1AZDlnvdHlFlD?= =?utf-8?q?9rZLsUiVGj1+LUWsVnICSA9j3I6rCBbpvP4Fb+aeoqtksnP2plo8xTlETebjKLy3U?= =?utf-8?q?/Pj8I76PjbveFdNcx4sKiABcVLnam7380rCWSJpV/jfyfDAjZMzvm775RWXsXfbqf?= =?utf-8?q?b4xevWFHSTgBE5HigUu7u07PIpir67Yt4DWtigF3bDAwDI11Fy7CBPIGQNpybpal6?= =?utf-8?q?WJy758u15RjPxreklH+jePLM3/4L8TBogJS1UDyp5iyZoydh/m5GCLE9jaeTHOQZY?= =?utf-8?q?m/6HK0Aeov2q?= X-OriginatorOrg: suse.com X-MS-Exchange-CrossTenant-Network-Message-Id: 45afa46d-ca31-40b4-164e-08da5e841954 X-MS-Exchange-CrossTenant-AuthSource: VE1PR04MB6560.eurprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 05 Jul 2022 12:44:28.1387 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: yED7xdMjgOAjActLXaU6gROgDvYZs4R4SKA/m5jVvFlV8sD5B1WBxM5n1mcrSqNA0MwgyoY22v+OvtKweJ7W6A== X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB6PR0401MB2613 For vendor specific code to support superpages we need to be able to deal with a superpage mapping replacing an intermediate page table (or hierarchy thereof). Consequently an iommu_alloc_pgtable() counterpart is needed to free individual page tables while a domain is still alive. Since the freeing needs to be deferred until after a suitable IOTLB flush was performed, released page tables get queued for processing by a tasklet. Signed-off-by: Jan Beulich Reviewed-by: Roger Pau Monné Reviewed-by: Paul Durrant --- I was considering whether to use a softirq-tasklet instead. This would have the benefit of avoiding extra scheduling operations, but come with the risk of the freeing happening prematurely because of a process_pending_softirqs() somewhere. --- v7: Add cf_check to free_queued_pgtables(). Add ASSERT(). v6: Extend comment on the use of process_pending_softirqs(). v5: Fix CPU_UP_PREPARE for BIGMEM. Schedule tasklet in CPU_DOWN_FAILED when list is not empty. Skip all processing in CPU_DEAD when list is empty. v4: Change type of iommu_queue_free_pgtable()'s 1st parameter. Re-base. v3: Call process_pending_softirqs() from free_queued_pgtables(). --- a/xen/arch/x86/include/asm/iommu.h +++ b/xen/arch/x86/include/asm/iommu.h @@ -147,6 +147,7 @@ void iommu_free_domid(domid_t domid, uns int __must_check iommu_free_pgtables(struct domain *d); struct domain_iommu; struct page_info *__must_check iommu_alloc_pgtable(struct domain_iommu *hd); +void iommu_queue_free_pgtable(struct domain_iommu *hd, struct page_info *pg); #endif /* !__ARCH_X86_IOMMU_H__ */ /* --- a/xen/drivers/passthrough/x86/iommu.c +++ b/xen/drivers/passthrough/x86/iommu.c @@ -12,6 +12,7 @@ * this program; If not, see . */ +#include #include #include #include @@ -556,6 +557,105 @@ struct page_info *iommu_alloc_pgtable(st return pg; } +/* + * Intermediate page tables which get replaced by large pages may only be + * freed after a suitable IOTLB flush. Hence such pages get queued on a + * per-CPU list, with a per-CPU tasklet processing the list on the assumption + * that the necessary IOTLB flush will have occurred by the time tasklets get + * to run. (List and tasklet being per-CPU has the benefit of accesses not + * requiring any locking.) + */ +static DEFINE_PER_CPU(struct page_list_head, free_pgt_list); +static DEFINE_PER_CPU(struct tasklet, free_pgt_tasklet); + +static void cf_check free_queued_pgtables(void *arg) +{ + struct page_list_head *list = arg; + struct page_info *pg; + unsigned int done = 0; + + ASSERT(list == &this_cpu(free_pgt_list)); + + while ( (pg = page_list_remove_head(list)) ) + { + free_domheap_page(pg); + + /* + * Just to be on the safe side, check for processing softirqs every + * once in a while. Generally it is expected that parties queuing + * pages for freeing will find a need for preemption before too many + * pages can be queued. Granularity of checking is somewhat arbitrary. + */ + if ( !(++done & 0x1ff) ) + process_pending_softirqs(); + } +} + +void iommu_queue_free_pgtable(struct domain_iommu *hd, struct page_info *pg) +{ + unsigned int cpu = smp_processor_id(); + + spin_lock(&hd->arch.pgtables.lock); + page_list_del(pg, &hd->arch.pgtables.list); + spin_unlock(&hd->arch.pgtables.lock); + + page_list_add_tail(pg, &per_cpu(free_pgt_list, cpu)); + + tasklet_schedule(&per_cpu(free_pgt_tasklet, cpu)); +} + +static int cf_check cpu_callback( + struct notifier_block *nfb, unsigned long action, void *hcpu) +{ + unsigned int cpu = (unsigned long)hcpu; + struct page_list_head *list = &per_cpu(free_pgt_list, cpu); + struct tasklet *tasklet = &per_cpu(free_pgt_tasklet, cpu); + + switch ( action ) + { + case CPU_DOWN_PREPARE: + tasklet_kill(tasklet); + break; + + case CPU_DEAD: + if ( !page_list_empty(list) ) + { + page_list_splice(list, &this_cpu(free_pgt_list)); + INIT_PAGE_LIST_HEAD(list); + tasklet_schedule(&this_cpu(free_pgt_tasklet)); + } + break; + + case CPU_UP_PREPARE: + INIT_PAGE_LIST_HEAD(list); + fallthrough; + case CPU_DOWN_FAILED: + tasklet_init(tasklet, free_queued_pgtables, list); + if ( !page_list_empty(list) ) + tasklet_schedule(tasklet); + break; + } + + return NOTIFY_DONE; +} + +static struct notifier_block cpu_nfb = { + .notifier_call = cpu_callback, +}; + +static int __init cf_check bsp_init(void) +{ + if ( iommu_enabled ) + { + cpu_callback(&cpu_nfb, CPU_UP_PREPARE, + (void *)(unsigned long)smp_processor_id()); + register_cpu_notifier(&cpu_nfb); + } + + return 0; +} +presmp_initcall(bsp_init); + bool arch_iommu_use_permitted(const struct domain *d) { /*