From patchwork Fri Jan 20 22:46:21 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Elliot Berman X-Patchwork-Id: 13110649 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 7DE4AC05027 for ; Fri, 20 Jan 2023 23:02:42 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-ID:Date:Subject:CC:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=RHY0QTCOJUEoTk0p+jUBDX0OeIKBkKAnKfA/FL5SwVw=; b=Gv1Tw1zaC9zhnI EDWLpQr0MMMrXcGNaFdWthGJEF5ULT5rRbCqfS7tNrt/220f67GHkAdGz0DFglai4yrwNhjeDx4wU QgHH9q7VLOmBqUGUcmWAZx8Frp8BvHH8MKKT/y0xfgg9wQyBnDXg3EJjQQ2umpbTphp0dO25BmLLd Ed1OjGmdUT5Kf8hqASt0+6UAaYqBPU989Yg+VVyqT5V6eMmcsxBLLd7B3sQoS36xKiCOf/2eQ/BK8 ZA/IH8LBl55vEECVsrDQ5zAVPBdiCNY+3CLp0nXSA+L+IjOuJ3I/9mBeyGNlBA2l9VEegLHkWnOmA 9su2qT47Mu9zdVhw+ZhA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1pJ0Nu-00CdIa-LU; Fri, 20 Jan 2023 23:01:23 +0000 Received: from mx0a-0031df01.pphosted.com ([205.220.168.131]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1pJ0BD-00CXxN-HT for linux-arm-kernel@lists.infradead.org; Fri, 20 Jan 2023 22:48:20 +0000 Received: from pps.filterd (m0279862.ppops.net [127.0.0.1]) by mx0a-0031df01.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 30KMDFiY020834; Fri, 20 Jan 2023 22:48:05 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=qcppdkim1; bh=7HbBt3e0a1vAlc9eCKkIzVLn1j5GxGmq0P3U4Vw3h48=; b=ZN9RAm/VAbnLc4zWttEDFDOsbtzD6gDxw7/fXXnQFtwtuktwIxFlQ3T3+MCeeo3l8vsA 4qPXBwfCV5/58CSSGHF3dg9N0kSEgOIBP2bQVk3YMOGLHSH2R+YsT1pyzvR4YYsagHoM tW6cuKX2GXhwldS4bZpQMu9WGEuIf9YNGNJBxAikOrmTfkKWWM9eHzjRpm8C+METidgI eqCKysPkJtNygXf+tYMNrNfbIdQ6TOHkPbKSzESwf112nzwUFlJtTAEDw6ZgIqj9XwJO rEsZq0i8KsJpRIcAfFsSHbmWzfda7MuIiuZ5yy9Rl2oyPGO8oanmSIpzUNYqatad5dYq WQ== Received: from nasanppmta04.qualcomm.com (i-global254.qualcomm.com [199.106.103.254]) by mx0a-0031df01.pphosted.com (PPS) with ESMTPS id 3n81yq062k-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 20 Jan 2023 22:48:05 +0000 Received: from nasanex01b.na.qualcomm.com (nasanex01b.na.qualcomm.com [10.46.141.250]) by NASANPPMTA04.qualcomm.com (8.17.1.5/8.17.1.5) with ESMTPS id 30KMm4MA024424 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 20 Jan 2023 22:48:04 GMT Received: from hu-eberman-lv.qualcomm.com (10.49.16.6) by nasanex01b.na.qualcomm.com (10.46.141.250) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.36; Fri, 20 Jan 2023 14:48:03 -0800 From: Elliot Berman To: Bjorn Andersson , Alex Elder , Elliot Berman , Murali Nalajala CC: Trilok Soni , Srivatsa Vaddagiri , Carl van Schaik , Prakruthi Deepak Heragu , Dmitry Baryshkov , Arnd Bergmann , "Greg Kroah-Hartman" , Rob Herring , Krzysztof Kozlowski , Jonathan Corbet , Bagas Sanjaya , Catalin Marinas , Will Deacon , Marc Zyngier , Jassi Brar , Sudeep Holla , , , , , Subject: [PATCH v9 22/27] virt: gunyah: Add resource tickets Date: Fri, 20 Jan 2023 14:46:21 -0800 Message-ID: <20230120224627.4053418-23-quic_eberman@quicinc.com> X-Mailer: git-send-email 2.39.0 In-Reply-To: <20230120224627.4053418-1-quic_eberman@quicinc.com> References: <20230120224627.4053418-1-quic_eberman@quicinc.com> MIME-Version: 1.0 X-Originating-IP: [10.49.16.6] X-ClientProxiedBy: nalasex01c.na.qualcomm.com (10.47.97.35) To nasanex01b.na.qualcomm.com (10.46.141.250) X-QCInternal: smtphost X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-ORIG-GUID: 1B8-ZnTqIwdwSlHPVIzxct3lZ3qbYisu X-Proofpoint-GUID: 1B8-ZnTqIwdwSlHPVIzxct3lZ3qbYisu X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.219,Aquarius:18.0.930,Hydra:6.0.562,FMLib:17.11.122.1 definitions=2023-01-20_11,2023-01-20_01,2022-06-22_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 clxscore=1015 spamscore=0 priorityscore=1501 phishscore=0 suspectscore=0 lowpriorityscore=0 mlxlogscore=999 impostorscore=0 bulkscore=0 mlxscore=0 malwarescore=0 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2212070000 definitions=main-2301200218 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20230120_144815_939007_18147D50 X-CRM114-Status: GOOD ( 25.08 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Some VM functions need to acquire Gunyah resources. For instance, Gunyah vCPUs are exposed to the host as a resource. The Gunyah vCPU function will register a resource ticket and be able to interact with the hypervisor once the resource ticket is filled. Signed-off-by: Elliot Berman --- drivers/virt/gunyah/vm_mgr.c | 105 +++++++++++++++++++++++++++++++++- drivers/virt/gunyah/vm_mgr.h | 5 ++ include/linux/gunyah.h | 4 ++ include/linux/gunyah_vm_mgr.h | 14 +++++ 4 files changed, 127 insertions(+), 1 deletion(-) diff --git a/drivers/virt/gunyah/vm_mgr.c b/drivers/virt/gunyah/vm_mgr.c index 1e795f3d19d5..ee593cf1b074 100644 --- a/drivers/virt/gunyah/vm_mgr.c +++ b/drivers/virt/gunyah/vm_mgr.c @@ -135,6 +135,56 @@ static long gh_vm_rm_function(struct gunyah_vm *ghvm, struct gh_vm_function *fn) return 0; } +int ghvm_add_resource_ticket(struct gunyah_vm *ghvm, struct gunyah_vm_resource_ticket *ticket) +{ + struct gunyah_vm_resource_ticket *iter; + struct gunyah_resource *ghrsc; + int ret = 0; + + mutex_lock(&ghvm->resources_lock); + list_for_each_entry(iter, &ghvm->resource_tickets, list) { + if (iter->resource_type == ticket->resource_type && iter->label == ticket->label) { + ret = -EEXIST; + goto out; + } + } + + if (!try_module_get(ticket->owner)) { + ret = -ENODEV; + goto out; + } + + list_add(&ticket->list, &ghvm->resource_tickets); + INIT_LIST_HEAD(&ticket->resources); + + list_for_each_entry(ghrsc, &ghvm->resources, list) { + if (ghrsc->type == ticket->resource_type && ghrsc->rm_label == ticket->label) { + if (!ticket->populate(ticket, ghrsc)) + list_move(&ghrsc->list, &ticket->resources); + } + } +out: + mutex_unlock(&ghvm->resources_lock); + return ret; +} +EXPORT_SYMBOL_GPL(ghvm_add_resource_ticket); + +void ghvm_remove_resource_ticket(struct gunyah_vm *ghvm, struct gunyah_vm_resource_ticket *ticket) +{ + struct gunyah_resource *ghrsc, *iter; + + mutex_lock(&ghvm->resources_lock); + list_for_each_entry_safe(ghrsc, iter, &ticket->resources, list) { + ticket->unpopulate(ticket, ghrsc); + list_move(&ghrsc->list, &ghvm->resources); + } + + module_put(ticket->owner); + list_del(&ticket->list); + mutex_unlock(&ghvm->resources_lock); +} +EXPORT_SYMBOL_GPL(ghvm_remove_resource_ticket); + static void ghvm_put(struct kref *kref) { struct gunyah_vm *ghvm = container_of(kref, struct gunyah_vm, kref); @@ -180,17 +230,40 @@ static __must_check struct gunyah_vm *gunyah_vm_alloc(struct gh_rm *rm) INIT_LIST_HEAD(&ghvm->memory_mappings); init_rwsem(&ghvm->status_lock); kref_init(&ghvm->kref); + mutex_init(&ghvm->resources_lock); + INIT_LIST_HEAD(&ghvm->resources); + INIT_LIST_HEAD(&ghvm->resource_tickets); INIT_LIST_HEAD(&ghvm->functions); return ghvm; } +static void gh_vm_add_resource(struct gunyah_vm *ghvm, struct gunyah_resource *ghrsc) +{ + struct gunyah_vm_resource_ticket *ticket; + + mutex_lock(&ghvm->resources_lock); + list_for_each_entry(ticket, &ghvm->resource_tickets, list) { + if (ghrsc->type == ticket->resource_type && ghrsc->rm_label == ticket->label) { + if (!ticket->populate(ticket, ghrsc)) { + list_add(&ghrsc->list, &ticket->resources); + goto found; + } + } + } + list_add(&ghrsc->list, &ghvm->resources); +found: + mutex_unlock(&ghvm->resources_lock); +} + static int gh_vm_start(struct gunyah_vm *ghvm) { struct gunyah_vm_memory_mapping *mapping; u64 dtb_offset; u32 mem_handle; - int ret; + struct gunyah_resource *ghrsc; + struct gh_rm_hyp_resources *resources; + int ret, i; down_write(&ghvm->status_lock); if (ghvm->vm_status != GH_RM_VM_STATUS_NO_STATE) { @@ -241,6 +314,22 @@ static int gh_vm_start(struct gunyah_vm *ghvm) goto err; } + ret = gh_rm_get_hyp_resources(ghvm->rm, ghvm->vmid, &resources); + if (ret) { + pr_warn("Failed to get hypervisor resources for VM: %d\n", ret); + goto err; + } + + for (i = 0; i < le32_to_cpu(resources->n_entries); i++) { + ghrsc = gh_rm_alloc_resource(ghvm->rm, &resources->entries[i]); + if (!ghrsc) { + ret = -ENOMEM; + goto err; + } + + gh_vm_add_resource(ghvm, ghrsc); + } + ret = gh_rm_vm_start(ghvm->rm, ghvm->vmid); if (ret) { pr_warn("Failed to start VM: %d\n", ret); @@ -380,6 +469,8 @@ static int gh_vm_release(struct inode *inode, struct file *filp) struct gunyah_vm *ghvm = filp->private_data; struct gunyah_vm_memory_mapping *mapping, *tmp; struct gunyah_vm_function *f, *fiter; + struct gunyah_vm_resource_ticket *ticket, *titer; + struct gunyah_resource *ghrsc, *riter; gh_vm_stop(ghvm); @@ -394,6 +485,18 @@ static int gh_vm_release(struct inode *inode, struct file *filp) kfree(mapping); } + if (!list_empty(&ghvm->resource_tickets)) { + pr_warn("Dangling resource tickets:\n"); + list_for_each_entry_safe(ticket, titer, &ghvm->resource_tickets, list) { + pr_warn(" %pS\n", ticket->populate); + ghvm_remove_resource_ticket(ghvm, ticket); + } + } + + list_for_each_entry_safe(ghrsc, riter, &ghvm->resources, list) { + gh_rm_free_resource(ghrsc); + } + put_gh_rm(ghvm->rm); put_gunyah_vm(ghvm); return 0; diff --git a/drivers/virt/gunyah/vm_mgr.h b/drivers/virt/gunyah/vm_mgr.h index 8d3b0678fb96..f52350145010 100644 --- a/drivers/virt/gunyah/vm_mgr.h +++ b/drivers/virt/gunyah/vm_mgr.h @@ -7,6 +7,7 @@ #define _GH_PRIV_VM_MGR_H #include +#include #include #include #include @@ -47,6 +48,10 @@ struct gunyah_vm { struct mutex mm_lock; struct list_head memory_mappings; + struct mutex resources_lock; + struct list_head resources; + struct list_head resource_tickets; + struct list_head functions; }; diff --git a/include/linux/gunyah.h b/include/linux/gunyah.h index 91d2c9f5b2f0..d745916cabcc 100644 --- a/include/linux/gunyah.h +++ b/include/linux/gunyah.h @@ -27,6 +27,10 @@ struct gunyah_resource { enum gunyah_resource_type type; u64 capid; int irq; + + /* To help allocator of resource manager */ + struct list_head list; + u32 rm_label; }; /** diff --git a/include/linux/gunyah_vm_mgr.h b/include/linux/gunyah_vm_mgr.h index 69f98eb503e9..995a9b0eb6b7 100644 --- a/include/linux/gunyah_vm_mgr.h +++ b/include/linux/gunyah_vm_mgr.h @@ -65,4 +65,18 @@ void gunyah_vm_function_unregister(struct gunyah_vm_function_driver *f); } \ module_exit(_name##_mod_exit) +struct gunyah_vm_resource_ticket { + struct list_head list; + struct list_head resources; + enum gunyah_resource_type resource_type; + u32 label; + + struct module *owner; + int (*populate)(struct gunyah_vm_resource_ticket *ticket, struct gunyah_resource *ghrsc); + void (*unpopulate)(struct gunyah_vm_resource_ticket *ticket, struct gunyah_resource *ghrsc); +}; + +int ghvm_add_resource_ticket(struct gunyah_vm *ghvm, struct gunyah_vm_resource_ticket *ticket); +void ghvm_remove_resource_ticket(struct gunyah_vm *ghvm, struct gunyah_vm_resource_ticket *ticket); + #endif