From patchwork Wed Feb 23 23:37:19 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Elliot Berman X-Patchwork-Id: 12757666 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 34D47C433FE for ; Wed, 23 Feb 2022 23:38:03 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S244880AbiBWXi3 (ORCPT ); Wed, 23 Feb 2022 18:38:29 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58872 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S244877AbiBWXi1 (ORCPT ); Wed, 23 Feb 2022 18:38:27 -0500 Received: from alexa-out-sd-02.qualcomm.com (alexa-out-sd-02.qualcomm.com [199.106.114.39]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id BEA8B5A580; Wed, 23 Feb 2022 15:37:55 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; i=@quicinc.com; q=dns/txt; s=qcdkim; t=1645659475; x=1677195475; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=q6I2iTlgYSdcCh5IbDuMtO1zYfoBSNPNnrKA7SE3n7E=; b=f9AQlyRP2Vt6sL5IoyQN05i3GDEGnsCgktSWpgf58HsaO1SySGXvKKs/ tFrAMB1ognO/gSwnhbOnm44X3CmSXvLSCOxRr3pM4zb0+C5XPNlTmouGn +zbINDsWbRKe1cbMGe7EmQ9bxb08yzen90h/syPtsbnvAXugTuIDNe64I w=; Received: from unknown (HELO ironmsg01-sd.qualcomm.com) ([10.53.140.141]) by alexa-out-sd-02.qualcomm.com with ESMTP; 23 Feb 2022 15:37:55 -0800 X-QCInternal: smtphost Received: from nasanex01b.na.qualcomm.com ([10.46.141.250]) by ironmsg01-sd.qualcomm.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 23 Feb 2022 15:37:55 -0800 Received: from hu-eberman-lv.qualcomm.com (10.49.16.6) by nasanex01b.na.qualcomm.com (10.46.141.250) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.15; Wed, 23 Feb 2022 15:37:54 -0800 From: Elliot Berman To: Bjorn Andersson , , , Jonathan Corbet CC: Elliot Berman , Trilok Soni , Murali Nalajala , Srivatsa Vaddagiri , Carl van Schaik , Andy Gross , Subject: [PATCH 01/11] docs: gunyah: Introduce Gunyah Hypervisor Date: Wed, 23 Feb 2022 15:37:19 -0800 Message-ID: <20220223233729.1571114-2-quic_eberman@quicinc.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220223233729.1571114-1-quic_eberman@quicinc.com> References: <20220223233729.1571114-1-quic_eberman@quicinc.com> MIME-Version: 1.0 X-Originating-IP: [10.49.16.6] X-ClientProxiedBy: nalasex01a.na.qualcomm.com (10.47.209.196) To nasanex01b.na.qualcomm.com (10.46.141.250) Precedence: bulk List-ID: X-Mailing-List: linux-arm-msm@vger.kernel.org Gunyah is an open-source Type-1 hypervisor developed by Qualcomm. It does not depend on any lower-privileged OS/kernel code for its core functionality. This increases its security and can support a smaller trusted computing based when compared to Type-2 hypervisors. Add documentation describing the Gunyah hypervisor and the main components of the Gunyah hypervisor which are of interest to Linux virtualization development. Signed-off-by: Elliot Berman --- Documentation/virt/gunyah/index.rst | 92 +++++++++++++++++++++ Documentation/virt/gunyah/message-queue.rst | 52 ++++++++++++ Documentation/virt/index.rst | 1 + MAINTAINERS | 7 ++ 4 files changed, 152 insertions(+) create mode 100644 Documentation/virt/gunyah/index.rst create mode 100644 Documentation/virt/gunyah/message-queue.rst diff --git a/Documentation/virt/gunyah/index.rst b/Documentation/virt/gunyah/index.rst new file mode 100644 index 000000000000..e7bb2b14543e --- /dev/null +++ b/Documentation/virt/gunyah/index.rst @@ -0,0 +1,92 @@ +.. SPDX-License-Identifier: GPL-2.0 + +================= +Gunyah Hypervisor +================= + +.. toctree:: + :maxdepth: 1 + + message-queue + +Gunyah is a Type-1 hypervisor which is independent of any OS kernel, and runs in +a higher CPU privilege level. It does not depend on any lower-privileged operating system +for its core functionality. This increases its security and can support a much smaller +trusted computing base than a Type-2 hypervisor. + +Gunyah is an open source hypervisor. The source repo is available at +https://github.com/quic/gunyah-hypervisor. + +Gunyah provides these following features. + +- Scheduling: + A scheduler for virtual CPUs (VCPUs) on physical CPUs and enables time-sharing + of the CPUs. +- Memory Management: + APIs handling memory, abstracted as objects, limiting direct use of physical + addresses. Memory ownership and usage tracking of all memory under its control. + Memory partitioning between VMs is a fundamental security feature. +- Interrupt Virtualization: + Uses CPU hardware interrupt virtualization capabilities. Interrupts are handled + in the hypervisor and routed to the assigned VM. +- Inter-VM Communication: + There are several different mechanisms provided for communicating between VMs. +- Virtual platform: + Architectural devices such as interrupt controllers and CPU timers are directly provided + by the hypervisor as well as core virtual platform devices and system APIs such as ARM PSCI. +- Device Virtualization: + Para-virtualization of devices is supported using inter-VM communication. + +Architectures supported +======================= +AArch64 with a GIC + +Resources and Capabilities +========================== + +Some services or resources provided by the Gunyah hypervisor are described by capability IDs. +For instance, inter-VM communication is performed with doorbells and message queues. The specific +instance of a doorbell is described by a capability ID. These devices are described in Linux as a +struct gunyah_device. + +High level management of these resources is performed by the resource manager VM. RM informs a +guest VM about resources it can access through either the device tree or via guest-initiated RPC. + +Resource Manager +================ + +The resource manager (RM) is a privileged application VM supporting the Gunyah Hypervisor. +It provides policy enforcement aspects of the virtualization system. The resource manager can +be treated as an extension of the Hypervisor but is separated to its own partition to ensure +that the hypervisor layer itself remains small and secure and to maintain a separation of policy +and mechanism in the platform. On arm64, RM runs at NS-EL1 similar to other virtual machines. + +Communication with the resource manager from each guest VM happens with message-queue.rst. Details +about the specific messages can be found in drivers/virt/gunyah/rsc_mgr.c + +:: + +-------+ +--------+ +--------+ + | RM | | VM_A | | VM_B | + +-.-.-.-+ +---.----+ +---.----+ + | | | | + +-.-.-----------.------------.----+ + | | \==========/ | | + | \========================/ | + | Gunyah | + +---------------------------------+ + +The source for the resource manager is available at https://github.com/quic/gunyah-resource-manager. + +The resource manager provides the following features: + +- Generate device-tree overlay +- VM creation and deletion +- VM device-tree management +- VM access control policy +- Interrupt routing configuration + +When booting a virtual machine which uses a devicetree, resource manager overlays a +/hypervisor node. This node can let Linux know it is running as a Gunyah guest VM, +how to communicate with resource manager, and basic description and capabilities of +this VM. See Documentation/devicetree/bindings/gunyah/qcom,hypervisor.yml for a description +of this node. diff --git a/Documentation/virt/gunyah/message-queue.rst b/Documentation/virt/gunyah/message-queue.rst new file mode 100644 index 000000000000..afd405f3a5e1 --- /dev/null +++ b/Documentation/virt/gunyah/message-queue.rst @@ -0,0 +1,52 @@ +.. SPDX-License-Identifier: GPL-2.0 + +Message Queues +============== +Message queue is a simple low-capacity IPC channel between two VMs. It is +intended for sending small control and configuration messages. Each message +queue object is unidirectional, so a full-duplex IPC channel requires a pair of +objects. + +Messages can be up to 240 bytes in length. Longer messages require a further +protocol on top of the message queue messages themselves. For instance, communication +with the resource manager adds a header field for sending longer messages via multiple +message fragments. + +The diagram below shows how message queue works. A typical configuration involves +2 message queues. Message queue 1 allows VM_A to send messages to VM_B. Message +queue 2 allows VM_B to send messages to VM_A. + +1. VM_A sends a message of up to 240 bytes in length. It raises a hypercall + with the message to inform the hypervisor to add the message to + message queue 1's queue. +2. Gunyah raises the corresponding interrupt for VM_B when any of these happens: + a. gh_msgq_send has PUSH flag. Queue is immediately flushed. This is the typical case. + b. Explicility with gh_msgq_push command from VM_A. + c. Message queue has reached a threshold depth. +3. VM_B calls gh_msgq_recv and Gunyah copies message to requested buffer. + +For VM_B to send a message to VM_A, the process is identical, except that hypercalls +reference message queue 2's capability ID. + +:: + + +---------------+ +-----------------+ +---------------+ + | VM_A | |Gunyah hypervisor| | VM_B | + | | | | | | + | | | | | | + | | Tx | | | | + | |-------->| | Rx vIRQ | | + |gh_msgq_send() | Tx vIRQ |Message queue 1 |-------->|gh_msgq_recv() | + | |<------- | | | | + | | | | | | + | Message Queue | | | | Message Queue | + | driver | | | | driver | + | | | | | | + | | | | | | + | | | | Tx | | + | | Rx vIRQ | |<--------| | + |gh_msgq_recv() |<--------|Message queue 2 | Tx vIRQ |gh_msgq_send() | + | | | |-------->| | + | | | | | | + | | | | | | + +---------------+ +-----------------+ +---------------+ diff --git a/Documentation/virt/index.rst b/Documentation/virt/index.rst index edea7fea95a8..4080e7f5cad8 100644 --- a/Documentation/virt/index.rst +++ b/Documentation/virt/index.rst @@ -13,6 +13,7 @@ Linux Virtualization Support guest-halt-polling ne_overview acrn/index + gunyah/index .. only:: html and subproject diff --git a/MAINTAINERS b/MAINTAINERS index 777cd6fa2b3d..bed175adc4c3 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -8395,6 +8395,13 @@ L: linux-efi@vger.kernel.org S: Maintained F: block/partitions/efi.* +GUNYAH HYPERVISOR DRIVER +M: Elliot Berman +M: Murali Nalajala +L: linux-arm-msm@vger.kernel.org +S: Maintained +F: Documentation/virt/gunyah/ + H8/300 ARCHITECTURE M: Yoshinori Sato L: uclinux-h8-devel@lists.sourceforge.jp (moderated for non-subscribers) From patchwork Wed Feb 23 23:37:20 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Elliot Berman X-Patchwork-Id: 12757667 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 079EEC433EF for ; Wed, 23 Feb 2022 23:38:05 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S244896AbiBWXia (ORCPT ); Wed, 23 Feb 2022 18:38:30 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58874 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S244878AbiBWXi1 (ORCPT ); Wed, 23 Feb 2022 18:38:27 -0500 Received: from alexa-out-sd-01.qualcomm.com (alexa-out-sd-01.qualcomm.com [199.106.114.38]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 24E0A5A589; Wed, 23 Feb 2022 15:37:56 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; i=@quicinc.com; q=dns/txt; s=qcdkim; t=1645659476; x=1677195476; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=+MKLa2yx3nWTCH8/dR6dYi3DXpTMspvp8pf1dvGQZKQ=; b=BxT4wK+FRMFIICKFGVQQTIT9V2dermF6KI2W939JQZ/uU1GHSAoKXMow gBWu/gcBP7yxpvr9pXF7+wZQxqC3/raDURV5AitTNTbgpmK+hW7srEWKi 63iNYhtMdDpCjW/grMeTzN2dH7SoiCQQdw7sY7U14hih4HDTxc0DkRL1m 8=; Received: from unknown (HELO ironmsg03-sd.qualcomm.com) ([10.53.140.143]) by alexa-out-sd-01.qualcomm.com with ESMTP; 23 Feb 2022 15:37:55 -0800 X-QCInternal: smtphost Received: from nasanex01b.na.qualcomm.com ([10.46.141.250]) by ironmsg03-sd.qualcomm.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 23 Feb 2022 15:37:55 -0800 Received: from hu-eberman-lv.qualcomm.com (10.49.16.6) by nasanex01b.na.qualcomm.com (10.46.141.250) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.15; Wed, 23 Feb 2022 15:37:54 -0800 From: Elliot Berman To: Bjorn Andersson , , Rob Herring , CC: Elliot Berman , Trilok Soni , Murali Nalajala , Srivatsa Vaddagiri , Carl van Schaik , Andy Gross , Subject: [PATCH 02/11] dt-bindings: Add binding for gunyah hypervisor Date: Wed, 23 Feb 2022 15:37:20 -0800 Message-ID: <20220223233729.1571114-3-quic_eberman@quicinc.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220223233729.1571114-1-quic_eberman@quicinc.com> References: <20220223233729.1571114-1-quic_eberman@quicinc.com> MIME-Version: 1.0 X-Originating-IP: [10.49.16.6] X-ClientProxiedBy: nalasex01a.na.qualcomm.com (10.47.209.196) To nasanex01b.na.qualcomm.com (10.46.141.250) Precedence: bulk List-ID: X-Mailing-List: linux-arm-msm@vger.kernel.org When Linux is booted as a guest under the Gunyah hypervisor, Gunyah applies a devicetree overlay describing the virtual platform configuration of the guest VM, such as the message queue capability IDs for communicating with the Resource Manager. Add the DT bindings that Gunyah adheres for the hypervisor node and message queues. Signed-off-by: Elliot Berman --- .../bindings/gunyah/message-queue.yml | 100 ++++++++++++++ .../bindings/gunyah/qcom,hypervisor.yml | 122 ++++++++++++++++++ MAINTAINERS | 1 + 3 files changed, 223 insertions(+) create mode 100644 Documentation/devicetree/bindings/gunyah/message-queue.yml create mode 100644 Documentation/devicetree/bindings/gunyah/qcom,hypervisor.yml diff --git a/Documentation/devicetree/bindings/gunyah/message-queue.yml b/Documentation/devicetree/bindings/gunyah/message-queue.yml new file mode 100644 index 000000000000..1a96d3de2a19 --- /dev/null +++ b/Documentation/devicetree/bindings/gunyah/message-queue.yml @@ -0,0 +1,100 @@ +# SPDX-License-Identifier: (GPL-2.0 OR BSD-2-Clause) +%YAML 1.2 +--- +$id: http://devicetree.org/schemas/gunyah/qcom,hypervisor.yaml# +$schema: http://devicetree.org/meta-schemas/core.yaml# + +title: Gunyah message queue + +maintainers: + - Murali Nalajala + - Elliot Berman + +properties: + compatible: + items: + - const: qcom,gunyah-message-queue + - const: qcom,gunyah-capability + peer: + description: VMID of the VM on the other end of message queue + $ref: /schemas/types.yaml#/definitions/uint32 + allOf: + - if: + anyOf: + - properties: + qcom,is-sender: true + - properties: + qcom,is-full-duplex: true + then: + properties: + qcom,tx-message-size: + description: Maximum size in bytes of a message which can be sent by this queue + $ref: /schemas/types.yaml#/definitions/int32 + qcom,tx-queue-depth: + description: Depth of transmit queue for messages sent by this queue + $ref: /schemas/types.yaml#/definitions/int32 + - if: + anyOf: + - properties: + qcom,is-receiver: true + - properties: + qcom,is-full-duplex: true + then: + properties: + qcom,rx-message-size: + description: Maximum size in bytes of a message which can be received by this queue + $ref: /schemas/types.yaml#/definitions/int32 + qcom,rx-queue-depth: + description: Depth of transmit queue for messages received by this queue + $ref: /schemas/types.yaml#/definitions/int32 + - if: + anyOf: + - properties: + qcom,is-receiver: true + - properties: + qcom,is-sender: true + then: + properties: + reg: + description: Hypervisor capability ID of the message queue + $ref: /schemas/types.yaml#/definitions/uint32 + minItems: 1 + maxItems: 1 + interrupts: + minItems: 1 + maxItems: 1 + - if: + properties: + qcom,is-full-duplex: true + then: + properties: + reg: + description: + Hypervisor capability IDs of the message queue + The first is tx side, the second is rx side + $ref: /schemas/types.yaml#/definitions/uint32 + minItems: 2 + maxItems: 2 + interrupts: + description: The first is tx interrupt, second is rx interrupt + minItems: 2 + maxItems: 2 + required: + - compatible + - reg + - interrupts + + +examples: + - | + display-msgq-pair@abbf0da3c3c965cc { + compatible = "qcom,gunyah-message-queue", "qcom,gunyah-capability"; + interrupts = , /* TX full IRQ */ + ; /* RX empty IRQ */ + reg = <0x00000000 0x00000000>, <0x00000000 0x00000001>; /* TX, RX cap ids */ + qcom,is-full-duplex; + qcom,tx-queue-depth = <8>; + qcom,tx-message-size = <0xf0>; + qcom,rx-queue-depth = <8>; + qcom,rx-message-size = <0xf0>; + }; \ No newline at end of file diff --git a/Documentation/devicetree/bindings/gunyah/qcom,hypervisor.yml b/Documentation/devicetree/bindings/gunyah/qcom,hypervisor.yml new file mode 100644 index 000000000000..f637d51c52f0 --- /dev/null +++ b/Documentation/devicetree/bindings/gunyah/qcom,hypervisor.yml @@ -0,0 +1,122 @@ +# SPDX-License-Identifier: (GPL-2.0 OR BSD-2-Clause) +%YAML 1.2 +--- +$id: http://devicetree.org/schemas/gunyah/qcom,hypervisor.yaml# +$schema: http://devicetree.org/meta-schemas/core.yaml# + +title: Hypervisor node to define virtual devices and other services provided by a Gunyah hypervisor + to this virtual machine. + +maintainers: + - Murali Nalajala + - Elliot Berman + +description: |+ + On systems which support devicetree, Gunyah generates and overlays a deviceetree overlay which + describes the basic configuration of the hypervisor. Virtual machines use this information for + initial discovery that they are running as a Gunyah guest VM. + See also: https://github.com/quic/gunyah-resource-manager/blob/develop/src/vm_creation/dto_construct.c + +properties: + compatible: + oneOf: + - items: + - const: qcom,gunyah-hypervisor-1.0 + - const: qcom,gunyah-hypervisor + + "#address-cells": + description: Number of cells needed to represent 64-bit capability IDs. + const: 2 + "#size-cells": + description: must be 0, because capability IDs are not memory address + ranges and do not have a size. + const: 0 + + qcom,gunyah-vm: + type: object + description: + The VM Identification is a virtual node that conveys to the VM information + about this virtual machine in the context of the hypervisor-based system + properties: + compatible: + oneOf: + - items: + - const: qcom,gunyah-vm-id-1.0 + - const: qcom,gunyah-vm-id + qcom,vendor: + $ref: /schemas/types.yaml#/definitions/string + description: Vendor of the Virtual Machine, e.g. Qualcomm + qcom,vmid: + $ref: /schemas/types.yaml#/definitions/uint32 + description: contains the VMID of this VM as a 32-bit value + qcom,owner-vmid: + $ref: /schemas/types.yaml#/definitions/uint32 + description: Contains the hypervisor VMID of the VM's owner. The owner + is the VM that allocated and created the VM. VMs directly + managed by the resource manager, such as the primary VM do + not have an owner. + required: + - compatible + - qcom,vmid + - qcom,owner-vmid + +patternProperties: + "^qcom,resource-manager-rpc(@.*)?": + type: object + description: + Resource Manager node which is required to communicate to Resource + Manager VM using Gunyah Message Queues. + allOf: "message-queue.yml#" + + properties: + compatible: + oneOf: + items: + - const: qcom,resource-manager-1-0 + - const: qcom,resource-manager + qcom,console-dev: + $ref: /schemas/types.yaml#/definitions/flag + description: if set, the resource-manger will accept console logs from the VM + qcom,free-irq-start: + $ref: /schemas/types.yaml#/definitions/uint32 + description: Set on ARM systems which use a GIC. First VIRQ number which is free + for virtual interrupt use. + required: + - qcom,is-full-duplex + + +required: +- compatible +- "#address-cells" +- "#size-cells" + +examples: + - | + hypervisor { + #address-cells = <2>; + #size-cells = <0>; + compatible = "qcom,gunyah-hypervisor-1.0", "qcom,gunyah-hypervisor", "simple-bus"; + name = "hypervisor"; + + qcom,gunyah-vm { + compatible = "qcom,gunyah-vm-id-1.0", "qcom,gunyah-vm-id"; + qcom,vendor = "Qualcomm Technologies, Inc."; + qcom,vmid = <45>; + qcom,owner-vmid = <3>; + }; + + qcom,resource-manager-rpc@0000000000000001 { + compatible = "qcom,resource-manager-1-0", "qcom,resource-manager", + "qcom,gunyah-message-queue", "qcom,gunyah-capability"; + interrupts = , /* TX full IRQ */ + ; /* RX empty IRQ */ + reg = <0x00000000 0x00000000>, <0x00000000 0x00000001>; + /* TX, RX cap ids */ + qcom,is-full-duplex; + qcom,free-irq-start = <0>; + qcom,tx-queue-depth = <8>; + qcom,tx-message-size = <0xf0>; + qcom,rx-queue-depth = <8>; + qcom,rx-message-size = <0xf0>; + }; + }; diff --git a/MAINTAINERS b/MAINTAINERS index bed175adc4c3..6a918f653eac 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -8400,6 +8400,7 @@ M: Elliot Berman M: Murali Nalajala L: linux-arm-msm@vger.kernel.org S: Maintained +F: Documentation/devicetree/bindings/gunyah/ F: Documentation/virt/gunyah/ H8/300 ARCHITECTURE From patchwork Wed Feb 23 23:37:21 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Elliot Berman X-Patchwork-Id: 12757670 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 98E6CC4332F for ; Wed, 23 Feb 2022 23:38:07 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S244905AbiBWXid (ORCPT ); Wed, 23 Feb 2022 18:38:33 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58874 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S241915AbiBWXi2 (ORCPT ); Wed, 23 Feb 2022 18:38:28 -0500 Received: from alexa-out-sd-02.qualcomm.com (alexa-out-sd-02.qualcomm.com [199.106.114.39]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 57BC45A58D; Wed, 23 Feb 2022 15:37:56 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; i=@quicinc.com; q=dns/txt; s=qcdkim; t=1645659476; x=1677195476; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=O348AWxXja34F/CxfeoastVacG7K7AciL4ROR2FvJFI=; b=Eyh/XzcXabgRmqrUK0dy/QYd5j4h8dKedNwTXYctTdhp+GcZIpGmlcNz 9Ir3gUsKYTFF7URNq9lDzUdIVTfOr8HTlAnL7lf7/3l4NTefhZeJVUSVn Uw3x7hZ4l0Dn39JaBetk7zNAXERsHEPq60BjRSJll4GQPXabhmdYVcRIS o=; Received: from unknown (HELO ironmsg01-sd.qualcomm.com) ([10.53.140.141]) by alexa-out-sd-02.qualcomm.com with ESMTP; 23 Feb 2022 15:37:56 -0800 X-QCInternal: smtphost Received: from nasanex01b.na.qualcomm.com ([10.46.141.250]) by ironmsg01-sd.qualcomm.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 23 Feb 2022 15:37:56 -0800 Received: from hu-eberman-lv.qualcomm.com (10.49.16.6) by nasanex01b.na.qualcomm.com (10.46.141.250) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.15; Wed, 23 Feb 2022 15:37:55 -0800 From: Elliot Berman To: Bjorn Andersson , CC: Elliot Berman , Trilok Soni , Murali Nalajala , Srivatsa Vaddagiri , Carl van Schaik , Andy Gross , , Subject: [PATCH 03/11] arm64: gunyah: Add Gunyah hypercalls ABI Date: Wed, 23 Feb 2022 15:37:21 -0800 Message-ID: <20220223233729.1571114-4-quic_eberman@quicinc.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220223233729.1571114-1-quic_eberman@quicinc.com> References: <20220223233729.1571114-1-quic_eberman@quicinc.com> MIME-Version: 1.0 X-Originating-IP: [10.49.16.6] X-ClientProxiedBy: nalasex01a.na.qualcomm.com (10.47.209.196) To nasanex01b.na.qualcomm.com (10.46.141.250) Precedence: bulk List-ID: X-Mailing-List: linux-arm-msm@vger.kernel.org Add initial support to perform Gunyah hypercalls. The arm64 ABI for Gunyah hypercalls generally follows the AAPCS64, and can be summarized: - Function identifier is passed through the imm operand - [r0,r7] are parameter and result registers - [r8-r18] are temporary and saved by the caller (VM) - [r19-r31] are preserved and saved by the hypervisor The preprocessor macors for creating the necessary HVC instruction roughly follows the SMCCC 1.1 implementation in include/linux/arm-smccc.h. Signed-off-by: Elliot Berman --- MAINTAINERS | 1 + arch/arm64/include/asm/gunyah/hypercall.h | 193 ++++++++++++++++++++++ 2 files changed, 194 insertions(+) create mode 100644 arch/arm64/include/asm/gunyah/hypercall.h diff --git a/MAINTAINERS b/MAINTAINERS index 6a918f653eac..7e6a8488fa3e 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -8402,6 +8402,7 @@ L: linux-arm-msm@vger.kernel.org S: Maintained F: Documentation/devicetree/bindings/gunyah/ F: Documentation/virt/gunyah/ +F: arch/arm64/include/asm/gunyah/ H8/300 ARCHITECTURE M: Yoshinori Sato diff --git a/arch/arm64/include/asm/gunyah/hypercall.h b/arch/arm64/include/asm/gunyah/hypercall.h new file mode 100644 index 000000000000..626163500e32 --- /dev/null +++ b/arch/arm64/include/asm/gunyah/hypercall.h @@ -0,0 +1,193 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (c) 2022 Qualcomm Innovation Center, Inc. All rights reserved. + */ +#ifndef __ASM_GH_HYPERCALL_H +#define __ASM_GH_HYPERCALL_H + +#include + +#define ___gh_count_args(_0, _1, _2, _3, _4, _5, _6, _7, _8, x, ...) x + +#define __gh_count_args(...) \ + ___gh_count_args(_, ## __VA_ARGS__, 8, 7, 6, 5, 4, 3, 2, 1, 0) + +#define __gh_skip_0(...) __VA_ARGS__ +#define __gh_skip_1(a, ...) __VA_ARGS__ +#define __gh_skip_2(a, b, ...) __VA_ARGS__ +#define __gh_skip_3(a, b, c, ...) __VA_ARGS__ +#define __gh_skip_4(a, b, c, d, ...) __VA_ARGS__ +#define __gh_skip_5(a, b, c, d, e, ...) __VA_ARGS__ +#define __gh_skip_6(a, b, c, d, e, f, ...) __VA_ARGS__ +#define __gh_skip_7(a, b, c, d, e, f, g, ...) __VA_ARGS__ +#define __gh_skip_8(a, b, c, d, e, f, g, h, ...) __VA_ARGS__ + +#define __gh_declare_arg_0(...) + +#define __gh_declare_arg_1(a1, ...) \ + typeof(a1) __gh_a1 = (a1); \ + register uintptr_t arg1 asm("r0") = __gh_a1 + +#define __gh_declare_arg_2(a1, a2, ...) \ + __gh_declare_arg_1(a1); \ + typeof(a2) __gh_a2 = (a2); \ + register uintptr_t arg2 asm("r1") = __gh_a2 + +#define __gh_declare_arg_3(a1, a2, a3, ...) \ + __gh_declare_arg_2(a1, a2); \ + typeof(a3) __gh_a3 = (a3); \ + register uintptr_t arg3 asm("r2") = __gh_a3 + +#define __gh_declare_arg_4(a1, a2, a3, a4, ...) \ + __gh_declare_arg_3(a1, a2, a3); \ + typeof(a4) __gh_a4 = (a4); \ + register uintptr_t arg4 asm("r3") = __gh_a4 + +#define __gh_declare_arg_5(a1, a2, a3, a4, a5, ...) \ + __gh_declare_arg_4(a1, a2, a3, a4); \ + typeof(a5) __gh_a5 = (a5); \ + register uintptr_t arg5 asm("r4") = __gh_a5 + +#define __gh_declare_arg_6(a1, a2, a3, a4, a5, a6, ...) \ + __gh_declare_arg_5(a1, a2, a3, a4, a5); \ + typeof(a6) __gh_a6 = (a6); \ + register uintptr_t arg6 asm("r5") = __gh_a6 + +#define __gh_declare_arg_7(a1, a2, a3, a4, a5, a6, a7, ...) \ + __gh_declare_arg_6(a1, a2, a3, a4, a5, a6); \ + typeof(a7) __gh_a7 = (a7); \ + register uintptr_t arg7 asm("r6") = __gh_a7 + +#define __gh_declare_arg_8(a1, a2, a3, a4, a5, a6, a7, a8, ...) \ + __gh_declare_arg_7(a1, a2, a3, a4, a5, a6, a7); \ + typeof(a8) __gh_a8 = (a8); \ + register uintptr_t arg8 asm("r7") = __gh_a8 + +#define ___gh_declare_args(nargs) __gh_declare_arg_ ## nargs +#define __gh_declare_args(nargs) ___gh_declare_args(nargs) +#define _gh_declare_args(nargs, ...) __gh_declare_args(nargs)(__VA_ARGS__) + +#define __gh_constraint_arg_0 +#define __gh_constraint_arg_1 "r" (arg1), +#define __gh_constraint_arg_2 __gh_constraint_arg_1 "r" (arg2), +#define __gh_constraint_arg_3 __gh_constraint_arg_2 "r" (arg3), +#define __gh_constraint_arg_4 __gh_constraint_arg_3 "r" (arg4), +#define __gh_constraint_arg_5 __gh_constraint_arg_4 "r" (arg5), +#define __gh_constraint_arg_6 __gh_constraint_arg_5 "r" (arg6), +#define __gh_constraint_arg_7 __gh_constraint_arg_6 "r" (arg7), +#define __gh_constraint_arg_8 __gh_constraint_arg_7 "r" (arg8), + +#define _gh_constraint_args(nargs) __gh_constraint_arg_ ## nargs + +#define __gh_to_res(nargs, ...) __gh_skip_ ## nargs (__VA_ARGS__) + +#define __gh_declare_res_0 + +#define __gh_declare_res_1 \ + register uintptr_t res1 asm("r0") + +#define __gh_declare_res_2 \ + __gh_declare_res_1; \ + register uintptr_t res2 asm("r1") + +#define __gh_declare_res_3 \ + __gh_declare_res_2; \ + register uintptr_t res3 asm("r2") + +#define __gh_declare_res_4 \ + __gh_declare_res_3; \ + register uintptr_t res4 asm("r3") + +#define __gh_declare_res_5 \ + __gh_declare_res_4; \ + register uintptr_t res5 asm("r4") + +#define __gh_declare_res_6 \ + __gh_declare_res_5; \ + register uintptr_t res6 asm("r5") + +#define __gh_declare_res_7 \ + __gh_declare_res_6; \ + register uintptr_t res7 asm("r6") + +#define __gh_declare_res_8 \ + __gh_declare_res_7; \ + register uintptr_t res8 asm("r7") + +#define ___gh_declare_res(nargs) __gh_declare_res_ ## nargs +#define __gh_declare_res(nargs) ___gh_declare_res(nargs) +#define _gh_declare_res(...) __gh_declare_res(__gh_count_args(__VA_ARGS__)) + +#define __gh_constraint_res_0 +#define __gh_constraint_res_1 "=r" (res1) +#define __gh_constraint_res_2 __gh_constraint_res_1, "=r" (res2) +#define __gh_constraint_res_3 __gh_constraint_res_2, "=r" (res3) +#define __gh_constraint_res_4 __gh_constraint_res_3, "=r" (res4) +#define __gh_constraint_res_5 __gh_constraint_res_4, "=r" (res5) +#define __gh_constraint_res_6 __gh_constraint_res_5, "=r" (res6) +#define __gh_constraint_res_7 __gh_constraint_res_6, "=r" (res7) +#define __gh_constraint_res_8 __gh_constraint_res_7, "=r" (res8) + +#define ___gh_constraint_res(nargs) __gh_constraint_res_ ## nargs +#define __gh_constraint_res(nargs) ___gh_constraint_res(nargs) +#define _gh_constraint_res(...) \ + __gh_constraint_res(__gh_count_args(__VA_ARGS__)) + +#define __gh_assign_res_0(...) + +#define __gh_assign_res_1(r1) \ + r1 = res1; + +#define __gh_assign_res_2(r1, r2) \ + __gh_assign_res_1(r1); \ + r2 = res2 + +#define __gh_assign_res_3(r1, r2, r3) \ + __gh_assign_res_2(r1, r2); \ + r3 = res3 + +#define __gh_assign_res_4(r1, r2, r3, r4) \ + __gh_assign_res_3(r1, r2, r3); \ + r4 = res4 + +#define __gh_assign_res_5(r1, r2, r3, r4, r5) \ + __gh_assign_res_4(r1, r2, r3, r4); \ + r5 = res5 + +#define __gh_assign_res_6(r1, r2, r3, r4, r5, r6) \ + __gh_assign_res_5(r1, r2, r3, r4, r5); \ + r6 = res6 + +#define __gh_assign_res_7(r1, r2, r3, r4, r5, r6, r7) \ + __gh_assign_res_6(r1, r2, r3, r4, r5, r6); \ + r7 = res7 + +#define __gh_assign_res_8(r1, r2, r3, r4, r5, r6, r7, r8) \ + __gh_assign_res_7(r1, r2, r3, r4, r5, r6, r7); \ + r8 = res8 + +#define ___gh_assign_res(nargs) __gh_assign_res_ ## nargs +#define __gh_assign_res(nargs) ___gh_assign_res(nargs) +#define _gh_assign_res(...) __gh_assign_res(__gh_count_args(__VA_ARGS__))(__VA_ARGS__) + +/** + * arch_gh_hypercall() - Performs an AArch64-specific call into hypervisor using Gunyah ABI + * @hcall_num: Hypercall function ID to invoke + * @nargs: Number of input arguments + * @...: First nargs are the input arguments. Remaining arguments are output variables. + */ +#define arch_gh_hypercall(hcall_num, nargs, ...) \ + do { \ + _gh_declare_res(__gh_to_res(nargs, __VA_ARGS__)); \ + _gh_declare_args(nargs, __VA_ARGS__); \ + asm volatile( \ + "hvc %[num]\n" \ + : _gh_constraint_res(__gh_to_res(nargs, __VA_ARGS__)) \ + : _gh_constraint_args(nargs) \ + [num] "i" (hcall_num) \ + : "x9", "x10", "x11", "x12", "x13", "x14", "x15", "x16", "x17", \ + "memory"); \ + _gh_assign_res(__gh_to_res(nargs, __VA_ARGS__)); \ + } while (0) + +#endif From patchwork Wed Feb 23 23:37:22 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Elliot Berman X-Patchwork-Id: 12757671 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id F344DC433EF for ; Wed, 23 Feb 2022 23:38:07 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S244918AbiBWXif (ORCPT ); Wed, 23 Feb 2022 18:38:35 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58872 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S244881AbiBWXi2 (ORCPT ); Wed, 23 Feb 2022 18:38:28 -0500 Received: from alexa-out-sd-02.qualcomm.com (alexa-out-sd-02.qualcomm.com [199.106.114.39]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B0A5E5A09B; Wed, 23 Feb 2022 15:37:56 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; i=@quicinc.com; q=dns/txt; s=qcdkim; t=1645659476; x=1677195476; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=wJO28cZ5d4onrWSSPwvwkejj4FA3oYjfcxSdxHBpzKY=; b=S+ys3OdwOMfnpBk52oM1FUhMEENXpOvn3mR2D/DpC0BaxxAwPrKvgOvd W0k9fXRGthrd9D+aXDRdQ5TICawPPCmjfYxm8MCReEK00rmddWLCNK8nN X5eU5GB7Cav0ujZH8GmtAGs8aQCS6KWRFhmL9L0yofjOZ3pAhruFoUzwF c=; Received: from unknown (HELO ironmsg01-sd.qualcomm.com) ([10.53.140.141]) by alexa-out-sd-02.qualcomm.com with ESMTP; 23 Feb 2022 15:37:56 -0800 X-QCInternal: smtphost Received: from nasanex01b.na.qualcomm.com ([10.46.141.250]) by ironmsg01-sd.qualcomm.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 23 Feb 2022 15:37:56 -0800 Received: from hu-eberman-lv.qualcomm.com (10.49.16.6) by nasanex01b.na.qualcomm.com (10.46.141.250) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.15; Wed, 23 Feb 2022 15:37:55 -0800 From: Elliot Berman To: Bjorn Andersson , CC: Elliot Berman , Trilok Soni , Murali Nalajala , Srivatsa Vaddagiri , Carl van Schaik , Andy Gross , Subject: [PATCH 04/11] gunyah: Common types and error codes for Gunyah hypercalls Date: Wed, 23 Feb 2022 15:37:22 -0800 Message-ID: <20220223233729.1571114-5-quic_eberman@quicinc.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220223233729.1571114-1-quic_eberman@quicinc.com> References: <20220223233729.1571114-1-quic_eberman@quicinc.com> MIME-Version: 1.0 X-Originating-IP: [10.49.16.6] X-ClientProxiedBy: nalasex01a.na.qualcomm.com (10.47.209.196) To nasanex01b.na.qualcomm.com (10.46.141.250) Precedence: bulk List-ID: X-Mailing-List: linux-arm-msm@vger.kernel.org Add architecture-independent standard error codes, types, and macros for Gunyah hypercalls. Signed-off-by: Elliot Berman --- MAINTAINERS | 1 + include/linux/gunyah.h | 74 ++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 75 insertions(+) create mode 100644 include/linux/gunyah.h diff --git a/MAINTAINERS b/MAINTAINERS index 7e6a8488fa3e..59e7070f726a 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -8403,6 +8403,7 @@ S: Maintained F: Documentation/devicetree/bindings/gunyah/ F: Documentation/virt/gunyah/ F: arch/arm64/include/asm/gunyah/ +F: include/linux/gunyah.h H8/300 ARCHITECTURE M: Yoshinori Sato diff --git a/include/linux/gunyah.h b/include/linux/gunyah.h new file mode 100644 index 000000000000..8743bf4978e2 --- /dev/null +++ b/include/linux/gunyah.h @@ -0,0 +1,74 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (c) 2022 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#ifndef _GUNYAH_H +#define _GUNYAH_H + +#include +#include +#include + +typedef u64 gh_capid_t; + +/* Common Gunyah macros */ +#define GH_CAPID_INVAL U64_MAX + +#define GH_ERROR_OK 0 +#define GH_ERROR_UNIMPLEMENTED -1 + +#define GH_ERROR_ARG_INVAL 1 +#define GH_ERROR_ARG_SIZE 2 +#define GH_ERROR_ARG_ALIGN 3 + +#define GH_ERROR_NOMEM 10 + +#define GH_ERROR_ADDR_OVFL 20 +#define GH_ERROR_ADDR_UNFL 21 +#define GH_ERROR_ADDR_INVAL 22 + +#define GH_ERROR_DENIED 30 +#define GH_ERROR_BUSY 31 +#define GH_ERROR_IDLE 32 + +#define GH_ERROR_IRQ_BOUND 40 +#define GH_ERROR_IRQ_UNBOUND 41 + +#define GH_ERROR_CSPACE_CAP_NULL 50 +#define GH_ERROR_CSPACE_CAP_REVOKED 51 +#define GH_ERROR_CSPACE_WRONG_OBJ_TYPE 52 +#define GH_ERROR_CSPACE_INSUF_RIGHTS 53 +#define GH_ERROR_CSPACE_FULL 54 + +#define GH_ERROR_MSGQUEUE_EMPTY 60 +#define GH_ERROR_MSGQUEUE_FULL 61 + +static inline int gh_remap_error(int gh_error) +{ + switch (gh_error) { + case GH_ERROR_OK: + return 0; + case GH_ERROR_NOMEM: + return -ENOMEM; + case GH_ERROR_DENIED: + case GH_ERROR_CSPACE_CAP_NULL: + case GH_ERROR_CSPACE_CAP_REVOKED: + case GH_ERROR_CSPACE_WRONG_OBJ_TYPE: + case GH_ERROR_CSPACE_INSUF_RIGHTS: + case GH_ERROR_CSPACE_FULL: + return -EACCES; + case GH_ERROR_BUSY: + case GH_ERROR_IDLE: + return -EBUSY; + case GH_ERROR_IRQ_BOUND: + case GH_ERROR_IRQ_UNBOUND: + case GH_ERROR_MSGQUEUE_FULL: + case GH_ERROR_MSGQUEUE_EMPTY: + return -EPERM; + default: + return -EINVAL; + } +} + +#endif From patchwork Wed Feb 23 23:37:23 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Elliot Berman X-Patchwork-Id: 12757669 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1570DC433F5 for ; Wed, 23 Feb 2022 23:38:07 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S244873AbiBWXic (ORCPT ); Wed, 23 Feb 2022 18:38:32 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58878 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S244885AbiBWXi2 (ORCPT ); Wed, 23 Feb 2022 18:38:28 -0500 Received: from alexa-out-sd-01.qualcomm.com (alexa-out-sd-01.qualcomm.com [199.106.114.38]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 814B45A597; Wed, 23 Feb 2022 15:37:57 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; i=@quicinc.com; q=dns/txt; s=qcdkim; t=1645659477; x=1677195477; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=+MRbB5DRgFvuMH3fvObVcrNFhjluEvf0jPhElKZgs1w=; b=gi+qzBsaFfqENBDMIaAUscS6RffAlP1N+78FKmTntti2oP9hI04z0kZm nC0BJuyWrdq2AnKwfw6F1ogyVgCk3oWqolT8i302cHYrBUff4IcWfd8bH DzYFOU8MqQRerlSboYYwjogmD4f64pamb+LvC1DgNwU5PIUXI8z1GcA8f 8=; Received: from unknown (HELO ironmsg02-sd.qualcomm.com) ([10.53.140.142]) by alexa-out-sd-01.qualcomm.com with ESMTP; 23 Feb 2022 15:37:57 -0800 X-QCInternal: smtphost Received: from nasanex01b.na.qualcomm.com ([10.46.141.250]) by ironmsg02-sd.qualcomm.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 23 Feb 2022 15:37:57 -0800 Received: from hu-eberman-lv.qualcomm.com (10.49.16.6) by nasanex01b.na.qualcomm.com (10.46.141.250) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.15; Wed, 23 Feb 2022 15:37:56 -0800 From: Elliot Berman To: Bjorn Andersson , CC: Elliot Berman , Trilok Soni , Murali Nalajala , Srivatsa Vaddagiri , Carl van Schaik , Andy Gross , Subject: [PATCH 05/11] virt: gunyah: Add sysfs nodes Date: Wed, 23 Feb 2022 15:37:23 -0800 Message-ID: <20220223233729.1571114-6-quic_eberman@quicinc.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220223233729.1571114-1-quic_eberman@quicinc.com> References: <20220223233729.1571114-1-quic_eberman@quicinc.com> MIME-Version: 1.0 X-Originating-IP: [10.49.16.6] X-ClientProxiedBy: nalasex01a.na.qualcomm.com (10.47.209.196) To nasanex01b.na.qualcomm.com (10.46.141.250) Precedence: bulk List-ID: X-Mailing-List: linux-arm-msm@vger.kernel.org Add /sys/hypervisor support when detecting that Linux is running in a Gunyah environment. Export the version of Gunyah which is reported via the hyp_identify hypercall. Signed-off-by: Elliot Berman --- .../ABI/testing/sysfs-hypervisor-gunyah | 37 ++++++ MAINTAINERS | 2 + arch/arm64/include/asm/gunyah/hypercall.h | 2 + drivers/virt/Kconfig | 2 + drivers/virt/Makefile | 1 + drivers/virt/gunyah/Kconfig | 13 ++ drivers/virt/gunyah/Makefile | 4 + drivers/virt/gunyah/sysfs.c | 116 ++++++++++++++++++ 8 files changed, 177 insertions(+) create mode 100644 Documentation/ABI/testing/sysfs-hypervisor-gunyah create mode 100644 drivers/virt/gunyah/Kconfig create mode 100644 drivers/virt/gunyah/Makefile create mode 100644 drivers/virt/gunyah/sysfs.c diff --git a/Documentation/ABI/testing/sysfs-hypervisor-gunyah b/Documentation/ABI/testing/sysfs-hypervisor-gunyah new file mode 100644 index 000000000000..ebbdd0aead7b --- /dev/null +++ b/Documentation/ABI/testing/sysfs-hypervisor-gunyah @@ -0,0 +1,37 @@ +What: /sys/hypervisor/type +Date: January 2022 +KernelVersion: 5.17 +Contact: linux-arm-msm@vger.kernel.org +Description: If running under Gunyah: + Type of hypervisor: + "gunyah": Gunyah hypervisor + +What: /sys/hypervisor/features +Date: January 2022 +KernelVersion: 5.17 +Contact: linux-arm-msm@vger.kernel.org +Description: If running under Gunyah: + Space separated list of features supported by Linux and Gunyah: + "cspace": Gunyah devices + "doorbell": Sending/receiving virtual interrupts via Gunyah doorbells + "message-queue": Sending/receiving messages via Gunyah message queues + "vic": Interrupt lending + "vpm": Virtual platform management + "vcpu": Virtual CPU management + "memextent": Memory lending/management + "trace": Gunyah hypervisor tracing + + +What: /sys/hypervisor/version/api +Date: April 2020 +KernelVersion: 5.17 +Contact: linux-arm-msm@vger.kernel.org +Description: If running under Gunyah: + The Gunyah API version. + +What: /sys/hypervisor/version/variant +Date: April 2020 +KernelVersion: 5.17 +Contact: linux-arm-msm@vger.kernel.org +Description: If running under Gunyah: + The Gunyah variant (build) version. \ No newline at end of file diff --git a/MAINTAINERS b/MAINTAINERS index 59e7070f726a..10c59c8767ff 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -8400,9 +8400,11 @@ M: Elliot Berman M: Murali Nalajala L: linux-arm-msm@vger.kernel.org S: Maintained +F: Documentation/ABI/testing/sysfs-hypervisor-gunyah F: Documentation/devicetree/bindings/gunyah/ F: Documentation/virt/gunyah/ F: arch/arm64/include/asm/gunyah/ +F: drivers/virt/gunyah/ F: include/linux/gunyah.h H8/300 ARCHITECTURE diff --git a/arch/arm64/include/asm/gunyah/hypercall.h b/arch/arm64/include/asm/gunyah/hypercall.h index 626163500e32..a8e68ece074e 100644 --- a/arch/arm64/include/asm/gunyah/hypercall.h +++ b/arch/arm64/include/asm/gunyah/hypercall.h @@ -7,6 +7,8 @@ #include +#define GH_HYPERCALL_HYP_IDENTIFY 0x6000 + #define ___gh_count_args(_0, _1, _2, _3, _4, _5, _6, _7, _8, x, ...) x #define __gh_count_args(...) \ diff --git a/drivers/virt/Kconfig b/drivers/virt/Kconfig index 8061e8ef449f..823663d67a95 100644 --- a/drivers/virt/Kconfig +++ b/drivers/virt/Kconfig @@ -36,4 +36,6 @@ source "drivers/virt/vboxguest/Kconfig" source "drivers/virt/nitro_enclaves/Kconfig" source "drivers/virt/acrn/Kconfig" + +source "drivers/virt/gunyah/Kconfig" endif diff --git a/drivers/virt/Makefile b/drivers/virt/Makefile index 3e272ea60cd9..ca2141b6837e 100644 --- a/drivers/virt/Makefile +++ b/drivers/virt/Makefile @@ -5,6 +5,7 @@ obj-$(CONFIG_FSL_HV_MANAGER) += fsl_hypervisor.o obj-y += vboxguest/ +obj-$(CONFIG_GUNYAH) += gunyah/ obj-$(CONFIG_NITRO_ENCLAVES) += nitro_enclaves/ obj-$(CONFIG_ACRN_HSM) += acrn/ diff --git a/drivers/virt/gunyah/Kconfig b/drivers/virt/gunyah/Kconfig new file mode 100644 index 000000000000..e88289963518 --- /dev/null +++ b/drivers/virt/gunyah/Kconfig @@ -0,0 +1,13 @@ +# SPDX-License-Identifier: GPL-2.0-only + +config GUNYAH + tristate "Gunyah Virtualization drivers" + depends on ARM64 + select SYS_HYPERVISOR + help + The Gunyah drivers are the helper interfaces that runs in a guest VM + such as basic inter-VM IPC and signaling mechanism,s and higher level + services such as memory/device sharing, IRQ sharing, and so on. + + Say Y here to enable the drivers needed to interact in a Gunyah + virtual environment. diff --git a/drivers/virt/gunyah/Makefile b/drivers/virt/gunyah/Makefile new file mode 100644 index 000000000000..0aa086f9149f --- /dev/null +++ b/drivers/virt/gunyah/Makefile @@ -0,0 +1,4 @@ +# SPDX-License-Identifier: GPL-2.0-only + +gunyah-y += sysfs.o +obj-$(CONFIG_GUNYAH) += gunyah.o \ No newline at end of file diff --git a/drivers/virt/gunyah/sysfs.c b/drivers/virt/gunyah/sysfs.c new file mode 100644 index 000000000000..3d22f08360db --- /dev/null +++ b/drivers/virt/gunyah/sysfs.c @@ -0,0 +1,116 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (c) 2022 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#define pr_fmt(fmt) "gunyah: " fmt + +#include +#include +#include +#include +#include +#include + +#include "gunyah_private.h" + +#define GH_API_INFO_API_VERSION(x) (((x) >> 0) & 0x3fff) +#define GH_API_INFO_BIG_ENDIAN(x) (((x) >> 14) & 1) +#define GH_API_INFO_IS_64BIT(x) (((x) >> 15) & 1) +#define GH_API_INFO_VARIANT(x) (((x) >> 56) & 0xff) + +#define GH_IDENTIFY_PARTITION_CSPACE(flags) (((flags)[0] >> 0) & 1) +#define GH_IDENTIFY_DOORBELL(flags) (((flags)[0] >> 1) & 1) +#define GH_IDENTIFY_MSGQUEUE(flags) (((flags)[0] >> 2) & 1) +#define GH_IDENTIFY_VIC(flags) (((flags)[0] >> 3) & 1) +#define GH_IDENTIFY_VPM(flags) (((flags)[0] >> 4) & 1) +#define GH_IDENTIFY_VCPU(flags) (((flags)[0] >> 5) & 1) +#define GH_IDENTIFY_MEMEXTENT(flags) (((flags)[0] >> 6) & 1) +#define GH_IDENTIFY_TRACE_CTRL(flags) (((flags)[0] >> 7) & 1) + +struct gh_hypercall_hyp_identify_resp { + u64 api_info; + u64 flags[3]; +}; + +static struct gh_hypercall_hyp_identify_resp gunyah_api; + +static ssize_t type_show(struct kobject *kobj, struct kobj_attribute *attr, char *buffer) +{ + return sysfs_emit(buffer, "gunyah\n"); +} +static struct kobj_attribute type_attr = __ATTR_RO(type); + +static ssize_t api_show(struct kobject *kobj, struct kobj_attribute *attr, char *buffer) +{ + return sysfs_emit(buffer, "%d\n", (int)GH_API_INFO_API_VERSION(gunyah_api.api_info)); +} +static struct kobj_attribute api_attr = __ATTR_RO(api); + +static ssize_t variant_show(struct kobject *kobj, struct kobj_attribute *attr, char *buffer) +{ + return sysfs_emit(buffer, "%d\n", (int)GH_API_INFO_VARIANT(gunyah_api.api_info)); +} +static struct kobj_attribute variant_attr = __ATTR_RO(variant); + +static ssize_t features_show(struct kobject *kobj, struct kobj_attribute *attr, char *buffer) +{ + return sysfs_emit(buffer, "\n"); +} +static struct kobj_attribute features_attr = __ATTR_RO(features); + +static struct attribute *version_attrs[] = { + &api_attr.attr, + &variant_attr.attr, + NULL +}; + +static const struct attribute_group version_group = { + .name = "version", + .attrs = version_attrs, +}; + +static int __init gh_sysfs_register(void) +{ + int ret; + + ret = sysfs_create_file(hypervisor_kobj, &type_attr.attr); + if (ret) + return ret; + + ret = sysfs_create_group(hypervisor_kobj, &version_group); + if (ret) + return ret; + + return sysfs_create_file(hypervisor_kobj, &features_attr.attr); +} + +static void gh_sysfs_unregister(void) +{ + sysfs_remove_file(hypervisor_kobj, &type_attr.attr); + sysfs_remove_group(hypervisor_kobj, &version_group); +} + +static int __init gunyah_init(void) +{ + arch_gh_hypercall(GH_HYPERCALL_HYP_IDENTIFY, 0, gunyah_api.api_info, + gunyah_api.flags[0], gunyah_api.flags[1], gunyah_api.flags[2]); + + if (GH_API_INFO_API_VERSION(gunyah_api.api_info) != 1) { + pr_warn("Unrecognized gunyah version: %llu. Currently supported: 1\n", + GH_API_INFO_API_VERSION(gunyah_api.api_info)); + return 0; + } + + return gh_sysfs_register(); +} +module_init(gunyah_init); + +static void __exit gunyah_exit(void) +{ + gh_sysfs_unregister(); +} +module_exit(gunyah_exit); + +MODULE_LICENSE("GPL v2"); +MODULE_DESCRIPTION("Gunyah Hypervisor Driver"); From patchwork Wed Feb 23 23:37:24 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Elliot Berman X-Patchwork-Id: 12757676 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 359B1C4321E for ; Wed, 23 Feb 2022 23:38:12 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S244921AbiBWXii (ORCPT ); Wed, 23 Feb 2022 18:38:38 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58882 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S244887AbiBWXi2 (ORCPT ); Wed, 23 Feb 2022 18:38:28 -0500 Received: from alexa-out-sd-02.qualcomm.com (alexa-out-sd-02.qualcomm.com [199.106.114.39]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B30E15A5A0; Wed, 23 Feb 2022 15:37:57 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; i=@quicinc.com; q=dns/txt; s=qcdkim; t=1645659477; x=1677195477; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=PtK747GnjuyC0WUIVmHWvW0DGS4uGZItQvyQSo8I7T8=; b=l4jH0vdACg2AumFfjaO0fny8hxcb7RhexOUGL78pgYEZo6hbVP3WBm5U 5i1uyCYydngHRvMAUNys6luAD03k2qjNXjUCz6IKfdtGFfbGya7UKp25T 1zXtj/ZlQ5MMONmrSvwt80mVUV0FOuRepv2zh34cRQPs0PY7AeHmnt5OY 8=; Received: from unknown (HELO ironmsg-SD-alpha.qualcomm.com) ([10.53.140.30]) by alexa-out-sd-02.qualcomm.com with ESMTP; 23 Feb 2022 15:37:57 -0800 X-QCInternal: smtphost Received: from nasanex01b.na.qualcomm.com ([10.46.141.250]) by ironmsg-SD-alpha.qualcomm.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 23 Feb 2022 15:37:57 -0800 Received: from hu-eberman-lv.qualcomm.com (10.49.16.6) by nasanex01b.na.qualcomm.com (10.46.141.250) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.15; Wed, 23 Feb 2022 15:37:56 -0800 From: Elliot Berman To: Bjorn Andersson , CC: Elliot Berman , Trilok Soni , Murali Nalajala , Srivatsa Vaddagiri , Carl van Schaik , Andy Gross , Subject: [PATCH 06/11] virt: gunyah: Add capabilities bus and devices Date: Wed, 23 Feb 2022 15:37:24 -0800 Message-ID: <20220223233729.1571114-7-quic_eberman@quicinc.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220223233729.1571114-1-quic_eberman@quicinc.com> References: <20220223233729.1571114-1-quic_eberman@quicinc.com> MIME-Version: 1.0 X-Originating-IP: [10.49.16.6] X-ClientProxiedBy: nalasex01a.na.qualcomm.com (10.47.209.196) To nasanex01b.na.qualcomm.com (10.46.141.250) Precedence: bulk List-ID: X-Mailing-List: linux-arm-msm@vger.kernel.org Some resources provided by the Gunyah hypervisor are described as objects. The objects are identified with a capability ID. For instance, Inter-VM communication is performed with doorbells and message queues. Each doorbell and message queue endpoint can be described consisely as a Linux device. These resources are discovered either on the devicetree or reported by the Resource Manager. Devices on the Gunyah bus are matched with drivers according to the type ID reported by resource manager. Most resources will be discovered directly from the resource manager, so matching directly on type ID seems like sensible design. Each resource may also optionally have an interrupt associated with it and a known partner VM (e.g. which VM is the receiver of a message queue). Signed-off-by: Elliot Berman --- drivers/virt/gunyah/Makefile | 2 +- drivers/virt/gunyah/device.c | 108 +++++++++++++++++++++++++++ drivers/virt/gunyah/gunyah_private.h | 12 +++ drivers/virt/gunyah/sysfs.c | 24 +++++- include/linux/gunyah.h | 45 +++++++++++ 5 files changed, 188 insertions(+), 3 deletions(-) create mode 100644 drivers/virt/gunyah/device.c create mode 100644 drivers/virt/gunyah/gunyah_private.h diff --git a/drivers/virt/gunyah/Makefile b/drivers/virt/gunyah/Makefile index 0aa086f9149f..3869fb7371df 100644 --- a/drivers/virt/gunyah/Makefile +++ b/drivers/virt/gunyah/Makefile @@ -1,4 +1,4 @@ # SPDX-License-Identifier: GPL-2.0-only -gunyah-y += sysfs.o +gunyah-y += sysfs.o device.o obj-$(CONFIG_GUNYAH) += gunyah.o \ No newline at end of file diff --git a/drivers/virt/gunyah/device.c b/drivers/virt/gunyah/device.c new file mode 100644 index 000000000000..93595f9a65b9 --- /dev/null +++ b/drivers/virt/gunyah/device.c @@ -0,0 +1,108 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (c) 2022 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#define pr_fmt(fmt) "ghdev: " fmt + +#include +#include +#include +#include +#include + +#include "gunyah_private.h" + +static int gunyah_match(struct device *dev, struct device_driver *drv) +{ + struct gunyah_device *ghdev = to_gunyah_device(dev); + struct gunyah_driver *ghdrv = to_gunyah_driver(drv); + + return ghdev->type == ghdrv->type; +} + +static int gunyah_probe(struct device *dev) +{ + struct gunyah_device *ghdev = to_gunyah_device(dev); + struct gunyah_driver *ghdrv = to_gunyah_driver(dev->driver); + + return ghdrv->probe ? ghdrv->probe(ghdev) : 0; +} + +static void gunyah_remove(struct device *dev) +{ + struct gunyah_device *ghdev = to_gunyah_device(dev); + struct gunyah_driver *ghdrv = to_gunyah_driver(dev->driver); + + if (ghdrv->remove) + ghdrv->remove(ghdev); +} + +static struct bus_type gunyah_bus = { + .name = "gunyah", + .match = gunyah_match, + .probe = gunyah_probe, + .remove = gunyah_remove, +}; + +int gunyah_register_driver(struct gunyah_driver *ghdrv) +{ + ghdrv->driver.bus = &gunyah_bus; + return driver_register(&ghdrv->driver); +} + +void gunyah_unregister_driver(struct gunyah_driver *ghdrv) +{ + driver_unregister(&ghdrv->driver); +} + +static void gunyah_device_release(struct device *dev) +{ + struct gunyah_device *ghdev = to_gunyah_device(dev); + + kfree(ghdev); +} + +struct gunyah_device *gunyah_device_alloc(struct device *parent, gh_capid_t capid, u8 type) +{ + struct gunyah_device *ghdev; + + ghdev = kzalloc(sizeof(*ghdev), GFP_KERNEL); + if (!ghdev) + return NULL; + + ghdev->capid = capid; + ghdev->type = type; + ghdev->irq = IRQ_NOTCONNECTED; + ghdev->dev.parent = parent; + ghdev->dev.release = gunyah_device_release; + ghdev->dev.bus = &gunyah_bus; + device_initialize(&ghdev->dev); + return ghdev; +} + +int gunyah_device_add(struct gunyah_device *ghdev) +{ + int ret; + + ret = dev_set_name(&ghdev->dev, "%u.%08llx", ghdev->type, ghdev->capid); + if (ret) + return ret; + + return device_add(&ghdev->dev); +} + +void gunyah_device_remove(struct gunyah_device *ghdev) +{ + device_unregister(&ghdev->dev); +} + +int __init gunyah_bus_init(void) +{ + return bus_register(&gunyah_bus); +} + +void gunyah_bus_exit(void) +{ + bus_unregister(&gunyah_bus); +} diff --git a/drivers/virt/gunyah/gunyah_private.h b/drivers/virt/gunyah/gunyah_private.h new file mode 100644 index 000000000000..5f3832608020 --- /dev/null +++ b/drivers/virt/gunyah/gunyah_private.h @@ -0,0 +1,12 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (c) 2022 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#ifndef _GUNYAH_PRIVATE_H +#define _GUNYAH_PRIVATE_H + +int __init gunyah_bus_init(void); +void gunyah_bus_exit(void); + +#endif diff --git a/drivers/virt/gunyah/sysfs.c b/drivers/virt/gunyah/sysfs.c index 3d22f08360db..d66e6275fe32 100644 --- a/drivers/virt/gunyah/sysfs.c +++ b/drivers/virt/gunyah/sysfs.c @@ -55,7 +55,13 @@ static struct kobj_attribute variant_attr = __ATTR_RO(variant); static ssize_t features_show(struct kobject *kobj, struct kobj_attribute *attr, char *buffer) { - return sysfs_emit(buffer, "\n"); + int len = 0; + + if (GH_IDENTIFY_PARTITION_CSPACE(gunyah_api.flags)) + len += sysfs_emit_at(buffer, len, "cspace "); + + len += sysfs_emit_at(buffer, len, "\n"); + return len; } static struct kobj_attribute features_attr = __ATTR_RO(features); @@ -93,6 +99,8 @@ static void gh_sysfs_unregister(void) static int __init gunyah_init(void) { + int ret; + arch_gh_hypercall(GH_HYPERCALL_HYP_IDENTIFY, 0, gunyah_api.api_info, gunyah_api.flags[0], gunyah_api.flags[1], gunyah_api.flags[2]); @@ -102,12 +110,24 @@ static int __init gunyah_init(void) return 0; } - return gh_sysfs_register(); + ret = gh_sysfs_register(); + if (ret) + return ret; + + ret = gunyah_bus_init(); + if (ret) + goto err_sysfs; + + return ret; +err_sysfs: + gh_sysfs_unregister(); + return ret; } module_init(gunyah_init); static void __exit gunyah_exit(void) { + gunyah_bus_exit(); gh_sysfs_unregister(); } module_exit(gunyah_exit); diff --git a/include/linux/gunyah.h b/include/linux/gunyah.h index 8743bf4978e2..f169c78881cb 100644 --- a/include/linux/gunyah.h +++ b/include/linux/gunyah.h @@ -8,6 +8,7 @@ #include #include +#include #include typedef u64 gh_capid_t; @@ -71,4 +72,48 @@ static inline int gh_remap_error(int gh_error) } } +/* Follows resource manager's resource types for VM_GET_HYP_RESOURCES */ +#define GUNYAH_DEVICE_TYPE_BELL_TX 0 +#define GUNYAH_DEVICE_TYPE_BELL_RX 1 +#define GUNYAH_DEVICE_TYPE_MSGQ_TX 2 +#define GUNYAH_DEVICE_TYPE_MSGQ_RX 3 +#define GUNYAH_DEVICE_TYPE_VCPU 4 + +struct gunyah_device { + u8 type; + gh_capid_t capid; + int irq; + + struct device dev; +}; + +#define to_gunyah_device(dev) container_of(dev, struct gunyah_device, dev) + +static inline void *ghdev_get_drvdata(const struct gunyah_device *ghdev) +{ + return dev_get_drvdata(&ghdev->dev); +} + +static inline void ghdev_set_drvdata(struct gunyah_device *ghdev, void *data) +{ + dev_set_drvdata(&ghdev->dev, data); +} + +struct gunyah_device *gunyah_device_alloc(struct device *parent, gh_capid_t capid, u8 type); + +int gunyah_device_add(struct gunyah_device *ghdev); +void gunyah_device_remove(struct gunyah_device *ghdev); + +struct gunyah_driver { + struct device_driver driver; + u8 type; + int (*probe)(struct gunyah_device *ghdev); + int (*remove)(struct gunyah_device *ghdev); +}; + +#define to_gunyah_driver(drv) container_of(drv, struct gunyah_driver, driver) + +int gunyah_register_driver(struct gunyah_driver *ghdrv); +void gunyah_unregister_driver(struct gunyah_driver *ghdrv); + #endif From patchwork Wed Feb 23 23:37:25 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Elliot Berman X-Patchwork-Id: 12757668 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 01A10C43219 for ; Wed, 23 Feb 2022 23:38:05 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S244899AbiBWXib (ORCPT ); Wed, 23 Feb 2022 18:38:31 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58884 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S244886AbiBWXi2 (ORCPT ); Wed, 23 Feb 2022 18:38:28 -0500 Received: from alexa-out-sd-02.qualcomm.com (alexa-out-sd-02.qualcomm.com [199.106.114.39]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 81F8C5A5A4; Wed, 23 Feb 2022 15:37:58 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; i=@quicinc.com; q=dns/txt; s=qcdkim; t=1645659478; x=1677195478; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=2kdOYPCLqgFfKRajMJBjKF0jVs1Vu57J310qUgU6YFU=; b=L7VoxBt6+fHGr4FjmhXjNF3luOFMBPDr0fEq5PEDisHYuj+9nLp2lct2 msVjx+aeN+T7FIKQF5Q9f8BkMKkQN6GS/jwyjUYxOLMebPhFsVFt6v9kQ W358zhX3t7+2mgbYskU9Pyk7j/zl+R4cTksLHYaiVvt+nN2ZZMcby/N8u E=; Received: from unknown (HELO ironmsg04-sd.qualcomm.com) ([10.53.140.144]) by alexa-out-sd-02.qualcomm.com with ESMTP; 23 Feb 2022 15:37:58 -0800 X-QCInternal: smtphost Received: from nasanex01b.na.qualcomm.com ([10.46.141.250]) by ironmsg04-sd.qualcomm.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 23 Feb 2022 15:37:57 -0800 Received: from hu-eberman-lv.qualcomm.com (10.49.16.6) by nasanex01b.na.qualcomm.com (10.46.141.250) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.15; Wed, 23 Feb 2022 15:37:57 -0800 From: Elliot Berman To: Bjorn Andersson , CC: Elliot Berman , Trilok Soni , Murali Nalajala , Srivatsa Vaddagiri , Carl van Schaik , Andy Gross , Subject: [PATCH 07/11] gunyah: msgq: Add Gunyah message queues Date: Wed, 23 Feb 2022 15:37:25 -0800 Message-ID: <20220223233729.1571114-8-quic_eberman@quicinc.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220223233729.1571114-1-quic_eberman@quicinc.com> References: <20220223233729.1571114-1-quic_eberman@quicinc.com> MIME-Version: 1.0 X-Originating-IP: [10.49.16.6] X-ClientProxiedBy: nalasex01a.na.qualcomm.com (10.47.209.196) To nasanex01b.na.qualcomm.com (10.46.141.250) Precedence: bulk List-ID: X-Mailing-List: linux-arm-msm@vger.kernel.org Gunyah message queues are unidirectional pipelines to communicate between 2 virtual machines, but are typically paired to allow bidirectional communication. The intended use case is for small control messages between 2 VMs, as they support a maximum of 240 bytes. Message queues can be discovered either by resource manager or on the devicetree. To support discovery on the devicetree, client drivers can use gh_msgq_platform_host_attach to allocate the tx and rx message queues according to Documentation/devicetree/bindings/gunyah/qcom,hypervisor.yml. Signed-off-by: Elliot Berman --- arch/arm64/include/asm/gunyah/hypercall.h | 4 + drivers/virt/gunyah/Makefile | 2 +- drivers/virt/gunyah/gunyah_private.h | 3 + drivers/virt/gunyah/msgq.c | 295 ++++++++++++++++++++++ drivers/virt/gunyah/sysfs.c | 9 + include/linux/gunyah.h | 19 ++ 6 files changed, 331 insertions(+), 1 deletion(-) create mode 100644 drivers/virt/gunyah/msgq.c diff --git a/arch/arm64/include/asm/gunyah/hypercall.h b/arch/arm64/include/asm/gunyah/hypercall.h index a8e68ece074e..7c6eb82ecd88 100644 --- a/arch/arm64/include/asm/gunyah/hypercall.h +++ b/arch/arm64/include/asm/gunyah/hypercall.h @@ -8,6 +8,10 @@ #include #define GH_HYPERCALL_HYP_IDENTIFY 0x6000 +#define GH_HYPERCALL_MSGQ_SEND 0x601B +#define GH_HYPERCALL_MSGQ_RECV 0x601C + +#define GH_HYPERCALL_MSGQ_SEND_FLAGS_PUSH BIT(0) #define ___gh_count_args(_0, _1, _2, _3, _4, _5, _6, _7, _8, x, ...) x diff --git a/drivers/virt/gunyah/Makefile b/drivers/virt/gunyah/Makefile index 3869fb7371df..94dc8e738911 100644 --- a/drivers/virt/gunyah/Makefile +++ b/drivers/virt/gunyah/Makefile @@ -1,4 +1,4 @@ # SPDX-License-Identifier: GPL-2.0-only -gunyah-y += sysfs.o device.o +gunyah-y += sysfs.o device.o msgq.o obj-$(CONFIG_GUNYAH) += gunyah.o \ No newline at end of file diff --git a/drivers/virt/gunyah/gunyah_private.h b/drivers/virt/gunyah/gunyah_private.h index 5f3832608020..2ade32bd9bdf 100644 --- a/drivers/virt/gunyah/gunyah_private.h +++ b/drivers/virt/gunyah/gunyah_private.h @@ -9,4 +9,7 @@ int __init gunyah_bus_init(void); void gunyah_bus_exit(void); +int __init gh_msgq_init(void); +void gh_msgq_exit(void); + #endif diff --git a/drivers/virt/gunyah/msgq.c b/drivers/virt/gunyah/msgq.c new file mode 100644 index 000000000000..1c79b3fff30c --- /dev/null +++ b/drivers/virt/gunyah/msgq.c @@ -0,0 +1,295 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (c) 2022 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include "gunyah_private.h" + +struct gh_msgq { + bool ready; + wait_queue_head_t wq; + spinlock_t lock; +}; + +static irqreturn_t gh_msgq_irq_handler(int irq, void *dev) +{ + struct gh_msgq *msgq = dev; + + spin_lock(&msgq->lock); + msgq->ready = true; + spin_unlock(&msgq->lock); + wake_up_interruptible(&msgq->wq); + + return IRQ_HANDLED; +} + +static int __gh_msgq_send(struct gunyah_device *ghdev, void *buff, size_t size, u64 tx_flags) +{ + unsigned long flags, gh_error; + struct gh_msgq *msgq = ghdev_get_drvdata(ghdev); + ssize_t ret; + bool ready; + + spin_lock_irqsave(&msgq->lock, flags); + arch_gh_hypercall(GH_HYPERCALL_MSGQ_SEND, 5, + ghdev->capid, size, (uintptr_t)buff, tx_flags, 0, + gh_error, ready); + switch (gh_error) { + case GH_ERROR_OK: + ret = 0; + msgq->ready = ready; + break; + case GH_ERROR_MSGQUEUE_FULL: + ret = -EAGAIN; + msgq->ready = false; + break; + default: + ret = gh_remap_error(gh_error); + break; + } + + spin_unlock_irqrestore(&msgq->lock, flags); + + return ret; +} + +/** + * gh_msgq_send() - Send a message to the client running on a different VM + * @client: The client descriptor that was obtained via gh_msgq_register() + * @buff: Pointer to the buffer where the received data must be placed + * @buff_size: The size of the buffer space available + * @flags: Optional flags to pass to receive the data. For the list of flags, + * see linux/gunyah/gh_msgq.h + * + * Returns: The number of bytes copied to buff. <0 if there was an error. + * + * Note: this function may sleep and should not be called from interrupt context + */ +int gh_msgq_send(struct gunyah_device *ghdev, void *buff, size_t size, unsigned long flags) +{ + struct gh_msgq *msgq = ghdev_get_drvdata(ghdev); + int ret; + u64 tx_flags = 0; + + if (flags & GH_MSGQ_TX_PUSH) + tx_flags |= GH_HYPERCALL_MSGQ_SEND_FLAGS_PUSH; + + do { + ret = __gh_msgq_send(ghdev, buff, size, tx_flags); + + if (ret == -EAGAIN) { + if (flags & GH_MSGQ_NONBLOCK) + goto out; + if (wait_event_interruptible(msgq->wq, msgq->ready)) + ret = -ERESTARTSYS; + } + } while (ret == -EAGAIN); + +out: + return ret; +} +EXPORT_SYMBOL_GPL(gh_msgq_send); + +static ssize_t __gh_msgq_recv(struct gunyah_device *ghdev, void *buff, size_t size) +{ + unsigned long flags, gh_error; + size_t recv_size; + struct gh_msgq *msgq = ghdev_get_drvdata(ghdev); + ssize_t ret; + bool ready; + + spin_lock_irqsave(&msgq->lock, flags); + + arch_gh_hypercall(GH_HYPERCALL_MSGQ_RECV, 4, + ghdev->capid, (uintptr_t)buff, size, 0, + gh_error, recv_size, ready); + switch (gh_error) { + case GH_ERROR_OK: + ret = recv_size; + msgq->ready = ready; + break; + case GH_ERROR_MSGQUEUE_EMPTY: + ret = -EAGAIN; + msgq->ready = false; + break; + default: + ret = gh_remap_error(gh_error); + break; + } + + spin_unlock_irqrestore(&msgq->lock, flags); + + return ret; +} + +/** + * gh_msgq_recv() - Receive a message from the client running on a different VM + * @client: The client descriptor that was obtained via gh_msgq_register() + * @buff: Pointer to the buffer where the received data must be placed + * @buff_size: The size of the buffer space available + * @flags: Optional flags to pass to receive the data. For the list of flags, + * see linux/gunyah/gh_msgq.h + * + * Returns: The number of bytes copied to buff. <0 if there was an error. + * + * Note: this function may sleep and should not be called from interrupt context + */ +ssize_t gh_msgq_recv(struct gunyah_device *ghdev, void *buff, size_t size, unsigned long flags) +{ + struct gh_msgq *msgq = ghdev_get_drvdata(ghdev); + ssize_t ret; + + do { + ret = __gh_msgq_recv(ghdev, buff, size); + + if (ret == -EAGAIN) { + if (flags & GH_MSGQ_NONBLOCK) + goto out; + if (wait_event_interruptible(msgq->wq, msgq->ready)) + ret = -ERESTARTSYS; + } + } while (ret == -EAGAIN); + +out: + return ret; +} +EXPORT_SYMBOL_GPL(gh_msgq_recv); + +static int gh_msgq_probe(struct gunyah_device *ghdev) +{ + struct gh_msgq *msgq; + + msgq = devm_kzalloc(&ghdev->dev, sizeof(*msgq), GFP_KERNEL); + ghdev_set_drvdata(ghdev, msgq); + + msgq->ready = true; /* Assume we can use the message queue right away */ + init_waitqueue_head(&msgq->wq); + + return devm_request_irq(&ghdev->dev, ghdev->irq, gh_msgq_irq_handler, 0, + dev_name(&ghdev->dev), msgq); +} + +static struct gunyah_driver gh_msgq_tx_driver = { + .driver = { + .name = "gh_msgq_tx", + .owner = THIS_MODULE, + }, + .type = GUNYAH_DEVICE_TYPE_MSGQ_TX, + .probe = gh_msgq_probe, +}; + +static struct gunyah_driver gh_msgq_rx_driver = { + .driver = { + .name = "gh_msgq_rx", + .owner = THIS_MODULE, + }, + .type = GUNYAH_DEVICE_TYPE_MSGQ_RX, + .probe = gh_msgq_probe, +}; + +static struct gunyah_device *gh_msgq_platform_probe_direction(struct platform_device *pdev, + u8 gh_type, int idx) +{ + int irq, ret; + u64 capid; + struct device_node *node = pdev->dev.of_node; + struct gunyah_device *ghdev; + + irq = platform_get_irq(pdev, idx); + if (irq < 0) { + dev_err(&pdev->dev, "Failed to get irq%d: %d\n", idx, irq); + return ERR_PTR(irq); + } + + ret = of_property_read_u64_index(node, "reg", idx, &capid); + if (ret) { + dev_err(&pdev->dev, "Failed to get capid%d: %d\n", idx, ret); + return ERR_PTR(ret); + } + + ghdev = gunyah_device_alloc(&pdev->dev, capid, gh_type); + ghdev->irq = irq; + ret = gunyah_device_add(ghdev); + if (ret) { + kfree(ghdev); + return ERR_PTR(ret); + } + + return ghdev; +} + +int gh_msgq_platform_host_attach(struct platform_device *pdev, struct gh_msgq_platform_host *host) +{ + struct gunyah_device *tx_dev = NULL, *rx_dev = NULL; + struct device_node *node = pdev->dev.of_node; + int idx = 0; + bool duplex; + + duplex = of_property_read_bool(node, "qcom,is-full-duplex"); + + if (duplex || of_property_read_bool(node, "qcom,is-sender")) { + tx_dev = gh_msgq_platform_probe_direction(pdev, GUNYAH_DEVICE_TYPE_MSGQ_TX, idx); + if (IS_ERR(tx_dev)) + return PTR_ERR(tx_dev); + idx++; + } + + if (duplex || of_property_read_bool(node, "qcom,is-receiver")) { + rx_dev = gh_msgq_platform_probe_direction(pdev, GUNYAH_DEVICE_TYPE_MSGQ_RX, idx); + if (IS_ERR(rx_dev)) { + if (!IS_ERR_OR_NULL(tx_dev)) + gunyah_device_remove(tx_dev); + return PTR_ERR(rx_dev); + } + } + + host->tx = tx_dev; + host->rx = rx_dev; + return 0; +} + +void gh_msgq_platform_host_unattach(struct gh_msgq_platform_host *host) +{ + if (host->tx) { + gunyah_device_remove(host->tx); + host->tx = NULL; + } + if (host->rx) { + gunyah_device_remove(host->rx); + host->rx = NULL; + } +} + +int __init gh_msgq_init(void) +{ + int ret; + + ret = gunyah_register_driver(&gh_msgq_tx_driver); + if (ret) + return ret; + + ret = gunyah_register_driver(&gh_msgq_rx_driver); + if (ret) + goto err_rx; + + return ret; +err_rx: + gunyah_unregister_driver(&gh_msgq_tx_driver); + return ret; +} + +void gh_msgq_exit(void) +{ + gunyah_unregister_driver(&gh_msgq_rx_driver); + gunyah_unregister_driver(&gh_msgq_tx_driver); +} diff --git a/drivers/virt/gunyah/sysfs.c b/drivers/virt/gunyah/sysfs.c index d66e6275fe32..7bf39fe1b6e6 100644 --- a/drivers/virt/gunyah/sysfs.c +++ b/drivers/virt/gunyah/sysfs.c @@ -59,6 +59,8 @@ static ssize_t features_show(struct kobject *kobj, struct kobj_attribute *attr, if (GH_IDENTIFY_PARTITION_CSPACE(gunyah_api.flags)) len += sysfs_emit_at(buffer, len, "cspace "); + if (GH_IDENTIFY_MSGQUEUE(gunyah_api.flags)) + len += sysfs_emit_at(buffer, len, "message-queue "); len += sysfs_emit_at(buffer, len, "\n"); return len; @@ -118,7 +120,13 @@ static int __init gunyah_init(void) if (ret) goto err_sysfs; + ret = gh_msgq_init(); + if (ret) + goto err_bus; + return ret; +err_bus: + gunyah_bus_exit(); err_sysfs: gh_sysfs_unregister(); return ret; @@ -127,6 +135,7 @@ module_init(gunyah_init); static void __exit gunyah_exit(void) { + gh_msgq_exit(); gunyah_bus_exit(); gh_sysfs_unregister(); } diff --git a/include/linux/gunyah.h b/include/linux/gunyah.h index f169c78881cb..66c1dba73cc5 100644 --- a/include/linux/gunyah.h +++ b/include/linux/gunyah.h @@ -10,6 +10,7 @@ #include #include #include +#include typedef u64 gh_capid_t; @@ -116,4 +117,22 @@ struct gunyah_driver { int gunyah_register_driver(struct gunyah_driver *ghdrv); void gunyah_unregister_driver(struct gunyah_driver *ghdrv); +#define GH_MSGQ_MAX_MSG_SIZE 1024 + +/* Possible flags to pass for Tx or Rx */ +#define GH_MSGQ_TX_PUSH BIT(0) +#define GH_MSGQ_NONBLOCK BIT(32) + +int gh_msgq_send(struct gunyah_device *ghdev, void *buff, size_t size, unsigned long flags); +ssize_t gh_msgq_recv(struct gunyah_device *ghdev, void *buff, size_t size, unsigned long flags); + +struct gh_msgq_platform_host { + struct gunyah_device *tx; + struct gunyah_device *rx; +}; + +int gh_msgq_platform_host_attach(struct platform_device *pdev, struct gh_msgq_platform_host *host); +void gh_msgq_platform_host_unattach(struct gh_msgq_platform_host *host); + + #endif From patchwork Wed Feb 23 23:37:26 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Elliot Berman X-Patchwork-Id: 12757674 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 46962C433FE for ; Wed, 23 Feb 2022 23:38:10 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S244920AbiBWXig (ORCPT ); Wed, 23 Feb 2022 18:38:36 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58886 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S244888AbiBWXi2 (ORCPT ); Wed, 23 Feb 2022 18:38:28 -0500 Received: from alexa-out-sd-01.qualcomm.com (alexa-out-sd-01.qualcomm.com [199.106.114.38]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 68B925A5A1; Wed, 23 Feb 2022 15:37:58 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; i=@quicinc.com; q=dns/txt; s=qcdkim; t=1645659478; x=1677195478; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=i+uc4Od5DQuCVegoL2U1VxjxpHjuf1Mi0ziMxBIZ3AM=; b=x0xb8xZnCsuiDMiZysRRzBccNeE/qiJWJvk69aiF5a2mMzJGvkXvW31s GudmKbiW2Pw/q5xGVvj8DbzQNl//jqPhI9vIbHpbgYdY1YvzoJmI2Y0C/ X5XvrwCI4KjuGzchVQci6EMf3JAatOkc3MoeB58nQvdDL6AUrMILLh7/z E=; Received: from unknown (HELO ironmsg02-sd.qualcomm.com) ([10.53.140.142]) by alexa-out-sd-01.qualcomm.com with ESMTP; 23 Feb 2022 15:37:58 -0800 X-QCInternal: smtphost Received: from nasanex01b.na.qualcomm.com ([10.46.141.250]) by ironmsg02-sd.qualcomm.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 23 Feb 2022 15:37:58 -0800 Received: from hu-eberman-lv.qualcomm.com (10.49.16.6) by nasanex01b.na.qualcomm.com (10.46.141.250) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.15; Wed, 23 Feb 2022 15:37:57 -0800 From: Elliot Berman To: Bjorn Andersson , CC: Elliot Berman , Trilok Soni , Murali Nalajala , Srivatsa Vaddagiri , Carl van Schaik , Andy Gross , Subject: [PATCH 08/11] gunyah: rsc_mgr: Add resource manager RPC core Date: Wed, 23 Feb 2022 15:37:26 -0800 Message-ID: <20220223233729.1571114-9-quic_eberman@quicinc.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220223233729.1571114-1-quic_eberman@quicinc.com> References: <20220223233729.1571114-1-quic_eberman@quicinc.com> MIME-Version: 1.0 X-Originating-IP: [10.49.16.6] X-ClientProxiedBy: nalasex01a.na.qualcomm.com (10.47.209.196) To nasanex01b.na.qualcomm.com (10.46.141.250) Precedence: bulk List-ID: X-Mailing-List: linux-arm-msm@vger.kernel.org The resource manager is a special virtual machine which is always running on a Gunyah system. It provides APIs for creating and destroying VMs, secure memory management, sharing/lending of memory between VMs, and setup of inter-VM communication. Calls to the resource manager are made via message queues. This patch implements the basic probing and RPC mechanism to make those API calls. Request/response calls can be made with gh_rm_call. Drivers can also register to notifications pushed by RM via gh_rm_register_notifier Specific API calls that resource manager supports will be implemented in subsequent patches. Signed-off-by: Elliot Berman --- MAINTAINERS | 2 +- drivers/virt/gunyah/Kconfig | 1 + drivers/virt/gunyah/Makefile | 1 + drivers/virt/gunyah/gunyah_private.h | 3 + drivers/virt/gunyah/rsc_mgr.c | 574 +++++++++++++++++++++++++++ drivers/virt/gunyah/rsc_mgr.h | 34 ++ drivers/virt/gunyah/sysfs.c | 7 + include/linux/gunyah_rsc_mgr.h | 29 ++ 8 files changed, 650 insertions(+), 1 deletion(-) create mode 100644 drivers/virt/gunyah/rsc_mgr.c create mode 100644 drivers/virt/gunyah/rsc_mgr.h create mode 100644 include/linux/gunyah_rsc_mgr.h diff --git a/MAINTAINERS b/MAINTAINERS index 10c59c8767ff..b05359adc4f7 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -8405,7 +8405,7 @@ F: Documentation/devicetree/bindings/gunyah/ F: Documentation/virt/gunyah/ F: arch/arm64/include/asm/gunyah/ F: drivers/virt/gunyah/ -F: include/linux/gunyah.h +F: include/linux/gunyah*.h H8/300 ARCHITECTURE M: Yoshinori Sato diff --git a/drivers/virt/gunyah/Kconfig b/drivers/virt/gunyah/Kconfig index e88289963518..2ef4887e280d 100644 --- a/drivers/virt/gunyah/Kconfig +++ b/drivers/virt/gunyah/Kconfig @@ -4,6 +4,7 @@ config GUNYAH tristate "Gunyah Virtualization drivers" depends on ARM64 select SYS_HYPERVISOR + select AUXILIARY_BUS help The Gunyah drivers are the helper interfaces that runs in a guest VM such as basic inter-VM IPC and signaling mechanism,s and higher level diff --git a/drivers/virt/gunyah/Makefile b/drivers/virt/gunyah/Makefile index 94dc8e738911..86655bca8944 100644 --- a/drivers/virt/gunyah/Makefile +++ b/drivers/virt/gunyah/Makefile @@ -1,4 +1,5 @@ # SPDX-License-Identifier: GPL-2.0-only gunyah-y += sysfs.o device.o msgq.o +gunyah-y += rsc_mgr.o obj-$(CONFIG_GUNYAH) += gunyah.o \ No newline at end of file diff --git a/drivers/virt/gunyah/gunyah_private.h b/drivers/virt/gunyah/gunyah_private.h index 2ade32bd9bdf..6483ffa8c15d 100644 --- a/drivers/virt/gunyah/gunyah_private.h +++ b/drivers/virt/gunyah/gunyah_private.h @@ -12,4 +12,7 @@ void gunyah_bus_exit(void); int __init gh_msgq_init(void); void gh_msgq_exit(void); +int __init gh_rsc_mgr_init(void); +void gh_rsc_mgr_exit(void); + #endif diff --git a/drivers/virt/gunyah/rsc_mgr.c b/drivers/virt/gunyah/rsc_mgr.c new file mode 100644 index 000000000000..c8c4e1617566 --- /dev/null +++ b/drivers/virt/gunyah/rsc_mgr.c @@ -0,0 +1,574 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (c) 2022 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#define pr_fmt(fmt) "gh_rsc_mgr: " fmt + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include "gunyah_private.h" +#include "rsc_mgr.h" + +/* Resource Manager Header */ +struct gh_rm_rpc_hdr { + u8 version:4, + hdr_words:4; + u8 type:2, + fragments:6; + u16 seq; + u32 msg_id; +} __packed; + +/* Standard reply header */ +struct gh_rm_rpc_reply_hdr { + struct gh_rm_rpc_hdr rpc_hdr; + u32 err_code; +} __packed; + +/* RPC Header versions */ +#define GH_RM_RPC_HDR_VERSION_ONE 0x1 + +/* RPC Header words */ +#define GH_RM_RPC_HDR_WORDS 0x2 + +/* RPC Message types */ +#define GH_RM_RPC_TYPE_CONT 0x0 +#define GH_RM_RPC_TYPE_REQ 0x1 +#define GH_RM_RPC_TYPE_RPLY 0x2 +#define GH_RM_RPC_TYPE_NOTIF 0x3 + +#define GH_RM_MAX_NUM_FRAGMENTS 62 + +#define GH_RM_MAX_MSG_SIZE (GH_MSGQ_MAX_MSG_SIZE - sizeof(struct gh_rm_rpc_hdr)) + +/** + * struct gh_rm_connection - Represents a complete message from resource manager + * @buff: Combined payload of all the fragments (i.e. msg headers stripped off). + * @size: Size of the payload. + * @ret: Linux return code, set in case there was an error processing the connection. + * @msg_id: Message ID from the header. + * @type: GH_RM_RPC_TYPE_RPLY or GH_RM_RPC_TYPE_NOTIF. + * @num_fragments: total number of fragments expected to be received for this connection. + * @fragments_recieved: fragments received so far. + * @rm_error: For request/reply sequences with standard replies. + * @seq: Sequence ID for the main message. + */ +struct gh_rm_connection { + void *buff; + size_t size; + int ret; + u32 msg_id; + u8 type; + + u8 num_fragments; + u8 fragments_received; + + /* only for req/reply sequence */ + u32 rm_error; + u16 seq; + struct completion seq_done; +}; + +struct gh_rm_notif_complete { + struct gh_rm_connection *conn; + struct work_struct work; +}; + +struct gh_rsc_mgr { + struct task_struct *recv_task; + struct gh_msgq_platform_host msgq; + + struct idr call_idr; + struct mutex call_idr_lock; + + struct mutex send_lock; +}; + +static struct gh_rsc_mgr *__rsc_mgr; +SRCU_NOTIFIER_HEAD_STATIC(gh_rm_notifier); + +static struct gh_rm_connection *gh_rm_alloc_connection(u32 msg_id, u8 type) +{ + struct gh_rm_connection *connection; + + connection = kzalloc(sizeof(*connection), GFP_KERNEL); + if (!connection) + return NULL; + + connection->type = type; + connection->msg_id = msg_id; + + return connection; +} + +/** + * gh_rm_init_connection_buff() - Fills the first message for a connection. + */ +static int gh_rm_init_connection_buff(struct gh_rm_connection *connection, void *msg, + size_t hdr_size, size_t payload_size) +{ + struct gh_rm_rpc_hdr *hdr = msg; + size_t max_buf_size; + + connection->num_fragments = hdr->fragments; + connection->fragments_received = 0; + connection->type = hdr->type; + + /* There's not going to be any payload, no need to allocate buffer. */ + if (!payload_size && !connection->num_fragments) + return 0; + + /* + * maximum payload size is GH_MSGQ_MAX_MSG_SIZE - hdr_size + * and can received (hdr->fragments + 1) of those + */ + max_buf_size = (GH_MSGQ_MAX_MSG_SIZE - hdr_size) * (hdr->fragments + 1); + + connection->buff = kzalloc(max_buf_size, GFP_KERNEL); + if (!connection->buff) + return -ENOMEM; + + memcpy(connection->buff, msg + hdr_size, payload_size); + connection->size = payload_size; + return 0; +} + +static void gh_rm_notif_work(struct work_struct *work) +{ + struct gh_rm_notif_complete *notif = container_of(work, struct gh_rm_notif_complete, work); + struct gh_rm_connection *connection = notif->conn; + u32 notif_id = connection->msg_id; + struct gh_rm_notification notification = { + .buff = connection->buff, + .size = connection->size, + }; + + srcu_notifier_call_chain(&gh_rm_notifier, notif_id, ¬ification); + + kfree(connection->buff); + kfree(connection); + kfree(notif); +} + +static struct gh_rm_connection *gh_rm_process_notif(struct gh_rsc_mgr *rsc_mgr, + void *msg, size_t msg_size) +{ + struct gh_rm_rpc_hdr *hdr = msg; + struct gh_rm_connection *connection; + + connection = gh_rm_alloc_connection(hdr->msg_id, hdr->type); + if (!connection) { + pr_err("Failed to alloc connection for notification, dropping.\n"); + return NULL; + } + + if (gh_rm_init_connection_buff(connection, msg, sizeof(*hdr), msg_size - sizeof(*hdr))) { + pr_err("Failed to alloc connection buffer for notification, dropping.\n"); + kfree(connection); + return NULL; + } + + return connection; +} + +static struct gh_rm_connection *gh_rm_process_rply(struct gh_rsc_mgr *rsc_mgr, + void *msg, size_t msg_size) +{ + struct gh_rm_rpc_reply_hdr *reply_hdr = msg; + struct gh_rm_rpc_hdr *hdr = msg; + struct gh_rm_connection *connection; + + if (mutex_lock_interruptible(&rsc_mgr->call_idr_lock)) + return ERR_PTR(-ERESTARTSYS); + + connection = idr_find(&rsc_mgr->call_idr, hdr->seq); + mutex_unlock(&rsc_mgr->call_idr_lock); + + if (!connection) { + pr_err("Failed to find connection for sequence %u\n", hdr->seq); + return NULL; + } + if (connection->msg_id != hdr->msg_id) { + pr_err("Reply for sequence %u expected msg_id: %x but got %x\n", hdr->seq, + connection->msg_id, hdr->msg_id); + /* + * Don't complete connection and error the client, maybe resource manager will + * send us the expected reply sequence soon. + */ + return NULL; + } + + if (gh_rm_init_connection_buff(connection, msg, sizeof(*reply_hdr), + msg_size - sizeof(*reply_hdr))) { + pr_err("Failed to alloc connection buffer for sequence %d\n", hdr->seq); + /* Send connection complete and error the client. */ + connection->ret = -ENOMEM; + complete(&connection->seq_done); + return NULL; + } + + connection->rm_error = reply_hdr->err_code; + return connection; +} + +static void gh_rm_process_cont(struct gh_rm_connection *connection, void *msg, size_t msg_size) +{ + struct gh_rm_rpc_hdr *hdr = msg; + size_t payload_size = msg_size - sizeof(*hdr); + + /* + * hdr->fragments and hdr->msg_id preserves the value from first reply or notif message. + * For sake of sanity, check if it's still intact. + */ + if (connection->msg_id != hdr->msg_id) + pr_warn("Appending mismatched continuation with id %d to connection with id %d\n", + hdr->msg_id, connection->msg_id); + if (connection->num_fragments != hdr->fragments) + pr_warn("Number of fragments mismatch for seq: %d\n", hdr->seq); + + memcpy(connection->buff + connection->size, msg + sizeof(*hdr), payload_size); + connection->size += payload_size; + connection->fragments_received++; +} + +static bool gh_rm_complete_connection(struct gh_rm_connection *connection) +{ + struct gh_rm_notif_complete *notif_work; + + if (!connection) + return false; + + if (connection->fragments_received != connection->num_fragments) + return false; + + switch (connection->type) { + case GH_RM_RPC_TYPE_RPLY: + complete(&connection->seq_done); + break; + case GH_RM_RPC_TYPE_NOTIF: + notif_work = kzalloc(sizeof(*notif_work), GFP_KERNEL); + if (notif_work == NULL) + break; + + notif_work->conn = connection; + INIT_WORK(¬if_work->work, gh_rm_notif_work); + + schedule_work(¬if_work->work); + break; + default: + pr_err("Invalid message type (%d) received\n", connection->type); + break; + } + + return true; +} + +static int gh_rm_recv_task_fn(void *data) +{ + struct gh_rsc_mgr *rsc_mgr = data; + struct gh_rm_connection *connection = NULL; + struct gh_rm_rpc_hdr *hdr = NULL; + ssize_t msg_size; + void *msg; + + msg = kzalloc(GH_MSGQ_MAX_MSG_SIZE, GFP_KERNEL); + if (!msg) + return -ENOMEM; + + while (!kthread_should_stop()) { + /* Block until a new message is received */ + msg_size = gh_msgq_recv(rsc_mgr->msgq.rx, msg, GH_MSGQ_MAX_MSG_SIZE, 0); + if (msg_size < 0) { + pr_err("Failed to receive the message: %ld\n", msg_size); + continue; + } else if (msg_size <= sizeof(struct gh_rm_rpc_hdr)) { + pr_err("Invalid message size received: %ld is too small\n", msg_size); + continue; + } + + hdr = msg; + switch (hdr->type) { + case GH_RM_RPC_TYPE_NOTIF: + if (connection) { + /* Not possible per protocol. Do something better than BUG_ON */ + pr_warn("Received start of new notification without finishing existing message series.\n"); + kfree(connection->buff); + kfree(connection); + } + connection = gh_rm_process_notif(rsc_mgr, msg, msg_size); + break; + case GH_RM_RPC_TYPE_RPLY: + if (connection) { + /* Not possible per protocol. Do something better than BUG_ON */ + pr_warn("Received start of new reply without finishing existing message series.\n"); + kfree(connection->buff); + kfree(connection); + } + connection = gh_rm_process_rply(rsc_mgr, msg, msg_size); + break; + case GH_RM_RPC_TYPE_CONT: + if (!connection) { + pr_warn("Received a continuation message without receiving initial message\n"); + break; + } + gh_rm_process_cont(connection, msg, msg_size); + break; + default: + pr_err("Invalid message type (%d) received\n", hdr->type); + continue; + } + + if (gh_rm_complete_connection(connection)) + connection = NULL; + } + + return 0; +} + +static int gh_rm_send_request(struct gh_rsc_mgr *rsc_mgr, u32 message_id, + const void *req_buff, size_t req_buff_size, + struct gh_rm_connection *connection) +{ + size_t buff_size_remaining = req_buff_size; + const void *req_buff_curr = req_buff; + struct gh_rm_rpc_hdr *hdr; + unsigned long tx_flags; + u32 num_fragments = 0; + size_t payload_size; + void *msg; + int i, ret = 0; + + if (req_buff_size > GH_RM_MAX_MSG_SIZE) + num_fragments = req_buff_size / GH_RM_MAX_MSG_SIZE; + + if (num_fragments > GH_RM_MAX_NUM_FRAGMENTS) { + pr_err("Limit exceeded for the number of fragments: %u\n", num_fragments); + return -E2BIG; + } + + /* + * The above calculation also includes the count for the 'request' packet. + * Exclude it as the header needs to fill the num. of fragments to follow. + */ + if (num_fragments) + num_fragments--; + + if (mutex_lock_interruptible(&rsc_mgr->send_lock)) + return -ERESTARTSYS; + + msg = kzalloc(GH_MSGQ_MAX_MSG_SIZE, GFP_KERNEL); + if (!msg) { + mutex_unlock(&rsc_mgr->send_lock); + return -ENOMEM; + } + + /* Consider also the 'request' packet for the loop count */ + for (i = 0; i <= num_fragments; i++) { + if (buff_size_remaining > GH_RM_MAX_MSG_SIZE) { + payload_size = GH_RM_MAX_MSG_SIZE; + buff_size_remaining -= payload_size; + } else { + payload_size = buff_size_remaining; + } + + memset(msg, 0, GH_MSGQ_MAX_MSG_SIZE); + + /* Fill header */ + hdr = msg; + hdr->version = GH_RM_RPC_HDR_VERSION_ONE; + hdr->hdr_words = GH_RM_RPC_HDR_WORDS; + hdr->type = i == 0 ? GH_RM_RPC_TYPE_REQ : GH_RM_RPC_TYPE_CONT; + hdr->fragments = num_fragments; + hdr->seq = connection->seq; + hdr->msg_id = message_id; + + /* Copy payload */ + memcpy(msg + sizeof(*hdr), req_buff_curr, payload_size); + req_buff_curr += payload_size; + + /* Force the last fragment to be sent immediately to the receiver */ + tx_flags = (i == num_fragments) ? GH_MSGQ_TX_PUSH : 0; + + ret = gh_msgq_send(rsc_mgr->msgq.tx, msg, sizeof(*hdr) + payload_size, tx_flags); + + if (ret) + break; + } + + mutex_unlock(&rsc_mgr->send_lock); + return ret; +} + +/** + * gh_rm_call: Achieve request-response type communication with RPC + * @message_id: The RM RPC message-id + * @req_buff: Request buffer that contains the payload + * @req_buff_size: Total size of the payload + * @resp_buf: Pointer to a response buffer + * @resp_buff_size: Size of the response buffer + * @reply_err_code: Returns Gunyah standard error code for the response + * + * Make a request to the RM-VM and wait for reply back. For a successful + * response, the function returns the payload. The size of the payload is set in resp_buff_size. + * The resp_buf should be freed by the caller. + * + * Context: Process context. Will sleep waiting for reply. + * Return: >0 is standard reply error from RM. <0 on internal error. + */ +int gh_rm_call(u32 message_id, void *req_buff, size_t req_buff_size, + void **resp_buf, size_t *resp_buff_size) +{ + struct gh_rm_connection *connection; + int ret; + struct gh_rsc_mgr *rsc_mgr = __rsc_mgr; + + /* messaged_id 0 is reserved */ + if (!message_id) + return -EINVAL; + + if (!rsc_mgr) + return -EPROBE_DEFER; + + connection = gh_rm_alloc_connection(message_id, GH_RM_RPC_TYPE_RPLY); + if (!connection) + return -ENOMEM; + + init_completion(&connection->seq_done); + + /* Allocate a new seq number for this connection */ + if (mutex_lock_interruptible(&rsc_mgr->call_idr_lock)) { + kfree(connection); + return -ERESTARTSYS; + } + connection->seq = idr_alloc_cyclic(&rsc_mgr->call_idr, connection, 0, U16_MAX, GFP_KERNEL); + mutex_unlock(&rsc_mgr->call_idr_lock); + + /* Send the request to the Resource Manager */ + ret = gh_rm_send_request(rsc_mgr, message_id, req_buff, req_buff_size, connection); + if (ret < 0) + goto out; + + /* Wait for response */ + wait_for_completion(&connection->seq_done); + + if (connection->ret) { + ret = connection->ret; + kfree(connection->buff); + goto out; + } + + mutex_lock(&rsc_mgr->call_idr_lock); + idr_remove(&rsc_mgr->call_idr, connection->seq); + mutex_unlock(&rsc_mgr->call_idr_lock); + + *resp_buf = connection->buff; + *resp_buff_size = connection->size; + ret = connection->rm_error; + +out: + kfree(connection); + return ret; +} + +int gh_rm_register_notifier(struct notifier_block *nb) +{ + return srcu_notifier_chain_register(&gh_rm_notifier, nb); +} +EXPORT_SYMBOL_GPL(gh_rm_register_notifier); + +int gh_rm_unregister_notifier(struct notifier_block *nb) +{ + return srcu_notifier_chain_unregister(&gh_rm_notifier, nb); +} +EXPORT_SYMBOL_GPL(gh_rm_unregister_notifier); + +static int gh_rm_drv_probe(struct platform_device *pdev) +{ + struct device *dev = &pdev->dev; + struct gh_rsc_mgr *rsc_mgr; + struct list_head *l, *n; + int ret, i; + + rsc_mgr = devm_kzalloc(&pdev->dev, sizeof(*rsc_mgr), GFP_KERNEL); + if (!rsc_mgr) + return -ENOMEM; + platform_set_drvdata(pdev, rsc_mgr); + + mutex_init(&rsc_mgr->call_idr_lock); + idr_init(&rsc_mgr->call_idr); + mutex_init(&rsc_mgr->send_lock); + + ret = gh_msgq_platform_host_attach(pdev, &rsc_mgr->msgq); + if (ret) + return ret; + if (!rsc_mgr->msgq.tx || !rsc_mgr->msgq.rx) { + dev_warn(dev, "Expected both tx and rx message queues\n"); + ret = -ENODEV; + goto err_msgq; + } + + rsc_mgr->recv_task = kthread_run(gh_rm_recv_task_fn, rsc_mgr, "gh_rm_recv_task"); + if (IS_ERR_OR_NULL(rsc_mgr->recv_task)) { + ret = PTR_ERR(rsc_mgr->recv_task); + goto err_msgq; + } + + __rsc_mgr = rsc_mgr; + + return 0; + +err_msgq: + gh_msgq_platform_host_unattach(&rsc_mgr->msgq); + return ret; +} + +static int gh_rm_drv_remove(struct platform_device *pdev) +{ + struct gh_rsc_mgr *rsc_mgr = platform_get_drvdata(pdev); + + gh_msgq_platform_host_unattach(&rsc_mgr->msgq); + + return 0; +} + +static const struct of_device_id gh_rm_of_match[] = { + { .compatible = "qcom,resource-manager-1-0" }, + { } +}; +MODULE_DEVICE_TABLE(of, gh_rm_of_match); + +static struct platform_driver gh_rsc_mgr_driver = { + .probe = gh_rm_drv_probe, + .remove = gh_rm_drv_remove, + .driver = { + .name = "gh_rsc_mgr", + .of_match_table = gh_rm_of_match, + }, +}; + +int __init gh_rsc_mgr_init(void) +{ + return platform_driver_register(&gh_rsc_mgr_driver); +} + +void gh_rsc_mgr_exit(void) +{ + platform_driver_unregister(&gh_rsc_mgr_driver); +} diff --git a/drivers/virt/gunyah/rsc_mgr.h b/drivers/virt/gunyah/rsc_mgr.h new file mode 100644 index 000000000000..e4f2499267bf --- /dev/null +++ b/drivers/virt/gunyah/rsc_mgr.h @@ -0,0 +1,34 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (c) 2022 Qualcomm Innovation Center, Inc. All rights reserved. + */ +#ifndef __GH_RSC_MGR_PRIV_H +#define __GH_RSC_MGR_PRIV_H + +#include + +/* RM Error codes */ +#define GH_RM_ERROR_OK 0x0 +#define GH_RM_ERROR_UNIMPLEMENTED 0xFFFFFFFF +#define GH_RM_ERROR_NOMEM 0x1 +#define GH_RM_ERROR_NORESOURCE 0x2 +#define GH_RM_ERROR_DENIED 0x3 +#define GH_RM_ERROR_INVALID 0x4 +#define GH_RM_ERROR_BUSY 0x5 +#define GH_RM_ERROR_ARGUMENT_INVALID 0x6 +#define GH_RM_ERROR_HANDLE_INVALID 0x7 +#define GH_RM_ERROR_VALIDATE_FAILED 0x8 +#define GH_RM_ERROR_MAP_FAILED 0x9 +#define GH_RM_ERROR_MEM_INVALID 0xA +#define GH_RM_ERROR_MEM_INUSE 0xB +#define GH_RM_ERROR_MEM_RELEASED 0xC +#define GH_RM_ERROR_VMID_INVALID 0xD +#define GH_RM_ERROR_LOOKUP_FAILED 0xE +#define GH_RM_ERROR_IRQ_INVALID 0xF +#define GH_RM_ERROR_IRQ_INUSE 0x10 +#define GH_RM_ERROR_IRQ_RELEASED 0x11 + +int gh_rm_call(u32 message_id, void *req_buff, size_t req_buff_size, + void **resp_buf, size_t *resp_buff_size); + +#endif diff --git a/drivers/virt/gunyah/sysfs.c b/drivers/virt/gunyah/sysfs.c index 7bf39fe1b6e6..389feeb5cc03 100644 --- a/drivers/virt/gunyah/sysfs.c +++ b/drivers/virt/gunyah/sysfs.c @@ -124,7 +124,13 @@ static int __init gunyah_init(void) if (ret) goto err_bus; + ret = gh_rsc_mgr_init(); + if (ret) + goto err_msgq; + return ret; +err_msgq: + gh_msgq_exit(); err_bus: gunyah_bus_exit(); err_sysfs: @@ -135,6 +141,7 @@ module_init(gunyah_init); static void __exit gunyah_exit(void) { + gh_rsc_mgr_exit(); gh_msgq_exit(); gunyah_bus_exit(); gh_sysfs_unregister(); diff --git a/include/linux/gunyah_rsc_mgr.h b/include/linux/gunyah_rsc_mgr.h new file mode 100644 index 000000000000..015bd851e1a3 --- /dev/null +++ b/include/linux/gunyah_rsc_mgr.h @@ -0,0 +1,29 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (c) 2022 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#ifndef _GUNYAH_RSC_MGR_H +#define _GUNYAH_RSC_MGR_H + +#include +#include +#include + +typedef u16 gh_vmid_t; +typedef u32 gh_virq_handle_t; + +#define GH_VMID_INVAL U16_MAX + +/* Gunyah recognizes VMID0 as an alias to the current VM's ID */ +#define GH_VMID_SELF 0 + +struct gh_rm_notification { + const void *buff; + const size_t size; +}; + +int gh_rm_register_notifier(struct notifier_block *nb); +int gh_rm_unregister_notifier(struct notifier_block *nb); + +#endif From patchwork Wed Feb 23 23:37:27 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Elliot Berman X-Patchwork-Id: 12757673 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 744EFC433F5 for ; Wed, 23 Feb 2022 23:38:09 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S244903AbiBWXig (ORCPT ); Wed, 23 Feb 2022 18:38:36 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58890 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S244889AbiBWXi2 (ORCPT ); Wed, 23 Feb 2022 18:38:28 -0500 Received: from alexa-out-sd-02.qualcomm.com (alexa-out-sd-02.qualcomm.com [199.106.114.39]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D501C5A5A5; Wed, 23 Feb 2022 15:37:58 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; i=@quicinc.com; q=dns/txt; s=qcdkim; t=1645659478; x=1677195478; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=2VUkdZNWBUow2l5oaAUbfqT7RLs4Yas3+07dDbQamTk=; b=BWguezkOMEDur/3XOhGnVawv8Pgs3XzyzvEmGk2VyEWNOVQQuOXF5+Z9 6qFI9EXfRnnYaihkPTRr3SbBxn2IY7Mc998Ic/TR/+HtWGH0g8IxYcGAy MiLEnZyexJhstT/NrOrMcM9zHquxS3X85II53eeFFHgPHeiIY3nmCMmPi 0=; Received: from unknown (HELO ironmsg05-sd.qualcomm.com) ([10.53.140.145]) by alexa-out-sd-02.qualcomm.com with ESMTP; 23 Feb 2022 15:37:58 -0800 X-QCInternal: smtphost Received: from nasanex01b.na.qualcomm.com ([10.46.141.250]) by ironmsg05-sd.qualcomm.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 23 Feb 2022 15:37:58 -0800 Received: from hu-eberman-lv.qualcomm.com (10.49.16.6) by nasanex01b.na.qualcomm.com (10.46.141.250) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.15; Wed, 23 Feb 2022 15:37:58 -0800 From: Elliot Berman To: Bjorn Andersson , CC: Elliot Berman , Trilok Soni , Murali Nalajala , Srivatsa Vaddagiri , Carl van Schaik , Andy Gross , Subject: [PATCH 09/11] gunyah: rsc_mgr: Add auxiliary devices for console Date: Wed, 23 Feb 2022 15:37:27 -0800 Message-ID: <20220223233729.1571114-10-quic_eberman@quicinc.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220223233729.1571114-1-quic_eberman@quicinc.com> References: <20220223233729.1571114-1-quic_eberman@quicinc.com> MIME-Version: 1.0 X-Originating-IP: [10.49.16.6] X-ClientProxiedBy: nalasex01a.na.qualcomm.com (10.47.209.196) To nasanex01b.na.qualcomm.com (10.46.141.250) Precedence: bulk List-ID: X-Mailing-List: linux-arm-msm@vger.kernel.org Gunyah resource manager exposes a concrete functionalities which complicate a single resource manager driver. Use auxiliary bus to help split high level functions for the resource manager and keep the primary resource manager driver focused on the RPC with RM itself. Delegate Resource Manager's console functionality to the auxiliary bus. Signed-off-by: Elliot Berman --- drivers/virt/gunyah/rsc_mgr.c | 58 +++++++++++++++++++++++++++++++++++ 1 file changed, 58 insertions(+) diff --git a/drivers/virt/gunyah/rsc_mgr.c b/drivers/virt/gunyah/rsc_mgr.c index c8c4e1617566..9ca63053dd38 100644 --- a/drivers/virt/gunyah/rsc_mgr.c +++ b/drivers/virt/gunyah/rsc_mgr.c @@ -91,6 +91,11 @@ struct gh_rm_notif_complete { struct work_struct work; }; +struct gh_rsc_mgr_adev { + struct auxiliary_device adev; + struct list_head list; +}; + struct gh_rsc_mgr { struct task_struct *recv_task; struct gh_msgq_platform_host msgq; @@ -99,6 +104,13 @@ struct gh_rsc_mgr { struct mutex call_idr_lock; struct mutex send_lock; + + struct list_head adevs; +}; + +/* List of auxiliary devices which resource manager creates */ +static const char *adev_names[] = { + "console", }; static struct gh_rsc_mgr *__rsc_mgr; @@ -499,10 +511,19 @@ int gh_rm_unregister_notifier(struct notifier_block *nb) } EXPORT_SYMBOL_GPL(gh_rm_unregister_notifier); +static void gh_rm_adev_release(struct device *dev) +{ + struct gh_rsc_mgr_adev *rm_adev = container_of(dev, struct gh_rsc_mgr_adev, adev.dev); + + list_del(&rm_adev->list); + kfree(rm_adev); +} + static int gh_rm_drv_probe(struct platform_device *pdev) { struct device *dev = &pdev->dev; struct gh_rsc_mgr *rsc_mgr; + struct gh_rsc_mgr_adev *rm_adev; struct list_head *l, *n; int ret, i; @@ -514,6 +535,7 @@ static int gh_rm_drv_probe(struct platform_device *pdev) mutex_init(&rsc_mgr->call_idr_lock); idr_init(&rsc_mgr->call_idr); mutex_init(&rsc_mgr->send_lock); + INIT_LIST_HEAD(&rsc_mgr->adevs); ret = gh_msgq_platform_host_attach(pdev, &rsc_mgr->msgq); if (ret) @@ -530,10 +552,38 @@ static int gh_rm_drv_probe(struct platform_device *pdev) goto err_msgq; } + for (i = 0; i < ARRAY_SIZE(adev_names); i++) { + rm_adev = kzalloc(sizeof(*rm_adev), GFP_KERNEL); + + rm_adev->adev.dev.parent = dev; + rm_adev->adev.dev.release = gh_rm_adev_release; + rm_adev->adev.name = adev_names[i]; + ret = auxiliary_device_init(&rm_adev->adev); + if (ret) { + kfree(rm_adev); + goto err_adevs; + } + + list_add(&rm_adev->list, &rsc_mgr->adevs); + + ret = auxiliary_device_add(&rm_adev->adev); + if (ret) { + auxiliary_device_uninit(&rm_adev->adev); + goto err_adevs; + } + } + __rsc_mgr = rsc_mgr; return 0; +err_adevs: + list_for_each_safe(l, n, &rsc_mgr->adevs) { + rm_adev = container_of(l, struct gh_rsc_mgr_adev, list); + auxiliary_device_delete(&rm_adev->adev); + auxiliary_device_uninit(&rm_adev->adev); + } + err_msgq: gh_msgq_platform_host_unattach(&rsc_mgr->msgq); return ret; @@ -542,6 +592,14 @@ static int gh_rm_drv_probe(struct platform_device *pdev) static int gh_rm_drv_remove(struct platform_device *pdev) { struct gh_rsc_mgr *rsc_mgr = platform_get_drvdata(pdev); + struct gh_rsc_mgr_adev *rm_adev; + struct list_head *l, *n; + + list_for_each_safe(l, n, &rsc_mgr->adevs) { + rm_adev = container_of(l, struct gh_rsc_mgr_adev, list); + auxiliary_device_delete(&rm_adev->adev); + auxiliary_device_uninit(&rm_adev->adev); + } gh_msgq_platform_host_unattach(&rsc_mgr->msgq); From patchwork Wed Feb 23 23:37:28 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Elliot Berman X-Patchwork-Id: 12757672 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id CAEE8C4321E for ; Wed, 23 Feb 2022 23:38:08 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S244923AbiBWXif (ORCPT ); Wed, 23 Feb 2022 18:38:35 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58888 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S244875AbiBWXi2 (ORCPT ); Wed, 23 Feb 2022 18:38:28 -0500 Received: from alexa-out-sd-02.qualcomm.com (alexa-out-sd-02.qualcomm.com [199.106.114.39]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A2D075A5B5; Wed, 23 Feb 2022 15:37:59 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; i=@quicinc.com; q=dns/txt; s=qcdkim; t=1645659479; x=1677195479; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=SUtFhxETXDaiGCtcN5VjrfvX86YbCrloV5EQK2dGRjo=; b=lsrJ7Mwo0SrUH7h8he7GvubBLFa/vzVU6VhuUyLZ8WjR09VpKK5RL2RI iDbtWOUuXa1aXS2QONYBEdjmjxvUApFeTaOKNW6m/Zv9xvbfnuZkhREOX dhXCG/BmKGComfNSoRqhqi8+xeXoNCh1YsRld+hLtvzVBhBUQh9ooq+X1 A=; Received: from unknown (HELO ironmsg04-sd.qualcomm.com) ([10.53.140.144]) by alexa-out-sd-02.qualcomm.com with ESMTP; 23 Feb 2022 15:37:59 -0800 X-QCInternal: smtphost Received: from nasanex01b.na.qualcomm.com ([10.46.141.250]) by ironmsg04-sd.qualcomm.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 23 Feb 2022 15:37:59 -0800 Received: from hu-eberman-lv.qualcomm.com (10.49.16.6) by nasanex01b.na.qualcomm.com (10.46.141.250) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.15; Wed, 23 Feb 2022 15:37:58 -0800 From: Elliot Berman To: Bjorn Andersson , CC: Elliot Berman , Trilok Soni , Murali Nalajala , Srivatsa Vaddagiri , Carl van Schaik , Andy Gross , Subject: [PATCH 10/11] gunyah: rsc_mgr: Add RPC for console services Date: Wed, 23 Feb 2022 15:37:28 -0800 Message-ID: <20220223233729.1571114-11-quic_eberman@quicinc.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220223233729.1571114-1-quic_eberman@quicinc.com> References: <20220223233729.1571114-1-quic_eberman@quicinc.com> MIME-Version: 1.0 X-Originating-IP: [10.49.16.6] X-ClientProxiedBy: nalasex01a.na.qualcomm.com (10.47.209.196) To nasanex01b.na.qualcomm.com (10.46.141.250) Precedence: bulk List-ID: X-Mailing-List: linux-arm-msm@vger.kernel.org Gunyah resource manager defines a simple API for virtual machine log sharing with the console service. A VM's own log can be opened by using GH_VMID_SELF. Another VM's log can be accessed via its VMID. Once opened, characters can be written to the log with a write command. Characters are received with resource manager notifications (using ID GH_RM_NOTIF_VM_CONSOLE_CHARS). These high level rpc calls are kept in drivers/virt/gunyah/rsc_mgr_rpc.c. Future RPC calls, e.g. to launch a VM will also be maintained in this file. Signed-off-by: Elliot Berman --- drivers/virt/gunyah/Makefile | 4 +- drivers/virt/gunyah/rsc_mgr.h | 19 +++++ drivers/virt/gunyah/rsc_mgr_rpc.c | 129 ++++++++++++++++++++++++++++++ include/linux/gunyah_rsc_mgr.h | 15 ++++ 4 files changed, 165 insertions(+), 2 deletions(-) create mode 100644 drivers/virt/gunyah/rsc_mgr_rpc.c diff --git a/drivers/virt/gunyah/Makefile b/drivers/virt/gunyah/Makefile index 86655bca8944..b3f15c052297 100644 --- a/drivers/virt/gunyah/Makefile +++ b/drivers/virt/gunyah/Makefile @@ -1,5 +1,5 @@ # SPDX-License-Identifier: GPL-2.0-only gunyah-y += sysfs.o device.o msgq.o -gunyah-y += rsc_mgr.o -obj-$(CONFIG_GUNYAH) += gunyah.o \ No newline at end of file +gunyah-y += rsc_mgr.o rsc_mgr_rpc.o +obj-$(CONFIG_GUNYAH) += gunyah.o diff --git a/drivers/virt/gunyah/rsc_mgr.h b/drivers/virt/gunyah/rsc_mgr.h index e4f2499267bf..8cbc04b9938e 100644 --- a/drivers/virt/gunyah/rsc_mgr.h +++ b/drivers/virt/gunyah/rsc_mgr.h @@ -28,6 +28,25 @@ #define GH_RM_ERROR_IRQ_INUSE 0x10 #define GH_RM_ERROR_IRQ_RELEASED 0x11 +/* Message IDs: VM Services */ +#define GH_RM_RPC_VM_CONSOLE_OPEN_ID 0x56000081 +#define GH_RM_RPC_VM_CONSOLE_CLOSE_ID 0x56000082 +#define GH_RM_RPC_VM_CONSOLE_WRITE_ID 0x56000083 +#define GH_RM_RPC_VM_CONSOLE_FLUSH_ID 0x56000084 + +/* Call: CONSOLE_OPEN, CONSOLE_CLOSE, CONSOLE_FLUSH */ +struct gh_vm_console_common_req { + gh_vmid_t vmid; + u16 reserved0; +} __packed; + +/* Call: CONSOLE_WRITE */ +struct gh_vm_console_write_req { + gh_vmid_t vmid; + u16 num_bytes; + u8 data[0]; +} __packed; + int gh_rm_call(u32 message_id, void *req_buff, size_t req_buff_size, void **resp_buf, size_t *resp_buff_size); diff --git a/drivers/virt/gunyah/rsc_mgr_rpc.c b/drivers/virt/gunyah/rsc_mgr_rpc.c new file mode 100644 index 000000000000..23e0febc1567 --- /dev/null +++ b/drivers/virt/gunyah/rsc_mgr_rpc.c @@ -0,0 +1,129 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (c) 2022 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#define pr_fmt(fmt) "gh_rsc_mgr: " fmt + +/* Contains the high level interface used by other drivers (or RM driver itself) */ +#include +#include +#include +#include + +#include "rsc_mgr.h" + +/** + * gh_rm_console_open: Open a console with a VM + * @vmid: VMID of the other vmid whose console to open. If VMID is GH_VMID_SELF, the + * console associated with this VM is opened. + */ +int gh_rm_console_open(gh_vmid_t vmid) +{ + void *resp; + struct gh_vm_console_common_req req_payload = {0}; + size_t resp_size; + int ret; + + req_payload.vmid = vmid; + + ret = gh_rm_call(GH_RM_RPC_VM_CONSOLE_OPEN_ID, + &req_payload, sizeof(req_payload), + &resp, &resp_size); + kfree(resp); + + if (!ret && resp_size) + pr_warn("Received unexpected payload for CONSOLE_OPEN: %lu\n", resp_size); + + return ret; +} +EXPORT_SYMBOL_GPL(gh_rm_console_open); + +/** + * gh_rm_console_close: Close a console with a VM + * @vmid: The vmid of the vm whose console to close. + */ +int gh_rm_console_close(gh_vmid_t vmid) +{ + void *resp; + struct gh_vm_console_common_req req_payload = {0}; + size_t resp_size; + int ret; + + req_payload.vmid = vmid; + + ret = gh_rm_call(GH_RM_RPC_VM_CONSOLE_CLOSE_ID, + &req_payload, sizeof(req_payload), + &resp, &resp_size); + kfree(resp); + + if (!ret && resp_size) + pr_warn("Received unexpected payload for CONSOLE_CLOSE: %lu\n", resp_size); + + return ret; +} +EXPORT_SYMBOL_GPL(gh_rm_console_close); + +/** + * gh_rm_console_write: Write to a VM's console + * @vmid: The vmid of the vm whose console to write to. + * @buf: Buffer to write to the VM's console + * @size: Size of the buffer + */ +int gh_rm_console_write(gh_vmid_t vmid, const char *buf, size_t size) +{ + void *resp; + struct gh_vm_console_write_req *req_payload; + size_t resp_size; + int ret = 0; + size_t req_payload_size = sizeof(*req_payload) + size; + + if (size < 1 || size > (U32_MAX - sizeof(*req_payload))) + return -EINVAL; + + req_payload = kzalloc(req_payload_size, GFP_KERNEL); + + if (!req_payload) + return -ENOMEM; + + req_payload->vmid = vmid; + req_payload->num_bytes = size; + memcpy(req_payload->data, buf, size); + + ret = gh_rm_call(GH_RM_RPC_VM_CONSOLE_WRITE_ID, + req_payload, req_payload_size, + &resp, &resp_size); + kfree(req_payload); + kfree(resp); + + if (!ret && resp_size) + pr_warn("Received unexpected payload for CONSOLE_WRITE: %lu\n", resp_size); + + return ret; +} +EXPORT_SYMBOL_GPL(gh_rm_console_write); + +/** + * gh_rm_console_flush: Flush a console with a VM + * @vmid: The vmid of the vm whose console to flush + */ +int gh_rm_console_flush(gh_vmid_t vmid) +{ + void *resp; + struct gh_vm_console_common_req req_payload = {0}; + size_t resp_size; + int ret; + + req_payload.vmid = vmid; + + ret = gh_rm_call(GH_RM_RPC_VM_CONSOLE_FLUSH_ID, + &req_payload, sizeof(req_payload), + &resp, &resp_size); + kfree(resp); + + if (!ret && resp_size) + pr_warn("Received unexpected payload for CONSOLE_FLUSH: %lu\n", resp_size); + + return ret; +} +EXPORT_SYMBOL_GPL(gh_rm_console_flush); diff --git a/include/linux/gunyah_rsc_mgr.h b/include/linux/gunyah_rsc_mgr.h index 015bd851e1a3..035bbf601e1f 100644 --- a/include/linux/gunyah_rsc_mgr.h +++ b/include/linux/gunyah_rsc_mgr.h @@ -26,4 +26,19 @@ struct gh_rm_notification { int gh_rm_register_notifier(struct notifier_block *nb); int gh_rm_unregister_notifier(struct notifier_block *nb); +/* Notification type Message IDs */ +#define GH_RM_NOTIF_VM_CONSOLE_CHARS 0X56100080 + +struct gh_rm_notif_vm_console_chars { + gh_vmid_t vmid; + u16 num_bytes; + u8 bytes[0]; +} __packed; + +/* RPC Calls */ +int gh_rm_console_open(gh_vmid_t vmid); +int gh_rm_console_close(gh_vmid_t vmid); +int gh_rm_console_write(gh_vmid_t vmid, const char *buf, size_t size); +int gh_rm_console_flush(gh_vmid_t vmid); + #endif From patchwork Wed Feb 23 23:37:29 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Elliot Berman X-Patchwork-Id: 12757675 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id A1A58C433F5 for ; Wed, 23 Feb 2022 23:38:11 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S244913AbiBWXih (ORCPT ); Wed, 23 Feb 2022 18:38:37 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58872 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S244895AbiBWXi2 (ORCPT ); Wed, 23 Feb 2022 18:38:28 -0500 Received: from alexa-out-sd-01.qualcomm.com (alexa-out-sd-01.qualcomm.com [199.106.114.38]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A6EB45A5B8; Wed, 23 Feb 2022 15:37:59 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; i=@quicinc.com; q=dns/txt; s=qcdkim; t=1645659479; x=1677195479; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=DDRe7Pv56LCQJcqlWwmFLq2OP2NC5qVr02qNNdMapis=; b=OspBUknzgSVXcvxAW26jfbAdaX4Vwyy9/ZekJVAXFhCPVQqM9whmTJQl I3XiDFrfQRa66hgyJx3h6U+ClJz30JLnI9s5r/s8FVIpQwX+QfYPBUvVm rp0lqsrOC0g+b/hX3qEFTCkMoI6Mf6YL8zbP1UJ0XyTJQbLCNoEacg1G2 E=; Received: from unknown (HELO ironmsg03-sd.qualcomm.com) ([10.53.140.143]) by alexa-out-sd-01.qualcomm.com with ESMTP; 23 Feb 2022 15:37:59 -0800 X-QCInternal: smtphost Received: from nasanex01b.na.qualcomm.com ([10.46.141.250]) by ironmsg03-sd.qualcomm.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 23 Feb 2022 15:37:59 -0800 Received: from hu-eberman-lv.qualcomm.com (10.49.16.6) by nasanex01b.na.qualcomm.com (10.46.141.250) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.15; Wed, 23 Feb 2022 15:37:58 -0800 From: Elliot Berman To: Bjorn Andersson , CC: Elliot Berman , Trilok Soni , Murali Nalajala , Srivatsa Vaddagiri , Carl van Schaik , Andy Gross , Subject: [PATCH 11/11] gunyah: Add tty console driver for RM Console Serivces Date: Wed, 23 Feb 2022 15:37:29 -0800 Message-ID: <20220223233729.1571114-12-quic_eberman@quicinc.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220223233729.1571114-1-quic_eberman@quicinc.com> References: <20220223233729.1571114-1-quic_eberman@quicinc.com> MIME-Version: 1.0 X-Originating-IP: [10.49.16.6] X-ClientProxiedBy: nalasex01a.na.qualcomm.com (10.47.209.196) To nasanex01b.na.qualcomm.com (10.46.141.250) Precedence: bulk List-ID: X-Mailing-List: linux-arm-msm@vger.kernel.org Gunyah provides a console for each VM using the VM console resource manager APIs. This driver allows console data from other VMs to be accessed via a TTY device and exports a console device to dump Linux's own logs to our console. Signed-off-by: Elliot Berman --- Documentation/virt/gunyah/index.rst | 7 + drivers/virt/gunyah/Kconfig | 13 + drivers/virt/gunyah/Makefile | 3 + drivers/virt/gunyah/rsc_mgr_console.c | 410 ++++++++++++++++++++++++++ 4 files changed, 433 insertions(+) create mode 100644 drivers/virt/gunyah/rsc_mgr_console.c diff --git a/Documentation/virt/gunyah/index.rst b/Documentation/virt/gunyah/index.rst index e7bb2b14543e..95ba9b71ab30 100644 --- a/Documentation/virt/gunyah/index.rst +++ b/Documentation/virt/gunyah/index.rst @@ -90,3 +90,10 @@ When booting a virtual machine which uses a devicetree, resource manager overlay how to communicate with resource manager, and basic description and capabilities of this VM. See Documentation/devicetree/bindings/gunyah/qcom,hypervisor.yml for a description of this node. + +Resource Manager Consoles +------------------------- +RM provides infrastructure for virtual machines to share an interactive console. This can be used to +interact with a VM which may not have access to a serial port. Linux will register a printk console: +ttyGH0. That console and other VM's consoles can be accessed via ttyGHX. +/sys/class/tty/ttyGHX/vmid will print the VM which is associated with that TTY. diff --git a/drivers/virt/gunyah/Kconfig b/drivers/virt/gunyah/Kconfig index 2ef4887e280d..86d5ca292c76 100644 --- a/drivers/virt/gunyah/Kconfig +++ b/drivers/virt/gunyah/Kconfig @@ -12,3 +12,16 @@ config GUNYAH Say Y here to enable the drivers needed to interact in a Gunyah virtual environment. + +if GUNYAH +config GUNYAH_RESOURCE_MANAGER_CONSOLE + tristate "Gunyah Resource Manager Consoles" + depends on TTY + help + This enables support for console output using Gunyah's Resource Manager RPC. + This is normally used when a secondary VM which does not have exclusive access + to a real serial device. + + If you don't have Gunyah or have other console options for secondary VMs, + you probably don't want this option. +endif diff --git a/drivers/virt/gunyah/Makefile b/drivers/virt/gunyah/Makefile index b3f15c052297..001cf1630c03 100644 --- a/drivers/virt/gunyah/Makefile +++ b/drivers/virt/gunyah/Makefile @@ -3,3 +3,6 @@ gunyah-y += sysfs.o device.o msgq.o gunyah-y += rsc_mgr.o rsc_mgr_rpc.o obj-$(CONFIG_GUNYAH) += gunyah.o + +gunyah_console-y += rsc_mgr_console.o +obj-$(CONFIG_GUNYAH_RESOURCE_MANAGER_CONSOLE) += gunyah_console.o diff --git a/drivers/virt/gunyah/rsc_mgr_console.c b/drivers/virt/gunyah/rsc_mgr_console.c new file mode 100644 index 000000000000..72267bc9a315 --- /dev/null +++ b/drivers/virt/gunyah/rsc_mgr_console.c @@ -0,0 +1,410 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (c) 2022 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#define pr_fmt(fmt) "gh_rsc_mgr_console: " fmt + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +/* + * The Linux TTY code does not support dynamic addition of tty derived devices + * so we need to know how many tty devices we might need when space is allocated + * for the tty device. Since this driver supports hotplug of vty adapters we + * need to make sure we have enough allocated. + */ +#define RSC_MGR_TTY_ADAPTERS 16 + +/* # of payload bytes that can fit in a 1-fragment CONSOLE_WRITE message */ +#define RM_CONS_WRITE_MSG_SIZE ((1 * (GH_MSGQ_MAX_MSG_SIZE - 8)) - 4) + +struct rm_cons_port { + struct tty_port port; + gh_vmid_t vmid; + bool open; + unsigned int index; + + DECLARE_KFIFO(put_fifo, char, 1024); + spinlock_t fifo_lock; + struct work_struct put_work; + + struct rm_cons_data *cons_data; +}; + +struct rm_cons_data { + struct tty_driver *tty_driver; + struct device *dev; + + spinlock_t ports_lock; + struct rm_cons_port *ports[RSC_MGR_TTY_ADAPTERS]; + + struct notifier_block rsc_mgr_notif; + struct console console; +}; + +static void put_work_fn(struct work_struct *ws) +{ + char buf[RM_CONS_WRITE_MSG_SIZE]; + int count, ret; + struct rm_cons_port *port = container_of(ws, struct rm_cons_port, put_work); + + while (!kfifo_is_empty(&port->put_fifo)) { + count = kfifo_out_spinlocked(&port->put_fifo, buf, sizeof(buf), &port->fifo_lock); + if (count <= 0) + continue; + + ret = gh_rm_console_write(port->vmid, buf, count); + if (ret) { + pr_warn_once("failed to send characters: %d\n", ret); + break; + } + } +} + +static int rsc_mgr_console_notif(struct notifier_block *nb, unsigned long cmd, void *data) +{ + int count, i; + struct rm_cons_port *rm_port; + struct tty_port *tty_port = NULL; + struct rm_cons_data *cons_data = container_of(nb, struct rm_cons_data, rsc_mgr_notif); + const struct gh_rm_notification *notif = data; + struct gh_rm_notif_vm_console_chars const * const msg = notif->buff; + + if (cmd != GH_RM_NOTIF_VM_CONSOLE_CHARS || + notif->size < sizeof(*msg)) + return NOTIFY_DONE; + + spin_lock(&cons_data->ports_lock); + for (i = 0; i < RSC_MGR_TTY_ADAPTERS; i++) { + if (!cons_data->ports[i]) + continue; + if (cons_data->ports[i]->vmid == msg->vmid) { + rm_port = cons_data->ports[i]; + break; + } + } + if (rm_port) + tty_port = tty_port_get(&rm_port->port); + spin_unlock(&cons_data->ports_lock); + + if (!rm_port) + pr_warn("Received unexpected console characters for VMID %u\n", msg->vmid); + if (!tty_port) + return NOTIFY_DONE; + + count = tty_buffer_request_room(tty_port, msg->num_bytes); + tty_insert_flip_string(tty_port, msg->bytes, count); + tty_flip_buffer_push(tty_port); + + tty_port_put(tty_port); + return NOTIFY_OK; +} + +static ssize_t vmid_show(struct device *dev, struct device_attribute *attr, char *buf) +{ + struct rm_cons_port *rm_port = dev_get_drvdata(dev); + + if (rm_port->vmid == GH_VMID_SELF) + return sysfs_emit(buf, "self\n"); + + return sysfs_emit(buf, "%u\n", rm_port->vmid); +} + +static DEVICE_ATTR_RO(vmid); + +static struct attribute *rsc_mgr_tty_dev_attrs[] = { + &dev_attr_vmid.attr, + NULL +}; + +static const struct attribute_group rsc_mgr_tty_dev_attr_group = { + .attrs = rsc_mgr_tty_dev_attrs, +}; + +static const struct attribute_group *rsc_mgr_tty_dev_attr_groups[] = { + &rsc_mgr_tty_dev_attr_group, + NULL +}; + +static int rsc_mgr_tty_open(struct tty_struct *tty, struct file *filp) +{ + int ret; + struct rm_cons_port *rm_port = dev_get_drvdata(tty->dev); + + if (!rm_port->open) { + ret = gh_rm_console_open(rm_port->vmid); + if (ret) { + pr_err("Failed to open RM console for vmid %x: %d\n", rm_port->vmid, ret); + return ret; + } + rm_port->open = true; + } + + return tty_port_open(&rm_port->port, tty, filp); +} + +static void rsc_mgr_tty_close(struct tty_struct *tty, struct file *filp) +{ + int ret; + struct rm_cons_port *rm_port = dev_get_drvdata(tty->dev); + + if (rm_port->open) { + if (rm_port->vmid != GH_VMID_SELF) { + ret = gh_rm_console_close(rm_port->vmid); + if (ret) + pr_warn("Failed to close RM console for vmid %d: %d\n", + rm_port->vmid, ret); + } + rm_port->open = false; + + tty_port_close(&rm_port->port, tty, filp); + } + +} + +static int rsc_mgr_tty_write(struct tty_struct *tty, const unsigned char *buf, int count) +{ + struct rm_cons_port *rm_port = dev_get_drvdata(tty->dev); + int ret; + + ret = kfifo_in_spinlocked(&rm_port->put_fifo, buf, count, &rm_port->fifo_lock); + if (ret > 0) + schedule_work(&rm_port->put_work); + + return ret; +} + +static unsigned int rsc_mgr_mgr_tty_write_room(struct tty_struct *tty) +{ + struct rm_cons_port *rm_port = dev_get_drvdata(tty->dev); + + return kfifo_avail(&rm_port->put_fifo); +} + +static void rsc_mgr_console_write(struct console *co, const char *buf, unsigned count) +{ + struct rm_cons_port *rm_port = co->data; + int ret; + + ret = kfifo_in_spinlocked(&rm_port->put_fifo, buf, count, &rm_port->fifo_lock); + if (ret > 0) + schedule_work(&rm_port->put_work); +} + +static struct tty_driver *rsc_mgr_console_device(struct console *co, int *index) +{ + struct rm_cons_port *rm_port = co->data; + + *index = rm_port->index; + return rm_port->port.tty->driver; +} + +static int rsc_mgr_console_setup(struct console *co, char *unused) +{ + int ret; + struct rm_cons_port *rm_port = co->data; + + if (!rm_port->open) { + ret = gh_rm_console_open(rm_port->vmid); + if (ret) { + pr_err("Failed to open RM console for vmid %x: %d\n", rm_port->vmid, ret); + return ret; + } + rm_port->open = true; + } + + return 0; +} + +static int rsc_mgr_console_exit(struct console *co) +{ + int ret; + struct rm_cons_port *rm_port = co->data; + + if (rm_port->open) { + ret = gh_rm_console_close(rm_port->vmid); + if (ret) { + pr_err("Failed to close RM console for vmid %x: %d\n", rm_port->vmid, ret); + return ret; + } + rm_port->open = false; + } + + return 0; +} + +static const struct tty_operations rsc_mgr_tty_ops = { + .open = rsc_mgr_tty_open, + .close = rsc_mgr_tty_close, + .write = rsc_mgr_tty_write, + .write_room = rsc_mgr_mgr_tty_write_room, +}; + +static void rsc_mgr_port_destruct(struct tty_port *port) +{ + struct rm_cons_port *rm_port = container_of(port, struct rm_cons_port, port); + struct rm_cons_data *cons_data = rm_port->cons_data; + + spin_lock(&cons_data->ports_lock); + WARN_ON(cons_data->ports[rm_port->index] != rm_port); + cons_data->ports[rm_port->index] = NULL; + spin_unlock(&cons_data->ports_lock); + kfree(rm_port); +} + +static const struct tty_port_operations rsc_mgr_port_ops = { + .destruct = rsc_mgr_port_destruct, +}; + +static struct rm_cons_port *rsc_mgr_port_create(struct rm_cons_data *cons_data, gh_vmid_t vmid) +{ + struct rm_cons_port *rm_port; + struct device *ttydev; + unsigned int index; + int ret; + + rm_port = kzalloc(sizeof(*rm_port), GFP_KERNEL); + rm_port->vmid = vmid; + INIT_KFIFO(rm_port->put_fifo); + spin_lock_init(&rm_port->fifo_lock); + INIT_WORK(&rm_port->put_work, put_work_fn); + tty_port_init(&rm_port->port); + rm_port->port.ops = &rsc_mgr_port_ops; + + spin_lock(&cons_data->ports_lock); + for (index = 0; index < RSC_MGR_TTY_ADAPTERS; index++) { + if (!cons_data->ports[index]) { + cons_data->ports[index] = rm_port; + rm_port->index = index; + break; + } + } + spin_unlock(&cons_data->ports_lock); + if (index >= RSC_MGR_TTY_ADAPTERS) { + ret = -ENOSPC; + goto err_put_port; + } + + ttydev = tty_port_register_device_attr(&rm_port->port, cons_data->tty_driver, index, + cons_data->dev, rm_port, rsc_mgr_tty_dev_attr_groups); + if (IS_ERR(ttydev)) { + ret = PTR_ERR(ttydev); + goto err_put_port; + } + + return rm_port; +err_put_port: + tty_port_put(&rm_port->port); + return ERR_PTR(ret); +} + +static int rsc_mgr_console_probe(struct auxiliary_device *auxdev, + const struct auxiliary_device_id *aux_dev_id) +{ + struct rm_cons_data *cons_data; + struct rm_cons_port *rm_port; + int ret; + struct device_node *hyp_node, *vm_of_node; + u32 vmid; + + cons_data = devm_kzalloc(&auxdev->dev, sizeof(*cons_data), GFP_KERNEL); + if (!cons_data) + return -ENOMEM; + dev_set_drvdata(&auxdev->dev, cons_data); + cons_data->dev = &auxdev->dev; + + cons_data->tty_driver = tty_alloc_driver(RSC_MGR_TTY_ADAPTERS, + TTY_DRIVER_REAL_RAW | TTY_DRIVER_DYNAMIC_DEV); + if (IS_ERR(cons_data->tty_driver)) + return PTR_ERR(cons_data->tty_driver); + + cons_data->tty_driver->driver_name = "gh"; + cons_data->tty_driver->name = "ttyGH"; + cons_data->tty_driver->type = TTY_DRIVER_TYPE_SYSTEM; + cons_data->tty_driver->init_termios = tty_std_termios; + tty_set_operations(cons_data->tty_driver, &rsc_mgr_tty_ops); + + ret = tty_register_driver(cons_data->tty_driver); + if (ret) { + dev_err(&auxdev->dev, "Could not register tty driver: %d\n", ret); + goto err_put_tty; + } + + spin_lock_init(&cons_data->ports_lock); + + cons_data->rsc_mgr_notif.notifier_call = rsc_mgr_console_notif; + ret = gh_rm_register_notifier(&cons_data->rsc_mgr_notif); + if (ret) { + dev_err(&auxdev->dev, "Could not register for resource manager notifications: %d\n", + ret); + goto err_put_tty; + } + + rm_port = rsc_mgr_port_create(cons_data, GH_VMID_SELF); + if (IS_ERR(rm_port)) { + ret = PTR_ERR(rm_port); + dev_err(&auxdev->dev, "Could not create own console: %d\n", ret); + goto err_unreg_notif; + } + + strncpy(cons_data->console.name, "ttyGH", sizeof(cons_data->console.name)); + cons_data->console.write = rsc_mgr_console_write; + cons_data->console.device = rsc_mgr_console_device; + cons_data->console.setup = rsc_mgr_console_setup; + cons_data->console.exit = rsc_mgr_console_exit; + cons_data->console.index = rm_port->index; + cons_data->console.data = rm_port; + register_console(&cons_data->console); + + hyp_node = of_get_parent(auxdev->dev.parent->of_node); + vm_of_node = of_find_compatible_node(hyp_node, NULL, "qcom,gunyah-vm-id"); + if (vm_of_node) { + ret = of_property_read_u32(vm_of_node, "qcom,vmid", &vmid); + if (ret) + return 0; + rm_port = rsc_mgr_port_create(cons_data, vmid); + } + + return 0; +err_unreg_notif: + gh_rm_unregister_notifier(&cons_data->rsc_mgr_notif); +err_put_tty: + tty_driver_kref_put(cons_data->tty_driver); + return ret; +} + +static void rsc_mgr_console_remove(struct auxiliary_device *auxdev) +{ + struct rm_cons_data *cons_data = dev_get_drvdata(&auxdev->dev); + + unregister_console(&cons_data->console); + gh_rm_unregister_notifier(&cons_data->rsc_mgr_notif); + tty_driver_kref_put(cons_data->tty_driver); +} + +static struct auxiliary_device_id rsc_mgr_console_ids[] = { + { .name = "gunyah.console" }, + {} +}; +MODULE_DEVICE_TABLE(auxiliary, rsc_mgr_console_ids); + +static struct auxiliary_driver rsc_mgr_console_drv = { + .probe = rsc_mgr_console_probe, + .remove = rsc_mgr_console_remove, + .id_table = rsc_mgr_console_ids, +}; +module_auxiliary_driver(rsc_mgr_console_drv); + +MODULE_LICENSE("GPL v2"); +MODULE_DESCRIPTION("Gunyah Console");