From patchwork Fri Apr 23 09:51:36 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vivek Kumar Gautam X-Patchwork-Id: 12220021 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-11.8 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2C0D5C433ED for ; Fri, 23 Apr 2021 09:54:40 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 73906613E4 for ; Fri, 23 Apr 2021 09:54:39 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 73906613E4 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=arm.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding :Content-Type:MIME-Version:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:Message-Id:Date:Subject:Cc:To:From:Reply-To: Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender: Resent-To:Resent-Cc:Resent-Message-ID:In-Reply-To:References:List-Owner; bh=vkT+1zbmFEIE7eM9icxWhH72bsg5hc8NATaykSmNnO0=; b=lZkV70XLl6S622G3H2gnpbfIUc 4QHZuonEI9RFJbMu/7KNkzyEwQDM5e1EpGQAdE4X8ZP6WGIHr+kpUrpB5m03p2ZaiuK94nwsMT5qN Y4lcWOIJ3UhLwanFxTgTtkdzjRxldXSaArrp9aCJ5r1fDVUYWbY4qw/0mHEsBUygSgEmBSoEVN0zo 6gMTIia9FO6ZGzbV4KyIxJd9Ksa4WGGfRUJ4eTpMlGohTRn5tQl8O7rF1niCpQ+C0fzpRt59hvKKB DyWMc1y9rORPv4K/9IZXhKbua8189Pc62wyLq4UvVefMsYFxLvkU5FoIalH7lTFnmVrNioFEhpUf7 agsU98DQ==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1lZsTq-001DEs-DN; Fri, 23 Apr 2021 09:52:10 +0000 Received: from bombadil.infradead.org ([2607:7c80:54:e::133]) by desiato.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lZsTn-001DEd-0D for linux-arm-kernel@desiato.infradead.org; Fri, 23 Apr 2021 09:52:07 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=Message-Id:Date:Subject:Cc:To:From: Sender:Reply-To:MIME-Version:Content-Type:Content-Transfer-Encoding: Content-ID:Content-Description:In-Reply-To:References; bh=0pEPkw4j2As1APEhl3lqiizUDzgsQKIOMFP1xKmlAwE=; b=KLoXsYogvMtjh5Eb5z6HaJHqdz c6SjpWhDXLgu+1qFuMUY9zB1CTJZzlk41WQfGbR7VqaZ1CNXETVDj336jXYs/yFcdydxykDDsYnfM LPc66Y7gveE54WkdYslVrP5BEev9NZB/SZmBUFpzCZL0PUoQY6zBNQds271uewueR5b32nw/gMRpK tIbfHEvoL2cytkb2UVSZpaRyJgiCzX7oOCEciZSSeIs++EW/qm2w/tJmNrfzPfn5F9pJO9/DEFc2s fcm/8SGW4BXEYb8iA8y48RehX4fvKoH+Gpt7w7tDqHqom03imZQsI7ToTUrJjRTC60Gwaah7o5Tfd yWc9WVOw==; Received: from foss.arm.com ([217.140.110.172]) by bombadil.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1lZsTj-00EIcl-NI for linux-arm-kernel@lists.infradead.org; Fri, 23 Apr 2021 09:52:05 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 6F65C11D4; Fri, 23 Apr 2021 02:51:58 -0700 (PDT) Received: from usa.arm.com (a074945.blr.arm.com [10.162.16.71]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 7E1DD3F774; Fri, 23 Apr 2021 02:51:54 -0700 (PDT) From: Vivek Gautam To: linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, iommu@lists.linux-foundation.org, virtualization@lists.linux-foundation.org Cc: joro@8bytes.org, will.deacon@arm.com, mst@redhat.com, robin.murphy@arm.com, jean-philippe@linaro.org, eric.auger@redhat.com, kevin.tian@intel.com, jacob.jun.pan@linux.intel.com, yi.l.liu@intel.com, Lorenzo.Pieralisi@arm.com, shameerali.kolothum.thodi@huawei.com Subject: [PATCH RFC v1 00/11] iommu/virtio: vSVA support with Arm Date: Fri, 23 Apr 2021 15:21:36 +0530 Message-Id: <20210423095147.27922-1-vivek.gautam@arm.com> X-Mailer: git-send-email 2.17.1 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210423_025203_875332_7A066902 X-CRM114-Status: GOOD ( 19.22 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org This patch series aims at enabling vSVA (Shared virtual addressing) support in virtio-iommu driver for devices assigned to the guest kernel using pci-passthrough technique (also called as Device assignment). The changes are proposed to make virtio-iommu driver become intelligent to support Shared virtual addressing. The virtio-iommu device serves as the para-virtualized iommu for IO devices running in the guest. This work is done for PCI devices that are able to generate Address Translation Service (ATS) and Page Request Interface (PRI) requests. With vSVA support now devices running in guest can start using process address space (Guest virtual address, GVA) for DMA. The below diagram shows a simple system layout using iommus in guest and host. ------------------ | virtio-iommu | | driver | | (front-end) | |________________| Guest kernel -------------------------------------------------- ------------------ | virtio-iommu | | driver | | (back-end) | |________________| kvmtool -------------------------------------------------- ----------------- | arm-smmu-v3 | | driver | |________________| Host kernel -------------------------------------------------- ----------------- | arm-smmu-v3 | | hardware | |________________| Hardware -------------------------------------------------- The flow of various messages/requests looks like below: a) The stage-1 page tables are prepared by virtio-iommu driver using CPU page table info. This page table data then flow from guest kernel to host kernel using VFIO uapi changes [1]. The stage-1 page tables are then programmed into the hardware by the arm-smmu-v3 driver. b) The device can then start initiating DMA transactions using ATS request. c) When using GVA, SMMU encounters a translation fault and responds to the device with ATS success - translation failure. d) The device can then send a PRI request that eventually populates the PRIQ of arm-smmu-v3. e) The page fault info is captured from PRIQ, and sent to the guest kernel using VFIO dma fault region as added in [1]. f) The page fault is received on the virt-queue by virtio-iommu driver and is then passed to io-page-fault handler. The io page fault handler talks to mm fault handling engine in guest kernel and gets the CPU page tables updated. g) Virtio-iommu driver then sends page_response backend virtio-iommu in vmm. From there this page_response info is passed to host kernel. h) The arm-smmu-v3 driver running in the host will then use this page repsonse info and send a response to the requesting device. The series use nested page table support [2] as the base to build vSVA solution. The changes are inspired from the SVA support in arm-smmu-v3 and are making use of io-page-fault changes added by Jean Philippe [3]. The changes include: - The mmu notifier structure, and allocation and freeing up of shared context descriptors are moved to arm-smmu-v3-cd-lib library driver, and changes have been made to arm-smmu-v3-sva driver to use these shared cd alloc/free helpers. - In virtio-iommu driver: - changes have been added to add various iommu_ops to enable/disable feature, and to perform sva_bind/unbind. - A iopf queue has been added that accepts incoming page faults, and work with mm fault handler to get the page resident. - Incoming page fault information from vmm is parsed using a work-queue and passed to the iopf fault handler. - A new virt-queue request has been added to send page response back to the vmm back-end driver. The page response is populated from the mm fault handler response, and includes information on Page Request pasid, group-id, and response code, etc. [1] https://lore.kernel.org/linux-iommu/20210411114659.15051-1-eric.auger@redhat.com/ https://lore.kernel.org/linux-iommu/20210411111228.14386-1-eric.auger@redhat.com/ [2] https://lore.kernel.org/linux-iommu/20210115121342.15093-1-vivek.gautam@arm.com/ [3] https://www.spinics.net/lists/arm-kernel/msg886518.html Vivek Gautam (11): uapi/virtio-iommu: Add page request grp-id and flags information iommu/virtio: Maintain a list of endpoints served by viommu_dev iommu/virtio: Handle incoming page faults iommu/virtio: Add a io page fault queue iommu/virtio: Add SVA feature and related enable/disable callbacks iommu/pasid-table: Add pasid table ops for shared context management iommu/arm-smmu-v3: Move shared context descriptor code to cd-lib iommu/arm-smmu-v3: Implement shared context alloc and free ops iommu/virtio: Implement sva bind/unbind calls uapi/virtio-iommu: Add a new request type to send page response iommu/virtio: Add support to send page response .../arm/arm-smmu-v3/arm-smmu-v3-cd-lib.c | 165 ++++- .../iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c | 230 +------ drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 1 - drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h | 2 - drivers/iommu/iommu-pasid-table.h | 43 ++ drivers/iommu/virtio-iommu.c | 634 +++++++++++++++++- include/uapi/linux/virtio_iommu.h | 35 + 7 files changed, 904 insertions(+), 206 deletions(-)