From patchwork Fri Oct 6 13:31:48 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jean-Philippe Brucker X-Patchwork-Id: 9989509 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 362D46020F for ; Fri, 6 Oct 2017 13:52:13 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 1263D28D39 for ; Fri, 6 Oct 2017 13:52:13 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 072C028D96; Fri, 6 Oct 2017 13:52:13 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-4.2 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,RCVD_IN_DNSWL_MED autolearn=unavailable version=3.3.1 Received: from bombadil.infradead.org (bombadil.infradead.org [65.50.211.133]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 454EC28D39 for ; Fri, 6 Oct 2017 13:52:12 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:MIME-Version:Cc:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id:References: In-Reply-To:Message-Id:Date:Subject:To:From:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=qGhivatid63+XZGdJ6Kh3lVNNDBiLS9IelCJPMDaQXY=; b=o+2WpSaBEi6bVwtwdxZ3WwMssk 7Yc6lkqnxvmRfU5b4zRluB5f7lQb/39LGJSqwXUCCQTsyvROqdVseBI8SEodUkoNsKsyaPSJE4kvc vizDu441Y4EWON60TOSDGA8dbhOMQ0eXpFTOqyj6zJfSyY6YJenZYqctiDZQ/0SHfYtGw3yxt572C kOd1Wxj0U69uTnfpLhgyng+WJSWbeUjJT9Z+miLjnSa/LRc0pbv7W/+4abixMjGptq2e0srTsrd8+ eSidhBXuXwBl8JaiC31trvIiMVnyBk7jUIF0d16mtNMRXrqUJjAG2eqsZnsiaHT7547PP1fb+i5LL EawKF8kQ==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.87 #1 (Red Hat Linux)) id 1e0T2Z-0004sr-Vw; Fri, 06 Oct 2017 13:51:48 +0000 Received: from merlin.infradead.org ([2001:8b0:10b:1231::1]) by bombadil.infradead.org with esmtps (Exim 4.87 #1 (Red Hat Linux)) id 1e0SxR-0007ar-7Y for linux-arm-kernel@bombadil.infradead.org; Fri, 06 Oct 2017 13:46:29 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=merlin.20170209; h=References:In-Reply-To:Message-Id:Date: Subject:Cc:To:From:Sender:Reply-To:MIME-Version:Content-Type: Content-Transfer-Encoding:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Id: List-Help:List-Unsubscribe:List-Subscribe:List-Post:List-Owner:List-Archive; bh=DYCxJmfKgGT8G4iF/KPFvLMSWuD2fSrTPyN/fF0G2c8=; b=sOkgZ2jea06JgzFbTiWWmbrfz /i3yIMde3f3TcRErnKMuln7YOHIkyG+AG+5K87HOpp3R1EweiSiPp1q0LbJf0wqi/jThE0E7AzJFc j9fmx1p/D3W+a31RmhlCIJIDft+sOAb/cwiFsSTYGdzOOxqlwTeEZUedg5FoSNTMHkprLwwZVGz2O o1KHDwYoRe9bpn/hpbV8boZMdd/bO0DPvQbbwC4e1zz4mZM3AqdAz9f/MYeunCCM2FuMA2RFnrmHC gDT38Zv9XLSiox19qGle8PAYNenWaB/tqkz+G1Q+CpjcGmQLlnhjZvVQobINJCV9cnziZh7ozsUQ6 tyBw9hdiQ==; Received: from foss.arm.com ([217.140.101.70]) by merlin.infradead.org with esmtp (Exim 4.87 #1 (Red Hat Linux)) id 1e0ShT-0002EV-Ry for linux-arm-kernel@lists.infradead.org; Fri, 06 Oct 2017 13:30:00 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id BB4241610; Fri, 6 Oct 2017 06:29:39 -0700 (PDT) Received: from e106794-lin.cambridge.arm.com (e106794-lin.cambridge.arm.com [10.1.211.72]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id E18523F578; Fri, 6 Oct 2017 06:29:34 -0700 (PDT) From: Jean-Philippe Brucker To: linux-arm-kernel@lists.infradead.org, linux-pci@vger.kernel.org, linux-acpi@vger.kernel.org, devicetree@vger.kernel.org, iommu@lists.linux-foundation.org Subject: [RFCv2 PATCH 21/36] iommu/arm-smmu-v3: Implement process operations Date: Fri, 6 Oct 2017 14:31:48 +0100 Message-Id: <20171006133203.22803-22-jean-philippe.brucker@arm.com> X-Mailer: git-send-email 2.13.3 In-Reply-To: <20171006133203.22803-1-jean-philippe.brucker@arm.com> References: <20171006133203.22803-1-jean-philippe.brucker@arm.com> X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: mark.rutland@arm.com, xieyisheng1@huawei.com, gabriele.paoloni@huawei.com, catalin.marinas@arm.com, will.deacon@arm.com, okaya@codeaurora.org, yi.l.liu@intel.com, lorenzo.pieralisi@arm.com, ashok.raj@intel.com, tn@semihalf.com, joro@8bytes.org, rfranz@cavium.com, lenb@kernel.org, jacob.jun.pan@linux.intel.com, alex.williamson@redhat.com, robh+dt@kernel.org, thunder.leizhen@huawei.com, bhelgaas@google.com, dwmw2@infradead.org, liubo95@huawei.com, rjw@rjwysocki.net, robdclark@gmail.com, hanjun.guo@linaro.org, sudeep.holla@arm.com, robin.murphy@arm.com, nwatters@codeaurora.org MIME-Version: 1.0 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org X-Virus-Scanned: ClamAV using ClamSMTP Hook process operations to support PASID and page table sharing with the SMMUv3: * process_allocate pins down its ASID and initializes the context descriptor fields. * process_free releases the ASID. * process_attach checks device capabilities and writes the context descriptor. More work is required to ensure that the process' ASID isn't being used for io-pgtables. * process_detach clears the context descriptor and sends required invalidations. * process_invalidate sends required invalidations. * process_exit stops us of the PASID, clears the context descriptor and performs required invalidations. Signed-off-by: Jean-Philippe Brucker --- drivers/iommu/arm-smmu-v3.c | 207 ++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 207 insertions(+) diff --git a/drivers/iommu/arm-smmu-v3.c b/drivers/iommu/arm-smmu-v3.c index 71fc3a2c8a95..c86a1182c137 100644 --- a/drivers/iommu/arm-smmu-v3.c +++ b/drivers/iommu/arm-smmu-v3.c @@ -29,6 +29,7 @@ #include #include #include +#include #include #include #include @@ -37,6 +38,7 @@ #include #include #include +#include #include @@ -642,6 +644,7 @@ struct arm_smmu_strtab_cfg { struct arm_smmu_asid_state { struct arm_smmu_domain *domain; + unsigned long refs; }; /* An SMMUv3 instance */ @@ -712,6 +715,9 @@ struct arm_smmu_master_data { struct device *dev; size_t num_ssids; + bool can_fault; + /* Number of processes attached */ + int processes; }; /* SMMU private data for an IOMMU domain */ @@ -740,6 +746,11 @@ struct arm_smmu_domain { spinlock_t devices_lock; }; +struct arm_smmu_process { + struct iommu_process process; + struct arm_smmu_ctx_desc ctx_desc; +}; + struct arm_smmu_option_prop { u32 opt; const char *prop; @@ -766,6 +777,11 @@ static struct arm_smmu_domain *to_smmu_domain(struct iommu_domain *dom) return container_of(dom, struct arm_smmu_domain, domain); } +static struct arm_smmu_process *to_smmu_process(struct iommu_process *process) +{ + return container_of(process, struct arm_smmu_process, process); +} + static void parse_driver_options(struct arm_smmu_device *smmu) { int i = 0; @@ -2032,6 +2048,13 @@ static void arm_smmu_detach_dev(struct device *dev) struct arm_smmu_master_data *master = dev->iommu_fwspec->iommu_priv; struct arm_smmu_domain *smmu_domain = master->domain; + /* + * Core is preventing concurrent calls between attach and bind, so this + * read only races with process_exit (FIXME). + */ + if (master->processes) + __iommu_process_unbind_dev_all(&smmu_domain->domain, dev); + if (smmu_domain) { spin_lock_irqsave(&smmu_domain->devices_lock, flags); list_del(&master->list); @@ -2143,6 +2166,184 @@ arm_smmu_iova_to_phys(struct iommu_domain *domain, dma_addr_t iova) return ops->iova_to_phys(ops, iova); } +static int arm_smmu_process_init_pgtable(struct arm_smmu_process *smmu_process, + struct mm_struct *mm) +{ + int asid; + + asid = mm_context_get(mm); + if (!asid) + return -ENOSPC; + + smmu_process->ctx_desc.asid = asid; + /* TODO: init the rest */ + + return 0; +} + +static struct iommu_process *arm_smmu_process_alloc(struct task_struct *task) +{ + int ret; + struct mm_struct *mm; + struct arm_smmu_process *smmu_process; + + smmu_process = kzalloc(sizeof(*smmu_process), GFP_KERNEL); + + mm = get_task_mm(task); + if (!mm) { + kfree(smmu_process); + return NULL; + } + + ret = arm_smmu_process_init_pgtable(smmu_process, mm); + mmput(mm); + if (ret) { + kfree(smmu_process); + return NULL; + } + + return &smmu_process->process; +} + +static void arm_smmu_process_free(struct iommu_process *process) +{ + struct arm_smmu_process *smmu_process = to_smmu_process(process); + + /* Unpin ASID */ + mm_context_put(process->mm); + + kfree(smmu_process); +} + +static int arm_smmu_process_share(struct arm_smmu_domain *smmu_domain, + struct arm_smmu_process *smmu_process) +{ + int asid, ret; + struct arm_smmu_asid_state *asid_state; + struct arm_smmu_device *smmu = smmu_domain->smmu; + + asid = smmu_process->ctx_desc.asid; + + asid_state = idr_find(&smmu->asid_idr, asid); + if (asid_state && asid_state->domain) { + return -EEXIST; + } else if (asid_state) { + asid_state->refs++; + return 0; + } + + asid_state = kzalloc(sizeof(*asid_state), GFP_ATOMIC); + asid_state->refs = 1; + + if (!asid_state) + return -ENOMEM; + + ret = idr_alloc(&smmu->asid_idr, asid_state, asid, asid + 1, GFP_ATOMIC); + return ret < 0 ? ret : 0; +} + +static int arm_smmu_process_attach(struct iommu_domain *domain, + struct device *dev, + struct iommu_process *process, bool first) +{ + int ret; + struct arm_smmu_domain *smmu_domain = to_smmu_domain(domain); + struct arm_smmu_process *smmu_process = to_smmu_process(process); + struct arm_smmu_device *smmu = smmu_domain->smmu; + struct arm_smmu_master_data *master = dev->iommu_fwspec->iommu_priv; + + if (!(smmu->features & ARM_SMMU_FEAT_SVM)) + return -ENODEV; + + /* TODO: process->no_pasid */ + if (process->pasid >= master->num_ssids) + return -ENODEV; + + /* TODO: process->no_need_for_pri_ill_pin_everything */ + if (!master->can_fault) + return -ENODEV; + + master->processes++; + + if (!first) + return 0; + + spin_lock(&smmu->asid_lock); + ret = arm_smmu_process_share(smmu_domain, smmu_process); + spin_unlock(&smmu->asid_lock); + if (ret) + return ret; + + arm_smmu_write_ctx_desc(smmu_domain, process->pasid, &smmu_process->ctx_desc); + + return 0; +} + +static void arm_smmu_process_detach(struct iommu_domain *domain, + struct device *dev, + struct iommu_process *process, bool last) +{ + struct arm_smmu_asid_state *asid_state; + struct arm_smmu_domain *smmu_domain = to_smmu_domain(domain); + struct arm_smmu_process *smmu_process = to_smmu_process(process); + struct arm_smmu_master_data *master = dev->iommu_fwspec->iommu_priv; + struct arm_smmu_device *smmu = smmu_domain->smmu; + + master->processes--; + + if (last) { + spin_lock(&smmu->asid_lock); + asid_state = idr_find(&smmu->asid_idr, smmu_process->ctx_desc.asid); + if (--asid_state->refs == 0) { + idr_remove(&smmu->asid_idr, smmu_process->ctx_desc.asid); + kfree(asid_state); + } + spin_unlock(&smmu->asid_lock); + + arm_smmu_write_ctx_desc(smmu_domain, process->pasid, NULL); + } + + /* TODO: Invalidate ATC. */ + /* TODO: Invalidate all mappings if last and not DVM. */ +} + +static void arm_smmu_process_invalidate(struct iommu_domain *domain, + struct iommu_process *process, + unsigned long iova, size_t size) +{ + /* + * TODO: Invalidate ATC. + * TODO: Invalidate mapping if not DVM + */ +} + +static void arm_smmu_process_exit(struct iommu_domain *domain, + struct iommu_process *process) +{ + struct arm_smmu_master_data *master; + struct arm_smmu_domain *smmu_domain = to_smmu_domain(domain); + + if (!domain->process_exit) + return; + + spin_lock(&smmu_domain->devices_lock); + list_for_each_entry(master, &smmu_domain->devices, list) { + if (!master->processes) + continue; + + master->processes--; + domain->process_exit(domain, master->dev, process->pasid, + domain->process_exit_token); + + /* TODO: inval ATC */ + } + spin_unlock(&smmu_domain->devices_lock); + + arm_smmu_write_ctx_desc(smmu_domain, process->pasid, NULL); + + /* TODO: Invalidate all mappings if not DVM */ +} + static struct platform_driver arm_smmu_driver; static int arm_smmu_match_node(struct device *dev, void *data) @@ -2351,6 +2552,12 @@ static struct iommu_ops arm_smmu_ops = { .domain_alloc = arm_smmu_domain_alloc, .domain_free = arm_smmu_domain_free, .attach_dev = arm_smmu_attach_dev, + .process_alloc = arm_smmu_process_alloc, + .process_free = arm_smmu_process_free, + .process_attach = arm_smmu_process_attach, + .process_detach = arm_smmu_process_detach, + .process_invalidate = arm_smmu_process_invalidate, + .process_exit = arm_smmu_process_exit, .map = arm_smmu_map, .unmap = arm_smmu_unmap, .map_sg = default_iommu_map_sg,