From patchwork Wed Aug 2 09:53:18 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vivek Gautam X-Patchwork-Id: 9876493 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 4270B6041F for ; Wed, 2 Aug 2017 09:54:55 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 22CE32879A for ; Wed, 2 Aug 2017 09:54:55 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 0735C2879B; Wed, 2 Aug 2017 09:54:55 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-1.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,RCVD_IN_DNSWL_NONE autolearn=unavailable version=3.3.1 Received: from bombadil.infradead.org (bombadil.infradead.org [65.50.211.133]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 6DF0828798 for ; Wed, 2 Aug 2017 09:54:54 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:MIME-Version:Cc:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id:References: In-Reply-To:Message-Id:Date:Subject:To:From:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=nNmTsarEOorlimvRfraVUiVhOdEBCmhGRQ8/ZBZIptY=; b=KUJJc/H6eNg7hichH8tiskc+ss EiNTL8oRr0/OPsi7cpZhkVbQynuKB3+saS85Y65GOejQV9YpO/mckZHzQge9pjj0jA1sggP+a7Jkk kcmrb1plJW15cmZoAa0O7hDZsQF6DD60V9sl+ik/0DQCq9W+uZQQER26m6MzWGAy00d59XWTnLPLE WnNIOenbvtx3Axqh3bcI/whrofot1E8s+ZIS2djAc/JOTKGlFF1nqZQsVtFcku1JToxpWrhc31jms BTygE12JodxccqRTck/pETvpcvdIu7fPqh8v4v2WRbrBXw14o+WnSyqqZj6UrfdkIVYZ5TPmbaY5F ZJulF0hA==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.87 #1 (Red Hat Linux)) id 1dcqLo-0003Cq-DX; Wed, 02 Aug 2017 09:54:00 +0000 Received: from smtp.codeaurora.org ([198.145.29.96]) by bombadil.infradead.org with esmtps (Exim 4.87 #1 (Red Hat Linux)) id 1dcqLk-0003B0-KQ for linux-arm-kernel@lists.infradead.org; Wed, 02 Aug 2017 09:53:58 +0000 Received: by smtp.codeaurora.org (Postfix, from userid 1000) id F1FEB607E2; Wed, 2 Aug 2017 09:53:35 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=codeaurora.org; s=default; t=1501667615; bh=0e6hOif/hziWewYH0dG2B4fKoMcXFmPf12RG6cVh0nw=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=lhkL+E2DRtHLy4wNnuo4CWTIdZNzro1c3DhjYKXbdJ0TZv1xHtYsTjshH6xh2m0Bd fKhRccX6/V4DqhWY3QtSDWTvZceAStIZrQ70pxaF+KMxXuu7OQ45HFg8Q4/mScMTWu dCtI5QVz0SKR7MqHqI4frGDKlbO10j/SeiEpUbrw= Received: from blr-ubuntu-41.ap.qualcomm.com (blr-bdr-fw-01_globalnat_allzones-outside.qualcomm.com [103.229.18.19]) (using TLSv1.1 with cipher ECDHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) (Authenticated sender: vivek.gautam@smtp.codeaurora.org) by smtp.codeaurora.org (Postfix) with ESMTPSA id 45275602BC; Wed, 2 Aug 2017 09:53:30 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=codeaurora.org; s=default; t=1501667614; bh=0e6hOif/hziWewYH0dG2B4fKoMcXFmPf12RG6cVh0nw=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=GOtcVN2/16GMy3ZdmrsCPSrNjKE/AdsTdj3/HCUHMmShTe3BOZkY3AbxuJ+5LI9x0 hjlI+yUGRMdS5PB95bat2Yk3xIvjynTlL8QnRDhBYkP8UP63rNLosEwKnDLmcUuPpq fuH3XeKwgdBU6Jsf3HV34kAYYB8mvtN+X8U+EKQw= DMARC-Filter: OpenDMARC Filter v1.3.2 smtp.codeaurora.org 45275602BC Authentication-Results: pdx-caf-mail.web.codeaurora.org; dmarc=none (p=none dis=none) header.from=codeaurora.org Authentication-Results: pdx-caf-mail.web.codeaurora.org; spf=none smtp.mailfrom=vivek.gautam@codeaurora.org From: Vivek Gautam To: iommu@lists.linux-foundation.org, linux-arm-msm@vger.kernel.org Subject: [PATCH] iommu/arm-smmu: Defer TLB flush in case of unmap op Date: Wed, 2 Aug 2017 15:23:18 +0530 Message-Id: <1501667598-16404-1-git-send-email-vivek.gautam@codeaurora.org> X-Mailer: git-send-email 1.9.1 In-Reply-To: References: X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20170802_025356_712604_BCBE2831 X-CRM114-Status: GOOD ( 17.35 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: mark.rutland@arm.com, sricharan@codeaurora.org, joro@8bytes.org, will.deacon@arm.com, linux-kernel@vger.kernel.org, stanimir.varbanov@linaro.org, robdclark@gmail.com, robh+dt@kernel.org, Vivek Gautam , robin.murphy@arm.com, sboyd@codeaurora.org, linux-arm-kernel@lists.infradead.org, m.szyprowski@samsung.com MIME-Version: 1.0 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org X-Virus-Scanned: ClamAV using ClamSMTP We don't want to touch the TLB when smmu is suspended. Defer it until resume. Signed-off-by: Vivek Gautam --- Hi all, Here's the small patch in response of suggestion to defer tlb operations when smmu is in suspend state. The patch stores the TLB requests in 'unmap' when the smmu device is suspended. On resume, it checks all the pending TLB requests, and performs the unmap over those. Right now, I have applied the patch on top of the pm runtime series. Let me know what you think of the change. It will also be helpful if somebody can please test a valid use case with this. regards Vivek drivers/iommu/arm-smmu.c | 59 +++++++++++++++++++++++++++++++++++++++++++----- 1 file changed, 53 insertions(+), 6 deletions(-) diff --git a/drivers/iommu/arm-smmu.c b/drivers/iommu/arm-smmu.c index fe8e7fd61282..1f9c2b16aabb 100644 --- a/drivers/iommu/arm-smmu.c +++ b/drivers/iommu/arm-smmu.c @@ -51,6 +51,7 @@ #include #include #include +#include #include @@ -151,6 +152,14 @@ struct arm_smmu_master_cfg { #define for_each_cfg_sme(fw, i, idx) \ for (i = 0; idx = fwspec_smendx(fw, i), i < fw->num_ids; ++i) +struct arm_smmu_tlb_req_info { + struct iommu_domain *domain; + unsigned long iova; + size_t size; + bool tlb_flush_pending; + struct list_head list; +}; + struct arm_smmu_device { struct device *dev; @@ -182,6 +191,7 @@ struct arm_smmu_device { u32 num_s2_context_banks; DECLARE_BITMAP(context_map, ARM_SMMU_MAX_CBS); atomic_t irptndx; + struct list_head domain_list; u32 num_mapping_groups; u16 streamid_mask; @@ -1239,17 +1249,32 @@ static size_t arm_smmu_unmap(struct iommu_domain *domain, unsigned long iova, size_t size) { struct arm_smmu_domain *smmu_domain = to_smmu_domain(domain); + struct arm_smmu_device *smmu = smmu_domain->smmu; struct io_pgtable_ops *ops = smmu_domain->pgtbl_ops; - size_t ret; + struct arm_smmu_tlb_req_info *tlb_info; if (!ops) return 0; - pm_runtime_get_sync(smmu_domain->smmu->dev); - ret = ops->unmap(ops, iova, size); - pm_runtime_put_sync(smmu_domain->smmu->dev); + /* if the device is suspended; we can't unmap, defer any tlb operations */ + if (pm_runtime_suspended(smmu->dev)) { + tlb_info = devm_kzalloc(smmu->dev, sizeof(*tlb_info), GFP_ATOMIC); + if (!tlb_info) + return -ENOMEM; - return ret; + tlb_info->domain = domain; + tlb_info->iova = iova; + tlb_info->size = size; + tlb_info->tlb_flush_pending = true; + INIT_LIST_HEAD(&tlb_info->list); + + /* XXX: We need locks here, but that again introduce the slowpath ? */ + list_add_tail(&tlb_info->list, &smmu->domain_list); + + return size; + } + + return ops->unmap(ops, iova, size); } static phys_addr_t arm_smmu_iova_to_phys_hard(struct iommu_domain *domain, @@ -2166,6 +2191,8 @@ static int arm_smmu_device_probe(struct platform_device *pdev) smmu->irqs[i] = irq; } + INIT_LIST_HEAD(&smmu->domain_list); + err = arm_smmu_init_clocks(smmu); if (err) return err; @@ -2268,8 +2295,28 @@ static int arm_smmu_device_remove(struct platform_device *pdev) static int arm_smmu_resume(struct device *dev) { struct arm_smmu_device *smmu = dev_get_drvdata(dev); + struct arm_smmu_tlb_req_info *tlb_info, *temp; + int ret; + + ret = arm_smmu_enable_clocks(smmu); + if (ret) + return ret; + + list_for_each_entry_safe(tlb_info, temp, &smmu->domain_list, list) { + printk("\n\n %s %d :: iterating over pending tlb request\n\n", __func__, __LINE__); + if (tlb_info->tlb_flush_pending) { + ret = arm_smmu_unmap(tlb_info->domain, tlb_info->iova, tlb_info->size); + if (!ret) + return -EINVAL; - return arm_smmu_enable_clocks(smmu); + tlb_info->tlb_flush_pending = false; + + /* we are done with this request; delete it */ + list_del(&tlb_info->list); + } + } + + return 0; } static int arm_smmu_suspend(struct device *dev)