From patchwork Fri Oct 6 13:31:36 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jean-Philippe Brucker X-Patchwork-Id: 9989433 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 437076029B for ; Fri, 6 Oct 2017 13:30:56 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 39C0028DB0 for ; Fri, 6 Oct 2017 13:30:56 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 2E2A428DAE; Fri, 6 Oct 2017 13:30:56 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-4.2 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,RCVD_IN_DNSWL_MED autolearn=unavailable version=3.3.1 Received: from bombadil.infradead.org (bombadil.infradead.org [65.50.211.133]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 27CA428DCF for ; Fri, 6 Oct 2017 13:30:55 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:MIME-Version:Cc:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id:References: In-Reply-To:Message-Id:Date:Subject:To:From:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=+r6THs/AEdfp66pgGfNhEbgaAgXOxLV8iH1NlQBrUsk=; b=dNF/Cdv6U6nMi9Scz/N6fj3Aba 0PyvLGv0aiWDTNa3Ae2fD8mGJK+H2wekQ9QpgFVpEJIn5vXKzMu1RIakRxdeoMMgt6aRrAOL6x5Vr jo9CLT6piM+WQJjT+WaTbFSZpRZnKjP732Mys+H5kzW98NAtzvGTcPKqw0PZjeR9Wh1m/H6d8g2M1 hSBW6HNzYEvT5ecxJ6jjejzvc4q4M2mOBijArO7GXCOEWasetSt4HFLh25bNE2b897b74xEFIoNT2 ZpDguGxWfQVW9eh3mBO0XhV1CWE3neanQmq64X3axk4dhwqShdohgN8CHenSrHPeKLs78QzNoDff7 FLiCxJtA==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.87 #1 (Red Hat Linux)) id 1e0SiH-0005HF-VP; Fri, 06 Oct 2017 13:30:50 +0000 Received: from usa-sjc-mx-foss1.foss.arm.com ([217.140.101.70] helo=foss.arm.com) by bombadil.infradead.org with esmtp (Exim 4.87 #1 (Red Hat Linux)) id 1e0SgT-0002Jg-Vr for linux-arm-kernel@lists.infradead.org; Fri, 06 Oct 2017 13:30:07 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 977EF15BF; Fri, 6 Oct 2017 06:28:38 -0700 (PDT) Received: from e106794-lin.cambridge.arm.com (e106794-lin.cambridge.arm.com [10.1.211.72]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id BE8E93F578; Fri, 6 Oct 2017 06:28:33 -0700 (PDT) From: Jean-Philippe Brucker To: linux-arm-kernel@lists.infradead.org, linux-pci@vger.kernel.org, linux-acpi@vger.kernel.org, devicetree@vger.kernel.org, iommu@lists.linux-foundation.org Subject: [RFCv2 PATCH 09/36] iommu/fault: Allow blocking fault handlers Date: Fri, 6 Oct 2017 14:31:36 +0100 Message-Id: <20171006133203.22803-10-jean-philippe.brucker@arm.com> X-Mailer: git-send-email 2.13.3 In-Reply-To: <20171006133203.22803-1-jean-philippe.brucker@arm.com> References: <20171006133203.22803-1-jean-philippe.brucker@arm.com> X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20171006_062858_873123_5735DA34 X-CRM114-Status: GOOD ( 19.04 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: mark.rutland@arm.com, xieyisheng1@huawei.com, gabriele.paoloni@huawei.com, catalin.marinas@arm.com, will.deacon@arm.com, okaya@codeaurora.org, yi.l.liu@intel.com, lorenzo.pieralisi@arm.com, ashok.raj@intel.com, tn@semihalf.com, joro@8bytes.org, rfranz@cavium.com, lenb@kernel.org, jacob.jun.pan@linux.intel.com, alex.williamson@redhat.com, robh+dt@kernel.org, thunder.leizhen@huawei.com, bhelgaas@google.com, dwmw2@infradead.org, liubo95@huawei.com, rjw@rjwysocki.net, robdclark@gmail.com, hanjun.guo@linaro.org, sudeep.holla@arm.com, robin.murphy@arm.com, nwatters@codeaurora.org MIME-Version: 1.0 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org X-Virus-Scanned: ClamAV using ClamSMTP Allow device driver to register their fault handler at various stages of the handling path, by adding flags to iommu_set_ext_fault_handler. Since we now have a fault workqueue, it is quite easy to call their handler from thread context instead of IRQ handler. A driver can request to be called both in blocking and non-blocking context, so it can filter faults early and only execute the blocking code for some of them. Add the IOMMU_FAULT_ATOMIC fault flag to tell the driver where we're calling it from. Signed-off-by: Jean-Philippe Brucker --- Rob, would this do what you want? The MSM driver can register its handler with ATOMIC | BLOCKING flags. When called in IRQ context, it can ignore the fault by returning IOMMU_FAULT_STATUS_NONE, or drop it by returning IOMMU_FAULT_STATUS_INVALID. When called in thread context, it can sleep and then return IOMMU_FAULT_STATUS_INVALID to terminate the fault. --- drivers/iommu/io-pgfault.c | 16 ++++++++++++++-- drivers/iommu/iommu.c | 12 +++++++++--- include/linux/iommu.h | 20 +++++++++++++++++++- 3 files changed, 42 insertions(+), 6 deletions(-) diff --git a/drivers/iommu/io-pgfault.c b/drivers/iommu/io-pgfault.c index 532bdb9ce519..3ec8179f58b5 100644 --- a/drivers/iommu/io-pgfault.c +++ b/drivers/iommu/io-pgfault.c @@ -91,6 +91,14 @@ static int iommu_fault_handle_single(struct iommu_fault_context *fault) unsigned int access_flags = 0; unsigned int fault_flags = FAULT_FLAG_REMOTE; struct iommu_fault *params = &fault->params; + struct iommu_domain *domain = fault->domain; + + if (domain->handler_flags & IOMMU_FAULT_HANDLER_BLOCKING) { + ret = domain->ext_handler(domain, fault->dev, &fault->params, + domain->handler_token); + if (ret != IOMMU_FAULT_STATUS_NONE) + return ret; + } if (!(params->flags & IOMMU_FAULT_PASID)) return ret; @@ -274,7 +282,8 @@ int handle_iommu_fault(struct iommu_domain *domain, struct device *dev, * if upper layers showed interest and installed a fault handler, * invoke it. */ - if (domain->ext_handler) { + if (domain->handler_flags & IOMMU_FAULT_HANDLER_ATOMIC) { + fault->flags |= IOMMU_FAULT_ATOMIC; ret = domain->ext_handler(domain, dev, fault, domain->handler_token); @@ -290,8 +299,11 @@ int handle_iommu_fault(struct iommu_domain *domain, struct device *dev, } /* If the handler is blocking, handle fault in the workqueue */ - if (fault->flags & IOMMU_FAULT_RECOVERABLE) + if ((fault->flags & IOMMU_FAULT_RECOVERABLE) || + (domain->handler_flags & IOMMU_FAULT_HANDLER_BLOCKING)) { + fault->flags &= ~IOMMU_FAULT_ATOMIC; ret = iommu_queue_fault(domain, dev, fault); + } return iommu_fault_finish(domain, dev, fault, ret); } diff --git a/drivers/iommu/iommu.c b/drivers/iommu/iommu.c index ee956b5fc301..c189648ab7b4 100644 --- a/drivers/iommu/iommu.c +++ b/drivers/iommu/iommu.c @@ -1258,7 +1258,9 @@ EXPORT_SYMBOL_GPL(iommu_set_fault_handler); * @dev: the device * @handler: fault handler * @token: user data, will be passed back to the fault handler - * @flags: IOMMU_FAULT_HANDLER_* parameters. + * @flags: IOMMU_FAULT_HANDLER_* parameters. Allows the driver to tell when it + * wants to be notified. By default the handler will only be called from atomic + * context. * * This function should be used by IOMMU users which want to be notified * whenever an IOMMU fault happens. @@ -1275,11 +1277,15 @@ void iommu_set_ext_fault_handler(struct device *dev, if (WARN_ON(!domain)) return; + if (!flags) + flags |= IOMMU_FAULT_HANDLER_ATOMIC; + if (WARN_ON(domain->handler || domain->ext_handler)) return; domain->ext_handler = handler; domain->handler_token = token; + domain->handler_flags = flags; } EXPORT_SYMBOL_GPL(iommu_set_ext_fault_handler); @@ -1824,7 +1830,7 @@ int report_iommu_fault(struct iommu_domain *domain, struct device *dev, int ret = -ENOSYS; struct iommu_fault fault = { .address = iova, - .flags = flags, + .flags = flags | IOMMU_FAULT_ATOMIC, }; /* @@ -1834,7 +1840,7 @@ int report_iommu_fault(struct iommu_domain *domain, struct device *dev, if (domain->handler) ret = domain->handler(domain, dev, iova, flags, domain->handler_token); - else if (domain->ext_handler) + else if (domain->handler_flags & IOMMU_FAULT_HANDLER_ATOMIC) ret = domain->ext_handler(domain, dev, &fault, domain->handler_token); diff --git a/include/linux/iommu.h b/include/linux/iommu.h index 37fafaf07ee2..a6d417785c7b 100644 --- a/include/linux/iommu.h +++ b/include/linux/iommu.h @@ -66,6 +66,8 @@ struct notifier_block; #define IOMMU_FAULT_GROUP (1 << 6) /* Fault is last of its group */ #define IOMMU_FAULT_LAST (1 << 7) +/* The fault handler is being called from atomic context */ +#define IOMMU_FAULT_ATOMIC (1 << 8) /** * enum iommu_fault_status - Return status of fault handlers, telling the IOMMU @@ -97,6 +99,21 @@ enum iommu_fault_status { typedef int (*iommu_fault_handler_t)(struct iommu_domain *, struct device *, unsigned long, int, void *); +/* + * IOMMU_FAULT_HANDLER_ATOMIC: Notify device driver from within atomic context + * (IRQ handler). The callback is not allowed to sleep. If the fault is + * recoverable, the driver must either return a fault status telling the IOMMU + * driver how to complete the fault (FAILURE, INVALID, HANDLED) or complete the + * fault later with iommu_fault_response. + */ +#define IOMMU_FAULT_HANDLER_ATOMIC (1 << 0) +/* + * IOMMU_FAULT_HANDLER_BLOCKING: Notify device driver from a thread. If the fault + * is recoverable, the driver must return a fault status telling the IOMMU + * driver how to complete the fault (FAILURE, INVALID, HANDLED) + */ +#define IOMMU_FAULT_HANDLER_BLOCKING (1 << 1) + struct iommu_fault { /* Faulting address */ unsigned long address; @@ -161,6 +178,7 @@ struct iommu_domain { iommu_fault_handler_t handler; iommu_ext_fault_handler_t ext_handler; void *handler_token; + int handler_flags; iommu_process_exit_handler_t process_exit; void *process_exit_token; struct iommu_domain_geometry geometry; @@ -633,7 +651,7 @@ static inline phys_addr_t iommu_iova_to_phys(struct iommu_domain *domain, dma_ad } static inline void iommu_set_fault_handler(struct iommu_domain *domain, - iommu_fault_handler_t handler, void *token) + iommu_fault_handler_t handler, void *token, int flags) { }