From patchwork Mon Feb 24 16:58:41 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jean-Philippe Brucker X-Patchwork-Id: 11401057 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id AB2251580 for ; Mon, 24 Feb 2020 16:59:09 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 8AF2320CC7 for ; Mon, 24 Feb 2020 16:59:09 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b="RRx8U11v" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727939AbgBXQ7I (ORCPT ); Mon, 24 Feb 2020 11:59:08 -0500 Received: from mail-wm1-f65.google.com ([209.85.128.65]:51751 "EHLO mail-wm1-f65.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728022AbgBXQ7F (ORCPT ); Mon, 24 Feb 2020 11:59:05 -0500 Received: by mail-wm1-f65.google.com with SMTP id t23so55275wmi.1 for ; Mon, 24 Feb 2020 08:59:03 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=ffFCJH69MWS3gu5TdGiWJyiqlV+lu2ttPiLstT+myYY=; b=RRx8U11vMPRTlGnIGn4sCB8SnlGdKWqavfSqD6SR8YQbgWxZkfH48PSJ+5CDfsbk5S Ze+3wOWEkqnI8YBqOFSDJz5FBNrpacYvYZgTsDkmdA5n4wlRZVjddXIXMc1Swc0FusDG ID0exI27uMOdvgwnrKyEkHPZhPAkCE5/JcqFqErWtxLy03ZXcHvBaYquM+8n9pZzbnmf JF5yKMQAnnlPwCUdQ417vEY8sBmXYM2Du+VPCiVVgh3ewo9JEWYEU99AdEfFfMi7eRqo Nb3D2kPURsTnpbcyeQ1NnVKVHrxjKbWjHpk2O3LengMczoRsjyCdMZXBKdW27vABPch/ uALg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=ffFCJH69MWS3gu5TdGiWJyiqlV+lu2ttPiLstT+myYY=; b=AYVDyYZErg4M2W/NOsSm8zJ3JSAbaoseVqAueRgx+iI4iBAYrOXREwldOFDHJ2earK 392Hb+GGLCcuD7CafoDJUb6/ow0hOyGIXloVOtMNs/YALe/fz4MBxhIPLJ2Tsf/ide6q c5r8vmm7txocR1EdKJZWKfSv3RbIgkKHZhCy0DxdNipJQwT7iCNjcN6ooujjUCNNt1kB TcQUHFmnmXW2yz1khsDJVMFk5cljeBFUXwcHW1CuJD29rZ7KE7pmfDA8APEcC2hKsKJt Oid4XUdq11K1ueqGlLh0kJxZX/ptGD8YzMdC9O0VRXMZYhOtVeqn+uc+fkTJJLbpyn6V USGw== X-Gm-Message-State: APjAAAVqwqejFhMDp+76PBPl+fop81FDgdUGMIWaHq5PnC945DUPBlgd wgUUiAqMg2CP4BAyEpxD00bOdB1R0tg= X-Google-Smtp-Source: APXvYqzuI5JL7DJSxxC2MA8748RGx58CDQkzuT+eunhF/IfX/BLY+BsUCaKxLYTvlGXreVTEpIeERg== X-Received: by 2002:a7b:cb86:: with SMTP id m6mr23887005wmi.51.1582563543020; Mon, 24 Feb 2020 08:59:03 -0800 (PST) Received: from localhost.localdomain ([2001:171b:c9a8:fbc0:116c:c27a:3e7f:5eaf]) by smtp.gmail.com with ESMTPSA id b10sm19473978wrt.90.2020.02.24.08.59.02 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 24 Feb 2020 08:59:02 -0800 (PST) From: Jean-Philippe Brucker To: linux-pci@vger.kernel.org, linux-arm-kernel@lists.infradead.org, iommu@lists.linux-foundation.org, will@kernel.org, bhelgaas@google.com Cc: joro@8bytes.org, robin.murphy@arm.com, jonathan.cameron@huawei.com, zhangfei.gao@linaro.org, robh@kernel.org Subject: [PATCH v2 1/6] PCI/ATS: Export symbols of PASID functions Date: Mon, 24 Feb 2020 17:58:41 +0100 Message-Id: <20200224165846.345993-2-jean-philippe@linaro.org> X-Mailer: git-send-email 2.25.0 In-Reply-To: <20200224165846.345993-1-jean-philippe@linaro.org> References: <20200224165846.345993-1-jean-philippe@linaro.org> MIME-Version: 1.0 Sender: linux-pci-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org The Arm SMMUv3 driver uses pci_{enable,disable}_pasid() and related functions. Export them to allow the driver to be built as a module. Signed-off-by: Jean-Philippe Brucker Acked-by: Bjorn Helgaas --- drivers/pci/ats.c | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/drivers/pci/ats.c b/drivers/pci/ats.c index 3ef0bb281e7c..390e92f2d8d1 100644 --- a/drivers/pci/ats.c +++ b/drivers/pci/ats.c @@ -366,6 +366,7 @@ int pci_enable_pasid(struct pci_dev *pdev, int features) return 0; } +EXPORT_SYMBOL_GPL(pci_enable_pasid); /** * pci_disable_pasid - Disable the PASID capability @@ -390,6 +391,7 @@ void pci_disable_pasid(struct pci_dev *pdev) pdev->pasid_enabled = 0; } +EXPORT_SYMBOL_GPL(pci_disable_pasid); /** * pci_restore_pasid_state - Restore PASID capabilities @@ -441,6 +443,7 @@ int pci_pasid_features(struct pci_dev *pdev) return supported; } +EXPORT_SYMBOL_GPL(pci_pasid_features); #define PASID_NUMBER_SHIFT 8 #define PASID_NUMBER_MASK (0x1f << PASID_NUMBER_SHIFT) @@ -469,4 +472,5 @@ int pci_max_pasids(struct pci_dev *pdev) return (1 << supported); } +EXPORT_SYMBOL_GPL(pci_max_pasids); #endif /* CONFIG_PCI_PASID */ From patchwork Mon Feb 24 16:58:42 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jean-Philippe Brucker X-Patchwork-Id: 11401055 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 73A0514D5 for ; Mon, 24 Feb 2020 16:59:09 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 53C2120CC7 for ; Mon, 24 Feb 2020 16:59:09 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b="jR7RaN40" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728040AbgBXQ7I (ORCPT ); Mon, 24 Feb 2020 11:59:08 -0500 Received: from mail-wr1-f67.google.com ([209.85.221.67]:43670 "EHLO mail-wr1-f67.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727644AbgBXQ7G (ORCPT ); Mon, 24 Feb 2020 11:59:06 -0500 Received: by mail-wr1-f67.google.com with SMTP id r11so11240639wrq.10 for ; Mon, 24 Feb 2020 08:59:04 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=L9Q0qcxnkV13c6MhKRP3gkBpAW/Xjj6KTCyBYPPUbo8=; b=jR7RaN40drIO3Z8YN3DdhMKXNAVqDCtLzCnR6gbvLpVA/Aln4pckiC3TMYCvU/Hr3o ijNGM9EOE+8fOESdkaILMtKqGiSU3jxQlhAv81EPv1g4giBq6E+GNU9NF1XS9+XUxqpv Z1Y4k7JAjRn6JCf0HnoMAAsuLrLx6R7gc8PnPUB807vFu6YF2Gv6Cus2I62H+2DXg+OJ rMbRmUuc/BfoBIRbQ6OS0ujUx/hVsYU5Y7neAAcunsZuxM7GhpXI1pYohEA6mlYujQSE 0aaoLw1BVr7jMCzsX+NBL8CH7zND4vxqR9LRD4eFmI21ORXyI1FeB/gv0pyptSkP+kjp 0bjw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=L9Q0qcxnkV13c6MhKRP3gkBpAW/Xjj6KTCyBYPPUbo8=; b=uHMS23LttQzfikPOkr0x06mTURxZMeKPzIoieFjT8hAYYEgnXCMys16BK0C52c7fPS Ek2wk9XQ92IOkstHAZ8q6LQyH0nsaD4iaxmFW/T29S/+VQ6HjuTmN229CsEHf2I4O8HX 9DsYrSRr91KJ9EbmfwfgLAmvIytHsnRhoATYN6ZFEG7iTpWcArkeT+5NjTI+HPOiEsN0 ZH4Fvt+XOp0OvzjENFZm5opFplZay60qPWEGfliL+vZR7RB08NTldaLI9nbkHIbeHYmi 3sKV+2FORkspcvBGoaOjFaJQpN1DHNiY9W8h7QyUDcD6YL5uDIZfhL1xdCPquAPyrPKO Y1cA== X-Gm-Message-State: APjAAAVX5eFJ8IZ2e26vO3yWAX/viJXtCsk/fHcP8afCq9k4YcwPZl4C ljoWlOc720hzCM4X6Fqi54NJzVVnGFo= X-Google-Smtp-Source: APXvYqwBvkXYS+FZhWNBv6R7Dnf8ywFoid6nCAlXnksnCN/yapBQ30GW0ZCSqCk1G3DmpN0THhlTRg== X-Received: by 2002:a5d:4f89:: with SMTP id d9mr67669582wru.391.1582563544022; Mon, 24 Feb 2020 08:59:04 -0800 (PST) Received: from localhost.localdomain ([2001:171b:c9a8:fbc0:116c:c27a:3e7f:5eaf]) by smtp.gmail.com with ESMTPSA id b10sm19473978wrt.90.2020.02.24.08.59.03 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 24 Feb 2020 08:59:03 -0800 (PST) From: Jean-Philippe Brucker To: linux-pci@vger.kernel.org, linux-arm-kernel@lists.infradead.org, iommu@lists.linux-foundation.org, will@kernel.org, bhelgaas@google.com Cc: joro@8bytes.org, robin.murphy@arm.com, jonathan.cameron@huawei.com, zhangfei.gao@linaro.org, robh@kernel.org Subject: [PATCH v2 2/6] iommu/arm-smmu-v3: Add support for PCI PASID Date: Mon, 24 Feb 2020 17:58:42 +0100 Message-Id: <20200224165846.345993-3-jean-philippe@linaro.org> X-Mailer: git-send-email 2.25.0 In-Reply-To: <20200224165846.345993-1-jean-philippe@linaro.org> References: <20200224165846.345993-1-jean-philippe@linaro.org> MIME-Version: 1.0 Sender: linux-pci-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org Enable PASID for PCI devices that support it. Initialize PASID early in add_device() because it must be enabled before ATS. Tested-by: Zhangfei Gao Reviewed-by: Jonathan Cameron Signed-off-by: Jean-Philippe Brucker --- drivers/iommu/arm-smmu-v3.c | 62 ++++++++++++++++++++++++++++++++++++- 1 file changed, 61 insertions(+), 1 deletion(-) diff --git a/drivers/iommu/arm-smmu-v3.c b/drivers/iommu/arm-smmu-v3.c index aa3ac2a03807..6b76df37025e 100644 --- a/drivers/iommu/arm-smmu-v3.c +++ b/drivers/iommu/arm-smmu-v3.c @@ -2628,6 +2628,53 @@ static void arm_smmu_disable_ats(struct arm_smmu_master *master) atomic_dec(&smmu_domain->nr_ats_masters); } +static int arm_smmu_enable_pasid(struct arm_smmu_master *master) +{ + int ret; + int features; + int num_pasids; + struct pci_dev *pdev; + + if (!dev_is_pci(master->dev)) + return -ENODEV; + + pdev = to_pci_dev(master->dev); + + features = pci_pasid_features(pdev); + if (features < 0) + return features; + + num_pasids = pci_max_pasids(pdev); + if (num_pasids <= 0) + return num_pasids; + + ret = pci_enable_pasid(pdev, features); + if (ret) { + dev_err(&pdev->dev, "Failed to enable PASID\n"); + return ret; + } + + master->ssid_bits = min_t(u8, ilog2(num_pasids), + master->smmu->ssid_bits); + return 0; +} + +static void arm_smmu_disable_pasid(struct arm_smmu_master *master) +{ + struct pci_dev *pdev; + + if (!dev_is_pci(master->dev)) + return; + + pdev = to_pci_dev(master->dev); + + if (!pdev->pasid_enabled) + return; + + master->ssid_bits = 0; + pci_disable_pasid(pdev); +} + static void arm_smmu_detach_dev(struct arm_smmu_master *master) { unsigned long flags; @@ -2831,13 +2878,23 @@ static int arm_smmu_add_device(struct device *dev) master->ssid_bits = min(smmu->ssid_bits, fwspec->num_pasid_bits); + /* + * Note that PASID must be enabled before, and disabled after ATS: + * PCI Express Base 4.0r1.0 - 10.5.1.3 ATS Control Register + * + * Behavior is undefined if this bit is Set and the value of the PASID + * Enable, Execute Requested Enable, or Privileged Mode Requested bits + * are changed. + */ + arm_smmu_enable_pasid(master); + if (!(smmu->features & ARM_SMMU_FEAT_2_LVL_CDTAB)) master->ssid_bits = min_t(u8, master->ssid_bits, CTXDESC_LINEAR_CDMAX); ret = iommu_device_link(&smmu->iommu, dev); if (ret) - goto err_free_master; + goto err_disable_pasid; group = iommu_group_get_for_dev(dev); if (IS_ERR(group)) { @@ -2850,6 +2907,8 @@ static int arm_smmu_add_device(struct device *dev) err_unlink: iommu_device_unlink(&smmu->iommu, dev); +err_disable_pasid: + arm_smmu_disable_pasid(master); err_free_master: kfree(master); fwspec->iommu_priv = NULL; @@ -2870,6 +2929,7 @@ static void arm_smmu_remove_device(struct device *dev) arm_smmu_detach_dev(master); iommu_group_remove_device(dev); iommu_device_unlink(&smmu->iommu, dev); + arm_smmu_disable_pasid(master); kfree(master); iommu_fwspec_free(dev); } From patchwork Mon Feb 24 16:58:43 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jean-Philippe Brucker X-Patchwork-Id: 11401053 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 4AB7492A for ; Mon, 24 Feb 2020 16:59:09 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 2A18D20836 for ; Mon, 24 Feb 2020 16:59:09 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b="h5rEfPIu" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727978AbgBXQ7H (ORCPT ); Mon, 24 Feb 2020 11:59:07 -0500 Received: from mail-wr1-f65.google.com ([209.85.221.65]:38409 "EHLO mail-wr1-f65.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728042AbgBXQ7H (ORCPT ); Mon, 24 Feb 2020 11:59:07 -0500 Received: by mail-wr1-f65.google.com with SMTP id e8so11253427wrm.5 for ; Mon, 24 Feb 2020 08:59:06 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=8HIxi7ZL1ytSixldA0X7x1yzZ6pxdx7GHOJTgPsbD+w=; b=h5rEfPIuduIQsDpoF+aKzWlaxFkIu7NDgF+GUcFIIt/VEnEoT0L5HPk9fkl7MEJle/ Ud2/GAlx627mLkL/h0OM7o+RKQDlXhqbE11lN9bmjiCsn4DNjmUERD/jrlTqzVJWNjj4 GKsFp79swWzlVETvQ6zeOwXBwvIXoqOkTvFEc5iELSunxPmlJa3C9IoktYgPNSri85j6 dOsjcm5aopJ5Wg/HN4EeLpWdLgJf2/AhAp5ryL71MVMahP0aLdvn5AtDO+E46a0UO5F/ xsZMf+Gj1g+5qy4Nt1bFFOG67DXu65lwChW+sVk7kYxBwmSFM94vz6m+bs8D32j8JX1B kE3g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=8HIxi7ZL1ytSixldA0X7x1yzZ6pxdx7GHOJTgPsbD+w=; b=Mdox3KLuyPzbazIjH2ATvkH/0sFLCu0x2LHlDfyxdtzA0VmGKxSrqO6NIaP2ZMZiew DsL+hP8TLlkkpAI7fcyDD4SZwP3HBHblkxsUg/fuKVj2huAMfzirM1s1A4QyqEu7nge5 s++rPYTSfvNwpsZEG5IWrW9i5fHfOwbXKd+clFiD0umMi5Gw2W3YvKFMtEzN1kQ/S5Dg yB33wJhrNrPFgBOGDE0C7UGgiB0NlJj+fkneu9eUu+BUbZvwgSpigcPIIXHpYYJdiNdp 7jRgvE8MVs21aOV7Vm8cEOgj+rzBKIv0IwOUbi36gCxNFLi5q05bH4a0bxexFfwR7Fcv QOag== X-Gm-Message-State: APjAAAW9T2CNV7d8sCecF/QXrKjm/O6nk9hD/UcXQ9DZn3ZtTTQhc0IV jmfEBU0bHemW0Ql/gXVFpGPKe6TYr/0= X-Google-Smtp-Source: APXvYqzKuk1DCPA9bZvrQ+EiIsobd8Fn/PAZ2IzLOJ9Fytea0eB2ueRowKL1GRPNmrrjY6GtmuesjA== X-Received: by 2002:adf:e6d2:: with SMTP id y18mr14316725wrm.270.1582563545042; Mon, 24 Feb 2020 08:59:05 -0800 (PST) Received: from localhost.localdomain ([2001:171b:c9a8:fbc0:116c:c27a:3e7f:5eaf]) by smtp.gmail.com with ESMTPSA id b10sm19473978wrt.90.2020.02.24.08.59.04 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 24 Feb 2020 08:59:04 -0800 (PST) From: Jean-Philippe Brucker To: linux-pci@vger.kernel.org, linux-arm-kernel@lists.infradead.org, iommu@lists.linux-foundation.org, will@kernel.org, bhelgaas@google.com Cc: joro@8bytes.org, robin.murphy@arm.com, jonathan.cameron@huawei.com, zhangfei.gao@linaro.org, robh@kernel.org Subject: [PATCH v2 3/6] iommu/arm-smmu-v3: Write level-1 descriptors atomically Date: Mon, 24 Feb 2020 17:58:43 +0100 Message-Id: <20200224165846.345993-4-jean-philippe@linaro.org> X-Mailer: git-send-email 2.25.0 In-Reply-To: <20200224165846.345993-1-jean-philippe@linaro.org> References: <20200224165846.345993-1-jean-philippe@linaro.org> MIME-Version: 1.0 Sender: linux-pci-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org Use WRITE_ONCE() to make sure that the SMMU doesn't read incomplete stream table descriptors. Refer to the comment about 64-bit accesses, and add the comment to the equivalent context descriptor code. Signed-off-by: Jean-Philippe Brucker --- drivers/iommu/arm-smmu-v3.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/drivers/iommu/arm-smmu-v3.c b/drivers/iommu/arm-smmu-v3.c index 6b76df37025e..068a16d0eabe 100644 --- a/drivers/iommu/arm-smmu-v3.c +++ b/drivers/iommu/arm-smmu-v3.c @@ -1531,6 +1531,7 @@ static void arm_smmu_write_cd_l1_desc(__le64 *dst, u64 val = (l1_desc->l2ptr_dma & CTXDESC_L1_DESC_L2PTR_MASK) | CTXDESC_L1_DESC_V; + /* See comment in arm_smmu_write_ctx_desc() */ WRITE_ONCE(*dst, cpu_to_le64(val)); } @@ -1726,7 +1727,8 @@ arm_smmu_write_strtab_l1_desc(__le64 *dst, struct arm_smmu_strtab_l1_desc *desc) val |= FIELD_PREP(STRTAB_L1_DESC_SPAN, desc->span); val |= desc->l2ptr_dma & STRTAB_L1_DESC_L2PTR_MASK; - *dst = cpu_to_le64(val); + /* See comment in arm_smmu_write_ctx_desc() */ + WRITE_ONCE(*dst, cpu_to_le64(val)); } static void arm_smmu_sync_ste_for_sid(struct arm_smmu_device *smmu, u32 sid) From patchwork Mon Feb 24 16:58:44 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jean-Philippe Brucker X-Patchwork-Id: 11401061 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 58CB31871 for ; Mon, 24 Feb 2020 16:59:10 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 3899B21775 for ; Mon, 24 Feb 2020 16:59:10 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b="cYA2Ki37" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728039AbgBXQ7J (ORCPT ); Mon, 24 Feb 2020 11:59:09 -0500 Received: from mail-wm1-f68.google.com ([209.85.128.68]:36733 "EHLO mail-wm1-f68.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727746AbgBXQ7I (ORCPT ); Mon, 24 Feb 2020 11:59:08 -0500 Received: by mail-wm1-f68.google.com with SMTP id p17so61666wma.1 for ; Mon, 24 Feb 2020 08:59:06 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=XLHzcG1rM7CKaHACIaWqlr851SfTiMG1TWiRRSjmhnI=; b=cYA2Ki3712y1AnkYyhSFyI0gPtpDlkJM+1q36yhaQXLM5yHq3MpQEGpQGFn5acTJNq F0BBJHCxoFTbdxkeT2vlUqDSU2bBZcU0lb8elvwHyd4Vg/zVkkyjhd5+jJVBYTxNoe/1 T887AvB5daQAFRCqRCHFVryjl79Ik52OOPX76rCW6nsW+PTelLhaY0/XaXm/hm7XJZOT bDtyCARPGJe9qEjvAqV4E1zbPS4JHztKZyP48RYwOQ+6UtYGoT41rVTvmBpGTezTRswQ jOK+z0mNca9ZAwbWfU/DpnD214mbXaRo45wfSK3F3fh4dc+SQHsQqOySm7gSx2MTFu0R hVPA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=XLHzcG1rM7CKaHACIaWqlr851SfTiMG1TWiRRSjmhnI=; b=Ly+PtY5GLA1cpZ4NXANWW1DWKFcFp5w9A8tXHI/JJvTn4MYIUBbTAwk0AVJMdFth3E lzSDzlgk2GqSYU2Q7mPZHl2lwzB4iv/QfnERbd9GVmlQcHNFaFc7xUvZWlWXvCpqPYkO o6Zmc12ETH0/jQ27Xd1hlvu7BSCErGgNTY+H/1nFMvml58vuGkPbBbZkzsTtW8WoP03/ Y4WbeW6USe9nC8bslTFsNtM4htLeCzRYToxtLUGi5dp5Kk5NqzGymwvv5LpqgYlY5YHk gK6+aXlxOOOBA8dyoWJLnM3OnGPBlS4WsGYxQSe5EqTfaTzbzyxpRb6sinwTZMmsH37r BF7A== X-Gm-Message-State: APjAAAW62p7j7ZaBnWnztQvsmXYZSDXjmQOUaNLFx/zzOzybK/rAdmfm 5EVjG0gjAcgtoiMx2FliUEPOh2xDUDQ= X-Google-Smtp-Source: APXvYqyAQvDHFPlwol084YtmoxeK9ALvce8BoTf5MrLynyThLVJi07+L0kfZvMdagU1zen+M1p+zPA== X-Received: by 2002:a7b:c088:: with SMTP id r8mr33668wmh.18.1582563545996; Mon, 24 Feb 2020 08:59:05 -0800 (PST) Received: from localhost.localdomain ([2001:171b:c9a8:fbc0:116c:c27a:3e7f:5eaf]) by smtp.gmail.com with ESMTPSA id b10sm19473978wrt.90.2020.02.24.08.59.05 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 24 Feb 2020 08:59:05 -0800 (PST) From: Jean-Philippe Brucker To: linux-pci@vger.kernel.org, linux-arm-kernel@lists.infradead.org, iommu@lists.linux-foundation.org, will@kernel.org, bhelgaas@google.com Cc: joro@8bytes.org, robin.murphy@arm.com, jonathan.cameron@huawei.com, zhangfei.gao@linaro.org, robh@kernel.org Subject: [PATCH v2 4/6] iommu/arm-smmu-v3: Add command queue batching helpers Date: Mon, 24 Feb 2020 17:58:44 +0100 Message-Id: <20200224165846.345993-5-jean-philippe@linaro.org> X-Mailer: git-send-email 2.25.0 In-Reply-To: <20200224165846.345993-1-jean-philippe@linaro.org> References: <20200224165846.345993-1-jean-philippe@linaro.org> MIME-Version: 1.0 Sender: linux-pci-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org As more functions will implement command queue batching, add two helpers to simplify building a command list. Signed-off-by: Jean-Philippe Brucker --- drivers/iommu/arm-smmu-v3.c | 37 ++++++++++++++++++++++++++----------- 1 file changed, 26 insertions(+), 11 deletions(-) diff --git a/drivers/iommu/arm-smmu-v3.c b/drivers/iommu/arm-smmu-v3.c index 068a16d0eabe..beeec366bc41 100644 --- a/drivers/iommu/arm-smmu-v3.c +++ b/drivers/iommu/arm-smmu-v3.c @@ -548,6 +548,11 @@ struct arm_smmu_cmdq { atomic_t lock; }; +struct arm_smmu_cmdq_batch { + u64 cmds[CMDQ_BATCH_ENTRIES * CMDQ_ENT_DWORDS]; + int num; +}; + struct arm_smmu_evtq { struct arm_smmu_queue q; u32 max_stalls; @@ -1482,6 +1487,24 @@ static int arm_smmu_cmdq_issue_sync(struct arm_smmu_device *smmu) return arm_smmu_cmdq_issue_cmdlist(smmu, NULL, 0, true); } +static void arm_smmu_cmdq_batch_add(struct arm_smmu_device *smmu, + struct arm_smmu_cmdq_batch *cmds, + struct arm_smmu_cmdq_ent *cmd) +{ + if (cmds->num == CMDQ_BATCH_ENTRIES) { + arm_smmu_cmdq_issue_cmdlist(smmu, cmds->cmds, cmds->num, false); + cmds->num = 0; + } + arm_smmu_cmdq_build_cmd(&cmds->cmds[cmds->num * CMDQ_ENT_DWORDS], cmd); + cmds->num++; +} + +static int arm_smmu_cmdq_batch_submit(struct arm_smmu_device *smmu, + struct arm_smmu_cmdq_batch *cmds) +{ + return arm_smmu_cmdq_issue_cmdlist(smmu, cmds->cmds, cmds->num, true); +} + /* Context descriptor manipulation functions */ static void arm_smmu_sync_cd(struct arm_smmu_domain *smmu_domain, int ssid, bool leaf) @@ -2220,10 +2243,9 @@ static void arm_smmu_tlb_inv_range(unsigned long iova, size_t size, size_t granule, bool leaf, struct arm_smmu_domain *smmu_domain) { - u64 cmds[CMDQ_BATCH_ENTRIES * CMDQ_ENT_DWORDS]; struct arm_smmu_device *smmu = smmu_domain->smmu; unsigned long start = iova, end = iova + size; - int i = 0; + struct arm_smmu_cmdq_batch cmds = {}; struct arm_smmu_cmdq_ent cmd = { .tlbi = { .leaf = leaf, @@ -2242,18 +2264,11 @@ static void arm_smmu_tlb_inv_range(unsigned long iova, size_t size, } while (iova < end) { - if (i == CMDQ_BATCH_ENTRIES) { - arm_smmu_cmdq_issue_cmdlist(smmu, cmds, i, false); - i = 0; - } - cmd.tlbi.addr = iova; - arm_smmu_cmdq_build_cmd(&cmds[i * CMDQ_ENT_DWORDS], &cmd); + arm_smmu_cmdq_batch_add(smmu, &cmds, &cmd); iova += granule; - i++; } - - arm_smmu_cmdq_issue_cmdlist(smmu, cmds, i, true); + arm_smmu_cmdq_batch_submit(smmu, &cmds); /* * Unfortunately, this can't be leaf-only since we may have From patchwork Mon Feb 24 16:58:45 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jean-Philippe Brucker X-Patchwork-Id: 11401063 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 0538192A for ; Mon, 24 Feb 2020 16:59:11 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id D919721556 for ; Mon, 24 Feb 2020 16:59:10 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b="fxdWSvL7" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727746AbgBXQ7J (ORCPT ); Mon, 24 Feb 2020 11:59:09 -0500 Received: from mail-wm1-f65.google.com ([209.85.128.65]:55848 "EHLO mail-wm1-f65.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727980AbgBXQ7I (ORCPT ); Mon, 24 Feb 2020 11:59:08 -0500 Received: by mail-wm1-f65.google.com with SMTP id q9so31919wmj.5 for ; Mon, 24 Feb 2020 08:59:07 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=HsNqDpM5GYOb0rP3SKvjSjiXAgrPp7/dDnFzXTvMfm0=; b=fxdWSvL7/PlqMuOjCbpoyDK0oOMZfWMx8Bn0sQViiqiE3H2JO3U4xq75c/gD0NU1IB 5LX+dV8sT5h1EGNCfmF3/Oc4OwWIiPSscSgJ9EdoXDkOLYlFea7GgkHZEKGQ4n3RX2xc tE0Z9fTNGrynG92HHnK+qNkjvWnd1/LmhQMR8Wkp+U8ifjROHa1WaQQXNc85lp2jIC+P lz56h7v5lvnlEFIVnmE4jx/Q/BDcbY3YEs++7gzkztWZ4dCkTt2gdyf4zasHzdwGusEP V4s2WLd6VesfOZIZPnXJjsJsg1FHR6SX/xW3H+Vqf2K7q/htUfeBaETDNaFwfGHo9nb3 TUyw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=HsNqDpM5GYOb0rP3SKvjSjiXAgrPp7/dDnFzXTvMfm0=; b=iPXPj85TnvnhdbFMfG5nUXIM/e6zkjizZpwBJebMs7Hn8wCy15Y1o9CcyTYq+qgT0S rQ5xvIZdx7MkYt3MQ5f3Wnmi7Cg1vzlGNvVPJJOmo02rdv7U6sr8bH3y6FPZ9LmZPAo3 JIDyCbguhYi4lOiUeVzpFnxS6g1N5CUYphc8q4w6gVyPQHJLFFY8Pxcp7bWD08uhn2YT FZNJMk5PvHXNaP6KNEyDF06yhz+it9+JSdc9yu/M+3i2rV8n8UNepCriscRfFaNISbD0 7G+POn/xI/X5lKnIbaoxB8tmVwA3mWeAG4OGiJxDJXz9K0MS7ZviG+4Kheko4rtC25VI hTIg== X-Gm-Message-State: APjAAAWfZU6HYnEpii9dRfZDYcL3m0WFyvANSKf5Pl+Ow3joU+pvDdiV tAKBlgcB19lAQqr53BpMkqhAxh8f3+c= X-Google-Smtp-Source: APXvYqxzLvSxV15TQeDzuS7Vx3zfQgb1/0PPrdplrfLn1QNB4Up5cq5+1azMjxbcKStUM/cjgwhbxA== X-Received: by 2002:a7b:c416:: with SMTP id k22mr35940wmi.10.1582563546869; Mon, 24 Feb 2020 08:59:06 -0800 (PST) Received: from localhost.localdomain ([2001:171b:c9a8:fbc0:116c:c27a:3e7f:5eaf]) by smtp.gmail.com with ESMTPSA id b10sm19473978wrt.90.2020.02.24.08.59.06 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 24 Feb 2020 08:59:06 -0800 (PST) From: Jean-Philippe Brucker To: linux-pci@vger.kernel.org, linux-arm-kernel@lists.infradead.org, iommu@lists.linux-foundation.org, will@kernel.org, bhelgaas@google.com Cc: joro@8bytes.org, robin.murphy@arm.com, jonathan.cameron@huawei.com, zhangfei.gao@linaro.org, robh@kernel.org Subject: [PATCH v2 5/6] iommu/arm-smmu-v3: Batch context descriptor invalidation Date: Mon, 24 Feb 2020 17:58:45 +0100 Message-Id: <20200224165846.345993-6-jean-philippe@linaro.org> X-Mailer: git-send-email 2.25.0 In-Reply-To: <20200224165846.345993-1-jean-philippe@linaro.org> References: <20200224165846.345993-1-jean-philippe@linaro.org> MIME-Version: 1.0 Sender: linux-pci-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org Rather than publishing one command at a time when invalidating a context descriptor, batch the commands for all SIDs in the domain. Signed-off-by: Jean-Philippe Brucker --- drivers/iommu/arm-smmu-v3.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/drivers/iommu/arm-smmu-v3.c b/drivers/iommu/arm-smmu-v3.c index beeec366bc41..12b2a0fa747e 100644 --- a/drivers/iommu/arm-smmu-v3.c +++ b/drivers/iommu/arm-smmu-v3.c @@ -1512,6 +1512,7 @@ static void arm_smmu_sync_cd(struct arm_smmu_domain *smmu_domain, size_t i; unsigned long flags; struct arm_smmu_master *master; + struct arm_smmu_cmdq_batch cmds = {}; struct arm_smmu_device *smmu = smmu_domain->smmu; struct arm_smmu_cmdq_ent cmd = { .opcode = CMDQ_OP_CFGI_CD, @@ -1525,12 +1526,12 @@ static void arm_smmu_sync_cd(struct arm_smmu_domain *smmu_domain, list_for_each_entry(master, &smmu_domain->devices, domain_head) { for (i = 0; i < master->num_sids; i++) { cmd.cfgi.sid = master->sids[i]; - arm_smmu_cmdq_issue_cmd(smmu, &cmd); + arm_smmu_cmdq_batch_add(smmu, &cmds, &cmd); } } spin_unlock_irqrestore(&smmu_domain->devices_lock, flags); - arm_smmu_cmdq_issue_sync(smmu); + arm_smmu_cmdq_batch_submit(smmu, &cmds); } static int arm_smmu_alloc_cd_leaf_table(struct arm_smmu_device *smmu, From patchwork Mon Feb 24 16:58:46 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jean-Philippe Brucker X-Patchwork-Id: 11401065 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 886FD1580 for ; Mon, 24 Feb 2020 16:59:11 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 6857C21927 for ; Mon, 24 Feb 2020 16:59:11 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b="MVNhbcg8" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727980AbgBXQ7K (ORCPT ); Mon, 24 Feb 2020 11:59:10 -0500 Received: from mail-wm1-f66.google.com ([209.85.128.66]:35577 "EHLO mail-wm1-f66.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728022AbgBXQ7K (ORCPT ); Mon, 24 Feb 2020 11:59:10 -0500 Received: by mail-wm1-f66.google.com with SMTP id b17so71451wmb.0 for ; Mon, 24 Feb 2020 08:59:08 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=8wR4JvjjnPlxADINM207T0IRGzt73ZsNoWPOZadfSNk=; b=MVNhbcg8wD57oQ9I2JyLERVR2yKCiLO+U8+KIMgmDXq5uE3Xz17SRqISJlYfd4ir0f R4kd49dYtzzGIjmUKtm2Nil6peOKseDxSX6LLgZymQYn3PjxOlFG7DMwufOltiWTpMwW nAvv/0bXQehpLWgHnz/JM8FzL1XTmsAaoNNKJDC2OGvlpi33JinznQOQFLaMEP8oSMB6 p2D8+W5IrG6gxs6dgpvB2oEc9zTKeiWuWpX4iz8pcAiJETSzXOdN8W2cI7+0K3InfFhi mhvq6KoZzQD35eEblEKVN/M5SfSNB4cjLRfBZyi4z0D6gsum8FD4M5GXqNy0vqyUQ8Ek kgJA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=8wR4JvjjnPlxADINM207T0IRGzt73ZsNoWPOZadfSNk=; b=ptsN54sIQlaAiystK9pa+Qih934TCkkhy4p5/r3lLe4fwYv1oZsGpwEIG78QPbdZml /urxQHO8QhuCG1665kjfMrRyHpjx2EeTeI5zvxCRrl9/rnri55VXvPa+ZTvTyX2LY1Vq 1YHG8PqfzoKCDLgVz2fLIbjOh1hmTYTIrtK3L7iTFcBxch/sG+1VN2zuO8a/eRuYXViG /7MKeiF46xM8orRFwB04nzqoCqV2e+f5dR9GbdtsGHyvswtkq9PbzNKyb4g2ovf8QQI4 QkuTZ5izr76s3jaNkkCRCvPFo90oIBbq6rwG3nTQqwFdsDIHreYTog/RYvgmAwpFJCMq a4pw== X-Gm-Message-State: APjAAAWJr0qIQu/XJMfH6yLS97MCC3j6hUpOBAYvAQR6hMXG0dH+kKoE CvQpqQBMs81TwEtHycFkjY2A20paPcE= X-Google-Smtp-Source: APXvYqy5NCQ4cadqxMUKN5tm7gQWKMw6tJ7/zLaq1h3uCOJopLS3L3IpJR06WXC/akjTuCTTeW4gnw== X-Received: by 2002:a05:600c:54e:: with SMTP id k14mr22152694wmc.115.1582563547735; Mon, 24 Feb 2020 08:59:07 -0800 (PST) Received: from localhost.localdomain ([2001:171b:c9a8:fbc0:116c:c27a:3e7f:5eaf]) by smtp.gmail.com with ESMTPSA id b10sm19473978wrt.90.2020.02.24.08.59.06 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 24 Feb 2020 08:59:07 -0800 (PST) From: Jean-Philippe Brucker To: linux-pci@vger.kernel.org, linux-arm-kernel@lists.infradead.org, iommu@lists.linux-foundation.org, will@kernel.org, bhelgaas@google.com Cc: joro@8bytes.org, robin.murphy@arm.com, jonathan.cameron@huawei.com, zhangfei.gao@linaro.org, robh@kernel.org Subject: [PATCH v2 6/6] iommu/arm-smmu-v3: Batch ATC invalidation commands Date: Mon, 24 Feb 2020 17:58:46 +0100 Message-Id: <20200224165846.345993-7-jean-philippe@linaro.org> X-Mailer: git-send-email 2.25.0 In-Reply-To: <20200224165846.345993-1-jean-philippe@linaro.org> References: <20200224165846.345993-1-jean-philippe@linaro.org> MIME-Version: 1.0 Sender: linux-pci-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org From: Rob Herring Similar to commit 2af2e72b18b4 ("iommu/arm-smmu-v3: Defer TLB invalidation until ->iotlb_sync()"), build up a list of ATC invalidation commands and submit them all at once to the command queue instead of one-by-one. As there is only one caller of arm_smmu_atc_inv_master() left, we can simplify it and avoid passing in struct arm_smmu_cmdq_ent. Cc: Jean-Philippe Brucker Cc: Will Deacon Cc: Robin Murphy Cc: Joerg Roedel Signed-off-by: Rob Herring Signed-off-by: Jean-Philippe Brucker --- drivers/iommu/arm-smmu-v3.c | 31 ++++++++++++++++++------------- 1 file changed, 18 insertions(+), 13 deletions(-) diff --git a/drivers/iommu/arm-smmu-v3.c b/drivers/iommu/arm-smmu-v3.c index 12b2a0fa747e..4f0a38dae6db 100644 --- a/drivers/iommu/arm-smmu-v3.c +++ b/drivers/iommu/arm-smmu-v3.c @@ -2158,17 +2158,16 @@ arm_smmu_atc_inv_to_cmd(int ssid, unsigned long iova, size_t size, cmd->atc.size = log2_span; } -static int arm_smmu_atc_inv_master(struct arm_smmu_master *master, - struct arm_smmu_cmdq_ent *cmd) +static int arm_smmu_atc_inv_master(struct arm_smmu_master *master) { int i; + struct arm_smmu_cmdq_ent cmd; - if (!master->ats_enabled) - return 0; + arm_smmu_atc_inv_to_cmd(0, 0, 0, &cmd); for (i = 0; i < master->num_sids; i++) { - cmd->atc.sid = master->sids[i]; - arm_smmu_cmdq_issue_cmd(master->smmu, cmd); + cmd.atc.sid = master->sids[i]; + arm_smmu_cmdq_issue_cmd(master->smmu, &cmd); } return arm_smmu_cmdq_issue_sync(master->smmu); @@ -2177,10 +2176,11 @@ static int arm_smmu_atc_inv_master(struct arm_smmu_master *master, static int arm_smmu_atc_inv_domain(struct arm_smmu_domain *smmu_domain, int ssid, unsigned long iova, size_t size) { - int ret = 0; + int i; unsigned long flags; struct arm_smmu_cmdq_ent cmd; struct arm_smmu_master *master; + struct arm_smmu_cmdq_batch cmds = {}; if (!(smmu_domain->smmu->features & ARM_SMMU_FEAT_ATS)) return 0; @@ -2205,11 +2205,18 @@ static int arm_smmu_atc_inv_domain(struct arm_smmu_domain *smmu_domain, arm_smmu_atc_inv_to_cmd(ssid, iova, size, &cmd); spin_lock_irqsave(&smmu_domain->devices_lock, flags); - list_for_each_entry(master, &smmu_domain->devices, domain_head) - ret |= arm_smmu_atc_inv_master(master, &cmd); + list_for_each_entry(master, &smmu_domain->devices, domain_head) { + if (!master->ats_enabled) + continue; + + for (i = 0; i < master->num_sids; i++) { + cmd.atc.sid = master->sids[i]; + arm_smmu_cmdq_batch_add(smmu_domain->smmu, &cmds, &cmd); + } + } spin_unlock_irqrestore(&smmu_domain->devices_lock, flags); - return ret ? -ETIMEDOUT : 0; + return arm_smmu_cmdq_batch_submit(smmu_domain->smmu, &cmds); } /* IO_PGTABLE API */ @@ -2629,7 +2636,6 @@ static void arm_smmu_enable_ats(struct arm_smmu_master *master) static void arm_smmu_disable_ats(struct arm_smmu_master *master) { - struct arm_smmu_cmdq_ent cmd; struct arm_smmu_domain *smmu_domain = master->domain; if (!master->ats_enabled) @@ -2641,8 +2647,7 @@ static void arm_smmu_disable_ats(struct arm_smmu_master *master) * ATC invalidation via the SMMU. */ wmb(); - arm_smmu_atc_inv_to_cmd(0, 0, 0, &cmd); - arm_smmu_atc_inv_master(master, &cmd); + arm_smmu_atc_inv_master(master); atomic_dec(&smmu_domain->nr_ats_masters); }