From patchwork Fri Jan 26 01:39:58 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ethan Zhao X-Patchwork-Id: 13531931 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.10]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7D364522E; Fri, 26 Jan 2024 01:40:26 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.175.65.10 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706233229; cv=none; b=mCLAhwEMrgSB+51dMLL4OorBUZvaD9dnjSIb0HeiYUJpBV8P8OPahQ2K8BFQMmEfk19YtOd3GYiQWjbABLyaQclWqcnl5wsPg4JjeK78ilOd7KaJEWG1VM/RRFi4Zljx9swLPrzHuSc7Xj6f+xsKcWPsHu62Bd5/wHEoNRllsAo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706233229; c=relaxed/simple; bh=dEKy3mt/WPL6E30nghtcx9sbGmG9T5AiLwENf8K/kU4=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=Y0JEP5XIEjmP0HMl3vEK3GiWZKPxC0vHgmwR8WHH0gVPjY5eTFa9Ow9hXhh3K4cW2FDyrnj2tGiJ6XKBtxPtX+6XtL/D6RI5obwXYg82vCrdMfVaZ345rWoLIRGNygu8YEsgzLTkV/d0Lh3cCGdV7mGpDK4lex+WIH8vjuNYybg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com; spf=none smtp.mailfrom=linux.intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=hFpKaysN; arc=none smtp.client-ip=198.175.65.10 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.mailfrom=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="hFpKaysN" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1706233226; x=1737769226; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=dEKy3mt/WPL6E30nghtcx9sbGmG9T5AiLwENf8K/kU4=; b=hFpKaysN3cQSoyCaMPhka0oe39bMBe4cnlyo2WTwxRZPJ9E3WmLq/Jwr 1YnfoNT/tB5STV24o3KPT3wkzHMqHXPKy7CubtCzFsCynx6MJ/O6CE6kx mUGh22/7+/B0pkYhaqez//m/uJ9AroX6SifdpGcgEjX92AhxabvQmSy4F oiDzMo4L8cADJcjKNcDths22JyigytfFrYtzGtX28KqbI0PVakdTSLhg0 koMCU3drEbbU6paCh0w5K6sXmrfBw0xx0wu/e1G/EwGRgjfTm56J23SHF De6hbK7CHR9xkZvLX2mqpsoFSO5dNU0DcF5NZ09OwS1hZ5fzNSLeqBytY g==; X-IronPort-AV: E=McAfee;i="6600,9927,10964"; a="15718453" X-IronPort-AV: E=Sophos;i="6.05,216,1701158400"; d="scan'208";a="15718453" Received: from fmviesa003.fm.intel.com ([10.60.135.143]) by orvoesa102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 25 Jan 2024 17:40:26 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.05,216,1701158400"; d="scan'208";a="2533094" Received: from unknown (HELO ply01-vm-store.amr.corp.intel.com) ([10.238.153.201]) by fmviesa003.fm.intel.com with ESMTP; 25 Jan 2024 17:40:23 -0800 From: Ethan Zhao To: baolu.lu@linux.intel.com, bhelgaas@google.com, robin.murphy@arm.com, jgg@ziepe.ca Cc: kevin.tian@intel.com, dwmw2@infradead.org, will@kernel.org, lukas@wunner.de, yi.l.liu@intel.com, iommu@lists.linux.dev, linux-kernel@vger.kernel.org, linux-pci@vger.kernel.org, Ethan Zhao , Haorong Ye Subject: [PATCH v11 1/5] PCI: make pci_dev_is_disconnected() helper public for other drivers Date: Thu, 25 Jan 2024 20:39:58 -0500 Message-Id: <20240126014002.481294-2-haifeng.zhao@linux.intel.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20240126014002.481294-1-haifeng.zhao@linux.intel.com> References: <20240126014002.481294-1-haifeng.zhao@linux.intel.com> Precedence: bulk X-Mailing-List: linux-pci@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Make pci_dev_is_disconnected() public so that it can be called from Intel VT-d driver to quickly fix/workaround the surprise removal unplug hang issue for those ATS capable devices on PCIe switch downstream hotplug capable ports. Beside pci_device_is_present() function, this one has no config space space access, so is light enough to optimize the normal pure surprise removal and safe removal flow. Tested-by: Haorong Ye Signed-off-by: Ethan Zhao --- drivers/pci/pci.h | 5 ----- include/linux/pci.h | 5 +++++ 2 files changed, 5 insertions(+), 5 deletions(-) diff --git a/drivers/pci/pci.h b/drivers/pci/pci.h index 2336a8d1edab..94118c4cff54 100644 --- a/drivers/pci/pci.h +++ b/drivers/pci/pci.h @@ -368,11 +368,6 @@ static inline int pci_dev_set_disconnected(struct pci_dev *dev, void *unused) return 0; } -static inline bool pci_dev_is_disconnected(const struct pci_dev *dev) -{ - return dev->error_state == pci_channel_io_perm_failure; -} - /* pci_dev priv_flags */ #define PCI_DEV_ADDED 0 #define PCI_DPC_RECOVERED 1 diff --git a/include/linux/pci.h b/include/linux/pci.h index add9368e6314..19dfe5f16fc9 100644 --- a/include/linux/pci.h +++ b/include/linux/pci.h @@ -2512,6 +2512,11 @@ static inline struct pci_dev *pcie_find_root_port(struct pci_dev *dev) return NULL; } +static inline bool pci_dev_is_disconnected(const struct pci_dev *dev) +{ + return dev->error_state == pci_channel_io_perm_failure; +} + void pci_request_acs(void); bool pci_acs_enabled(struct pci_dev *pdev, u16 acs_flags); bool pci_acs_path_enabled(struct pci_dev *start, From patchwork Fri Jan 26 01:39:59 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ethan Zhao X-Patchwork-Id: 13531932 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.10]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A5E638F66; Fri, 26 Jan 2024 01:40:30 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.175.65.10 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706233232; cv=none; b=gnPX3ufqS5ullOOGbG56shTkVy1Ye0ZUc5goEAoL0OJeFltm8/ulC3b4HoTDnbYk3N4x8tIuWplavzN5IiG3oSI3awc4ULf0t7WMEL/56c1nhY2/JeY5yB3xRt6w9QzQOO4lb6WZ15XK5ohzytLEsaWO1jO5LiQs+x2Fwl2VYzE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706233232; c=relaxed/simple; bh=q2qOvYluDvLjuk2djtt50RKGW5uxSvlbJiVtYpV+LNo=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=OmpSBG6s4RReS5jwYxUNM6XPejijV/DbimpavImzC8KE7SUZjPCex3pIp/a4KZ48ZbSrGLKV+r2onsXz0/tse5p3zJqiPznFUY+T0fTjKB4O+zOANZwh8JNtZfj/SqRlXe1QJdgUxU7LDeSaR1fWhjjDTieSaSXRloXT69aoE98= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com; spf=none smtp.mailfrom=linux.intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=nl9HJmOX; arc=none smtp.client-ip=198.175.65.10 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.mailfrom=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="nl9HJmOX" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1706233231; x=1737769231; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=q2qOvYluDvLjuk2djtt50RKGW5uxSvlbJiVtYpV+LNo=; b=nl9HJmOXc5Gs2iDjLIlxc2rKDJ+Oh0KQE72V7Dti9oCVAYXBxGKeHNDW 5OY+wP0gzEdpB1qfIsS0W/yA03N7TTEFK+jpB5MHUB0IQNHNBrj7bi/Om DyIh9S27IM9xGQHWhaG7XYcSdvwGLRvZkaPvSmRe4pRVmWN8iXtnC4MHo pw0efkeOLFMVHdya6izdzwDLBTB2zgqh17YJxj9jmHvZahRY8ON3HODCP gw/xfLCA4s7EBC2SyWzM3Kn4v+dnMwNcGmwARuREio6AU9Yn1PMP2lFZZ aGJEmWoi/ySh4mA/dLtEKT60/Z0E+EVPXQWs3SPNtHsAfJ1g/nADnvqIQ g==; X-IronPort-AV: E=McAfee;i="6600,9927,10964"; a="15718469" X-IronPort-AV: E=Sophos;i="6.05,216,1701158400"; d="scan'208";a="15718469" Received: from fmviesa003.fm.intel.com ([10.60.135.143]) by orvoesa102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 25 Jan 2024 17:40:30 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.05,216,1701158400"; d="scan'208";a="2533117" Received: from unknown (HELO ply01-vm-store.amr.corp.intel.com) ([10.238.153.201]) by fmviesa003.fm.intel.com with ESMTP; 25 Jan 2024 17:40:26 -0800 From: Ethan Zhao To: baolu.lu@linux.intel.com, bhelgaas@google.com, robin.murphy@arm.com, jgg@ziepe.ca Cc: kevin.tian@intel.com, dwmw2@infradead.org, will@kernel.org, lukas@wunner.de, yi.l.liu@intel.com, iommu@lists.linux.dev, linux-kernel@vger.kernel.org, linux-pci@vger.kernel.org, Ethan Zhao , Haorong Ye Subject: [PATCH v11 2/5] iommu/vt-d: don't issue ATS Invalidation request when device is disconnected Date: Thu, 25 Jan 2024 20:39:59 -0500 Message-Id: <20240126014002.481294-3-haifeng.zhao@linux.intel.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20240126014002.481294-1-haifeng.zhao@linux.intel.com> References: <20240126014002.481294-1-haifeng.zhao@linux.intel.com> Precedence: bulk X-Mailing-List: linux-pci@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 For those endpoint devices connect to system via hotplug capable ports, users could request a hot reset to the device by flapping device's link through setting the slot's link control register, as pciehp_ist() DLLSC interrupt sequence response, pciehp will unload the device driver and then power it off. thus cause an IOMMU device-TLB invalidation (Intel VT-d spec, or ATS Invalidation in PCIe spec r6.1) request for non-existence target device to be sent and deadly loop to retry that request after ITE fault triggered in interrupt context. That would cause following continuous hard lockup warning and system hang [ 4211.433662] pcieport 0000:17:01.0: pciehp: Slot(108): Link Down [ 4211.433664] pcieport 0000:17:01.0: pciehp: Slot(108): Card not present [ 4223.822591] NMI watchdog: Watchdog detected hard LOCKUP on cpu 144 [ 4223.822622] CPU: 144 PID: 1422 Comm: irq/57-pciehp Kdump: loaded Tainted: G S OE kernel version xxxx [ 4223.822623] Hardware name: vendorname xxxx 666-106, BIOS 01.01.02.03.01 05/15/2023 [ 4223.822623] RIP: 0010:qi_submit_sync+0x2c0/0x490 [ 4223.822624] Code: 48 be 00 00 00 00 00 08 00 00 49 85 74 24 20 0f 95 c1 48 8b 57 10 83 c1 04 83 3c 1a 03 0f 84 a2 01 00 00 49 8b 04 24 8b 70 34 <40> f6 c6 1 0 74 17 49 8b 04 24 8b 80 80 00 00 00 89 c2 d3 fa 41 39 [ 4223.822624] RSP: 0018:ffffc4f074f0bbb8 EFLAGS: 00000093 [ 4223.822625] RAX: ffffc4f040059000 RBX: 0000000000000014 RCX: 0000000000000005 [ 4223.822625] RDX: ffff9f3841315800 RSI: 0000000000000000 RDI: ffff9f38401a8340 [ 4223.822625] RBP: ffff9f38401a8340 R08: ffffc4f074f0bc00 R09: 0000000000000000 [ 4223.822626] R10: 0000000000000010 R11: 0000000000000018 R12: ffff9f384005e200 [ 4223.822626] R13: 0000000000000004 R14: 0000000000000046 R15: 0000000000000004 [ 4223.822626] FS: 0000000000000000(0000) GS:ffffa237ae400000(0000) knlGS:0000000000000000 [ 4223.822627] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4223.822627] CR2: 00007ffe86515d80 CR3: 000002fd3000a001 CR4: 0000000000770ee0 [ 4223.822627] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 4223.822628] DR3: 0000000000000000 DR6: 00000000fffe07f0 DR7: 0000000000000400 [ 4223.822628] PKRU: 55555554 [ 4223.822628] Call Trace: [ 4223.822628] qi_flush_dev_iotlb+0xb1/0xd0 [ 4223.822628] __dmar_remove_one_dev_info+0x224/0x250 [ 4223.822629] dmar_remove_one_dev_info+0x3e/0x50 [ 4223.822629] intel_iommu_release_device+0x1f/0x30 [ 4223.822629] iommu_release_device+0x33/0x60 [ 4223.822629] iommu_bus_notifier+0x7f/0x90 [ 4223.822630] blocking_notifier_call_chain+0x60/0x90 [ 4223.822630] device_del+0x2e5/0x420 [ 4223.822630] pci_remove_bus_device+0x70/0x110 [ 4223.822630] pciehp_unconfigure_device+0x7c/0x130 [ 4223.822631] pciehp_disable_slot+0x6b/0x100 [ 4223.822631] pciehp_handle_presence_or_link_change+0xd8/0x320 [ 4223.822631] pciehp_ist+0x176/0x180 [ 4223.822631] ? irq_finalize_oneshot.part.50+0x110/0x110 [ 4223.822632] irq_thread_fn+0x19/0x50 [ 4223.822632] irq_thread+0x104/0x190 [ 4223.822632] ? irq_forced_thread_fn+0x90/0x90 [ 4223.822632] ? irq_thread_check_affinity+0xe0/0xe0 [ 4223.822633] kthread+0x114/0x130 [ 4223.822633] ? __kthread_cancel_work+0x40/0x40 [ 4223.822633] ret_from_fork+0x1f/0x30 [ 4223.822633] Kernel panic - not syncing: Hard LOCKUP [ 4223.822634] CPU: 144 PID: 1422 Comm: irq/57-pciehp Kdump: loaded Tainted: G S OE kernel version xxxx [ 4223.822634] Hardware name: vendorname xxxx 666-106, BIOS 01.01.02.03.01 05/15/2023 [ 4223.822634] Call Trace: [ 4223.822634] [ 4223.822635] dump_stack+0x6d/0x88 [ 4223.822635] panic+0x101/0x2d0 [ 4223.822635] ? ret_from_fork+0x11/0x30 [ 4223.822635] nmi_panic.cold.14+0xc/0xc [ 4223.822636] watchdog_overflow_callback.cold.8+0x6d/0x81 [ 4223.822636] __perf_event_overflow+0x4f/0xf0 [ 4223.822636] handle_pmi_common+0x1ef/0x290 [ 4223.822636] ? __set_pte_vaddr+0x28/0x40 [ 4223.822637] ? flush_tlb_one_kernel+0xa/0x20 [ 4223.822637] ? __native_set_fixmap+0x24/0x30 [ 4223.822637] ? ghes_copy_tofrom_phys+0x70/0x100 [ 4223.822637] ? __ghes_peek_estatus.isra.16+0x49/0xa0 [ 4223.822637] intel_pmu_handle_irq+0xba/0x2b0 [ 4223.822638] perf_event_nmi_handler+0x24/0x40 [ 4223.822638] nmi_handle+0x4d/0xf0 [ 4223.822638] default_do_nmi+0x49/0x100 [ 4223.822638] exc_nmi+0x134/0x180 [ 4223.822639] end_repeat_nmi+0x16/0x67 [ 4223.822639] RIP: 0010:qi_submit_sync+0x2c0/0x490 [ 4223.822639] Code: 48 be 00 00 00 00 00 08 00 00 49 85 74 24 20 0f 95 c1 48 8b 57 10 83 c1 04 83 3c 1a 03 0f 84 a2 01 00 00 49 8b 04 24 8b 70 34 <40> f6 c6 10 74 17 49 8b 04 24 8b 80 80 00 00 00 89 c2 d3 fa 41 39 [ 4223.822640] RSP: 0018:ffffc4f074f0bbb8 EFLAGS: 00000093 [ 4223.822640] RAX: ffffc4f040059000 RBX: 0000000000000014 RCX: 0000000000000005 [ 4223.822640] RDX: ffff9f3841315800 RSI: 0000000000000000 RDI: ffff9f38401a8340 [ 4223.822641] RBP: ffff9f38401a8340 R08: ffffc4f074f0bc00 R09: 0000000000000000 [ 4223.822641] R10: 0000000000000010 R11: 0000000000000018 R12: ffff9f384005e200 [ 4223.822641] R13: 0000000000000004 R14: 0000000000000046 R15: 0000000000000004 [ 4223.822641] ? qi_submit_sync+0x2c0/0x490 [ 4223.822642] ? qi_submit_sync+0x2c0/0x490 [ 4223.822642] [ 4223.822642] qi_flush_dev_iotlb+0xb1/0xd0 [ 4223.822642] __dmar_remove_one_dev_info+0x224/0x250 [ 4223.822643] dmar_remove_one_dev_info+0x3e/0x50 [ 4223.822643] intel_iommu_release_device+0x1f/0x30 [ 4223.822643] iommu_release_device+0x33/0x60 [ 4223.822643] iommu_bus_notifier+0x7f/0x90 [ 4223.822644] blocking_notifier_call_chain+0x60/0x90 [ 4223.822644] device_del+0x2e5/0x420 [ 4223.822644] pci_remove_bus_device+0x70/0x110 [ 4223.822644] pciehp_unconfigure_device+0x7c/0x130 [ 4223.822644] pciehp_disable_slot+0x6b/0x100 [ 4223.822645] pciehp_handle_presence_or_link_change+0xd8/0x320 [ 4223.822645] pciehp_ist+0x176/0x180 [ 4223.822645] ? irq_finalize_oneshot.part.50+0x110/0x110 [ 4223.822645] irq_thread_fn+0x19/0x50 [ 4223.822646] irq_thread+0x104/0x190 [ 4223.822646] ? irq_forced_thread_fn+0x90/0x90 [ 4223.822646] ? irq_thread_check_affinity+0xe0/0xe0 [ 4223.822646] kthread+0x114/0x130 [ 4223.822647] ? __kthread_cancel_work+0x40/0x40 [ 4223.822647] ret_from_fork+0x1f/0x30 [ 4223.822647] Kernel Offset: 0x6400000 from 0xffffffff81000000 (relocation range: 0xffffffff80000000-0xffffffffbfffffff) Such issue could be triggered by all kinds of regular surprise removal hotplug operation. like: 1. pull EP(endpoint device) out directly. 2. turn off EP's power. 3. bring the link down. etc. this patch aims to work for regular safe removal and surprise removal unplug. these hot unplug handling process could be optimized to fix the ATS Invalidation hang issue by calling pci_dev_is_disconnected() in function devtlb_invalidation_with_pasid() to check target device state to avoid sending meaningless ATS Invalidation request to iommu when device is gone. (see IMPLEMENTATION NOTE in PCIe spec r6.1 section 10.3.1) For safe removal, device wouldn't be removed until the whole software handling process is done, it wouldn't trigger the hard lock up issue caused by too long ATS Invalidation timeout wait. In safe removal path, device state isn't set to pci_channel_io_perm_failure in pciehp_unconfigure_device() by checking 'presence' parameter, calling pci_dev_is_disconnected() in devtlb_invalidation_with_pasid() will return false there, wouldn't break the function. For surprise removal, device state is set to pci_channel_io_perm_failure in pciehp_unconfigure_device(), means device is already gone (disconnected) call pci_dev_is_disconnected() in devtlb_invalidation_with_pasid() will return true to break the function not to send ATS Invalidation request to the disconnected device blindly, thus avoid to trigger further ITE fault, and ITE fault will block all invalidation request to be handled. furthermore retry the timeout request could trigger hard lockup. safe removal (present) & surprise removal (not present) pciehp_ist() pciehp_handle_presence_or_link_change() pciehp_disable_slot() remove_board() pciehp_unconfigure_device(presence) { if (!presence) pci_walk_bus(parent, pci_dev_set_disconnected, NULL); } this patch works for regular safe removal and surprise removal of ATS capable endpoint on PCIe switch downstream ports. Fixes: 6f7db75e1c46 ("iommu/vt-d: Add second level page table interface") Tested-by: Haorong Ye Signed-off-by: Ethan Zhao --- drivers/iommu/intel/pasid.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/drivers/iommu/intel/pasid.c b/drivers/iommu/intel/pasid.c index 3239cefa4c33..953592125e4a 100644 --- a/drivers/iommu/intel/pasid.c +++ b/drivers/iommu/intel/pasid.c @@ -214,6 +214,9 @@ devtlb_invalidation_with_pasid(struct intel_iommu *iommu, if (!info || !info->ats_enabled) return; + if (pci_dev_is_disconnected(to_pci_dev(dev))) + return; + sid = info->bus << 8 | info->devfn; qdep = info->ats_qdep; pfsid = info->pfsid; From patchwork Fri Jan 26 01:40:00 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ethan Zhao X-Patchwork-Id: 13531933 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.10]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7841EAD49; Fri, 26 Jan 2024 01:40:34 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.175.65.10 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706233236; cv=none; b=FUXLoXyjgcx28LHWSU/HyOaGZs61Ya8GiCfMzSYPQT38spFcdkYOofMdXKFJ/2G16kQ6KzyDy9BsXX8OsdMEqn4x3fUcSsOn455h69PLKSE1t7hXRXCeR/I8H/Awf2ony0apOzepOUzlNXYVKRYtgwqMVla3SxNNJo+BsGrdHIA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706233236; c=relaxed/simple; bh=CXF2rO6vNLbMO2Xepa62X8TKjLOzx8JqzSpWfxi/P1o=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=XytKy78fwvtULrwrJmndeYuHHg8Lm5qXlD8QwLZQbo99MMtWgTLpj+RXX0YjzR5M1Rvx3FBvJPycaIMPpizPwWF42TkvFKFO5lCcVvbkcYUiG2deA4t+QpcdGk7vpnZrMsddK0XbN8qXfy4n+lr98NLQqiGwB6TmtUkGU9t6eEI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com; spf=none smtp.mailfrom=linux.intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=GmtPGp5Q; arc=none smtp.client-ip=198.175.65.10 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.mailfrom=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="GmtPGp5Q" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1706233234; x=1737769234; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=CXF2rO6vNLbMO2Xepa62X8TKjLOzx8JqzSpWfxi/P1o=; b=GmtPGp5QysgKMBQXcedCph5byAWJ0JoEBzm8d0ke6+/copJbqESQVgRP pAFVRKmQ+mVKVjpFYX3pL8X+V7ASbLiOGYYz+jZeqX3Su6Yp5fKdnm+cT abd4r67w06ByFc8EU/vkyP0NC4j9bnbzUKfF8QKT3JMMdAxL5FXxO+85r 66qHlLSszIeaohO6jlxR160QMjDUgR6YbvJKftw3gQ0hNnn8ghU0sWac9 MpHXMKG/hXBuI3B23+V2SInA926SU8kQ8LKOtZb5LvwGQhclnGkgQV3G5 gL8GiUO49YOESt6F94LJ62r0/LOuZ6TixlcSNdxIY7Svf/+mLplBV6t/O w==; X-IronPort-AV: E=McAfee;i="6600,9927,10964"; a="15718487" X-IronPort-AV: E=Sophos;i="6.05,216,1701158400"; d="scan'208";a="15718487" Received: from fmviesa003.fm.intel.com ([10.60.135.143]) by orvoesa102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 25 Jan 2024 17:40:34 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.05,216,1701158400"; d="scan'208";a="2533129" Received: from unknown (HELO ply01-vm-store.amr.corp.intel.com) ([10.238.153.201]) by fmviesa003.fm.intel.com with ESMTP; 25 Jan 2024 17:40:30 -0800 From: Ethan Zhao To: baolu.lu@linux.intel.com, bhelgaas@google.com, robin.murphy@arm.com, jgg@ziepe.ca Cc: kevin.tian@intel.com, dwmw2@infradead.org, will@kernel.org, lukas@wunner.de, yi.l.liu@intel.com, iommu@lists.linux.dev, linux-kernel@vger.kernel.org, linux-pci@vger.kernel.org, Ethan Zhao Subject: [PATCH v11 3/5] iommu/vt-d: simplify parameters of qi_submit_sync() ATS invalidation callers Date: Thu, 25 Jan 2024 20:40:00 -0500 Message-Id: <20240126014002.481294-4-haifeng.zhao@linux.intel.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20240126014002.481294-1-haifeng.zhao@linux.intel.com> References: <20240126014002.481294-1-haifeng.zhao@linux.intel.com> Precedence: bulk X-Mailing-List: linux-pci@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 fold parameters back to struct device_domain_info *info instead of extract and pass them, thus decrease the number of the parameter passed for following functions qi_flush_dev_iotlb() qi_flush_dev_iotlb_pasid() quirk_extra_dev_tlb_flush() no function change. Signed-off-by: Ethan Zhao --- drivers/iommu/intel/dmar.c | 26 ++++++++++++++++++++++---- drivers/iommu/intel/iommu.c | 26 +++++++------------------- drivers/iommu/intel/iommu.h | 17 ++++++++--------- drivers/iommu/intel/nested.c | 9 ++------- drivers/iommu/intel/pasid.c | 9 ++------- drivers/iommu/intel/svm.c | 7 +++---- 6 files changed, 44 insertions(+), 50 deletions(-) diff --git a/drivers/iommu/intel/dmar.c b/drivers/iommu/intel/dmar.c index 23cb80d62a9a..ab5e1760bd59 100644 --- a/drivers/iommu/intel/dmar.c +++ b/drivers/iommu/intel/dmar.c @@ -1517,11 +1517,20 @@ void qi_flush_iotlb(struct intel_iommu *iommu, u16 did, u64 addr, qi_submit_sync(iommu, &desc, 1, 0); } -void qi_flush_dev_iotlb(struct intel_iommu *iommu, u16 sid, u16 pfsid, - u16 qdep, u64 addr, unsigned mask) +void qi_flush_dev_iotlb(struct intel_iommu *iommu, + struct device_domain_info *info, u64 addr, + unsigned int mask) { + u16 sid, qdep, pfsid; struct qi_desc desc; + if (!info || !info->ats_enabled) + return; + + sid = info->bus << 8 | info->devfn; + qdep = info->ats_qdep; + pfsid = info->pfsid; + /* * VT-d spec, section 4.3: * @@ -1590,11 +1599,20 @@ void qi_flush_piotlb(struct intel_iommu *iommu, u16 did, u32 pasid, u64 addr, } /* PASID-based device IOTLB Invalidate */ -void qi_flush_dev_iotlb_pasid(struct intel_iommu *iommu, u16 sid, u16 pfsid, - u32 pasid, u16 qdep, u64 addr, unsigned int size_order) +void qi_flush_dev_iotlb_pasid(struct intel_iommu *iommu, + struct device_domain_info *info, u64 addr, u32 pasid, + unsigned int size_order) { unsigned long mask = 1UL << (VTD_PAGE_SHIFT + size_order - 1); struct qi_desc desc = {.qw1 = 0, .qw2 = 0, .qw3 = 0}; + u16 sid, qdep, pfsid; + + if (!info || !dev_is_pci(info->dev)) + return; + + sid = info->bus << 8 | info->devfn; + qdep = info->ats_qdep; + pfsid = info->pfsid; /* * VT-d spec, section 4.3: diff --git a/drivers/iommu/intel/iommu.c b/drivers/iommu/intel/iommu.c index 6fb5f6fceea1..65bd867dafdd 100644 --- a/drivers/iommu/intel/iommu.c +++ b/drivers/iommu/intel/iommu.c @@ -1310,16 +1310,11 @@ static void iommu_disable_pci_caps(struct device_domain_info *info) static void __iommu_flush_dev_iotlb(struct device_domain_info *info, u64 addr, unsigned int mask) { - u16 sid, qdep; - if (!info || !info->ats_enabled) return; - sid = info->bus << 8 | info->devfn; - qdep = info->ats_qdep; - qi_flush_dev_iotlb(info->iommu, sid, info->pfsid, - qdep, addr, mask); - quirk_extra_dev_tlb_flush(info, addr, mask, IOMMU_NO_PASID, qdep); + qi_flush_dev_iotlb(info->iommu, info, addr, mask); + quirk_extra_dev_tlb_flush(info, addr, IOMMU_NO_PASID, mask); } static void iommu_flush_dev_iotlb(struct dmar_domain *domain, @@ -1342,11 +1337,7 @@ static void iommu_flush_dev_iotlb(struct dmar_domain *domain, if (!info->ats_enabled) continue; - qi_flush_dev_iotlb_pasid(info->iommu, - PCI_DEVID(info->bus, info->devfn), - info->pfsid, dev_pasid->pasid, - info->ats_qdep, addr, - mask); + qi_flush_dev_iotlb_pasid(info->iommu, info, addr, dev_pasid->pasid, mask); } spin_unlock_irqrestore(&domain->lock, flags); } @@ -4990,9 +4981,8 @@ static void __init check_tylersburg_isoch(void) * * As a reminder, #6 will *NEED* this quirk as we enable nested translation. */ -void quirk_extra_dev_tlb_flush(struct device_domain_info *info, - unsigned long address, unsigned long mask, - u32 pasid, u16 qdep) +void quirk_extra_dev_tlb_flush(struct device_domain_info *info, u32 pasid, + unsigned long address, unsigned long mask) { u16 sid; @@ -5001,11 +4991,9 @@ void quirk_extra_dev_tlb_flush(struct device_domain_info *info, sid = PCI_DEVID(info->bus, info->devfn); if (pasid == IOMMU_NO_PASID) { - qi_flush_dev_iotlb(info->iommu, sid, info->pfsid, - qdep, address, mask); + qi_flush_dev_iotlb(info->iommu, info, address, mask); } else { - qi_flush_dev_iotlb_pasid(info->iommu, sid, info->pfsid, - pasid, qdep, address, mask); + qi_flush_dev_iotlb_pasid(info->iommu, info, address, pasid, mask); } } diff --git a/drivers/iommu/intel/iommu.h b/drivers/iommu/intel/iommu.h index d02f916d8e59..f68f17476d85 100644 --- a/drivers/iommu/intel/iommu.h +++ b/drivers/iommu/intel/iommu.h @@ -1037,18 +1037,17 @@ void qi_flush_context(struct intel_iommu *iommu, u16 did, u16 sid, u8 fm, u64 type); void qi_flush_iotlb(struct intel_iommu *iommu, u16 did, u64 addr, unsigned int size_order, u64 type); -void qi_flush_dev_iotlb(struct intel_iommu *iommu, u16 sid, u16 pfsid, - u16 qdep, u64 addr, unsigned mask); - +void qi_flush_dev_iotlb(struct intel_iommu *iommu, + struct device_domain_info *info, u64 addr, + unsigned int mask); void qi_flush_piotlb(struct intel_iommu *iommu, u16 did, u32 pasid, u64 addr, unsigned long npages, bool ih); -void qi_flush_dev_iotlb_pasid(struct intel_iommu *iommu, u16 sid, u16 pfsid, - u32 pasid, u16 qdep, u64 addr, - unsigned int size_order); -void quirk_extra_dev_tlb_flush(struct device_domain_info *info, - unsigned long address, unsigned long pages, - u32 pasid, u16 qdep); +void qi_flush_dev_iotlb_pasid(struct intel_iommu *iommu, + struct device_domain_info *info, u64 addr, + u32 pasid, unsigned int size_order); +void quirk_extra_dev_tlb_flush(struct device_domain_info *info, u32 pasid, + unsigned long address, unsigned long mask); void qi_flush_pasid_cache(struct intel_iommu *iommu, u16 did, u64 granu, u32 pasid); diff --git a/drivers/iommu/intel/nested.c b/drivers/iommu/intel/nested.c index f26c7f1c46cc..d15f72b55940 100644 --- a/drivers/iommu/intel/nested.c +++ b/drivers/iommu/intel/nested.c @@ -78,18 +78,13 @@ static void nested_flush_dev_iotlb(struct dmar_domain *domain, u64 addr, { struct device_domain_info *info; unsigned long flags; - u16 sid, qdep; spin_lock_irqsave(&domain->lock, flags); list_for_each_entry(info, &domain->devices, link) { if (!info->ats_enabled) continue; - sid = info->bus << 8 | info->devfn; - qdep = info->ats_qdep; - qi_flush_dev_iotlb(info->iommu, sid, info->pfsid, - qdep, addr, mask); - quirk_extra_dev_tlb_flush(info, addr, mask, - IOMMU_NO_PASID, qdep); + qi_flush_dev_iotlb(info->iommu, info, addr, mask); + quirk_extra_dev_tlb_flush(info, IOMMU_NO_PASID, addr, mask); } spin_unlock_irqrestore(&domain->lock, flags); } diff --git a/drivers/iommu/intel/pasid.c b/drivers/iommu/intel/pasid.c index 953592125e4a..5dacdea3cab7 100644 --- a/drivers/iommu/intel/pasid.c +++ b/drivers/iommu/intel/pasid.c @@ -208,7 +208,6 @@ devtlb_invalidation_with_pasid(struct intel_iommu *iommu, struct device *dev, u32 pasid) { struct device_domain_info *info; - u16 sid, qdep, pfsid; info = dev_iommu_priv_get(dev); if (!info || !info->ats_enabled) @@ -217,10 +216,6 @@ devtlb_invalidation_with_pasid(struct intel_iommu *iommu, if (pci_dev_is_disconnected(to_pci_dev(dev))) return; - sid = info->bus << 8 | info->devfn; - qdep = info->ats_qdep; - pfsid = info->pfsid; - /* * When PASID 0 is used, it indicates RID2PASID(DMA request w/o PASID), * devTLB flush w/o PASID should be used. For non-zero PASID under @@ -228,9 +223,9 @@ devtlb_invalidation_with_pasid(struct intel_iommu *iommu, * efficient to flush devTLB specific to the PASID. */ if (pasid == IOMMU_NO_PASID) - qi_flush_dev_iotlb(iommu, sid, pfsid, qdep, 0, 64 - VTD_PAGE_SHIFT); + qi_flush_dev_iotlb(iommu, info, 0, 64 - VTD_PAGE_SHIFT); else - qi_flush_dev_iotlb_pasid(iommu, sid, pfsid, pasid, qdep, 0, 64 - VTD_PAGE_SHIFT); + qi_flush_dev_iotlb_pasid(iommu, info, 0, pasid, 64 - VTD_PAGE_SHIFT); } void intel_pasid_tear_down_entry(struct intel_iommu *iommu, struct device *dev, diff --git a/drivers/iommu/intel/svm.c b/drivers/iommu/intel/svm.c index 40edd282903f..abc226220029 100644 --- a/drivers/iommu/intel/svm.c +++ b/drivers/iommu/intel/svm.c @@ -181,11 +181,10 @@ static void __flush_svm_range_dev(struct intel_svm *svm, qi_flush_piotlb(sdev->iommu, sdev->did, svm->pasid, address, pages, ih); if (info->ats_enabled) { - qi_flush_dev_iotlb_pasid(sdev->iommu, sdev->sid, info->pfsid, - svm->pasid, sdev->qdep, address, + qi_flush_dev_iotlb_pasid(sdev->iommu, info, address, svm->pasid, order_base_2(pages)); - quirk_extra_dev_tlb_flush(info, address, order_base_2(pages), - svm->pasid, sdev->qdep); + quirk_extra_dev_tlb_flush(info, svm->pasid, address, + order_base_2(pages)); } } From patchwork Fri Jan 26 01:40:01 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ethan Zhao X-Patchwork-Id: 13531934 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.10]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 85A03BE4E; Fri, 26 Jan 2024 01:40:39 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.175.65.10 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706233241; cv=none; b=fXSgTt4wKr8rWgHYDOOVl3Cd2Z1AR1dSTIHokVEnWIzUUM54T2YbCSVE6NLTFP/gjNHk++xZAeFRliY2KVh4xzJSgv+A1quTf0sLFnlRhqotLrfHUcBvjdR0WZSLja+rsNaz7AIGvnAyY3TMbzQQ5a0vRdZ/sVdoy1dFb+l7YWQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706233241; c=relaxed/simple; bh=ItgP093gIYVsRymsymHY0/QTT+jDlduQBou/FdU9F6Y=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=f/7bqPMZGLkpSZcIBvOkglkboJx+AYfMamTHNRIjUoIX90U+/KPkQmLdZjMk/iqvgKgYdZEXC6u/YnEULG3hLpVluWYXctU+J1AlnV1YY4o4XmK5QKzhuWUd86DSk/7Cj08vWMbscRnIUTZybROZPeUg94t9nxCFWfbOD2iy/Jg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com; spf=none smtp.mailfrom=linux.intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=DxX8dsga; arc=none smtp.client-ip=198.175.65.10 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.mailfrom=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="DxX8dsga" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1706233239; x=1737769239; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=ItgP093gIYVsRymsymHY0/QTT+jDlduQBou/FdU9F6Y=; b=DxX8dsgaRKdj/OnTyQL74Zlldno1b7jHZ8NE9MR+kWscjNFXUA+FrCfI kdFaeEVh1jMxSF+QohUJwhnD/awG/yEJWCMDABGmQWYgYatwZvnCtuC+U wRGuK3wibDGoF66+JiFZvH/VhFY/heJBTj6O8hYTQnAR9crsmMtjFOaPo MCDmO4b+pzbSrY7/eaTOnSWLqARp9N6mbIfWMMo2ltDNXYbyBx1Vp+mgb 3sRVlIn2TG6jmR97+AXbo4E2SDJ/Pw70HZXMbmZ+lNgchbQRoVUej6MgT 59faqGS0QF48heAUbabNYBRnnPrKFP8p8bwRR7Ywft7reNYWvjUCxm4XA Q==; X-IronPort-AV: E=McAfee;i="6600,9927,10964"; a="15718503" X-IronPort-AV: E=Sophos;i="6.05,216,1701158400"; d="scan'208";a="15718503" Received: from fmviesa003.fm.intel.com ([10.60.135.143]) by orvoesa102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 25 Jan 2024 17:40:39 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.05,216,1701158400"; d="scan'208";a="2533142" Received: from unknown (HELO ply01-vm-store.amr.corp.intel.com) ([10.238.153.201]) by fmviesa003.fm.intel.com with ESMTP; 25 Jan 2024 17:40:34 -0800 From: Ethan Zhao To: baolu.lu@linux.intel.com, bhelgaas@google.com, robin.murphy@arm.com, jgg@ziepe.ca Cc: kevin.tian@intel.com, dwmw2@infradead.org, will@kernel.org, lukas@wunner.de, yi.l.liu@intel.com, iommu@lists.linux.dev, linux-kernel@vger.kernel.org, linux-pci@vger.kernel.org, Ethan Zhao Subject: [PATCH v11 4/5] iommu/vt-d: pass pdev parameter for qi_check_fault() and refactor callers Date: Thu, 25 Jan 2024 20:40:01 -0500 Message-Id: <20240126014002.481294-5-haifeng.zhao@linux.intel.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20240126014002.481294-1-haifeng.zhao@linux.intel.com> References: <20240126014002.481294-1-haifeng.zhao@linux.intel.com> Precedence: bulk X-Mailing-List: linux-pci@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 to check state of ATS capable pci device in qi_check_fault() for surprise removal case, we need to pass the target pci device of ATS invalidation request to qi_check_fault(). if pdev is valid, means current request is for ATS invalidation, vice vesa. no function change. Signed-off-by: Ethan Zhao --- drivers/iommu/intel/dmar.c | 31 ++++++++++++++++++----------- drivers/iommu/intel/iommu.h | 3 ++- drivers/iommu/intel/irq_remapping.c | 2 +- drivers/iommu/intel/pasid.c | 2 +- drivers/iommu/intel/svm.c | 6 +++--- 5 files changed, 26 insertions(+), 18 deletions(-) diff --git a/drivers/iommu/intel/dmar.c b/drivers/iommu/intel/dmar.c index ab5e1760bd59..814134e9aa5a 100644 --- a/drivers/iommu/intel/dmar.c +++ b/drivers/iommu/intel/dmar.c @@ -1267,7 +1267,8 @@ static void qi_dump_fault(struct intel_iommu *iommu, u32 fault) (unsigned long long)desc->qw1); } -static int qi_check_fault(struct intel_iommu *iommu, int index, int wait_index) +static int qi_check_fault(struct intel_iommu *iommu, int index, int wait_index, + struct pci_dev *pdev) { u32 fault; int head, tail; @@ -1344,7 +1345,8 @@ static int qi_check_fault(struct intel_iommu *iommu, int index, int wait_index) * can be part of the submission but it will not be polled for completion. */ int qi_submit_sync(struct intel_iommu *iommu, struct qi_desc *desc, - unsigned int count, unsigned long options) + unsigned int count, unsigned long options, + struct pci_dev *pdev) { struct q_inval *qi = iommu->qi; s64 devtlb_start_ktime = 0; @@ -1430,7 +1432,7 @@ int qi_submit_sync(struct intel_iommu *iommu, struct qi_desc *desc, * a deadlock where the interrupt context can wait indefinitely * for free slots in the queue. */ - rc = qi_check_fault(iommu, index, wait_index); + rc = qi_check_fault(iommu, index, wait_index, pdev); if (rc) break; @@ -1476,7 +1478,7 @@ void qi_global_iec(struct intel_iommu *iommu) desc.qw3 = 0; /* should never fail */ - qi_submit_sync(iommu, &desc, 1, 0); + qi_submit_sync(iommu, &desc, 1, 0, NULL); } void qi_flush_context(struct intel_iommu *iommu, u16 did, u16 sid, u8 fm, @@ -1490,7 +1492,7 @@ void qi_flush_context(struct intel_iommu *iommu, u16 did, u16 sid, u8 fm, desc.qw2 = 0; desc.qw3 = 0; - qi_submit_sync(iommu, &desc, 1, 0); + qi_submit_sync(iommu, &desc, 1, 0, NULL); } void qi_flush_iotlb(struct intel_iommu *iommu, u16 did, u64 addr, @@ -1514,23 +1516,27 @@ void qi_flush_iotlb(struct intel_iommu *iommu, u16 did, u64 addr, desc.qw2 = 0; desc.qw3 = 0; - qi_submit_sync(iommu, &desc, 1, 0); + qi_submit_sync(iommu, &desc, 1, 0, NULL); } void qi_flush_dev_iotlb(struct intel_iommu *iommu, struct device_domain_info *info, u64 addr, unsigned int mask) { + struct pci_dev *pdev = NULL; u16 sid, qdep, pfsid; struct qi_desc desc; if (!info || !info->ats_enabled) return; + if (info->dev || !dev_is_pci(info->dev)) + return; + + pdev = to_pci_dev(info->dev); sid = info->bus << 8 | info->devfn; qdep = info->ats_qdep; pfsid = info->pfsid; - /* * VT-d spec, section 4.3: * @@ -1554,7 +1560,7 @@ void qi_flush_dev_iotlb(struct intel_iommu *iommu, desc.qw2 = 0; desc.qw3 = 0; - qi_submit_sync(iommu, &desc, 1, 0); + qi_submit_sync(iommu, &desc, 1, 0, pdev); } /* PASID-based IOTLB invalidation */ @@ -1595,7 +1601,7 @@ void qi_flush_piotlb(struct intel_iommu *iommu, u16 did, u32 pasid, u64 addr, QI_EIOTLB_AM(mask); } - qi_submit_sync(iommu, &desc, 1, 0); + qi_submit_sync(iommu, &desc, 1, 0, NULL); } /* PASID-based device IOTLB Invalidate */ @@ -1605,15 +1611,16 @@ void qi_flush_dev_iotlb_pasid(struct intel_iommu *iommu, { unsigned long mask = 1UL << (VTD_PAGE_SHIFT + size_order - 1); struct qi_desc desc = {.qw1 = 0, .qw2 = 0, .qw3 = 0}; + struct pci_dev *pdev = NULL; u16 sid, qdep, pfsid; if (!info || !dev_is_pci(info->dev)) return; + pdev = to_pci_dev(info->dev); sid = info->bus << 8 | info->devfn; qdep = info->ats_qdep; pfsid = info->pfsid; - /* * VT-d spec, section 4.3: * @@ -1657,7 +1664,7 @@ void qi_flush_dev_iotlb_pasid(struct intel_iommu *iommu, desc.qw1 |= QI_DEV_EIOTLB_SIZE; } - qi_submit_sync(iommu, &desc, 1, 0); + qi_submit_sync(iommu, &desc, 1, 0, pdev); } void qi_flush_pasid_cache(struct intel_iommu *iommu, u16 did, @@ -1667,7 +1674,7 @@ void qi_flush_pasid_cache(struct intel_iommu *iommu, u16 did, desc.qw0 = QI_PC_PASID(pasid) | QI_PC_DID(did) | QI_PC_GRAN(granu) | QI_PC_TYPE; - qi_submit_sync(iommu, &desc, 1, 0); + qi_submit_sync(iommu, &desc, 1, 0, NULL); } /* diff --git a/drivers/iommu/intel/iommu.h b/drivers/iommu/intel/iommu.h index f68f17476d85..72e1d5fb2114 100644 --- a/drivers/iommu/intel/iommu.h +++ b/drivers/iommu/intel/iommu.h @@ -1052,7 +1052,8 @@ void qi_flush_pasid_cache(struct intel_iommu *iommu, u16 did, u64 granu, u32 pasid); int qi_submit_sync(struct intel_iommu *iommu, struct qi_desc *desc, - unsigned int count, unsigned long options); + unsigned int count, unsigned long options, + struct pci_dev *pdev); /* * Options used in qi_submit_sync: * QI_OPT_WAIT_DRAIN - Wait for PRQ drain completion, spec 6.5.2.8. diff --git a/drivers/iommu/intel/irq_remapping.c b/drivers/iommu/intel/irq_remapping.c index 566297bc87dd..09276bfa127d 100644 --- a/drivers/iommu/intel/irq_remapping.c +++ b/drivers/iommu/intel/irq_remapping.c @@ -153,7 +153,7 @@ static int qi_flush_iec(struct intel_iommu *iommu, int index, int mask) desc.qw2 = 0; desc.qw3 = 0; - return qi_submit_sync(iommu, &desc, 1, 0); + return qi_submit_sync(iommu, &desc, 1, 0, NULL); } static int modify_irte(struct irq_2_iommu *irq_iommu, diff --git a/drivers/iommu/intel/pasid.c b/drivers/iommu/intel/pasid.c index 5dacdea3cab7..8bba5721aeba 100644 --- a/drivers/iommu/intel/pasid.c +++ b/drivers/iommu/intel/pasid.c @@ -200,7 +200,7 @@ pasid_cache_invalidation_with_pasid(struct intel_iommu *iommu, desc.qw2 = 0; desc.qw3 = 0; - qi_submit_sync(iommu, &desc, 1, 0); + qi_submit_sync(iommu, &desc, 1, 0, NULL); } static void diff --git a/drivers/iommu/intel/svm.c b/drivers/iommu/intel/svm.c index abc226220029..2a2cd82d781b 100644 --- a/drivers/iommu/intel/svm.c +++ b/drivers/iommu/intel/svm.c @@ -538,7 +538,7 @@ void intel_drain_pasid_prq(struct device *dev, u32 pasid) QI_DEV_IOTLB_PFSID(info->pfsid); qi_retry: reinit_completion(&iommu->prq_complete); - qi_submit_sync(iommu, desc, 3, QI_OPT_WAIT_DRAIN); + qi_submit_sync(iommu, desc, 3, QI_OPT_WAIT_DRAIN, NULL); if (readl(iommu->reg + DMAR_PRS_REG) & DMA_PRS_PRO) { wait_for_completion(&iommu->prq_complete); goto qi_retry; @@ -641,7 +641,7 @@ static void handle_bad_prq_event(struct intel_iommu *iommu, desc.qw3 = 0; } - qi_submit_sync(iommu, &desc, 1, 0); + qi_submit_sync(iommu, &desc, 1, 0, NULL); } static irqreturn_t prq_event_thread(int irq, void *d) @@ -797,7 +797,7 @@ int intel_svm_page_response(struct device *dev, ktime_to_ns(ktime_get()) - prm->private_data[0]); } - qi_submit_sync(iommu, &desc, 1, 0); + qi_submit_sync(iommu, &desc, 1, 0, NULL); } out: return ret; From patchwork Fri Jan 26 01:40:02 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ethan Zhao X-Patchwork-Id: 13531935 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.10]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6AF6EBE5B; Fri, 26 Jan 2024 01:40:42 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.175.65.10 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706233243; cv=none; b=Csoz0jTfCp3Q/nKg0seATY10r4FX5peHozRaPdseu/uxuDwylcSVYiKmdGCWSWQHxTNjKX8W3dw8HbevepK2/4lmHiF41TCDBp8tHIcHv3F3tpv9koYSn+2D1xuUKhB5nx4bUAtiwy0mqNFO8qh2z8uRAhCcWduLN17z4QNHGHQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706233243; c=relaxed/simple; bh=FcKQPcK9DSVvVLptnofw+fKVNv4QmsDqpMonokbwkz0=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=Net+WrS96aNeh2JiCbJCttvaWKEgpk+0vwdrZ/FE86UDB41qdLKI83J/o/2iZL6c3dciTPNMJj4Sve4KETuTZUQJ/klhhKsWh23E0j+6sZnxl6zGZ9zeP/8lJY89kNoK3icVXknCZvpw20wXSB7bUaKpUcs/PKVO0ioKeaeBYnY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com; spf=none smtp.mailfrom=linux.intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=DZsOyMVR; arc=none smtp.client-ip=198.175.65.10 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.mailfrom=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="DZsOyMVR" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1706233242; x=1737769242; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=FcKQPcK9DSVvVLptnofw+fKVNv4QmsDqpMonokbwkz0=; b=DZsOyMVRFBgq6h9Ix+AhJEFRBDRy9IYt1Pq38rn6zvJK57FGsNRz7Jjn 2zq0FN3i09Gnu+UASLTrL0zQB1KA8I/cLSQg/TggWoWmNiyXUwAnN2rrb elR+PCTFhpPlYU+xKuQBI5/Z+hZWcGOAbYDKqlqMfswtdE8Rnlj52jsfR y9Ybh2BZR9Y41RaYEyg5TPYQ/OOo3Zxa/geSwdTq8TqMl6vTue/MDW4EA kqGV1jUX2LaJI64zRT5G1nZgtH4IHqPZi4OCxn/wicztqCsDwfwgKt7yX rbxsP5s4yUfmcrlYektoS3Hg1ZjLeqJBvnuYqu79l5D23+eOd+JOal/LV g==; X-IronPort-AV: E=McAfee;i="6600,9927,10964"; a="15718519" X-IronPort-AV: E=Sophos;i="6.05,216,1701158400"; d="scan'208";a="15718519" Received: from fmviesa003.fm.intel.com ([10.60.135.143]) by orvoesa102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 25 Jan 2024 17:40:42 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.05,216,1701158400"; d="scan'208";a="2533153" Received: from unknown (HELO ply01-vm-store.amr.corp.intel.com) ([10.238.153.201]) by fmviesa003.fm.intel.com with ESMTP; 25 Jan 2024 17:40:39 -0800 From: Ethan Zhao To: baolu.lu@linux.intel.com, bhelgaas@google.com, robin.murphy@arm.com, jgg@ziepe.ca Cc: kevin.tian@intel.com, dwmw2@infradead.org, will@kernel.org, lukas@wunner.de, yi.l.liu@intel.com, iommu@lists.linux.dev, linux-kernel@vger.kernel.org, linux-pci@vger.kernel.org, Ethan Zhao Subject: [PATCH v11 5/5] iommu/vt-d: improve ITE fault handling if target device isn't present Date: Thu, 25 Jan 2024 20:40:02 -0500 Message-Id: <20240126014002.481294-6-haifeng.zhao@linux.intel.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20240126014002.481294-1-haifeng.zhao@linux.intel.com> References: <20240126014002.481294-1-haifeng.zhao@linux.intel.com> Precedence: bulk X-Mailing-List: linux-pci@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Because surprise removal could happen anytime, e.g. user could request safe removal to EP(endpoint device) via sysfs and brings its link down to do surprise removal cocurrently. such aggressive cases would cause ATS invalidation request issued to non-existence target device, then deadly loop to retry that request after ITE fault triggered in interrupt context. this patch aims to optimize the ITE handling by checking the target device presence state to avoid retrying the timeout request blindly, thus avoid hard lockup or system hang. Signed-off-by: Ethan Zhao --- drivers/iommu/intel/dmar.c | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+) diff --git a/drivers/iommu/intel/dmar.c b/drivers/iommu/intel/dmar.c index 814134e9aa5a..e2d5ea2b0683 100644 --- a/drivers/iommu/intel/dmar.c +++ b/drivers/iommu/intel/dmar.c @@ -1272,6 +1272,7 @@ static int qi_check_fault(struct intel_iommu *iommu, int index, int wait_index, { u32 fault; int head, tail; + u64 iqe_err, ite_sid; struct q_inval *qi = iommu->qi; int shift = qi_shift(iommu); @@ -1316,6 +1317,13 @@ static int qi_check_fault(struct intel_iommu *iommu, int index, int wait_index, tail = readl(iommu->reg + DMAR_IQT_REG); tail = ((tail >> shift) - 1 + QI_LENGTH) % QI_LENGTH; + /* + * SID field is valid only when the ITE field is Set in FSTS_REG + * see Intel VT-d spec r4.1, section 11.4.9.9 + */ + iqe_err = dmar_readq(iommu->reg + DMAR_IQER_REG); + ite_sid = DMAR_IQER_REG_ITESID(iqe_err); + writel(DMA_FSTS_ITE, iommu->reg + DMAR_FSTS_REG); pr_info("Invalidation Time-out Error (ITE) cleared\n"); @@ -1325,6 +1333,16 @@ static int qi_check_fault(struct intel_iommu *iommu, int index, int wait_index, head = (head - 2 + QI_LENGTH) % QI_LENGTH; } while (head != tail); + /* + * If got ITE, we need to check if the sid of ITE is the same as + * current ATS invalidation target device, if yes, don't try this + * request anymore if the target device isn't present. + * 0 value of ite_sid means old VT-d device, no ite_sid value. + */ + if (pdev && ite_sid && !pci_device_is_present(pdev) && + ite_sid == pci_dev_id(pci_physfn(pdev))) + return -ETIMEDOUT; + if (qi->desc_status[wait_index] == QI_ABORT) return -EAGAIN; }