From patchwork Mon Aug 23 11:56:38 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kunkun Jiang X-Patchwork-Id: 12452561 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id F25C4C4338F for ; Mon, 23 Aug 2021 11:59:33 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 9A0F96054E for ; Mon, 23 Aug 2021 11:59:33 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 9A0F96054E Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=huawei.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=nongnu.org Received: from localhost ([::1]:45712 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mI8c0-0003au-Sk for qemu-devel@archiver.kernel.org; Mon, 23 Aug 2021 07:59:32 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:37644) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mI8Zi-0008IF-FB for qemu-devel@nongnu.org; Mon, 23 Aug 2021 07:57:10 -0400 Received: from szxga02-in.huawei.com ([45.249.212.188]:3117) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mI8Zf-0000qd-BZ for qemu-devel@nongnu.org; Mon, 23 Aug 2021 07:57:10 -0400 Received: from dggemv711-chm.china.huawei.com (unknown [172.30.72.56]) by szxga02-in.huawei.com (SkyGuard) with ESMTP id 4GtVx06TmJz8slN; Mon, 23 Aug 2021 19:52:52 +0800 (CST) Received: from dggema765-chm.china.huawei.com (10.1.198.207) by dggemv711-chm.china.huawei.com (10.1.198.66) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256) id 15.1.2176.2; Mon, 23 Aug 2021 19:56:59 +0800 Received: from DESKTOP-6NKE0BC.china.huawei.com (10.174.185.210) by dggema765-chm.china.huawei.com (10.1.198.207) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.2176.2; Mon, 23 Aug 2021 19:56:59 +0800 From: Kunkun Jiang To: Alex Williamson , Eric Auger , "open list:All patches CC here" Subject: [PATCH 1/2] vfio/common: Add trace point when a MMIO RAM section less than minimum size Date: Mon, 23 Aug 2021 19:56:38 +0800 Message-ID: <20210823115640.817-2-jiangkunkun@huawei.com> X-Mailer: git-send-email 2.26.2.windows.1 In-Reply-To: <20210823115640.817-1-jiangkunkun@huawei.com> References: <20210823115640.817-1-jiangkunkun@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.174.185.210] X-ClientProxiedBy: dggems706-chm.china.huawei.com (10.3.19.183) To dggema765-chm.china.huawei.com (10.1.198.207) X-CFilter-Loop: Reflected Received-SPF: pass client-ip=45.249.212.188; envelope-from=jiangkunkun@huawei.com; helo=szxga02-in.huawei.com X-Spam_score_int: -41 X-Spam_score: -4.2 X-Spam_bar: ---- X-Spam_report: (-4.2 / 5.0 requ) BAYES_00=-1.9, RCVD_IN_DNSWL_MED=-2.3, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: wanghaibin.wang@huawei.com, Kunkun Jiang Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" I recently did some tests about the 82599 NIC, and found a strange scenario. The MSIX-Table size of this NIC is 0x30 and the offset in Bar 3(64KB) is 0x0. And CPU page size is 64KB. The region_add() will return early at 'int128_ge((int128_make64(iova), llend))' and hasn't any message. Let's add a trace point to informed users. Signed-off-by: Kunkun Jiang --- hw/vfio/common.c | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/hw/vfio/common.c b/hw/vfio/common.c index 7d80f43e39..bbb8d1ea0c 100644 --- a/hw/vfio/common.c +++ b/hw/vfio/common.c @@ -892,6 +892,13 @@ static void vfio_listener_region_add(MemoryListener *listener, llend = int128_and(llend, int128_exts64(qemu_real_host_page_mask)); if (int128_ge(int128_make64(iova), llend)) { + if (memory_region_is_ram_device(section->mr)) { + trace_vfio_listener_region_add_no_dma_map( + memory_region_name(section->mr), + section->offset_within_address_space, + int128_getlo(section->size), + qemu_real_host_page_size); + } return; } end = int128_get64(int128_sub(llend, int128_one())); From patchwork Mon Aug 23 11:56:39 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kunkun Jiang X-Patchwork-Id: 12452557 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id CE83AC4338F for ; Mon, 23 Aug 2021 11:58:17 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 17FD26127B for ; Mon, 23 Aug 2021 11:58:17 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 17FD26127B Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=huawei.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=nongnu.org Received: from localhost ([::1]:42252 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mI8am-0001GA-0X for qemu-devel@archiver.kernel.org; Mon, 23 Aug 2021 07:58:16 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:37634) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mI8Zh-0008Gs-AK for qemu-devel@nongnu.org; Mon, 23 Aug 2021 07:57:09 -0400 Received: from szxga01-in.huawei.com ([45.249.212.187]:2471) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mI8Ze-0000sD-MZ for qemu-devel@nongnu.org; Mon, 23 Aug 2021 07:57:09 -0400 Received: from dggemv704-chm.china.huawei.com (unknown [172.30.72.55]) by szxga01-in.huawei.com (SkyGuard) with ESMTP id 4GtVxP68xYzbhLv; Mon, 23 Aug 2021 19:53:13 +0800 (CST) Received: from dggema765-chm.china.huawei.com (10.1.198.207) by dggemv704-chm.china.huawei.com (10.3.19.47) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256) id 15.1.2176.2; Mon, 23 Aug 2021 19:57:01 +0800 Received: from DESKTOP-6NKE0BC.china.huawei.com (10.174.185.210) by dggema765-chm.china.huawei.com (10.1.198.207) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.2176.2; Mon, 23 Aug 2021 19:57:01 +0800 From: Kunkun Jiang To: Alex Williamson , Eric Auger , "open list:All patches CC here" Subject: [PATCH 2/2] vfio/common: Fix address alignment in region_add/region_del Date: Mon, 23 Aug 2021 19:56:39 +0800 Message-ID: <20210823115640.817-3-jiangkunkun@huawei.com> X-Mailer: git-send-email 2.26.2.windows.1 In-Reply-To: <20210823115640.817-1-jiangkunkun@huawei.com> References: <20210823115640.817-1-jiangkunkun@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.174.185.210] X-ClientProxiedBy: dggems706-chm.china.huawei.com (10.3.19.183) To dggema765-chm.china.huawei.com (10.1.198.207) X-CFilter-Loop: Reflected Received-SPF: pass client-ip=45.249.212.187; envelope-from=jiangkunkun@huawei.com; helo=szxga01-in.huawei.com X-Spam_score_int: -41 X-Spam_score: -4.2 X-Spam_bar: ---- X-Spam_report: (-4.2 / 5.0 requ) BAYES_00=-1.9, RCVD_IN_DNSWL_MED=-2.3, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: wanghaibin.wang@huawei.com, Kunkun Jiang Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" The page sizes supported by IOMMU may not match the CPU page size. For example, the CPU page size is 16KB, but ARM SMMU may not support 16KB. So it is inappropriate to use qemu_real_host_page_mask in region_add/region_del. The vfio iommu page sizes exposed via VFIO_IOMMU_GET_INFO. So use the smallest page size to align the address. Fixes: 1eb7f642750 (vfio: Support host translation granule size) Signed-off-by: Kunkun Jiang --- hw/vfio/common.c | 30 +++++++++++++++++++++--------- 1 file changed, 21 insertions(+), 9 deletions(-) diff --git a/hw/vfio/common.c b/hw/vfio/common.c index bbb8d1ea0c..62f338cd78 100644 --- a/hw/vfio/common.c +++ b/hw/vfio/common.c @@ -859,10 +859,13 @@ static void vfio_unregister_ram_discard_listener(VFIOContainer *container, g_free(vrdl); } +#define VFIO_IOMMU_MIN_PAGE_ALIGN(addr, pgsize) ROUND_UP((addr), (pgsize)) + static void vfio_listener_region_add(MemoryListener *listener, MemoryRegionSection *section) { VFIOContainer *container = container_of(listener, VFIOContainer, listener); + uint64_t vfio_iommu_min_page_size, vfio_iommu_min_page_mask; hwaddr iova, end; Int128 llend, llsize; void *vaddr; @@ -879,17 +882,21 @@ static void vfio_listener_region_add(MemoryListener *listener, return; } + vfio_iommu_min_page_size = 1 << ctz64(container->pgsizes); + vfio_iommu_min_page_mask = ~(vfio_iommu_min_page_size - 1); + if (unlikely((section->offset_within_address_space & - ~qemu_real_host_page_mask) != - (section->offset_within_region & ~qemu_real_host_page_mask))) { + ~vfio_iommu_min_page_mask) != + (section->offset_within_region & ~vfio_iommu_min_page_mask))) { error_report("%s received unaligned region", __func__); return; } - iova = REAL_HOST_PAGE_ALIGN(section->offset_within_address_space); + iova = VFIO_IOMMU_MIN_PAGE_ALIGN(section->offset_within_address_space, + vfio_iommu_min_page_size); llend = int128_make64(section->offset_within_address_space); llend = int128_add(llend, section->size); - llend = int128_and(llend, int128_exts64(qemu_real_host_page_mask)); + llend = int128_and(llend, int128_exts64(vfio_iommu_min_page_mask)); if (int128_ge(int128_make64(iova), llend)) { if (memory_region_is_ram_device(section->mr)) { @@ -897,7 +904,7 @@ static void vfio_listener_region_add(MemoryListener *listener, memory_region_name(section->mr), section->offset_within_address_space, int128_getlo(section->size), - qemu_real_host_page_size); + vfio_iommu_min_page_size); } return; } @@ -1102,6 +1109,7 @@ static void vfio_listener_region_del(MemoryListener *listener, MemoryRegionSection *section) { VFIOContainer *container = container_of(listener, VFIOContainer, listener); + uint64_t vfio_iommu_min_page_size, vfio_iommu_min_page_mask; hwaddr iova, end; Int128 llend, llsize; int ret; @@ -1115,9 +1123,12 @@ static void vfio_listener_region_del(MemoryListener *listener, return; } + vfio_iommu_min_page_size = 1 << ctz64(container->pgsizes); + vfio_iommu_min_page_mask = ~(vfio_iommu_min_page_size - 1); + if (unlikely((section->offset_within_address_space & - ~qemu_real_host_page_mask) != - (section->offset_within_region & ~qemu_real_host_page_mask))) { + ~vfio_iommu_min_page_mask) != + (section->offset_within_region & ~vfio_iommu_min_page_mask))) { error_report("%s received unaligned region", __func__); return; } @@ -1145,10 +1156,11 @@ static void vfio_listener_region_del(MemoryListener *listener, */ } - iova = REAL_HOST_PAGE_ALIGN(section->offset_within_address_space); + iova = VFIO_IOMMU_MIN_PAGE_ALIGN(section->offset_within_address_space, + vfio_iommu_min_page_size); llend = int128_make64(section->offset_within_address_space); llend = int128_add(llend, section->size); - llend = int128_and(llend, int128_exts64(qemu_real_host_page_mask)); + llend = int128_and(llend, int128_exts64(vfio_iommu_min_page_mask)); if (int128_ge(int128_make64(iova), llend)) { return;