From patchwork Tue Feb 19 23:08:12 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael D Labriola X-Patchwork-Id: 10821679 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id EF246922 for ; Wed, 20 Feb 2019 08:05:54 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id DC1CB2DB45 for ; Wed, 20 Feb 2019 08:05:54 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id D03F42D186; Wed, 20 Feb 2019 08:05:54 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-4.2 required=2.0 tests=BAYES_00,DKIM_ADSP_CUSTOM_MED, FREEMAIL_FROM,HEXHASH_WORD,MAILING_LIST_MULTI,RCVD_IN_DNSWL_MED autolearn=unavailable version=3.3.1 Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 5EB862D1C8 for ; Wed, 20 Feb 2019 08:05:54 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 20475892AC; Wed, 20 Feb 2019 08:05:19 +0000 (UTC) X-Original-To: dri-devel@lists.freedesktop.org Delivered-To: dri-devel@lists.freedesktop.org Received: from mail-pg1-x544.google.com (mail-pg1-x544.google.com [IPv6:2607:f8b0:4864:20::544]) by gabe.freedesktop.org (Postfix) with ESMTPS id 9A8D789220; Tue, 19 Feb 2019 23:08:18 +0000 (UTC) Received: by mail-pg1-x544.google.com with SMTP id y1so10827539pgk.11; Tue, 19 Feb 2019 15:08:18 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id; bh=eMDwzIIraovpyvUL0fWbbqV8zgStL3UhkGc2Gfxoe2M=; b=Z/tvW0nDY2HAfAMBK5UY3SaSTcousouLeUVdiYzBeEjpE8BYYeZOn591ENHhIbPBz9 vQy6BvJdH4BV0p1baPA2pKa3MH7JsZ4zsqWO+J3BwObIOpc4kXZGFjJ5kv3qPOjDWR7j Dzf/yFqYfJGBnQSN6RNlt6FQvOlDVrXQ1z2Qf1JG8oVz1vDHWv4mJE+Wcz1uxfYFYD0A 1Cdd9OTgyXk2qbhc4XWU26k4e8S5F90QuzS3/I/5cO8lpDnyl4COmpiA3Y0tPHkuoLFz Cd/oVOuIIHZnFvXOEZoCV2HHsXh2Nrvte2Xc5BKCun8sjcuWJ8fRBQuL8jrkp3BQYr+T ciHA== X-Gm-Message-State: AHQUAuZHJWvkypjRPUqYfm4nclM3qYatCOu9ngrMXVQvVFXmBW4paWoX CKKPm8lGI6z5U1Tvbt+UJAwgU5C0vsE= X-Google-Smtp-Source: AHgI3IYMmC63X+p9crU7ysFSpwT74WMTtulf7P9ZJUT0S3GO/mIT+2Xr/FfM5j9ZbUSEFbnysIfrOQ== X-Received: by 2002:a63:d453:: with SMTP id i19mr11212492pgj.237.1550617697854; Tue, 19 Feb 2019 15:08:17 -0800 (PST) Received: from eomer.localdomain (wsip-70-167-21-27.ri.ri.cox.net. [70.167.21.27]) by smtp.gmail.com with ESMTPSA id 22sm36210519pfk.53.2019.02.19.15.08.15 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 19 Feb 2019 15:08:17 -0800 (PST) From: Michael D Labriola To: dri-devel@lists.freedesktop.org, Alex Deucher , Christian Koenig , Chunming Zhou , amd-gfx@lists.freedesktop.org, Monk Liu Subject: [PATCH] drm: add func to better detect wether swiotlb is needed Date: Tue, 19 Feb 2019 18:08:12 -0500 Message-Id: <1550617692-27306-1-git-send-email-michael.d.labriola@gmail.com> X-Mailer: git-send-email 1.8.3.1 X-Mailman-Approved-At: Wed, 20 Feb 2019 08:05:14 +0000 X-Mailman-Original-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id; bh=eMDwzIIraovpyvUL0fWbbqV8zgStL3UhkGc2Gfxoe2M=; b=qhA4Py1xIjlZYP9gwt2PKtlZD0jDi9+jmMgy0TwFrKo38Bc4HBEwKR/a9Dq3pqAMxr bte6w5rf5F6tGvIgkTddvw7HG1xFiRrZpoPjFMSXswdu34Ik9qzVhMnYpaF+vpjeC5YG MDbja7mmK3Q1KQ8egbmVv3Z7VIXJOw0Qu4pr0WTOWQOzmwkYB98mECXjGxkuwBqxOGCa I3QpYaRialxtK6U+3iG3cHX3mPQVWVGLR1qCouifridC5JsF/kmOltLKFk3xLIhAtjhM 3QZNF2UY+bcuKXkK0RJTd4XaPSQjpDt741la/t19k72SOKfMVpmD1HN92HOXQaqZ/PxO C+NQ== X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Juergen Gross , Konrad Rzeszutek Wilk , Andrew Cooper , Michael D Labriola , xen-devel@lists.xen.org, Christoph Hellwig , Paul Durrant , Jan Beulich MIME-Version: 1.0 Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" X-Virus-Scanned: ClamAV using ClamSMTP This commit fixes DRM failures on Xen PV systems that were introduced in v4.17 by the following commits: 82626363 drm: add func to get max iomem address v2 fd5fd480 drm/amdgpu: only enable swiotlb alloc when need v2 1bc3d3cc drm/radeon: only enable swiotlb path when need v2 The introduction of ->need_swiotlb to the ttm_dma_populate() conditionals in the radeon and amdgpu device drivers causes Gnome to immediately crash on Xen PV systems, returning the user to the login screen. The following kernel errors get logged: [ 28.554259] radeon_dp_aux_transfer_native: 200 callbacks suppressed [ 31.219821] radeon 0000:01:00.0: swiotlb buffer is full (sz: 2097152 bytes) [ 31.220030] [drm:radeon_gem_object_create [radeon]] *ERROR* Failed to allocate GEM object (16384000, 2, 4096, -14) [ 31.226109] radeon 0000:01:00.0: swiotlb buffer is full (sz: 2097152 bytes) [ 31.226300] [drm:radeon_gem_object_create [radeon]] *ERROR* Failed to allocate GEM object (16384000, 2, 4096, -14) [ 31.300734] gnome-shell[1935]: segfault at 88 ip 00007f39151cd904 sp 00007ffc97611ad8 error 4 in libmutter-cogl.so[7f3915178000+aa000] [ 31.300745] Code: 5f c3 0f 1f 40 00 48 8b 47 78 48 8b 40 40 ff e0 66 0f 1f 44 00 00 48 8b 47 78 48 8b 40 48 ff e0 66 0f 1f 44 00 00 48 8b 47 78 <48> 8b 80 88 00 00 00 ff e0 0f 1f 00 48 8b 47 78 48 8b 40 68 ff e0 [ 38.193302] radeon_dp_aux_transfer_native: 116 callbacks suppressed [ 40.009317] radeon 0000:01:00.0: swiotlb buffer is full (sz: 2097152 bytes) [ 40.009488] [drm:radeon_gem_object_create [radeon]] *ERROR* Failed to allocate GEM object (16384000, 2, 4096, -14) [ 40.015114] radeon 0000:01:00.0: swiotlb buffer is full (sz: 2097152 bytes) [ 40.015297] [drm:radeon_gem_object_create [radeon]] *ERROR* Failed to allocate GEM object (16384000, 2, 4096, -14) [ 40.028302] gnome-shell[2431]: segfault at 2dadf40 ip 0000000002dadf40 sp 00007ffcd24ea5f8 error 15 [ 40.028306] Code: 20 6e 31 00 00 00 00 00 00 00 00 37 e3 3d 2d 7f 00 00 80 f4 e6 3d 2d 7f 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 <00> 00 00 00 00 00 00 00 c1 00 00 00 00 00 00 00 80 e1 d2 03 00 00 This commit renames drm_get_max_iomem() to drm_need_swiotlb(), adds a xen_pv_domain() check to it, and moves the bit shifting comparison that always follows its usage into the function (simplifying the drm driver code). Signed-off-by: Michael D Labriola --- drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c | 2 +- drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c | 2 +- drivers/gpu/drm/amd/amdgpu/gmc_v9_0.c | 2 +- drivers/gpu/drm/drm_memory.c | 19 ++++++++++++++++--- drivers/gpu/drm/radeon/radeon_device.c | 2 +- include/drm/drm_cache.h | 2 +- 6 files changed, 21 insertions(+), 8 deletions(-) diff --git a/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c b/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c index 910c4ce..6bc0266 100644 --- a/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c +++ b/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c @@ -1029,7 +1029,7 @@ static int gmc_v7_0_sw_init(void *handle) pci_set_consistent_dma_mask(adev->pdev, DMA_BIT_MASK(32)); pr_warn("amdgpu: No coherent DMA available\n"); } - adev->need_swiotlb = drm_get_max_iomem() > ((u64)1 << dma_bits); + adev->need_swiotlb = drm_need_swiotlb(dma_bits); r = gmc_v7_0_init_microcode(adev); if (r) { diff --git a/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c b/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c index 747c068..8638adf 100644 --- a/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c +++ b/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c @@ -1155,7 +1155,7 @@ static int gmc_v8_0_sw_init(void *handle) pci_set_consistent_dma_mask(adev->pdev, DMA_BIT_MASK(32)); pr_warn("amdgpu: No coherent DMA available\n"); } - adev->need_swiotlb = drm_get_max_iomem() > ((u64)1 << dma_bits); + adev->need_swiotlb = drm_need_swiotlb(dma_bits); r = gmc_v8_0_init_microcode(adev); if (r) { diff --git a/drivers/gpu/drm/amd/amdgpu/gmc_v9_0.c b/drivers/gpu/drm/amd/amdgpu/gmc_v9_0.c index f35d7a5..4f67093 100644 --- a/drivers/gpu/drm/amd/amdgpu/gmc_v9_0.c +++ b/drivers/gpu/drm/amd/amdgpu/gmc_v9_0.c @@ -989,7 +989,7 @@ static int gmc_v9_0_sw_init(void *handle) pci_set_consistent_dma_mask(adev->pdev, DMA_BIT_MASK(32)); printk(KERN_WARNING "amdgpu: No coherent DMA available.\n"); } - adev->need_swiotlb = drm_get_max_iomem() > ((u64)1 << dma_bits); + adev->need_swiotlb = drm_need_swiotlb(dma_bits); if (adev->asic_type == CHIP_VEGA20) { r = gfxhub_v1_1_get_xgmi_info(adev); diff --git a/drivers/gpu/drm/drm_memory.c b/drivers/gpu/drm/drm_memory.c index d69e4fc..6af59a6 100644 --- a/drivers/gpu/drm/drm_memory.c +++ b/drivers/gpu/drm/drm_memory.c @@ -35,6 +35,7 @@ #include #include +#include #include #include "drm_legacy.h" @@ -150,15 +151,27 @@ void drm_legacy_ioremapfree(struct drm_local_map *map, struct drm_device *dev) } EXPORT_SYMBOL(drm_legacy_ioremapfree); -u64 drm_get_max_iomem(void) +bool drm_need_swiotlb(int dma_bits) { struct resource *tmp; resource_size_t max_iomem = 0; + /* + * Xen paravirtual hosts require swiotlb regardless of requested dma + * transfer size. + * + * NOTE: Really, what it requires is use of the dma_alloc_coherent + * allocator used in ttm_dma_populate() instead of + * ttm_populate_and_map_pages(), which bounce buffers so much in + * Xen it leads to swiotlb buffer exhaustion. + */ + if (xen_pv_domain()) + return true; + for (tmp = iomem_resource.child; tmp; tmp = tmp->sibling) { max_iomem = max(max_iomem, tmp->end); } - return max_iomem; + return max_iomem > ((u64)1 << dma_bits); } -EXPORT_SYMBOL(drm_get_max_iomem); +EXPORT_SYMBOL(drm_need_swiotlb); diff --git a/drivers/gpu/drm/radeon/radeon_device.c b/drivers/gpu/drm/radeon/radeon_device.c index 59c8a66..a1d3c62 100644 --- a/drivers/gpu/drm/radeon/radeon_device.c +++ b/drivers/gpu/drm/radeon/radeon_device.c @@ -1387,7 +1387,7 @@ int radeon_device_init(struct radeon_device *rdev, pci_set_consistent_dma_mask(rdev->pdev, DMA_BIT_MASK(32)); pr_warn("radeon: No coherent DMA available\n"); } - rdev->need_swiotlb = drm_get_max_iomem() > ((u64)1 << dma_bits); + rdev->need_swiotlb = drm_need_swiotlb(dma_bits); /* Registers mapping */ /* TODO: block userspace mapping of io register */ diff --git a/include/drm/drm_cache.h b/include/drm/drm_cache.h index bfe1639..633eaaf 100644 --- a/include/drm/drm_cache.h +++ b/include/drm/drm_cache.h @@ -38,7 +38,7 @@ void drm_clflush_pages(struct page *pages[], unsigned long num_pages); void drm_clflush_sg(struct sg_table *st); void drm_clflush_virt_range(void *addr, unsigned long length); -u64 drm_get_max_iomem(void); +bool drm_need_swiotlb(int dma_bits); static inline bool drm_arch_can_wc_memory(void)