From patchwork Sun Jun 9 22:24:07 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Stefan Wahren X-Patchwork-Id: 13691398 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 7F12DC27C53 for ; Sun, 9 Jun 2024 22:24:44 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:Content-Type: Content-Transfer-Encoding:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:Subject:From:Cc:To:MIME-Version:Date:Message-ID: Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender :Resent-To:Resent-Cc:Resent-Message-ID:In-Reply-To:References:List-Owner; bh=wnI11amDJafQeDRpQxOI7g2fDu+FId07YTDzvxF30/8=; b=z0+YVVxevibbD5kqSd4Dl3Q3Gh hj9I3SNl23NjP3EX1qwkAxKzlh3FWXev3vRPQg/UrWDndHSaVhsRiWyUnGccIVmFhmMwzSC0NjZzA 3WDBf1r8pEW1icrfDPkiIEP6JwxTWkqG2VygW2yg/w7l8BooAxjw9zjpGgS6uV5lc/e4tkVCH8iRb UMetWtct2yvakiw0pJVatk86pPGcGNBDOkH5es0yX/i6O4RH/FRLtenTLksfKOzdAPaaasSpFxux2 HK/aE6kjL/eufvEVvQWVSUJIRoSbQRebuYNQQ9u79gnfrCL7CSV8AxeorfXFVnY1pCRz1o/G6mcKj gnXUyYyQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1sGQxc-00000003LdC-1CHM; Sun, 09 Jun 2024 22:24:24 +0000 Received: from mout.gmx.net ([212.227.15.19]) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1sGQxY-00000003Lcc-0Tvu for linux-arm-kernel@lists.infradead.org; Sun, 09 Jun 2024 22:24:22 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmx.net; s=s31663417; t=1717971848; x=1718576648; i=wahrenst@gmx.net; bh=CwlA2XWpylmghwW82kcJuuHVOtVeAgY64AJcXqvAtrU=; h=X-UI-Sender-Class:Message-ID:Date:MIME-Version:To:Cc:From: Subject:Content-Type:Content-Transfer-Encoding:cc: content-transfer-encoding:content-type:date:from:message-id: mime-version:reply-to:subject:to; b=bHXXUw4Cc+Hoq9Buys5bRPujwwjJtbVOli7ACK84gl58ltCikF3JCadroD+Sdu0y oey9gXNQsqgJizAPJALN1KscIJOApcpuJDaGveJTtMe07qEQLFbU7Ts+Cd5jOgxEa lnwv91E+Q0A5yxzq3uCyYYlvVZ6ziYQMTCVwqIWcAh59UJc2Uk36VciMInA8wnmII /hbIpsesQW4sHx6CkmebR0W9ggc3tFU/ZNRD3ajs7EIsKFgfwmsNB6F8WvFGE//nG WyLeBOghy6RZTCyd6PUX57tO9g+eZyoyAkXBYkZcd2uauvB8Rw4r8dwcs2Lnl2YjT bLXVOIx2DXzZxdV6xQ== X-UI-Sender-Class: 724b4f7f-cbec-4199-ad4e-598c01a50d3a Received: from [192.168.1.127] ([37.4.248.43]) by mail.gmx.net (mrgmx005 [212.227.17.190]) with ESMTPSA (Nemesis) id 1MQe9s-1s1fna0mVK-00TorN; Mon, 10 Jun 2024 00:24:08 +0200 Message-ID: Date: Mon, 10 Jun 2024 00:24:07 +0200 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Content-Language: en-US To: Catalin Marinas , Phil Elwell , Laurent Pinchart Cc: Arnd Bergmann , Will Deacon , Christoph Hellwig , Florian Fainelli , Robin Murphy , Greg Kroah-Hartman , Linux ARM , linux-staging@lists.linux.dev From: Stefan Wahren Subject: WARNING: drivers/staging/vc04_services/interface/vchiq_arm/vchiq_arm.c:364 vchiq_prepare_bulk_data X-Provags-ID: V03:K1:qNsRZmqGHH75DWNfgijucHO8ezT35SGbbSRMhjuhmxsPIoJwW8f rcKs90R3NfFU3niHoWmjoQIx5TiDWcsBLiPzuEYzCe1IUA84umzHJi2KYTLDP7ZLEFsqcON JAXNueIw93HWQDP9sbyiPM6L39waaDb9Hcsh3cVEEGZcfsx3n3v42rN6rUamsh5Y15wEYc8 CwWLd/ycMRZ3EMGG1HFIA== UI-OutboundReport: notjunk:1;M01:P0:TbTv1iOZxWw=;Bi7nP9MiSrZOnsB5xv+RUT7Sioo YFcV/kpQP7sf/QSRpEeDZHKGfQ9ZoU4KV7JqzJrd2WwY0FY3WNbXhLUwunWPWLyf9bjUvzNx3 bB2FVq5A1HPvPtwRO5aHDgrrrvKTR5OcuZbNrhvSxPHDXwsOh2UNzcFZvHXKnUB9Sv8q1bcml SQClmNAaPFNM3MNIokHcfe2FE3GzEm785X01OQdE7drxNyymbkA+hwpoPB7CVZ/UeVASXbBzT yTzgADb4TI92snP+byeDWkvYvGGxEQ79gaoxCYFJjBNdSfyTSeVJWvWi7d8SMjvLzVGJDYmrn nLMY9/9YtR0We3G76uAdTCalUzPsRawBRwMnkpiJIbxE2dhJGgyw5FkGuPGP8nZxHsrwUkmIV RlYw1RocI+fHsTUFn10UXGsFlXdaU3zzAfCRhoZOZmI6ArHVgOaBH+KmSxFuJ619C4j+WXbq0 cynq1+NIsEdMDYOPEdwOhRD/em2Ww4NM34lQ6zrunuEDdAxYjzxGrHHP07X+Depatr98argKQ 1LLlHu+7NnJ79fjBSTyTNvh6HTyhC+BYeH98Hwsp6P63rhIZfRlhOK4CK06C4m4hVuiMlCkOs IYQJJ17QC04CZ/nnN1LXkv+QKmxl0BEnQGISVt+uiJb1++hxf1xHNX1c5FvT1OEVVdpxVicsS AvYz0+siqBjAu1mu0nSa30LNFn7sK31fSocnJLHPG5OTkRKDk9DfgGNo2rJbj404dBpp95JlJ RJ9DVphkvRxkBldIfoDlP9X9hluEUqAJpFNFADx+nBKEWkOthlgQJmyDSzPKlDTVSjrg6YMrA xSf17OCrgoxVe2hpKXCVIuaeD7ftFTg0yFNqIxaZ8ZLJY= X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240609_152420_474369_5BD1DAA7 X-CRM114-Status: GOOD ( 11.40 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Hi, i was able to trigger a flood of WARNINGs with Raspberry Pi 3 B Plus ( arm64/defconfig + CONFIG_BCM2835_VCHIQ=m + CONFIG_VCHIQ_CDEV=y ) running the following userspace tool (triggers DMA transfer) vchiq_test -f 1 The command seems to succeed as usual except of the warnings. But these warnings are not reproducible with 32 bit configuration like multi_v7_defconfig on the same Raspberry Pi 3 B Plus: [   64.531826] WARNING: CPU: 3 PID: 867 at drivers/staging/vc04_services/interface/vchiq_arm/vchiq_arm.c:364 vchiq_prepare_bulk_data+0x2cc/0x4e8 [vchiq] [   64.531880] Modules linked in: aes_neon_blk af_alg 8021q garp mrp stp llc microchip raspberrypi_cpufreq brcmfmac_wcc snd_bcm2835(C) lan78xx brcmfmac brcmutil vc4 cfg80211 hci_uart snd_soc_hdmi_codec btqca cec btbcm onboard_usb_hub drm_display_helper bluetooth crct10dif_ce drm_dma_helper raspberrypi_hwmon clk_raspberrypi drm_kms_helper ecdh_generic bcm2835_thermal i2c_bcm2835 ecc pwm_bcm2835 rfkill vchiq(C) drm fuse ip_tables x_tables ipv6 [   64.532010] CPU: 3 PID: 867 Comm: vchiq_test Tainted: G WC         6.4.0-rc4-00297-gf999f38b4e6f #24 [   64.532022] Hardware name: Raspberry Pi 3 Model B Plus Rev 1.3 (DT) [   64.532029] pstate: 00000005 (nzcv daif -PAN -UAO -TCO -DIT -SSBS BTYPE=--) [   64.532038] pc : vchiq_prepare_bulk_data+0x2cc/0x4e8 [vchiq] [   64.532060] lr : vchiq_prepare_bulk_data+0x320/0x4e8 [vchiq] [   64.532080] sp : ffff8000093dbb10 [   64.532086] x29: ffff8000093dbb10 x28: 00000000000000a4 x27: ffff80000b065000 [   64.532101] x26: ffff80000b065010 x25: ffff80000b065060 x24: 0000000000000000 [   64.532114] x23: 0000000000000001 x22: 0000000000000001 x21: 0000000000000001 [   64.532127] x20: ffff80000b065040 x19: 0000000000000002 x18: 0000000000000800 [   64.532139] x17: 001fffffffffffff x16: ffffdefad16fc6d0 x15: ffff1449b720713c [   64.532151] x14: 0000000000000000 x13: ffff1449b7207130 x12: ffff1449b7207100 [   64.532163] x11: 0000000000000030 x10: 00000000000000ff x9 : 000000003f000000 [   64.532175] x8 : 00000000351ae800 x7 : 00000000351ae800 x6 : 00000000000000ff [   64.532187] x5 : ffff1449b51ae801 x4 : ffff1449b9efc001 x3 : 0000000000000000 [   64.532201] x2 : 00000000f51ae800 x1 : fffffc5126e7bf02 x0 : 0000000000000001 [   64.532218] Call trace: [   64.532222]  vchiq_prepare_bulk_data+0x2cc/0x4e8 [vchiq] [   64.532247]  vchiq_bulk_transfer+0x25c/0x3bc [vchiq] [   64.532272]  vchiq_irq_queue_bulk_tx_rx+0x6c/0x348 [vchiq] [   64.532293]  vchiq_ioctl+0x684/0xbe8 [vchiq] [   64.532313]  __arm64_sys_ioctl+0xa8/0xec [   64.532327]  invoke_syscall+0x48/0x114 [   64.532341]  el0_svc_common.constprop.0+0x44/0xf4 [   64.532353]  do_el0_svc+0x3c/0xa8 [   64.532362]  el0_svc+0x2c/0x84 [   64.532379]  el0t_64_sync_handler+0xbc/0x138 [   64.532389]  el0t_64_sync+0x190/0x194 [   64.532400] ---[ end trace 0000000000000000 ]--- Since there are a lot of WARN_ON in the function, i applied the following patch: @@ -357,15 +357,19 @@ create_pagelist(struct vchiq_instance *instance, char *buf, char __user *ubuf,      k = 0;      for_each_sg(scatterlist, sg, dma_buffers, i) {          u32 len = sg_dma_len(sg); -        u32 addr = sg_dma_address(sg); +        dma_addr_t addr = sg_dma_address(sg);          /* Note: addrs is the address + page_count - 1           * The firmware expects blocks after the first to be page-           * aligned and a multiple of the page size           */ -        WARN_ON(len == 0); -        WARN_ON(i && (i != (dma_buffers - 1)) && (len & ~PAGE_MASK)); -        WARN_ON(i && (addr & ~PAGE_MASK)); +        if (len == 0) +            pr_warn_once("%s: sg_dma_len() == 0\n", __func__); +        else if (i && (i != (dma_buffers - 1)) && (len & ~PAGE_MASK)) +            pr_warn_once("%s: following block not page aligned\n", __func__); +        else if (i && (addr & ~PAGE_MASK)) +            pr_warn_once("%s: block %u, DMA address %pad doesn't align with PAGE_MASK 0x%lx\n", __func__, i, &addr, PAGE_MASK); +          if (is_adjacent_block(addrs, addr, k))              addrs[k - 1] += ((len + PAGE_SIZE - 1) >> PAGE_SHIFT);          else @@ -385,11 +389,15 @@ create_pagelist(struct vchiq_instance *instance, char *buf, char __user *ubuf,              return NULL;          } -        WARN_ON(!g_free_fragments); +        if (!g_free_fragments) +            pr_warn_once("%s: !g_free_fragments\n", __func__);          down(&g_free_fragments_mutex);          fragments = g_free_fragments; -        WARN_ON(!fragments); + +        if (!fragments) +            pr_warn_once("%s: !fragments\n", __func__); +          g_free_fragments = *(char **)g_free_fragments;          up(&g_free_fragments_mutex);          pagelist->type = PAGELIST_READ_WITH_FRAGMENTS + After that i'm getting the following warning:   create_pagelist: block 1, DMA address 0x00000000f5f62800 doesn't align with PAGE_MASK 0xfffffffffffff000 Then i bisected the issue to this commit: commit 1c1a429efd4ee8ca244cc2401365c983cda4ed76 Author: Catalin Marinas Date:   Mon Jun 12 16:32:01 2023 +0100     arm64: enable ARCH_WANT_KMALLOC_DMA_BOUNCE for arm64     With the DMA bouncing of unaligned kmalloc() buffers now in place, enable     it for arm64 to allow the kmalloc-{8,16,32,48,96} caches.  In addition,     always create the swiotlb buffer even when the end of RAM is within the     32-bit physical address range (the swiotlb buffer can still be disabled on     the kernel command line). So how can the root cause of these warnings be fixed? Or is the specific WARN_ON() now obsolete? diff --git a/drivers/staging/vc04_services/interface/vchiq_arm/vchiq_arm.c b/drivers/staging/vc04_services/interface/vchiq_arm/vchiq_arm.c index 9fb8f657cc78..2beeb94f629e 100644 --- a/drivers/staging/vc04_services/interface/vchiq_arm/vchiq_arm.c +++ b/drivers/staging/vc04_services/interface/vchiq_arm/vchiq_arm.c @@ -192,7 +192,7 @@ cleanup_pagelistinfo(struct vchiq_instance *instance, struct vchiq_pagelist_info  }  static inline bool -is_adjacent_block(u32 *addrs, u32 addr, unsigned int k) +is_adjacent_block(u32 *addrs, dma_addr_t addr, unsigned int k)  {      u32 tmp;