From patchwork Wed Dec 26 08:25:08 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andy Duan X-Patchwork-Id: 10742925 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id A7ABC13B5 for ; Wed, 26 Dec 2018 08:25:19 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 98080283E7 for ; Wed, 26 Dec 2018 08:25:19 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 8C0D528957; Wed, 26 Dec 2018 08:25:19 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=unavailable version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id DFFAF283E7 for ; Wed, 26 Dec 2018 08:25:18 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726013AbeLZIZS (ORCPT ); Wed, 26 Dec 2018 03:25:18 -0500 Received: from mail-eopbgr30079.outbound.protection.outlook.com ([40.107.3.79]:57871 "EHLO EUR03-AM5-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1725987AbeLZIZS (ORCPT ); Wed, 26 Dec 2018 03:25:18 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nxp.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=zwgHVODRL8A4eEw5TVUe7wsj9KWQfivMX/21ZOqfkUs=; b=a1H0VwhR775SRCNbcKn4d5CWai38USY+JvFjlToKIxD4zSiJQkc+JfphUazGjULrwLOZpykH7lXiXA5C1fhmlGcEv+G06GfHGw+T6NaaBUce0AIkFyUof6V0ZJyuimnsXU/pPss9/wtCDQhwJK7hYb8LUXOJXwV9w2lPoIDmz+g= Received: from VI1PR0402MB3600.eurprd04.prod.outlook.com (52.134.5.23) by VI1PR0402MB3520.eurprd04.prod.outlook.com (52.134.4.25) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.1446.24; Wed, 26 Dec 2018 08:25:09 +0000 Received: from VI1PR0402MB3600.eurprd04.prod.outlook.com ([fe80::e101:27fc:e3fd:a1c4]) by VI1PR0402MB3600.eurprd04.prod.outlook.com ([fe80::e101:27fc:e3fd:a1c4%2]) with mapi id 15.20.1446.027; Wed, 26 Dec 2018 08:25:09 +0000 From: Andy Duan To: "bjorn.andersson@linaro.org" , "ohad@wizery.com" CC: "linux-remoteproc@vger.kernel.org" , "linux-mm@kvack.org" , "anup@brainfault.org" , "loic.pallardy@st.com" , "ard.biesheuvel@linaro.org" , dl-linux-imx , Richard Zhu , Jason Liu , Peng Fan Subject: [rpmsg PATCH v2 1/1] rpmsg: virtio_rpmsg_bus: fix unexpected huge vmap mappings Thread-Topic: [rpmsg PATCH v2 1/1] rpmsg: virtio_rpmsg_bus: fix unexpected huge vmap mappings Thread-Index: AQHUnPSDrjCGi2kU3UyftbYgSkxi4Q== Date: Wed, 26 Dec 2018 08:25:08 +0000 Message-ID: <1545812449-32455-1-git-send-email-fugang.duan@nxp.com> Accept-Language: zh-CN, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-mailer: git-send-email 1.9.1 x-clientproxiedby: HK2PR02CA0164.apcprd02.prod.outlook.com (2603:1096:201:1f::24) To VI1PR0402MB3600.eurprd04.prod.outlook.com (2603:10a6:803:a::23) authentication-results: spf=none (sender IP is ) smtp.mailfrom=fugang.duan@nxp.com; x-ms-exchange-messagesentrepresentingtype: 1 x-originating-ip: [119.31.174.66] x-ms-publictraffictype: Email x-microsoft-exchange-diagnostics: 1;VI1PR0402MB3520;6:rzXHnYIrF4jd1ntQQeGAdlRzPDuzSc9GLstzAsa0XQdsH7WKFfs1RNYSzkrjTn2T9Cxf1yLGcJTCjGDW5CVLga4soZ3BAr/SKIp0XlQOMObV1B/lM2xYW9OXiJHr89LN5bG44Wkqp0qjRd5ZUX1bPlfKbelArpBZQwGE6J2K53M4GoUg42FHh5zHTji2sxa9xlu3qCu5SqmPa62+z0AXMms9PiLWBXDtzw/lf1chtVkGT/tJwUaEWdcW3IzmW5wGxH0Gy3zBXmP8/Oc0Ksqz/X8pZOVM8jK4q3ThaVXsN4chD6af1GVhhXSpN/29FqbAvJdLWn/jde/YLqqctjyHbx1j+LJZRLy6kylMjFt0r5zAu4kGicH16iMvE6Yxrr1FC5un0ICreZhtG+0sqsnZeb2Xoy3vXyO+cmzxj+udyqpoBy9welVRBBdAkyTA6HrMQTyUF5Q+YFWL+FKDUBbq2g==;5:BHDHKmXXk77E6tSB5FHYrmZfEw5Y/3Fh03FFxYrVezsHWxiPN7o23ZsX0ARoix98gT59GCWN8itb5lTHjoutLhgC3PuoAtL39Lq1yXbFs9heTb4eHf36rUNMUkTkPX3U06IWRhPBcC/ex9UuPJPnyF+7/0lInlvj0ga508g8KXk=;7:MZ+FfxDv/vagugieVFRgRPlTMxdr8sntfDve/BbYa3y9T2z00sEi1++ZXoRoYWnhLpNvjd44ckAdXcdAxOi3B2oFGeebNsxs6RQnltRhqRBOd5F4eIDFcSQjKMsmK9SS+84V1ST231sAOUV5XduD0A== x-ms-office365-filtering-correlation-id: 474e31c2-e14e-4a51-46ca-08d66b0ba5a1 x-ms-office365-filtering-ht: Tenant x-microsoft-antispam: BCL:0;PCL:0;RULEID:(2390118)(7020095)(4652040)(8989299)(4534185)(4627221)(201703031133081)(201702281549075)(8990200)(5600074)(711020)(4618075)(2017052603328)(7153060)(7193020);SRVR:VI1PR0402MB3520; x-ms-traffictypediagnostic: VI1PR0402MB3520: x-microsoft-antispam-prvs: x-exchange-antispam-report-cfa-test: BCL:0;PCL:0;RULEID:(8211001083)(3230021)(908002)(999002)(5005026)(6040522)(2401047)(8121501046)(3002001)(3231475)(944501520)(52105112)(93006095)(93001095)(10201501046)(6055026)(6041310)(20161123560045)(201703131423095)(201702281528075)(20161123555045)(201703061421075)(201703061406153)(20161123564045)(20161123562045)(20161123558120)(201708071742011)(7699051)(76991095);SRVR:VI1PR0402MB3520;BCL:0;PCL:0;RULEID:;SRVR:VI1PR0402MB3520; x-forefront-prvs: 0898A6E028 x-forefront-antispam-report: SFV:NSPM;SFS:(10009020)(136003)(346002)(376002)(366004)(39860400002)(396003)(189003)(199004)(478600001)(5660300001)(71190400001)(26005)(97736004)(4326008)(81166006)(186003)(14454004)(45080400002)(8676002)(8936002)(50226002)(6512007)(6486002)(2616005)(81156014)(68736007)(53936002)(71200400001)(6436002)(7736002)(25786009)(102836004)(305945005)(2501003)(476003)(486006)(86362001)(575784001)(5024004)(386003)(6116002)(3846002)(99286004)(316002)(6506007)(110136005)(54906003)(256004)(52116002)(14444005)(36756003)(105586002)(106356001)(2906002)(66066001)(14583001);DIR:OUT;SFP:1101;SCL:1;SRVR:VI1PR0402MB3520;H:VI1PR0402MB3600.eurprd04.prod.outlook.com;FPR:;SPF:None;LANG:en;PTR:InfoNoRecords;MX:1;A:1; received-spf: None (protection.outlook.com: nxp.com does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 x-microsoft-antispam-message-info: AsrJwTF7+mdLsMKE69GQJpRg7ao5eo7+R9a6DzbjbAbvl6sA9kexVcAH6m2I1IraJ5UZiY6eyT2WC0KzP/txgiOhf1F2QE4NOYpFl2XaDI7sVLEMwB5EX2Ad36mxM9oYoKjHGQKxud4JI7KB1yhYUA7tvfAEQRiyzG4mZ7KWDKJfTwZSEM4Dz395zo4PtO5knggePKB2rDDaQ36N52dUwBofhiMwWiccn7HQqxlCpYuQacWcxSvkmViO4PvvHgvyn72NfAfDVaT9sNkHL3fPpmcNkElfc6kVOQlDVags0IhzoF9vP/YbaS0TW2P0WO9M spamdiagnosticoutput: 1:99 spamdiagnosticmetadata: NSPM MIME-Version: 1.0 X-OriginatorOrg: nxp.com X-MS-Exchange-CrossTenant-Network-Message-Id: 474e31c2-e14e-4a51-46ca-08d66b0ba5a1 X-MS-Exchange-CrossTenant-originalarrivaltime: 26 Dec 2018 08:25:08.9788 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 686ea1d3-bc2b-4c6f-a92c-d99c5c301635 X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0402MB3520 Sender: linux-remoteproc-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-remoteproc@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Fugang Duan If RPMSG dma memory allocate from per-device mem pool by calling .dma_alloc_coherent(), the size is bigger than 2M bytes and alignment with 2M (PMD_SIZE), then kernel dump by calling .vmalloc_to_page(). Since per-device dma pool do vmap mappings by __ioremap(), __ioremap() might use the hugepage mapping, which in turn will cause the vmalloc_page failed to return the correct page due to the PTE not setup. For exp, when reserve 8M bytes per-device dma mem pool, __ioremap() will use hugepage mapping: __ioremap ioremap_page_range ioremap_pud_range ioremap_pmd_range pmd_set_huge(pmd, phys_addr + addr, prot) Commit:029c54b09599 ("mm/vmalloc.c: huge-vmap: fail gracefully on unexpected huge vmap mapping") ensure that vmalloc_to_page() does not go off into the weeds trying to dereference huge PUDs or PMDs as table entries: rpmsg_sg_init -> vmalloc_to_page-> WARN_ON_ONCE(pmd_bad(*pmd)); In generally, .dma_alloc_coherent() allocate memory from CMA pool/DMA pool/atomic_pool, or swiotlb slabs pool, the virt address mapping to physical address should be lineal, so for the rpmsg scatterlist initialization can use pfn to find the page to avoid to call .vmalloc_to_page(). Kernel dump: [ 0.881722] WARNING: CPU: 0 PID: 1 at mm/vmalloc.c:301 vmalloc_to_page+0xbc/0xc8 [ 0.889094] Modules linked in: [ 0.892139] CPU: 0 PID: 1 Comm: swapper/0 Not tainted 4.14.78-05581-gc61a572 #206 [ 0.899604] Hardware name: Freescale i.MX8QM MEK (DT) [ 0.904643] task: ffff8008f6c98000 task.stack: ffff000008068000 [ 0.910549] PC is at vmalloc_to_page+0xbc/0xc8 [ 0.914987] LR is at rpmsg_sg_init+0x70/0xcc [ 0.919238] pc : [] lr : [] pstate: 40000045 [ 0.926619] sp : ffff00000806b8b0 [ 0.929923] x29: ffff00000806b8b0 x28: ffff00000961cdf0 [ 0.935220] x27: ffff00000961cdf0 x26: 0000000000000000 [ 0.940519] x25: 0000000000040000 x24: ffff00000961ce40 [ 0.945819] x23: ffff00000f000000 x22: ffff00000961ce30 [ 0.951118] x21: 0000000000000000 x20: ffff00000806b950 [ 0.956417] x19: 0000000000000000 x18: 000000000000000e [ 0.961717] x17: 0000000000000001 x16: 0000000000000019 [ 0.967016] x15: 0000000000000033 x14: 616d64202c303030 [ 0.972316] x13: 3030306630303030 x12: 3066666666206176 [ 0.977615] x11: 203a737265666675 x10: 62203334394c203a [ 0.982914] x9 : 000000000000009f x8 : ffff00000806b970 [ 0.988214] x7 : 0000000000000000 x6 : ffff000009690712 [ 0.993513] x5 : 0000000000000000 x4 : 0000000080000000 [ 0.998812] x3 : 00e8000090800f0d x2 : ffff8008ffffd3c0 [ 1.004112] x1 : 0000000000000000 x0 : ffff00000f000000 [ 1.009416] Call trace: [ 1.011849] Exception stack(0xffff00000806b770 to 0xffff00000806b8b0) [ 1.018279] b760: ffff00000f000000 0000000000000000 [ 1.026094] b780: ffff8008ffffd3c0 00e8000090800f0d 0000000080000000 0000000000000000 [ 1.033915] b7a0: ffff000009690712 0000000000000000 ffff00000806b970 000000000000009f [ 1.041731] b7c0: 62203334394c203a 203a737265666675 3066666666206176 3030306630303030 [ 1.049550] b7e0: 616d64202c303030 0000000000000033 0000000000000019 0000000000000001 [ 1.057368] b800: 000000000000000e 0000000000000000 ffff00000806b950 0000000000000000 [ 1.065188] b820: ffff00000961ce30 ffff00000f000000 ffff00000961ce40 0000000000040000 [ 1.073008] b840: 0000000000000000 ffff00000961cdf0 ffff00000961cdf0 ffff00000806b8b0 [ 1.080825] b860: ffff000008ac471c ffff00000806b8b0 ffff0000081c80d4 0000000040000045 [ 1.088646] b880: ffff0000092c8528 ffff00000806b890 ffffffffffffffff ffff000008ac4710 [ 1.096461] b8a0: ffff00000806b8b0 ffff0000081c80d4 [ 1.101327] [] vmalloc_to_page+0xbc/0xc8 [ 1.106800] [] rpmsg_probe+0x1f0/0x49c [ 1.112107] [] virtio_dev_probe+0x198/0x210 [ 1.117839] [] driver_probe_device+0x220/0x2d4 [ 1.123829] [] __device_attach_driver+0x98/0xc8 [ 1.129913] [] bus_for_each_drv+0x54/0x94 [ 1.135470] [] __device_attach+0xc4/0x12c [ 1.141029] [] device_initial_probe+0x10/0x18 [ 1.146937] [] bus_probe_device+0x90/0x98 [ 1.152501] [] device_add+0x3f4/0x570 [ 1.157709] [] device_register+0x1c/0x28 [ 1.163182] [] register_virtio_device+0xb8/0x114 [ 1.169353] [] imx_rpmsg_probe+0x3a0/0x5d0 [ 1.175003] [] platform_drv_probe+0x50/0xbc [ 1.180730] [] driver_probe_device+0x220/0x2d4 [ 1.186725] [] __driver_attach+0xa4/0xa8 [ 1.192199] [] bus_for_each_dev+0x58/0x98 [ 1.197759] [] driver_attach+0x20/0x28 [ 1.203058] [] bus_add_driver+0x1c0/0x224 [ 1.208619] [] driver_register+0x68/0x108 [ 1.214178] [] __platform_driver_register+0x4c/0x54 [ 1.220614] [] imx_rpmsg_init+0x1c/0x50 [ 1.225999] [] do_one_initcall+0x38/0x124 [ 1.231560] [] kernel_init_freeable+0x18c/0x228 [ 1.237640] [] kernel_init+0x10/0x100 [ 1.242849] [] ret_from_fork+0x10/0x18 [ 1.248154] ---[ end trace bcc95d4e07033434 ]--- v2: - use pfn_to_page(PHYS_PFN(x)) instead of phys_to_page(x) since .phys_to_page() interface has arch platform limitation. Reviewed-by: Richard Zhu Suggested-and-reviewed-by: Jason Liu Signed-off-by: Fugang Duan --- drivers/rpmsg/virtio_rpmsg_bus.c | 25 +++++++++++++------------ 1 file changed, 13 insertions(+), 12 deletions(-) diff --git a/drivers/rpmsg/virtio_rpmsg_bus.c b/drivers/rpmsg/virtio_rpmsg_bus.c index 664f957..d548bd0 100644 --- a/drivers/rpmsg/virtio_rpmsg_bus.c +++ b/drivers/rpmsg/virtio_rpmsg_bus.c @@ -196,16 +196,17 @@ static int virtio_rpmsg_trysend_offchannel(struct rpmsg_endpoint *ept, u32 src, * location (in vmalloc or in kernel). */ static void -rpmsg_sg_init(struct scatterlist *sg, void *cpu_addr, unsigned int len) +rpmsg_sg_init(struct virtproc_info *vrp, struct scatterlist *sg, + void *cpu_addr, unsigned int len) { - if (is_vmalloc_addr(cpu_addr)) { - sg_init_table(sg, 1); - sg_set_page(sg, vmalloc_to_page(cpu_addr), len, - offset_in_page(cpu_addr)); - } else { - WARN_ON(!virt_addr_valid(cpu_addr)); - sg_init_one(sg, cpu_addr, len); - } + unsigned int offset; + dma_addr_t dev_add = vrp->bufs_dma + (cpu_addr - vrp->rbufs); + struct page *page = pfn_to_page(PHYS_PFN(dma_to_phys(vrp->bufs_dev, + dev_add))); + + offset = offset_in_page(cpu_addr); + sg_init_table(sg, 1); + sg_set_page(sg, page, len, offset); } /** @@ -626,7 +627,7 @@ static int rpmsg_send_offchannel_raw(struct rpmsg_device *rpdev, msg, sizeof(*msg) + msg->len, true); #endif - rpmsg_sg_init(&sg, msg, sizeof(*msg) + len); + rpmsg_sg_init(vrp, &sg, msg, sizeof(*msg) + len); mutex_lock(&vrp->tx_lock); @@ -750,7 +751,7 @@ static int rpmsg_recv_single(struct virtproc_info *vrp, struct device *dev, dev_warn(dev, "msg received with no recipient\n"); /* publish the real size of the buffer */ - rpmsg_sg_init(&sg, msg, vrp->buf_size); + rpmsg_sg_init(vrp, &sg, msg, vrp->buf_size); /* add the buffer back to the remote processor's virtqueue */ err = virtqueue_add_inbuf(vrp->rvq, &sg, 1, msg, GFP_KERNEL); @@ -934,7 +935,7 @@ static int rpmsg_probe(struct virtio_device *vdev) struct scatterlist sg; void *cpu_addr = vrp->rbufs + i * vrp->buf_size; - rpmsg_sg_init(&sg, cpu_addr, vrp->buf_size); + rpmsg_sg_init(vrp, &sg, cpu_addr, vrp->buf_size); err = virtqueue_add_inbuf(vrp->rvq, &sg, 1, cpu_addr, GFP_KERNEL);