From patchwork Fri Aug 16 02:07:23 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Huang, Ying" X-Patchwork-Id: 13765355 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2483CC52D7C for ; Fri, 16 Aug 2024 02:08:43 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 72AE98D0031; Thu, 15 Aug 2024 22:08:43 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 6DAF28D002B; Thu, 15 Aug 2024 22:08:43 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 57C5F8D0031; Thu, 15 Aug 2024 22:08:43 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 3A69B8D002B for ; Thu, 15 Aug 2024 22:08:43 -0400 (EDT) Received: from smtpin13.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 9C0D51418A3 for ; Fri, 16 Aug 2024 02:08:42 +0000 (UTC) X-FDA: 82456474884.13.098744C Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.13]) by imf27.hostedemail.com (Postfix) with ESMTP id C4D304000A for ; Fri, 16 Aug 2024 02:08:39 +0000 (UTC) Authentication-Results: imf27.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=HIhvf60f; dmarc=pass (policy=none) header.from=intel.com; spf=pass (imf27.hostedemail.com: domain of ying.huang@intel.com designates 192.198.163.13 as permitted sender) smtp.mailfrom=ying.huang@intel.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1723774066; a=rsa-sha256; cv=none; b=bXn5v/L7ZX9vP4+wkkLvY/Igllpi/9ERmBVYg1vMtTZgx2V+OKMc2Cmp0sml/V5/xqRDat XI7acPnQNKAz4W8vmz3Q8oMT+2YhihiZhGKEKrXI2OPGHYf8aPt2/n9R+HTkhH0wRD91G2 mguZjgPwnUtmhAJQcTHO8SV3pbeKl2c= ARC-Authentication-Results: i=1; imf27.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=HIhvf60f; dmarc=pass (policy=none) header.from=intel.com; spf=pass (imf27.hostedemail.com: domain of ying.huang@intel.com designates 192.198.163.13 as permitted sender) smtp.mailfrom=ying.huang@intel.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1723774066; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:references:dkim-signature; bh=2NTCXqp5U5jdOsDJ705EqTO/gDOYPV5b6eBC2YekxK8=; b=hMgvyKqpKRh4cRSLeu/WBghjYi9EN2Eo+QBX2g7nxxVLxMBxvmcws5DD3zBhecKTDmeIcm zBrvONlknYBxfLF/Lf3mKyPL9wuXQ/Jzu1CLwK5lyetqfPNrPVEMYHaV1jtz3ioVeLkKXA KOpVkUQfj1ljCHDZ4QRUejtTbhtmxjU= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1723774120; x=1755310120; h=from:to:cc:subject:date:message-id:mime-version: content-transfer-encoding; bh=Cn8Icje7WZwUT1SVd8Grw4AeWoUygNe+OPhYpqavebw=; b=HIhvf60fruW+qR6shSPRgXSI+5cfAnBIqe4C/j+NSkIwZ0D5J0ImUCY7 6kEFtCOVtAuL5TiqYnzvTu2JkTIUtrR6Snt3xvewiS2M0g5dvWXa+5tVq 5ScJKXjEeaE/drPqrHsfKPenC+OkAlnjz7Y1Zf40WqkNiRicfZsMY65ku 1ef/yGyPwyXm798syPVenAjSSq+U+vPRsGDeeE7sKjzVV9Bvh47Et2MYH VUjVEJwFDlDrcVHmQv11KxSorBrQDQx4MnIGHYf7/Ry3LZdjtQDb3ABcT L9SHAOKVvj7EbL4d8EkFzabxWjrmoiX3maf0r4B6vQmRXNfV5OivudN/T Q==; X-CSE-ConnectionGUID: spyBLZgiQP+yUkLVg0FXrA== X-CSE-MsgGUID: 79BkBZSFRLe3Q/6yWxD+2Q== X-IronPort-AV: E=McAfee;i="6700,10204,11165"; a="24961573" X-IronPort-AV: E=Sophos;i="6.10,150,1719903600"; d="scan'208";a="24961573" Received: from fmviesa005.fm.intel.com ([10.60.135.145]) by fmvoesa107.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 15 Aug 2024 19:08:38 -0700 X-CSE-ConnectionGUID: FNewU1WHQw+9ewoOr15BIQ== X-CSE-MsgGUID: 1oIgxPjmQROTb9IJeYv+8w== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.10,150,1719903600"; d="scan'208";a="63945215" Received: from unknown (HELO yhuang6-mobl2.ccr.corp.intel.com) ([10.245.243.178]) by fmviesa005-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 15 Aug 2024 19:08:15 -0700 From: Huang Ying To: Andrew Morton Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, linux-cxl@vger.kernel.org, Huang Ying , Dan Williams , Davidlohr Bueso , Jonathan Cameron , Dave Jiang , Alison Schofield , Vishal Verma , Ira Weiny , Alistair Popple , Andy Shevchenko , Bjorn Helgaas , Baoquan He Subject: [PATCH] Resource: fix region_intersects() for CXL memory Date: Fri, 16 Aug 2024 10:07:23 +0800 Message-Id: <20240816020723.771196-1-ying.huang@intel.com> X-Mailer: git-send-email 2.39.2 MIME-Version: 1.0 X-Rspamd-Queue-Id: C4D304000A X-Rspam-User: X-Rspamd-Server: rspam05 X-Stat-Signature: a1w93tqq4qfiiuixd3mcyfmhtumsoe3g X-HE-Tag: 1723774119-750741 X-HE-Meta: U2FsdGVkX1/KDc1ok3+3kshXKdLQS/TjMratueqSPWU+20vuNcDfrid1OIsZnjVbdoEGgoTHRxB1iY320QGY5k0PAkwaJ8knyTwTCpoZe0Cd6fVWQtM5Y3DpjNcqfqLiWuQ6ZAA0mAVuD/pGxQth2sW5+4tIbVvtiiFbJN4QlotSMC5eNUSBwhhpzOj0qz6lIoZwJkNnqBiuyoRMdAoH2/vMNlSo6yKooOrfF2LI5NKkf6L7GKWMfHEQs5GyPEGlLt/LCXhkALyHYtQNiDWvvBY8G0oogTs1j30YHXl1jnoCMbiU3N1wNmWZRSVL2mNVL1WdX94cd6PVh6dVkw36nV2g1pKbebFnCFEwgh9ah/qUpU2TExlYI8hsNsrNRKF035v9gVESvoYQaAUxTwX6yJEbo8GY61PBzEliG8f4qccuJiE2ouiqH6lHrCzpasQvCiU6RQ+lbsmlAf/zvcBsJTK2rSAyUhb2YwLfhxwNLhNW3mWFiplageybP4e9nJYBNKmAM0MaAw8EdkbnwJUr0eAQR5Ad4C1hUafYDGVaXAAK2jUNUblEgHhVG7A2uLIjES93jZpg0qMFzcXdeIh4Iune2wa5hUl0Jse3uD0+3jBLurYBy0C1IyaVotzvbyhZodQwAsKudxOn4xdOB1aCZ0Br+hwdkF9Ii20sSeEDcky9neKmQNzq+5sCiWLrRc4gl0xBDFniFgFylH5rjXgmRTJvGo7HySlvitPsidnyXF7tPZ6hS1uGRS0OHrQUVgyYpsucsb33Z2TVCkz3CiE0E5PVXu08W6dKrvv2+GHH1IW2wfcf4Ws14wZhwDUimJobtaZMW9/XHZEiuCy+9z7NE1ZT+25ToABqIG4EhE6lpNYopuxyIXzGO3ZsFdvhaUxW9FmUIrDOsC4X96ffEOq1M9p4FHBi299HGf6B/FV18usc0UHPn8zJ41yqljYtn8WPcaWKSMxKOxa8TSvQhxv YNE7R36B iufYCReGlC5GWqUyOHcZGEh+mlEJYN7BO+cj7ZvBsMhzzqESaoirAUI06rL0tvQ/vcec6vKd8nu0Ke0tLeAkh7CNG5hh5N3Or/rhOJuHiaTWmIU56+ygtf3nVTKWvY4Avh2oRGJoEIcFW8Rus0S1lj0C00NtF9nIzGyrPchI3v/wgGuY2/Omnc2KjypicTDtDy62kK+eH9ry0esArVYSDn744wJiKUaxyN9ukobr0CGymJcxPsyZ72dm7IiS2R8t0VuM4X7PaDD3tLThrjH0HDUa0kWze5IqTkIuX7WAd40w2ZsveJoLSbXrC9mQFc0IvYkoF0YUGz/NLHyCRgzCjwQlW7BHGiHsk0svGAVqZt1cusH+qHSAppuwvS37p9c10GjWVRjPrIvWrtYpbOeOs6De9ZrhYjezl2qlE X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On a system with CXL memory installed, the resource tree (/proc/iomem) related to CXL memory looks like something as follows. 490000000-50fffffff : CXL Window 0 490000000-50fffffff : region0 490000000-50fffffff : dax0.0 490000000-50fffffff : System RAM (kmem) When the following command line is run to try writing some memory in CXL memory range, $ dd if=data of=/dev/mem bs=1k seek=19136512 count=1 dd: error writing '/dev/mem': Bad address 1+0 records in 0+0 records out 0 bytes copied, 0.0283507 s, 0.0 kB/s the command fails as expected. However, the error code is wrong. It should be "Operation not permitted" instead of "Bad address". And, the following warning is reported in kernel log. ioremap on RAM at 0x0000000490000000 - 0x0000000490000fff WARNING: CPU: 2 PID: 416 at arch/x86/mm/ioremap.c:216 __ioremap_caller.constprop.0+0x131/0x35d Modules linked in: cxl_pmem libnvdimm cbc encrypted_keys cxl_pmu CPU: 2 UID: 0 PID: 416 Comm: dd Not tainted 6.11.0-rc3-kvm #40 Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.16.3-0-ga6ed6b701f0a-prebuilt.qemu.org 04/01/2014 RIP: 0010:__ioremap_caller.constprop.0+0x131/0x35d Code: 2d 80 3d 24 6a a1 02 00 75 c1 48 8d 54 24 70 48 8d b4 24 90 00 00 00 48 c7 c7 40 3a 05 8a c6 05 07 6a a1 02 01 e8 a3 a0 01 00 <0f> 0b eb 9d 48 8b 84 24 90 00 00 00 48 8d 4c 24 60 89 ea 48 bf 00 RSP: 0018:ffff888105387bf0 EFLAGS: 00010282 RAX: 0000000000000000 RBX: 0000000490000fff RCX: 0000000000000000 RDX: 0000000000000001 RSI: 0000000000000003 RDI: ffffed1020a70f73 RBP: 0000000000000000 R08: ffffed100d9fce92 R09: 0000000000000001 R10: ffffffff892348e7 R11: ffffed100d9fce91 R12: 0000000490000000 R13: 0000000000000001 R14: 0000000000000001 R15: ffff888105387ca0 FS: 00007f86c438c740(0000) GS:ffff88806ce00000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 000055ba75b1b818 CR3: 0000000005231000 CR4: 0000000000350eb0 Call Trace: ? __warn+0xd7/0x1b8 ? __ioremap_caller.constprop.0+0x131/0x35d ? report_bug+0x136/0x19e ? __ioremap_caller.constprop.0+0x131/0x35d ? handle_bug+0x3c/0x64 ? exc_invalid_op+0x13/0x38 ? asm_exc_invalid_op+0x16/0x20 ? irq_work_claim+0x16/0x38 ? __ioremap_caller.constprop.0+0x131/0x35d ? tracer_hardirqs_off+0x18/0x16d ? kmem_cache_debug_flags+0x16/0x23 ? memremap+0xcb/0x184 ? iounmap+0xfe/0xfe ? lock_sync+0xc7/0xc7 ? lock_sync+0xc7/0xc7 ? rcu_is_watching+0x1c/0x38 ? do_raw_read_unlock+0x37/0x42 ? _raw_read_unlock+0x1f/0x2f memremap+0xcb/0x184 ? pfn_valid+0x159/0x159 ? resource_is_exclusive+0xba/0xc5 xlate_dev_mem_ptr+0x25/0x2f write_mem+0x94/0xfb vfs_write+0x128/0x26d ? kernel_write+0x89/0x89 ? rcu_is_watching+0x1c/0x38 ? __might_fault+0x72/0xba ? __might_fault+0x72/0xba ? rcu_is_watching+0x1c/0x38 ? lock_release+0xba/0x13e ? files_lookup_fd_raw+0x40/0x4b ? __fget_light+0x64/0x89 ksys_write+0xac/0xfe ? __ia32_sys_read+0x40/0x40 ? tracer_hardirqs_off+0x18/0x16d ? tracer_hardirqs_on+0x11/0x146 do_syscall_64+0x9a/0xfd entry_SYSCALL_64_after_hwframe+0x4b/0x53 RIP: 0033:0x7f86c4487140 Code: 40 00 48 8b 15 c1 9c 0d 00 f7 d8 64 89 02 48 c7 c0 ff ff ff ff eb b7 0f 1f 00 80 3d a1 24 0e 00 00 74 17 b8 01 00 00 00 0f 05 <48> 3d 00 f0 ff ff 77 58 c3 0f 1f 80 00 00 00 00 48 83 ec 28 48 89 RSP: 002b:00007ffca9f62af8 EFLAGS: 00000202 ORIG_RAX: 0000000000000001 RAX: ffffffffffffffda RBX: 0000000000000400 RCX: 00007f86c4487140 RDX: 0000000000000400 RSI: 000055ba75b1a000 RDI: 0000000000000001 RBP: 000055ba75b1a000 R08: 0000000000000000 R09: 00007f86c457c080 R10: 00007f86c43a84d0 R11: 0000000000000202 R12: 0000000000000000 R13: 0000000000000000 R14: 000055ba75b1a000 R15: 0000000000000400 irq event stamp: 0 hardirqs last enabled at (0): [<0000000000000000>] 0x0 hardirqs last disabled at (0): [] copy_process+0xb60/0x255f softirqs last enabled at (0): [] copy_process+0xb60/0x255f softirqs last disabled at (0): [<0000000000000000>] 0x0 After investigation, we found the following bug. In the above resource tree, "System RAM" is a descendant of "CXL Window 0" instead of a top level resource. So, region_intersects() will report no System RAM resources in the CXL memory region incorrectly, because it only checks the top level resources. Consequently, devmem_is_allowed() will return 1 (allow access via /dev/mem) for CXL memory region incorrectly. Fortunately, ioremap() doesn't allow to map System RAM and reject the access. However, region_intersects() needs to be fixed to work correctly with the resources tree with CXL Window as above. To fix it, we will search matched resources in the descendant resources too. So, we will not miss any matched resources anymore. Signed-off-by: "Huang, Ying" Cc: Dan Williams Cc: Davidlohr Bueso Cc: Jonathan Cameron Cc: Dave Jiang Cc: Alison Schofield Cc: Vishal Verma Cc: Ira Weiny Cc: Alistair Popple Cc: Andy Shevchenko Cc: Bjorn Helgaas Cc: Baoquan He --- kernel/resource.c | 44 +++++++++++++++++++++++++++++++++++++------- 1 file changed, 37 insertions(+), 7 deletions(-) diff --git a/kernel/resource.c b/kernel/resource.c index 14777afb0a99..c97a5add9394 100644 --- a/kernel/resource.c +++ b/kernel/resource.c @@ -542,18 +542,48 @@ static int __region_intersects(struct resource *parent, resource_size_t start, { struct resource res; int type = 0; int other = 0; - struct resource *p; + struct resource *p, *dp; + resource_size_t ostart, oend; + bool is_type; res.start = start; res.end = start + size - 1; for (p = parent->child; p ; p = p->sibling) { - bool is_type = (((p->flags & flags) == flags) && - ((desc == IORES_DESC_NONE) || - (desc == p->desc))); - - if (resource_overlaps(p, &res)) - is_type ? type++ : other++; + if (!resource_overlaps(p, &res)) + continue; + is_type = (((p->flags & flags) == flags) && + ((desc == IORES_DESC_NONE) || (desc == p->desc))); + if (is_type) { + type++; + continue; + } + /* + * Continue to search in descendant resources. Unless + * the matched descendant resources cover the whole + * overlapped range, increase 'other', because it + * overlaps with 'p' at least. + */ + other++; + ostart = max(res.start, p->start); + oend = min(res.end, p->end); + for_each_resource(p, dp, false) { + if (!resource_overlaps(dp, &res)) + continue; + is_type = (((dp->flags & flags) == flags) && + ((desc == IORES_DESC_NONE) || + (desc == dp->desc))); + if (is_type) { + type++; + if (dp->start > ostart) + break; + if (dp->end >= oend) { + other--; + break; + } + ostart = dp->end + 1; + } + } } if (type == 0)