From patchwork Thu Aug 29 17:42:23 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrii Nakryiko X-Patchwork-Id: 13783492 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6162DC87FCD for ; Thu, 29 Aug 2024 17:42:43 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id E42076B00A0; Thu, 29 Aug 2024 13:42:42 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id D7C886B00A4; Thu, 29 Aug 2024 13:42:42 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id BA7966B00AB; Thu, 29 Aug 2024 13:42:42 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 95D956B00A0 for ; Thu, 29 Aug 2024 13:42:42 -0400 (EDT) Received: from smtpin24.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 37F3F80F55 for ; Thu, 29 Aug 2024 17:42:42 +0000 (UTC) X-FDA: 82506002964.24.657E1AB Received: from ams.source.kernel.org (ams.source.kernel.org [145.40.68.75]) by imf16.hostedemail.com (Postfix) with ESMTP id 5838618000B for ; Thu, 29 Aug 2024 17:42:40 +0000 (UTC) Authentication-Results: imf16.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=urHzLm72; spf=pass (imf16.hostedemail.com: domain of andrii@kernel.org designates 145.40.68.75 as permitted sender) smtp.mailfrom=andrii@kernel.org; dmarc=pass (policy=quarantine) header.from=kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1724953315; a=rsa-sha256; cv=none; b=qAUe+CpkVEUNwgdd3pNTC7lOZdd0BC1oqxoRzo+p6cjjJMiT7CSgNIyyJEvI+X22xXUXnv JmNF77GGWZICpPAKCspJlr9v9/iw7flbHhzqH2feCKrReVKzB73D4GxdPmtV8HJjLbOXl2 uhC+ewkSrfo8c4iTAeYBZfxEaEqrIj0= ARC-Authentication-Results: i=1; imf16.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=urHzLm72; spf=pass (imf16.hostedemail.com: domain of andrii@kernel.org designates 145.40.68.75 as permitted sender) smtp.mailfrom=andrii@kernel.org; dmarc=pass (policy=quarantine) header.from=kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1724953315; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=C6Ea2t2suroOwHkv8688e4INVvktX0WEK1owCc2+iOY=; b=TvyTxRngbuwhxu7V5Khm6WThbK5EZd4UfLGP1+ke13S7/AuzBL32FQZ+jQwMrIAXsdh3Ur ecbYcqnDhli5a9BH7bV9FRRefMLuyOc17DiYxS1+FVefacLH9MjYbERZzC8LW2rAS7cuKp 9FijGfp5jKVZrVRSqJjdFXuRZHEI1ts= Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by ams.source.kernel.org (Postfix) with ESMTP id 445CDAE2CBA; Thu, 29 Aug 2024 17:42:33 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 0D37AC4CEC1; Thu, 29 Aug 2024 17:42:38 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1724953358; bh=wR1R0e1Zi9YKCuyt9M3VgHLYSWK2su/QlRrjtoWcppc=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=urHzLm72UFR1B7nmShcEOGVpnFSl6PfYIqK/V3jPtWPlpy89zKCsy7GLfVrcobWIz v+SiX1zosX/t0XpzBqYd3GWtGMK/EIya1ThmeOfj0M6glnTzk7+kAkNx8RSOaj40uK 8JUhVdJ6aQXkI9PMwpCar36PnUqK9xO1DZhGZJu+Rckc+hzbbibh0u1qrhDyl9BUa9 yN0psa9zIYwkb4dM9OHFRbabe+b0TnxKHuBpFHUb0sfHJZWXtnEcLzRcozVjQ8LIVY dCkTmULMjQmi4oDOxE85M6i0Qapac8t3NfxCI+ZYHk3gqYu9A/PQQKx4N5uvzOE7dm +y1Dl1GOIDpHA== From: Andrii Nakryiko To: bpf@vger.kernel.org Cc: linux-mm@kvack.org, akpm@linux-foundation.org, adobriyan@gmail.com, shakeel.butt@linux.dev, hannes@cmpxchg.org, ak@linux.intel.com, osandov@osandov.com, song@kernel.org, jannh@google.com, linux-fsdevel@vger.kernel.org, willy@infradead.org, Andrii Nakryiko , stable@vger.kernel.org, Eduard Zingerman Subject: [PATCH v7 bpf-next 01/10] lib/buildid: harden build ID parsing logic Date: Thu, 29 Aug 2024 10:42:23 -0700 Message-ID: <20240829174232.3133883-2-andrii@kernel.org> X-Mailer: git-send-email 2.43.5 In-Reply-To: <20240829174232.3133883-1-andrii@kernel.org> References: <20240829174232.3133883-1-andrii@kernel.org> MIME-Version: 1.0 X-Stat-Signature: g9ikpf1epa9xzdr1awmurpxygpz6uoo3 X-Rspamd-Queue-Id: 5838618000B X-Rspam-User: X-Rspamd-Server: rspam10 X-HE-Tag: 1724953360-366640 X-HE-Meta: U2FsdGVkX1+CyikupURUCpCSyU/WPDzvTEdKadxsBlW/oza3/1piYvqt92t5ntzy3OIFVRcW5iCW/1zBwZh0KRp60lZ7YUTIadCzhZq2ktbYNwv+HlvCYLy5dMYv005nInj1wYJoZPXAtqWJlSbIFjh7G++iw5b85WVzMGzZ/HXGRSG62OexUr+D/S1I7H9bOm4+lTIgRbwegV8XCuKEC2xGpag+azVsy97KXnJGhSJCNnwj07zXk1NI+qLrqx9CKDZsXn7sp6eUGKPPJURMFceXceyf8kriknt7tdanWlleqRlbDljujtpjNArW4kmNfhB7es+UmBWcVWCATlM9AoMjwkrvlt9D83uNfUa+WBhhPOgDll2xBSgou0krSz+W+O4Nlcm3UMLYSR+pmDZ/9iZhIq+nS6Sluy1DTxW6UndDl13oR50jFNZW0kd0zCicnpRpk3o7W37GqFIZ8PPJ3E7fYCmW8sVx6VaUoInxMoIqv8I+1ivMGFyW7vATUM83/jJFUbe2WqmIBar9N9NNom7jWnPvnwCsYQzSNaXigdsLkDX5IFpQb0jG9tiQ7++w525v02lSCyaKHqfr3H8OpSRG/cOhHPkbpzru+fwh3XQPgW9t78LWIuONqYRfq48G82MsZksiyxSANtlAUx+/xT1IjbBkYAfb5tn1B3jlr8kR9pwbfw7sNteW143cb4OFEZGVFgh2mlSQ4qPA5hk8aRqU4OgwKBU2NWRcxZEQjHgbPq27RwAB/O583i7dLhrS8K/FhiLqp1oz7pA5PtuuOMEUogxOpOQnixhf3gqqJGoUQG125d9ie9IPX0DoSHkjl6U81dXZgPjclkSlvEU+SWsOFmXQ47Hw51LG5LC8Rgt6OJEd8qd7Y0a6+ZR4Xz3VeeIaPaSJeOpi5ImzqAtptxMkiufIAzvt6rLHyA76wA6dnSVyctd6j8Ua3z+rsF57v6rlDNsZHeBic/0hf5+ 9muLDkbV ptpf5GXRm2CAFG1GtNK3OmKee8iIV/o4A+bmTZF0gNxJ1iIzehodvQZQL/fMzTvaOwNLuBpc6z+AasfFMAxVsdmN95/dUMt8u2a06GnrLSkme2x2X/Kt09V+eTTPR0FQ+SmYgXcUm6yAr3Xnm8x/pVtb2PGf3TSSqnBSpkn3oty/RWH89DK6YWkUu4uQ5wHV1RFrkGI3TQVvWF5/O/La+z1/nYcbMQiRTXJG6+g8GEa4rxHl7PpZLb7RlRmKxMe07n2KwNsVxBh9Md9V8SRgrPqkyQ8bXHNqgERPWjkWOr0cQN78= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Harden build ID parsing logic, adding explicit READ_ONCE() where it's important to have a consistent value read and validated just once. Also, as pointed out by Andi Kleen, we need to make sure that entire ELF note is within a page bounds, so move the overflow check up and add an extra note_size boundaries validation. Fixes tag below points to the code that moved this code into lib/buildid.c, and then subsequently was used in perf subsystem, making this code exposed to perf_event_open() users in v5.12+. Cc: stable@vger.kernel.org Reviewed-by: Eduard Zingerman Reviewed-by: Jann Horn Suggested-by: Andi Kleen Fixes: bd7525dacd7e ("bpf: Move stack_map_get_build_id into lib") Signed-off-by: Andrii Nakryiko Signed-off-by: Jiri Olsa Acked-by: Andrii Nakryiko --- lib/buildid.c | 76 +++++++++++++++++++++++++++++---------------------- 1 file changed, 44 insertions(+), 32 deletions(-) diff --git a/lib/buildid.c b/lib/buildid.c index e02b5507418b..26007cc99a38 100644 --- a/lib/buildid.c +++ b/lib/buildid.c @@ -18,31 +18,37 @@ static int parse_build_id_buf(unsigned char *build_id, const void *note_start, Elf32_Word note_size) { - Elf32_Word note_offs = 0, new_offs; - - while (note_offs + sizeof(Elf32_Nhdr) < note_size) { - Elf32_Nhdr *nhdr = (Elf32_Nhdr *)(note_start + note_offs); + const char note_name[] = "GNU"; + const size_t note_name_sz = sizeof(note_name); + u64 note_off = 0, new_off, name_sz, desc_sz; + const char *data; + + while (note_off + sizeof(Elf32_Nhdr) < note_size && + note_off + sizeof(Elf32_Nhdr) > note_off /* overflow */) { + Elf32_Nhdr *nhdr = (Elf32_Nhdr *)(note_start + note_off); + + name_sz = READ_ONCE(nhdr->n_namesz); + desc_sz = READ_ONCE(nhdr->n_descsz); + + new_off = note_off + sizeof(Elf32_Nhdr); + if (check_add_overflow(new_off, ALIGN(name_sz, 4), &new_off) || + check_add_overflow(new_off, ALIGN(desc_sz, 4), &new_off) || + new_off > note_size) + break; if (nhdr->n_type == BUILD_ID && - nhdr->n_namesz == sizeof("GNU") && - !strcmp((char *)(nhdr + 1), "GNU") && - nhdr->n_descsz > 0 && - nhdr->n_descsz <= BUILD_ID_SIZE_MAX) { - memcpy(build_id, - note_start + note_offs + - ALIGN(sizeof("GNU"), 4) + sizeof(Elf32_Nhdr), - nhdr->n_descsz); - memset(build_id + nhdr->n_descsz, 0, - BUILD_ID_SIZE_MAX - nhdr->n_descsz); + name_sz == note_name_sz && + memcmp(nhdr + 1, note_name, note_name_sz) == 0 && + desc_sz > 0 && desc_sz <= BUILD_ID_SIZE_MAX) { + data = note_start + note_off + ALIGN(note_name_sz, 4); + memcpy(build_id, data, desc_sz); + memset(build_id + desc_sz, 0, BUILD_ID_SIZE_MAX - desc_sz); if (size) - *size = nhdr->n_descsz; + *size = desc_sz; return 0; } - new_offs = note_offs + sizeof(Elf32_Nhdr) + - ALIGN(nhdr->n_namesz, 4) + ALIGN(nhdr->n_descsz, 4); - if (new_offs <= note_offs) /* overflow */ - break; - note_offs = new_offs; + + note_off = new_off; } return -EINVAL; @@ -71,7 +77,7 @@ static int get_build_id_32(const void *page_addr, unsigned char *build_id, { Elf32_Ehdr *ehdr = (Elf32_Ehdr *)page_addr; Elf32_Phdr *phdr; - int i; + __u32 i, phnum; /* * FIXME @@ -80,18 +86,19 @@ static int get_build_id_32(const void *page_addr, unsigned char *build_id, */ if (ehdr->e_phoff != sizeof(Elf32_Ehdr)) return -EINVAL; + + phnum = READ_ONCE(ehdr->e_phnum); /* only supports phdr that fits in one page */ - if (ehdr->e_phnum > - (PAGE_SIZE - sizeof(Elf32_Ehdr)) / sizeof(Elf32_Phdr)) + if (phnum > (PAGE_SIZE - sizeof(Elf32_Ehdr)) / sizeof(Elf32_Phdr)) return -EINVAL; phdr = (Elf32_Phdr *)(page_addr + sizeof(Elf32_Ehdr)); - for (i = 0; i < ehdr->e_phnum; ++i) { + for (i = 0; i < phnum; ++i) { if (phdr[i].p_type == PT_NOTE && !parse_build_id(page_addr, build_id, size, - page_addr + phdr[i].p_offset, - phdr[i].p_filesz)) + page_addr + READ_ONCE(phdr[i].p_offset), + READ_ONCE(phdr[i].p_filesz))) return 0; } return -EINVAL; @@ -103,7 +110,7 @@ static int get_build_id_64(const void *page_addr, unsigned char *build_id, { Elf64_Ehdr *ehdr = (Elf64_Ehdr *)page_addr; Elf64_Phdr *phdr; - int i; + __u32 i, phnum; /* * FIXME @@ -112,18 +119,19 @@ static int get_build_id_64(const void *page_addr, unsigned char *build_id, */ if (ehdr->e_phoff != sizeof(Elf64_Ehdr)) return -EINVAL; + + phnum = READ_ONCE(ehdr->e_phnum); /* only supports phdr that fits in one page */ - if (ehdr->e_phnum > - (PAGE_SIZE - sizeof(Elf64_Ehdr)) / sizeof(Elf64_Phdr)) + if (phnum > (PAGE_SIZE - sizeof(Elf64_Ehdr)) / sizeof(Elf64_Phdr)) return -EINVAL; phdr = (Elf64_Phdr *)(page_addr + sizeof(Elf64_Ehdr)); - for (i = 0; i < ehdr->e_phnum; ++i) { + for (i = 0; i < phnum; ++i) { if (phdr[i].p_type == PT_NOTE && !parse_build_id(page_addr, build_id, size, - page_addr + phdr[i].p_offset, - phdr[i].p_filesz)) + page_addr + READ_ONCE(phdr[i].p_offset), + READ_ONCE(phdr[i].p_filesz))) return 0; } return -EINVAL; @@ -152,6 +160,10 @@ int build_id_parse(struct vm_area_struct *vma, unsigned char *build_id, page = find_get_page(vma->vm_file->f_mapping, 0); if (!page) return -EFAULT; /* page not mapped */ + if (!PageUptodate(page)) { + put_page(page); + return -EFAULT; + } ret = -EINVAL; page_addr = kmap_local_page(page); From patchwork Thu Aug 29 17:42:24 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrii Nakryiko X-Patchwork-Id: 13783494 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7C7E6C8303E for ; Thu, 29 Aug 2024 17:42:51 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id DFD5E6B00AF; Thu, 29 Aug 2024 13:42:50 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id D7DA36B00B0; Thu, 29 Aug 2024 13:42:50 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B38A86B00B1; Thu, 29 Aug 2024 13:42:50 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 872BF6B00AF for ; Thu, 29 Aug 2024 13:42:50 -0400 (EDT) Received: from smtpin07.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 4156F141120 for ; Thu, 29 Aug 2024 17:42:50 +0000 (UTC) X-FDA: 82506003300.07.1CC7DE9 Received: from sin.source.kernel.org (sin.source.kernel.org [145.40.73.55]) by imf29.hostedemail.com (Postfix) with ESMTP id F208B12000A for ; Thu, 29 Aug 2024 17:42:46 +0000 (UTC) Authentication-Results: imf29.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=OMhk+aEB; dmarc=pass (policy=quarantine) header.from=kernel.org; spf=pass (imf29.hostedemail.com: domain of andrii@kernel.org designates 145.40.73.55 as permitted sender) smtp.mailfrom=andrii@kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1724953346; a=rsa-sha256; cv=none; b=EQSt2G11knQC+uCkA54dTKyeMbVHkdyoRGrZSd3NBRh+yWQ+iKXGuCoaSy1fP+mJlHDsxC +bFX2hdd4JNkycb021eTzYslzW8ex78QCi+XBPqpoQbaG2QG/r6b7y5OxDL/hr28K/m2nJ 2PeLIZL0uYjVlrGLr1lEbjcGP+ZzkEA= ARC-Authentication-Results: i=1; imf29.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=OMhk+aEB; dmarc=pass (policy=quarantine) header.from=kernel.org; spf=pass (imf29.hostedemail.com: domain of andrii@kernel.org designates 145.40.73.55 as permitted sender) smtp.mailfrom=andrii@kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1724953346; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=w6Wm7dwmzAqOVNMjjtYly260d+BRg60go05/44VJcBM=; b=GPjYzA7R+rQ6ibZAaJQbVqTzFZZyV7nxsdacaKIHJKIIJfbVIPADCvKNv78X99RmeHLCtC r5fi0eP8WIQjuefTICuF9bnpy/YIelpXMQMLg75cd6vP0irpIlTzIzcH8Gge7JizgzuzCh dOG0UKFEun6AJy0khAcq9+TV0WDQ4wc= Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by sin.source.kernel.org (Postfix) with ESMTP id 4F6DCCE1C2F; Thu, 29 Aug 2024 17:42:42 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 4A68CC4CEC1; Thu, 29 Aug 2024 17:42:41 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1724953361; bh=Z1WbIc+u/F3T63+9FQlx6/STGu7yunPqQEuluURvw7s=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=OMhk+aEBKgMD3dOciiLBq0tvfvkASQSTgVydC2iWvRlfkobSqWw56M8ZXBjGUvunW gX3Q4hOBnW1bw3tsjz9dUtJ5Dv6ZtajqVp3yPPlQLa75eRr1CVr4ROQC+xF6O6ORkp EgpWDO97voSIwkAgGa6UDGOiejrLmqfh64Tz2ER47SclBLINxYUkyW+ze7LTyRgVsK +Uj0P5mPc7Qjoi7iRX2PvwH7gMYMY85xbZASXv04RIE2wYevWpJ47SLEeGB62v7/m7 lg9/90VqyGsueUC1aB65/zQqvIvbwpTKCBg91k4G2/v5mBG7G3VsSadZkq9y69cFvf WhEGF+H1BOC5g== From: Andrii Nakryiko To: bpf@vger.kernel.org Cc: linux-mm@kvack.org, akpm@linux-foundation.org, adobriyan@gmail.com, shakeel.butt@linux.dev, hannes@cmpxchg.org, ak@linux.intel.com, osandov@osandov.com, song@kernel.org, jannh@google.com, linux-fsdevel@vger.kernel.org, willy@infradead.org, Andrii Nakryiko , Eduard Zingerman Subject: [PATCH v7 bpf-next 02/10] lib/buildid: add single folio-based file reader abstraction Date: Thu, 29 Aug 2024 10:42:24 -0700 Message-ID: <20240829174232.3133883-3-andrii@kernel.org> X-Mailer: git-send-email 2.43.5 In-Reply-To: <20240829174232.3133883-1-andrii@kernel.org> References: <20240829174232.3133883-1-andrii@kernel.org> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Queue-Id: F208B12000A X-Rspamd-Server: rspam01 X-Stat-Signature: hgd6dhzhmnzgo8xrg8bgb8iu1q8mcqfp X-HE-Tag: 1724953366-396740 X-HE-Meta: U2FsdGVkX19H7cq2odRsKIoJzhtok4c5DMEx1gwkpIZJbiWGnbpzL2+p4dDi+E9ADJDXBitqLUAH6rV1PSf5u1FLkZsVCHSe6eKY26zauEs2c3IzTQxHdKNiXnjQLrtdEjJ3nZVxP2mej76RsPWFdy6bsfo6UjgGGeOOmCIrrvbLKH/QeJSw1aCq1sljRjWgtQu8+STTyj4CjXQ07nKPrRhK871OWlyRE9gzSjDq66msvLA8Gzgfd7/RBDrz/oxK5uj/utaIvaDe7+vduYya7IkKLqq24u4PO5WW/o9VAOmR3KLHJRuqoPSojx49S7NzKwyVxk8Wi9uGIt9CDVI8SzyjhEBXzAhEGIeuttD7GSFCSfsolmq7YEC3o0BwW9DBCO9SzaqYP+ezJ4r55UmSDTxH2B03u4bbuPUJciLu1d2rQ7QpJiK+QCWQVFOar4sin0MyE+ejdstDrtoH8zzh4InQ6qlatmwvD1mUtXgw5usq+61RgJ31d0I4qzonC3rgXt1r1fcnS7sbDLo7jXmSKpgngV8Chi+M/zJH8K1uuGYuCydPooE/AX1A3yuHyi6IdiImEH3EFT30mg6/9n78urRuHS0PhGTgSiihVHFUtWvQLUw8WNqEXp65c/oAfF0FcHzWBaPOq+4MDkJBQ0dAbJjM64exR6Vyhgl8ZdliR2MEIERSZX2YfjZVVC7A3qhhD1FZktAvtNJdnyfrTaWm/8bVXuhcQ83D0uPCFXZv1wjfel07P3S3Uz5HfcSRVJ1KCQ1N87+w2Rh77neS8qMSoXipPNKc0wNgoahY9kv/ilUSOyKgrfKCt3NPgusc61nxiFdJHjoxws3LQAXPZK82c4A5QNqx/F3auuStjqZu6zCsr2/mJg+SQVgk0khRCSgzTmfqIiuPOVk2sph6EYd0tEHyW7gCVPnQ2d27c4ocRASQa6w1Y5oTb3WRX9f5dRhP/MnsgHpJfEPBu7eTPXS /Uz7LnzM cwPbcd4oLKgR2XoswBsiasXFdUbpMIDIA++aXHLFjcVjJ+SbIsFAAqt05qyMj+gecWjo9NLk8btTp0NvwV1KP88qWh7r5hAfI6BxG8LxZScL0CDMunmyQ/wR9wJ1PP3jAUuWsHcOwTwvn4dFl+4h7iwI4JyIZv7MB2mxAZMZqbSW+Dwo6bd2nYp4Bqkq+4/jAuTI71RzSviMUiokYEvqLp+rDAdopiRLVaKexVag0R17rhXmOMiXTUWLJye6tDzKvXqkL X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Add freader abstraction that transparently manages fetching and local mapping of the underlying file page(s) and provides a simple direct data access interface. freader_fetch() is the only and single interface necessary. It accepts file offset and desired number of bytes that should be accessed, and will return a kernel mapped pointer that caller can use to dereference data up to requested size. Requested size can't be bigger than the size of the extra buffer provided during initialization (because, worst case, all requested data has to be copied into it, so it's better to flag wrongly sized buffer unconditionally, regardless if requested data range is crossing page boundaries or not). If folio is not paged in, or some of the conditions are not satisfied, NULL is returned and more detailed error code can be accessed through freader->err field. This approach makes the usage of freader_fetch() cleaner. To accommodate accessing file data that crosses folio boundaries, user has to provide an extra buffer that will be used to make a local copy, if necessary. This is done to maintain a simple linear pointer data access interface. We switch existing build ID parsing logic to it, without changing or lifting any of the existing constraints, yet. This will be done separately. Given existing code was written with the assumption that it's always working with a single (first) page of the underlying ELF file, logic passes direct pointers around, which doesn't really work well with freader approach and would be limiting when removing the single page (folio) limitation. So we adjust all the logic to work in terms of file offsets. There is also a memory buffer-based version (freader_init_from_mem()) for cases when desired data is already available in kernel memory. This is used for parsing vmlinux's own build ID note. In this mode assumption is that provided data starts at "file offset" zero, which works great when parsing ELF notes sections, as all the parsing logic is relative to note section's start. Reviewed-by: Eduard Zingerman Reviewed-by: Shakeel Butt Signed-off-by: Andrii Nakryiko --- lib/buildid.c | 263 ++++++++++++++++++++++++++++++++++++++++---------- 1 file changed, 210 insertions(+), 53 deletions(-) diff --git a/lib/buildid.c b/lib/buildid.c index 26007cc99a38..bfe00b66b1e8 100644 --- a/lib/buildid.c +++ b/lib/buildid.c @@ -8,24 +8,155 @@ #define BUILD_ID 3 +struct freader { + void *buf; + u32 buf_sz; + int err; + union { + struct { + struct address_space *mapping; + struct folio *folio; + void *addr; + loff_t folio_off; + }; + struct { + const char *data; + u64 data_sz; + }; + }; +}; + +static void freader_init_from_file(struct freader *r, void *buf, u32 buf_sz, + struct address_space *mapping) +{ + memset(r, 0, sizeof(*r)); + r->buf = buf; + r->buf_sz = buf_sz; + r->mapping = mapping; +} + +static void freader_init_from_mem(struct freader *r, const char *data, u64 data_sz) +{ + memset(r, 0, sizeof(*r)); + r->data = data; + r->data_sz = data_sz; +} + +static void freader_put_folio(struct freader *r) +{ + if (!r->folio) + return; + kunmap_local(r->addr); + folio_put(r->folio); + r->folio = NULL; +} + +static int freader_get_folio(struct freader *r, loff_t file_off) +{ + /* check if we can just reuse current folio */ + if (r->folio && file_off >= r->folio_off && + file_off < r->folio_off + folio_size(r->folio)) + return 0; + + freader_put_folio(r); + + r->folio = filemap_get_folio(r->mapping, file_off >> PAGE_SHIFT); + if (IS_ERR(r->folio) || !folio_test_uptodate(r->folio)) { + if (!IS_ERR(r->folio)) + folio_put(r->folio); + r->folio = NULL; + return -EFAULT; + } + + r->folio_off = folio_pos(r->folio); + r->addr = kmap_local_folio(r->folio, 0); + + return 0; +} + +static const void *freader_fetch(struct freader *r, loff_t file_off, size_t sz) +{ + size_t folio_sz; + + /* provided internal temporary buffer should be sized correctly */ + if (WARN_ON(r->buf && sz > r->buf_sz)) { + r->err = -E2BIG; + return NULL; + } + + if (unlikely(file_off + sz < file_off)) { + r->err = -EOVERFLOW; + return NULL; + } + + /* working with memory buffer is much more straightforward */ + if (!r->buf) { + if (file_off + sz > r->data_sz) { + r->err = -ERANGE; + return NULL; + } + return r->data + file_off; + } + + /* fetch or reuse folio for given file offset */ + r->err = freader_get_folio(r, file_off); + if (r->err) + return NULL; + + /* if requested data is crossing folio boundaries, we have to copy + * everything into our local buffer to keep a simple linear memory + * access interface + */ + folio_sz = folio_size(r->folio); + if (file_off + sz > r->folio_off + folio_sz) { + int part_sz = r->folio_off + folio_sz - file_off; + + /* copy the part that resides in the current folio */ + memcpy(r->buf, r->addr + (file_off - r->folio_off), part_sz); + + /* fetch next folio */ + r->err = freader_get_folio(r, r->folio_off + folio_sz); + if (r->err) + return NULL; + + /* copy the rest of requested data */ + memcpy(r->buf + part_sz, r->addr, sz - part_sz); + + return r->buf; + } + + /* if data fits in a single folio, just return direct pointer */ + return r->addr + (file_off - r->folio_off); +} + +static void freader_cleanup(struct freader *r) +{ + if (!r->buf) + return; /* non-file-backed mode */ + + freader_put_folio(r); +} + /* * Parse build id from the note segment. This logic can be shared between * 32-bit and 64-bit system, because Elf32_Nhdr and Elf64_Nhdr are * identical. */ -static int parse_build_id_buf(unsigned char *build_id, - __u32 *size, - const void *note_start, - Elf32_Word note_size) +static int parse_build_id_buf(struct freader *r, + unsigned char *build_id, __u32 *size, + loff_t note_off, Elf32_Word note_size) { const char note_name[] = "GNU"; const size_t note_name_sz = sizeof(note_name); - u64 note_off = 0, new_off, name_sz, desc_sz; + u32 build_id_off, new_off, note_end, name_sz, desc_sz; + const Elf32_Nhdr *nhdr; const char *data; - while (note_off + sizeof(Elf32_Nhdr) < note_size && - note_off + sizeof(Elf32_Nhdr) > note_off /* overflow */) { - Elf32_Nhdr *nhdr = (Elf32_Nhdr *)(note_start + note_off); + note_end = note_off + note_size; + while (note_end - note_off > sizeof(Elf32_Nhdr) + note_name_sz) { + nhdr = freader_fetch(r, note_off, sizeof(Elf32_Nhdr) + note_name_sz); + if (!nhdr) + return r->err; name_sz = READ_ONCE(nhdr->n_namesz); desc_sz = READ_ONCE(nhdr->n_descsz); @@ -33,14 +164,20 @@ static int parse_build_id_buf(unsigned char *build_id, new_off = note_off + sizeof(Elf32_Nhdr); if (check_add_overflow(new_off, ALIGN(name_sz, 4), &new_off) || check_add_overflow(new_off, ALIGN(desc_sz, 4), &new_off) || - new_off > note_size) + new_off > note_end) break; if (nhdr->n_type == BUILD_ID && name_sz == note_name_sz && memcmp(nhdr + 1, note_name, note_name_sz) == 0 && desc_sz > 0 && desc_sz <= BUILD_ID_SIZE_MAX) { - data = note_start + note_off + ALIGN(note_name_sz, 4); + build_id_off = note_off + sizeof(Elf32_Nhdr) + ALIGN(note_name_sz, 4); + + /* freader_fetch() will invalidate nhdr pointer */ + data = freader_fetch(r, build_id_off, desc_sz); + if (!data) + return r->err; + memcpy(build_id, data, desc_sz); memset(build_id + desc_sz, 0, BUILD_ID_SIZE_MAX - desc_sz); if (size) @@ -54,30 +191,33 @@ static int parse_build_id_buf(unsigned char *build_id, return -EINVAL; } -static inline int parse_build_id(const void *page_addr, +static inline int parse_build_id(struct freader *r, unsigned char *build_id, __u32 *size, - const void *note_start, + loff_t note_start_off, Elf32_Word note_size) { /* check for overflow */ - if (note_start < page_addr || note_start + note_size < note_start) + if (note_start_off + note_size < note_start_off) return -EINVAL; /* only supports note that fits in the first page */ - if (note_start + note_size > page_addr + PAGE_SIZE) + if (note_start_off + note_size > PAGE_SIZE) return -EINVAL; - return parse_build_id_buf(build_id, size, note_start, note_size); + return parse_build_id_buf(r, build_id, size, note_start_off, note_size); } /* Parse build ID from 32-bit ELF */ -static int get_build_id_32(const void *page_addr, unsigned char *build_id, - __u32 *size) +static int get_build_id_32(struct freader *r, unsigned char *build_id, __u32 *size) { - Elf32_Ehdr *ehdr = (Elf32_Ehdr *)page_addr; - Elf32_Phdr *phdr; - __u32 i, phnum; + const Elf32_Ehdr *ehdr; + const Elf32_Phdr *phdr; + __u32 phnum, i; + + ehdr = freader_fetch(r, 0, sizeof(Elf32_Ehdr)); + if (!ehdr) + return r->err; /* * FIXME @@ -87,30 +227,35 @@ static int get_build_id_32(const void *page_addr, unsigned char *build_id, if (ehdr->e_phoff != sizeof(Elf32_Ehdr)) return -EINVAL; + /* subsequent freader_fetch() calls invalidate pointers, so remember locally */ phnum = READ_ONCE(ehdr->e_phnum); /* only supports phdr that fits in one page */ if (phnum > (PAGE_SIZE - sizeof(Elf32_Ehdr)) / sizeof(Elf32_Phdr)) return -EINVAL; - phdr = (Elf32_Phdr *)(page_addr + sizeof(Elf32_Ehdr)); - for (i = 0; i < phnum; ++i) { - if (phdr[i].p_type == PT_NOTE && - !parse_build_id(page_addr, build_id, size, - page_addr + READ_ONCE(phdr[i].p_offset), - READ_ONCE(phdr[i].p_filesz))) + phdr = freader_fetch(r, i * sizeof(Elf32_Phdr), sizeof(Elf32_Phdr)); + if (!phdr) + return r->err; + + if (phdr->p_type == PT_NOTE && + !parse_build_id(r, build_id, size, READ_ONCE(phdr->p_offset), + READ_ONCE(phdr->p_filesz))) return 0; } return -EINVAL; } /* Parse build ID from 64-bit ELF */ -static int get_build_id_64(const void *page_addr, unsigned char *build_id, - __u32 *size) +static int get_build_id_64(struct freader *r, unsigned char *build_id, __u32 *size) { - Elf64_Ehdr *ehdr = (Elf64_Ehdr *)page_addr; - Elf64_Phdr *phdr; - __u32 i, phnum; + const Elf64_Ehdr *ehdr; + const Elf64_Phdr *phdr; + __u32 phnum, i; + + ehdr = freader_fetch(r, 0, sizeof(Elf64_Ehdr)); + if (!ehdr) + return r->err; /* * FIXME @@ -120,23 +265,29 @@ static int get_build_id_64(const void *page_addr, unsigned char *build_id, if (ehdr->e_phoff != sizeof(Elf64_Ehdr)) return -EINVAL; + /* subsequent freader_fetch() calls invalidate pointers, so remember locally */ phnum = READ_ONCE(ehdr->e_phnum); /* only supports phdr that fits in one page */ if (phnum > (PAGE_SIZE - sizeof(Elf64_Ehdr)) / sizeof(Elf64_Phdr)) return -EINVAL; - phdr = (Elf64_Phdr *)(page_addr + sizeof(Elf64_Ehdr)); - for (i = 0; i < phnum; ++i) { - if (phdr[i].p_type == PT_NOTE && - !parse_build_id(page_addr, build_id, size, - page_addr + READ_ONCE(phdr[i].p_offset), - READ_ONCE(phdr[i].p_filesz))) + phdr = freader_fetch(r, i * sizeof(Elf64_Phdr), sizeof(Elf64_Phdr)); + if (!phdr) + return r->err; + + if (phdr->p_type == PT_NOTE && + !parse_build_id(r, build_id, size, READ_ONCE(phdr->p_offset), + READ_ONCE(phdr->p_filesz))) return 0; } + return -EINVAL; } +/* enough for Elf64_Ehdr, Elf64_Phdr, and all the smaller requests */ +#define MAX_FREADER_BUF_SZ 64 + /* * Parse build ID of ELF file mapped to vma * @vma: vma object @@ -148,26 +299,25 @@ static int get_build_id_64(const void *page_addr, unsigned char *build_id, int build_id_parse(struct vm_area_struct *vma, unsigned char *build_id, __u32 *size) { - Elf32_Ehdr *ehdr; - struct page *page; - void *page_addr; + const Elf32_Ehdr *ehdr; + struct freader r; + char buf[MAX_FREADER_BUF_SZ]; int ret; /* only works for page backed storage */ if (!vma->vm_file) return -EINVAL; - page = find_get_page(vma->vm_file->f_mapping, 0); - if (!page) - return -EFAULT; /* page not mapped */ - if (!PageUptodate(page)) { - put_page(page); - return -EFAULT; + freader_init_from_file(&r, buf, sizeof(buf), vma->vm_file->f_mapping); + + /* fetch first 18 bytes of ELF header for checks */ + ehdr = freader_fetch(&r, 0, offsetofend(Elf32_Ehdr, e_type)); + if (!ehdr) { + ret = r.err; + goto out; } ret = -EINVAL; - page_addr = kmap_local_page(page); - ehdr = (Elf32_Ehdr *)page_addr; /* compare magic x7f "ELF" */ if (memcmp(ehdr->e_ident, ELFMAG, SELFMAG) != 0) @@ -178,12 +328,11 @@ int build_id_parse(struct vm_area_struct *vma, unsigned char *build_id, goto out; if (ehdr->e_ident[EI_CLASS] == ELFCLASS32) - ret = get_build_id_32(page_addr, build_id, size); + ret = get_build_id_32(&r, build_id, size); else if (ehdr->e_ident[EI_CLASS] == ELFCLASS64) - ret = get_build_id_64(page_addr, build_id, size); + ret = get_build_id_64(&r, build_id, size); out: - kunmap_local(page_addr); - put_page(page); + freader_cleanup(&r); return ret; } @@ -197,7 +346,15 @@ int build_id_parse(struct vm_area_struct *vma, unsigned char *build_id, */ int build_id_parse_buf(const void *buf, unsigned char *build_id, u32 buf_size) { - return parse_build_id_buf(build_id, NULL, buf, buf_size); + struct freader r; + int err; + + freader_init_from_mem(&r, buf, buf_size); + + err = parse_build_id(&r, build_id, NULL, 0, buf_size); + + freader_cleanup(&r); + return err; } #if IS_ENABLED(CONFIG_STACKTRACE_BUILD_ID) || IS_ENABLED(CONFIG_VMCORE_INFO) From patchwork Thu Aug 29 17:42:25 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrii Nakryiko X-Patchwork-Id: 13783493 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id A6216C87FCD for ; Thu, 29 Aug 2024 17:42:49 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 345326B00AB; Thu, 29 Aug 2024 13:42:49 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 2CDF76B00AF; Thu, 29 Aug 2024 13:42:49 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 125E96B00B0; Thu, 29 Aug 2024 13:42:49 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id DA84D6B00AB for ; Thu, 29 Aug 2024 13:42:48 -0400 (EDT) Received: from smtpin25.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 7F5C5C065E for ; Thu, 29 Aug 2024 17:42:48 +0000 (UTC) X-FDA: 82506003216.25.6B241CD Received: from ams.source.kernel.org (ams.source.kernel.org [145.40.68.75]) by imf06.hostedemail.com (Postfix) with ESMTP id ABB62180021 for ; Thu, 29 Aug 2024 17:42:46 +0000 (UTC) Authentication-Results: imf06.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=h9S7C4y+; dmarc=pass (policy=quarantine) header.from=kernel.org; spf=pass (imf06.hostedemail.com: domain of andrii@kernel.org designates 145.40.68.75 as permitted sender) smtp.mailfrom=andrii@kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1724953267; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=+slAdB4wS56GMUwmGYfitxITn7XvMmDTDZNI1FJGq7k=; b=ITbs/00luj1QZ2bBKdjUzLFgVa5JZK+ntdzzHvJiWh/Kh6Gn+ubM5SOcKr+wOs0NN3plp2 A6vYneDuWYIG81vbmAl3jX+9PQpfQlCT69NHl0M+QC10r4ybY++ONRYIEbKGjj8LeMxlTg uk5e84kAbKnJnkH1bfW70o2GPyKB3h8= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1724953267; a=rsa-sha256; cv=none; b=B/dSQGxaNMxMultsrAn15ma2GbQtp8jgLEhro4Sr//oFDReVKYTZo9A8nj3ePyx5+HBiQp 1eFYUlDN0cM3nC5u3AU5RM1BjF5T5Ha+DytxT7bz4Iiqx5Vy3izhnBC+uvLbIZ2JS8nBfc giTE5p+xGAm/RvZUmuzdtrVu+KThk38= ARC-Authentication-Results: i=1; imf06.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=h9S7C4y+; dmarc=pass (policy=quarantine) header.from=kernel.org; spf=pass (imf06.hostedemail.com: domain of andrii@kernel.org designates 145.40.68.75 as permitted sender) smtp.mailfrom=andrii@kernel.org Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by ams.source.kernel.org (Postfix) with ESMTP id BC98CAE2CD0; Thu, 29 Aug 2024 17:42:39 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 8C3DCC4CEC1; Thu, 29 Aug 2024 17:42:44 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1724953364; bh=Rp6otyrq1s70AfuJSm1rv8h2cgFz+vWWyi+WpWNJo0k=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=h9S7C4y+C06tQyD+Sl51neSMfzNbO3H7+XqTB+9IpLWNA5lP+7o39gvHMaHYMTSM1 NOme1U8VxqRTfLT9XS/9uBRx6l7NB72LK9SCN1+L+uvRjNIyIAZAFWeULC4Hz2BJm7 qfp5elwqfey2Rzwir5hsz1BcEG4n4MeOej78lpi6juzXR7URC5jNrQNEPXgOQ7fhif rTKpnnCjGuyYxfLhZecA2brCZkio51MjpJbulavW8CtHl3fiwLCJxk1z/jqJgwJGEy UXSe7/zUDwVUrveZGzktpl0G1JOYrU7B+QGYubKYXXQyOdSc6vsex6cDe+5LPPE7cw ol9ipOaSf/sig== From: Andrii Nakryiko To: bpf@vger.kernel.org Cc: linux-mm@kvack.org, akpm@linux-foundation.org, adobriyan@gmail.com, shakeel.butt@linux.dev, hannes@cmpxchg.org, ak@linux.intel.com, osandov@osandov.com, song@kernel.org, jannh@google.com, linux-fsdevel@vger.kernel.org, willy@infradead.org, Andrii Nakryiko , Eduard Zingerman Subject: [PATCH v7 bpf-next 03/10] lib/buildid: take into account e_phoff when fetching program headers Date: Thu, 29 Aug 2024 10:42:25 -0700 Message-ID: <20240829174232.3133883-4-andrii@kernel.org> X-Mailer: git-send-email 2.43.5 In-Reply-To: <20240829174232.3133883-1-andrii@kernel.org> References: <20240829174232.3133883-1-andrii@kernel.org> MIME-Version: 1.0 X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: ABB62180021 X-Stat-Signature: jp1a58km6yu3k1oitsjnx77rxrh6df6c X-Rspam-User: X-HE-Tag: 1724953366-748236 X-HE-Meta: U2FsdGVkX1/BzZXt1n2FzoS7CPPVUfTkaRwjT8yXTDODmYG2t9zYtoW+DBzR8xOipaEBdmGoUxQ1JVVROoqLKFmhLk4L6z7q+Wox4dPA7rCeVpkr4MKsbsVJV9+tcp2vCP5aQRG8qmlo3u4GDzOfzVdpKcyDfJOuOw4cGE2v7wo/pJciM4QIaAvGZe72Ikifsa44OIKLS7qWu7ZZ5B2JiZUdOASWz/VFdHO6re0RjEl6rWrnYmhRU4kCMh1yQltW+HxQ31iIOqmqWMf9mrlPFU/+HKFIOSmUpU6wJ1/dKJqwkm1ljYf9SZED8lURyrGNMsnFb/TvWDWutkY4U/ExfQpgnNTglmZYk6d2ADFsmQX3hsPCJU9cVexvzoed7wgBv3gusLCuFt2Ny723jW2raARkMLO/XjaLmRJW5QkSf5j/rb+yDw/CWYXZaHUVf0cUCk8GwZ/toxxZlpoQHPj5lK4DXShzLJf8Ox8P61nfFSiCch6ZnluUKLZr4UcEplVr7uZ7NNopQNzfJcrNOyxrNTU7BIBoI5F83epbykPGZYVZD2kdPpKST5Y7QgdMrAGy8rDXmfQSiRk4BjfG7oXXr3x48EZOEHuRTrK48Fqq42ZnNwP6xP08aO8yHKH910ioVyDiLnxmqm9S+5yyOaVQ8Go99L+KGqk0urjgC/1fIyOijE5AltX0iugRixcUK7EAGMkD9/sDoyub97LtABAIuyYNhqTzQDuKJeSeLm91d/uzqEkOSs8Jct7g7IQA29x4sIjjWcVMBKiBuBXFzyM6r6C8iB6qM2Tuxpu25NYbZhlk6tQXrpyIQ2S9DSx4Y6YExuqo5bYiq2F0fpRcqQq47FSgzp6sIvbITkMPu5X1mmSVPSFbGYZICl6Y/dbuGFv3eJpvioRMuEElS/dUzauRmHLe2lgUSrly1Y4IVYV7+rZj52kMdOKk3sLGxIXzt3il32ldUcgyAMlkGHeuaxT ztJo+A9W B1Elc3oeG5Mb2XSyK6ULq2bniE/7Rw/d2CeQe4cOcD5ndmETZ2hHXD7Ei+U1CXXT8D1QyGrgQgF54fI5D7Vch8aWmk8UkL3QLUmV6h/phMylCrO5Ynv4tGpHk2ovsfmHvfltKu0eV1qCRd2Qpf67aE5X4Qt9CaEgiyK2yZ4LsscbVaxZyTW74u/h1PHlwL3V/oIUbi9HXP5iNtYal82MUA7m/jwf9f7mLg1sw/LlKCZEiUDJo8LgCIlAgvgzvNvYQfwtcJ0k7rei92t/V6BiGnTXBAGnZZOFEQM9G7sYQzXBFs3A= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Current code assumption is that program (segment) headers are following ELF header immediately. This is a common case, but is not guaranteed. So take into account e_phoff field of the ELF header when accessing program headers. Reviewed-by: Eduard Zingerman Reported-by: Alexey Dobriyan Signed-off-by: Andrii Nakryiko --- lib/buildid.c | 35 ++++++++++++++++------------------- 1 file changed, 16 insertions(+), 19 deletions(-) diff --git a/lib/buildid.c b/lib/buildid.c index bfe00b66b1e8..7fb08a1d98bd 100644 --- a/lib/buildid.c +++ b/lib/buildid.c @@ -213,28 +213,26 @@ static int get_build_id_32(struct freader *r, unsigned char *build_id, __u32 *si { const Elf32_Ehdr *ehdr; const Elf32_Phdr *phdr; - __u32 phnum, i; + __u32 phnum, phoff, i; ehdr = freader_fetch(r, 0, sizeof(Elf32_Ehdr)); if (!ehdr) return r->err; - /* - * FIXME - * Neither ELF spec nor ELF loader require that program headers - * start immediately after ELF header. - */ - if (ehdr->e_phoff != sizeof(Elf32_Ehdr)) - return -EINVAL; - /* subsequent freader_fetch() calls invalidate pointers, so remember locally */ phnum = READ_ONCE(ehdr->e_phnum); + phoff = READ_ONCE(ehdr->e_phoff); + /* only supports phdr that fits in one page */ if (phnum > (PAGE_SIZE - sizeof(Elf32_Ehdr)) / sizeof(Elf32_Phdr)) return -EINVAL; + /* check that phoff is not large enough to cause an overflow */ + if (phoff + phnum * sizeof(Elf32_Phdr) < phoff) + return -EINVAL; + for (i = 0; i < phnum; ++i) { - phdr = freader_fetch(r, i * sizeof(Elf32_Phdr), sizeof(Elf32_Phdr)); + phdr = freader_fetch(r, phoff + i * sizeof(Elf32_Phdr), sizeof(Elf32_Phdr)); if (!phdr) return r->err; @@ -252,27 +250,26 @@ static int get_build_id_64(struct freader *r, unsigned char *build_id, __u32 *si const Elf64_Ehdr *ehdr; const Elf64_Phdr *phdr; __u32 phnum, i; + __u64 phoff; ehdr = freader_fetch(r, 0, sizeof(Elf64_Ehdr)); if (!ehdr) return r->err; - /* - * FIXME - * Neither ELF spec nor ELF loader require that program headers - * start immediately after ELF header. - */ - if (ehdr->e_phoff != sizeof(Elf64_Ehdr)) - return -EINVAL; - /* subsequent freader_fetch() calls invalidate pointers, so remember locally */ phnum = READ_ONCE(ehdr->e_phnum); + phoff = READ_ONCE(ehdr->e_phoff); + /* only supports phdr that fits in one page */ if (phnum > (PAGE_SIZE - sizeof(Elf64_Ehdr)) / sizeof(Elf64_Phdr)) return -EINVAL; + /* check that phoff is not large enough to cause an overflow */ + if (phoff + phnum * sizeof(Elf64_Phdr) < phoff) + return -EINVAL; + for (i = 0; i < phnum; ++i) { - phdr = freader_fetch(r, i * sizeof(Elf64_Phdr), sizeof(Elf64_Phdr)); + phdr = freader_fetch(r, phoff + i * sizeof(Elf64_Phdr), sizeof(Elf64_Phdr)); if (!phdr) return r->err; From patchwork Thu Aug 29 17:42:26 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrii Nakryiko X-Patchwork-Id: 13783514 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id ED01EC87FCD for ; Thu, 29 Aug 2024 17:42:53 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 542926B00B1; Thu, 29 Aug 2024 13:42:52 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 4CA696B00B2; Thu, 29 Aug 2024 13:42:52 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 31D9F6B00B3; Thu, 29 Aug 2024 13:42:52 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 021AF6B00B1 for ; Thu, 29 Aug 2024 13:42:51 -0400 (EDT) Received: from smtpin24.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 9FCABA11C2 for ; Thu, 29 Aug 2024 17:42:51 +0000 (UTC) X-FDA: 82506003342.24.361DB27 Received: from ams.source.kernel.org (ams.source.kernel.org [145.40.68.75]) by imf28.hostedemail.com (Postfix) with ESMTP id E1D31C001C for ; Thu, 29 Aug 2024 17:42:49 +0000 (UTC) Authentication-Results: imf28.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=Y13JdG8x; dmarc=pass (policy=quarantine) header.from=kernel.org; spf=pass (imf28.hostedemail.com: domain of andrii@kernel.org designates 145.40.68.75 as permitted sender) smtp.mailfrom=andrii@kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1724953297; a=rsa-sha256; cv=none; b=tKTpgKvcmCqXD+Zb9OqSXpjj6scfRFVZKcSZN5u0gaF8EpvAL0tA5IrnQbcZanuaCDWJYc TW94ceZ4aTD1nj4rPknC/rdFyY7Bq6uNue+FvCFSMjCppKJFduHv5CGgs0kG2Fa6qvly/R EIwwoafk1WVvOBeSdmTYS/GDvbg6Aoc= ARC-Authentication-Results: i=1; imf28.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=Y13JdG8x; dmarc=pass (policy=quarantine) header.from=kernel.org; spf=pass (imf28.hostedemail.com: domain of andrii@kernel.org designates 145.40.68.75 as permitted sender) smtp.mailfrom=andrii@kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1724953297; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=D6qr6cjavPLmp//bC38sq+z7dCMkMMFwxvjl4VMnEpk=; b=yjfdXGLMsSYM+CXnK4E5/b2LHH8W0z0MQuVvI141Ib0eDHj24iDXemxpWSuOHd827cBcT5 xxx8rwEkK8WW65nqFi6wRMNeTCTFO3Z2EX79foK2vSQP/m2rrpoRvcwu7Pb5LM91Lm/i5U Ge0TjPA5wBYM2041jwIKAHSrJjx6x8k= Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by ams.source.kernel.org (Postfix) with ESMTP id ED964AE2CA3; Thu, 29 Aug 2024 17:42:42 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id BC5C9C4CEC1; Thu, 29 Aug 2024 17:42:47 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1724953367; bh=5CLcAmrYIJlnXm0mQvpn/y1HhZ+JoZho6ovLxlNSww4=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Y13JdG8xb8iuDxI4s46rTFLZcD10pfqV7vza38UzKeRzQoCwxEVNo8hA2eTGhuETW Hyaadvu2kZkMMtxCumBkixN0Xvv5Y9MbdYW8QWzqfeg8ltNMJoiz/vf94PsTyv8uAC oneOA1xwnIxNnE+vQFX01ZZQjSWinI6oygsunq7alY+uWLbMljQTmMdnde3Edi4a1I vYivD6IyJT7izocTWPg65srGF6Py9G66+mTfeexfdBLnjehN5cj+lktTmyamZTWuhU /98hLgNJGq6USK0h7pkav5KZ6OCiM901bnwTnvOWgCbFm51BIy/pG8ht7rZrKJkbXk EDBG8rM8TzYAA== From: Andrii Nakryiko To: bpf@vger.kernel.org Cc: linux-mm@kvack.org, akpm@linux-foundation.org, adobriyan@gmail.com, shakeel.butt@linux.dev, hannes@cmpxchg.org, ak@linux.intel.com, osandov@osandov.com, song@kernel.org, jannh@google.com, linux-fsdevel@vger.kernel.org, willy@infradead.org, Andrii Nakryiko , Eduard Zingerman Subject: [PATCH v7 bpf-next 04/10] lib/buildid: remove single-page limit for PHDR search Date: Thu, 29 Aug 2024 10:42:26 -0700 Message-ID: <20240829174232.3133883-5-andrii@kernel.org> X-Mailer: git-send-email 2.43.5 In-Reply-To: <20240829174232.3133883-1-andrii@kernel.org> References: <20240829174232.3133883-1-andrii@kernel.org> MIME-Version: 1.0 X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: E1D31C001C X-Stat-Signature: zndcfrh3t9s7d7z459wfwrutwhdxw1e7 X-Rspam-User: X-HE-Tag: 1724953369-349792 X-HE-Meta: U2FsdGVkX18eFXnqiqlSJl2kG6F1/+UUiP6Vhxn5WM+C7xXObkPbpI1NYB82ap/lCrq4QqyUKzibhlgzZYP9aIc2Y+9trE8GGSvDVwCOv8WxytZyKl9G0iLLUhKDuWAVpf/L7Fpw1pTGuIT/fGKFhwi7anE7NZHy5cBxE599s6Ad3+prnOfFZ+9rCnXiIO6LidTRjy0pzEHrXgKWdZWZkAPfoDw2H8TagF8HBN+Ck60ffcehiYOPmR00r6xqIEqxILr/znIG7Tnypo80X+7xqyWTUEjomQXjnqqJ7dXXK0jKPpQICXHb4k/ZrL5Qa3XilkwWZPcqk2QYfSzuV97YOo/CK9pBzGLFQxTCcJUPMbmQkYWEP5EmEJIcwiOPS2gH+k1/ixCHWPTKQ96sWU50ZAMM0SxRvl7djmb2dpvN7iyv86QDd3p2GpCi/CfgPeQOJABuw3eJ8w0n2mLiY1tL+QwH+vKv/2BElg6QLIZJiQ0PTA/zguv+4Am5DEY7J6qRPsdUOg9dM4k1IkJLBpb8Y7WS4vl35nskjdUDc2BTC5OZqGoPZMGKhN4rmJSmM3E4v48+6QsekZNNt8LGPd2/o+awpnBttylUR91LbVp8dZVV0xZoZLfG1syO568GnUWv2mA7qr6JKeoDYvcYMviAUYY4TO3jnIS8yhHWy/B3wDu+bqbAcaGRgPeum1MdeS6OA9zA6yrf/XMWmMSufRPYzGxZaFrxSlUizR+Gofxpbko7u+UGy8EQTgHyDEV/+DQLFuEZV78k7ZSaTmUnvtOWpasxp9Gc/DkGSf1mgbhYn3xsvfHUw1ugmYXP+/v8KXVjqabLjzctDON7ZmZ2XDqkEEK4F0MSLYUkzx/h6NLBFiIoygl7TiV99rwaBVkqcZF7Xiy2XyUC2wyhVdGosmyTMN85nZ03esJfIH3EvK1Xb1x0vFaiIp5jQkEufP5jON0rl6NAvWmcG4IAk7L90S3 S2HTzJ58 FxCBQqE15n/brLVpYike/+RMZ4SmxOdMw4fo7ThOWJUHt4SwWyIojIwfaEE4RMpr659FBJrrnuUuihd5NEfBoPIeHfO2muOlZef1nTlXJEV4apFwt87Jetpare1GDIaePfRZaiKFw3eJVQpDzFPyslybrSbCF7zNqMHVnfcYiMJ5sMrYb412z4FhugPDNSVwziZpzyXzDM++BVO2NnEZnsUxEbSbAbXd4Bug3Ry9NL+1/aj+22i/4yCajTkkM67W76chW X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Now that freader allows to access multiple pages transparently, there is no need to limit program headers to the very first ELF file page. Remove this limitation, but still put some sane limit on amount of program headers that we are willing to iterate over (set arbitrarily to 256). Reviewed-by: Eduard Zingerman Signed-off-by: Andrii Nakryiko --- lib/buildid.c | 14 ++++++++------ 1 file changed, 8 insertions(+), 6 deletions(-) diff --git a/lib/buildid.c b/lib/buildid.c index 7fb08a1d98bd..e8fc4aeb01f2 100644 --- a/lib/buildid.c +++ b/lib/buildid.c @@ -8,6 +8,8 @@ #define BUILD_ID 3 +#define MAX_PHDR_CNT 256 + struct freader { void *buf; u32 buf_sz; @@ -223,9 +225,9 @@ static int get_build_id_32(struct freader *r, unsigned char *build_id, __u32 *si phnum = READ_ONCE(ehdr->e_phnum); phoff = READ_ONCE(ehdr->e_phoff); - /* only supports phdr that fits in one page */ - if (phnum > (PAGE_SIZE - sizeof(Elf32_Ehdr)) / sizeof(Elf32_Phdr)) - return -EINVAL; + /* set upper bound on amount of segments (phdrs) we iterate */ + if (phnum > MAX_PHDR_CNT) + phnum = MAX_PHDR_CNT; /* check that phoff is not large enough to cause an overflow */ if (phoff + phnum * sizeof(Elf32_Phdr) < phoff) @@ -260,9 +262,9 @@ static int get_build_id_64(struct freader *r, unsigned char *build_id, __u32 *si phnum = READ_ONCE(ehdr->e_phnum); phoff = READ_ONCE(ehdr->e_phoff); - /* only supports phdr that fits in one page */ - if (phnum > (PAGE_SIZE - sizeof(Elf64_Ehdr)) / sizeof(Elf64_Phdr)) - return -EINVAL; + /* set upper bound on amount of segments (phdrs) we iterate */ + if (phnum > MAX_PHDR_CNT) + phnum = MAX_PHDR_CNT; /* check that phoff is not large enough to cause an overflow */ if (phoff + phnum * sizeof(Elf64_Phdr) < phoff) From patchwork Thu Aug 29 17:42:27 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrii Nakryiko X-Patchwork-Id: 13783515 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id D52D8C87FCD for ; Thu, 29 Aug 2024 17:42:56 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id CBE216B00B4; Thu, 29 Aug 2024 13:42:55 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id BF8226B00B5; Thu, 29 Aug 2024 13:42:55 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 95EA16B00B6; Thu, 29 Aug 2024 13:42:55 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 722B86B00B4 for ; Thu, 29 Aug 2024 13:42:55 -0400 (EDT) Received: from smtpin09.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 237D91A0701 for ; Thu, 29 Aug 2024 17:42:55 +0000 (UTC) X-FDA: 82506003510.09.592B68F Received: from ams.source.kernel.org (ams.source.kernel.org [145.40.68.75]) by imf13.hostedemail.com (Postfix) with ESMTP id 45B3C20018 for ; Thu, 29 Aug 2024 17:42:53 +0000 (UTC) Authentication-Results: imf13.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=WBvaoWsW; dmarc=pass (policy=quarantine) header.from=kernel.org; spf=pass (imf13.hostedemail.com: domain of andrii@kernel.org designates 145.40.68.75 as permitted sender) smtp.mailfrom=andrii@kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1724953274; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=D40VGjjdavlr4wHnP2dKrdOmyuV2VKSC1QVisv7AqNc=; b=PtX1Jd0bMS1LQSoRsFHzxbmDV4/5XT/jzaOyRrbc7vAl8+CNmpWaUu6zzei002ZLcv54PU 9xB8/yOqgKqk0QQfZ/atIoXt4q6zQJwG1PsCfRBe7Dt6XTbgv6p4hKSP8iwPxGgshDBxh5 eZDBcnCkhOuYrR0jwu7ms/kP04+JzwA= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1724953274; a=rsa-sha256; cv=none; b=SnUxqkSj3G+o1OJ6MsCZvjAg1yPwByFPPcMv3lZTPQM4WZB3ryK6uZTkXoSyZJjffXVwkY +daj85L6mplD74FbPF73j2XbGZmek2lr6Sqngb5NiN3FIAWi5zZTbyk9AeCwMNEij/HQar 2pcI+P+VSNtIpjo/CMy4YKiYRFrjfqI= ARC-Authentication-Results: i=1; imf13.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=WBvaoWsW; dmarc=pass (policy=quarantine) header.from=kernel.org; spf=pass (imf13.hostedemail.com: domain of andrii@kernel.org designates 145.40.68.75 as permitted sender) smtp.mailfrom=andrii@kernel.org Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by ams.source.kernel.org (Postfix) with ESMTP id 6220BAE2CD0; Thu, 29 Aug 2024 17:42:46 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 06A05C4CEC2; Thu, 29 Aug 2024 17:42:51 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1724953371; bh=wB28U31vHwT6YnSflEoFZZLqW1/CPRZJ0NPgjrn1UhQ=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=WBvaoWsWRNVaVVuv2xXmqKIidjDuVADjq04fALxrs8n8+Z5Z3hEHKp/HA+4xhHFSD brILKgXRIkFDpN5DzPx+mllCuOIHi9V7mdzAZXcEmd+s8zHKfXjA+0KyMip+Vi9/wp CCirREQsumrgyna4aVPLp/Ya9O/58JxFnA/FkPTa06cM6ckklB4Rz5djcyHcyHCyN5 GGsnbNf4Hh1qf+tAB8uplWEqkEVpbgwnyqP5HY+favLHYtKQCJujOUxrTpshZSSQ1s bLAQuj0/TAY1FGELr/AIHUMOx/y9QWbQZbs7xjvfJrYNhYjyjBzOAOOj+fupfgiymK xaHRkSHKGHyqg== From: Andrii Nakryiko To: bpf@vger.kernel.org Cc: linux-mm@kvack.org, akpm@linux-foundation.org, adobriyan@gmail.com, shakeel.butt@linux.dev, hannes@cmpxchg.org, ak@linux.intel.com, osandov@osandov.com, song@kernel.org, jannh@google.com, linux-fsdevel@vger.kernel.org, willy@infradead.org, Andrii Nakryiko , Eduard Zingerman Subject: [PATCH v7 bpf-next 05/10] lib/buildid: rename build_id_parse() into build_id_parse_nofault() Date: Thu, 29 Aug 2024 10:42:27 -0700 Message-ID: <20240829174232.3133883-6-andrii@kernel.org> X-Mailer: git-send-email 2.43.5 In-Reply-To: <20240829174232.3133883-1-andrii@kernel.org> References: <20240829174232.3133883-1-andrii@kernel.org> MIME-Version: 1.0 X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: 45B3C20018 X-Stat-Signature: c46ta4o9mjysx95t9rymtwqp4htwxe4j X-Rspam-User: X-HE-Tag: 1724953373-757888 X-HE-Meta: U2FsdGVkX18sXVMNzM04nKrTNRZGWst1D3eEOHisGhF0CC1e52YSVNCTLVxfuKgGfIQaFKPw/SQqqHhQxzIOANWiIJXe7TCYQJo+pqyauKzjVmcc6CkS/q0Opaev23gMY0CTrlONGaFBdDsv5pGr8isDPfECfd5r7r2hf9JzEe5KcXRydF7fYeT4CsQuygN2BbJX3QrNgnguI1x3V79Y6tjeQOh0M2TWaqGiJYNx5hYKk5L2TwQpYoPfBVK2xcflK6cnNgVgO7zS4Ek0YYuCmOfLL58UPGxgK/4fyN8euaYMOK64z/+VNVQvuiDdCDeEwOS9UqEf4V6nxkNBIJhvEZSBC1MiZ9F5x2+qF7eE9sHIPHuep4tK5i1KcuWgYqSq26oNeyKUJ7S7OzZlEk6fngcgiv0YoqRfsD7oaXOQe39lHoke12Rpw4ZW5jQMOy87l3ctlsy3SB1vsuOY7pXXwBw4tfZ/Qua8qo40vq9VJDTGYP4yH8Kjp9rvsp8epn575mrKhpmEY8/DQdh3RtNkDMC7wuF0ALtvZJhjRD1PpmQCd8gHyeC77riGga1W8ylCdAFDnlzE8pNNgKEaEapkhy3sovQUI+JHHYcJl9Rt0fihjyQNRufwqUar6wwF45lBL0CdKeKv+rT0vunwUFvRd8xPiW6kTVKTyyBoDLeYzWp4MdspBg5pzc134YcrtT1EHYCwNstJYMeP1AFUqnoYv3QG8EMgqf0wfoGXby4W9TPWdvMQfE/IGrc1oeRm7ugbGpPr4EGOgN/KUdoeLmorYasLJ0pl3/pEiVjXG7aEXnPdK3q2r00EWCnmivxXgsH+wImg7+nULn49Qlhuv1lOn5bj10gESheVR5ho9Eld7ToK7mao3KRKfHOOPoiaGdhqCnJg1fs9MUpr+AdFZWlVkFKSTVH9FTUdA8lL17JpkY5vb0sakQx6rTnLPwwIqzffpUPp04UtkVoJPuhqzO6 qfo9mktK e075opgfJ+gKpsQnhAeGfTSWEGfUNIbFHCxEqzD6iOuT3+w05tosWlbmqlcFwT1mmznTfIfKLjPZjOoHm+Rs0eg9NXuteBbbaF8mq0WGsgxYxmKyAJmUs9c4OJQzJAVSftQBruU5jbTjj8XiJfgbEjP9HVMhM9eoH8LBsEHC73R99olhT4n3sgZ7sgJ3499FxqCmtNuvLLbQVqAp8+UX9qx/pdWENvnBNhs/UtTyCw6OzadW0m4G0AWrQoV8BvQQIEnjZWtk/M2BSXwIsMhGlMvtblfRr93M11kBT/Et6TWheyVlYpIH7pEecfA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Make it clear that build_id_parse() assumes that it can take no page fault by renaming it and current few users to build_id_parse_nofault(). Also add build_id_parse() stub which for now falls back to non-sleepable implementation, but will be changed in subsequent patches to take advantage of sleepable context. PROCMAP_QUERY ioctl() on /proc//maps file is using build_id_parse() and will automatically take advantage of more reliable sleepable context implementation. Reviewed-by: Eduard Zingerman Signed-off-by: Andrii Nakryiko --- include/linux/buildid.h | 4 ++-- kernel/bpf/stackmap.c | 2 +- kernel/events/core.c | 2 +- lib/buildid.c | 25 ++++++++++++++++++++++--- 4 files changed, 26 insertions(+), 7 deletions(-) diff --git a/include/linux/buildid.h b/include/linux/buildid.h index 20aa3c2d89f7..014a88c41073 100644 --- a/include/linux/buildid.h +++ b/include/linux/buildid.h @@ -7,8 +7,8 @@ #define BUILD_ID_SIZE_MAX 20 struct vm_area_struct; -int build_id_parse(struct vm_area_struct *vma, unsigned char *build_id, - __u32 *size); +int build_id_parse(struct vm_area_struct *vma, unsigned char *build_id, __u32 *size); +int build_id_parse_nofault(struct vm_area_struct *vma, unsigned char *build_id, __u32 *size); int build_id_parse_buf(const void *buf, unsigned char *build_id, u32 buf_size); #if IS_ENABLED(CONFIG_STACKTRACE_BUILD_ID) || IS_ENABLED(CONFIG_VMCORE_INFO) diff --git a/kernel/bpf/stackmap.c b/kernel/bpf/stackmap.c index c99f8e5234ac..770ae8e88016 100644 --- a/kernel/bpf/stackmap.c +++ b/kernel/bpf/stackmap.c @@ -156,7 +156,7 @@ static void stack_map_get_build_id_offset(struct bpf_stack_build_id *id_offs, goto build_id_valid; } vma = find_vma(current->mm, ips[i]); - if (!vma || build_id_parse(vma, id_offs[i].build_id, NULL)) { + if (!vma || build_id_parse_nofault(vma, id_offs[i].build_id, NULL)) { /* per entry fall back to ips */ id_offs[i].status = BPF_STACK_BUILD_ID_IP; id_offs[i].ip = ips[i]; diff --git a/kernel/events/core.c b/kernel/events/core.c index c973e3c11e03..c78a77f9dce4 100644 --- a/kernel/events/core.c +++ b/kernel/events/core.c @@ -8851,7 +8851,7 @@ static void perf_event_mmap_event(struct perf_mmap_event *mmap_event) mmap_event->event_id.header.size = sizeof(mmap_event->event_id) + size; if (atomic_read(&nr_build_id_events)) - build_id_parse(vma, mmap_event->build_id, &mmap_event->build_id_size); + build_id_parse_nofault(vma, mmap_event->build_id, &mmap_event->build_id_size); perf_iterate_sb(perf_event_mmap_output, mmap_event, diff --git a/lib/buildid.c b/lib/buildid.c index e8fc4aeb01f2..c1cbd34f3685 100644 --- a/lib/buildid.c +++ b/lib/buildid.c @@ -293,10 +293,12 @@ static int get_build_id_64(struct freader *r, unsigned char *build_id, __u32 *si * @build_id: buffer to store build id, at least BUILD_ID_SIZE long * @size: returns actual build id size in case of success * - * Return: 0 on success, -EINVAL otherwise + * Assumes no page fault can be taken, so if relevant portions of ELF file are + * not already paged in, fetching of build ID fails. + * + * Return: 0 on success; negative error, otherwise */ -int build_id_parse(struct vm_area_struct *vma, unsigned char *build_id, - __u32 *size) +int build_id_parse_nofault(struct vm_area_struct *vma, unsigned char *build_id, __u32 *size) { const Elf32_Ehdr *ehdr; struct freader r; @@ -335,6 +337,23 @@ int build_id_parse(struct vm_area_struct *vma, unsigned char *build_id, return ret; } +/* + * Parse build ID of ELF file mapped to VMA + * @vma: vma object + * @build_id: buffer to store build id, at least BUILD_ID_SIZE long + * @size: returns actual build id size in case of success + * + * Assumes faultable context and can cause page faults to bring in file data + * into page cache. + * + * Return: 0 on success; negative error, otherwise + */ +int build_id_parse(struct vm_area_struct *vma, unsigned char *build_id, __u32 *size) +{ + /* fallback to non-faultable version for now */ + return build_id_parse_nofault(vma, build_id, size); +} + /** * build_id_parse_buf - Get build ID from a buffer * @buf: ELF note section(s) to parse From patchwork Thu Aug 29 17:42:28 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrii Nakryiko X-Patchwork-Id: 13783516 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 97E0DC87FCD for ; Thu, 29 Aug 2024 17:42:59 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id E1DC56B0089; Thu, 29 Aug 2024 13:42:58 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id DA5466B00A6; Thu, 29 Aug 2024 13:42:58 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id BF9006B00B6; Thu, 29 Aug 2024 13:42:58 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 90C796B0089 for ; Thu, 29 Aug 2024 13:42:58 -0400 (EDT) Received: from smtpin30.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 4981014053A for ; Thu, 29 Aug 2024 17:42:58 +0000 (UTC) X-FDA: 82506003636.30.67C63B0 Received: from ams.source.kernel.org (ams.source.kernel.org [145.40.68.75]) by imf01.hostedemail.com (Postfix) with ESMTP id 752CE40011 for ; Thu, 29 Aug 2024 17:42:56 +0000 (UTC) Authentication-Results: imf01.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=PWZIwYJt; spf=pass (imf01.hostedemail.com: domain of andrii@kernel.org designates 145.40.68.75 as permitted sender) smtp.mailfrom=andrii@kernel.org; dmarc=pass (policy=quarantine) header.from=kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1724953287; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=CgwGfgjZwwNKNiR5h2Kb486VbQFMf/adnAPBvAZvt3E=; b=wGRPKyugONotmfYUQ0TXASaDRYlqW6wYWvszTEbO5fBJDBICHQDleJiOd744J8MHRIk0dH ShhglVaI2C7DU+pcRUJUncWG6joUK9AU3AeaVc/c1+p8moUlKcqlQOB9HJys+l0Tvx/Q5e b5nwzAaqyMch08HMFkjmWuVkgcH6fIM= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1724953287; a=rsa-sha256; cv=none; b=VYhhSTtVoQ5Lw3eNP2e1Qe9m/e2Gf7mvlaIMaZKh7WmZcZxorR/VC+B1xBtJGtCqhC8z/N /0weC+GbHxLCnKoAiSKG/CWiWCHtKjvBmoJiu6Cf/QYTuCSSM8qSTqYqwXu5mvxBEpRKG+ qjlFjb/Hqqr1VsbvWt3p2S6Hz30bA7A= ARC-Authentication-Results: i=1; imf01.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=PWZIwYJt; spf=pass (imf01.hostedemail.com: domain of andrii@kernel.org designates 145.40.68.75 as permitted sender) smtp.mailfrom=andrii@kernel.org; dmarc=pass (policy=quarantine) header.from=kernel.org Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by ams.source.kernel.org (Postfix) with ESMTP id A444FAE2820; Thu, 29 Aug 2024 17:42:49 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 418F3C4CEC1; Thu, 29 Aug 2024 17:42:54 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1724953374; bh=fCwSimqQps0x37cNpSBPuZZv0BI1f3GW/HwGWSK6VXw=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=PWZIwYJtQmn+LkoSZ3Jlq0eKfKvTkd3g+cHFDhLEZBB3AhDDZl/QK43WNZgezjRmJ yC+o+yHV4bFHUWPVkJWYnJfkS4h2EHVSyMIxwNiTbFiadPCMzgjwEqC5cgoUO7DtVT dGeSrHgmQ7UDfHYUFl+j9bT1HlwnANmxZFm7+1HY/vljZaJq4gSftnP3gzJkN+GLfw gY/pRCGUx7VHVKzG4kykFHkKRfy8E5wH/bpt5Aju2XNXAzfUq77/ZM6zaTPiqSkseK XsPHvmQZ4YWofUaw4k9BBOgZCXeIaKfnnaJ2Ja2/7Ri7uPzQ68X2YF5RmdsA0UDySi mvNcyn2zb/KIA== From: Andrii Nakryiko To: bpf@vger.kernel.org Cc: linux-mm@kvack.org, akpm@linux-foundation.org, adobriyan@gmail.com, shakeel.butt@linux.dev, hannes@cmpxchg.org, ak@linux.intel.com, osandov@osandov.com, song@kernel.org, jannh@google.com, linux-fsdevel@vger.kernel.org, willy@infradead.org, Andrii Nakryiko , Omar Sandoval , Eduard Zingerman Subject: [PATCH v7 bpf-next 06/10] lib/buildid: implement sleepable build_id_parse() API Date: Thu, 29 Aug 2024 10:42:28 -0700 Message-ID: <20240829174232.3133883-7-andrii@kernel.org> X-Mailer: git-send-email 2.43.5 In-Reply-To: <20240829174232.3133883-1-andrii@kernel.org> References: <20240829174232.3133883-1-andrii@kernel.org> MIME-Version: 1.0 X-Stat-Signature: 1yza8dhrxs7kcxgf8os1bsqrmm36qox6 X-Rspamd-Queue-Id: 752CE40011 X-Rspam-User: X-Rspamd-Server: rspam08 X-HE-Tag: 1724953376-855371 X-HE-Meta: U2FsdGVkX18HkwcJhUNjhc6Ikjx/6XkWjXbtwDPwEZawxFiZNJh3EYK1APTYeZSt9kTIKg+wYibWf3qF1NM0dx6D8WzlG3SFokp2x5y3E/sM28ooUeGP2UW5wqyWsVwn13UTcks1qGmnynyqJIdGU4HoKelr0muMemgOsxy0/uutbLaeBjo71SzJaJ+N27DuWu1UXWtN0Co7I8QI6eKrDHIJkgHY3cPqoT2gwviQl+7ow0mvCVqqo7jUCHkpkEoyOKlaZLwkYokmjdTaxbICXPKxfqWhnsoLT5jHFgIL0O/kEUjhB5VibzHWuwhqHzj0tM9wCrMcQkUz9E/wzCd4LA0QrMIuNJi6AkWIeZo+lXHBuXGKTw3bGY6ggyb11+ONZHSLp99OQH6D6TNwyVe8usMH96nEqmnLu1h5XT5TfzVEmzorF8qbNGY7oWcncLTkFWVNAmspa0KQNK/LOiLAtShejHx+PqGxoVWhOAF9DkDhdgsSjGweAqGKH8zdmMxjV/IhlqOez1va965kU5y5VjfEgYs9uRWAEcqqpH8bD9ypoTcGBqEmsXhe5g62o/UB4ItWAFpvIyOOO8fcjnzsOP01MZGKfNY5dFgancqkZODqR8PLacCZevwLppne/q1O05VVmx1234lAfVdc+o3vNDjjZXnF7kp+rYLHHU5xHsekyMOUsz9ubsWq9E9Xsy5RtV2pIXVG45iFhdtU8VnoByOjaKaIf51tqoBGW7Gb5Vis8Ds7KEpkPJ1j7ucNI5FCYax3Fa/mFgrlSLieBzutwnRK9GaCDh8nPOvJB2p3TQGPAdRHhoiBIe3k9rcYD47Qz9wKDPE7vvImzjwDRraBbfhttlWntu8ksSHy3tAYAtlooPGe61P6jPx9s+eeaot+QE1iQ+BotPYO2vOVbPJwbqd52u3G7cnjqd7ZrAslPBgn2vbNsYQosOpE2szx4zazDNsPEGhvlZtM+dj17cY RPfh2BnY KcsbzBNIq8VMLxjwu6beZ+n7sRy7TTFhXeN8DihDSVaEtVze5yayA5FpdypmZjWl1+ZtAhdUechoSUtvWD2QB3xAZmU4pv/pTh9hymENxea0QyYH2/rlwsDLjmjPtNiSfCRfwVHqKbK2fDGnSEUy8air7oY9O/T/xzGdDCNmvyElOg+G2sIu9EtV+dsKp++x9ppBiffFdFHcKWdzUwVasXXCPgw358/eozVFGtIjProxT7rcmg6JTX/0e2NPpMBdL8vrhDLBIhxmd2qTjIZ66DDDgPAqBJ0OcTeInXElxtBYpXhr5ub1Vu4sVuKptv/Qymc3NCUtkqIGy0SUvaFmo6SR/+csahAHceo8NoPGTkvrPjU+MAex1ckO9oIKu0YMxM9zM X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Extend freader with a flag specifying whether it's OK to cause page fault to fetch file data that is not already physically present in memory. With this, it's now easy to wait for data if the caller is running in sleepable (faultable) context. We utilize read_cache_folio() to bring the desired folio into page cache, after which the rest of the logic works just the same at folio level. Suggested-by: Omar Sandoval Cc: Shakeel Butt Cc: Johannes Weiner Reviewed-by: Eduard Zingerman Reviewed-by: Shakeel Butt Signed-off-by: Andrii Nakryiko --- lib/buildid.c | 54 +++++++++++++++++++++++++++++++++------------------ 1 file changed, 35 insertions(+), 19 deletions(-) diff --git a/lib/buildid.c b/lib/buildid.c index c1cbd34f3685..18ef55812c64 100644 --- a/lib/buildid.c +++ b/lib/buildid.c @@ -16,10 +16,11 @@ struct freader { int err; union { struct { - struct address_space *mapping; + struct file *file; struct folio *folio; void *addr; loff_t folio_off; + bool may_fault; }; struct { const char *data; @@ -29,12 +30,13 @@ struct freader { }; static void freader_init_from_file(struct freader *r, void *buf, u32 buf_sz, - struct address_space *mapping) + struct file *file, bool may_fault) { memset(r, 0, sizeof(*r)); r->buf = buf; r->buf_sz = buf_sz; - r->mapping = mapping; + r->file = file; + r->may_fault = may_fault; } static void freader_init_from_mem(struct freader *r, const char *data, u64 data_sz) @@ -62,7 +64,16 @@ static int freader_get_folio(struct freader *r, loff_t file_off) freader_put_folio(r); - r->folio = filemap_get_folio(r->mapping, file_off >> PAGE_SHIFT); + r->folio = filemap_get_folio(r->file->f_mapping, file_off >> PAGE_SHIFT); + + /* if sleeping is allowed, wait for the page, if necessary */ + if (r->may_fault && (IS_ERR(r->folio) || !folio_test_uptodate(r->folio))) { + filemap_invalidate_lock_shared(r->file->f_mapping); + r->folio = read_cache_folio(r->file->f_mapping, file_off >> PAGE_SHIFT, + NULL, r->file); + filemap_invalidate_unlock_shared(r->file->f_mapping); + } + if (IS_ERR(r->folio) || !folio_test_uptodate(r->folio)) { if (!IS_ERR(r->folio)) folio_put(r->folio); @@ -287,18 +298,8 @@ static int get_build_id_64(struct freader *r, unsigned char *build_id, __u32 *si /* enough for Elf64_Ehdr, Elf64_Phdr, and all the smaller requests */ #define MAX_FREADER_BUF_SZ 64 -/* - * Parse build ID of ELF file mapped to vma - * @vma: vma object - * @build_id: buffer to store build id, at least BUILD_ID_SIZE long - * @size: returns actual build id size in case of success - * - * Assumes no page fault can be taken, so if relevant portions of ELF file are - * not already paged in, fetching of build ID fails. - * - * Return: 0 on success; negative error, otherwise - */ -int build_id_parse_nofault(struct vm_area_struct *vma, unsigned char *build_id, __u32 *size) +static int __build_id_parse(struct vm_area_struct *vma, unsigned char *build_id, + __u32 *size, bool may_fault) { const Elf32_Ehdr *ehdr; struct freader r; @@ -309,7 +310,7 @@ int build_id_parse_nofault(struct vm_area_struct *vma, unsigned char *build_id, if (!vma->vm_file) return -EINVAL; - freader_init_from_file(&r, buf, sizeof(buf), vma->vm_file->f_mapping); + freader_init_from_file(&r, buf, sizeof(buf), vma->vm_file, may_fault); /* fetch first 18 bytes of ELF header for checks */ ehdr = freader_fetch(&r, 0, offsetofend(Elf32_Ehdr, e_type)); @@ -337,6 +338,22 @@ int build_id_parse_nofault(struct vm_area_struct *vma, unsigned char *build_id, return ret; } +/* + * Parse build ID of ELF file mapped to vma + * @vma: vma object + * @build_id: buffer to store build id, at least BUILD_ID_SIZE long + * @size: returns actual build id size in case of success + * + * Assumes no page fault can be taken, so if relevant portions of ELF file are + * not already paged in, fetching of build ID fails. + * + * Return: 0 on success; negative error, otherwise + */ +int build_id_parse_nofault(struct vm_area_struct *vma, unsigned char *build_id, __u32 *size) +{ + return __build_id_parse(vma, build_id, size, false /* !may_fault */); +} + /* * Parse build ID of ELF file mapped to VMA * @vma: vma object @@ -350,8 +367,7 @@ int build_id_parse_nofault(struct vm_area_struct *vma, unsigned char *build_id, */ int build_id_parse(struct vm_area_struct *vma, unsigned char *build_id, __u32 *size) { - /* fallback to non-faultable version for now */ - return build_id_parse_nofault(vma, build_id, size); + return __build_id_parse(vma, build_id, size, true /* may_fault */); } /** From patchwork Thu Aug 29 17:42:29 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrii Nakryiko X-Patchwork-Id: 13783517 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id A2717C8303E for ; Thu, 29 Aug 2024 17:43:02 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 100506B00B5; Thu, 29 Aug 2024 13:43:02 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 00F526B00B7; Thu, 29 Aug 2024 13:43:01 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D7D996B00B8; Thu, 29 Aug 2024 13:43:01 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id A926E6B00B5 for ; Thu, 29 Aug 2024 13:43:01 -0400 (EDT) Received: from smtpin22.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 6490DA04B0 for ; Thu, 29 Aug 2024 17:43:01 +0000 (UTC) X-FDA: 82506003762.22.5E45B02 Received: from ams.source.kernel.org (ams.source.kernel.org [145.40.68.75]) by imf07.hostedemail.com (Postfix) with ESMTP id A4D8640009 for ; Thu, 29 Aug 2024 17:42:59 +0000 (UTC) Authentication-Results: imf07.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=Iij0aW8j; spf=pass (imf07.hostedemail.com: domain of andrii@kernel.org designates 145.40.68.75 as permitted sender) smtp.mailfrom=andrii@kernel.org; dmarc=pass (policy=quarantine) header.from=kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1724953309; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=7A7Lt1FSDUlLzoHLFtmVPVQLw9Jv32/MfGnKFXKkguI=; b=I+dO8IqqGbT4dpYmBvaJ5HOBsxg4ReACA6o7omOgHO3oqorQMKDxlneaHIEKtsX8i16hEK N/vzKCuCPhfuHEiHH9OxXWyXxySxMHxwqwrztc5Y8Qkwh5M2ZA4mUAWY71T1UoRDKl9cB1 /NIfCvNHo6R7NYaHrMX/92EPXuaJwCs= ARC-Authentication-Results: i=1; imf07.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=Iij0aW8j; spf=pass (imf07.hostedemail.com: domain of andrii@kernel.org designates 145.40.68.75 as permitted sender) smtp.mailfrom=andrii@kernel.org; dmarc=pass (policy=quarantine) header.from=kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1724953309; a=rsa-sha256; cv=none; b=s/5KF9UBxwBwsaYqG0VEO/dCCNaLDoz+f70Dhpc1pqNfGgFSStHlY3FJriWmEzDSjsyq4v jtfQrwSKQ1iAw2uBiY3xIHQET3/wGYu35TLn7m5qlzit0UJuGraQAyGm03M0TLq4UD9hHH nO1tD+pMasZ5ReP4nlgFejROHRAel+k= Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by ams.source.kernel.org (Postfix) with ESMTP id C12C0AE287B; Thu, 29 Aug 2024 17:42:52 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 8E162C4CEC1; Thu, 29 Aug 2024 17:42:57 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1724953377; bh=2jVZvSaSmUP1eILbkNsVbAsPQzYYbYwqDjRGc+xq+N4=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Iij0aW8jjwJqgQiw3EdbzhJ/zMX2hOtDDDPvqLD6dR//5hE2vlBaFNc1Zns9oO8Of fbYp4hyEnZlujg6/CwOVGXypguIDPW8QB1Q8pOqX55ZuVvyzbejBkFqvg4eB//Bwkw Ql2O4bvlqmgE5bDgmeHYob++eJop6Sq5bENwENUgVel5WIDH2dogQ19O97M+HxhjZa 4lR64OjtEtn+0wJOpHHvZdIHAa24eN8rv8AVfdm0rqsm5ow3yZGLedfPASdB+8xa3X 02bHyiJawK+xM63XbjtQExRwHZOnHvqLT6SRaeE63Rl0/R7SRQXbCPWVUbF1nsrfu6 RLwIVPacoRcsg== From: Andrii Nakryiko To: bpf@vger.kernel.org Cc: linux-mm@kvack.org, akpm@linux-foundation.org, adobriyan@gmail.com, shakeel.butt@linux.dev, hannes@cmpxchg.org, ak@linux.intel.com, osandov@osandov.com, song@kernel.org, jannh@google.com, linux-fsdevel@vger.kernel.org, willy@infradead.org, Andrii Nakryiko , Eduard Zingerman Subject: [PATCH v7 bpf-next 07/10] lib/buildid: don't limit .note.gnu.build-id to the first page in ELF Date: Thu, 29 Aug 2024 10:42:29 -0700 Message-ID: <20240829174232.3133883-8-andrii@kernel.org> X-Mailer: git-send-email 2.43.5 In-Reply-To: <20240829174232.3133883-1-andrii@kernel.org> References: <20240829174232.3133883-1-andrii@kernel.org> MIME-Version: 1.0 X-Stat-Signature: 1mhgf6mhdt3hhmdd4z5o87qctgumdj19 X-Rspam-User: X-Rspamd-Queue-Id: A4D8640009 X-Rspamd-Server: rspam02 X-HE-Tag: 1724953379-584797 X-HE-Meta: U2FsdGVkX1+5OsEQtozZv23QfVvTKc/oRryv69cW9kVWZQflVCxAjrL+x6noWwqjx7C8RN6gcxEHLVl+2w7DOXI3p1vNAIZqBdnUXVB2gO1Yxl+KFahXyw8MrvDkndTBxai9+eVtndzUrU0Jk/MKvJZd3gi/RjT4VRDiQ3LxIObtSzvpX32aHaD6qGGQLwHNlsaOUm5HRJSdFWy6cqg51KP9hjtNpu6POrLoUkPQS0ltfIT7e01qq3kpmP7mVwRabjz3v5EtffHFJLwiPju5VCDcsonEM7CdTfYD60923DOz+qY8r6Ar38AqbMsH87QkKud7Ympr3XeHDyT4P0k691B00CrM+A04NaoEC8n+Xp76/rAoptXkwtwjFTUGXdcem33UjtWQNxYYEYVs87mvIY5yX4LQYzCDnMIHZmvbPBwldlWgXoU23MLNRDdUABu7L2L+Lbba9ErW88gZak9xG1hysiGB7+oGsrkL4LMPIXaxqKWXS4/Oo63CNMmlwLjVNBOd8bXYg+4Cxtprmp/zrKqcm/JK2KVWwa4cT0v4/hQ1fVVZWf7PmGD0ALVdEsgl3RAORX1F6chPRxFV+H8ut85l/byR3yZlkc6ZppiVG+wu8P4xdJ/B5jvfKudR45t/SorwrLuLIEmh8En0V0dAd26rN5k1Tnhy4qZoywQacXWsDKjY9jDlPVABLEDmmbWx0LtLpo4TAS8oSRcR1IzehbN7sr1QoB57lxwDD6ZlTH+i6ca9ukm6eBvGWobdpQgxsnBMWNR48SFgnFSMU4tve6rW6ZNi1nFPIofWKpaBC52zLvN9KlvoG75rP9tnEEsgPOG1264uQPK4FS/ZNvu2JZobVnR8lTCrYvRt8MQCzQLDPP3wHKpyFteC+LqFWQxeyMkmEK2UpMJR8lyAbgkGy6ru7bHBrMzKsp6qJ3kZaTDtkDr3qdcu07whzZq+U+NC/luzyBjqtP7IeYvAvru UsZg0LGQ 3syAytgDcYmagXZzW3/iWVQgopcktTMpI758njzmSOePORj/cX/UTQ2NQQGLdN2mZYavOhUYXkuON2KUvc/kn8KIGiJusRBP06E05agpeOO1Av4GlWS3ED4NvmxcITHzKfjbLCYfWMBwxSdsGK1REdU4ot8xdenSIFcOQFZdp7Vvf+pfsUhOO6Kjz3D+Tj7Q91va/inlpubB10LPg9uMz7WrSZocXIbSQOg7qHw5I/13PSmX4o5vC6NoXIFAflrheNjES X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: With freader we don't need to restrict ourselves to a single page, so let's allow ELF notes to be at any valid position with the file. We also merge parse_build_id() and parse_build_id_buf() as now the only difference between them is note offset overflow, which makes sense to check in all situations. Reviewed-by: Eduard Zingerman Signed-off-by: Andrii Nakryiko --- lib/buildid.c | 26 +++++--------------------- 1 file changed, 5 insertions(+), 21 deletions(-) diff --git a/lib/buildid.c b/lib/buildid.c index 18ef55812c64..290641d92ac1 100644 --- a/lib/buildid.c +++ b/lib/buildid.c @@ -155,9 +155,8 @@ static void freader_cleanup(struct freader *r) * 32-bit and 64-bit system, because Elf32_Nhdr and Elf64_Nhdr are * identical. */ -static int parse_build_id_buf(struct freader *r, - unsigned char *build_id, __u32 *size, - loff_t note_off, Elf32_Word note_size) +static int parse_build_id(struct freader *r, unsigned char *build_id, __u32 *size, + loff_t note_off, Elf32_Word note_size) { const char note_name[] = "GNU"; const size_t note_name_sz = sizeof(note_name); @@ -165,7 +164,9 @@ static int parse_build_id_buf(struct freader *r, const Elf32_Nhdr *nhdr; const char *data; - note_end = note_off + note_size; + if (check_add_overflow(note_off, note_size, ¬e_end)) + return -EINVAL; + while (note_end - note_off > sizeof(Elf32_Nhdr) + note_name_sz) { nhdr = freader_fetch(r, note_off, sizeof(Elf32_Nhdr) + note_name_sz); if (!nhdr) @@ -204,23 +205,6 @@ static int parse_build_id_buf(struct freader *r, return -EINVAL; } -static inline int parse_build_id(struct freader *r, - unsigned char *build_id, - __u32 *size, - loff_t note_start_off, - Elf32_Word note_size) -{ - /* check for overflow */ - if (note_start_off + note_size < note_start_off) - return -EINVAL; - - /* only supports note that fits in the first page */ - if (note_start_off + note_size > PAGE_SIZE) - return -EINVAL; - - return parse_build_id_buf(r, build_id, size, note_start_off, note_size); -} - /* Parse build ID from 32-bit ELF */ static int get_build_id_32(struct freader *r, unsigned char *build_id, __u32 *size) { From patchwork Thu Aug 29 17:42:30 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrii Nakryiko X-Patchwork-Id: 13783518 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id E466BC87FCE for ; Thu, 29 Aug 2024 17:43:05 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 6F1746B00B9; Thu, 29 Aug 2024 13:43:05 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 652526B00BA; Thu, 29 Aug 2024 13:43:05 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 4A4EC6B00BB; Thu, 29 Aug 2024 13:43:05 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 1F0686B00B9 for ; Thu, 29 Aug 2024 13:43:05 -0400 (EDT) Received: from smtpin20.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id CF0801A0CCD for ; Thu, 29 Aug 2024 17:43:04 +0000 (UTC) X-FDA: 82506003888.20.1BE436D Received: from ams.source.kernel.org (ams.source.kernel.org [145.40.68.75]) by imf09.hostedemail.com (Postfix) with ESMTP id 09F05140023 for ; Thu, 29 Aug 2024 17:43:02 +0000 (UTC) Authentication-Results: imf09.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b="o/PzqoW7"; dmarc=pass (policy=quarantine) header.from=kernel.org; spf=pass (imf09.hostedemail.com: domain of andrii@kernel.org designates 145.40.68.75 as permitted sender) smtp.mailfrom=andrii@kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1724953283; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=4WGFVVD8NNfz/tzbo/ADWproGkeU+mb1JZaG+UH0brI=; b=K1ADJjOU1BU75lRpV4ialNgJugF3OYcrDozxG0AGOiXKLBm45o7TOxyFIpj4XNtjs1KcCE 8CfykLxsSF4yPA0N8WQwqqwhO7/6wum7Wc7jcatYa8Lfwr7h5IEnJtjkRmhbU1ap37NEEi GAbTfpsbhDm6FiQDaFmA+8i0KbXwumM= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1724953283; a=rsa-sha256; cv=none; b=5C3yCm0XY9tK35aPY0qvqvB0VaP7YlR42vxj+IXuYr6rXafoCojwgu/A9tiOm3V8IJO2gD 2WFsKODIro5PDArxwpM9sRPDuc4y22TzZ2lVCVmrg3aTnEgLkK7UgcA/X5g+Du3XD4rRNJ CEzT6JLJG8Olar9rBZlv8C0mH7Q6EHQ= ARC-Authentication-Results: i=1; imf09.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b="o/PzqoW7"; dmarc=pass (policy=quarantine) header.from=kernel.org; spf=pass (imf09.hostedemail.com: domain of andrii@kernel.org designates 145.40.68.75 as permitted sender) smtp.mailfrom=andrii@kernel.org Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by ams.source.kernel.org (Postfix) with ESMTP id 0BCAAAE089A; Thu, 29 Aug 2024 17:42:56 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id C5B75C4CEC5; Thu, 29 Aug 2024 17:43:00 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1724953381; bh=xyGq5opgp+BqdKV8FxoUZpF+NuXZlsagAy54RWJaSuw=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=o/PzqoW7INNxwdo+fytTAWkWzKGO8/uy0SFJKvvcPHhhsz5O2HhqxHA4KI+/GhPQz Bn9QmPVSB7OugoetJuA8uo6ECiphr+3cFzGxSmODcqdcTuksQqL7wRgbXXG33IBHoy zt/HZlHJvmQgrVOFFFTmq737JXTGh4TIPW+1qWjCObdKP954I7xiY05+Iv8YIHyyk3 NKMX3wdvOKwSDRW7xewHZZ8JtQZR+0JFawkOFN1CMYubkW08RwWMCWHkVFT6ganQ9G h+fven9XVydBH8J6AEXF+5brs2SdRrtL6KVG89VKJZS6roxKuBTfVfj3MzdMxgEzss Ek7rLSRjqNlMQ== From: Andrii Nakryiko To: bpf@vger.kernel.org Cc: linux-mm@kvack.org, akpm@linux-foundation.org, adobriyan@gmail.com, shakeel.butt@linux.dev, hannes@cmpxchg.org, ak@linux.intel.com, osandov@osandov.com, song@kernel.org, jannh@google.com, linux-fsdevel@vger.kernel.org, willy@infradead.org, Andrii Nakryiko , Eduard Zingerman Subject: [PATCH v7 bpf-next 08/10] bpf: decouple stack_map_get_build_id_offset() from perf_callchain_entry Date: Thu, 29 Aug 2024 10:42:30 -0700 Message-ID: <20240829174232.3133883-9-andrii@kernel.org> X-Mailer: git-send-email 2.43.5 In-Reply-To: <20240829174232.3133883-1-andrii@kernel.org> References: <20240829174232.3133883-1-andrii@kernel.org> MIME-Version: 1.0 X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: 09F05140023 X-Stat-Signature: tsiywjrdqwxzz79s5ibjknchrgi4q36i X-Rspam-User: X-HE-Tag: 1724953382-332328 X-HE-Meta: U2FsdGVkX18PIr0wMhXGX1F1IP/GhQ6H8XaHuRInywFQvdYJakWAmeaBCTq24Pwdf1v813wpRaGjvR8Lhy7c0qQPBUzOgzm5KB3Qpo3DCvCTQuQdw/7ERGqOWFOmX2E56B2rTL/GXJ3rrfpzz/dXiC4F5r0/2Wi3KaddywLF3Uqq7GOuvlQ8BSkQYKqN2vlIDTakyA3DZmLdhKbWVljNaI5eSHf3Pm/Eug+Dnk6xNr4mEo/GdPDFiwm/cPJtxD5QAbj16tfi8ovHgf1BkH5y6waCIx/dd2rdtdPHG9nKAhoqk3JD1b2wkI+4HSAWI6LrSZeJo6fM+CNTkneDSvJcLXZ5Z3IxdWwjt5+7Go1hVcP5PYsYYKmrPn4LP43oS5uAmwBD6tZoQ7acwSs5NdfwHDUX9NGdhycXebv8Wwx0aQx3a4NaJ+YyANQS7l/2JBIxlICXkQDwNbo7a96OgYRWnydHrmn8+ff8iTOPYIwJX69+rPk46238rXy0FZ1hLo+imQ8BjRoEMoaPElN6zswN1WmwUaiIU5I1JWw4adwQhBMYARsxxgirr/PwDL49sccBMK9URxbOF9TJptx/R4JboJIihik5weKsYdJiXhrpx8iVd/86siof2gS5vuBjnogOQDGkhEpnJERaTtsJ38I7xV0hMOmJVIOsPKYGf1mlStPYjzaOvFtbds01yD6DUKXsGcLHF1mxaXYj027iV8UXiO4cm7ABKoBpI+XKtriRQ/6iLMAfbUZavmny8d/kZaz8QqarcIHNkn2xhyLkWT0JvwQNE5uUOCBIdP1YQkAJaCxp+4UfHPx4/g6mZIozX2YGiX/Yu/EIjFmvxBjFQrOMPbcp5RUw8/Ajvak81dqq/RtrqJgCk7lairoorAVL/O23xTXotSG3X57fjgTkduCnbp6WVMk1lDfV1+0zN7kbcDsAm4DARq7+68ksbFq1HmfaDonVUzQPYjT72H9EucB TYGP4Ysm lWGVeXvLrfWAGBVKbhbuNpMUls1j0STYALx0XIVQbwerD49wwDJ1i3s1QSNmbpKHc5bEmcjwA+kiwqiyEcHvw+rcBO/UedqwoUdxb8IH1XTnEYjW9kpISPqm7mtZ4AVMspKkXAZWb8arCE5HqkA14IEvWUe6/7wNID5HFi149vtSMFek5o27Wo85oZ5HsKEwsuR3EFNNHhO/cDU79cwHLpvv074va813H0OMeGLPmKrfx3Z0pJiqVuyHugqmr4Hd0T5M5Z92U1wy09Yc6Ks7v59gCfHbQCD0Wvn7mMfsWdHyhZJSJWuRQXSHVFg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Change stack_map_get_build_id_offset() which is used to convert stack trace IP addresses into build ID+offset pairs. Right now this function accepts an array of u64s as an input, and uses array of struct bpf_stack_build_id as an output. This is problematic because u64 array is coming from perf_callchain_entry, which is (non-sleepable) RCU protected, so once we allows sleepable build ID fetching, this all breaks down. But its actually pretty easy to make stack_map_get_build_id_offset() works with array of struct bpf_stack_build_id as both input and output. Which is what this patch is doing, eliminating the dependency on perf_callchain_entry. We require caller to fill out bpf_stack_build_id.ip fields (all other can be left uninitialized), and update in place as we do build ID resolution. We make sure to READ_ONCE() and cache locally current IP value as we used it in a few places to find matching VMA and so on. Given this data is directly accessible and modifiable by user's BPF code, we should make sure to have a consistent view of it. Reviewed-by: Eduard Zingerman Signed-off-by: Andrii Nakryiko --- kernel/bpf/stackmap.c | 49 +++++++++++++++++++++++++++++-------------- 1 file changed, 33 insertions(+), 16 deletions(-) diff --git a/kernel/bpf/stackmap.c b/kernel/bpf/stackmap.c index 770ae8e88016..6457222b0b46 100644 --- a/kernel/bpf/stackmap.c +++ b/kernel/bpf/stackmap.c @@ -124,8 +124,18 @@ static struct bpf_map *stack_map_alloc(union bpf_attr *attr) return ERR_PTR(err); } +/* + * Expects all id_offs[i].ip values to be set to correct initial IPs. + * They will be subsequently: + * - either adjusted in place to a file offset, if build ID fetching + * succeeds; in this case id_offs[i].build_id is set to correct build ID, + * and id_offs[i].status is set to BPF_STACK_BUILD_ID_VALID; + * - or IP will be kept intact, if build ID fetching failed; in this case + * id_offs[i].build_id is zeroed out and id_offs[i].status is set to + * BPF_STACK_BUILD_ID_IP. + */ static void stack_map_get_build_id_offset(struct bpf_stack_build_id *id_offs, - u64 *ips, u32 trace_nr, bool user) + u32 trace_nr, bool user) { int i; struct mmap_unlock_irq_work *work = NULL; @@ -142,30 +152,28 @@ static void stack_map_get_build_id_offset(struct bpf_stack_build_id *id_offs, /* cannot access current->mm, fall back to ips */ for (i = 0; i < trace_nr; i++) { id_offs[i].status = BPF_STACK_BUILD_ID_IP; - id_offs[i].ip = ips[i]; memset(id_offs[i].build_id, 0, BUILD_ID_SIZE_MAX); } return; } for (i = 0; i < trace_nr; i++) { - if (range_in_vma(prev_vma, ips[i], ips[i])) { + u64 ip = READ_ONCE(id_offs[i].ip); + + if (range_in_vma(prev_vma, ip, ip)) { vma = prev_vma; - memcpy(id_offs[i].build_id, prev_build_id, - BUILD_ID_SIZE_MAX); + memcpy(id_offs[i].build_id, prev_build_id, BUILD_ID_SIZE_MAX); goto build_id_valid; } - vma = find_vma(current->mm, ips[i]); + vma = find_vma(current->mm, ip); if (!vma || build_id_parse_nofault(vma, id_offs[i].build_id, NULL)) { /* per entry fall back to ips */ id_offs[i].status = BPF_STACK_BUILD_ID_IP; - id_offs[i].ip = ips[i]; memset(id_offs[i].build_id, 0, BUILD_ID_SIZE_MAX); continue; } build_id_valid: - id_offs[i].offset = (vma->vm_pgoff << PAGE_SHIFT) + ips[i] - - vma->vm_start; + id_offs[i].offset = (vma->vm_pgoff << PAGE_SHIFT) + ip - vma->vm_start; id_offs[i].status = BPF_STACK_BUILD_ID_VALID; prev_vma = vma; prev_build_id = id_offs[i].build_id; @@ -216,7 +224,7 @@ static long __bpf_get_stackid(struct bpf_map *map, struct bpf_stack_map *smap = container_of(map, struct bpf_stack_map, map); struct stack_map_bucket *bucket, *new_bucket, *old_bucket; u32 skip = flags & BPF_F_SKIP_FIELD_MASK; - u32 hash, id, trace_nr, trace_len; + u32 hash, id, trace_nr, trace_len, i; bool user = flags & BPF_F_USER_STACK; u64 *ips; bool hash_matches; @@ -238,15 +246,18 @@ static long __bpf_get_stackid(struct bpf_map *map, return id; if (stack_map_use_build_id(map)) { + struct bpf_stack_build_id *id_offs; + /* for build_id+offset, pop a bucket before slow cmp */ new_bucket = (struct stack_map_bucket *) pcpu_freelist_pop(&smap->freelist); if (unlikely(!new_bucket)) return -ENOMEM; new_bucket->nr = trace_nr; - stack_map_get_build_id_offset( - (struct bpf_stack_build_id *)new_bucket->data, - ips, trace_nr, user); + id_offs = (struct bpf_stack_build_id *)new_bucket->data; + for (i = 0; i < trace_nr; i++) + id_offs[i].ip = ips[i]; + stack_map_get_build_id_offset(id_offs, trace_nr, user); trace_len = trace_nr * sizeof(struct bpf_stack_build_id); if (hash_matches && bucket->nr == trace_nr && memcmp(bucket->data, new_bucket->data, trace_len) == 0) { @@ -445,10 +456,16 @@ static long __bpf_get_stack(struct pt_regs *regs, struct task_struct *task, copy_len = trace_nr * elem_size; ips = trace->ip + skip; - if (user && user_build_id) - stack_map_get_build_id_offset(buf, ips, trace_nr, user); - else + if (user && user_build_id) { + struct bpf_stack_build_id *id_offs = buf; + u32 i; + + for (i = 0; i < trace_nr; i++) + id_offs[i].ip = ips[i]; + stack_map_get_build_id_offset(buf, trace_nr, user); + } else { memcpy(buf, ips, copy_len); + } if (size > copy_len) memset(buf + copy_len, 0, size - copy_len); From patchwork Thu Aug 29 17:42:31 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrii Nakryiko X-Patchwork-Id: 13783519 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9EDE0C87FCD for ; Thu, 29 Aug 2024 17:43:11 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 322946B00BB; Thu, 29 Aug 2024 13:43:11 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 280226B00BC; Thu, 29 Aug 2024 13:43:11 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 0AC3A6B00BD; Thu, 29 Aug 2024 13:43:11 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id DD9FB6B00BB for ; Thu, 29 Aug 2024 13:43:10 -0400 (EDT) Received: from smtpin30.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 921C4C10BC for ; Thu, 29 Aug 2024 17:43:10 +0000 (UTC) X-FDA: 82506004140.30.C6973FC Received: from sin.source.kernel.org (sin.source.kernel.org [145.40.73.55]) by imf24.hostedemail.com (Postfix) with ESMTP id 3EB2518002B for ; Thu, 29 Aug 2024 17:43:07 +0000 (UTC) Authentication-Results: imf24.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=hs1MIpnt; spf=pass (imf24.hostedemail.com: domain of andrii@kernel.org designates 145.40.73.55 as permitted sender) smtp.mailfrom=andrii@kernel.org; dmarc=pass (policy=quarantine) header.from=kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1724953299; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=25UArbly4WmkaZG/XTF6s+mH82SPPcMqL+Ukxg5kEzE=; b=Krjbt/ntKb8XclyIuB2yyyw0K8JCgoE2sFmXtvW2nbymtE2O34PWabe1nu7eVBhdPhupmz yXTWRpuXxTAN/OeursnSoSrGW6q0AuGqItz7Y1vQP0KKY3LxJ/YdGz7W92zogza3i6MWAY qOAuTG2hLSS9ePRUeFtar+NCcGvbeMs= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1724953299; a=rsa-sha256; cv=none; b=gxzYryx4Dn/kN83IZQvF5wNGlwTyjQNxX6FPZL4WvDjpmsyxqkdgZRoeT7rMjz9S+i3Mrc 2qwjhSk59WcDnpLHillYecQnm5vt33E6N8rofn8gOD4u08xpsbNlFF2Ixs4DwuaRl5dzd6 4hxhYOkHPrbnkF0yPPCxaz/lGSXuhw8= ARC-Authentication-Results: i=1; imf24.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=hs1MIpnt; spf=pass (imf24.hostedemail.com: domain of andrii@kernel.org designates 145.40.73.55 as permitted sender) smtp.mailfrom=andrii@kernel.org; dmarc=pass (policy=quarantine) header.from=kernel.org Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by sin.source.kernel.org (Postfix) with ESMTP id 1B42ECE1C17; Thu, 29 Aug 2024 17:43:05 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 15F58C4CEC2; Thu, 29 Aug 2024 17:43:04 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1724953384; bh=tGQF6ovExCYSlrHl8E/yX57XV5uTWFexTl5fBsGRDU8=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=hs1MIpntkQvpwQlU4c4x/diHzQWvWgf0T2Hph5+25K3ukHIA4ZriED3RvPn85UR9n WSBmUgv/YZQmQTAtkrEVripIbuTGfeg9qZqarlYKyuglBX0SEeIxcj9eSbQbdVoo2Q o4Ncqm7lZ3RQOVDe3criRIrtuOo2QN/i/mq2NFZbEwNj/9SDNGtNESL1FKizvRi6t3 BYe+V74G/1Jsr4ehgUslBYQhlxphvWpzDOMB4AompZRIIGQlUdChn6AI5lE7Wyljrg tq3AFThkKozgOyHQLsbyGFMJHspsldH36OrgzRK9mRxLoTGOaeM2ycpbJIiYbtNZVy HPAmbSACRAhAA== From: Andrii Nakryiko To: bpf@vger.kernel.org Cc: linux-mm@kvack.org, akpm@linux-foundation.org, adobriyan@gmail.com, shakeel.butt@linux.dev, hannes@cmpxchg.org, ak@linux.intel.com, osandov@osandov.com, song@kernel.org, jannh@google.com, linux-fsdevel@vger.kernel.org, willy@infradead.org, Andrii Nakryiko , Eduard Zingerman Subject: [PATCH v7 bpf-next 09/10] bpf: wire up sleepable bpf_get_stack() and bpf_get_task_stack() helpers Date: Thu, 29 Aug 2024 10:42:31 -0700 Message-ID: <20240829174232.3133883-10-andrii@kernel.org> X-Mailer: git-send-email 2.43.5 In-Reply-To: <20240829174232.3133883-1-andrii@kernel.org> References: <20240829174232.3133883-1-andrii@kernel.org> MIME-Version: 1.0 X-Stat-Signature: 8a6dzpc4qjtidfd5ysprgrioqkzbxzae X-Rspamd-Queue-Id: 3EB2518002B X-Rspam-User: X-Rspamd-Server: rspam08 X-HE-Tag: 1724953387-263866 X-HE-Meta: U2FsdGVkX18ohxgyEkxXuVl44NA1Q/ZAFhC0jPqmzpTb25Cnx9aDDmbrYW2faUiZv45wxkC5ftYn95FlK2w+Q4K1fWj2DUnJmuLea+Fa1CiERKcAc7cQIg3wy7kSXtgesezRG84ojE808l18YYVuJZKBo+ULvbA2KkW2ZytVsthyKccN/w4a293JGqVr8AGUpNUICSemUVfO9Ug/j7qa8oCzeZ4YsToObBelveCVxzt3Xem0kfdzYdrNVu5HBb2zgDLiGUDucFg+D6Jc3qrZDTAe+31qht5pmFJtLQqJ0hhVU4bU5tqAcJtMzx4jlkIzSoRePWoO81JlKdOEzamost5G475zhPPSkFPrhKg8pvc71/bTPXU0AEzPXkpOIdfi2GWEfqOxGtEpjktafTyEYaMrv+dFEtBjvqyi4UFuQjdNEDO7TguR1ea1An9BpHi0ml/blK+tH9AsLs6vxNS6GNa0abxyX3kBg8DLVb/7nzZOnsC6ixZkvIxo3vxUZkF1EwNzadcCvbI1Ps4S/d+vPS0UXbUCoQCiwbBq63mYXER17sJd8n8lk1QNFZRtMbkrQiNtTckGtRFEQOT7Bd60wuOBrWchyxALgoLKfdwwve3qID0PG9CWKC/W76JJNtkxgkfygyBD7qKXrJm+aiRdOqL5xalAdDqrnPWjanju7tEjdJ1w/gMZU7RxqnIGgZUKgg6Fk9odDsn92s5JmvwHb7Wr4i2wY423gtvpPjrMI8BeXNWqAbCfTGdj5vn/C5eIbhT8aKN1iS29g5nW/RkX5+Fq58nfyBQo/QBYSdwznDhHucSrAQrQDvZMDytMfngIbjAa7biKojhvX8bz/de3Nzzm6ot8yJ/w2vu4HAOfAexAqF+wOOty0MK7njawkZ7rfkGeGy72hZflVJJmd3n3958r+vhFd+2GfNKpMBmNQoF7CjL7fK6agUG4uj7xejnBv73wW4N6Kf5i5Iz9yJY kXiAZ+PY AbGe2tZN164HXG0bCth9OwnIHopqFocmyaIglfpIhfrjjCKfRzPzzyazlGhQpekm35HSwhLQ4UTxARiWQ0G6JK/A5MWC2JHTr4+meKrpqgEBekELLEazN3WzQoY0Q405s1OpF8VTccl6leptQD2mJISXsvv+dwwzj8ZsJcOEkGavcIdMGF+H8oYDgihGUjXmJunU8PDjI0KnLDUbhGBLckB26dakLYD7xhEbTqxffvnWj0n/W/9q6WwOd7LVGpN3S+T/isZSCjgIyUmtDDCxPpVbfk6AjwA0FCxtxrRtolWCPgySecqbNHEXYkw== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Add sleepable implementations of bpf_get_stack() and bpf_get_task_stack() helpers and allow them to be used from sleepable BPF program (e.g., sleepable uprobes). Note, the stack trace IPs capturing itself is not sleepable (that would need to be a separate project), only build ID fetching is sleepable and thus more reliable, as it will wait for data to be paged in, if necessary. For that we make use of sleepable build_id_parse() implementation. Now that build ID related internals in kernel/bpf/stackmap.c can be used both in sleepable and non-sleepable contexts, we need to add additional rcu_read_lock()/rcu_read_unlock() protection around fetching perf_callchain_entry, but with the refactoring in previous commit it's now pretty straightforward. We make sure to do rcu_read_unlock (in sleepable mode only) right before stack_map_get_build_id_offset() call which can sleep. By that time we don't have any more use of perf_callchain_entry. Note, bpf_get_task_stack() will fail for user mode if task != current. And for kernel mode build ID are irrelevant. So in that sense adding sleepable bpf_get_task_stack() implementation is a no-op. It feel right to wire this up for symmetry and completeness, but I'm open to just dropping it until we support `user && crosstask` condition. Reviewed-by: Eduard Zingerman Signed-off-by: Andrii Nakryiko --- include/linux/bpf.h | 2 + kernel/bpf/stackmap.c | 90 ++++++++++++++++++++++++++++++++-------- kernel/trace/bpf_trace.c | 5 ++- 3 files changed, 77 insertions(+), 20 deletions(-) diff --git a/include/linux/bpf.h b/include/linux/bpf.h index dc63083f76b7..f3b0083dd29b 100644 --- a/include/linux/bpf.h +++ b/include/linux/bpf.h @@ -3198,7 +3198,9 @@ extern const struct bpf_func_proto bpf_get_current_uid_gid_proto; extern const struct bpf_func_proto bpf_get_current_comm_proto; extern const struct bpf_func_proto bpf_get_stackid_proto; extern const struct bpf_func_proto bpf_get_stack_proto; +extern const struct bpf_func_proto bpf_get_stack_sleepable_proto; extern const struct bpf_func_proto bpf_get_task_stack_proto; +extern const struct bpf_func_proto bpf_get_task_stack_sleepable_proto; extern const struct bpf_func_proto bpf_get_stackid_proto_pe; extern const struct bpf_func_proto bpf_get_stack_proto_pe; extern const struct bpf_func_proto bpf_sock_map_update_proto; diff --git a/kernel/bpf/stackmap.c b/kernel/bpf/stackmap.c index 6457222b0b46..3615c06b7dfa 100644 --- a/kernel/bpf/stackmap.c +++ b/kernel/bpf/stackmap.c @@ -124,6 +124,12 @@ static struct bpf_map *stack_map_alloc(union bpf_attr *attr) return ERR_PTR(err); } +static int fetch_build_id(struct vm_area_struct *vma, unsigned char *build_id, bool may_fault) +{ + return may_fault ? build_id_parse(vma, build_id, NULL) + : build_id_parse_nofault(vma, build_id, NULL); +} + /* * Expects all id_offs[i].ip values to be set to correct initial IPs. * They will be subsequently: @@ -135,7 +141,7 @@ static struct bpf_map *stack_map_alloc(union bpf_attr *attr) * BPF_STACK_BUILD_ID_IP. */ static void stack_map_get_build_id_offset(struct bpf_stack_build_id *id_offs, - u32 trace_nr, bool user) + u32 trace_nr, bool user, bool may_fault) { int i; struct mmap_unlock_irq_work *work = NULL; @@ -166,7 +172,7 @@ static void stack_map_get_build_id_offset(struct bpf_stack_build_id *id_offs, goto build_id_valid; } vma = find_vma(current->mm, ip); - if (!vma || build_id_parse_nofault(vma, id_offs[i].build_id, NULL)) { + if (!vma || fetch_build_id(vma, id_offs[i].build_id, may_fault)) { /* per entry fall back to ips */ id_offs[i].status = BPF_STACK_BUILD_ID_IP; memset(id_offs[i].build_id, 0, BUILD_ID_SIZE_MAX); @@ -257,7 +263,7 @@ static long __bpf_get_stackid(struct bpf_map *map, id_offs = (struct bpf_stack_build_id *)new_bucket->data; for (i = 0; i < trace_nr; i++) id_offs[i].ip = ips[i]; - stack_map_get_build_id_offset(id_offs, trace_nr, user); + stack_map_get_build_id_offset(id_offs, trace_nr, user, false /* !may_fault */); trace_len = trace_nr * sizeof(struct bpf_stack_build_id); if (hash_matches && bucket->nr == trace_nr && memcmp(bucket->data, new_bucket->data, trace_len) == 0) { @@ -398,7 +404,7 @@ const struct bpf_func_proto bpf_get_stackid_proto_pe = { static long __bpf_get_stack(struct pt_regs *regs, struct task_struct *task, struct perf_callchain_entry *trace_in, - void *buf, u32 size, u64 flags) + void *buf, u32 size, u64 flags, bool may_fault) { u32 trace_nr, copy_len, elem_size, num_elem, max_depth; bool user_build_id = flags & BPF_F_USER_BUILD_ID; @@ -416,8 +422,7 @@ static long __bpf_get_stack(struct pt_regs *regs, struct task_struct *task, if (kernel && user_build_id) goto clear; - elem_size = (user && user_build_id) ? sizeof(struct bpf_stack_build_id) - : sizeof(u64); + elem_size = user_build_id ? sizeof(struct bpf_stack_build_id) : sizeof(u64); if (unlikely(size % elem_size)) goto clear; @@ -438,6 +443,9 @@ static long __bpf_get_stack(struct pt_regs *regs, struct task_struct *task, if (sysctl_perf_event_max_stack < max_depth) max_depth = sysctl_perf_event_max_stack; + if (may_fault) + rcu_read_lock(); /* need RCU for perf's callchain below */ + if (trace_in) trace = trace_in; else if (kernel && task) @@ -445,28 +453,35 @@ static long __bpf_get_stack(struct pt_regs *regs, struct task_struct *task, else trace = get_perf_callchain(regs, 0, kernel, user, max_depth, crosstask, false); - if (unlikely(!trace)) - goto err_fault; - if (trace->nr < skip) + if (unlikely(!trace) || trace->nr < skip) { + if (may_fault) + rcu_read_unlock(); goto err_fault; + } trace_nr = trace->nr - skip; trace_nr = (trace_nr <= num_elem) ? trace_nr : num_elem; copy_len = trace_nr * elem_size; ips = trace->ip + skip; - if (user && user_build_id) { + if (user_build_id) { struct bpf_stack_build_id *id_offs = buf; u32 i; for (i = 0; i < trace_nr; i++) id_offs[i].ip = ips[i]; - stack_map_get_build_id_offset(buf, trace_nr, user); } else { memcpy(buf, ips, copy_len); } + /* trace/ips should not be dereferenced after this point */ + if (may_fault) + rcu_read_unlock(); + + if (user_build_id) + stack_map_get_build_id_offset(buf, trace_nr, user, may_fault); + if (size > copy_len) memset(buf + copy_len, 0, size - copy_len); return copy_len; @@ -481,7 +496,7 @@ static long __bpf_get_stack(struct pt_regs *regs, struct task_struct *task, BPF_CALL_4(bpf_get_stack, struct pt_regs *, regs, void *, buf, u32, size, u64, flags) { - return __bpf_get_stack(regs, NULL, NULL, buf, size, flags); + return __bpf_get_stack(regs, NULL, NULL, buf, size, flags, false /* !may_fault */); } const struct bpf_func_proto bpf_get_stack_proto = { @@ -494,8 +509,24 @@ const struct bpf_func_proto bpf_get_stack_proto = { .arg4_type = ARG_ANYTHING, }; -BPF_CALL_4(bpf_get_task_stack, struct task_struct *, task, void *, buf, - u32, size, u64, flags) +BPF_CALL_4(bpf_get_stack_sleepable, struct pt_regs *, regs, void *, buf, u32, size, + u64, flags) +{ + return __bpf_get_stack(regs, NULL, NULL, buf, size, flags, true /* may_fault */); +} + +const struct bpf_func_proto bpf_get_stack_sleepable_proto = { + .func = bpf_get_stack_sleepable, + .gpl_only = true, + .ret_type = RET_INTEGER, + .arg1_type = ARG_PTR_TO_CTX, + .arg2_type = ARG_PTR_TO_UNINIT_MEM, + .arg3_type = ARG_CONST_SIZE_OR_ZERO, + .arg4_type = ARG_ANYTHING, +}; + +static long __bpf_get_task_stack(struct task_struct *task, void *buf, u32 size, + u64 flags, bool may_fault) { struct pt_regs *regs; long res = -EINVAL; @@ -505,12 +536,18 @@ BPF_CALL_4(bpf_get_task_stack, struct task_struct *, task, void *, buf, regs = task_pt_regs(task); if (regs) - res = __bpf_get_stack(regs, task, NULL, buf, size, flags); + res = __bpf_get_stack(regs, task, NULL, buf, size, flags, may_fault); put_task_stack(task); return res; } +BPF_CALL_4(bpf_get_task_stack, struct task_struct *, task, void *, buf, + u32, size, u64, flags) +{ + return __bpf_get_task_stack(task, buf, size, flags, false /* !may_fault */); +} + const struct bpf_func_proto bpf_get_task_stack_proto = { .func = bpf_get_task_stack, .gpl_only = false, @@ -522,6 +559,23 @@ const struct bpf_func_proto bpf_get_task_stack_proto = { .arg4_type = ARG_ANYTHING, }; +BPF_CALL_4(bpf_get_task_stack_sleepable, struct task_struct *, task, void *, buf, + u32, size, u64, flags) +{ + return __bpf_get_task_stack(task, buf, size, flags, true /* !may_fault */); +} + +const struct bpf_func_proto bpf_get_task_stack_sleepable_proto = { + .func = bpf_get_task_stack_sleepable, + .gpl_only = false, + .ret_type = RET_INTEGER, + .arg1_type = ARG_PTR_TO_BTF_ID, + .arg1_btf_id = &btf_tracing_ids[BTF_TRACING_TYPE_TASK], + .arg2_type = ARG_PTR_TO_UNINIT_MEM, + .arg3_type = ARG_CONST_SIZE_OR_ZERO, + .arg4_type = ARG_ANYTHING, +}; + BPF_CALL_4(bpf_get_stack_pe, struct bpf_perf_event_data_kern *, ctx, void *, buf, u32, size, u64, flags) { @@ -533,7 +587,7 @@ BPF_CALL_4(bpf_get_stack_pe, struct bpf_perf_event_data_kern *, ctx, __u64 nr_kernel; if (!(event->attr.sample_type & PERF_SAMPLE_CALLCHAIN)) - return __bpf_get_stack(regs, NULL, NULL, buf, size, flags); + return __bpf_get_stack(regs, NULL, NULL, buf, size, flags, false /* !may_fault */); if (unlikely(flags & ~(BPF_F_SKIP_FIELD_MASK | BPF_F_USER_STACK | BPF_F_USER_BUILD_ID))) @@ -553,7 +607,7 @@ BPF_CALL_4(bpf_get_stack_pe, struct bpf_perf_event_data_kern *, ctx, __u64 nr = trace->nr; trace->nr = nr_kernel; - err = __bpf_get_stack(regs, NULL, trace, buf, size, flags); + err = __bpf_get_stack(regs, NULL, trace, buf, size, flags, false /* !may_fault */); /* restore nr */ trace->nr = nr; @@ -565,7 +619,7 @@ BPF_CALL_4(bpf_get_stack_pe, struct bpf_perf_event_data_kern *, ctx, goto clear; flags = (flags & ~BPF_F_SKIP_FIELD_MASK) | skip; - err = __bpf_get_stack(regs, NULL, trace, buf, size, flags); + err = __bpf_get_stack(regs, NULL, trace, buf, size, flags, false /* !may_fault */); } return err; diff --git a/kernel/trace/bpf_trace.c b/kernel/trace/bpf_trace.c index b69a39316c0c..81a94130d873 100644 --- a/kernel/trace/bpf_trace.c +++ b/kernel/trace/bpf_trace.c @@ -1507,7 +1507,8 @@ bpf_tracing_func_proto(enum bpf_func_id func_id, const struct bpf_prog *prog) case BPF_FUNC_jiffies64: return &bpf_jiffies64_proto; case BPF_FUNC_get_task_stack: - return &bpf_get_task_stack_proto; + return prog->sleepable ? &bpf_get_task_stack_sleepable_proto + : &bpf_get_task_stack_proto; case BPF_FUNC_copy_from_user: return &bpf_copy_from_user_proto; case BPF_FUNC_copy_from_user_task: @@ -1563,7 +1564,7 @@ kprobe_prog_func_proto(enum bpf_func_id func_id, const struct bpf_prog *prog) case BPF_FUNC_get_stackid: return &bpf_get_stackid_proto; case BPF_FUNC_get_stack: - return &bpf_get_stack_proto; + return prog->sleepable ? &bpf_get_stack_sleepable_proto : &bpf_get_stack_proto; #ifdef CONFIG_BPF_KPROBE_OVERRIDE case BPF_FUNC_override_return: return &bpf_override_return_proto; From patchwork Thu Aug 29 17:42:32 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrii Nakryiko X-Patchwork-Id: 13783520 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id A3E0DC8303E for ; Thu, 29 Aug 2024 17:43:13 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 14B356B00BC; Thu, 29 Aug 2024 13:43:12 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 0D3306B00BD; Thu, 29 Aug 2024 13:43:12 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id DA8616B00BE; Thu, 29 Aug 2024 13:43:11 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id B4DB86B00BC for ; Thu, 29 Aug 2024 13:43:11 -0400 (EDT) Received: from smtpin30.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 73E55AACD9 for ; Thu, 29 Aug 2024 17:43:11 +0000 (UTC) X-FDA: 82506004182.30.2F2FAFF Received: from ams.source.kernel.org (ams.source.kernel.org [145.40.68.75]) by imf13.hostedemail.com (Postfix) with ESMTP id 9918720008 for ; Thu, 29 Aug 2024 17:43:09 +0000 (UTC) Authentication-Results: imf13.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b="pCQ/coWl"; spf=pass (imf13.hostedemail.com: domain of andrii@kernel.org designates 145.40.68.75 as permitted sender) smtp.mailfrom=andrii@kernel.org; dmarc=pass (policy=quarantine) header.from=kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1724953345; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=SdNzVe4DBSRlAJo6CNeiMMT/ri86V36cdYA1xdK86yM=; b=up5HFyYS0H6M7cAFDClPrGdh+3gyGvDCFvMEnYZfd/gw/E/vHTnVegL9YqqzXIHeQ9Br2c CL6Im+/NNf0t7LnbzEpzdHwx8+6OMPQX6+12bEBCr50tvE9VWYY+GR7WG1FVbcHbvDnXy+ dxXDzdVa+yz9oXNhDjIIl/hpqYDolPc= ARC-Authentication-Results: i=1; imf13.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b="pCQ/coWl"; spf=pass (imf13.hostedemail.com: domain of andrii@kernel.org designates 145.40.68.75 as permitted sender) smtp.mailfrom=andrii@kernel.org; dmarc=pass (policy=quarantine) header.from=kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1724953345; a=rsa-sha256; cv=none; b=214NBtXR7vM4txkEhVIryRPWtZbJcAn5jZPoqE/rBzUiezhonkDG6WRLgd7CRSWyW7YxI7 dnF40USBsgYLT/gsoqUSywa5EVNoZuBHwIzCHvbyU1okKl1JsB7KhbCzvxWccyAVmnfe1K w/gLyNwQUCVP3u2dOAGpZocBEu/rxV4= Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by ams.source.kernel.org (Postfix) with ESMTP id 93CE4AE2CAD; Thu, 29 Aug 2024 17:43:02 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 5D9EBC4CEC1; Thu, 29 Aug 2024 17:43:07 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1724953387; bh=X97j92yUSN/YIJFWooWtS6FnLn2/foHfKK8DgE5pkPM=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=pCQ/coWlxELV2e9dG7DPTFDkSYBOCl764rRVpgXaomvACgIEgQLQ1I+HSWLCvbP3j buvAtEDakO9xIZ7ovIhxaSxf98o97QsyJ0gDB8kII0tjRGpWlbUpJFAY4rD3TiO5u5 1sI9b6y30lbZscxftSg3sk4TF7YaberGU32Xi2YeQF88aMEgB7B6t3UT910+p31/HI q+vRTmkWfnNwojKGeKDdmTXh05T57BKtEE9Xf9AA0XtPSYaeksZNPJDaKZrxv9pdT0 5ljSREa6FbOv1qFsJiv+4JLMB7KebgayPM3cUlQEZHYpudK7ZpsR4/Jmh0I4C/wvTr Wq7vI1f3cZFzA== From: Andrii Nakryiko To: bpf@vger.kernel.org Cc: linux-mm@kvack.org, akpm@linux-foundation.org, adobriyan@gmail.com, shakeel.butt@linux.dev, hannes@cmpxchg.org, ak@linux.intel.com, osandov@osandov.com, song@kernel.org, jannh@google.com, linux-fsdevel@vger.kernel.org, willy@infradead.org, Andrii Nakryiko , Eduard Zingerman Subject: [PATCH v7 bpf-next 10/10] selftests/bpf: add build ID tests Date: Thu, 29 Aug 2024 10:42:32 -0700 Message-ID: <20240829174232.3133883-11-andrii@kernel.org> X-Mailer: git-send-email 2.43.5 In-Reply-To: <20240829174232.3133883-1-andrii@kernel.org> References: <20240829174232.3133883-1-andrii@kernel.org> MIME-Version: 1.0 X-Rspamd-Server: rspam03 X-Rspam-User: X-Rspamd-Queue-Id: 9918720008 X-Stat-Signature: fdwuxudtuut97zh51nwd61rjyprp1pqc X-HE-Tag: 1724953389-289374 X-HE-Meta: U2FsdGVkX1/+aGYqz2ZYg490PYLAlerNPy82Z5vExzHT6fmsYEGInN1EThoMn/iih8cSfvu58k3QoDIKHzaUrQh0bSdpxFifmXa9WLjYTNGigKdWjVKSCIeFdhRBP11M7vVYpeHZRqJ6iNkrYTJlY4cqMiQVOOdn8fQsgADcrjIXy1wzFhfdvBG2hprADV2vGwDliQbMaE6VBDSgDQn0WO48wQ/m6+VjBONNxSv5gI/MFiZ7hvZIjWU/1uCyN8Uh2rGG0WnWnN7BDYKrZREv03QWtgAa/hjDBiNyl8isUK817WCfZrp06hOSvm01uR8573H3/fgIIM0/sG2IhG+ESfQeenm8udvh8+We02x5mpNc0U3m5URchEVf7mOiw/ylH7vQN8JxQ/FjWIV4yaGzD4BhiiIcKx5V3aQgP2I2lrPubir/pgX3xn0K6DhtaouPDa5Mv+fMduY8JvtUmRZUUqG0YGiMTczppxwChyvVUQh/L8eySImkSTjnXwBndO6TflHNbdDhgLCo0R9yTv3aKk80irEjkJbICBFH6GPevu7prBlZhHm+sQsKKE7WXvHRriq6RiMlYY4cRaxSbbLZ+SKj5tphMxz8qA+GG4WGq1xlp9BsfyEcYAix7Y7qpsgyNg2IuN9/mv8my1qWiXDpLicx2A/Ov0MQiobFNZB7p9XUMr9nzS0iQCx6o0uKwkjDal9TJi4f6m9JFNNipwwtAzhel6GJKut4cUSbC1GD63kTX/B8VlOjwj2+NGJi6trR5F134sAWnoYm85o0a7ITr19UA5kCTIgf0koscZU5WAgxqF0n1OKOKI/mo70XtjMGz6dEsAmiMVwhZXcFII/kZPFQIadcLpihRsv2R1fnKYeG0ECuEhFEea/azVq6HRjQGpQZGlMwh1Tct9fmgZtgDR6w3AwhOLrM//MmngC0hHFzSAaIdMiDbrG6DTrTwJZ2I+UA4piBKq7fSIdVf5p l5LPNGC+ yJqdY6qI//a5o3lqFY4wRPUzJxJypRUPtn06VMaYxfTVbPb9Ei9gPVwP+5DowQpSEu0P3rvz6ccU0r2M6FH4p2aBYytY1dqyZ2EFpVm5wtDnxT1JVs+bcIaMZiVWjOQtXgNxrlhNRtcMMnH0PHz3R09tl1VsoBb83B5wtUeamPyhGvOlmm5RWF3xdqUF0seLc78m0e99cXX0eYZ86Pq8+qtgni81D9HwvgaHEqHs80yxYWKB3pE8m72Gyk3G8GiVILNN+ X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Add a new set of tests validating behavior of capturing stack traces with build ID. We extend uprobe_multi target binary with ability to trigger uprobe (so that we can capture stack traces from it), but also we allow to force build ID data to be either resident or non-resident in memory (see also a comment about quirks of MADV_PAGEOUT). That way we can validate that in non-sleepable context we won't get build ID (as expected), but with sleepable uprobes we will get that build ID regardless of it being physically present in memory. Also, we add a small add-on linker script which reorders .note.gnu.build-id section and puts it after (big) .text section, putting build ID data outside of the very first page of ELF file. This will test all the relaxations we did in build ID parsing logic in kernel thanks to freader abstraction. Reviewed-by: Eduard Zingerman Signed-off-by: Andrii Nakryiko --- tools/testing/selftests/bpf/Makefile | 5 +- .../selftests/bpf/prog_tests/build_id.c | 118 ++++++++++++++++++ .../selftests/bpf/progs/test_build_id.c | 31 +++++ tools/testing/selftests/bpf/uprobe_multi.c | 41 ++++++ tools/testing/selftests/bpf/uprobe_multi.ld | 11 ++ 5 files changed, 204 insertions(+), 2 deletions(-) create mode 100644 tools/testing/selftests/bpf/prog_tests/build_id.c create mode 100644 tools/testing/selftests/bpf/progs/test_build_id.c create mode 100644 tools/testing/selftests/bpf/uprobe_multi.ld diff --git a/tools/testing/selftests/bpf/Makefile b/tools/testing/selftests/bpf/Makefile index c120617b64ad..4c021eb2e857 100644 --- a/tools/testing/selftests/bpf/Makefile +++ b/tools/testing/selftests/bpf/Makefile @@ -832,9 +832,10 @@ $(OUTPUT)/veristat: $(OUTPUT)/veristat.o # Linking uprobe_multi can fail due to relocation overflows on mips. $(OUTPUT)/uprobe_multi: CFLAGS += $(if $(filter mips, $(ARCH)),-mxgot) -$(OUTPUT)/uprobe_multi: uprobe_multi.c +$(OUTPUT)/uprobe_multi: uprobe_multi.c uprobe_multi.ld $(call msg,BINARY,,$@) - $(Q)$(CC) $(CFLAGS) -O0 $(LDFLAGS) $^ $(LDLIBS) -o $@ + $(Q)$(CC) $(CFLAGS) -Wl,-T,uprobe_multi.ld -O0 $(LDFLAGS) \ + $(filter-out %.ld,$^) $(LDLIBS) -o $@ EXTRA_CLEAN := $(SCRATCH_DIR) $(HOST_SCRATCH_DIR) \ prog_tests/tests.h map_tests/tests.h verifier/tests.h \ diff --git a/tools/testing/selftests/bpf/prog_tests/build_id.c b/tools/testing/selftests/bpf/prog_tests/build_id.c new file mode 100644 index 000000000000..aec9c8d6bc96 --- /dev/null +++ b/tools/testing/selftests/bpf/prog_tests/build_id.c @@ -0,0 +1,118 @@ +// SPDX-License-Identifier: GPL-2.0 +/* Copyright (c) 2024 Meta Platforms, Inc. and affiliates. */ +#include + +#include "test_build_id.skel.h" + +static char build_id[BPF_BUILD_ID_SIZE]; +static int build_id_sz; + +static void print_stack(struct bpf_stack_build_id *stack, int frame_cnt) +{ + int i, j; + + for (i = 0; i < frame_cnt; i++) { + printf("FRAME #%02d: ", i); + switch (stack[i].status) { + case BPF_STACK_BUILD_ID_EMPTY: + printf("\n"); + break; + case BPF_STACK_BUILD_ID_VALID: + printf("BUILD ID = "); + for (j = 0; j < BPF_BUILD_ID_SIZE; j++) + printf("%02hhx", (unsigned)stack[i].build_id[j]); + printf(" OFFSET = %llx", (unsigned long long)stack[i].offset); + break; + case BPF_STACK_BUILD_ID_IP: + printf("IP = %llx", (unsigned long long)stack[i].ip); + break; + default: + printf("UNEXPECTED STATUS %d ", stack[i].status); + break; + } + printf("\n"); + } +} + +static void subtest_nofault(bool build_id_resident) +{ + struct test_build_id *skel; + struct bpf_stack_build_id *stack; + int frame_cnt; + + skel = test_build_id__open_and_load(); + if (!ASSERT_OK_PTR(skel, "skel_open")) + return; + + skel->links.uprobe_nofault = bpf_program__attach(skel->progs.uprobe_nofault); + if (!ASSERT_OK_PTR(skel->links.uprobe_nofault, "link")) + goto cleanup; + + if (build_id_resident) + ASSERT_OK(system("./uprobe_multi uprobe-paged-in"), "trigger_uprobe"); + else + ASSERT_OK(system("./uprobe_multi uprobe-paged-out"), "trigger_uprobe"); + + if (!ASSERT_GT(skel->bss->res_nofault, 0, "res")) + goto cleanup; + + stack = skel->bss->stack_nofault; + frame_cnt = skel->bss->res_nofault / sizeof(struct bpf_stack_build_id); + if (env.verbosity >= VERBOSE_NORMAL) + print_stack(stack, frame_cnt); + + if (build_id_resident) { + ASSERT_EQ(stack[0].status, BPF_STACK_BUILD_ID_VALID, "build_id_status"); + ASSERT_EQ(memcmp(stack[0].build_id, build_id, build_id_sz), 0, "build_id_match"); + } else { + ASSERT_EQ(stack[0].status, BPF_STACK_BUILD_ID_IP, "build_id_status"); + } + +cleanup: + test_build_id__destroy(skel); +} + +static void subtest_sleepable(void) +{ + struct test_build_id *skel; + struct bpf_stack_build_id *stack; + int frame_cnt; + + skel = test_build_id__open_and_load(); + if (!ASSERT_OK_PTR(skel, "skel_open")) + return; + + skel->links.uprobe_sleepable = bpf_program__attach(skel->progs.uprobe_sleepable); + if (!ASSERT_OK_PTR(skel->links.uprobe_sleepable, "link")) + goto cleanup; + + /* force build ID to not be paged in */ + ASSERT_OK(system("./uprobe_multi uprobe-paged-out"), "trigger_uprobe"); + + if (!ASSERT_GT(skel->bss->res_sleepable, 0, "res")) + goto cleanup; + + stack = skel->bss->stack_sleepable; + frame_cnt = skel->bss->res_sleepable / sizeof(struct bpf_stack_build_id); + if (env.verbosity >= VERBOSE_NORMAL) + print_stack(stack, frame_cnt); + + ASSERT_EQ(stack[0].status, BPF_STACK_BUILD_ID_VALID, "build_id_status"); + ASSERT_EQ(memcmp(stack[0].build_id, build_id, build_id_sz), 0, "build_id_match"); + +cleanup: + test_build_id__destroy(skel); +} + +void serial_test_build_id(void) +{ + build_id_sz = read_build_id("uprobe_multi", build_id, sizeof(build_id)); + ASSERT_EQ(build_id_sz, BPF_BUILD_ID_SIZE, "parse_build_id"); + + if (test__start_subtest("nofault-paged-out")) + subtest_nofault(false /* not resident */); + if (test__start_subtest("nofault-paged-in")) + subtest_nofault(true /* resident */); + if (test__start_subtest("sleepable")) + subtest_sleepable(); +} diff --git a/tools/testing/selftests/bpf/progs/test_build_id.c b/tools/testing/selftests/bpf/progs/test_build_id.c new file mode 100644 index 000000000000..32ce59f9aa27 --- /dev/null +++ b/tools/testing/selftests/bpf/progs/test_build_id.c @@ -0,0 +1,31 @@ +// SPDX-License-Identifier: GPL-2.0 +/* Copyright (c) 2024 Meta Platforms, Inc. and affiliates. */ + +#include "vmlinux.h" +#include + +struct bpf_stack_build_id stack_sleepable[128]; +int res_sleepable; + +struct bpf_stack_build_id stack_nofault[128]; +int res_nofault; + +SEC("uprobe.multi/./uprobe_multi:uprobe") +int uprobe_nofault(struct pt_regs *ctx) +{ + res_nofault = bpf_get_stack(ctx, stack_nofault, sizeof(stack_nofault), + BPF_F_USER_STACK | BPF_F_USER_BUILD_ID); + + return 0; +} + +SEC("uprobe.multi.s/./uprobe_multi:uprobe") +int uprobe_sleepable(struct pt_regs *ctx) +{ + res_sleepable = bpf_get_stack(ctx, stack_sleepable, sizeof(stack_sleepable), + BPF_F_USER_STACK | BPF_F_USER_BUILD_ID); + + return 0; +} + +char _license[] SEC("license") = "GPL"; diff --git a/tools/testing/selftests/bpf/uprobe_multi.c b/tools/testing/selftests/bpf/uprobe_multi.c index 7ffa563ffeba..c7828b13e5ff 100644 --- a/tools/testing/selftests/bpf/uprobe_multi.c +++ b/tools/testing/selftests/bpf/uprobe_multi.c @@ -2,8 +2,21 @@ #include #include +#include +#include +#include +#include #include +#ifndef MADV_POPULATE_READ +#define MADV_POPULATE_READ 22 +#endif + +int __attribute__((weak)) uprobe(void) +{ + return 0; +} + #define __PASTE(a, b) a##b #define PASTE(a, b) __PASTE(a, b) @@ -75,6 +88,30 @@ static int usdt(void) return 0; } +extern char build_id_start[]; +extern char build_id_end[]; + +int __attribute__((weak)) trigger_uprobe(bool build_id_resident) +{ + int page_sz = sysconf(_SC_PAGESIZE); + void *addr; + + /* page-align build ID start */ + addr = (void *)((uintptr_t)&build_id_start & ~(page_sz - 1)); + + /* to guarantee MADV_PAGEOUT work reliably, we need to ensure that + * memory range is mapped into current process, so we unconditionally + * do MADV_POPULATE_READ, and then MADV_PAGEOUT, if necessary + */ + madvise(addr, page_sz, MADV_POPULATE_READ); + if (!build_id_resident) + madvise(addr, page_sz, MADV_PAGEOUT); + + (void)uprobe(); + + return 0; +} + int main(int argc, char **argv) { if (argc != 2) @@ -84,6 +121,10 @@ int main(int argc, char **argv) return bench(); if (!strcmp("usdt", argv[1])) return usdt(); + if (!strcmp("uprobe-paged-out", argv[1])) + return trigger_uprobe(false /* page-out build ID */); + if (!strcmp("uprobe-paged-in", argv[1])) + return trigger_uprobe(true /* page-in build ID */); error: fprintf(stderr, "usage: %s \n", argv[0]); diff --git a/tools/testing/selftests/bpf/uprobe_multi.ld b/tools/testing/selftests/bpf/uprobe_multi.ld new file mode 100644 index 000000000000..a2e94828bc8c --- /dev/null +++ b/tools/testing/selftests/bpf/uprobe_multi.ld @@ -0,0 +1,11 @@ +SECTIONS +{ + . = ALIGN(4096); + .note.gnu.build-id : { *(.note.gnu.build-id) } + . = ALIGN(4096); +} +INSERT AFTER .text; + +build_id_start = ADDR(.note.gnu.build-id); +build_id_end = ADDR(.note.gnu.build-id) + SIZEOF(.note.gnu.build-id); +