From patchwork Thu Jul 23 03:46:41 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: changhuaixin X-Patchwork-Id: 11679845 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 947B2138C for ; Thu, 23 Jul 2020 03:46:53 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 842B320737 for ; Thu, 23 Jul 2020 03:46:53 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1725849AbgGWDqx (ORCPT ); Wed, 22 Jul 2020 23:46:53 -0400 Received: from out30-131.freemail.mail.aliyun.com ([115.124.30.131]:52076 "EHLO out30-131.freemail.mail.aliyun.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725774AbgGWDqw (ORCPT ); Wed, 22 Jul 2020 23:46:52 -0400 X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R791e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=e01e07425;MF=changhuaixin@linux.alibaba.com;NM=1;PH=DS;RN=13;SR=0;TI=SMTPD_---0U3Y0Kj4_1595476010; Received: from localhost(mailfrom:changhuaixin@linux.alibaba.com fp:SMTPD_---0U3Y0Kj4_1595476010) by smtp.aliyun-inc.com(127.0.0.1); Thu, 23 Jul 2020 11:46:50 +0800 From: Huaixin Chang To: jpoimboe@redhat.com Cc: bp@alien8.de, changhuaixin@linux.alibaba.com, hpa@zytor.com, linux-kbuild@vger.kernel.org, linux-kernel@vger.kernel.org, luto@amacapital.net, michal.lkml@markovi.net, mingo@redhat.com, peterz@infradead.org, tglx@linutronix.de, x86@kernel.org, yamada.masahiro@socionext.com Subject: [PATCH 1/3] scripts/sorttable: Change section type of orc_lookup to SHT_PROGBITS Date: Thu, 23 Jul 2020 11:46:41 +0800 Message-Id: <20200723034643.33537-2-changhuaixin@linux.alibaba.com> X-Mailer: git-send-email 2.14.4.44.g2045bb6 In-Reply-To: <20200723034643.33537-1-changhuaixin@linux.alibaba.com> References: <20200723034643.33537-1-changhuaixin@linux.alibaba.com> MIME-Version: 1.0 Sender: linux-kbuild-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kbuild@vger.kernel.org In order to edit orc_lookup table via sorttable, type of section orc_lookup needs to be SHT_PROGBITS instead of SHT_NOBITS. Linker script doesn't seem to allow manual specification of the section type, so just write a byte into the section instead. Signed-off-by: Josh Poimboeuf Signed-off-by: Huaixin Chang Signed-off-by: Shile Zhang --- include/asm-generic/vmlinux.lds.h | 2 ++ 1 file changed, 2 insertions(+) diff --git a/include/asm-generic/vmlinux.lds.h b/include/asm-generic/vmlinux.lds.h index db600ef218d7..49f4f5bc6165 100644 --- a/include/asm-generic/vmlinux.lds.h +++ b/include/asm-generic/vmlinux.lds.h @@ -826,6 +826,8 @@ . += (((SIZEOF(.text) + LOOKUP_BLOCK_SIZE - 1) / \ LOOKUP_BLOCK_SIZE) + 1) * 4; \ orc_lookup_end = .; \ + /* HACK: force SHT_PROGBITS so sorttable can edit: */ \ + BYTE(1); \ } #else #define ORC_UNWIND_TABLE From patchwork Thu Jul 23 03:46:42 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: changhuaixin X-Patchwork-Id: 11679849 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id C2B1B138C for ; Thu, 23 Jul 2020 03:47:05 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id B44B32068F for ; Thu, 23 Jul 2020 03:47:05 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726109AbgGWDq5 (ORCPT ); Wed, 22 Jul 2020 23:46:57 -0400 Received: from out30-45.freemail.mail.aliyun.com ([115.124.30.45]:36323 "EHLO out30-45.freemail.mail.aliyun.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725843AbgGWDq4 (ORCPT ); Wed, 22 Jul 2020 23:46:56 -0400 X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R881e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=e01f04397;MF=changhuaixin@linux.alibaba.com;NM=1;PH=DS;RN=13;SR=0;TI=SMTPD_---0U3Y0Kj9_1595476010; Received: from localhost(mailfrom:changhuaixin@linux.alibaba.com fp:SMTPD_---0U3Y0Kj9_1595476010) by smtp.aliyun-inc.com(127.0.0.1); Thu, 23 Jul 2020 11:46:50 +0800 From: Huaixin Chang To: jpoimboe@redhat.com Cc: bp@alien8.de, changhuaixin@linux.alibaba.com, hpa@zytor.com, linux-kbuild@vger.kernel.org, linux-kernel@vger.kernel.org, luto@amacapital.net, michal.lkml@markovi.net, mingo@redhat.com, peterz@infradead.org, tglx@linutronix.de, x86@kernel.org, yamada.masahiro@socionext.com Subject: [PATCH 2/3] scripts/sorttable: Build orc fast lookup table via sorttable tool Date: Thu, 23 Jul 2020 11:46:42 +0800 Message-Id: <20200723034643.33537-3-changhuaixin@linux.alibaba.com> X-Mailer: git-send-email 2.14.4.44.g2045bb6 In-Reply-To: <20200723034643.33537-1-changhuaixin@linux.alibaba.com> References: <20200723034643.33537-1-changhuaixin@linux.alibaba.com> MIME-Version: 1.0 Sender: linux-kbuild-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kbuild@vger.kernel.org Since orc tables are already sorted by sorttable tool, let us move building of fast lookup table into sorttable tool too. This saves us 6380us from boot time under Intel(R) Xeon(R) CPU E5-2682 v4 @ 2.50GHz with 64 cores. Signed-off-by: Huaixin Chang Signed-off-by: Shile Zhang --- scripts/sorttable.h | 99 +++++++++++++++++++++++++++++++++++++++++++++++++---- 1 file changed, 92 insertions(+), 7 deletions(-) diff --git a/scripts/sorttable.h b/scripts/sorttable.h index a2baa2fefb13..a36c76c17be4 100644 --- a/scripts/sorttable.h +++ b/scripts/sorttable.h @@ -93,12 +93,50 @@ char g_err[ERRSTR_MAXSZ]; int *g_orc_ip_table; struct orc_entry *g_orc_table; +static unsigned long orc_ip_table_offset; pthread_t orc_sort_thread; +struct orc_sort_param { + size_t lookup_table_size; + unsigned int *orc_lookup_table; + unsigned long start_ip; + size_t text_size; + unsigned int orc_num_entries; +}; + static inline unsigned long orc_ip(const int *ip) { - return (unsigned long)ip + *ip; + return (unsigned long)ip + *ip + orc_ip_table_offset; +} + +static struct orc_entry *__orc_find(int *ip_table, struct orc_entry *u_table, + unsigned int num_entries, unsigned long ip) +{ + int *first = ip_table; + int *last = ip_table + num_entries - 1; + int *mid = first, *found = first; + + if (!num_entries) + return NULL; + + /* + * Do a binary range search to find the rightmost duplicate of a given + * starting address. Some entries are section terminators which are + * "weak" entries for ensuring there are no gaps. They should be + * ignored when they conflict with a real entry. + */ + while (first <= last) { + mid = first + ((last - first) / 2); + + if (orc_ip(mid) <= ip) { + found = mid; + first = mid + 1; + } else + last = mid - 1; + } + + return u_table + (found - ip_table); } static int orc_sort_cmp(const void *_a, const void *_b) @@ -130,18 +168,24 @@ static void *sort_orctable(void *arg) int *idxs = NULL; int *tmp_orc_ip_table = NULL; struct orc_entry *tmp_orc_table = NULL; - unsigned int *orc_ip_size = (unsigned int *)arg; - unsigned int num_entries = *orc_ip_size / sizeof(int); + struct orc_sort_param *param = (struct orc_sort_param *)arg; + unsigned int num_entries = param->orc_num_entries; + unsigned int orc_ip_size = num_entries * sizeof(int); unsigned int orc_size = num_entries * sizeof(struct orc_entry); + unsigned int lookup_num_blocks = param->lookup_table_size / sizeof(int); + unsigned int *orc_lookup = param->orc_lookup_table; + unsigned long lookup_start_ip = param->start_ip; + unsigned long lookup_stop_ip = param->start_ip + param->text_size; + struct orc_entry *orc; - idxs = (int *)malloc(*orc_ip_size); + idxs = (int *)malloc(orc_ip_size); if (!idxs) { snprintf(g_err, ERRSTR_MAXSZ, "malloc idxs: %s", strerror(errno)); pthread_exit(g_err); } - tmp_orc_ip_table = (int *)malloc(*orc_ip_size); + tmp_orc_ip_table = (int *)malloc(orc_ip_size); if (!tmp_orc_ip_table) { snprintf(g_err, ERRSTR_MAXSZ, "malloc tmp_orc_ip_table: %s", strerror(errno)); @@ -173,6 +217,31 @@ static void *sort_orctable(void *arg) g_orc_table[i] = tmp_orc_table[idxs[i]]; } +#define LOOKUP_BLOCK_ORDER 8 +#define LOOKUP_BLOCK_SIZE (1 << LOOKUP_BLOCK_ORDER) + + for (i = 0; i < lookup_num_blocks-1; i++) { + orc = __orc_find(g_orc_ip_table, g_orc_table, + num_entries, + lookup_start_ip + (LOOKUP_BLOCK_SIZE * i)); + if (!orc) { + snprintf(g_err, ERRSTR_MAXSZ, + "Corrupt .orc_unwind table\n"); + pthread_exit(g_err); + } + + orc_lookup[i] = orc - g_orc_table; + } + + /* Initialize the ending block: */ + orc = __orc_find(g_orc_ip_table, g_orc_table, num_entries, + lookup_stop_ip); + if (!orc) { + snprintf(g_err, ERRSTR_MAXSZ, "Corrupt .orc_unwind table\n"); + pthread_exit(g_err); + } + orc_lookup[lookup_num_blocks-1] = orc - g_orc_table; + free(idxs); free(tmp_orc_ip_table); free(tmp_orc_table); @@ -221,6 +290,8 @@ static int do_sort(Elf_Ehdr *ehdr, unsigned int orc_ip_size = 0; unsigned int orc_size = 0; unsigned int orc_num_entries = 0; + unsigned long orc_ip_addr = 0; + struct orc_sort_param param; #endif shstrndx = r2(&ehdr->e_shstrndx); @@ -259,17 +330,27 @@ static int do_sort(Elf_Ehdr *ehdr, orc_ip_size = s->sh_size; g_orc_ip_table = (int *)((void *)ehdr + s->sh_offset); + orc_ip_addr = s->sh_addr; } if (!strcmp(secstrings + idx, ".orc_unwind")) { orc_size = s->sh_size; g_orc_table = (struct orc_entry *)((void *)ehdr + s->sh_offset); } + if (!strcmp(secstrings + idx, ".orc_lookup")) { + param.lookup_table_size = s->sh_size; + param.orc_lookup_table = (unsigned int *) + ((void *)ehdr + s->sh_offset); + } + if (!strcmp(secstrings + idx, ".text")) { + param.text_size = s->sh_size; + param.start_ip = s->sh_addr; + } #endif } /* for loop */ #if defined(SORTTABLE_64) && defined(UNWINDER_ORC_ENABLED) - if (!g_orc_ip_table || !g_orc_table) { + if (!g_orc_ip_table || !g_orc_table || !param.orc_lookup_table) { fprintf(stderr, "incomplete ORC unwind tables in file: %s\n", fname); goto out; @@ -285,9 +366,13 @@ static int do_sort(Elf_Ehdr *ehdr, goto out; } + /* Make orc_ip return virtual address at execution. */ + orc_ip_table_offset = orc_ip_addr - (unsigned long)g_orc_ip_table; + /* create thread to sort ORC unwind tables concurrently */ + param.orc_num_entries = orc_num_entries; if (pthread_create(&orc_sort_thread, NULL, - sort_orctable, &orc_ip_size)) { + sort_orctable, ¶m)) { fprintf(stderr, "pthread_create orc_sort_thread failed '%s': %s\n", strerror(errno), fname); From patchwork Thu Jul 23 03:46:43 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: changhuaixin X-Patchwork-Id: 11679851 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id D4C631709 for ; Thu, 23 Jul 2020 03:47:05 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id C6B3C2068F for ; Thu, 23 Jul 2020 03:47:05 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726178AbgGWDrE (ORCPT ); Wed, 22 Jul 2020 23:47:04 -0400 Received: from out4436.biz.mail.alibaba.com ([47.88.44.36]:32589 "EHLO out4436.biz.mail.alibaba.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725843AbgGWDrE (ORCPT ); Wed, 22 Jul 2020 23:47:04 -0400 X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R141e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=e01e04407;MF=changhuaixin@linux.alibaba.com;NM=1;PH=DS;RN=13;SR=0;TI=SMTPD_---0U3YAYRP_1595476011; Received: from localhost(mailfrom:changhuaixin@linux.alibaba.com fp:SMTPD_---0U3YAYRP_1595476011) by smtp.aliyun-inc.com(127.0.0.1); Thu, 23 Jul 2020 11:46:51 +0800 From: Huaixin Chang To: jpoimboe@redhat.com Cc: bp@alien8.de, changhuaixin@linux.alibaba.com, hpa@zytor.com, linux-kbuild@vger.kernel.org, linux-kernel@vger.kernel.org, luto@amacapital.net, michal.lkml@markovi.net, mingo@redhat.com, peterz@infradead.org, tglx@linutronix.de, x86@kernel.org, yamada.masahiro@socionext.com Subject: [PATCH 3/3] x86/unwind/orc: Simplify unwind_init() for x86 boot Date: Thu, 23 Jul 2020 11:46:43 +0800 Message-Id: <20200723034643.33537-4-changhuaixin@linux.alibaba.com> X-Mailer: git-send-email 2.14.4.44.g2045bb6 In-Reply-To: <20200723034643.33537-1-changhuaixin@linux.alibaba.com> References: <20200723034643.33537-1-changhuaixin@linux.alibaba.com> MIME-Version: 1.0 Sender: linux-kbuild-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kbuild@vger.kernel.org The orc fast lookup table is built by scripts/sorttable tool. All that is left is setting lookup_num_blocks. Signed-off-by: Huaixin Chang Signed-off-by: Shile Zhang --- arch/x86/kernel/unwind_orc.c | 41 ++--------------------------------------- 1 file changed, 2 insertions(+), 39 deletions(-) diff --git a/arch/x86/kernel/unwind_orc.c b/arch/x86/kernel/unwind_orc.c index 7f969b2d240f..0ca4d02d1a69 100644 --- a/arch/x86/kernel/unwind_orc.c +++ b/arch/x86/kernel/unwind_orc.c @@ -264,48 +264,11 @@ void unwind_module_init(struct module *mod, void *_orc_ip, size_t orc_ip_size, void __init unwind_init(void) { - size_t orc_ip_size = (void *)__stop_orc_unwind_ip - (void *)__start_orc_unwind_ip; - size_t orc_size = (void *)__stop_orc_unwind - (void *)__start_orc_unwind; - size_t num_entries = orc_ip_size / sizeof(int); - struct orc_entry *orc; - int i; - - if (!num_entries || orc_ip_size % sizeof(int) != 0 || - orc_size % sizeof(struct orc_entry) != 0 || - num_entries != orc_size / sizeof(struct orc_entry)) { - orc_warn("WARNING: Bad or missing .orc_unwind table. Disabling unwinder.\n"); - return; - } - /* - * Note, the orc_unwind and orc_unwind_ip tables were already - * sorted at build time via the 'sorttable' tool. - * It's ready for binary search straight away, no need to sort it. + * All orc tables are sorted and built via sorttable tool. Initialize + * lookup_num_blocks only. */ - - /* Initialize the fast lookup table: */ lookup_num_blocks = orc_lookup_end - orc_lookup; - for (i = 0; i < lookup_num_blocks-1; i++) { - orc = __orc_find(__start_orc_unwind_ip, __start_orc_unwind, - num_entries, - LOOKUP_START_IP + (LOOKUP_BLOCK_SIZE * i)); - if (!orc) { - orc_warn("WARNING: Corrupt .orc_unwind table. Disabling unwinder.\n"); - return; - } - - orc_lookup[i] = orc - __start_orc_unwind; - } - - /* Initialize the ending block: */ - orc = __orc_find(__start_orc_unwind_ip, __start_orc_unwind, num_entries, - LOOKUP_STOP_IP); - if (!orc) { - orc_warn("WARNING: Corrupt .orc_unwind table. Disabling unwinder.\n"); - return; - } - orc_lookup[lookup_num_blocks-1] = orc - __start_orc_unwind; - orc_init = true; }