From patchwork Wed Oct 2 16:07:20 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Ryabinin X-Patchwork-Id: 13820017 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6EDAECF6D3B for ; Wed, 2 Oct 2024 16:09:09 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id A685D4401BF; Wed, 2 Oct 2024 12:09:07 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id A19424401B5; Wed, 2 Oct 2024 12:09:07 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 86BEA4401BF; Wed, 2 Oct 2024 12:09:07 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 5952B4401B5 for ; Wed, 2 Oct 2024 12:09:07 -0400 (EDT) Received: from smtpin19.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 0711E1A0AFF for ; Wed, 2 Oct 2024 16:09:07 +0000 (UTC) X-FDA: 82629146334.19.F617FB9 Received: from forwardcorp1d.mail.yandex.net (forwardcorp1d.mail.yandex.net [178.154.239.200]) by imf01.hostedemail.com (Postfix) with ESMTP id EA1E74001E for ; Wed, 2 Oct 2024 16:09:04 +0000 (UTC) Authentication-Results: imf01.hostedemail.com; dkim=pass header.d=yandex-team.com header.s=default header.b=Q0TLXURt; spf=pass (imf01.hostedemail.com: domain of arbn@yandex-team.com designates 178.154.239.200 as permitted sender) smtp.mailfrom=arbn@yandex-team.com; dmarc=pass (policy=none) header.from=yandex-team.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1727885175; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=vDCJtHtK3uU2sdBk8J/tnuJVXLHuR9McbyPPpAX4VPs=; b=yMVz/C6Zw9om6QvPF9cHWiprzWS9eIEvM93yxUFkqTzCQ6zP5WSs3Uh4LzV22ZJQZUuJar d7tLLaMiaZIfm8uzZGz/jglrF0YmB8UryMWug2PWCoPXKwWk0rBSp0PoBHKQ/3PLzMKTSd q4o7L+X96Zynlb/qygFCPtC9zJLxk3Y= ARC-Authentication-Results: i=1; imf01.hostedemail.com; dkim=pass header.d=yandex-team.com header.s=default header.b=Q0TLXURt; spf=pass (imf01.hostedemail.com: domain of arbn@yandex-team.com designates 178.154.239.200 as permitted sender) smtp.mailfrom=arbn@yandex-team.com; dmarc=pass (policy=none) header.from=yandex-team.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1727885175; a=rsa-sha256; cv=none; b=xJFAASRPodW4bcUbtpGY4CX0XwzxXp9UlKcX8k0AaoEdQzmm1v5sE0QMoG5dgGYZMR/Nvj nIo4e1ZneYH9NBTV+oslRMdkZ+g+5MYD0TzESpu+vjFzpjbNgd9gEZlmHuN8Hc76W2ukjk Gxq18alLCiCAjedU92L+WDc+NOUrFDM= Received: from mail-nwsmtp-smtp-corp-main-56.klg.yp-c.yandex.net (mail-nwsmtp-smtp-corp-main-56.klg.yp-c.yandex.net [IPv6:2a02:6b8:c42:b1cb:0:640:2a1e:0]) by forwardcorp1d.mail.yandex.net (Yandex) with ESMTPS id 62E2A609C4; Wed, 2 Oct 2024 19:09:03 +0300 (MSK) Received: from dellarbn.yandex.net (unknown [10.214.35.248]) by mail-nwsmtp-smtp-corp-main-56.klg.yp-c.yandex.net (smtpcorp/Yandex) with ESMTPSA id Z8emWD2IhiE0-vYLh3gEa; Wed, 02 Oct 2024 19:09:02 +0300 X-Yandex-Fwd: 1 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yandex-team.com; s=default; t=1727885342; bh=vDCJtHtK3uU2sdBk8J/tnuJVXLHuR9McbyPPpAX4VPs=; h=Message-ID:Date:In-Reply-To:Cc:Subject:References:To:From; b=Q0TLXURt2M11h05HQcTCotDXqV17/pTO/eYHxyvF7CpIRwLEfbBfbhK5guCrZhW7i FTOE8WK8gOdDKUCtCqsclUk1GbjcM0KngfLxMtHpSJRo9cn3UXM9NGQ39FLI8zLwTl y0g4IX5ZfCyGA9n7LHUYQ9btKkyR6EMQ9Tfr2MTA= From: Andrey Ryabinin To: linux-kernel@vger.kernel.org Cc: Alexander Graf , James Gowans , Mike Rapoport , Andrew Morton , linux-mm@kvack.org, Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H. Peter Anvin" , Eric Biederman , kexec@lists.infradead.org, Steven Rostedt , Masami Hiramatsu , Mathieu Desnoyers , linux-trace-kernel@vger.kernel.org, valesini@yandex-team.com, Andrey Ryabinin Subject: [RFC PATCH 5/7] kstate: Add mechanism to preserved specified memory pages across kexec. Date: Wed, 2 Oct 2024 18:07:20 +0200 Message-ID: <20241002160722.20025-6-arbn@yandex-team.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20241002160722.20025-1-arbn@yandex-team.com> References: <20241002160722.20025-1-arbn@yandex-team.com> MIME-Version: 1.0 X-Yandex-Filter: 1 X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: EA1E74001E X-Stat-Signature: dk1ckbfrhjsfq77x11y3tqmx53dk6hue X-Rspam-User: X-HE-Tag: 1727885344-182220 X-HE-Meta: U2FsdGVkX18Zg8FG4U4MWFaQA8wbcGy4GbveRMQ7sTB8emh+ejPAwumsvNX3Yg/GsnOvapQjGJvodd69q5lNDHB1C/cvhpaumsbQNJTW8Ad/a7bS105Lqhd8T/3ziArLjfn0ZrmknynSqanBIsVF+qz9a/zpP3CZijf65OEqoAY5SDfklYva49hgCynEmmSeAF9UH9bw3TLdiZk2AQn1Dl7riYHg4/BwDchyw9qX9N3yxw6ng/MB3pJVq9QpXxp5uVgYWxSSl6cNbfd+OHyC8n/BOHoH3kTdHaZBsLyr8YLLSYKvON9znP1hiiVp7zVUpddI5efbIQ8y/i+HhQBiV+7w84S3TkMw60SrOHKOYdBOM0kTmmAACstZojI9ZPEY8jkQspQmbE9ltZRScMP2OW5UUc1eYNg0aY/XoTmdKc52G/g6MC2jJfRI3E9EWvcqks78QfZc5csEa7vAizPPA73sJoIrdwyXRlx7WuZjQOwESC4a+bkECh1qu4KKI6vPYBuNcr2ej98x51flJ4RW1ryHPXfawBy29JlPdSTiCnpy5CF9Fgoax1sACH4A2/xeeJaHxyzmiTXH/4s0uP4ERX0p77KD9NbrO4A5IBmTPQKkheV55sTna1mK0vcpPJkgLeKhp04/+KYHCj2j9aucijxA39j8dCkckPlB7BYJdUXpw4Ongscfoj+iR+AL8bZuq9jp80oY66pkV6qIXaWVNp7MC2LR5Sm93bqH/rii3U+iIiQsMFxUa8oApzq5+pbrqOsG2wSz/8a48+jDmh5HwVlCKWU5kjT0AiIBABPSUzvqgKlpiBmTK2dXz+CutKmAiaBUNrOLVt0loHWEs4IDbt6QigO6U/Mj9zpMsfvvQ+EdMs+oq+1xNZoDPkhR1/w/IBbnn7prOrCWrUbOXfdvk//vCEzFHokzihhttOQH51vamuwGFBRQ1JhNqcP9L66a63f6W28cVGVO0rvqVXx EtiH1wJm ydg1b8XJ45gacWZehPOh6RgKQVWldRziWsokaUMQ/+k61rWOjo6DFXTOL9QN045Cbfe4J5mBkeX4+zLeb2SpeXTNyFAGCqmQDrVHoae6StQVwQwva0RBOyOQH3DHyjzGwN5BrK7PkyOsgsOuYVakBDF45YK/p4B6zLVT92ciVxrFFP3CCkAsvy62vYA/g3osM+OVDxGPP8cpoCvv+IpCdJsUvB9KcJXSf3GdPlDjFeHw2zrPFiDwtjvGg8UTPJjAyGjZOQ7CkF+8vrecTuF7FVOTBEHC+klyH/HdhwMI8XTSiCSoSETfzKQEvJgv5gm8TagGnUHKIwVPRTc6FabUIeixsq/5CyaTS8/W7iQt9Qwrsim4= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: This adds functionality to preserve memory pages across kexec. kstate_register_page() stores struct page in the special list of 'struct page_state's. At kexec reboot stage this list iterated, pfns saved into kstate's migrate stream. The new kernel after kexec reads pfns from the stream and marks memory as reserved to keep it intact. Also it marked with MEMBLOCK_PRSRV flag indicating that 'struct page' itself shouldn't be reinitialized. Signed-off-by: Andrey Ryabinin --- arch/x86/kernel/kexec-bzimage64.c | 2 +- arch/x86/kernel/setup.c | 81 +++++++++++++++++++++++++++++++ include/linux/kstate.h | 6 +++ kernel/kstate.c | 7 +++ 4 files changed, 95 insertions(+), 1 deletion(-) diff --git a/arch/x86/kernel/kexec-bzimage64.c b/arch/x86/kernel/kexec-bzimage64.c index 71c82841e6b12..d769d08cf9a8a 100644 --- a/arch/x86/kernel/kexec-bzimage64.c +++ b/arch/x86/kernel/kexec-bzimage64.c @@ -406,7 +406,7 @@ static int load_migrate_segments(struct kimage *image) kbuf.memsz = 8*1024*1024; - kbuf.buf_align = ELF_CORE_HEADER_ALIGN; + kbuf.buf_align = PAGE_SIZE; kbuf.mem = KEXEC_BUF_MEM_UNKNOWN; ret = kexec_add_buffer(&kbuf); if (ret) diff --git a/arch/x86/kernel/setup.c b/arch/x86/kernel/setup.c index f1fea506e20f4..cfddc902e266b 100644 --- a/arch/x86/kernel/setup.c +++ b/arch/x86/kernel/setup.c @@ -15,6 +15,7 @@ #include #include #include +#include #include #include #include @@ -638,6 +639,85 @@ static void __init e820_add_kernel_range(void) e820__range_add(start, size, E820_TYPE_RAM); } +#ifdef CONFIG_KSTATE +struct state_entry mem_kstate; + +struct mem_state { + unsigned int nr_pages; + struct list_head list; +}; +struct page_state { + struct list_head list; + int order; + struct page *page; +}; + +struct mem_state m_state = { .list = LIST_HEAD_INIT(m_state.list) }; + +int kstate_register_page(struct page *page, int order) +{ + struct page_state *state; + + state = kmalloc(sizeof(*state), GFP_KERNEL); + if (!state) + return -ENOMEM; + + state->page = page; + state->order = order; + list_add(&state->list, &m_state.list); + m_state.nr_pages++; + return 0; +} + +static int kstate_pages_save(void *mig_stream, void *obj, + const struct kstate_field *field) +{ + struct page_state *p_state; + void *start = mig_stream; + + list_for_each_entry(p_state, &m_state.list, list) { + mig_stream = kstate_save_byte(mig_stream, p_state->order); + mig_stream = kstate_save_ulong(mig_stream, page_to_phys(p_state->page)); + } + return mig_stream - start; +} + +static int __init kstate_pages_restore(void *mig_stream, void *obj, + const struct kstate_field *field) +{ + struct mem_state *m_state = obj; + int nr_pages, i; + + nr_pages = m_state->nr_pages; + for (i = 0; i < nr_pages; i++) { + int order = kstate_get_byte(&mig_stream); + unsigned long phys = kstate_get_ulong(&mig_stream); + + memblock_reserve(phys, PAGE_SIZE << order); + memblock_reserved_mark_preserved(phys, PAGE_SIZE << order); + } + return 0; +} + +struct kstate_description kstate_reserved = { + .name = "reserved_mem", + .id = KSTATE_RSVD_MEM_ID, + .state_list = LIST_HEAD_INIT(kstate_reserved.state_list), + .fields = (const struct kstate_field[]) { + KSTATE_SIMPLE(nr_pages, struct mem_state), + { + .name = "pages", + .flags = KS_CUSTOM, + .size = sizeof(struct mem_state), + .save = kstate_pages_save, + .restore = kstate_pages_restore, + }, + + KSTATE_END_OF_LIST() + }, +}; +#endif + static void __init early_reserve_memory(void) { /* @@ -989,6 +1069,7 @@ void __init setup_arch(char **cmdline_p) memblock_set_current_limit(ISA_END_ADDRESS); e820__memblock_setup(); + __kstate_register(&kstate_reserved, &m_state, &mem_kstate); /* * Needs to run after memblock setup because it needs the physical diff --git a/include/linux/kstate.h b/include/linux/kstate.h index c97804d0243ea..855acb339d5d7 100644 --- a/include/linux/kstate.h +++ b/include/linux/kstate.h @@ -29,6 +29,8 @@ struct kstate_field { }; enum kstate_ids { + KSTATE_PAGE_ID, + KSTATE_RSVD_MEM_ID, KSTATE_LAST_ID = -1, }; @@ -87,6 +89,10 @@ void *save_kstate(void *stream, int id, const struct kstate_description *kstate, void *obj); void *restore_kstate(struct kstate_entry *ke, int id, const struct kstate_description *kstate, void *obj); + +int kstate_page_save(void *mig_stream, void *obj, + const struct kstate_field *field); +int kstate_register_page(struct page *page, int order); #else #define __kstate_register(state, obj, se) diff --git a/kernel/kstate.c b/kernel/kstate.c index 0ef228baef94e..7f7e135bafd81 100644 --- a/kernel/kstate.c +++ b/kernel/kstate.c @@ -182,6 +182,13 @@ int kstate_register(struct kstate_description *state, void *obj) return 0; } +int kstate_page_save(void *mig_stream, void *obj, + const struct kstate_field *field) +{ + kstate_register_page(*(struct page **)obj, 0); + return 0; +} + static int __init setup_migrate(char *arg) { char *end;