From patchwork Thu Nov 22 10:09:38 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Hildenbrand X-Patchwork-Id: 10693847 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 4D09E1734 for ; Thu, 22 Nov 2018 10:09:45 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 3D0E72CA2B for ; Thu, 22 Nov 2018 10:09:45 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 2F81A2CA35; Thu, 22 Nov 2018 10:09:45 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=unavailable version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id A508D2CA2B for ; Thu, 22 Nov 2018 10:09:44 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id B0CA96B2B00; Thu, 22 Nov 2018 05:09:43 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id ABDB46B2B06; Thu, 22 Nov 2018 05:09:43 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 985716B2B07; Thu, 22 Nov 2018 05:09:43 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-qt1-f200.google.com (mail-qt1-f200.google.com [209.85.160.200]) by kanga.kvack.org (Postfix) with ESMTP id 6E4B46B2B00 for ; Thu, 22 Nov 2018 05:09:43 -0500 (EST) Received: by mail-qt1-f200.google.com with SMTP id w15so5843393qtk.19 for ; Thu, 22 Nov 2018 02:09:43 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references; bh=fTDu+zcvoyt860lBP9SOwff5lZQtyQCUtLhoHL5R3VE=; b=n+MIEd4AilV9Pb0HxqB+92h0RCasolgTpwkCeBWkrc086YgH9BCqzK7JGTZGfBTLyw zEI5lQFv85rXzP2QmVefZuIrWbmA/EipnazhTZzEA6ApuDKeCZTD93JxIz9rfExvb6BC +s92BYlmgKBUROKyCsQmu9hjWVA/vfBOG6jYu6MU03DdYBB7Q6larJ2b01N6fyRryJc8 52oQAXv9VXdBJuIBX1k8gFuklVIWCpbSzxpS9N8zszdOH6WV07Rz4Vr5iAtckt5T61aQ 9r0W3RyZaZXH4KI6d8H8i0/LTJI7647xVcXDHM0w5VIReXpYiN9kvzFJNf/2k7W3t2gt RGCA== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of david@redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=david@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com X-Gm-Message-State: AA+aEWYd7uYZBjtVWha5ouH5kgcaYHXN7VaDmaLx0L4Rpf/USZobxeP4 BljPvv4S1CXub9xwFZOsT/BusElYJlrXJna8vNFz8boWc1dW8uAYj6OWHMEpaRaqLp7nMiBHYwd 24dO3rv/ZPJIquuAaQ/sfH3mcvGs2/JrxBdnKzFFtmhJAWqoyu12haQW5EMARhbXDlw== X-Received: by 2002:a37:15a5:: with SMTP id 37mr9083219qkv.292.1542881383171; Thu, 22 Nov 2018 02:09:43 -0800 (PST) X-Google-Smtp-Source: AFSGD/WPSqk92utUXQu1pXr2n9WBX6Djg+xUg6qj47pJuIup7OrT46s0AB4OcrJdvZ2H8DTrT698 X-Received: by 2002:a37:15a5:: with SMTP id 37mr9083183qkv.292.1542881382436; Thu, 22 Nov 2018 02:09:42 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1542881382; cv=none; d=google.com; s=arc-20160816; b=E3XuXC9cyip01QJiBIOb8OUFyxY+PxeBL4LhgwKabvAi5BITJs7OGsyDeNSXAmiAGS SaerqHkQqsO/1tr6qiwSXmIj3qZUbG6rOkXTPaXlGr9sOzSLKLgO97iEU6vtA/5UoJ9S ZNp6nwLyKxI8SZDW5dVp+ERi9IXjuqxavQKh/pehGowgq+h0sslIvHv+NbPcQ0mQYMu9 WV7I4NERYes06SozYos8ZLGqhg/aU7zK/7qUA64IdWE43ZEVJiSAa5tcC0M+nzbpQheL FLtDctFhy5UP2eWtPbxE7d7qU6Ap8i9KajqpzcbOT7ofN/Iw5yCdnVqi+ZTflBCpniFB d9yA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:cc:to:from; bh=fTDu+zcvoyt860lBP9SOwff5lZQtyQCUtLhoHL5R3VE=; b=sDCh1fp+qd16FFengu5PvSf0tlArxG8QZ8g4rMjFOAzs+0uiCq/asSbKBpX2F6MnPw i9K//vxK2M89RXXDMhSpOovcqibB1WN9j88R/RxU7Q6KYDESUbTRfTrRjpoc9GLExNBR Aq2FoCIilN52gHPGWgPwsoczoj6QxKy0ti11EDE7aWudZGolBblzoBW0zanjV+kRGIh9 MgoDdJuHcZW9apQ8qgJrf+26yJfpPfn9uBzl8zqj8Rb8JvoGfY9n6UOV4NMRjXY4D+cP AvYlf4IcM2M9lh8OBkvGAZy4ykkdrQ46PYSwQYUZ3jEuiKSr7Zuk+k75tdFNDCWKgUcY uNhQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of david@redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=david@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from mx1.redhat.com (mx1.redhat.com. [209.132.183.28]) by mx.google.com with ESMTPS id s17si6381331qkl.40.2018.11.22.02.09.42 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 22 Nov 2018 02:09:42 -0800 (PST) Received-SPF: pass (google.com: domain of david@redhat.com designates 209.132.183.28 as permitted sender) client-ip=209.132.183.28; Authentication-Results: mx.google.com; spf=pass (google.com: domain of david@redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=david@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 679518EC; Thu, 22 Nov 2018 10:09:41 +0000 (UTC) Received: from t460s.redhat.com (ovpn-116-206.ams2.redhat.com [10.36.116.206]) by smtp.corp.redhat.com (Postfix) with ESMTP id 36E6557B0; Thu, 22 Nov 2018 10:09:39 +0000 (UTC) From: David Hildenbrand To: linux-mm@kvack.org Cc: linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, devel@linuxdriverproject.org, linux-fsdevel@vger.kernel.org, linux-pm@vger.kernel.org, xen-devel@lists.xenproject.org, kexec-ml , pv-drivers@vmware.com, Kazuhito Hagio , David Hildenbrand Subject: [PATCH v2] makedumpfile: exclude pages that are logically offline Date: Thu, 22 Nov 2018 11:09:38 +0100 Message-Id: <20181122100938.5567-1-david@redhat.com> In-Reply-To: <20181122100627.5189-1-david@redhat.com> References: <20181122100627.5189-1-david@redhat.com> X-Scanned-By: MIMEDefang 2.84 on 10.5.11.23 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.28]); Thu, 22 Nov 2018 10:09:41 +0000 (UTC) X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP Linux marks pages that are logically offline via a page flag (map count). Such pages e.g. include pages infated as part of a balloon driver or pages that were not actually onlined when onlining the whole section. While the hypervisor usually allows to read such inflated memory, we basically read and dump data that is completely irrelevant. Also, this might result in quite some overhead in the hypervisor. In addition, we saw some problems under Hyper-V, whereby we can crash the kernel by dumping, when reading memory of a partially onlined memory segment (for memory added by the Hyper-V balloon driver). Therefore, don't read and dump pages that are marked as being logically offline. Signed-off-by: David Hildenbrand --- v1 -> v2: - Fix PAGE_BUDDY_MAPCOUNT_VALUE vs. PAGE_OFFLINE_MAPCOUNT_VALUE makedumpfile.c | 34 ++++++++++++++++++++++++++++++---- makedumpfile.h | 1 + 2 files changed, 31 insertions(+), 4 deletions(-) diff --git a/makedumpfile.c b/makedumpfile.c index 8923538..a5f2ea9 100644 --- a/makedumpfile.c +++ b/makedumpfile.c @@ -88,6 +88,7 @@ mdf_pfn_t pfn_cache_private; mdf_pfn_t pfn_user; mdf_pfn_t pfn_free; mdf_pfn_t pfn_hwpoison; +mdf_pfn_t pfn_offline; mdf_pfn_t num_dumped; @@ -249,6 +250,21 @@ isHugetlb(unsigned long dtor) && (SYMBOL(free_huge_page) == dtor)); } +static int +isOffline(unsigned long flags, unsigned int _mapcount) +{ + if (NUMBER(PAGE_OFFLINE_MAPCOUNT_VALUE) == NOT_FOUND_NUMBER) + return FALSE; + + if (flags & (1UL << NUMBER(PG_slab))) + return FALSE; + + if (_mapcount == (int)NUMBER(PAGE_OFFLINE_MAPCOUNT_VALUE)) + return TRUE; + + return FALSE; +} + static int is_cache_page(unsigned long flags) { @@ -2287,6 +2303,8 @@ write_vmcoreinfo_data(void) WRITE_NUMBER("PG_hwpoison", PG_hwpoison); WRITE_NUMBER("PAGE_BUDDY_MAPCOUNT_VALUE", PAGE_BUDDY_MAPCOUNT_VALUE); + WRITE_NUMBER("PAGE_OFFLINE_MAPCOUNT_VALUE", + PAGE_OFFLINE_MAPCOUNT_VALUE); WRITE_NUMBER("phys_base", phys_base); WRITE_NUMBER("HUGETLB_PAGE_DTOR", HUGETLB_PAGE_DTOR); @@ -2687,6 +2705,7 @@ read_vmcoreinfo(void) READ_SRCFILE("pud_t", pud_t); READ_NUMBER("PAGE_BUDDY_MAPCOUNT_VALUE", PAGE_BUDDY_MAPCOUNT_VALUE); + READ_NUMBER("PAGE_OFFLINE_MAPCOUNT_VALUE", PAGE_OFFLINE_MAPCOUNT_VALUE); READ_NUMBER("phys_base", phys_base); #ifdef __aarch64__ READ_NUMBER("VA_BITS", VA_BITS); @@ -6041,6 +6060,12 @@ __exclude_unnecessary_pages(unsigned long mem_map, else if (isHWPOISON(flags)) { pfn_counter = &pfn_hwpoison; } + /* + * Exclude pages that are logically offline. + */ + else if (isOffline(flags, _mapcount)) { + pfn_counter = &pfn_offline; + } /* * Unexcludable page */ @@ -7522,7 +7547,7 @@ write_elf_pages_cyclic(struct cache_data *cd_header, struct cache_data *cd_page) */ if (info->flag_cyclic) { pfn_zero = pfn_cache = pfn_cache_private = 0; - pfn_user = pfn_free = pfn_hwpoison = 0; + pfn_user = pfn_free = pfn_hwpoison = pfn_offline = 0; pfn_memhole = info->max_mapnr; } @@ -8804,7 +8829,7 @@ write_kdump_pages_and_bitmap_cyclic(struct cache_data *cd_header, struct cache_d * Reset counter for debug message. */ pfn_zero = pfn_cache = pfn_cache_private = 0; - pfn_user = pfn_free = pfn_hwpoison = 0; + pfn_user = pfn_free = pfn_hwpoison = pfn_offline = 0; pfn_memhole = info->max_mapnr; /* @@ -9749,7 +9774,7 @@ print_report(void) pfn_original = info->max_mapnr - pfn_memhole; pfn_excluded = pfn_zero + pfn_cache + pfn_cache_private - + pfn_user + pfn_free + pfn_hwpoison; + + pfn_user + pfn_free + pfn_hwpoison + pfn_offline; shrinking = (pfn_original - pfn_excluded) * 100; shrinking = shrinking / pfn_original; @@ -9763,6 +9788,7 @@ print_report(void) REPORT_MSG(" User process data pages : 0x%016llx\n", pfn_user); REPORT_MSG(" Free pages : 0x%016llx\n", pfn_free); REPORT_MSG(" Hwpoison pages : 0x%016llx\n", pfn_hwpoison); + REPORT_MSG(" Offline pages : 0x%016llx\n", pfn_offline); REPORT_MSG(" Remaining pages : 0x%016llx\n", pfn_original - pfn_excluded); REPORT_MSG(" (The number of pages is reduced to %lld%%.)\n", @@ -9790,7 +9816,7 @@ print_mem_usage(void) pfn_original = info->max_mapnr - pfn_memhole; pfn_excluded = pfn_zero + pfn_cache + pfn_cache_private - + pfn_user + pfn_free + pfn_hwpoison; + + pfn_user + pfn_free + pfn_hwpoison + pfn_offline; shrinking = (pfn_original - pfn_excluded) * 100; shrinking = shrinking / pfn_original; total_size = info->page_size * pfn_original; diff --git a/makedumpfile.h b/makedumpfile.h index f02f86d..e3a2b29 100644 --- a/makedumpfile.h +++ b/makedumpfile.h @@ -1927,6 +1927,7 @@ struct number_table { long PG_hwpoison; long PAGE_BUDDY_MAPCOUNT_VALUE; + long PAGE_OFFLINE_MAPCOUNT_VALUE; long SECTION_SIZE_BITS; long MAX_PHYSMEM_BITS; long HUGETLB_PAGE_DTOR;