From patchwork Thu Nov 22 10:06:19 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Hildenbrand X-Patchwork-Id: 10693795 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id C45E35A4 for ; Thu, 22 Nov 2018 10:06:42 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id B41C02BA70 for ; Thu, 22 Nov 2018 10:06:42 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id A6B082BB0D; Thu, 22 Nov 2018 10:06:42 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 13FCA2BA70 for ; Thu, 22 Nov 2018 10:06:42 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 434296B2AF0; Thu, 22 Nov 2018 05:06:41 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 3B9F96B2AF1; Thu, 22 Nov 2018 05:06:41 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 235166B2AF2; Thu, 22 Nov 2018 05:06:41 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-qk1-f200.google.com (mail-qk1-f200.google.com [209.85.222.200]) by kanga.kvack.org (Postfix) with ESMTP id E9A326B2AF0 for ; Thu, 22 Nov 2018 05:06:40 -0500 (EST) Received: by mail-qk1-f200.google.com with SMTP id w185so8935842qka.9 for ; Thu, 22 Nov 2018 02:06:40 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id; bh=HKMex+6RqtkPC9k+FewK74/7+3nLkJykfp7FXNdGnFY=; b=fYD0gl3YyPpkCZcj0y2rUV3+Z9oPrit4uXbzGzHamGD6tw+M6xWYD7bjRioSKAuThr A0km8XNqYnYa+3buvr+WXlcLlVyN/kYgQ2c+j58ioFqw9FwvELEWCUNtmWdI5jgGFz6t 73osciTMmhCJFGdgFFMJYT18Ar5jZT48VlSE//eQKz1RdA+Ukp0YgKSFC7VTP95WjwIr QURUSLevNv0VUAm/sXxmwi9tRhrUs8P4e4f2gWTRoI/L/Mgk5KKjsM+yvh28llSKPdnF usNJ7lUJwTs3b21ZdK9FgLVqX3uBttZLjVbn1VqQn8VEpbnxcOzneB/AB/JEHnqDOpIV ZwlA== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of david@redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=david@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com X-Gm-Message-State: AA+aEWbOjlcC0f/aXdyVApaDuZG9rVVS2wOhNZ5/QzQsPlGRepmvIEuu D+UyI0zsGin/6MfRenSXS0L+hkY0fcpW5KFgyp4UGbh4RK6lhLpOKpZrL2JRW8sM8HSH3GneKyy WS+cvQcqA7YIvbas1E4UzlZre0q/Q2+cxhHM05xXZSYmTnmpwAlqpljc3a7/jSi5IKQ== X-Received: by 2002:ae9:d60c:: with SMTP id r12mr9269321qkk.9.1542881200673; Thu, 22 Nov 2018 02:06:40 -0800 (PST) X-Google-Smtp-Source: AFSGD/XuZrKVe0flhvySePEThgwyU7rlsv+6lIYGAYVvbljLe+eUkEN57nzJ1KwftumKxfotS8vh X-Received: by 2002:ae9:d60c:: with SMTP id r12mr9269269qkk.9.1542881199802; Thu, 22 Nov 2018 02:06:39 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1542881199; cv=none; d=google.com; s=arc-20160816; b=LdM7QSR5JB+qtkBoIAOQocfG2SP2yYTMfg5iVVr/ubShgYwJYaYVVz97gGxIgbM2HQ yoZGoYGTuwQAqkS9VeQUYmrouV++gCKEF9punPnhGibsTXCJ8X1XWnaOVlSNW9Cvaq2L LggvfVq2lLfOZ9/0NnKziwU57S0+jXqbC4eo7MKFK+DlRqsk8iYGRrCDLRHdnzSv2TGk 15T+5Lso7EqPvLiukwOiRLO1l3uT1gfk2f8+6udcmEfouDf/DHbQpXcLqtuQ+gZCRnuC Wc7Vt1LIfiu4VgGZf0VGX60Nu5kc1pqaBobLKP4G0t8GTieAGznRuvsX59gmz9npsytZ uvng== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=message-id:date:subject:cc:to:from; bh=HKMex+6RqtkPC9k+FewK74/7+3nLkJykfp7FXNdGnFY=; b=eqrM1kA9CfF8HpP9EIa5lXDNnrPliHyzRItUhf36h2JJDilS7/TfrlvMVlVj4NI6w5 gbnDr+gQ+OFq7ZwKzc0897btWEkB+XEpzDCspcp12716ui3DrPDdVM5XqPFnjgYKzA5S eMw8bh5mWjLSvk4VkwcXznSrh18AuP0n4XKOZFK5xAhYfQ+2Xn3coCaF6Ab0F6+3te5Q tVaTEufPLhHEQBOa0DjnTmnGy4MuMWnaiv+QmgMh50HsAKV5LMR4UFAA3Vi+WKrCMgoj aK8qnhuHAIuybRlXB24KucRD917CDasCzF8mcOX9YHhLThpcFX2VLFADSqgYdpl4ZMJ+ FOcg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of david@redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=david@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from mx1.redhat.com (mx1.redhat.com. [209.132.183.28]) by mx.google.com with ESMTPS id q10si25935081qvh.99.2018.11.22.02.06.39 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 22 Nov 2018 02:06:39 -0800 (PST) Received-SPF: pass (google.com: domain of david@redhat.com designates 209.132.183.28 as permitted sender) client-ip=209.132.183.28; Authentication-Results: mx.google.com; spf=pass (google.com: domain of david@redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=david@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id C5B5C80F8E; Thu, 22 Nov 2018 10:06:37 +0000 (UTC) Received: from t460s.redhat.com (ovpn-116-206.ams2.redhat.com [10.36.116.206]) by smtp.corp.redhat.com (Postfix) with ESMTP id CE32657B0; Thu, 22 Nov 2018 10:06:27 +0000 (UTC) From: David Hildenbrand To: linux-mm@kvack.org Cc: linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, devel@linuxdriverproject.org, linux-fsdevel@vger.kernel.org, linux-pm@vger.kernel.org, xen-devel@lists.xenproject.org, kexec-ml , pv-drivers@vmware.com, David Hildenbrand , Alexander Duyck , Alexey Dobriyan , Andrew Morton , Arnd Bergmann , Baoquan He , Borislav Petkov , Boris Ostrovsky , Christian Hansen , Dave Young , David Rientjes , Greg Kroah-Hartman , Haiyang Zhang , Jonathan Corbet , Juergen Gross , Julien Freche , Kairui Song , Kazuhito Hagio , "Kirill A. Shutemov" , Konstantin Khlebnikov , "K. Y. Srinivasan" , Len Brown , Lianbo Jiang , Matthew Wilcox , "Michael S. Tsirkin" , Michal Hocko , Michal Hocko , Mike Rapoport , Miles Chen , Nadav Amit , Naoya Horiguchi , Omar Sandoval , Pankaj gupta , Pavel Machek , Pavel Tatashin , "Rafael J. Wysocki" , "Rafael J. Wysocki" , Stefano Stabellini , Stephen Hemminger , Stephen Rothwell , Vitaly Kuznetsov , Vlastimil Babka , Xavier Deguillard Subject: [PATCH v2 0/8] mm/kdump: allow to exclude pages that are logically offline Date: Thu, 22 Nov 2018 11:06:19 +0100 Message-Id: <20181122100627.5189-1-david@redhat.com> X-Scanned-By: MIMEDefang 2.84 on 10.5.11.23 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.27]); Thu, 22 Nov 2018 10:06:39 +0000 (UTC) X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP Right now, pages inflated as part of a balloon driver will be dumped by dump tools like makedumpfile. While XEN is able to check in the crash kernel whether a certain pfn is actually backed by memory in the hypervisor (see xen_oldmem_pfn_is_ram) and optimize this case, dumps of virtio-balloon, hv-balloon and VMWare balloon inflated memory will essentially result in zero pages getting allocated by the hypervisor and the dump getting filled with this data. The allocation and reading of zero pages can directly be avoided if a dumping tool could know which pages only contain stale information not to be dumped. Also for XEN, calling into the kernel and asking the hypervisor if a pfn is backed can be avoided if the duming tool would skip such pages right from the beginning. Dumping tools have no idea whether a given page is part of a balloon driver and shall not be dumped. Esp. PG_reserved cannot be used for that purpose as all memory allocated during early boot is also PG_reserved, see discussion at [1]. So some other way of indication is required and a new page flag is frowned upon. We have PG_balloon (MAPCOUNT value), which is essentially unused now. I suggest renaming it to something more generic (PG_offline) to mark pages as logically offline. This flag can than e.g. also be used by virtio-mem in the future to mark subsections as offline. Or by other code that wants to put pages logically offline (e.g. later maybe poisoned pages that shall no longer be used). This series converts PG_balloon to PG_offline, allows dumping tools to query the value to detect such pages and marks pages in the hv-balloon and XEN balloon properly as PG_offline. Note that virtio-balloon already set pages to PG_balloon (and now PG_offline). Please note that this is also helpful for a problem we were seeing under Hyper-V: Dumping logically offline memory (pages kept fake offline while onlining a section via online_page_callback) would under some condicions result in a kernel panic when dumping them. As I don't have access to neither XEN nor Hyper-V nor VMWare installations, this was only tested with the virtio-balloon and pages were properly skipped when dumping. I'll also attach the makedumpfile patch to this series. [1] https://lkml.org/lkml/2018/7/20/566 v1 -> v2: - "kexec: export PG_offline to VMCOREINFO" -- Add description why it is exported as a macro - "vmw_balloon: mark inflated pages PG_offline" -- Use helper function + adapt comments - "PM / Hibernate: exclude all PageOffline() pages" -- Perform the check separate from swsusp checks. - Added RBs/ACKs David Hildenbrand (8): mm: balloon: update comment about isolation/migration/compaction mm: convert PG_balloon to PG_offline kexec: export PG_offline to VMCOREINFO xen/balloon: mark inflated pages PG_offline hv_balloon: mark inflated pages PG_offline vmw_balloon: mark inflated pages PG_offline PM / Hibernate: use pfn_to_online_page() PM / Hibernate: exclude all PageOffline() pages Documentation/admin-guide/mm/pagemap.rst | 9 ++++--- drivers/hv/hv_balloon.c | 14 ++++++++-- drivers/misc/vmw_balloon.c | 32 ++++++++++++++++++++++ drivers/xen/balloon.c | 3 +++ fs/proc/page.c | 4 +-- include/linux/balloon_compaction.h | 34 +++++++++--------------- include/linux/page-flags.h | 11 +++++--- include/uapi/linux/kernel-page-flags.h | 2 +- kernel/crash_core.c | 2 ++ kernel/power/snapshot.c | 17 +++++++----- tools/vm/page-types.c | 2 +- 11 files changed, 90 insertions(+), 40 deletions(-)