From patchwork Wed Sep 19 03:17:45 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pingfan Liu X-Patchwork-Id: 10605207 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id D4F4F15A6 for ; Wed, 19 Sep 2018 03:18:40 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id C454E2B8AC for ; Wed, 19 Sep 2018 03:18:40 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id B86702B8FF; Wed, 19 Sep 2018 03:18:40 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 1068A2B8AC for ; Wed, 19 Sep 2018 03:18:40 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id F02238E0005; Tue, 18 Sep 2018 23:18:38 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id E8A1B8E0001; Tue, 18 Sep 2018 23:18:38 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D2BB78E0005; Tue, 18 Sep 2018 23:18:38 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-pl1-f198.google.com (mail-pl1-f198.google.com [209.85.214.198]) by kanga.kvack.org (Postfix) with ESMTP id 8C93D8E0001 for ; Tue, 18 Sep 2018 23:18:38 -0400 (EDT) Received: by mail-pl1-f198.google.com with SMTP id 2-v6so1814095plc.11 for ; Tue, 18 Sep 2018 20:18:38 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references; bh=Ewd2RgeXG1E75HoAyPPCWH7kK+qG54Cpd4g/BO0a+8o=; b=kQ1phQG/SHieNuP+l3HCsYp0CRtST4FUbf3ZNiUSMhrC0DwqnwRiL6ACjZUe7cYFN0 m9OqaqvpOOIaTvZe/BXz4j2F2VaV6hHBjl4phJdtS2rwc/ECrzC5pmRR0z6gTxjX8W70 ZrGTN9Gb+FQeFV1HkErimYuVtIlPrhzbjlt9FZkfSlKsO0+CNAkpjGaZ+ANV7i0o/IEe B1GYA18yk+z4P01Ojt5KxJx/QO1MzrwX1+wwRKEIed3SWxNLIWfichYZcr6frJlcUZ3g jDWI7484Gj58fZtTaTQwLjkEGAGvQkUQV9XBQ2wreRUXhBSL83LrDBTRCJzF7SuRBwuc 4PYw== X-Gm-Message-State: APzg51DG8sJLwOwCD0vHcUrG6rw2LgqyX6ne+r+0ToMS2K3Zqg6ctxwh ZumHd6G3uSrynS5f5MwpTsL62SDQS4zFed6qvoSVnq4JhPqEvRAOUD4UOJ2B7gy3iZvdNu+TVwq GK9/2M+51CGcQKl1VhbfZPTDu8ZgOJRqvrz/w59OzdF89dV1mNw/A4nEMRyZSEi6sOeExnWl0xD /NBchItQlZfr5aP84eI57zgjNT4eXX9LGAM6W/F/xu9cXGMz8Feysg4p03AIf6IokuzPXRyjqDy mjCzrOt6mPf4bjO9Vlcb1wjqjZrCg7LacjEt9ubOdAc7Cwxe58UnmD0xtS18QG+sJPBIo8dnekb xigG0AYcm+whsJr9ePWt5R8F0s0mIEmXL4TQlUC7ypFeR+Zf9n888TW3beY47Wj0M2C6nbXW86S X X-Received: by 2002:a63:be4a:: with SMTP id g10-v6mr28910153pgo.378.1537327118214; Tue, 18 Sep 2018 20:18:38 -0700 (PDT) X-Received: by 2002:a63:be4a:: with SMTP id g10-v6mr28910098pgo.378.1537327117121; Tue, 18 Sep 2018 20:18:37 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1537327117; cv=none; d=google.com; s=arc-20160816; b=WPQjHdD9NEVDg2BmxLLcdM2HVqms4f4DfIjRF946nCyMAmzhDY2kZk4hxZMAXAL4Ci dGAhl7nSuVXisdrGPv0M6/JomzJQhcyKp2EwroYMNkSydYt63ot45OieDdDpiw5qK6JB dHmQdw/R5T3VKGojGLgOIaN24xi0bMcMbQcEwzpY9Nj9su4k7p04xC4tSLYFTOTs6+sT tiMaxV/pCv7sgiWSOPu75+bgny2VaiGlwcCeWIHkfRxYBQ5URKEQFHmoss/L4cynieML HPyFGKVgEsZC3tStsMVBpmHEc0h8sXUWDkLDyWzwSL+ikpHz5+uOKNO98qJl6HAxp8eM a6Wg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=Ewd2RgeXG1E75HoAyPPCWH7kK+qG54Cpd4g/BO0a+8o=; b=BDxAgwm4Sp17o8i1+Fd4mQ50Qp+FAHXB7psjQt3u+ldzF8talG4qZYAmVDh6I9BMah 5PvPneGtBv94oNlQkQKOPW/bjOXapfEyNQCrtPKWnl1ACL3NcGqGeKNnlD1Ww/oE/RAB oXAy2x7hci+P8TNXkqG+h1wUc7nASySeGxUsFGdiovz6xHV2URUfhw48kUPeZwJmsm6J eG8eDyxdUSXwipw1yGaSDEcl1SmkASXSTO6orwAMx4eWW/bMruh4gfNPN50EarjTDgzZ 4AiA+JIcXZp5f0xqhEbGZ2VzqYdCysONy3xHHsgH6vP8w64150zcElZPR2VEdrN8cpVW 1MbA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gmail.com header.s=20161025 header.b=tkgEV8g9; spf=pass (google.com: domain of kernelfans@gmail.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=kernelfans@gmail.com; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id c17-v6sor2234835pgf.80.2018.09.18.20.18.36 for (Google Transport Security); Tue, 18 Sep 2018 20:18:37 -0700 (PDT) Received-SPF: pass (google.com: domain of kernelfans@gmail.com designates 209.85.220.65 as permitted sender) client-ip=209.85.220.65; Authentication-Results: mx.google.com; dkim=pass header.i=@gmail.com header.s=20161025 header.b=tkgEV8g9; spf=pass (google.com: domain of kernelfans@gmail.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=kernelfans@gmail.com; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=Ewd2RgeXG1E75HoAyPPCWH7kK+qG54Cpd4g/BO0a+8o=; b=tkgEV8g97WDUQSQRot8oP5q/qMXID/wQUAi55F8wMgFQ53ZAc5HDID8f3kP4RGZ6RY BVjCJPUjzIhIeR0sW73F+Vtr3OWpwyeeSScdtiQpygFaQtGwkd6BOgjegWz0vHD0XfSB nBuDIxgzwauKeZHAwcyddYETYU7ixOfTJpwEwHn3Jpkh1EFxiT3Lili0eN3TDC3+9R3x OvtClffgttk7A2lQ67g0lEwLN46imSCpbofSdXL6N1yVunwF3sBBJv4KPWuLSgpqv7j+ Gc08j2eDb/GiAlyoqBmLlRRRIRL4JH5mZJYmb+QQRPrFQTpkzgGwMjwwO/9SJKJNMkOH M5Og== X-Google-Smtp-Source: ANB0VdazGVErhDxuMDSYC8FA2WDM58/eXWA5HCSlJuXSHP0XR/QC8OLnYh2+UjIE5VYwgpGMpPWGAQ== X-Received: by 2002:a62:59d5:: with SMTP id k82-v6mr33484040pfj.143.1537327116136; Tue, 18 Sep 2018 20:18:36 -0700 (PDT) Received: from mylaptop.redhat.com ([209.132.188.80]) by smtp.gmail.com with ESMTPSA id o20-v6sm53087673pfj.35.2018.09.18.20.18.30 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 18 Sep 2018 20:18:35 -0700 (PDT) From: Pingfan Liu To: linux-mm@kvack.org Cc: Pingfan Liu , Andrew Morton , KAMEZAWA Hiroyuki , Mel Gorman , Greg Kroah-Hartman , Pavel Tatashin , Michal Hocko , Bharata B Rao , Dan Williams , "H. Peter Anvin" , "Kirill A . Shutemov" Subject: [PATCH 2/3] drivers/base/memory: introduce a new state 'isolate' for memblock Date: Wed, 19 Sep 2018 11:17:45 +0800 Message-Id: <1537327066-27852-3-git-send-email-kernelfans@gmail.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1537327066-27852-1-git-send-email-kernelfans@gmail.com> References: <1537327066-27852-1-git-send-email-kernelfans@gmail.com> X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP Currently, offline pages in the unit of memblock, and normally, it is done one by one on each memblock. If there is only one numa node, then the dst pages may come from the next memblock to be offlined, which wastes time during memory offline. For a system with multi numa node, if only replacing part of mem on a node, and the migration dst page can be allocated from local node (which is done by [3/3]), it also faces such issue. This patch suggests to introduce a new state, named 'isolate', the state transition can be isolate -> online or reversion. And another slight benefit of "isolated" state is no further allocation on this memblock, which can block potential unmovable page allocated again from this memblock for a long time. After this patch, the suggested ops to offline pages will looks like: for i in {s..e}; do echo isolate > memory$i/state; done for i in {s..e}; do echo offline > memory$i/state; done Since this patch does not change the original offline path, hence for i in (s..e); do echo offline > memory$i/state; done still works. Signed-off-by: Pingfan Liu Cc: Andrew Morton Cc: KAMEZAWA Hiroyuki Cc: Mel Gorman Cc: Greg Kroah-Hartman Cc: Pavel Tatashin Cc: Michal Hocko Cc: Bharata B Rao Cc: Dan Williams Cc: "H. Peter Anvin" Cc: Kirill A. Shutemov --- drivers/base/memory.c | 31 ++++++++++++++++++++++++++++++- include/linux/memory.h | 1 + 2 files changed, 31 insertions(+), 1 deletion(-) diff --git a/drivers/base/memory.c b/drivers/base/memory.c index c8a1cb0..3b714be 100644 --- a/drivers/base/memory.c +++ b/drivers/base/memory.c @@ -19,6 +19,7 @@ #include #include #include +#include #include #include #include @@ -166,6 +167,9 @@ static ssize_t show_mem_state(struct device *dev, case MEM_GOING_OFFLINE: len = sprintf(buf, "going-offline\n"); break; + case MEM_ISOLATED: + len = sprintf(buf, "isolated\n"); + break; default: len = sprintf(buf, "ERROR-UNKNOWN-%ld\n", mem->state); @@ -323,6 +327,9 @@ store_mem_state(struct device *dev, { struct memory_block *mem = to_memory_block(dev); int ret, online_type; + int isolated = 0; + unsigned long start_pfn; + unsigned long nr_pages = PAGES_PER_SECTION * sections_per_block; ret = lock_device_hotplug_sysfs(); if (ret) @@ -336,7 +343,13 @@ store_mem_state(struct device *dev, online_type = MMOP_ONLINE_KEEP; else if (sysfs_streq(buf, "offline")) online_type = MMOP_OFFLINE; - else { + else if (sysfs_streq(buf, "isolate")) { + isolated = 1; + goto memblock_isolated; + } else if (sysfs_streq(buf, "unisolate")) { + isolated = -1; + goto memblock_isolated; + } else { ret = -EINVAL; goto err; } @@ -366,6 +379,20 @@ store_mem_state(struct device *dev, mem_hotplug_done(); err: +memblock_isolated: + if (isolated == 1 && mem->state == MEM_ONLINE) { + start_pfn = section_nr_to_pfn(mem->start_section_nr); + ret = start_isolate_page_range(start_pfn, start_pfn + nr_pages, + MIGRATE_MOVABLE, true, true); + if (!ret) + mem->state = MEM_ISOLATED; + } else if (isolated == -1 && mem->state == MEM_ISOLATED) { + start_pfn = section_nr_to_pfn(mem->start_section_nr); + ret = undo_isolate_page_range(start_pfn, start_pfn + nr_pages, + MIGRATE_MOVABLE, true); + if (!ret) + mem->state = MEM_ONLINE; + } unlock_device_hotplug(); if (ret < 0) @@ -455,6 +482,7 @@ static DEVICE_ATTR(phys_index, 0444, show_mem_start_phys_index, NULL); static DEVICE_ATTR(state, 0644, show_mem_state, store_mem_state); static DEVICE_ATTR(phys_device, 0444, show_phys_device, NULL); static DEVICE_ATTR(removable, 0444, show_mem_removable, NULL); +//static DEVICE_ATTR(isolate, 0600, show_mem_isolate, store_mem_isolate); /* * Block size attribute stuff @@ -631,6 +659,7 @@ static struct attribute *memory_memblk_attrs[] = { #ifdef CONFIG_MEMORY_HOTREMOVE &dev_attr_valid_zones.attr, #endif + //&dev_attr_isolate.attr, NULL }; diff --git a/include/linux/memory.h b/include/linux/memory.h index a6ddefc..e00f22c 100644 --- a/include/linux/memory.h +++ b/include/linux/memory.h @@ -47,6 +47,7 @@ int set_memory_block_size_order(unsigned int order); #define MEM_GOING_ONLINE (1<<3) #define MEM_CANCEL_ONLINE (1<<4) #define MEM_CANCEL_OFFLINE (1<<5) +#define MEM_ISOLATED (1<<6) struct memory_notify { unsigned long start_pfn;