From patchwork Fri Dec 14 23:02:56 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mel Gorman X-Patchwork-Id: 10731773 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 654EA13AD for ; Fri, 14 Dec 2018 23:03:18 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 17B562D639 for ; Fri, 14 Dec 2018 23:03:18 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 0B4962D63D; Fri, 14 Dec 2018 23:03:18 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 7F2C42D639 for ; Fri, 14 Dec 2018 23:03:17 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id B2DFC8E01DC; Fri, 14 Dec 2018 18:03:13 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id A82A58E021A; Fri, 14 Dec 2018 18:03:13 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 812CC8E0216; Fri, 14 Dec 2018 18:03:13 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-ed1-f72.google.com (mail-ed1-f72.google.com [209.85.208.72]) by kanga.kvack.org (Postfix) with ESMTP id 19D478E01DC for ; Fri, 14 Dec 2018 18:03:13 -0500 (EST) Received: by mail-ed1-f72.google.com with SMTP id l45so3463087edb.1 for ; Fri, 14 Dec 2018 15:03:13 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id; bh=R/bzNlwvflYXU48V8X1tuKm6Bc06LFozAAv6lToR55k=; b=ZotfOVFZz7BTqi/DwTmu7ETd7YgeT/e21Rj61NB13VldpJIPVSn3zNU73Q95Yu9l/E Rlb4A5zkuWbfKvvkGF06VnOnZ+FYtfscMTz6lzNo9PDtN0F9ZTdB2AmqmHoKQTN5V/Zr DAFt8DQWuMHCoVv/zpujw2tw74c5r1xSnrcIzz50Dgxt3PNkmnZKqJfb0pRqZvOkuAIm kYqvdKt/c5XTF5drERfl5c7V/+WbLhch+6AoTSAzxgyQ6wQJvf4VBVhFup85V7Wx8j1R eEp2TTyc14WrRaJ7IDkBLW7GFFyTm88ZplP9yrblq8ow/hJzYe8eMXalJP3s6GROqeMe mbTw== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of mgorman@techsingularity.net designates 46.22.139.230 as permitted sender) smtp.mailfrom=mgorman@techsingularity.net X-Gm-Message-State: AA+aEWYDSZNKOjXPsREiGt2g8DjMqEAR8tVXtyrVRV6tqkZ9/jdFnQo6 TyBaAK1apCM9YKdKydEWSgPqWNtsuQukS0dRTy38ankn0B+Eb4kRF6uEg2NBu/At+Oq4HaxpYhg BqcKuCX0TFIEcTUYfK3qqgus6narqDLcYRJP/UGTMAM6eoHLEqupOHQsUeDeTj9Ib4Q== X-Received: by 2002:a50:ae64:: with SMTP id c91mr4439499edd.222.1544828592465; Fri, 14 Dec 2018 15:03:12 -0800 (PST) X-Google-Smtp-Source: AFSGD/UfZkBXW3mp3wjlScKgGyATdiUNeQ50WhOwFaq4B0sFhk0yeG7m2TGotBi5gPxFpnN99sLz X-Received: by 2002:a50:ae64:: with SMTP id c91mr4439460edd.222.1544828591256; Fri, 14 Dec 2018 15:03:11 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1544828591; cv=none; d=google.com; s=arc-20160816; b=o6kyq5NpRzcIEz3AIr4W6RAchlv+myoCTBrmrSM0tvcsAwDyK8Cqf6ohFgTvb+BxLS KkZhm3nRtgaso6eSWF81wvVa68JwulEHhe0OJaPWgqtzbH3iGKVcqoWXDtUPUTlTzl2v cCnQ0zj4IjyXtn0iIVra3RNl6+RgcXMoVn9SkFv27NHW3T/24XX91uEpFuacmadiHJv3 zEiXsRAwuFAFCrhfKEUlV4zuUoWwvtlO/S3HxA34Dq2H4PEJl+yLDnVglxOFWBfH001J neIflGXxyvsH8rgD7c0OaPGFzvfvQeC2miru7VBaRVHZfGgx4SWDJL2e0a2Z+mnC9nhx aJTg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=message-id:date:subject:cc:to:from; bh=R/bzNlwvflYXU48V8X1tuKm6Bc06LFozAAv6lToR55k=; b=PUjSY+JsMf9dzLxH0p4iFyJFKabmb3oBGdDlgNYtGrpQbSumUOmiquoYhAeHtB0qEt 82TDE/om/QWFqgnXSd4pVm8najcdxVH8BDBjuVskwykiI3etU/7GYyp7DsR5a+Xdmrif LGu1x8Yn7c+qtsc4fhVxH3YGu+yWVThvFJe8C9VHKl2NS/N5R8mmV+aR7QahrUIF/CZz RRGug2ksejx+GVF4agTVWtcv8O0EPd5nXzfz4KoWcpXaTprWoZexszB64NgY8/KWG22T RYjQSwsdrwGGq6auCTXh/6zvsFl9BKCdUm6ALogITMWGas3+o/t6humpVhY7gHXGJzoJ V5bQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of mgorman@techsingularity.net designates 46.22.139.230 as permitted sender) smtp.mailfrom=mgorman@techsingularity.net Received: from outbound-smtp13.blacknight.com (outbound-smtp13.blacknight.com. [46.22.139.230]) by mx.google.com with ESMTPS id l1si2396298edn.1.2018.12.14.15.03.11 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 14 Dec 2018 15:03:11 -0800 (PST) Received-SPF: pass (google.com: domain of mgorman@techsingularity.net designates 46.22.139.230 as permitted sender) client-ip=46.22.139.230; Authentication-Results: mx.google.com; spf=pass (google.com: domain of mgorman@techsingularity.net designates 46.22.139.230 as permitted sender) smtp.mailfrom=mgorman@techsingularity.net Received: from mail.blacknight.com (unknown [81.17.254.10]) by outbound-smtp13.blacknight.com (Postfix) with ESMTPS id CC4B31C1D19 for ; Fri, 14 Dec 2018 23:03:10 +0000 (GMT) Received: (qmail 31145 invoked from network); 14 Dec 2018 23:03:10 -0000 Received: from unknown (HELO stampy.163woodhaven.lan) (mgorman@techsingularity.net@[37.228.245.71]) by 81.17.254.9 with ESMTPA; 14 Dec 2018 23:03:10 -0000 From: Mel Gorman To: Linux-MM Cc: David Rientjes , Andrea Arcangeli , Linus Torvalds , Michal Hocko , ying.huang@intel.com, kirill@shutemov.name, Andrew Morton , Linux List Kernel Mailing , Mel Gorman Subject: [RFC PATCH 00/14] Increase success rates and reduce latency of compaction v1 Date: Fri, 14 Dec 2018 23:02:56 +0000 Message-Id: <20181214230310.572-1-mgorman@techsingularity.net> X-Mailer: git-send-email 2.16.4 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP This is a very preliminary RFC. I'm posting this early as the __GFP_THISNODE discussion continues and has started looking at the compaction implementation and it'd be worth looking at this series fdirst. The cc list is based on that dicussion just to make them aware it exists. A v2 will have a significantly trimmed cc. This series reduces scan rates and success rates of compaction, primarily by using the free lists to shorten scans and better controlling of skip information and whether multiple scanners can target the same block. There still is much room for improvement but we probably should get these out of the way first as they are pre-requisites for anything smarter. Test data is still incomplete but I'm not expecting major differences on 2-socket vs 1-socket given the type of series. That might be wrong. Primarily I'm using thpscale to measure the impact of the series. The benchmark creates a large file, maps it, faults it, punches holes in the mapping so that the virtual address space is fragmented and then tries to allocate THP. It re-executes for different numbers of threads. From a fragmentation perspective, the workload is relatively benign but it does stress compaction. The overall impact on latencies for a 1-socket machine is 4.20.0-rc6 4.20.0-rc6 mmotm-20181210 capture-v1r8 Amean fault-both-3 3842.11 ( 0.00%) 2898.64 * 24.56%* Amean fault-both-5 5201.92 ( 0.00%) 4296.58 ( 17.40%) Amean fault-both-7 7086.15 ( 0.00%) 6203.55 ( 12.46%) Amean fault-both-12 11383.58 ( 0.00%) 9309.13 * 18.22%* Amean fault-both-18 16616.53 ( 0.00%) 6245.27 * 62.42%* Amean fault-both-24 18617.05 ( 0.00%) 15083.42 * 18.98%* Amean fault-both-30 24372.88 ( 0.00%) 11498.60 * 52.82%* Amean fault-both-32 22621.58 ( 0.00%) 9684.82 * 57.19%* 24-62% reduction in fault latency (be it base or huge page) The allocation success rates which are using MADV_HUGEPAGE are as follows; 4.20.0-rc6 4.20.0-rc6 mmotm-20181210 capture-v1r8 Percentage huge-3 85.74 ( 0.00%) 98.12 ( 14.44%) Percentage huge-5 89.16 ( 0.00%) 98.83 ( 10.85%) Percentage huge-7 85.98 ( 0.00%) 97.99 ( 13.97%) Percentage huge-12 84.19 ( 0.00%) 99.00 ( 17.59%) Percentage huge-18 81.20 ( 0.00%) 98.92 ( 21.83%) Percentage huge-24 82.60 ( 0.00%) 99.08 ( 19.95%) Percentage huge-30 82.87 ( 0.00%) 99.22 ( 19.74%) Percentage huge-32 81.98 ( 0.00%) 98.97 ( 20.72%) So, it's showing that the series is not far short of allocating 100% of the requested pages as huge pages. Finally the overall scan rates are as follows Compaction migrate scanned 677936161 4756927 Compaction free scanned 352045485 256525622 Kcompactd migrate scanned 751732 751080 Kcompactd free scanned 113784579 93099786 These are still pretty crazy scan rates but direct compaction migration scanning is reduced by 99% and the free scanner is reduced by 27% so it's a step in the right direction. include/linux/compaction.h | 3 +- include/linux/gfp.h | 7 +- include/linux/sched.h | 4 + kernel/sched/core.c | 3 + mm/compaction.c | 638 ++++++++++++++++++++++++++++++++++++++------- mm/internal.h | 21 +- mm/migrate.c | 2 +- mm/page_alloc.c | 70 ++++- 8 files changed, 637 insertions(+), 111 deletions(-)