From patchwork Fri Aug 14 17:31:24 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Minchan Kim X-Patchwork-Id: 11715047 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id A2969618 for ; Fri, 14 Aug 2020 17:31:43 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 6E61C208B3 for ; Fri, 14 Aug 2020 17:31:43 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="ZY/PZEiv" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 6E61C208B3 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id A338B8D0001; Fri, 14 Aug 2020 13:31:42 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 9E2EA6B0095; Fri, 14 Aug 2020 13:31:42 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8838E8D0001; Fri, 14 Aug 2020 13:31:42 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0036.hostedemail.com [216.40.44.36]) by kanga.kvack.org (Postfix) with ESMTP id 700606B000A for ; Fri, 14 Aug 2020 13:31:42 -0400 (EDT) Received: from smtpin24.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id 2DB45180AD815 for ; Fri, 14 Aug 2020 17:31:42 +0000 (UTC) X-FDA: 77149866444.24.magic94_240c20026ffe Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin24.hostedemail.com (Postfix) with ESMTP id DBA1C1A4AF for ; Fri, 14 Aug 2020 17:31:40 +0000 (UTC) X-Spam-Summary: 1,0,0,2c3da9f67f11b003,d41d8cd98f00b204,minchan.kim@gmail.com,,RULES_HIT:41:355:379:421:541:560:966:973:988:989:1260:1311:1314:1345:1437:1515:1535:1543:1605:1711:1730:1747:1777:1792:1801:2194:2196:2198:2199:2200:2201:2393:2559:2562:2693:2731:2736:3138:3139:3140:3141:3142:3865:3866:3867:3868:3870:3871:3872:4117:4250:4321:4385:4605:5007:6119:6120:6261:6630:6653:7875:7903:8603:9108:10004:11026:11473:11658:11914:12043:12291:12296:12297:12438:12517:12519:12555:12664:12679:12683:12895:12986:13161:13215:13229:13894:14096:14721:21080:21444:21451:21627:21740:21795:21990:30012:30034:30051:30054,0,RBL:209.85.216.67:@gmail.com:.lbl8.mailshell.net-66.100.201.100 62.50.0.100;04yftentytmfoophiip785bhks56qyphf76oit13gb4ywr3y9ajbauwn9j3upw3.h4ef7mpdwgfz66i171efb3eeq4j4fe6yy9a4hq3t3fr46i4ypd5q1yn389cxofb.q-lbl8.mailshell.net-223.238.255.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:25,LUA_SUM MARY:non X-HE-Tag: magic94_240c20026ffe X-Filterd-Recvd-Size: 6718 Received: from mail-pj1-f67.google.com (mail-pj1-f67.google.com [209.85.216.67]) by imf04.hostedemail.com (Postfix) with ESMTP for ; Fri, 14 Aug 2020 17:31:39 +0000 (UTC) Received: by mail-pj1-f67.google.com with SMTP id c6so4732345pje.1 for ; Fri, 14 Aug 2020 10:31:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=vJjxguHTrNHJFwdRjyMk1PMP4McDwPlH2CraBcQKKxY=; b=ZY/PZEivPpsjM1its4Z/fj+9y1Pw3WLdZc0PcrIXWTH4kug6JkXl191IRHz7Z1wMlb 2NI8Ch2KCOKqL+r7nz1YXXuzR+1u8xdlt6yaVh0Y13VGuVQEPl33l7RRbKMifsqGadTd lNW98+VJRsI6GIEo51fnKM35KHOxa6jNdYF03fcDOZlFCzhvNK6yQs7fN7tPS39ADxak vnFgdKVIe9RL3ERw/2Pjy00Fc11bq3ycUr18kz9Tz+EpCmpxo6LsB9BRaiYRsOPh88Tq N7TTZnAKL5FrpVQgSdzuIf8mH3t0h3h8zNUD+PQqRXs3MRV5wA2m9Y2leZ34Rnsv0IkN uG0A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:from:to:cc:subject:date:message-id :mime-version:content-transfer-encoding; bh=vJjxguHTrNHJFwdRjyMk1PMP4McDwPlH2CraBcQKKxY=; b=nplvGeNp25pebqiM+40hZiS6cZ5ns4oqW0ctCvNTzWgpD5gOtSdPzWvhLRj4Cdmq6H 1e+RP9TpUJJy1ysuchXyP7m0kyb1fHdz9wSdXsTy59Gfg/frBLU+CGua7ZwNSCputPfW tplhgDPSQawm5xBh38bnB5Joqp27fovto3WgiZ1vE39NlTQ7R4g4RUTTSZL3vLdPmekM s/0y31CpTYqxBxQ0aQOzHo11To05JqfHPqY25HdpXoyrBvlIlPU0R7RSF8P0VPxD8M1h 2pGmQnR6NZRQKhFu0tO3vfSGe3IeciJZCoyoUiDCwE2QPbXzTTFd7It9SMup0hIi2x33 I95Q== X-Gm-Message-State: AOAM531DMyIlVDc+kr98zAZ325z3QxGykZKlLcihQMCjkoKaFqYQw7tK Jjq/EfcFEFoUKy91WHrofMo= X-Google-Smtp-Source: ABdhPJz8kWhNlOPlyufPbCcZr+sgI50TYDepuKggl2gn5otMuRnG9sf+tydLD2c9hbQZGliMXWs/qQ== X-Received: by 2002:a17:90b:1092:: with SMTP id gj18mr3100491pjb.10.1597426298559; Fri, 14 Aug 2020 10:31:38 -0700 (PDT) Received: from bbox-1.mtv.corp.google.com ([2620:15c:211:1:7220:84ff:fe09:5e58]) by smtp.gmail.com with ESMTPSA id n22sm8522973pjq.25.2020.08.14.10.31.37 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 14 Aug 2020 10:31:37 -0700 (PDT) From: Minchan Kim To: Andrew Morton Cc: linux-mm , Joonsoo Kim , Vlastimil Babka , John Dias , Suren Baghdasaryan , pullip.cho@samsung.com, Minchan Kim Subject: [RFC 0/7] Support high-order page bulk allocation Date: Fri, 14 Aug 2020 10:31:24 -0700 Message-Id: <20200814173131.2803002-1-minchan@kernel.org> X-Mailer: git-send-email 2.28.0.220.ged08abb693-goog MIME-Version: 1.0 X-Rspamd-Queue-Id: DBA1C1A4AF X-Spamd-Result: default: False [0.00 / 100.00] X-Rspamd-Server: rspam04 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: There is a need for special HW to require bulk allocation of high-order pages. For example, 4800 * order-4 pages. To meet the requirement, a option is using CMA area because page allocator with compaction under memory pressure is easily failed to meet the requirement and too slow for 4800 times. However, CMA has also the following drawbacks: * 4800 of order-4 * cma_alloc is too slow To avoid the slowness, we could try to allocate 300M contiguous memory once and then split them into order-4 chunks. The problem of this approach is CMA allocation fails one of the pages in those range couldn't migrate out, which happens easily with fs write under memory pressure. To solve issues, this patch introduces alloc_pages_bulk. int alloc_pages_bulk(unsigned long start, unsigned long end, unsigned int migratetype, gfp_t gfp_mask, unsigned int order, unsigned int nr_elem, struct page **pages); It will investigate the [start, end) and migrate movable pages out there by best effort(by upcoming patches) to make requested order's free pages. The allocated pages will be returned using pages parameter. Return value represents how many of requested order pages we got. It could be less than user requested by nr_elem. /** * alloc_pages_bulk() -- tries to allocate high order pages * by batch from given range [start, end) * @start: start PFN to allocate * @end: one-past-the-last PFN to allocate * @migratetype: migratetype of the underlaying pageblocks (either * #MIGRATE_MOVABLE or #MIGRATE_CMA). All pageblocks * in range must have the same migratetype and it must * be either of the two. * @gfp_mask: GFP mask to use during compaction * @order: page order requested * @nr_elem: the number of high-order pages to allocate * @pages: page array pointer to store allocated pages (must * have space for at least nr_elem elements) * * The PFN range does not have to be pageblock or MAX_ORDER_NR_PAGES * aligned. The PFN range must belong to a single zone. * * Return: the number of pages allocated on success or negative error code. * The allocated pages should be freed using __free_pages */ The test goes order-4 * 4800 allocation(i.e., total 300MB) under kernel build workload. System RAM size is 1.5GB and CMA is 500M. With using CMA to allocate to 300M, ran 10 times trial, 10 time failed with big latency(up to several seconds). With this alloc_pages_bulk API, ran 10 time trial, 7 times are successful to allocate 4800 times. Rest 3 times are allocated 4799, 4789 and 4799. They are all done with 300ms. This patchset is against on next-20200813 Minchan Kim (7): mm: page_owner: split page by order mm: introduce split_page_by_order mm: compaction: deal with upcoming high-order page splitting mm: factor __alloc_contig_range out mm: introduce alloc_pages_bulk API mm: make alloc_pages_bulk best effort mm/page_isolation: avoid drain_all_pages for alloc_pages_bulk include/linux/gfp.h | 5 + include/linux/mm.h | 2 + include/linux/page-isolation.h | 1 + include/linux/page_owner.h | 10 +- mm/compaction.c | 64 +++++++---- mm/huge_memory.c | 2 +- mm/internal.h | 5 +- mm/page_alloc.c | 198 ++++++++++++++++++++++++++------- mm/page_isolation.c | 10 +- mm/page_owner.c | 7 +- 10 files changed, 230 insertions(+), 74 deletions(-)