From patchwork Fri Oct 12 21:24:57 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Rientjes X-Patchwork-Id: 10639299 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id A461E933 for ; Fri, 12 Oct 2018 21:25:04 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 91203201A4 for ; Fri, 12 Oct 2018 21:25:04 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 850782BA2F; Fri, 12 Oct 2018 21:25:04 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE, USER_IN_DEF_DKIM_WL autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 9421A201A4 for ; Fri, 12 Oct 2018 21:25:02 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 6F3AF6B0003; Fri, 12 Oct 2018 17:25:01 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 656756B0005; Fri, 12 Oct 2018 17:25:01 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 4CE396B0006; Fri, 12 Oct 2018 17:25:01 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-pf1-f199.google.com (mail-pf1-f199.google.com [209.85.210.199]) by kanga.kvack.org (Postfix) with ESMTP id 092EA6B0003 for ; Fri, 12 Oct 2018 17:25:01 -0400 (EDT) Received: by mail-pf1-f199.google.com with SMTP id h76-v6so13263853pfd.10 for ; Fri, 12 Oct 2018 14:25:01 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:date:from:to:cc:subject :message-id:user-agent:mime-version; bh=LF/YoHEd6r00UlYo875GaIQxjYz0VZX1XmCm0FXDGnY=; b=a+oIeg00u+u10gckNANVmwYO6OG1hlIQXXvFmuCIfvHSap0n3FUaW4zIP573Q2hznh wykSjwhHQsZy75B4D4Jkmm0Fom/3zVyb9Di50P/TrquL24DJbnBuj8QIJj/WKqOyc2jX bZfY/rN50VZTTv1IsZ2CcNUzgMaeI+GLCIBbecVDIM2Rf5QZ6WsNVUUsygCr74b9Xv41 4U+sAb4//PQkevRo7eTCtfgvXLodVAl5MF+7v1sTi+lmSnOvqwMMVOc8kih4dgw33nd1 b05D0OhfYj+xWd2fbluebZCEGbiJRwBCjXCZoc8Puhv2D/+59OjZM/e9Ys0IUaW3eMsn vOZw== X-Gm-Message-State: ABuFfojlm+VfvYxbMsWRZrxYYveMve4uYq5p5c2h0LsnoTsSUgHojD7s +tHghv2vaWc/zAOec1pZ68a4Wi3wb3Yzt1/avSp8a4f0BJNT9Dl1vNlpJzfxsZIl7YHKI5EhKyI PFiMSgtJQlUfAquQgv0s213lwnVB9sABDWG9uPwcafMFNBrWWHZ3kjBYbGwABns/LPY3Rj+CDn0 TAsZO8CntNghYtQcC0doa69+iOnxoBe/0b9hO1QtqcrW1cME1VkSaxactacby0kj4B00YOcdsly UtnDjQO3ObMxnA1zI3odx6g/PcDPD393ZQwWjNRvXy0/Q2g6gbEVJ9NJs5PYV9a8GOxvttT579q Fj/hy1o+EvdxGIeP4E6WQKlonY7O3nTwFLEnoMTw8ciN2R59ildxrlMt0IwY4CT4D7QPTZR9zTo 7 X-Received: by 2002:a63:c508:: with SMTP id f8-v6mr7039254pgd.412.1539379500630; Fri, 12 Oct 2018 14:25:00 -0700 (PDT) X-Received: by 2002:a63:c508:: with SMTP id f8-v6mr7039220pgd.412.1539379499641; Fri, 12 Oct 2018 14:24:59 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1539379499; cv=none; d=google.com; s=arc-20160816; b=T8fhQYma5KtSs6Rm8mU4+uJinqJmMr7Zglzwmd/gdxGBLLLQvQNRVvrrTEGVox4b5L Yve5a/pdX2Vtr6vdyyJsf/h4PBmAVXAXay61hdOiG9Vb6pwl20UnB+QWluds7mK5+XQG O6RwBBXmRC91/Ib2IrvWjSogBa4SrTTprUephbawMZ33PfP2YRNCet89QWHR4P7M9oQD bFszpKRF5WRZZ1lR5rV/z5l/CGv6WvFk8d6dHvsoQI41dSZVabad8xO6TyZGyxakqYjX juU8eo0Wi8NzaSzGjv2Zgl7BfgJUCap0GcAdmN49QdaBpQsmJlRMzdPgeXmo+b8bP7e2 c6og== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=mime-version:user-agent:message-id:subject:cc:to:from:date :dkim-signature; bh=LF/YoHEd6r00UlYo875GaIQxjYz0VZX1XmCm0FXDGnY=; b=sdJoHNDzz4s5/QD5OQzIldy0wsnVv2aV2/9UGxQWj0zy+s2XOX9C1lrTbHb7B5e0DH 4Fm532HPUJL8CyjaAZi1/Jh7lsrEgWPvez59oZvTlTkDdUYmnSMVL0GuTfIpF/ijJAis tWa5w23OfZJONBdDtFRHPnC3ta8VpTiIfTJuSfjGTmjhzEmt3F8Qxu5jPTOJx4I5yF/A 9t9MGXB6I1ckyK3GZT7b/Z4mqUeXc/x8ItgM3av/GGP5pmzDqw8ynSikIFXXbcgt1cXM I4XxPx8adLLJ/K+NWzLnlA239AAlYDcSfpgqRjuZCrODWnW3yVU/4ZfQhUdkQl3JYqGV h2SQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=CixZwotb; spf=pass (google.com: domain of rientjes@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=rientjes@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id u6-v6sor1998756plz.53.2018.10.12.14.24.59 for (Google Transport Security); Fri, 12 Oct 2018 14:24:59 -0700 (PDT) Received-SPF: pass (google.com: domain of rientjes@google.com designates 209.85.220.65 as permitted sender) client-ip=209.85.220.65; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=CixZwotb; spf=pass (google.com: domain of rientjes@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=rientjes@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:from:to:cc:subject:message-id:user-agent:mime-version; bh=LF/YoHEd6r00UlYo875GaIQxjYz0VZX1XmCm0FXDGnY=; b=CixZwotbFH8KgYAAQfsjqqPc8VX4nNMA3w3JlMX9LFnULIo2RvcC5Xa0rVBeVHRS7M THTu8xr5tkBslxN+iZkx/zAVU2oGvto3o4nw9bDs4jYrosdllNKh2yRNeIx2gR9SsGT6 DqEJtkPXG6FhZGx1WwlhqrLposZKAqyW3Cqu7CGcOufea11aJpb2bJ4tajgLYwZbFfhi fpz797F46bZshzvh8nTRYtlwTiZXdSNeR5YClakfoCwc1wbaDVrg8Fcjt4gbUuCxEIw1 JsdCHluRL6h1I9+i2KeTsfCkjth5tIl5tI85Ttgcos6r6hRrlx/Y4i06ub6uWPeUGOTz IUUw== X-Google-Smtp-Source: ACcGV61ZNx1mUBELOUYEHjUrEGA4ZFvKoEeT5JrrP495aULPzFHQxGuPrff/j0QsJYYmMhZEhqkruQ== X-Received: by 2002:a17:902:74c2:: with SMTP id f2-v6mr7269417plt.45.1539379499023; Fri, 12 Oct 2018 14:24:59 -0700 (PDT) Received: from [2620:15c:17:3:3a5:23a7:5e32:4598] ([2620:15c:17:3:3a5:23a7:5e32:4598]) by smtp.gmail.com with ESMTPSA id g3-v6sm2981344pgn.37.2018.10.12.14.24.58 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Fri, 12 Oct 2018 14:24:58 -0700 (PDT) Date: Fri, 12 Oct 2018 14:24:57 -0700 (PDT) From: David Rientjes X-X-Sender: rientjes@chino.kir.corp.google.com To: Christoph Lameter , Pekka Enberg , Joonsoo Kim , Andrew Morton cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [patch] mm, slab: avoid high-order slab pages when it does not reduce waste Message-ID: User-Agent: Alpine 2.21 (DEB 202 2017-01-01) MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP The slab allocator has a heuristic that checks whether the internal fragmentation is satisfactory and, if not, increases cachep->gfporder to try to improve this. If the amount of waste is the same at higher cachep->gfporder values, there is no significant benefit to allocating higher order memory. There will be fewer calls to the page allocator, but each call will require zone->lock and finding the page of best fit from the per-zone free areas. Instead, it is better to allocate order-0 memory if possible so that pages can be returned from the per-cpu pagesets (pcp). There are two reasons to prefer this over allocating high order memory: - allocating from the pcp lists does not require a per-zone lock, and - this reduces stranding of MIGRATE_UNMOVABLE pageblocks on pcp lists that increases slab fragmentation across a zone. We are particularly interested in the second point to eliminate cases where all other pages on a pageblock are movable (or free) and fallback to pageblocks of other migratetypes from the per-zone free areas causes high-order slab memory to be allocated from them rather than from free MIGRATE_UNMOVABLE pages on the pcp. Signed-off-by: David Rientjes --- mm/slab.c | 15 +++++++++++++++ 1 file changed, 15 insertions(+) diff --git a/mm/slab.c b/mm/slab.c --- a/mm/slab.c +++ b/mm/slab.c @@ -1748,6 +1748,7 @@ static size_t calculate_slab_order(struct kmem_cache *cachep, for (gfporder = 0; gfporder <= KMALLOC_MAX_ORDER; gfporder++) { unsigned int num; size_t remainder; + int order; num = cache_estimate(gfporder, size, flags, &remainder); if (!num) @@ -1803,6 +1804,20 @@ static size_t calculate_slab_order(struct kmem_cache *cachep, */ if (left_over * 8 <= (PAGE_SIZE << gfporder)) break; + + /* + * If a higher gfporder would not reduce internal fragmentation, + * no need to continue. The preference is to keep gfporder as + * small as possible so slab allocations can be served from + * MIGRATE_UNMOVABLE pcp lists to avoid stranding. + */ + for (order = gfporder + 1; order <= slab_max_order; order++) { + cache_estimate(order, size, flags, &remainder); + if (remainder < left_over) + break; + } + if (order > slab_max_order) + break; } return left_over; }