From patchwork Fri Sep 17 11:07:54 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Marco Elver X-Patchwork-Id: 12501677 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-26.3 required=3.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT,USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id B31A1C433F5 for ; Fri, 17 Sep 2021 11:08:13 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 5292361152 for ; Fri, 17 Sep 2021 11:08:13 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 5292361152 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=kvack.org Received: by kanga.kvack.org (Postfix) id A6ACD6B0071; Fri, 17 Sep 2021 07:08:12 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id A1B036B0072; Fri, 17 Sep 2021 07:08:12 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 90A0A6B0073; Fri, 17 Sep 2021 07:08:12 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0079.hostedemail.com [216.40.44.79]) by kanga.kvack.org (Postfix) with ESMTP id 83AA26B0071 for ; Fri, 17 Sep 2021 07:08:12 -0400 (EDT) Received: from smtpin18.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id 42DF82D4A3 for ; Fri, 17 Sep 2021 11:08:12 +0000 (UTC) X-FDA: 78596791224.18.60BD914 Received: from mail-wr1-f74.google.com (mail-wr1-f74.google.com [209.85.221.74]) by imf29.hostedemail.com (Postfix) with ESMTP id E88E89000261 for ; Fri, 17 Sep 2021 11:08:11 +0000 (UTC) Received: by mail-wr1-f74.google.com with SMTP id x7-20020a5d6507000000b0015dada209b1so3580682wru.15 for ; Fri, 17 Sep 2021 04:08:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=date:message-id:mime-version:subject:from:to:cc; bh=41pNklJY3f6bJq0CDDORqOJ/YTpKyKNMUQMh29DmX70=; b=pSdK3RNTGgb6ZR+LjmIjo/L8cGudhf0OuXhKFLqKwg05Z+MEl0PnXm93KhtX/6zjgK APC5h6GFg60E4Doe+AVgmgVE8a9vIURM87P3A0kDaMbvYHPvIgiMdQ16rQz/ETwku+ZX YV7mLjnAB6z9SVUo+mYW+unNoHeg0Fl4IWdade2tGwn19vQch3Wy0h9BSTHoKRTyOAR4 LdK7zQ1sSXCISNSl+FRgARCS8WtMhcsBPWa/El/0J1BgpzahUv8l7w4Q+AlrG4wB0PZb Bl1eVanwkN8RX703FmFcNk2xGWAO+itGWYxXjkqH4gZlBLDBXf22xpY2dA4CzVgAUedT qokg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:message-id:mime-version:subject:from:to:cc; bh=41pNklJY3f6bJq0CDDORqOJ/YTpKyKNMUQMh29DmX70=; b=2ebKckdrTtcy1bJEqqFAjLnuASf78xWKOQbB6wRcyLRmEIy/Oi3z/HrXuQeF+5fBju m3LmN8Ch5Ihl4HDXXbgbzYVprONQBiOoius9hT0NZDUfTX7zhscHyEJhCbH8Wl7k+P30 o7rhevSBazc/Z4gjAwbj/XEfkJB8bAwikCktMDRfGWomsl78bbBxog+TaW0YwrXASYeA bCyfieLCFLrgwgdPII1+gqMSyXouoYZDpsxJdopUe+XM+QUbZ8ZI/9u9c0i3dUwSD8qy KJm2bkbRSdECoYggnw5lk6+bJTVyD97pm6FNPApeOveGlo/J25cQR81+wXUkzF07QDVy +DWA== X-Gm-Message-State: AOAM531slfa5VH24toz1dj4uNubf9CQzjyckTq+DO4N0GFYo9/u1aQDA IJDSceNVZF9j5ngEytkdklIqAhBEOg== X-Google-Smtp-Source: ABdhPJzDljaqKYaFNol+T+wBqwa8PBOn2KgT0SEtgTO/YpQkKcD7gcZ64ORPSnxnS0X/AnbRm282L7BKiQ== X-Received: from elver.muc.corp.google.com ([2a00:79e0:15:13:1a57:84a3:9bae:8070]) (user=elver job=sendgmr) by 2002:a05:600c:8a9:: with SMTP id l41mr14384504wmp.29.1631876890388; Fri, 17 Sep 2021 04:08:10 -0700 (PDT) Date: Fri, 17 Sep 2021 13:07:54 +0200 Message-Id: <20210917110756.1121272-1-elver@google.com> Mime-Version: 1.0 X-Mailer: git-send-email 2.33.0.464.g1972c5931b-goog Subject: [PATCH 1/3] kfence: count unexpectedly skipped allocations From: Marco Elver To: elver@google.com, Andrew Morton Cc: Alexander Potapenko , Dmitry Vyukov , Aleksandr Nogikh , Taras Madan , linux-kernel@vger.kernel.org, linux-mm@kvack.org, kasan-dev@googlegroups.com X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: E88E89000261 X-Stat-Signature: royctqmm6ii11ayjwnew45mabhcjzxfx Authentication-Results: imf29.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=pSdK3RNT; spf=pass (imf29.hostedemail.com: domain of 3GndEYQUKCCsLScLYNVVNSL.JVTSPUbe-TTRcHJR.VYN@flex--elver.bounces.google.com designates 209.85.221.74 as permitted sender) smtp.mailfrom=3GndEYQUKCCsLScLYNVVNSL.JVTSPUbe-TTRcHJR.VYN@flex--elver.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com X-HE-Tag: 1631876891-146623 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Maintain a counter to count allocations that are skipped due to being incompatible (oversized, incompatible gfp flags) or no capacity. This is to compute the fraction of allocations that could not be serviced by KFENCE, which we expect to be rare. Signed-off-by: Marco Elver --- mm/kfence/core.c | 20 ++++++++++++++++---- 1 file changed, 16 insertions(+), 4 deletions(-) diff --git a/mm/kfence/core.c b/mm/kfence/core.c index 7a97db8bc8e7..2755800f3e2a 100644 --- a/mm/kfence/core.c +++ b/mm/kfence/core.c @@ -112,6 +112,8 @@ enum kfence_counter_id { KFENCE_COUNTER_FREES, KFENCE_COUNTER_ZOMBIES, KFENCE_COUNTER_BUGS, + KFENCE_COUNTER_SKIP_INCOMPAT, + KFENCE_COUNTER_SKIP_CAPACITY, KFENCE_COUNTER_COUNT, }; static atomic_long_t counters[KFENCE_COUNTER_COUNT]; @@ -121,6 +123,8 @@ static const char *const counter_names[] = { [KFENCE_COUNTER_FREES] = "total frees", [KFENCE_COUNTER_ZOMBIES] = "zombie allocations", [KFENCE_COUNTER_BUGS] = "total bugs", + [KFENCE_COUNTER_SKIP_INCOMPAT] = "skipped allocations (incompatible)", + [KFENCE_COUNTER_SKIP_CAPACITY] = "skipped allocations (capacity)", }; static_assert(ARRAY_SIZE(counter_names) == KFENCE_COUNTER_COUNT); @@ -272,7 +276,7 @@ static void *kfence_guarded_alloc(struct kmem_cache *cache, size_t size, gfp_t g } raw_spin_unlock_irqrestore(&kfence_freelist_lock, flags); if (!meta) - return NULL; + goto no_capacity; if (unlikely(!raw_spin_trylock_irqsave(&meta->lock, flags))) { /* @@ -289,7 +293,7 @@ static void *kfence_guarded_alloc(struct kmem_cache *cache, size_t size, gfp_t g list_add_tail(&meta->list, &kfence_freelist); raw_spin_unlock_irqrestore(&kfence_freelist_lock, flags); - return NULL; + goto no_capacity; } meta->addr = metadata_to_pageaddr(meta); @@ -349,6 +353,10 @@ static void *kfence_guarded_alloc(struct kmem_cache *cache, size_t size, gfp_t g atomic_long_inc(&counters[KFENCE_COUNTER_ALLOCS]); return addr; + +no_capacity: + atomic_long_inc(&counters[KFENCE_COUNTER_SKIP_CAPACITY]); + return NULL; } static void kfence_guarded_free(void *addr, struct kfence_metadata *meta, bool zombie) @@ -740,8 +748,10 @@ void *__kfence_alloc(struct kmem_cache *s, size_t size, gfp_t flags) * Perform size check before switching kfence_allocation_gate, so that * we don't disable KFENCE without making an allocation. */ - if (size > PAGE_SIZE) + if (size > PAGE_SIZE) { + atomic_long_inc(&counters[KFENCE_COUNTER_SKIP_INCOMPAT]); return NULL; + } /* * Skip allocations from non-default zones, including DMA. We cannot @@ -749,8 +759,10 @@ void *__kfence_alloc(struct kmem_cache *s, size_t size, gfp_t flags) * properties (e.g. reside in DMAable memory). */ if ((flags & GFP_ZONEMASK) || - (s->flags & (SLAB_CACHE_DMA | SLAB_CACHE_DMA32))) + (s->flags & (SLAB_CACHE_DMA | SLAB_CACHE_DMA32))) { + atomic_long_inc(&counters[KFENCE_COUNTER_SKIP_INCOMPAT]); return NULL; + } /* * allocation_gate only needs to become non-zero, so it doesn't make