From patchwork Tue Jul 12 02:28:05 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Rongwei Wang X-Patchwork-Id: 12914479 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 938D4CCA47B for ; Tue, 12 Jul 2022 02:28:21 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id BFECF940010; Mon, 11 Jul 2022 22:28:19 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id B3397940035; Mon, 11 Jul 2022 22:28:19 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 787E2940010; Mon, 11 Jul 2022 22:28:19 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 5DD65940033 for ; Mon, 11 Jul 2022 22:28:19 -0400 (EDT) Received: from smtpin26.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 37D9A3468D for ; Tue, 12 Jul 2022 02:28:19 +0000 (UTC) X-FDA: 79676863518.26.50E21BB Received: from out30-130.freemail.mail.aliyun.com (out30-130.freemail.mail.aliyun.com [115.124.30.130]) by imf17.hostedemail.com (Postfix) with ESMTP id 029EC40065 for ; Tue, 12 Jul 2022 02:28:15 +0000 (UTC) X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R141e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=ay29a033018045170;MF=rongwei.wang@linux.alibaba.com;NM=1;PH=DS;RN=10;SR=0;TI=SMTPD_---0VJ6Ep2C_1657592888; Received: from localhost.localdomain(mailfrom:rongwei.wang@linux.alibaba.com fp:SMTPD_---0VJ6Ep2C_1657592888) by smtp.aliyun-inc.com; Tue, 12 Jul 2022 10:28:10 +0800 From: Rongwei Wang To: akpm@linux-foundation.org, vbabka@suse.cz, 42.hyeyoo@gmail.com, roman.gushchin@linux.dev, iamjoonsoo.kim@lge.com, rientjes@google.com, penberg@kernel.org, cl@gentwo.de Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH v2 1/3] mm/slub: fix the race between validate_slab and slab_free Date: Tue, 12 Jul 2022 10:28:05 +0800 Message-Id: <20220712022807.44113-1-rongwei.wang@linux.alibaba.com> X-Mailer: git-send-email 2.32.0 MIME-Version: 1.0 ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1657592898; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:references; bh=MOJdFKxYXCIqbBknjodZOomTzrYrYkPDUOvvfZOx5/8=; b=nUM1JGXE7u0ZoPbilyZNh6SqqODrKGNJrtKAf6ua2q7DaLIXOh54JEEpCIfI5soOaEjZK3 PXFAx/BJP2ZbrAvWnoNCbNx4gjOb3GtyUx7qwXFiXtt4IhTL2LUX1xJpZn8gopnYNRR+PQ 1sPz/g04+YVsedkBho2xRjEpGWyCTSg= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1657592898; a=rsa-sha256; cv=none; b=YLReSjtwwUwAObkjIeiUxgTUSABazt3npQAqMVZDGK9kWgDCD8Os3qfNR30Bo5J5wWlbKJ RuyfXavZmjPqRsBGR3g0Au8Dfr+mzgwG03zoYbtRKB5omCXbVwkiuC+j78YzS2YSN+SKjW 28affDcgcG33+sgqJIylBRV7soLvc6g= ARC-Authentication-Results: i=1; imf17.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=alibaba.com; spf=pass (imf17.hostedemail.com: domain of rongwei.wang@linux.alibaba.com designates 115.124.30.130 as permitted sender) smtp.mailfrom=rongwei.wang@linux.alibaba.com X-Stat-Signature: rngfw8zc77eu897qctobgt5mrpk31nas X-Rspam-User: Authentication-Results: imf17.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=alibaba.com; spf=pass (imf17.hostedemail.com: domain of rongwei.wang@linux.alibaba.com designates 115.124.30.130 as permitted sender) smtp.mailfrom=rongwei.wang@linux.alibaba.com X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: 029EC40065 X-HE-Tag: 1657592895-861230 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: In use cases where allocating and freeing slab frequently, some error messages, such as "Left Redzone overwritten", "First byte 0xbb instead of 0xcc" would be printed when validating slabs. That's because an object has been filled with SLAB_RED_INACTIVE, but has not been added to slab's freelist. And between these two states, the behaviour of validating slab is likely to occur. Actually, it doesn't mean the slab can not work stably. But, these confusing messages will disturb slab debugging more or less. Signed-off-by: Rongwei Wang --- mm/slub.c | 43 +++++++++++++++++++++++++------------------ 1 file changed, 25 insertions(+), 18 deletions(-) diff --git a/mm/slub.c b/mm/slub.c index b1281b8654bd..e950d8df8380 100644 --- a/mm/slub.c +++ b/mm/slub.c @@ -1391,18 +1391,16 @@ static noinline int free_debug_processing( void *head, void *tail, int bulk_cnt, unsigned long addr) { - struct kmem_cache_node *n = get_node(s, slab_nid(slab)); void *object = head; int cnt = 0; - unsigned long flags, flags2; + unsigned long flags; int ret = 0; depot_stack_handle_t handle = 0; if (s->flags & SLAB_STORE_USER) handle = set_track_prepare(); - spin_lock_irqsave(&n->list_lock, flags); - slab_lock(slab, &flags2); + slab_lock(slab, &flags); if (s->flags & SLAB_CONSISTENCY_CHECKS) { if (!check_slab(s, slab)) @@ -1435,8 +1433,7 @@ static noinline int free_debug_processing( slab_err(s, slab, "Bulk freelist count(%d) invalid(%d)\n", bulk_cnt, cnt); - slab_unlock(slab, &flags2); - spin_unlock_irqrestore(&n->list_lock, flags); + slab_unlock(slab, &flags); if (!ret) slab_fix(s, "Object at 0x%p not freed", object); return ret; @@ -3330,7 +3327,7 @@ static void __slab_free(struct kmem_cache *s, struct slab *slab, { void *prior; - int was_frozen; + int was_frozen, to_take_off = 0; struct slab new; unsigned long counters; struct kmem_cache_node *n = NULL; @@ -3341,14 +3338,23 @@ static void __slab_free(struct kmem_cache *s, struct slab *slab, if (kfence_free(head)) return; - if (kmem_cache_debug(s) && - !free_debug_processing(s, slab, head, tail, cnt, addr)) - return; + n = get_node(s, slab_nid(slab)); + if (kmem_cache_debug(s)) { + int ret; - do { - if (unlikely(n)) { + spin_lock_irqsave(&n->list_lock, flags); + ret = free_debug_processing(s, slab, head, tail, cnt, addr); + if (!ret) { spin_unlock_irqrestore(&n->list_lock, flags); - n = NULL; + return; + } + } + + do { + if (unlikely(to_take_off)) { + if (!kmem_cache_debug(s)) + spin_unlock_irqrestore(&n->list_lock, flags); + to_take_off = 0; } prior = slab->freelist; counters = slab->counters; @@ -3369,8 +3375,6 @@ static void __slab_free(struct kmem_cache *s, struct slab *slab, new.frozen = 1; } else { /* Needs to be taken off a list */ - - n = get_node(s, slab_nid(slab)); /* * Speculatively acquire the list_lock. * If the cmpxchg does not succeed then we may @@ -3379,8 +3383,10 @@ static void __slab_free(struct kmem_cache *s, struct slab *slab, * Otherwise the list_lock will synchronize with * other processors updating the list of slabs. */ - spin_lock_irqsave(&n->list_lock, flags); + if (!kmem_cache_debug(s)) + spin_lock_irqsave(&n->list_lock, flags); + to_take_off = 1; } } @@ -3389,8 +3395,9 @@ static void __slab_free(struct kmem_cache *s, struct slab *slab, head, new.counters, "__slab_free")); - if (likely(!n)) { - + if (likely(!to_take_off)) { + if (kmem_cache_debug(s)) + spin_unlock_irqrestore(&n->list_lock, flags); if (likely(was_frozen)) { /* * The list lock was not taken therefore no list