From patchwork Mon Jul 18 19:28:29 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yury Norov X-Patchwork-Id: 12921645 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 67CA1CCA479 for ; Mon, 18 Jul 2022 19:29:14 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236018AbiGRT3E (ORCPT ); Mon, 18 Jul 2022 15:29:04 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45906 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235942AbiGRT2w (ORCPT ); Mon, 18 Jul 2022 15:28:52 -0400 Received: from mail-qk1-x72e.google.com (mail-qk1-x72e.google.com [IPv6:2607:f8b0:4864:20::72e]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D90F6FE2; Mon, 18 Jul 2022 12:28:49 -0700 (PDT) Received: by mail-qk1-x72e.google.com with SMTP id k7so6616102qkj.2; Mon, 18 Jul 2022 12:28:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=tW+JBzHw5uzW90VtxMVsK0vDh/ivXEOTk2ZO6iDHY3Y=; b=QfIBfiSefrMyF6U/utafae4Q2KqqgoV9UkXNUb/cNCDimVKBt0G1tfaL/mHBXzf1mQ jRw/wMkWn7yTRTAW5wqRwDMcLehBUnTyHP1bAf1Hapka0ilwXuG3pbwAsn/kLd8tfral jyOoXZ/TO57KVdLWf2b3SR/6OS0pvHPEJZCi69W4HoyaVwk/7o+Ed9CRJrGv/1O0uQZN HrF83DNgnBGRr4aYaIyOH8Av2zaMpfycUOm9pmFtDQvupTz59BQ/flMgqa+p4d0aFxQN P2jzNvocuGcejcl25q+JMXXR5RwO0bp5TXYdNVz4b2Y+gex8C66otzWU7VsXNaZNqpaz M+uw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=tW+JBzHw5uzW90VtxMVsK0vDh/ivXEOTk2ZO6iDHY3Y=; b=fQKdxWXpq+bElqIRWWN9kGkzjxCDvjDqC76Dy0L0MVYQ2kmxtsO0oSESlkJX1iiPI2 g3FHKIytbIjW00ritista2SYuEECWQ46PwAf10h/I+fwFBhmCuSyMkfFuUQFWOXH9Z5e 0Ok+t7RHg9eZnxIx73jbO28a0p2uHvm22yj66PP18QR+m3G9bmjMY4+nSVV58Sy/5tTu OzGd1wLzWagMs7CsyRKuGB+4yRALffbYqi6EM7NwH+O4WgNM8bf2/3xQVC5zyBSVpef1 /Rby6GBQmyeIL1yHnOJdhSEP3Tf2KpXV3Irs3cDEakNPJ9NgPsGJWMHYxQie+/xlChzc iGtw== X-Gm-Message-State: AJIora/Q8H16SpOjw/V29qSL+bSiNx0uQIfizuZeuDDl68ADbZPY4F+q IEvNQHRD1G4W4/SK1S1UwsWWkvCfF1L2xw== X-Google-Smtp-Source: AGRyM1tin0DA+F/kE9pZ3f6UhK4Fjyxi8XCE6itTUh8vPN2OIeGacYVTcS32W6yBP+j3ajMQQ0HxIA== X-Received: by 2002:a37:94b:0:b0:6b5:be32:4f2b with SMTP id 72-20020a37094b000000b006b5be324f2bmr17090382qkj.481.1658172528071; Mon, 18 Jul 2022 12:28:48 -0700 (PDT) Received: from localhost ([2601:4c1:c100:1230:ab01:d009:465a:5ab1]) by smtp.gmail.com with ESMTPSA id s11-20020a05620a0bcb00b006b5c5987ff2sm11273500qki.96.2022.07.18.12.28.47 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 18 Jul 2022 12:28:47 -0700 (PDT) From: Yury Norov To: linux-kernel@vger.kernel.org, Alexander Lobakin , Alexei Starovoitov , Alexey Klimov , Andrew Morton , Andrii Nakryiko , Andy Shevchenko , Ben Segall , Christoph Lameter , Dan Williams , Daniel Borkmann , Daniel Bristot de Oliveira , Dennis Zhou , Dietmar Eggemann , Eric Dumazet , Frederic Weisbecker , Guenter Roeck , Ingo Molnar , Isabella Basso , John Fastabend , Josh Poimboeuf , Juergen Gross , Juri Lelli , KP Singh , Kees Cook , Martin KaFai Lau , Mel Gorman , Miroslav Benes , Nathan Chancellor , "Paul E . McKenney" , Peter Zijlstra , Randy Dunlap , Rasmus Villemoes , Sebastian Andrzej Siewior , Song Liu , Steven Rostedt , Tejun Heo , Thomas Gleixner , Valentin Schneider , Vincent Guittot , Vlastimil Babka , Yonghong Song , Yury Norov , linux-mm@kvack.org, netdev@vger.kernel.org, bpf@vger.kernel.org Subject: [PATCH 01/16] lib/bitmap: add bitmap_check_params() Date: Mon, 18 Jul 2022 12:28:29 -0700 Message-Id: <20220718192844.1805158-2-yury.norov@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220718192844.1805158-1-yury.norov@gmail.com> References: <20220718192844.1805158-1-yury.norov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org bitmap_check_params() takes all arguments passed into bitmap functions and runs sanity checks. bitmap_check(), bitmap_check_op() and bitmap_check_move() are convenient wrappers for frequent cases. The following patches of this series clear all warnings found with bitmap_check_params() for x86_64, arm64 and powerpc64. The last patch introduces CONFIG_DEBUG_BITMAP option to let user enable bitmap_check_params(). No functional changes for existing kernel users, and for the following functions inline parameters checks removed: - bitmap_pos_to_ord; - bitmap_remap; - bitmap_onto; - bitmap_fold. Signed-off-by: Yury Norov --- include/linux/bitmap.h | 95 +++++++++++++++++++++++++++++++ lib/bitmap.c | 123 ++++++++++++++++++++++++++++++++++++++--- 2 files changed, 209 insertions(+), 9 deletions(-) diff --git a/include/linux/bitmap.h b/include/linux/bitmap.h index 035d4ac66641..6a0d9170c4f0 100644 --- a/include/linux/bitmap.h +++ b/include/linux/bitmap.h @@ -8,9 +8,46 @@ #include #include #include +#include #include #include +#define CHECK_B2 BIT(0) +#define CHECK_B3 BIT(1) +#define CHECK_START BIT(2) +#define CHECK_OFF BIT(3) +#define CHECK_OVERLAP12 BIT(4) +#define CHECK_OVERLAP13 BIT(5) +#define CHECK_OVERLAP23 BIT(6) +#define CHECK_OFF_EQ_0 BIT(7) +#define CHECK_START_LE_OFF BIT(8) + +#define NBITS_MAX (INT_MAX-1) + +#ifdef CONFIG_DEBUG_BITMAP +#define bitmap_check_params(b1, b2, b3, nbits, start, off, flags) \ + do { \ + if (__bitmap_check_params((b1), (b2), (b3), (nbits), \ + (start), (off), (flags))) { \ + pr_warn("Bitmap: parameters check failed"); \ + pr_warn("%s [%d]: %s\n", __FILE__, __LINE__, __func__); \ + } \ + } while (0) + +bool __bitmap_check_params(const unsigned long *b1, const unsigned long *b2, + const unsigned long *b3, const unsigned long nbits, + const unsigned long start, const unsigned long off, + const unsigned long flags); +#else +#define bitmap_check_params(b1, b2, b3, nbits, start, off, flags) +#endif + +#define bitmap_check(buf, nbits) bitmap_check_params(buf, NULL, NULL, nbits, 0, 0, 0) +#define bitmap_check_op(dst, src1, src2, nbits) \ + bitmap_check_params(dst, src1, src2, nbits, 0, 0, CHECK_B2 | CHECK_B3 | CHECK_OVERLAP23) +#define bitmap_check_move(dst, src, nbits) \ + bitmap_check_params(dst, src, NULL, nbits, 0, 0, CHECK_B2 | CHECK_OVERLAP12) + struct device; /* @@ -239,6 +276,8 @@ static inline void bitmap_zero(unsigned long *dst, unsigned int nbits) { unsigned int len = BITS_TO_LONGS(nbits) * sizeof(unsigned long); + bitmap_check(dst, nbits); + if (small_const_nbits(nbits)) *dst = 0; else @@ -249,6 +288,8 @@ static inline void bitmap_fill(unsigned long *dst, unsigned int nbits) { unsigned int len = BITS_TO_LONGS(nbits) * sizeof(unsigned long); + bitmap_check(dst, nbits); + if (small_const_nbits(nbits)) *dst = ~0UL; else @@ -260,6 +301,8 @@ static inline void bitmap_copy(unsigned long *dst, const unsigned long *src, { unsigned int len = BITS_TO_LONGS(nbits) * sizeof(unsigned long); + bitmap_check_move(dst, src, nbits); + if (small_const_nbits(nbits)) *dst = *src; else @@ -318,6 +361,8 @@ void bitmap_to_arr64(u64 *buf, const unsigned long *bitmap, unsigned int nbits); static inline bool bitmap_and(unsigned long *dst, const unsigned long *src1, const unsigned long *src2, unsigned int nbits) { + bitmap_check_op(dst, src1, src2, nbits); + if (small_const_nbits(nbits)) return (*dst = *src1 & *src2 & BITMAP_LAST_WORD_MASK(nbits)) != 0; return __bitmap_and(dst, src1, src2, nbits); @@ -326,6 +371,8 @@ static inline bool bitmap_and(unsigned long *dst, const unsigned long *src1, static inline void bitmap_or(unsigned long *dst, const unsigned long *src1, const unsigned long *src2, unsigned int nbits) { + bitmap_check_op(dst, src1, src2, nbits); + if (small_const_nbits(nbits)) *dst = *src1 | *src2; else @@ -335,6 +382,8 @@ static inline void bitmap_or(unsigned long *dst, const unsigned long *src1, static inline void bitmap_xor(unsigned long *dst, const unsigned long *src1, const unsigned long *src2, unsigned int nbits) { + bitmap_check_op(dst, src1, src2, nbits); + if (small_const_nbits(nbits)) *dst = *src1 ^ *src2; else @@ -344,6 +393,8 @@ static inline void bitmap_xor(unsigned long *dst, const unsigned long *src1, static inline bool bitmap_andnot(unsigned long *dst, const unsigned long *src1, const unsigned long *src2, unsigned int nbits) { + bitmap_check_op(dst, src1, src2, nbits); + if (small_const_nbits(nbits)) return (*dst = *src1 & ~(*src2) & BITMAP_LAST_WORD_MASK(nbits)) != 0; return __bitmap_andnot(dst, src1, src2, nbits); @@ -352,6 +403,8 @@ static inline bool bitmap_andnot(unsigned long *dst, const unsigned long *src1, static inline void bitmap_complement(unsigned long *dst, const unsigned long *src, unsigned int nbits) { + bitmap_check_move(dst, src, nbits); + if (small_const_nbits(nbits)) *dst = ~(*src); else @@ -368,6 +421,8 @@ static inline void bitmap_complement(unsigned long *dst, const unsigned long *sr static inline bool bitmap_equal(const unsigned long *src1, const unsigned long *src2, unsigned int nbits) { + bitmap_check_move(src1, src2, nbits); + if (small_const_nbits(nbits)) return !((*src1 ^ *src2) & BITMAP_LAST_WORD_MASK(nbits)); if (__builtin_constant_p(nbits & BITMAP_MEM_MASK) && @@ -390,6 +445,9 @@ static inline bool bitmap_or_equal(const unsigned long *src1, const unsigned long *src3, unsigned int nbits) { + bitmap_check_params(src1, src2, src3, nbits, 0, 0, CHECK_B2 | CHECK_B3 | + CHECK_OVERLAP12 | CHECK_OVERLAP13 | CHECK_OVERLAP23); + if (!small_const_nbits(nbits)) return __bitmap_or_equal(src1, src2, src3, nbits); @@ -400,6 +458,8 @@ static inline bool bitmap_intersects(const unsigned long *src1, const unsigned long *src2, unsigned int nbits) { + bitmap_check_move(src1, src2, nbits); + if (small_const_nbits(nbits)) return ((*src1 & *src2) & BITMAP_LAST_WORD_MASK(nbits)) != 0; else @@ -409,6 +469,8 @@ static inline bool bitmap_intersects(const unsigned long *src1, static inline bool bitmap_subset(const unsigned long *src1, const unsigned long *src2, unsigned int nbits) { + bitmap_check_move(src1, src2, nbits); + if (small_const_nbits(nbits)) return ! ((*src1 & ~(*src2)) & BITMAP_LAST_WORD_MASK(nbits)); else @@ -417,6 +479,8 @@ static inline bool bitmap_subset(const unsigned long *src1, static inline bool bitmap_empty(const unsigned long *src, unsigned nbits) { + bitmap_check(src, nbits); + if (small_const_nbits(nbits)) return ! (*src & BITMAP_LAST_WORD_MASK(nbits)); @@ -425,6 +489,8 @@ static inline bool bitmap_empty(const unsigned long *src, unsigned nbits) static inline bool bitmap_full(const unsigned long *src, unsigned int nbits) { + bitmap_check(src, nbits); + if (small_const_nbits(nbits)) return ! (~(*src) & BITMAP_LAST_WORD_MASK(nbits)); @@ -434,6 +500,8 @@ static inline bool bitmap_full(const unsigned long *src, unsigned int nbits) static __always_inline unsigned long bitmap_weight(const unsigned long *src, unsigned int nbits) { + bitmap_check(src, nbits); + if (small_const_nbits(nbits)) return hweight_long(*src & BITMAP_LAST_WORD_MASK(nbits)); return __bitmap_weight(src, nbits); @@ -442,6 +510,9 @@ unsigned long bitmap_weight(const unsigned long *src, unsigned int nbits) static __always_inline void bitmap_set(unsigned long *map, unsigned int start, unsigned int nbits) { + bitmap_check_params(map, NULL, NULL, start + nbits, start, nbits, + CHECK_START | CHECK_OFF | CHECK_OFF_EQ_0); + if (__builtin_constant_p(nbits) && nbits == 1) __set_bit(start, map); else if (small_const_nbits(start + nbits)) @@ -458,6 +529,9 @@ static __always_inline void bitmap_set(unsigned long *map, unsigned int start, static __always_inline void bitmap_clear(unsigned long *map, unsigned int start, unsigned int nbits) { + bitmap_check_params(map, NULL, NULL, start + nbits, start, nbits, + CHECK_START | CHECK_OFF | CHECK_OFF_EQ_0); + if (__builtin_constant_p(nbits) && nbits == 1) __clear_bit(start, map); else if (small_const_nbits(start + nbits)) @@ -474,6 +548,8 @@ static __always_inline void bitmap_clear(unsigned long *map, unsigned int start, static inline void bitmap_shift_right(unsigned long *dst, const unsigned long *src, unsigned int shift, unsigned int nbits) { + bitmap_check_params(dst, src, NULL, nbits, shift, 0, CHECK_START); + if (small_const_nbits(nbits)) *dst = (*src & BITMAP_LAST_WORD_MASK(nbits)) >> shift; else @@ -483,6 +559,8 @@ static inline void bitmap_shift_right(unsigned long *dst, const unsigned long *s static inline void bitmap_shift_left(unsigned long *dst, const unsigned long *src, unsigned int shift, unsigned int nbits) { + bitmap_check_params(dst, src, NULL, nbits, shift, 0, CHECK_START); + if (small_const_nbits(nbits)) *dst = (*src << shift) & BITMAP_LAST_WORD_MASK(nbits); else @@ -495,6 +573,10 @@ static inline void bitmap_replace(unsigned long *dst, const unsigned long *mask, unsigned int nbits) { + bitmap_check_op(dst, old, mask, nbits); + bitmap_check_op(dst, new, mask, nbits); + bitmap_check_op(dst, old, new, nbits); + if (small_const_nbits(nbits)) *dst = (*old & ~(*mask)) | (*new & *mask); else @@ -505,6 +587,8 @@ static inline void bitmap_next_set_region(unsigned long *bitmap, unsigned int *rs, unsigned int *re, unsigned int end) { + bitmap_check(bitmap, end); + *rs = find_next_bit(bitmap, end, *rs); *re = find_next_zero_bit(bitmap, end, *rs + 1); } @@ -571,6 +655,11 @@ static inline unsigned long bitmap_get_value8(const unsigned long *map, const size_t index = BIT_WORD(start); const unsigned long offset = start % BITS_PER_LONG; +#ifdef DEBUG_BITMAP + bitmap_check_params(map, NULL, NULL, start + 8, start, 0, CHECK_START); + WARN_ON(start % 8); +#endif + return (map[index] >> offset) & 0xFF; } @@ -586,6 +675,12 @@ static inline void bitmap_set_value8(unsigned long *map, unsigned long value, const size_t index = BIT_WORD(start); const unsigned long offset = start % BITS_PER_LONG; +#ifdef DEBUG_BITMAP + bitmap_check_params(map, NULL, NULL, start + 8, start, 0, CHECK_START); + WARN_ON(start % 8); + WARN_ON(value > 0xFFUL); +#endif + map[index] &= ~(0xFFUL << offset); map[index] |= value << offset; } diff --git a/lib/bitmap.c b/lib/bitmap.c index 2b67cd657692..cd4dd848ea6a 100644 --- a/lib/bitmap.c +++ b/lib/bitmap.c @@ -22,6 +22,71 @@ #include "kstrtox.h" +#ifdef CONFIG_DEBUG_BITMAP +static inline const bool check_overlap(const unsigned long *b1, const unsigned long *b2, + unsigned long nbits) +{ + return min(b1, b2) + DIV_ROUND_UP(nbits, BITS_PER_LONG) > max(b1, b2); +} + +bool __bitmap_check_params(const unsigned long *b1, const unsigned long *b2, + const unsigned long *b3, const unsigned long nbits, + const unsigned long start, const unsigned long off, + const unsigned long flags) +{ + bool warn = false; + + warn |= WARN_ON(b1 == NULL); + warn |= WARN_ON(nbits == 0); + warn |= WARN_ON(nbits > NBITS_MAX); + + if (flags & CHECK_B2) { + warn |= WARN_ON(b2 == NULL); + warn |= WARN_ON(flags & CHECK_OVERLAP12 && + check_overlap(b1, b2, nbits)); + } + + if (flags & CHECK_B3) { + warn |= WARN_ON(b3 == NULL); + warn |= WARN_ON(flags & CHECK_OVERLAP13 && + check_overlap(b1, b3, nbits)); + } + + if (flags & CHECK_OVERLAP23) + warn |= WARN_ON(check_overlap(b2, b3, nbits)); + + if (flags & CHECK_START) + warn |= WARN_ON(start >= nbits); + + if (flags & CHECK_OFF) + warn |= WARN_ON(off > nbits); + + if (flags & CHECK_OFF_EQ_0) + warn |= WARN_ON(off == 0); + + if (flags & CHECK_START_LE_OFF) + warn |= WARN_ON(start > off); + + if (flags & CHECK_B2 && flags & CHECK_B3) + warn |= WARN_ON(b2 == b3); + + if (warn) { + /* + * Convert kernel addresses to unsigned long because + * %pK hides actual values with the lack of randomization. + */ + pr_warn("b1:\t\t%lx\n", (unsigned long)b1); + pr_warn("b2:\t\t%lx\n", (unsigned long)b2); + pr_warn("b3:\t\t%lx\n", (unsigned long)b3); + pr_warn("nbits:\t%lu\n", nbits); + pr_warn("start:\t%lu\n", start); + pr_warn("off:\t%lu\n", off); + } + return warn; +} +EXPORT_SYMBOL(__bitmap_check_params); +#endif + /** * DOC: bitmap introduction * @@ -214,6 +279,9 @@ void bitmap_cut(unsigned long *dst, const unsigned long *src, unsigned long keep = 0, carry; int i; + bitmap_check_params(dst, src, NULL, nbits, first, first + cut, + CHECK_B2 | CHECK_START | CHECK_OFF); + if (first % BITS_PER_LONG) { keep = src[first / BITS_PER_LONG] & (~0UL >> (BITS_PER_LONG - first % BITS_PER_LONG)); @@ -410,6 +478,10 @@ unsigned long bitmap_find_next_zero_area_off(unsigned long *map, unsigned long align_offset) { unsigned long index, end, i; + + bitmap_check_params(map, NULL, NULL, size, start, start + nr, + CHECK_START | CHECK_OFF | CHECK_START_LE_OFF); + again: index = find_next_zero_bit(map, size, start); @@ -797,6 +869,8 @@ int bitmap_parselist(const char *buf, unsigned long *maskp, int nmaskbits) struct region r; long ret; + bitmap_check(maskp, nmaskbits); + r.nbits = nmaskbits; bitmap_zero(maskp, r.nbits); @@ -900,6 +974,8 @@ int bitmap_parse(const char *start, unsigned int buflen, int unset_bit; int chunk; + bitmap_check(maskp, nmaskbits); + for (chunk = 0; ; chunk++) { end = bitmap_find_region_reverse(start, end); if (start > end) @@ -950,7 +1026,9 @@ EXPORT_SYMBOL(bitmap_parse); */ static int bitmap_pos_to_ord(const unsigned long *buf, unsigned int pos, unsigned int nbits) { - if (pos >= nbits || !test_bit(pos, buf)) + bitmap_check_params(buf, NULL, NULL, nbits, pos, 0, CHECK_START); + + if (!test_bit(pos, buf)) return -1; return __bitmap_weight(buf, pos); @@ -1024,8 +1102,13 @@ void bitmap_remap(unsigned long *dst, const unsigned long *src, { unsigned int oldbit, w; - if (dst == src) /* following doesn't handle inplace remaps */ - return; + bitmap_check_params(dst, src, old, nbits, 0, 0, + CHECK_B2 | CHECK_B3 | CHECK_OVERLAP12 | + CHECK_OVERLAP13 | CHECK_OVERLAP23); + bitmap_check_params(dst, src, new, nbits, 0, 0, + CHECK_B2 | CHECK_B3 | CHECK_OVERLAP12 | + CHECK_OVERLAP13 | CHECK_OVERLAP23); + bitmap_zero(dst, nbits); w = bitmap_weight(new, nbits); @@ -1069,8 +1152,13 @@ EXPORT_SYMBOL(bitmap_remap); int bitmap_bitremap(int oldbit, const unsigned long *old, const unsigned long *new, int bits) { - int w = bitmap_weight(new, bits); - int n = bitmap_pos_to_ord(old, oldbit, bits); + int w, n; + + bitmap_check_params(old, new, NULL, bits, oldbit, 0, CHECK_B2 | + CHECK_START | CHECK_OVERLAP12); + + w = bitmap_weight(new, bits); + n = bitmap_pos_to_ord(old, oldbit, bits); if (n < 0 || w == 0) return oldbit; else @@ -1190,8 +1278,9 @@ void bitmap_onto(unsigned long *dst, const unsigned long *orig, { unsigned int n, m; /* same meaning as in above comment */ - if (dst == orig) /* following doesn't handle inplace mappings */ - return; + bitmap_check_params(dst, orig, relmap, bits, 0, 0, CHECK_B2 | CHECK_B3 | + CHECK_OVERLAP12 | CHECK_OVERLAP13 | CHECK_OVERLAP23); + bitmap_zero(dst, bits); /* @@ -1229,8 +1318,8 @@ void bitmap_fold(unsigned long *dst, const unsigned long *orig, { unsigned int oldbit; - if (dst == orig) /* following doesn't handle inplace mappings */ - return; + bitmap_check_move(dst, orig, sz); + bitmap_zero(dst, nbits); for_each_set_bit(oldbit, orig, nbits) @@ -1332,6 +1421,8 @@ int bitmap_find_free_region(unsigned long *bitmap, unsigned int bits, int order) { unsigned int pos, end; /* scans bitmap by regions of size order */ + bitmap_check_params(bitmap, NULL, NULL, bits, (1 << order), 0, CHECK_OFF); + for (pos = 0 ; (end = pos + (1U << order)) <= bits; pos = end) { if (!__reg_op(bitmap, pos, order, REG_OP_ISFREE)) continue; @@ -1355,6 +1446,8 @@ EXPORT_SYMBOL(bitmap_find_free_region); */ void bitmap_release_region(unsigned long *bitmap, unsigned int pos, int order) { + bitmap_check_params(bitmap, NULL, NULL, pos + (1 << order), pos, pos + (1 << order), + CHECK_START | CHECK_OFF); __reg_op(bitmap, pos, order, REG_OP_RELEASE); } EXPORT_SYMBOL(bitmap_release_region); @@ -1372,6 +1465,8 @@ EXPORT_SYMBOL(bitmap_release_region); */ int bitmap_allocate_region(unsigned long *bitmap, unsigned int pos, int order) { + bitmap_check_params(bitmap, NULL, NULL, pos + (1 << order), pos, pos + (1 << order), + CHECK_START | CHECK_OFF); if (!__reg_op(bitmap, pos, order, REG_OP_ISFREE)) return -EBUSY; return __reg_op(bitmap, pos, order, REG_OP_ALLOC); @@ -1391,6 +1486,8 @@ void bitmap_copy_le(unsigned long *dst, const unsigned long *src, unsigned int n { unsigned int i; + bitmap_check_move(dst, src, nbits); + for (i = 0; i < nbits/BITS_PER_LONG; i++) { if (BITS_PER_LONG == 64) dst[i] = cpu_to_le64(src[i]); @@ -1476,6 +1573,8 @@ void bitmap_from_arr32(unsigned long *bitmap, const u32 *buf, unsigned int nbits { unsigned int i, halfwords; + bitmap_check_move(bitmap, (unsigned long *)buf, nbits); + halfwords = DIV_ROUND_UP(nbits, 32); for (i = 0; i < halfwords; i++) { bitmap[i/2] = (unsigned long) buf[i]; @@ -1499,6 +1598,8 @@ void bitmap_to_arr32(u32 *buf, const unsigned long *bitmap, unsigned int nbits) { unsigned int i, halfwords; + bitmap_check_move(bitmap, (unsigned long *)buf, nbits); + halfwords = DIV_ROUND_UP(nbits, 32); for (i = 0; i < halfwords; i++) { buf[i] = (u32) (bitmap[i/2] & UINT_MAX); @@ -1524,6 +1625,8 @@ void bitmap_from_arr64(unsigned long *bitmap, const u64 *buf, unsigned int nbits { int n; + bitmap_check_move(bitmap, (unsigned long *)buf, nbits); + for (n = nbits; n > 0; n -= 64) { u64 val = *buf++; @@ -1554,6 +1657,8 @@ void bitmap_to_arr64(u64 *buf, const unsigned long *bitmap, unsigned int nbits) { const unsigned long *end = bitmap + BITS_TO_LONGS(nbits); + bitmap_check_move(bitmap, (unsigned long *)buf, nbits); + while (bitmap < end) { *buf = *bitmap++; if (bitmap < end) From patchwork Mon Jul 18 19:28:30 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yury Norov X-Patchwork-Id: 12921641 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6FF34CCA479 for ; Mon, 18 Jul 2022 19:28:57 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235962AbiGRT2z (ORCPT ); Mon, 18 Jul 2022 15:28:55 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45642 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235932AbiGRT2v (ORCPT ); Mon, 18 Jul 2022 15:28:51 -0400 Received: from mail-qk1-x72a.google.com (mail-qk1-x72a.google.com [IPv6:2607:f8b0:4864:20::72a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 42F4125D3; Mon, 18 Jul 2022 12:28:50 -0700 (PDT) Received: by mail-qk1-x72a.google.com with SMTP id e16so5626142qka.5; Mon, 18 Jul 2022 12:28:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=mds6TiJm2i/ciGRewHyQeRgrE3/eBxBJskgGjRCv8jE=; b=K+OO8GlSGfFoQ4u6Xgs/Jnqm5tx2JRuUl5xIk00RjcP+K/hkgRPZinOZ2EttuW5q5q hvFsBf0zmTfoBNpetZrWb+DCskXNBQW9Wr6HHqY575HwOkPBTVw4HS1BOky1KOrpEXOd tmKdebqLU2VTtvGMLQNsyBADet1rY6Vweu3mXy+o5kaVdj+jOz5T0mr49UwAGgGr8XLe 3JUtC9totttIeVL+NIKb0V3FiTtHU0zLxxZcy7PTaZsjaTiDd2H+U0uKvVz7FW8SLQbu DM4pMj5FgVTxaD/374QNS1ulsCavtIMeCQ9ntw6jXlM51lUnIbkhUR+mDJzqNU9A/tqz 2SoQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=mds6TiJm2i/ciGRewHyQeRgrE3/eBxBJskgGjRCv8jE=; b=c7ZWtbA9TDGQ75fVfWDU8r7rrJy+9I9pDimY0wD9nVzps8ZXPM4g0/QUkxjfX3/z97 Ax0aO8Y/9WNe4+dFxTSD9BedoTjjoSWYQ5cYjlVZHrv4hWdiD31/IB3MzeS+Wjr3jaH7 fXM/UuMsoQN0NOE8UREanC4JmFqfQJJ+NS72S09iQgHI3DnCsJyg9LtBBnEUI1rjdjoQ uy75CqD4ZzppFSNDxoPXalnPvQQpP5qujqz8pWEAZVY6ZK4zS3JRpKT1k5xZ2NEeJlxK OfzIMkhefswYYeQngx6K319npTu8k/egvJCufw97rh+Ciggqz1iv0ehKq1sBA3/yRdH1 AI0w== X-Gm-Message-State: AJIora+I+qWcbWtvXUZDyqCea7wJRh2qKo2zUAh1ZwdS+wfjaK7c/S6O Xpdw3xjJpFTCx5LbpifQWGRBW9UgsvUJDw== X-Google-Smtp-Source: AGRyM1sjmRb0z/jF/I9QRHkB+Qs97iyonIxrkCMHXwc46LvQcGE4P1mvDuBHw7JATaMF3VYkVo9uoA== X-Received: by 2002:a37:8245:0:b0:6b5:9078:267 with SMTP id e66-20020a378245000000b006b590780267mr17879707qkd.684.1658172529164; Mon, 18 Jul 2022 12:28:49 -0700 (PDT) Received: from localhost ([2601:4c1:c100:1230:ab01:d009:465a:5ab1]) by smtp.gmail.com with ESMTPSA id cc12-20020a05622a410c00b00316a384447fsm7028960qtb.16.2022.07.18.12.28.48 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 18 Jul 2022 12:28:48 -0700 (PDT) From: Yury Norov To: linux-kernel@vger.kernel.org, Alexander Lobakin , Alexei Starovoitov , Alexey Klimov , Andrew Morton , Andrii Nakryiko , Andy Shevchenko , Ben Segall , Christoph Lameter , Dan Williams , Daniel Borkmann , Daniel Bristot de Oliveira , Dennis Zhou , Dietmar Eggemann , Eric Dumazet , Frederic Weisbecker , Guenter Roeck , Ingo Molnar , Isabella Basso , John Fastabend , Josh Poimboeuf , Juergen Gross , Juri Lelli , KP Singh , Kees Cook , Martin KaFai Lau , Mel Gorman , Miroslav Benes , Nathan Chancellor , "Paul E . McKenney" , Peter Zijlstra , Randy Dunlap , Rasmus Villemoes , Sebastian Andrzej Siewior , Song Liu , Steven Rostedt , Tejun Heo , Thomas Gleixner , Valentin Schneider , Vincent Guittot , Vlastimil Babka , Yonghong Song , Yury Norov , linux-mm@kvack.org, netdev@vger.kernel.org, bpf@vger.kernel.org Subject: [PATCH 02/16] lib/bitmap: don't call bitmap_set() with len == 0 Date: Mon, 18 Jul 2022 12:28:30 -0700 Message-Id: <20220718192844.1805158-3-yury.norov@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220718192844.1805158-1-yury.norov@gmail.com> References: <20220718192844.1805158-1-yury.norov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org bitmap_parselist() format allows passing 0-length regions, but because len == 0 is not covered by small_const_nbits() macro, we have to call __bitmap_set() and do some prologue calculations just for nothing. Signed-off-by: Yury Norov --- lib/bitmap.c | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-) diff --git a/lib/bitmap.c b/lib/bitmap.c index cd4dd848ea6a..790df2ea02df 100644 --- a/lib/bitmap.c +++ b/lib/bitmap.c @@ -712,10 +712,13 @@ struct region { static void bitmap_set_region(const struct region *r, unsigned long *bitmap) { - unsigned int start; + unsigned int start, len; - for (start = r->start; start <= r->end; start += r->group_len) - bitmap_set(bitmap, start, min(r->end - start + 1, r->off)); + for (start = r->start; start <= r->end; start += r->group_len) { + len = min(r->end - start + 1, r->off); + if (len > 0) + bitmap_set(bitmap, start, len); + } } static int bitmap_check_region(const struct region *r) From patchwork Mon Jul 18 19:28:31 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yury Norov X-Patchwork-Id: 12921646 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9DB95CCA489 for ; Mon, 18 Jul 2022 19:29:14 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235991AbiGRT3D (ORCPT ); Mon, 18 Jul 2022 15:29:03 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45666 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235947AbiGRT2w (ORCPT ); Mon, 18 Jul 2022 15:28:52 -0400 Received: from mail-qt1-x833.google.com (mail-qt1-x833.google.com [IPv6:2607:f8b0:4864:20::833]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5081D2BFE; Mon, 18 Jul 2022 12:28:51 -0700 (PDT) Received: by mail-qt1-x833.google.com with SMTP id bz13so3157863qtb.7; Mon, 18 Jul 2022 12:28:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=h/b9Gor8OmaVbqlus0N8Yerswqy364XPSWWiuokjuPU=; b=fqFJufaEGeNpfSwDeHLXACT8OnWWXz+SS1j38SVvv2XI3V7V8WDZtxIfdvIaKQ8y9v 0B0WY2lTYT/i7+LGAA/Ip2Q7u/fJdoSJt/07tIAob70ImqR7TtZlTUPoT1LuMgFcuWY4 7HFka6L2jlYH/h5SMQBpAZF9w4mMJC5VPkP+4naKoLuGj5PEwF6S3Vgsnqe9/5kDZX5n E1qaAdoAy7EOEd3ji4eGl8J7TIvZC5UE2R5Le3yIwidlKTaWO/goPl8ml+dCO0JT7Ivy 7rMLL+xDqIc/xTUV4ajY3ZNx4vpyBfXc91TeNvsl4Lz9YhlMBkra7r0yzMzayL5h8PUk nvoQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=h/b9Gor8OmaVbqlus0N8Yerswqy364XPSWWiuokjuPU=; b=H1IGxrZocn1L1esbYq2CGQZz58XNJecCBmV2ErVkRaobHnabTvabDSKGDxpXRAyOZ0 +VcCtN9JmReKwliaFj5JUdqpfoaB9Y1IIr2fQu+d8xw0OovdYwATLJSFJlwkN6RJnt0n oV7WBh3YpQ30WtSP/bSfY8su9IpHLtwUtS/OAOkAhzjwaZKte4onZG1Ui27ko1B/j4pP OMAy2yM6PxlrEwmpoghmDqXZdvGszUpEGoXQ5lHz5fi8uclZFHTId2ZOLq42efOQfKXr lOCjNjqnOOSkiKfXyhcVS0iULhEarcedj95aTAPY+QoFN/dQ5g8ilhbhbSBRVmBncyse KmnQ== X-Gm-Message-State: AJIora94bnyIKu0vOi4OaLrgFIxMwJ47ZcSk11ZFA6jV+HMA1TDajSSF yiRqIltMQFVCIEpNpnO1Qdx7cJH7psfVOA== X-Google-Smtp-Source: AGRyM1uaGs+CAqWH8avpKeUoD3vmIQkUBNKvCB+RRyAV1xF9WVdeoxP71d/HveDQuxFH6mXJfEXYcA== X-Received: by 2002:a05:622a:198b:b0:31e:ec25:8ead with SMTP id u11-20020a05622a198b00b0031eec258eadmr5759733qtc.423.1658172530262; Mon, 18 Jul 2022 12:28:50 -0700 (PDT) Received: from localhost ([2601:4c1:c100:1230:ab01:d009:465a:5ab1]) by smtp.gmail.com with ESMTPSA id x27-20020a05620a0b5b00b006b5e43466ebsm4634117qkg.59.2022.07.18.12.28.49 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 18 Jul 2022 12:28:49 -0700 (PDT) From: Yury Norov To: linux-kernel@vger.kernel.org, Alexander Lobakin , Alexei Starovoitov , Alexey Klimov , Andrew Morton , Andrii Nakryiko , Andy Shevchenko , Ben Segall , Christoph Lameter , Dan Williams , Daniel Borkmann , Daniel Bristot de Oliveira , Dennis Zhou , Dietmar Eggemann , Eric Dumazet , Frederic Weisbecker , Guenter Roeck , Ingo Molnar , Isabella Basso , John Fastabend , Josh Poimboeuf , Juergen Gross , Juri Lelli , KP Singh , Kees Cook , Martin KaFai Lau , Mel Gorman , Miroslav Benes , Nathan Chancellor , "Paul E . McKenney" , Peter Zijlstra , Randy Dunlap , Rasmus Villemoes , Sebastian Andrzej Siewior , Song Liu , Steven Rostedt , Tejun Heo , Thomas Gleixner , Valentin Schneider , Vincent Guittot , Vlastimil Babka , Yonghong Song , Yury Norov , linux-mm@kvack.org, netdev@vger.kernel.org, bpf@vger.kernel.org Subject: [PATCH 03/16] lib/test_bitmap: don't test bitmap_set if nbits == 0 Date: Mon, 18 Jul 2022 12:28:31 -0700 Message-Id: <20220718192844.1805158-4-yury.norov@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220718192844.1805158-1-yury.norov@gmail.com> References: <20220718192844.1805158-1-yury.norov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org Don't test bitmap_set(bitmap, start, 0) as it's useless, most probably a sign of error in real code, and makes CONFIG_DEBUG_BITMAP barking. Signed-off-by: Yury Norov --- lib/test_bitmap.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/lib/test_bitmap.c b/lib/test_bitmap.c index 98754ff9fe68..2a70393ac011 100644 --- a/lib/test_bitmap.c +++ b/lib/test_bitmap.c @@ -622,7 +622,7 @@ static void noinline __init test_mem_optimisations(void) unsigned int start, nbits; for (start = 0; start < 1024; start += 8) { - for (nbits = 0; nbits < 1024 - start; nbits += 8) { + for (nbits = 1; nbits < 1024 - start; nbits += 8) { memset(bmap1, 0x5a, sizeof(bmap1)); memset(bmap2, 0x5a, sizeof(bmap2)); From patchwork Mon Jul 18 19:28:32 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yury Norov X-Patchwork-Id: 12921643 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id D9118C433EF for ; Mon, 18 Jul 2022 19:29:13 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235971AbiGRT3A (ORCPT ); Mon, 18 Jul 2022 15:29:00 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45620 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235933AbiGRT2y (ORCPT ); Mon, 18 Jul 2022 15:28:54 -0400 Received: from mail-qk1-x72d.google.com (mail-qk1-x72d.google.com [IPv6:2607:f8b0:4864:20::72d]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0D69ED5A; Mon, 18 Jul 2022 12:28:52 -0700 (PDT) Received: by mail-qk1-x72d.google.com with SMTP id o26so9561767qkl.6; Mon, 18 Jul 2022 12:28:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=WuubJCN67Jl+OjN8nViAm+NVBqJ2ywJqSxExI1yjjeo=; b=Tj6uBzad4HGB6EL0M1yus7P2xl8seTebl3KEY60QiHYHkiZzGlnZJc4zmMQ715lBQb V+6rCjlFtE36SY7lmQBsjFybnyNA/OpYbPg+Hxm3taVetxQSzz05FbXjQEqoSn3tr68N fufksI4iYgEDgfiqAfDBwg7RDixJD5/BbGOR3iYboPjgzPCnwUY3I9q81BKlZDy93R5+ TYLpPb/vjZhMTWj53wGRv+e2qfs4M6ZZ3ek9UJ68NbZRFgDae6SXyqs1nFFZVE0b4GJf TuETrm14lrx8VLkbDZWKHiMSYX1+sOP0ozMNOUt4xFlsRcKHdAtHqy1wiM2QBsaIJik6 D9HA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=WuubJCN67Jl+OjN8nViAm+NVBqJ2ywJqSxExI1yjjeo=; b=nArK0v2r1yode3l+iFyhdtsEdLmfZrSn8XAyJ9sCYe/FMI7ffT7RwcCycf8CWgMvW+ Cccl9/e6vz2d1L5RWzv/0bSgkTf0Usg4WNyV0xwDjQE8Z2xUsBn+n8rS1xhHAzjxC3Ep UykOc9GPSR4nK12nX7hRclPMA1xhjaAk6G7ifkkiDpHbcI5kYR+VctcyPMBk/vVNbCmF RQXhxXTN3YEf6MD40e0+7wkeD7VfkpnTHUheGe00FqwBQmyyjMO1Xaf/UxySBpeish/D 8zhEVvoasi8EYF56cFyl/n5EX3fWEdePoXRlFNOJSk22TCPiKigmOoX9TRNySbTnjh8i hTjw== X-Gm-Message-State: AJIora8CRPc4VGUOPscqLbrYugdFTVnlML7PKI4zrO37yMZ9ptLTb4of bZk3mxBeHERn/Y+QD72Hl2/T6M7dWh+yoQ== X-Google-Smtp-Source: AGRyM1utJxde5Ak8ZdjBXmB+6RTWRe3dLKOIt69li9dLPfU8C1nB6Z6dKZAjbNpuMZO2dr2cItKKhg== X-Received: by 2002:a37:b346:0:b0:6b2:8e4c:690c with SMTP id c67-20020a37b346000000b006b28e4c690cmr18431369qkf.654.1658172531539; Mon, 18 Jul 2022 12:28:51 -0700 (PDT) Received: from localhost ([2601:4c1:c100:1230:ab01:d009:465a:5ab1]) by smtp.gmail.com with ESMTPSA id r2-20020ac87ee2000000b0031ed590433bsm8607338qtc.78.2022.07.18.12.28.50 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 18 Jul 2022 12:28:51 -0700 (PDT) From: Yury Norov To: linux-kernel@vger.kernel.org, Alexander Lobakin , Alexei Starovoitov , Alexey Klimov , Andrew Morton , Andrii Nakryiko , Andy Shevchenko , Ben Segall , Christoph Lameter , Dan Williams , Daniel Borkmann , Daniel Bristot de Oliveira , Dennis Zhou , Dietmar Eggemann , Eric Dumazet , Frederic Weisbecker , Guenter Roeck , Ingo Molnar , Isabella Basso , John Fastabend , Josh Poimboeuf , Juergen Gross , Juri Lelli , KP Singh , Kees Cook , Martin KaFai Lau , Mel Gorman , Miroslav Benes , Nathan Chancellor , "Paul E . McKenney" , Peter Zijlstra , Randy Dunlap , Rasmus Villemoes , Sebastian Andrzej Siewior , Song Liu , Steven Rostedt , Tejun Heo , Thomas Gleixner , Valentin Schneider , Vincent Guittot , Vlastimil Babka , Yonghong Song , Yury Norov , linux-mm@kvack.org, netdev@vger.kernel.org, bpf@vger.kernel.org Subject: [PATCH 04/16] lib/test_bitmap: test test_bitmap_arr{32,64} starting from nbits == 1 Date: Mon, 18 Jul 2022 12:28:32 -0700 Message-Id: <20220718192844.1805158-5-yury.norov@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220718192844.1805158-1-yury.norov@gmail.com> References: <20220718192844.1805158-1-yury.norov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org nbits == 0 is useless. In a real code it's most probably a sign of error, and it makes CONFIG_DEBUG_BITMAP barking. Signed-off-by: Yury Norov --- lib/test_bitmap.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/lib/test_bitmap.c b/lib/test_bitmap.c index 2a70393ac011..bc48d992d10d 100644 --- a/lib/test_bitmap.c +++ b/lib/test_bitmap.c @@ -567,7 +567,7 @@ static void __init test_bitmap_arr32(void) memset(arr, 0xa5, sizeof(arr)); - for (nbits = 0; nbits < EXP1_IN_BITS; ++nbits) { + for (nbits = 1; nbits < EXP1_IN_BITS; ++nbits) { bitmap_to_arr32(arr, exp1, nbits); bitmap_from_arr32(bmap2, arr, nbits); expect_eq_bitmap(bmap2, exp1, nbits); @@ -593,7 +593,7 @@ static void __init test_bitmap_arr64(void) memset(arr, 0xa5, sizeof(arr)); - for (nbits = 0; nbits < EXP1_IN_BITS; ++nbits) { + for (nbits = 1; nbits < EXP1_IN_BITS; ++nbits) { memset(bmap2, 0xff, sizeof(arr)); bitmap_to_arr64(arr, exp1, nbits); bitmap_from_arr64(bmap2, arr, nbits); From patchwork Mon Jul 18 19:28:33 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yury Norov X-Patchwork-Id: 12921642 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7C19BCCA479 for ; Mon, 18 Jul 2022 19:29:00 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234713AbiGRT26 (ORCPT ); Mon, 18 Jul 2022 15:28:58 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45624 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235931AbiGRT2y (ORCPT ); Mon, 18 Jul 2022 15:28:54 -0400 Received: from mail-qk1-x72e.google.com (mail-qk1-x72e.google.com [IPv6:2607:f8b0:4864:20::72e]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id CA983FEB; Mon, 18 Jul 2022 12:28:53 -0700 (PDT) Received: by mail-qk1-x72e.google.com with SMTP id b25so8800394qka.11; Mon, 18 Jul 2022 12:28:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=fSIJ7a7j2AX31WFosI1tLZd0prS7r7q6CFeV8/OppVE=; b=VbkCt5PHj4oSczMa9BarPYV2CjxzDRvrtDRRMj4B07hzccBFOAfmYblYUUH9Ajgumv RKM4K28hzaiG92J/7vkuoCFu1rn9w8fhy0lWf6SIorEHqKyHwppnrzuULW+EmT6h7aZb iNnNyCpV5Al9CLDF5s+BHS+cuFK+7eXWgag89LdLI4R4kgeV2hbArCXUwMlgqtF2xWIm qfkeQzp0/+pXaxkZspiEkYN3yN4c8vfsqHzTwgjkQ3p2CVysWoVDuRXKrA2nXL5HlaIv LdBinorxw3H/YgC7eHfwweeumD20to6B5s4SYw9LVJdPLE0pFWEA8sn1G0OBVc/9tacU f7fg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=fSIJ7a7j2AX31WFosI1tLZd0prS7r7q6CFeV8/OppVE=; b=cD5ZeosHnFTEjr1F4y9szPkWBBod684uIM96g7mjDpMvrk+/K5W2ZQXtUIJfZY2G23 4WgWkXp1L7PTE6ZfJbfvTjaeSw40xIiqE6xPZJi5S2SDaDlkwB3hbsOF+rFzSU/S/+53 VGca8LhJVTk+1lmDwU712ED4EBIb/0jZNcuM6AgONNw5DusUqCJrZEz8jZBYZaoNybgG V8KmHFl31LjeFUEasX+4iOoEhnG8FRZ4XGx9UE0B8KGMLmWIHXBc3ppKapL6uaLTvsOr GTSV30b1nl2kt9d6g1qJiqd6qECLX/0fl9DPdb93nX+Wv+iL6aYtwTao/krYvPpEvPxg T4Rw== X-Gm-Message-State: AJIora+UWv+ZedSAh3kN4lSqQIAlrfbSrzzpWp37/O5tNkyH8x0Edv9K k2NieK7xPmrBbo1nrYTGAX2hRHhyw1SsKw== X-Google-Smtp-Source: AGRyM1vV0SLlM81qvktCdvHe8wlRlm35TgOxVMvgml/1hdumCxKzVqNhVheVD3VDTLe5gZR7GC4mqw== X-Received: by 2002:ae9:e402:0:b0:6a7:86a3:752e with SMTP id q2-20020ae9e402000000b006a786a3752emr18219422qkc.300.1658172532712; Mon, 18 Jul 2022 12:28:52 -0700 (PDT) Received: from localhost ([2601:4c1:c100:1230:ab01:d009:465a:5ab1]) by smtp.gmail.com with ESMTPSA id d19-20020a05620a241300b006a6d20386f6sm13557144qkn.42.2022.07.18.12.28.52 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 18 Jul 2022 12:28:52 -0700 (PDT) From: Yury Norov To: linux-kernel@vger.kernel.org, Alexander Lobakin , Alexei Starovoitov , Alexey Klimov , Andrew Morton , Andrii Nakryiko , Andy Shevchenko , Ben Segall , Christoph Lameter , Dan Williams , Daniel Borkmann , Daniel Bristot de Oliveira , Dennis Zhou , Dietmar Eggemann , Eric Dumazet , Frederic Weisbecker , Guenter Roeck , Ingo Molnar , Isabella Basso , John Fastabend , Josh Poimboeuf , Juergen Gross , Juri Lelli , KP Singh , Kees Cook , Martin KaFai Lau , Mel Gorman , Miroslav Benes , Nathan Chancellor , "Paul E . McKenney" , Peter Zijlstra , Randy Dunlap , Rasmus Villemoes , Sebastian Andrzej Siewior , Song Liu , Steven Rostedt , Tejun Heo , Thomas Gleixner , Valentin Schneider , Vincent Guittot , Vlastimil Babka , Yonghong Song , Yury Norov , linux-mm@kvack.org, netdev@vger.kernel.org, bpf@vger.kernel.org Subject: [PATCH 05/16] lib/test_bitmap: disable compile-time test if DEBUG_BITMAP() is enabled Date: Mon, 18 Jul 2022 12:28:33 -0700 Message-Id: <20220718192844.1805158-6-yury.norov@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220718192844.1805158-1-yury.norov@gmail.com> References: <20220718192844.1805158-1-yury.norov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org CONFIG_DEBUG_BITMAP, when enabled, injects __bitmap_check_params() into bitmap functions. It prevents compiler from compile-time optimizations, which makes corresponding test fail. Signed-off-by: Yury Norov --- lib/test_bitmap.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/lib/test_bitmap.c b/lib/test_bitmap.c index bc48d992d10d..8bd279a7633f 100644 --- a/lib/test_bitmap.c +++ b/lib/test_bitmap.c @@ -877,6 +877,7 @@ static void __init test_bitmap_print_buf(void) static void __init test_bitmap_const_eval(void) { +#ifndef CONFIG_DEBUG_BITMAP DECLARE_BITMAP(bitmap, BITS_PER_LONG); unsigned long initvar = BIT(2); unsigned long bitopvar = 0; @@ -934,6 +935,7 @@ static void __init test_bitmap_const_eval(void) /* ~BIT(25) */ BUILD_BUG_ON(!__builtin_constant_p(~var)); BUILD_BUG_ON(~var != ~BIT(25)); +#endif } static void __init selftest(void) From patchwork Mon Jul 18 19:28:34 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yury Norov X-Patchwork-Id: 12921644 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 55461CCA485 for ; Mon, 18 Jul 2022 19:29:14 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235976AbiGRT3B (ORCPT ); Mon, 18 Jul 2022 15:29:01 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45922 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229647AbiGRT24 (ORCPT ); Mon, 18 Jul 2022 15:28:56 -0400 Received: from mail-qv1-xf2b.google.com (mail-qv1-xf2b.google.com [IPv6:2607:f8b0:4864:20::f2b]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C8449A1AB; Mon, 18 Jul 2022 12:28:54 -0700 (PDT) Received: by mail-qv1-xf2b.google.com with SMTP id cw12so9538577qvb.12; Mon, 18 Jul 2022 12:28:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=kggq/8QR9J7+hQqC85Z7UvjwISsJybPyVWycNM36A94=; b=dhlOzRWSRCVDeB5G/ojYUTukZ+P2gcdtsU+jMLUstlXFlOrTp2ph3aG0NXBV8ztstI OXMYLzx6YxjDhp2P9YRr9zilu9HXR7fY5pvPhtwR3iRWzZwfVtgG9lVHCT2ZTqpw0+Ml +OgpMDSbWao5AC6+qrMwM3Db0YqXaBy6MaEv7wnj6amC+iUAq4cv2bT0A4L8zdy5Euao VAKc32hcR8GaNzT5A9JfavcEovAoouxP/ve/GxdRAbv82Ya4GcKRYbxbQkMPkB47IqRX kYT71Abd+YiP5O3A8ckxB+XNkCs6wKALUnGT5t/3o7qzxFLs7cpnxkdiCIV74UJKfkcN 7sOw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=kggq/8QR9J7+hQqC85Z7UvjwISsJybPyVWycNM36A94=; b=5BW/VUyenp3yQB7XiN74Oc7DdLNRhDtEqS6KBgj8nRDvJ5mk+E8AtO6eSsJs3MzDKq pmT0yinkz2c6d5PqbIxALQuPmTLADje39rMa1AwssWKLf4RqKePJWmKUVtynesXBidj/ 4jh5W9phBVQioPcVKT5BIh7csacLSvxxNMkm+SU7b3N0nLTSoLXInGballCL8Xa/yOwo Nu0X7m1uiNZTCJ53d096p5M9B2dPeKJD00c/9IWwvjxpZlel7M3x/IcAmiBXB9SpKJeF ISmBQ8WnZTmy9vpYZI/XCE06PEwG7j1QlOK1f9e4/FJSazt4i6ZJ7p0geysA1jGKyTBb fBwA== X-Gm-Message-State: AJIora/8Rk2zN6TMWvKdn4/qJj5xtpZAlrS0Kk9zy8eQjLscICYysakW EBNKdArLoiGIDMreheeGB3KEzSca+XfB/g== X-Google-Smtp-Source: AGRyM1u3EM0rXTHhAD/Z1aIEbA912gwLNc43tN+zBsepF5isd7Swws3o9Azy4V52O5TNXWBcbJiNlg== X-Received: by 2002:a05:6214:246f:b0:473:5145:d906 with SMTP id im15-20020a056214246f00b004735145d906mr22495118qvb.126.1658172533740; Mon, 18 Jul 2022 12:28:53 -0700 (PDT) Received: from localhost ([2601:4c1:c100:1230:ab01:d009:465a:5ab1]) by smtp.gmail.com with ESMTPSA id i21-20020a05620a405500b006b5f4b7b088sm1783681qko.108.2022.07.18.12.28.53 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 18 Jul 2022 12:28:53 -0700 (PDT) From: Yury Norov To: linux-kernel@vger.kernel.org, Alexander Lobakin , Alexei Starovoitov , Alexey Klimov , Andrew Morton , Andrii Nakryiko , Andy Shevchenko , Ben Segall , Christoph Lameter , Dan Williams , Daniel Borkmann , Daniel Bristot de Oliveira , Dennis Zhou , Dietmar Eggemann , Eric Dumazet , Frederic Weisbecker , Guenter Roeck , Ingo Molnar , Isabella Basso , John Fastabend , Josh Poimboeuf , Juergen Gross , Juri Lelli , KP Singh , Kees Cook , Martin KaFai Lau , Mel Gorman , Miroslav Benes , Nathan Chancellor , "Paul E . McKenney" , Peter Zijlstra , Randy Dunlap , Rasmus Villemoes , Sebastian Andrzej Siewior , Song Liu , Steven Rostedt , Tejun Heo , Thomas Gleixner , Valentin Schneider , Vincent Guittot , Vlastimil Babka , Yonghong Song , Yury Norov , linux-mm@kvack.org, netdev@vger.kernel.org, bpf@vger.kernel.org Subject: [PATCH 06/16] lib/test_bitmap: delete meaningless test for bitmap_cut Date: Mon, 18 Jul 2022 12:28:34 -0700 Message-Id: <20220718192844.1805158-7-yury.norov@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220718192844.1805158-1-yury.norov@gmail.com> References: <20220718192844.1805158-1-yury.norov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org One of bitmap_cut() tests passed it with: nbits = BITS_PER_LONG; first = BITS_PER_LONG; cut = BITS_PER_LONG; This test is useless because the range to cut is not inside the bitmap. This should normally raise an error, but bitmap_cut() is void and returns nothing. To check if the test is passed, it just tests that the memory is not touched by bitmap_cut(), which is probably not correct, because if a function is passed with wrong parameters, it's too optimistic to expect a correct, or even sane behavior. Now that we have bitmap_check_params(), there's a tool to detect such things in real code, and we can drop the test. Signed-off-by: Yury Norov --- lib/test_bitmap.c | 4 ---- 1 file changed, 4 deletions(-) diff --git a/lib/test_bitmap.c b/lib/test_bitmap.c index 8bd279a7633f..c1ea449aae2d 100644 --- a/lib/test_bitmap.c +++ b/lib/test_bitmap.c @@ -707,10 +707,6 @@ static struct test_bitmap_cut test_cut[] = { { 15, 16, 32, { 0xa5a5a5a5UL, }, { 0x0000a5a5UL, }, }, { 16, 15, 32, { 0xa5a5a5a5UL, }, { 0x0001a5a5UL, }, }, - { BITS_PER_LONG, BITS_PER_LONG, BITS_PER_LONG, - { 0xa5a5a5a5UL, 0xa5a5a5a5UL, }, - { 0xa5a5a5a5UL, 0xa5a5a5a5UL, }, - }, { 1, BITS_PER_LONG - 1, BITS_PER_LONG, { 0xa5a5a5a5UL, 0xa5a5a5a5UL, }, { 0x00000001UL, 0x00000001UL, }, From patchwork Mon Jul 18 19:28:35 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yury Norov X-Patchwork-Id: 12921648 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5566EC433EF for ; Mon, 18 Jul 2022 19:29:16 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234098AbiGRT3O (ORCPT ); Mon, 18 Jul 2022 15:29:14 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46342 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235860AbiGRT25 (ORCPT ); Mon, 18 Jul 2022 15:28:57 -0400 Received: from mail-qk1-x72e.google.com (mail-qk1-x72e.google.com [IPv6:2607:f8b0:4864:20::72e]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0B91B26F5; Mon, 18 Jul 2022 12:28:55 -0700 (PDT) Received: by mail-qk1-x72e.google.com with SMTP id o1so9549566qkg.9; Mon, 18 Jul 2022 12:28:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=m9V57xphOJe8FxCe80bZ1J3RuFelVtvXoENVWMfISb4=; b=B0XuB+yAeTJmK6B+vnHnd3BO8yIPsZ8lCES1PpoVp7Ti9ESY8L/5LseEzml+iCgDqI 1YlvnrUUgNx4DD7RQwT4vms4g+VmvWZpGoAnJNKJ0UYr9HyWBXYZufeEWv7Vuwxy/4nI qIrQ27tmeRvKsD0HQGA4yMLWkqHFveDpBWb4zZNi9MJuv/+VtQRCqaE5PFi0tfV7R98a 9bkurHLXilYTf8+/4Z6M5WqsGBng481jqtXDo5QizTdcTOcHRejuVHRFGd14+vBFF67B NUB7Epima9nbyqEkq0yYEeKIgt/HN7ct0hLat+eWTlW+qrZnqyzPXV06pMIjXg2GHmsY vUnQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=m9V57xphOJe8FxCe80bZ1J3RuFelVtvXoENVWMfISb4=; b=x2W7Yce8qD7GAnd11x8NoHIrMk5s0ETIBCgK/MKrwAaTqwnjMN5gvLv3sWOQ7zwmfs V3zPrRgb2ayJ95ctGN2m2w2cQKCKu7toIN2lGU0+SA7AqBL9aSVNPSEcK9xsBq3bA/t1 exa2Sw22nksud25LoouCuhj++9DC5lxutwbM/WgLn/F1mecuhWcFZ2fiHWJuZiAMm5mH OPl9FzyPClpwbCq2KxhAiSdlltG7tAQFXp1FIIG9CUewmqeoCWRg5lsVr73AIei1hBKr iWP3VOPgS8HjVCEN5LjHChIJcSh9JxYcykPhsWmHybwtyoXgknkhx4KmU/V4OQS8Bb5C aWJg== X-Gm-Message-State: AJIora8MDHGoaHeRScbu8fbfB9W1NrBoVPI4Bcwv+3uL97NgiDZjOuIO 0Eh20ChSsEW/tes+grOp2kpfqF82KyFafg== X-Google-Smtp-Source: AGRyM1vQdCay2nDpE1RIgzYmRYjdYqMGFdzO6hLEN38y55+zWOu2X7pBEMBNCfY35RCjup6wtifAoA== X-Received: by 2002:a05:620a:468e:b0:6b5:af5b:6e5f with SMTP id bq14-20020a05620a468e00b006b5af5b6e5fmr18034173qkb.288.1658172534753; Mon, 18 Jul 2022 12:28:54 -0700 (PDT) Received: from localhost ([2601:4c1:c100:1230:ab01:d009:465a:5ab1]) by smtp.gmail.com with ESMTPSA id x6-20020a05620a258600b006a65c58db99sm12736880qko.64.2022.07.18.12.28.54 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 18 Jul 2022 12:28:54 -0700 (PDT) From: Yury Norov To: linux-kernel@vger.kernel.org, Alexander Lobakin , Alexei Starovoitov , Alexey Klimov , Andrew Morton , Andrii Nakryiko , Andy Shevchenko , Ben Segall , Christoph Lameter , Dan Williams , Daniel Borkmann , Daniel Bristot de Oliveira , Dennis Zhou , Dietmar Eggemann , Eric Dumazet , Frederic Weisbecker , Guenter Roeck , Ingo Molnar , Isabella Basso , John Fastabend , Josh Poimboeuf , Juergen Gross , Juri Lelli , KP Singh , Kees Cook , Martin KaFai Lau , Mel Gorman , Miroslav Benes , Nathan Chancellor , "Paul E . McKenney" , Peter Zijlstra , Randy Dunlap , Rasmus Villemoes , Sebastian Andrzej Siewior , Song Liu , Steven Rostedt , Tejun Heo , Thomas Gleixner , Valentin Schneider , Vincent Guittot , Vlastimil Babka , Yonghong Song , Yury Norov , linux-mm@kvack.org, netdev@vger.kernel.org, bpf@vger.kernel.org Subject: [PATCH 07/16] smp: optimize smp_call_function_many_cond() Date: Mon, 18 Jul 2022 12:28:35 -0700 Message-Id: <20220718192844.1805158-8-yury.norov@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220718192844.1805158-1-yury.norov@gmail.com> References: <20220718192844.1805158-1-yury.norov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org smp_call_function_many_cond() is often passed with cpu_online_mask. If this is the case, we can use num_online_cpus(), which is O(1) instead of cpumask_{first,next}(), which is O(N). It can be optimized further: if cpu_online_mask has 0 or single bit set (depending on cpu_online(this_cpu), we can return result without AND'ing with user's mask. Caught with CONFIG_DEBUG_BITMAP: [ 7.830337] Call trace: [ 7.830397] __bitmap_check_params+0x1d8/0x260 [ 7.830499] smp_call_function_many_cond+0x1e8/0x45c [ 7.830607] kick_all_cpus_sync+0x44/0x80 [ 7.830698] bpf_int_jit_compile+0x34c/0x5cc [ 7.830796] bpf_prog_select_runtime+0x118/0x190 [ 7.830900] bpf_prepare_filter+0x3dc/0x51c [ 7.830995] __get_filter+0xd4/0x170 [ 7.831145] sk_attach_filter+0x18/0xb0 [ 7.831236] sock_setsockopt+0x5b0/0x1214 [ 7.831330] __sys_setsockopt+0x144/0x170 [ 7.831431] __arm64_sys_setsockopt+0x2c/0x40 [ 7.831541] invoke_syscall+0x48/0x114 [ 7.831634] el0_svc_common.constprop.0+0x44/0xfc [ 7.831745] do_el0_svc+0x30/0xc0 [ 7.831825] el0_svc+0x2c/0x84 [ 7.831899] el0t_64_sync_handler+0xbc/0x140 [ 7.831999] el0t_64_sync+0x18c/0x190 [ 7.832086] ---[ end trace 0000000000000000 ]--- [ 7.832375] b1: ffff24d1ffd98a48 [ 7.832385] b2: ffffa65533a29a38 [ 7.832393] b3: ffffa65533a29a38 [ 7.832400] nbits: 256 [ 7.832407] start: 0 [ 7.832412] off: 0 [ 7.832418] smp: Bitmap: parameters check failed [ 7.832432] smp: include/linux/bitmap.h [363]: bitmap_and Signed-off-by: Yury Norov --- kernel/smp.c | 29 +++++++++++++++++++++++------ 1 file changed, 23 insertions(+), 6 deletions(-) diff --git a/kernel/smp.c b/kernel/smp.c index dd215f439426..7ed2b9b12f74 100644 --- a/kernel/smp.c +++ b/kernel/smp.c @@ -880,6 +880,28 @@ EXPORT_SYMBOL_GPL(smp_call_function_any); #define SCF_WAIT (1U << 0) #define SCF_RUN_LOCAL (1U << 1) +/* Check if we need remote execution, i.e., any CPU excluding this one. */ +static inline bool __need_remote_exec(const struct cpumask *mask, unsigned int this_cpu) +{ + unsigned int cpu; + + switch (num_online_cpus()) { + case 0: + return false; + case 1: + return cpu_online(this_cpu) ? false : true; + default: + if (mask == cpu_online_mask) + return true; + } + + cpu = cpumask_first_and(mask, cpu_online_mask); + if (cpu == this_cpu) + cpu = cpumask_next_and(cpu, mask, cpu_online_mask); + + return cpu < nr_cpu_ids; +} + static void smp_call_function_many_cond(const struct cpumask *mask, smp_call_func_t func, void *info, unsigned int scf_flags, @@ -916,12 +938,7 @@ static void smp_call_function_many_cond(const struct cpumask *mask, if ((scf_flags & SCF_RUN_LOCAL) && cpumask_test_cpu(this_cpu, mask)) run_local = true; - /* Check if we need remote execution, i.e., any CPU excluding this one. */ - cpu = cpumask_first_and(mask, cpu_online_mask); - if (cpu == this_cpu) - cpu = cpumask_next_and(cpu, mask, cpu_online_mask); - if (cpu < nr_cpu_ids) - run_remote = true; + run_remote = __need_remote_exec(mask, this_cpu); if (run_remote) { cfd = this_cpu_ptr(&cfd_data); From patchwork Mon Jul 18 19:28:36 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yury Norov X-Patchwork-Id: 12921649 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 47CB3C43334 for ; Mon, 18 Jul 2022 19:29:28 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236021AbiGRT3R (ORCPT ); Mon, 18 Jul 2022 15:29:17 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46658 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S236001AbiGRT3D (ORCPT ); Mon, 18 Jul 2022 15:29:03 -0400 Received: from mail-qt1-x82f.google.com (mail-qt1-x82f.google.com [IPv6:2607:f8b0:4864:20::82f]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 73DB4B841; Mon, 18 Jul 2022 12:28:57 -0700 (PDT) Received: by mail-qt1-x82f.google.com with SMTP id r2so9570577qta.0; Mon, 18 Jul 2022 12:28:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=RIuxtr8d+TB6goTZIrQnEqj0NYsoVNuGdOrD+16hKvE=; b=K6FLy4kdLZiOyB3bLUhszl5Zd1puGwKh81hdQ6rMAxhulKYqedp7eUmoIZR0afJcpb nkiXT9uXP0O9WHgoCSQ2IR/sDvIeb+YkvalAFfr8I5dYLsp+gO1RFi5HdpTv9xwEjdQ2 auToBt5qQGIID9AHzaDc+sYfyIwWA0/KGjjZxYfmJ39gMRAk2oUkonyTSXkmCxoD4Fux yUPBSSyh4zYo4dxNsy901oFcH5LCqM2CA0xbWm/ZpSmbLWp+HfPmCnB1q1t1r0vv+BNU aNYNiAkMVnc3um9vm7dDymoR9WWvElT0wUSR1fdH1muLCWOWVQ3EyjMKXAO1Br2kBR00 Vx+A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=RIuxtr8d+TB6goTZIrQnEqj0NYsoVNuGdOrD+16hKvE=; b=gFWudZYVFqtuqMOmd4NJn95F2VqDGSm/mEU00IG7Tpmca2mgfOQP+1zvT1teLn7IkI /8Nk4GljTzgGPvvNArlWptwhruF9LwJxgzYRRexoR7v1jmjRZ0ogd3qx9u0IuFuCjSQi XB3+qwwaa2tF+VoDPzKOmezyC6deOvOhNOaVS/8O+h3ewkdwwkna1JUe9dR1F03edH9/ rk0C3IkeybFG1vGjO4uz/3zG8xG3yNrGitbjA82ew92Q0HzIn84ZOxkxcop892TpceIE frsOpSoY7zGma+ELq0xRcaFO18efr55C3aXgDNjec5WTYBn0LTLXpGFpYcZBmdBhgRGD Bvlw== X-Gm-Message-State: AJIora/DWbkpU1xWxTqBtLbwy4G0aCM34cNnZehS6wTrqhlovXJ7WAaj jFF6xLXN3O/AbF14imNAV6HtbGd4whNqaQ== X-Google-Smtp-Source: AGRyM1t1sw1jCN1ieW37kpkYQIAyMtVdPUyvBuYOD3/FMPrCUNrO4liqD1XLniGG0iHGpMVX+tZbUg== X-Received: by 2002:ac8:7e89:0:b0:31e:e0dd:9084 with SMTP id w9-20020ac87e89000000b0031ee0dd9084mr10850238qtj.37.1658172535998; Mon, 18 Jul 2022 12:28:55 -0700 (PDT) Received: from localhost ([2601:4c1:c100:1230:ab01:d009:465a:5ab1]) by smtp.gmail.com with ESMTPSA id bi14-20020a05620a318e00b006b5f85f8fa8sm1114706qkb.125.2022.07.18.12.28.55 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 18 Jul 2022 12:28:55 -0700 (PDT) From: Yury Norov To: linux-kernel@vger.kernel.org, Alexander Lobakin , Alexei Starovoitov , Alexey Klimov , Andrew Morton , Andrii Nakryiko , Andy Shevchenko , Ben Segall , Christoph Lameter , Dan Williams , Daniel Borkmann , Daniel Bristot de Oliveira , Dennis Zhou , Dietmar Eggemann , Eric Dumazet , Frederic Weisbecker , Guenter Roeck , Ingo Molnar , Isabella Basso , John Fastabend , Josh Poimboeuf , Juergen Gross , Juri Lelli , KP Singh , Kees Cook , Martin KaFai Lau , Mel Gorman , Miroslav Benes , Nathan Chancellor , "Paul E . McKenney" , Peter Zijlstra , Randy Dunlap , Rasmus Villemoes , Sebastian Andrzej Siewior , Song Liu , Steven Rostedt , Tejun Heo , Thomas Gleixner , Valentin Schneider , Vincent Guittot , Vlastimil Babka , Yonghong Song , Yury Norov , linux-mm@kvack.org, netdev@vger.kernel.org, bpf@vger.kernel.org Subject: [PATCH 08/16] smp: optimize smp_call_function_many_cond() for more Date: Mon, 18 Jul 2022 12:28:36 -0700 Message-Id: <20220718192844.1805158-9-yury.norov@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220718192844.1805158-1-yury.norov@gmail.com> References: <20220718192844.1805158-1-yury.norov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org smp_call_function_many_cond() is often passed with cpu_online_mask. If it's the case, we can use cpumask_copy instead of cpumask_and, which is faster. Caught with CONFIG_DEBUG_BITMAP: [ 7.830337] Call trace: [ 7.830397] __bitmap_check_params+0x1d8/0x260 [ 7.830499] smp_call_function_many_cond+0x1e8/0x45c [ 7.830607] kick_all_cpus_sync+0x44/0x80 [ 7.830698] bpf_int_jit_compile+0x34c/0x5cc [ 7.830796] bpf_prog_select_runtime+0x118/0x190 [ 7.830900] bpf_prepare_filter+0x3dc/0x51c [ 7.830995] __get_filter+0xd4/0x170 [ 7.831145] sk_attach_filter+0x18/0xb0 [ 7.831236] sock_setsockopt+0x5b0/0x1214 [ 7.831330] __sys_setsockopt+0x144/0x170 [ 7.831431] __arm64_sys_setsockopt+0x2c/0x40 [ 7.831541] invoke_syscall+0x48/0x114 [ 7.831634] el0_svc_common.constprop.0+0x44/0xfc [ 7.831745] do_el0_svc+0x30/0xc0 [ 7.831825] el0_svc+0x2c/0x84 [ 7.831899] el0t_64_sync_handler+0xbc/0x140 [ 7.831999] el0t_64_sync+0x18c/0x190 [ 7.832086] ---[ end trace 0000000000000000 ]--- [ 7.832375] b1: ffff24d1ffd98a48 [ 7.832385] b2: ffffa65533a29a38 [ 7.832393] b3: ffffa65533a29a38 [ 7.832400] nbits: 256 [ 7.832407] start: 0 [ 7.832412] off: 0 [ 7.832418] smp: Bitmap: parameters check failed [ 7.832432] smp: include/linux/bitmap.h [363]: bitmap_and Signed-off-by: Yury Norov --- kernel/smp.c | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/kernel/smp.c b/kernel/smp.c index 7ed2b9b12f74..f96fdf944b4a 100644 --- a/kernel/smp.c +++ b/kernel/smp.c @@ -942,7 +942,11 @@ static void smp_call_function_many_cond(const struct cpumask *mask, if (run_remote) { cfd = this_cpu_ptr(&cfd_data); - cpumask_and(cfd->cpumask, mask, cpu_online_mask); + if (mask == cpu_online_mask) + cpumask_copy(cfd->cpumask, cpu_online_mask); + else + cpumask_and(cfd->cpumask, mask, cpu_online_mask); + __cpumask_clear_cpu(this_cpu, cfd->cpumask); cpumask_clear(cfd->cpumask_ipi); From patchwork Mon Jul 18 19:28:37 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yury Norov X-Patchwork-Id: 12921656 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6850CCCA479 for ; Mon, 18 Jul 2022 19:31:36 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235413AbiGRTbF (ORCPT ); Mon, 18 Jul 2022 15:31:05 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46464 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S236070AbiGRT3M (ORCPT ); Mon, 18 Jul 2022 15:29:12 -0400 Received: from mail-qk1-x72e.google.com (mail-qk1-x72e.google.com [IPv6:2607:f8b0:4864:20::72e]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5C6D9DF16; Mon, 18 Jul 2022 12:28:58 -0700 (PDT) Received: by mail-qk1-x72e.google.com with SMTP id c24so4029697qkm.4; Mon, 18 Jul 2022 12:28:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=RIK4clDwX1oEMjN7pszaNKOSN2c//m8ehQ85KvlfQJc=; b=lNlO9SN3GGquIrZKMvt9zNOWmROiQ7UKGluxoJ/6490SfH4l3ZFvvrQ5XQ1qO64f2W p+I82uCQ8ZvuMZH+fAnQobCtCoq0Pxx28ZFgroRR13TygKFuozWxlJ/fBsXMsFi7JF8j LslOfWo/Dy/F6Ir4B6EskXMCoRMKBE9TwZvII3lNY3HkD0b5xpWqgQo6jfTqyMEE9GjF krBbndqao3O5qYyeZ+ps1kb1QUfD4EBcW10RtUjMZnVw3PPOlBGTUXkNXWRiSQp+594Y nQRk9aoZ3fTiFdlQFLY2WCbpyvmZnPd+Gmd7tWwjC5cucyR0Eq7BkmeA3NQwpHGdiKoM 400g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=RIK4clDwX1oEMjN7pszaNKOSN2c//m8ehQ85KvlfQJc=; b=579E0dGF+dgD8naDl20afHk9yX8ej94OmBI4vpL8x7NCxfeuZik/2uZkKQTuciIjU5 42BhHANdJh12Tglpee6m2egVaYmhUcePEzwR7s+fIaZTMjlwAsOpy3yrsswe5z79lPE+ yigYQvsgdKFwEGV79gNcATPAvc6qK7TOHwhWSa3dUFCHbS5RBqrUhosVI1qrsBNXrmu7 HMtzpNmT1vJoy6ehgJMTT6xhx9vhY96wPXSITJhFSxBnN3lqnBa7wNbkHFwpWad2Km5l YQRC6Hh46qv3EtT1Nn+ioDJL0sD4uNJ90YnaybSzHtIbBNtEKE9n+ZCodo/Guu3EUEJr uAMg== X-Gm-Message-State: AJIora9dNT99rOS5ito2Msf370F5brNuiGhC+p3cqiBUdBxX0xYZ2x5Y +ESh4ypzvlOpqVi8vV+RKSMaszXpBa82nw== X-Google-Smtp-Source: AGRyM1tudjg+bwULkYNm5Rqfm5wd3D6Bva3YUzb4kaoYEWEhL+2Z1rueeD9ZkF0QCclQf/uBzBI+zw== X-Received: by 2002:a05:620a:4725:b0:6b5:f6a9:950e with SMTP id bs37-20020a05620a472500b006b5f6a9950emr2013188qkb.464.1658172536990; Mon, 18 Jul 2022 12:28:56 -0700 (PDT) Received: from localhost ([2601:4c1:c100:1230:ab01:d009:465a:5ab1]) by smtp.gmail.com with ESMTPSA id dm53-20020a05620a1d7500b006b4880b08a9sm12668183qkb.88.2022.07.18.12.28.56 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 18 Jul 2022 12:28:56 -0700 (PDT) From: Yury Norov To: linux-kernel@vger.kernel.org, Alexander Lobakin , Alexei Starovoitov , Alexey Klimov , Andrew Morton , Andrii Nakryiko , Andy Shevchenko , Ben Segall , Christoph Lameter , Dan Williams , Daniel Borkmann , Daniel Bristot de Oliveira , Dennis Zhou , Dietmar Eggemann , Eric Dumazet , Frederic Weisbecker , Guenter Roeck , Ingo Molnar , Isabella Basso , John Fastabend , Josh Poimboeuf , Juergen Gross , Juri Lelli , KP Singh , Kees Cook , Martin KaFai Lau , Mel Gorman , Miroslav Benes , Nathan Chancellor , "Paul E . McKenney" , Peter Zijlstra , Randy Dunlap , Rasmus Villemoes , Sebastian Andrzej Siewior , Song Liu , Steven Rostedt , Tejun Heo , Thomas Gleixner , Valentin Schneider , Vincent Guittot , Vlastimil Babka , Yonghong Song , Yury Norov , linux-mm@kvack.org, netdev@vger.kernel.org, bpf@vger.kernel.org Subject: [PATCH 09/16] irq: don't copy cpu affinity mask if source is equal to destination Date: Mon, 18 Jul 2022 12:28:37 -0700 Message-Id: <20220718192844.1805158-10-yury.norov@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220718192844.1805158-1-yury.norov@gmail.com> References: <20220718192844.1805158-1-yury.norov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org irq_do_set_affinity() may be called with mask == irq_data_to_desc()->irq_common_data.affinity Copying in that case is useless. Caught with CONFIG_DEBUG_BITMAP: [ 1.089177] __bitmap_check_params+0x144/0x250 [ 1.089238] irq_do_set_affinity+0x120/0x470 [ 1.089298] irq_startup+0x140/0x16c [ 1.089350] __setup_irq+0x668/0x760 [ 1.089402] request_threaded_irq+0xe8/0x1b0 [ 1.089474] vp_find_vqs_msix+0x270/0x410 [ 1.089532] vp_find_vqs+0x48/0x1b4 [ 1.089584] vp_modern_find_vqs+0x1c/0x70 [ 1.089641] init_vq+0x2dc/0x34c [ 1.089690] virtblk_probe+0xdc/0x710 [ 1.089745] virtio_dev_probe+0x19c/0x270 [ 1.089802] really_probe.part.0+0x9c/0x2ac [ 1.089863] __driver_probe_device+0x98/0x144 [ 1.089923] driver_probe_device+0xac/0x140 [ 1.089985] __driver_attach+0xf8/0x1a0 [ 1.090047] bus_for_each_dev+0x70/0xd0 [ 1.090101] driver_attach+0x24/0x30 [ 1.090153] bus_add_driver+0x150/0x200 [ 1.090208] driver_register+0x78/0x130 [ 1.090266] register_virtio_driver+0x28/0x40 [ 1.090329] virtio_blk_init+0x68/0xa4 [ 1.090400] do_one_initcall+0x50/0x1c0 [ 1.090471] kernel_init_freeable+0x208/0x28c [ 1.090538] kernel_init+0x28/0x13c [ 1.090590] ret_from_fork+0x10/0x20 [ 1.090642] ---[ end trace 0000000000000000 ]--- [ 1.090705] b1: ffff2ec742b85e18 [ 1.090710] b2: ffff2ec742b85e18 [ 1.090715] b3: 0 [ 1.090719] nbits: 256 [ 1.090723] start: 0 [ 1.090727] off: 0 Signed-off-by: Yury Norov --- kernel/irq/manage.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/kernel/irq/manage.c b/kernel/irq/manage.c index 8c396319d5ac..f9c1b21584ec 100644 --- a/kernel/irq/manage.c +++ b/kernel/irq/manage.c @@ -284,7 +284,8 @@ int irq_do_set_affinity(struct irq_data *data, const struct cpumask *mask, switch (ret) { case IRQ_SET_MASK_OK: case IRQ_SET_MASK_OK_DONE: - cpumask_copy(desc->irq_common_data.affinity, mask); + if (desc->irq_common_data.affinity != mask) + cpumask_copy(desc->irq_common_data.affinity, mask); fallthrough; case IRQ_SET_MASK_OK_NOCOPY: irq_validate_effective_affinity(data); From patchwork Mon Jul 18 19:28:38 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yury Norov X-Patchwork-Id: 12921647 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 00288CCA485 for ; Mon, 18 Jul 2022 19:29:17 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235147AbiGRT3Q (ORCPT ); Mon, 18 Jul 2022 15:29:16 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46828 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S236066AbiGRT3M (ORCPT ); Mon, 18 Jul 2022 15:29:12 -0400 Received: from mail-qt1-x833.google.com (mail-qt1-x833.google.com [IPv6:2607:f8b0:4864:20::833]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 536D09FFF; Mon, 18 Jul 2022 12:28:59 -0700 (PDT) Received: by mail-qt1-x833.google.com with SMTP id l14so7163873qtv.4; Mon, 18 Jul 2022 12:28:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=3XYQv40S3S5GPzbQg2pMTtfGv4eD74y+tQ/YBLA8hPI=; b=TByPl+Xwiw2xqd6sHhrukw9bBi6aYZ9/z4hJI5n//rUO3S0J3LYVyW846tuoecx0/P +mwSlwBGZWSeexadVQp08LKtTu3yhD9GwqRpcIaZI+JTvwBTqsldLUYP7cMYJCj0qmee qzWvSzxAj6hHk8lfZ/znqRQjr8YYK6LhI35e/XjH607M4tjQzo2HZoBMdEmvG4Au2nUw 7VRp/BbOQlyMaiQHsMqpMueyXN70vJlljWT7heZMwoRcxsr+t7a84MhXDH0oqg1RW59u ELUqUPatBeLpVq9tLx/9dMB7Lb9eBn1unJzFEZ12o7YFpej8nHIclhzYzEdXcPOn0e7U PU/w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=3XYQv40S3S5GPzbQg2pMTtfGv4eD74y+tQ/YBLA8hPI=; b=1yUGvr2KBbqt5mRt3sIUdBoE7U9UOY1OOtJdKm7DB3fRXRnzF4GUHPeWf5rF/wis59 RgX746WJf7x+5ec+hvNPHEr7fxGIzR5wwk74H962dXLqvvYDIb5u/vIfdzUNVwAMLwBI knubeXMJkCJAtNPNkAizd+iYqLNKS8E5hbIkfgRCRgpFwd8VK/3kwIhPmSCodzre0HC5 Plj3JGItNcTFI3mjnXkoWwwsqPwTVeUfBgawFXy1MF5l+bCvFdgtijLS+YR8dtr7linD tyVPlNcXTcM1Td6WgGq9royTv3TlHF9gVCGKRQSbIfoNX1M23XnETn64waKqiTFBfrv8 r8Vw== X-Gm-Message-State: AJIora/RS7m8GyI6AgiG0CwcUGy31uSqwfbWkDDRX98yJMJ0iRDP52kf rUZszc0DqVYRQh47lqtanuxSIBj8Jwe0CQ== X-Google-Smtp-Source: AGRyM1sbR5MtKZUL9t8F0KGlqcEkHR0jVinkErRs9JvmFXVZhVXEvlih/lkWGgZ5SUq5r6VMJWzeKg== X-Received: by 2002:ac8:5cca:0:b0:31e:f51f:dd97 with SMTP id s10-20020ac85cca000000b0031ef51fdd97mr3070736qta.47.1658172538023; Mon, 18 Jul 2022 12:28:58 -0700 (PDT) Received: from localhost ([2601:4c1:c100:1230:ab01:d009:465a:5ab1]) by smtp.gmail.com with ESMTPSA id m13-20020ac8688d000000b0031bf484079esm8937407qtq.18.2022.07.18.12.28.57 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 18 Jul 2022 12:28:57 -0700 (PDT) From: Yury Norov To: linux-kernel@vger.kernel.org, Alexander Lobakin , Alexei Starovoitov , Alexey Klimov , Andrew Morton , Andrii Nakryiko , Andy Shevchenko , Ben Segall , Christoph Lameter , Dan Williams , Daniel Borkmann , Daniel Bristot de Oliveira , Dennis Zhou , Dietmar Eggemann , Eric Dumazet , Frederic Weisbecker , Guenter Roeck , Ingo Molnar , Isabella Basso , John Fastabend , Josh Poimboeuf , Juergen Gross , Juri Lelli , KP Singh , Kees Cook , Martin KaFai Lau , Mel Gorman , Miroslav Benes , Nathan Chancellor , "Paul E . McKenney" , Peter Zijlstra , Randy Dunlap , Rasmus Villemoes , Sebastian Andrzej Siewior , Song Liu , Steven Rostedt , Tejun Heo , Thomas Gleixner , Valentin Schneider , Vincent Guittot , Vlastimil Babka , Yonghong Song , Yury Norov , linux-mm@kvack.org, netdev@vger.kernel.org, bpf@vger.kernel.org Subject: [PATCH 10/16] sched: optimize __set_cpus_allowed_ptr_locked() Date: Mon, 18 Jul 2022 12:28:38 -0700 Message-Id: <20220718192844.1805158-11-yury.norov@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220718192844.1805158-1-yury.norov@gmail.com> References: <20220718192844.1805158-1-yury.norov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org Don't call cpumask_subset(new_mask, cpu_allowed_mask) if new_mask is cpu_allowed_mask. Caught with CONFIG_DEBUG_BITMAP: [ 0.132174] Call trace: [ 0.132189] __bitmap_check_params+0x144/0x250 [ 0.132216] __set_cpus_allowed_ptr_locked+0x8c/0x2c0 [ 0.132241] sched_init_smp+0x80/0xd8 [ 0.132273] kernel_init_freeable+0x12c/0x28c [ 0.132299] kernel_init+0x28/0x13c [ 0.132325] ret_from_fork+0x10/0x20 [ 0.132354] ---[ end trace 0000000000000000 ]--- [ 0.132378] b1: ffffcd0c07819a58 [ 0.132388] b2: ffffcd0c07819a58 [ 0.132397] b3: 0 [ 0.132405] nbits: 256 [ 0.132414] start: 0 [ 0.132422] off: 0 [ 0.132444] Bitmap: parameters check failed [ 0.132467] include/linux/bitmap.h [468]: bitmap_subset Signed-off-by: Yury Norov --- kernel/sched/core.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/kernel/sched/core.c b/kernel/sched/core.c index da0bf6fe9ecd..d6424336ef2d 100644 --- a/kernel/sched/core.c +++ b/kernel/sched/core.c @@ -2874,7 +2874,8 @@ static int __set_cpus_allowed_ptr_locked(struct task_struct *p, cpu_valid_mask = cpu_online_mask; } - if (!kthread && !cpumask_subset(new_mask, cpu_allowed_mask)) { + if (!kthread && new_mask != cpu_allowed_mask && + !cpumask_subset(new_mask, cpu_allowed_mask)) { ret = -EINVAL; goto out; } From patchwork Mon Jul 18 19:28:39 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yury Norov X-Patchwork-Id: 12921650 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5F4FDCCA479 for ; Mon, 18 Jul 2022 19:30:32 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233601AbiGRTaA (ORCPT ); Mon, 18 Jul 2022 15:30:00 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46824 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S236092AbiGRT3N (ORCPT ); Mon, 18 Jul 2022 15:29:13 -0400 Received: from mail-qk1-x736.google.com (mail-qk1-x736.google.com [IPv6:2607:f8b0:4864:20::736]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B66F76263; Mon, 18 Jul 2022 12:29:00 -0700 (PDT) Received: by mail-qk1-x736.google.com with SMTP id c24so4029775qkm.4; Mon, 18 Jul 2022 12:29:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=D0vGYuUrWK1A1pC+aGb2diJW4SXv6dDhykR2E3SvFc8=; b=acjdcKpDdV2PdvNi+eoEwihWKcpSrRAsrDHiFJG9tElETVBRpRl1BHIlBTcqTpJ1PZ dqPhoWLauRERrZA0j1cq0xNk0IkyjS+bk910OJd6v3ZC939j06bzpOv+oySvXXxzwccG DuquVnf7VDRuglTdL1A81oCQPyP9OiRPuTo9rV8rvvWPSirAXqZUVdJ9cVJF+N/tZAut cn4mvdBOyL7FeaLStoJT+VMKK5o0ToUzvTPwDDD4B5R4znOjfgPiLRfaP8jZoSwV2fdJ sN1KlbxFCs/ArEXTCRiHuBowwklPVPJUzv5nXSPFafZGnOR76aQqVOuKje8NqcNeZoVw N+xw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=D0vGYuUrWK1A1pC+aGb2diJW4SXv6dDhykR2E3SvFc8=; b=og/igHkWFYrWYeZz6vUsN6Pdr0D5z/e1KOxkdUYo6DWB+bdBTFUXXOihxAyekf8vOF P1A35YKN+9HI7Hd/SpAhGNd+lfFT/+0ZsGA1oRse/UfbOytoQNvS3q7XdPfq0p5ZRXLp rsZ/YrypKSy5pB8uBGBheq3N5vL98D8Aon8a+/9LgMJnmH6fB3oy/QYD/1+WfsVHrfPM KMIh7ZHZEsia2jX71oU9Ez6dt75XpHp31aENPTvJZV3R5YLPqd7Oot7V8FlF9q/ZCUeQ hp5UZk1KzMyud4GgalBHYC+z33IzrqM90WIH7Raedb28Im/IINmZfzx78nU92272hAA8 KeEA== X-Gm-Message-State: AJIora/4GaZwBYEP3zbBM4aCA+E75Cy8yAv+dsfDo967NsxhT3C88kmA lA6VGDptmtPOJ8DLdCNcq3lDtZAsJS6tAQ== X-Google-Smtp-Source: AGRyM1sfjfCy0QR9NsI2tMcO46D+TGA9UPgSgzjARo5pB7cKyOwUWXXbOFws+xYx88TQYg8ZRSCfgg== X-Received: by 2002:a37:648f:0:b0:6b5:ccd2:3ff9 with SMTP id y137-20020a37648f000000b006b5ccd23ff9mr11422805qkb.139.1658172539175; Mon, 18 Jul 2022 12:28:59 -0700 (PDT) Received: from localhost ([2601:4c1:c100:1230:ab01:d009:465a:5ab1]) by smtp.gmail.com with ESMTPSA id ez12-20020a05622a4c8c00b0031eb0bb5c3csm9878869qtb.28.2022.07.18.12.28.58 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 18 Jul 2022 12:28:58 -0700 (PDT) From: Yury Norov To: linux-kernel@vger.kernel.org, Alexander Lobakin , Alexei Starovoitov , Alexey Klimov , Andrew Morton , Andrii Nakryiko , Andy Shevchenko , Ben Segall , Christoph Lameter , Dan Williams , Daniel Borkmann , Daniel Bristot de Oliveira , Dennis Zhou , Dietmar Eggemann , Eric Dumazet , Frederic Weisbecker , Guenter Roeck , Ingo Molnar , Isabella Basso , John Fastabend , Josh Poimboeuf , Juergen Gross , Juri Lelli , KP Singh , Kees Cook , Martin KaFai Lau , Mel Gorman , Miroslav Benes , Nathan Chancellor , "Paul E . McKenney" , Peter Zijlstra , Randy Dunlap , Rasmus Villemoes , Sebastian Andrzej Siewior , Song Liu , Steven Rostedt , Tejun Heo , Thomas Gleixner , Valentin Schneider , Vincent Guittot , Vlastimil Babka , Yonghong Song , Yury Norov , linux-mm@kvack.org, netdev@vger.kernel.org, bpf@vger.kernel.org Subject: [PATCH 11/16] time: optimize tick_check_preferred() Date: Mon, 18 Jul 2022 12:28:39 -0700 Message-Id: <20220718192844.1805158-12-yury.norov@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220718192844.1805158-1-yury.norov@gmail.com> References: <20220718192844.1805158-1-yury.norov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org tick_check_preferred() calls cpumask_equal() even if curdev->cpumask == newdev->cpumask. Fix it. Caught with CONFIG_DEBUG_BITMAP: [ 0.079109] Call trace: [ 0.079124] __bitmap_check_params+0x144/0x250 [ 0.079161] tick_check_replacement+0x1a4/0x320 [ 0.079203] tick_check_new_device+0x50/0x110 [ 0.079237] clockevents_register_device+0x74/0x1c0 [ 0.079268] dummy_timer_starting_cpu+0x6c/0x80 [ 0.079310] cpuhp_invoke_callback+0x104/0x20c [ 0.079353] cpuhp_invoke_callback_range+0x70/0xf0 [ 0.079401] notify_cpu_starting+0xac/0xcc [ 0.079434] secondary_start_kernel+0xe4/0x154 [ 0.079471] __secondary_switched+0xa0/0xa4 [ 0.079516] ---[ end trace 0000000000000000 ]--- [ 0.079542] b1: ffffbfec4703b890 [ 0.079553] b2: ffffbfec4703b890 [ 0.079566] b3: 0 [ 0.079576] nbits: 256 [ 0.079588] start: 0 [ 0.079598] off: 0 [ 0.079609] Bitmap: parameters check failed [ 0.079619] include/linux/bitmap.h [419]: bitmap_equal Signed-off-by: Yury Norov --- kernel/time/tick-common.c | 10 +++++++--- 1 file changed, 7 insertions(+), 3 deletions(-) diff --git a/kernel/time/tick-common.c b/kernel/time/tick-common.c index 46789356f856..fdd5ae1a074b 100644 --- a/kernel/time/tick-common.c +++ b/kernel/time/tick-common.c @@ -316,9 +316,13 @@ static bool tick_check_preferred(struct clock_event_device *curdev, * Use the higher rated one, but prefer a CPU local device with a lower * rating than a non-CPU local device */ - return !curdev || - newdev->rating > curdev->rating || - !cpumask_equal(curdev->cpumask, newdev->cpumask); + if (!curdev || newdev->rating > curdev->rating) + return true; + + if (newdev->cpumask == curdev->cpumask) + return false; + + return !cpumask_equal(curdev->cpumask, newdev->cpumask); } /* From patchwork Mon Jul 18 19:28:40 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yury Norov X-Patchwork-Id: 12921652 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id B10D2CCA487 for ; Mon, 18 Jul 2022 19:30:32 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236089AbiGRTaD (ORCPT ); Mon, 18 Jul 2022 15:30:03 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46830 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S236095AbiGRT3N (ORCPT ); Mon, 18 Jul 2022 15:29:13 -0400 Received: from mail-qv1-xf2d.google.com (mail-qv1-xf2d.google.com [IPv6:2607:f8b0:4864:20::f2d]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 326A52A279; Mon, 18 Jul 2022 12:29:02 -0700 (PDT) Received: by mail-qv1-xf2d.google.com with SMTP id g9so9551028qvq.7; Mon, 18 Jul 2022 12:29:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=9jRBPxHetNaAYvNL/Zbk4Gz5JHeIZjINC1YVksHLpK0=; b=UWtmw4bkYSqZCrwkTdXvOJn0HuT45kCRhfcR25N1KZz4XPy5uHeP36ClHVvtxuQ0ml eR5MjNl5qmX49XbhugFRX9Q90pk0oydAsfvrCtMgEq6H+FFLr5M39bqMJV/2aDTL8QMd Mr5FnWMOsu+Oj9iWEBqUquNtUmVGFIUMVcfpkrmO4P1SAV9L6D4GUGE/7pXbBYWmGSvp yCoqENUMPT3+KVmnijAAPfz650MZZj/EIkfrWSELI+A5ssy2P71glhClbjpPNPVxkZHm vYHQO4TP3f3Ii52Sn3fr1U56mheGEatDLvDKTJFdhaq7zhn6Qn+rmyN5Vn2ChEuVs68B RCtQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=9jRBPxHetNaAYvNL/Zbk4Gz5JHeIZjINC1YVksHLpK0=; b=MMDx0q0vzdkhzCXSzUyTA2uNgME5QBkLeAYGw4UHdXWcW4RHJ0q/TFPGVHBcWCxIvD 9xqy/+nl60LJmSv3Uw3Cz3Z0FfNszUBVVWDsXYenoNgG6Ie9Z5D7FPN1A38/2MhDSdW4 lEWFdxJSeznu03Q983i/0J/basWTnsMpyuPi9kgIhOzTqLajyInzXuAm6EgRbFJY+ukQ mfJOC3IHSmKUFd0oKx/dIe7djaINEcssl5XzPKalPvdftIJu/LyjFyFbw+zYf6Z8w4zg q4Z2TFHBwH158mr912fa0sFWOrSQcGrh9i3jbQkOSCDEBeQ1rNWWtuhgOvEJhjWbTNVO CTfg== X-Gm-Message-State: AJIora9wNTh+BQcgvqtVvER3Gao1hq8DCUJQILkw8LKDezQpVBNq3vIG YIPVz83tpV07cJ+jOV4t1ZMo0IG4G88Teg== X-Google-Smtp-Source: AGRyM1tJL6xF5uFJZtWlPYKjynWj22iMoR4ixYcMYqU4mAIXJGlgzVLwHxDXYCNM6iwonqEgJ0BPsw== X-Received: by 2002:a05:6214:d8c:b0:473:5a78:cdd3 with SMTP id e12-20020a0562140d8c00b004735a78cdd3mr22081665qve.105.1658172540532; Mon, 18 Jul 2022 12:29:00 -0700 (PDT) Received: from localhost ([2601:4c1:c100:1230:ab01:d009:465a:5ab1]) by smtp.gmail.com with ESMTPSA id y14-20020a37640e000000b006b5869c1525sm11450264qkb.21.2022.07.18.12.28.59 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 18 Jul 2022 12:28:59 -0700 (PDT) From: Yury Norov To: linux-kernel@vger.kernel.org, Alexander Lobakin , Alexei Starovoitov , Alexey Klimov , Andrew Morton , Andrii Nakryiko , Andy Shevchenko , Ben Segall , Christoph Lameter , Dan Williams , Daniel Borkmann , Daniel Bristot de Oliveira , Dennis Zhou , Dietmar Eggemann , Eric Dumazet , Frederic Weisbecker , Guenter Roeck , Ingo Molnar , Isabella Basso , John Fastabend , Josh Poimboeuf , Juergen Gross , Juri Lelli , KP Singh , Kees Cook , Martin KaFai Lau , Mel Gorman , Miroslav Benes , Nathan Chancellor , "Paul E . McKenney" , Peter Zijlstra , Randy Dunlap , Rasmus Villemoes , Sebastian Andrzej Siewior , Song Liu , Steven Rostedt , Tejun Heo , Thomas Gleixner , Valentin Schneider , Vincent Guittot , Vlastimil Babka , Yonghong Song , Yury Norov , linux-mm@kvack.org, netdev@vger.kernel.org, bpf@vger.kernel.org Subject: [PATCH 12/16] time: optimize tick_check_percpu() Date: Mon, 18 Jul 2022 12:28:40 -0700 Message-Id: <20220718192844.1805158-13-yury.norov@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220718192844.1805158-1-yury.norov@gmail.com> References: <20220718192844.1805158-1-yury.norov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org tick_check_percpu() calls cpumask_equal() even if curdev->cpumask == cpumask_of(cpu). Fix it. Caught with CONFIG_DEBUG_BITMAP: [ 0.077622] Call trace: [ 0.077637] __bitmap_check_params+0x144/0x250 [ 0.077675] tick_check_replacement+0xac/0x320 [ 0.077716] tick_check_new_device+0x50/0x110 [ 0.077747] clockevents_register_device+0x74/0x1c0 [ 0.077779] dummy_timer_starting_cpu+0x6c/0x80 [ 0.077817] cpuhp_invoke_callback+0x104/0x20c [ 0.077856] cpuhp_invoke_callback_range+0x70/0xf0 [ 0.077890] notify_cpu_starting+0xac/0xcc [ 0.077921] secondary_start_kernel+0xe4/0x154 [ 0.077951] __secondary_switched+0xa0/0xa4 [ 0.077992] ---[ end trace 0000000000000000 ]--- [ 0.078021] b1: ffffbfec4703b890 [ 0.078031] b2: ffffbfec4703b890 [ 0.078043] b3: 0 [ 0.078052] nbits: 256 [ 0.078065] start: 0 [ 0.078075] off: 0 [ 0.078086] Bitmap: parameters check failed [ 0.078095] include/linux/bitmap.h [419]: bitmap_equal Signed-off-by: Yury Norov --- kernel/time/tick-common.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/kernel/time/tick-common.c b/kernel/time/tick-common.c index fdd5ae1a074b..7205f76f8d10 100644 --- a/kernel/time/tick-common.c +++ b/kernel/time/tick-common.c @@ -290,13 +290,15 @@ static bool tick_check_percpu(struct clock_event_device *curdev, { if (!cpumask_test_cpu(cpu, newdev->cpumask)) return false; - if (cpumask_equal(newdev->cpumask, cpumask_of(cpu))) + if (newdev->cpumask == cpumask_of(cpu) || + cpumask_equal(newdev->cpumask, cpumask_of(cpu))) return true; /* Check if irq affinity can be set */ if (newdev->irq >= 0 && !irq_can_set_affinity(newdev->irq)) return false; /* Prefer an existing cpu local device */ - if (curdev && cpumask_equal(curdev->cpumask, cpumask_of(cpu))) + if (curdev && (curdev->cpumask == cpumask_of(cpu) || + cpumask_equal(curdev->cpumask, cpumask_of(cpu)))) return false; return true; } From patchwork Mon Jul 18 19:28:41 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yury Norov X-Patchwork-Id: 12921651 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6FA99C433EF for ; Mon, 18 Jul 2022 19:30:32 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236085AbiGRTaC (ORCPT ); Mon, 18 Jul 2022 15:30:02 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46464 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S236099AbiGRT3N (ORCPT ); Mon, 18 Jul 2022 15:29:13 -0400 Received: from mail-qt1-x833.google.com (mail-qt1-x833.google.com [IPv6:2607:f8b0:4864:20::833]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 32D74A44A; Mon, 18 Jul 2022 12:29:03 -0700 (PDT) Received: by mail-qt1-x833.google.com with SMTP id l14so7164036qtv.4; Mon, 18 Jul 2022 12:29:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=nMdR+AEtByh2Sj3uhRet9r/mYlLTue7hiNXqJ4zPF7A=; b=ThZjcWWpivQL4LIMCYMLUyqh5o5kXMqBz+8OT2okIYTlrj/edxwBb2dbQa9UPOmtF6 4kSD+ZK2zlONe0lcAJjESBk9Nh+IMF2AWtzKWASNF/7e6AOQFfgrht6kvwCXXaBw+ZJ/ TuekgFubFOIx/+8XhYbzZTP0R9y2FHpWi0c1TNSBFoZBSAURAUsk02B8TI4+roOQrfEb rqvV8Zlz0kLm2wnKF7Eb5xNF8jQ/LSP7mPfSKTCz8rO9Som4BkQVoJkslp7Xj+XOaxbI /IAz4pqpN4giyGSLVmRa+JZ33lWzIiVJrwXSuZTIP487wCG2DZbEQ+zSsLJzYmv+Cs+/ XhuA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=nMdR+AEtByh2Sj3uhRet9r/mYlLTue7hiNXqJ4zPF7A=; b=qLIGhyV2409m4CxitSPQKxY9DA+HrMefJE9hJcR0YL9nXimwMT+/yC9S8rFm0jyXTI cqE1A37T4LeI0Hlnqzak0YXGFTysI9GPy23KR3DRuUeJFFVtc2LtYSg2e5Hy0kwKTBly Na4QKhuGB4H/vTeMJMKynqt2/m3n3rZWXNlo45RzihQEgF3lNlCb5+ScuufZLyt/cdPQ 2jFiwqWLQzoX+SASXBM2NieCTL/4YT1LqKodzyJy7ZyNivFYsC3oquo741CarDlhCH8q TSenN/RPrWNnaaA/8Fr1DnWNQyWS5jpaC7hpRl75lNYV1VCSOvcXsj7dRenG4Mzzwlsa GAPA== X-Gm-Message-State: AJIora+diTQeF2YOz/Bz+t5JYP55yN4z66jwxlmMPaYipdQp9JkXlDtb ILUnKVMNIEJxQ5aybK7CGAY6KIef0Tm29w== X-Google-Smtp-Source: AGRyM1vJb89RwvBBeMzFkakUtULOUGREDepYG3L9kNDYRN+9fj+cWruAf8vfYMA3CaqjYR907kc98w== X-Received: by 2002:a05:622a:213:b0:31e:c569:220e with SMTP id b19-20020a05622a021300b0031ec569220emr22576563qtx.436.1658172542486; Mon, 18 Jul 2022 12:29:02 -0700 (PDT) Received: from localhost ([2601:4c1:c100:1230:ab01:d009:465a:5ab1]) by smtp.gmail.com with ESMTPSA id n18-20020a05620a295200b006b5e45ff82csm5125690qkp.93.2022.07.18.12.29.01 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 18 Jul 2022 12:29:02 -0700 (PDT) From: Yury Norov To: linux-kernel@vger.kernel.org, Alexander Lobakin , Alexei Starovoitov , Alexey Klimov , Andrew Morton , Andrii Nakryiko , Andy Shevchenko , Ben Segall , Christoph Lameter , Dan Williams , Daniel Borkmann , Daniel Bristot de Oliveira , Dennis Zhou , Dietmar Eggemann , Eric Dumazet , Frederic Weisbecker , Guenter Roeck , Ingo Molnar , Isabella Basso , John Fastabend , Josh Poimboeuf , Juergen Gross , Juri Lelli , KP Singh , Kees Cook , Martin KaFai Lau , Mel Gorman , Miroslav Benes , Nathan Chancellor , "Paul E . McKenney" , Peter Zijlstra , Randy Dunlap , Rasmus Villemoes , Sebastian Andrzej Siewior , Song Liu , Steven Rostedt , Tejun Heo , Thomas Gleixner , Valentin Schneider , Vincent Guittot , Vlastimil Babka , Yonghong Song , Yury Norov , linux-mm@kvack.org, netdev@vger.kernel.org, bpf@vger.kernel.org Subject: [PATCH 13/16] time: optimize tick_setup_device() Date: Mon, 18 Jul 2022 12:28:41 -0700 Message-Id: <20220718192844.1805158-14-yury.norov@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220718192844.1805158-1-yury.norov@gmail.com> References: <20220718192844.1805158-1-yury.norov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org tick_setup_device() calls cpumask_equal(newdev->cpumask, cpumask) even if newdev->cpumask == cpumask. Fix it. Caught with CONFIG_DEBUG_BITMAP: [ 0.070960] Call trace: [ 0.070974] __bitmap_check_params+0x144/0x250 [ 0.071008] tick_setup_device+0x70/0x1a0 [ 0.071040] tick_check_new_device+0xc0/0x110 [ 0.071066] clockevents_register_device+0x74/0x1c0 [ 0.071090] clockevents_config_and_register+0x2c/0x3c [ 0.071114] arch_timer_starting_cpu+0x170/0x470 [ 0.071147] cpuhp_invoke_callback+0x104/0x20c [ 0.071180] cpuhp_invoke_callback_range+0x70/0xf0 [ 0.071205] notify_cpu_starting+0xac/0xcc [ 0.071229] secondary_start_kernel+0xe4/0x154 [ 0.071259] __secondary_switched+0xa0/0xa4 [ 0.071297] ---[ end trace 0000000000000000 ]--- [ 0.071328] b1: ffffa1f27323b890 [ 0.071339] b2: ffffa1f27323b890 [ 0.071348] b3: 0 [ 0.071356] nbits: 256 [ 0.071366] start: 0 [ 0.071374] off: 0 [ 0.071383] Bitmap: parameters check failed [ 0.071390] include/linux/bitmap.h [419]: bitmap_equal Signed-off-by: Yury Norov --- kernel/time/tick-common.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/kernel/time/tick-common.c b/kernel/time/tick-common.c index 7205f76f8d10..7b2da8ef09ef 100644 --- a/kernel/time/tick-common.c +++ b/kernel/time/tick-common.c @@ -255,7 +255,7 @@ static void tick_setup_device(struct tick_device *td, * When the device is not per cpu, pin the interrupt to the * current cpu: */ - if (!cpumask_equal(newdev->cpumask, cpumask)) + if (newdev->cpumask != cpumask && !cpumask_equal(newdev->cpumask, cpumask)) irq_set_affinity(newdev->irq, cpumask); /* From patchwork Mon Jul 18 19:28:42 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yury Norov X-Patchwork-Id: 12921655 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id D801CCCA481 for ; Mon, 18 Jul 2022 19:31:04 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235969AbiGRTak (ORCPT ); Mon, 18 Jul 2022 15:30:40 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46884 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235832AbiGRT3O (ORCPT ); Mon, 18 Jul 2022 15:29:14 -0400 Received: from mail-qv1-xf2d.google.com (mail-qv1-xf2d.google.com [IPv6:2607:f8b0:4864:20::f2d]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8D0432D1E3; Mon, 18 Jul 2022 12:29:04 -0700 (PDT) Received: by mail-qv1-xf2d.google.com with SMTP id m6so9543539qvq.10; Mon, 18 Jul 2022 12:29:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=/K1YhC0xN+k7O2FGj/yOUKehgh1HnqMHevQj0FQmavI=; b=hncdckdd/8keGq0BbMkUSkVvo58eaSfk46icZhjcfux7VapB1d8lOZsIlqh8gYAlwO nsH6TNRxQKop1KFXKAjUK6jXGpI96ZhzGYzEb1l58Uuid0BejJtLScjUXd1FwD7d6d0I n6nBpJsBMCWPJZOY2isxybjgVNgOUtIwy5syFw6dhjVkH48EFPjJXnsXMjILmWWh6aNt 1XjXQ+KmT1tei8OVv0kOBxiaSSBAGf1jRYjjXWTW/jEj/bm8gP4G3KDncFs30f51O1n9 KoibE09XlnBMUE30LzuwHeutbdP4Gw0vPBuJAXfM44toJn9n4BYblu9g3pXR4SHa5Cz4 j6aw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=/K1YhC0xN+k7O2FGj/yOUKehgh1HnqMHevQj0FQmavI=; b=1hsppSoi2ihKjmjPsCy+fDCVQq+lzVFsc/JfIYZ/HJn7JWk82VTaA14TsWON25hg18 WVPYJNICoIFtcmTuyxVQ8prv/pwuWkBWckx26KBsjG316GtlXK3jFetOHihXpHIUcZF3 nHz6FbE/H29W2E0VlmV2RP1+V2F2wGkHRlunDjHofHTuFb12VoZ4biK+xPH1R02Eu2mo 0Km8k6s/OY5sHoAefCSUav2wG7qUd36p7copYVcasv1cuzmLgbBI4g41HMkfHaGcyPYK BFkBRGLmhQw4YjlByy+CldPOuzQASabNxl0FBluyvlMxp04cY0OX0t2QS8QWJ5eL/fmJ 7Mpw== X-Gm-Message-State: AJIora+zmkzVeUO+3yuEbLkjClwFriNLYb7Jum0UjD3uf9gjsqRGXsOx /s90bE6B4IaeyOHa0bMAGXRcKVLn7thrDw== X-Google-Smtp-Source: AGRyM1uDAtG39PkyBUTwcaTq2/uDHYY4DcIItWFReB0uOGieqEEzLgaz94Wt8f3sFC0ByQPjoqZW6g== X-Received: by 2002:a05:6214:226d:b0:473:339e:3264 with SMTP id gs13-20020a056214226d00b00473339e3264mr22107594qvb.41.1658172543484; Mon, 18 Jul 2022 12:29:03 -0700 (PDT) Received: from localhost ([2601:4c1:c100:1230:ab01:d009:465a:5ab1]) by smtp.gmail.com with ESMTPSA id bb31-20020a05622a1b1f00b0031ef21aec36sm2216383qtb.32.2022.07.18.12.29.03 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 18 Jul 2022 12:29:03 -0700 (PDT) From: Yury Norov To: linux-kernel@vger.kernel.org, Alexander Lobakin , Alexei Starovoitov , Alexey Klimov , Andrew Morton , Andrii Nakryiko , Andy Shevchenko , Ben Segall , Christoph Lameter , Dan Williams , Daniel Borkmann , Daniel Bristot de Oliveira , Dennis Zhou , Dietmar Eggemann , Eric Dumazet , Frederic Weisbecker , Guenter Roeck , Ingo Molnar , Isabella Basso , John Fastabend , Josh Poimboeuf , Juergen Gross , Juri Lelli , KP Singh , Kees Cook , Martin KaFai Lau , Mel Gorman , Miroslav Benes , Nathan Chancellor , "Paul E . McKenney" , Peter Zijlstra , Randy Dunlap , Rasmus Villemoes , Sebastian Andrzej Siewior , Song Liu , Steven Rostedt , Tejun Heo , Thomas Gleixner , Valentin Schneider , Vincent Guittot , Vlastimil Babka , Yonghong Song , Yury Norov , linux-mm@kvack.org, netdev@vger.kernel.org, bpf@vger.kernel.org Subject: [PATCH 14/16] mm/percpu: optimize pcpu_alloc_area() Date: Mon, 18 Jul 2022 12:28:42 -0700 Message-Id: <20220718192844.1805158-15-yury.norov@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220718192844.1805158-1-yury.norov@gmail.com> References: <20220718192844.1805158-1-yury.norov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org Don't call bitmap_clear() to clear 0 bits. bitmap_clear() can handle 0-length requests properly, but it's not covered with static optimizations, and falls to __bitmap_set(). So we are paying a function call + prologue work cost just for nothing. Caught with CONFIG_DEBUG_BITMAP: [ 45.571799] [ 45.571801] pcpu_alloc_area+0x194/0x340 [ 45.571806] pcpu_alloc+0x2fb/0x8b0 [ 45.571811] ? kmem_cache_alloc_trace+0x177/0x2a0 [ 45.571815] __percpu_counter_init+0x22/0xa0 [ 45.571819] fprop_local_init_percpu+0x14/0x30 [ 45.571823] wb_get_create+0x15d/0x5f0 [ 45.571828] cleanup_offline_cgwb+0x73/0x210 [ 45.571831] cleanup_offline_cgwbs_workfn+0xcf/0x200 [ 45.571835] process_one_work+0x1e5/0x3b0 [ 45.571839] worker_thread+0x50/0x3a0 [ 45.571843] ? rescuer_thread+0x390/0x390 [ 45.571846] kthread+0xe8/0x110 [ 45.571849] ? kthread_complete_and_exit+0x20/0x20 [ 45.571853] ret_from_fork+0x22/0x30 [ 45.571858] [ 45.571859] ---[ end trace 0000000000000000 ]--- [ 45.571860] b1: ffffa8d5002e1000 [ 45.571861] b2: 0 [ 45.571861] b3: 0 [ 45.571862] nbits: 44638 [ 45.571863] start: 44638 [ 45.571864] off: 0 [ 45.571864] percpu: Bitmap: parameters check failed [ 45.571865] percpu: include/linux/bitmap.h [538]: bitmap_clear Signed-off-by: Yury Norov Acked-by: Dennis Zhou --- mm/percpu.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/mm/percpu.c b/mm/percpu.c index 3633eeefaa0d..f720f7c36b91 100644 --- a/mm/percpu.c +++ b/mm/percpu.c @@ -1239,7 +1239,8 @@ static int pcpu_alloc_area(struct pcpu_chunk *chunk, int alloc_bits, /* update boundary map */ set_bit(bit_off, chunk->bound_map); - bitmap_clear(chunk->bound_map, bit_off + 1, alloc_bits - 1); + if (alloc_bits > 1) + bitmap_clear(chunk->bound_map, bit_off + 1, alloc_bits - 1); set_bit(bit_off + alloc_bits, chunk->bound_map); chunk->free_bytes -= alloc_bits * PCPU_MIN_ALLOC_SIZE; From patchwork Mon Jul 18 19:28:43 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yury Norov X-Patchwork-Id: 12921654 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id A2E39CCA479 for ; Mon, 18 Jul 2022 19:31:04 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234294AbiGRTag (ORCPT ); Mon, 18 Jul 2022 15:30:36 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46846 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235933AbiGRT3P (ORCPT ); Mon, 18 Jul 2022 15:29:15 -0400 Received: from mail-qt1-x830.google.com (mail-qt1-x830.google.com [IPv6:2607:f8b0:4864:20::830]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id AB4812DA8E; Mon, 18 Jul 2022 12:29:05 -0700 (PDT) Received: by mail-qt1-x830.google.com with SMTP id e5so9537614qts.1; Mon, 18 Jul 2022 12:29:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=tA2rl8RctZ6HL9V5x9xFMH8oSgBBwUuElKQvWOHzbeI=; b=hP/RxbCCEkxGibcmzYfoczVfz2LGpVvZkrG5wlu75XNiAI1BfNZJPMtwQd0PYtFa4k 9sqDaS6kmGHiRQpzGl3YHVh0l8qAJUfrnfctlEXZsk+yJaCHbydmsJ/FIeyIMRe2QYYS AB02g9p+c9CoMw9OrSZ9omO4Oxfqs9oeUUaFdWEZpJcv2MJ7cdtPYz3Fd8zBnbGvz1Iw o7jHZpB3NBZRj1mxkRUcREMvO38UFX1tl3lIgHvmj+MBAc2h6f3RXJF3Q60pUT+dIr1T Zgt4vkyg0W1HEu5nB4dG2ctBT26OjozQ53zCuCaHayZmI0SZj41p5lheZfaXVwMwGzSd cd+Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=tA2rl8RctZ6HL9V5x9xFMH8oSgBBwUuElKQvWOHzbeI=; b=qtjOuBinv1cjHReODh9T0bjPhnyu5jC0QelJgxKYx3Arsce7Yigf0yxIcSts23lfb1 pAury/y9BIhoIkAzApCV9F8JcplIbrVrqxQ97D7k8HVFm4X8zsSVVnVUkbjOdBj0yAc3 9E6VSQROy+f/KIr8zpVMmc4O4oX6Dv8+4TKehi0lA6Amg6uWCqk+qxa7IoE4vpAzwGV/ C2NNz1w4gmz0xDH1ZpGl0lnPaaGycQ077ZA38BKll8euwrmSG6eh/Xblm5QoATD914I9 Lq699aNuQA7exlALJUyr7hvZaGobg4OvvJUf5/s7+fMsWI00X4J3leqy2UKgY+Jzmjef ffNg== X-Gm-Message-State: AJIora+QaTfqUYwF0qSHYlCkEyEoFJFW/ZybbU+sSRvjRdmjbCEffEla 4WF1dBbWJ7Bt/jgFgtH7eF14EZ224ObACw== X-Google-Smtp-Source: AGRyM1vQmvbkX1kTEORONuNSMwS4ZWos+Eg2Ylq19DBEaI61zIn7vgC2XHqbVwUo3RPShBlZ/+yVwg== X-Received: by 2002:ac8:5790:0:b0:31e:f69a:1b9e with SMTP id v16-20020ac85790000000b0031ef69a1b9emr2408133qta.103.1658172544528; Mon, 18 Jul 2022 12:29:04 -0700 (PDT) Received: from localhost ([2601:4c1:c100:1230:ab01:d009:465a:5ab1]) by smtp.gmail.com with ESMTPSA id v17-20020ac85791000000b0031ee2080c73sm6310369qta.54.2022.07.18.12.29.04 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 18 Jul 2022 12:29:04 -0700 (PDT) From: Yury Norov To: linux-kernel@vger.kernel.org, Alexander Lobakin , Alexei Starovoitov , Alexey Klimov , Andrew Morton , Andrii Nakryiko , Andy Shevchenko , Ben Segall , Christoph Lameter , Dan Williams , Daniel Borkmann , Daniel Bristot de Oliveira , Dennis Zhou , Dietmar Eggemann , Eric Dumazet , Frederic Weisbecker , Guenter Roeck , Ingo Molnar , Isabella Basso , John Fastabend , Josh Poimboeuf , Juergen Gross , Juri Lelli , KP Singh , Kees Cook , Martin KaFai Lau , Mel Gorman , Miroslav Benes , Nathan Chancellor , "Paul E . McKenney" , Peter Zijlstra , Randy Dunlap , Rasmus Villemoes , Sebastian Andrzej Siewior , Song Liu , Steven Rostedt , Tejun Heo , Thomas Gleixner , Valentin Schneider , Vincent Guittot , Vlastimil Babka , Yonghong Song , Yury Norov , linux-mm@kvack.org, netdev@vger.kernel.org, bpf@vger.kernel.org Subject: [PATCH 15/16] sched/topology: optimize topology_span_sane() Date: Mon, 18 Jul 2022 12:28:43 -0700 Message-Id: <20220718192844.1805158-16-yury.norov@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220718192844.1805158-1-yury.norov@gmail.com> References: <20220718192844.1805158-1-yury.norov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org topology_span_sane() checks if cpu == i before calling cpumask_equal(tl->mask(cpu), tl->mask(i)). However, tl->mask(cpu) and tl->mask(i) may point to the same cpumask even if i != cpu. Fix the check accordingly. While here, move tl->mask(cpu) out of the loop, and make the in-loop code calculating tl->mask(i) only once. Catched with CONFIG_DEBUG_BITMAP: [ 0.867917] Call Trace: [ 0.868209] [ 0.868471] build_sched_domains+0x36f/0x1a40 [ 0.868576] sched_init_smp+0x44/0xba [ 0.869012] ? mtrr_aps_init+0x84/0xa0 [ 0.869465] kernel_init_freeable+0x12e/0x26e [ 0.869982] ? rest_init+0xd0/0xd0 [ 0.870406] kernel_init+0x16/0x120 [ 0.870821] ret_from_fork+0x22/0x30 [ 0.871244] [ 0.871502] ---[ end trace 0000000000000000 ]--- [ 0.872040] b1: ffffffffb1fd3480 [ 0.872041] b2: ffffffffb1fd3480 [ 0.872041] b3: 0 [ 0.872042] nbits: 256 [ 0.872042] start: 0 [ 0.872042] off: 0 [ 0.872043] Bitmap: parameters check failed [ 0.872043] include/linux/bitmap.h [427]: bitmap_equal Signed-off-by: Yury Norov --- kernel/sched/topology.c | 10 ++++++++-- 1 file changed, 8 insertions(+), 2 deletions(-) diff --git a/kernel/sched/topology.c b/kernel/sched/topology.c index 05b6c2ad90b9..ad32d0a43424 100644 --- a/kernel/sched/topology.c +++ b/kernel/sched/topology.c @@ -2211,6 +2211,8 @@ static struct sched_domain *build_sched_domain(struct sched_domain_topology_leve static bool topology_span_sane(struct sched_domain_topology_level *tl, const struct cpumask *cpu_map, int cpu) { + const struct cpumask *mc = tl->mask(cpu); + const struct cpumask *mi; int i; /* NUMA levels are allowed to overlap */ @@ -2226,14 +2228,18 @@ static bool topology_span_sane(struct sched_domain_topology_level *tl, for_each_cpu(i, cpu_map) { if (i == cpu) continue; + + mi = tl->mask(i); + if (mi == mc) + continue; + /* * We should 'and' all those masks with 'cpu_map' to exactly * match the topology we're about to build, but that can only * remove CPUs, which only lessens our ability to detect * overlaps */ - if (!cpumask_equal(tl->mask(cpu), tl->mask(i)) && - cpumask_intersects(tl->mask(cpu), tl->mask(i))) + if (!cpumask_equal(mc, mi) && cpumask_intersects(mc, mi)) return false; } From patchwork Mon Jul 18 19:28:44 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yury Norov X-Patchwork-Id: 12921653 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id B86C3CCA485 for ; Mon, 18 Jul 2022 19:31:04 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235387AbiGRTai (ORCPT ); Mon, 18 Jul 2022 15:30:38 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46848 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235948AbiGRT3P (ORCPT ); Mon, 18 Jul 2022 15:29:15 -0400 Received: from mail-qt1-x835.google.com (mail-qt1-x835.google.com [IPv6:2607:f8b0:4864:20::835]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 928CF2F67A; Mon, 18 Jul 2022 12:29:06 -0700 (PDT) Received: by mail-qt1-x835.google.com with SMTP id b21so6415857qte.12; Mon, 18 Jul 2022 12:29:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=Y6DPpRFLDofGUL30b8+74GM4qwE4L2Zyq6sG1ZF6ebc=; b=dJrCdDdqoGBgCDi7uWoyd5h17sRa80Fs9GLm+P5DflfdZGuXJIVujeqX7lMRY06I8S t67bwzKyDstdRzhhUJT0vfUUmKANMlnUrNgw70sKglBzGUmW7C376hy4i3UQnjZifkek /9+PVCJu47AzGcSDT0BMA8vnweK45F4w3DfYiIC12NYsnjiEIPVVk5+271whtyjRYaNN N9ewHE3z8dFgGz8xroMMjlLM9s5XJQjkXW67JbGA6OgtOLqZ6+prLIpP2wjU2YHGx838 2Rnf2t9y4v4gfPTyeAgE+qUm/m8zeflm6dFi7d626bdAey5662yUyGEU3mpx0x/jNXL7 +mkg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Y6DPpRFLDofGUL30b8+74GM4qwE4L2Zyq6sG1ZF6ebc=; b=wP3GvDwFVRs9H7qvilG90HR6AhLNHDEI3Ycr/eTJnlFFRodczLlKLQEUG10y6tH+I0 DJEiWk6pHoWhVVU0SwmnMnufq+CNxOP5qMA8WsvSbDrZDmQeBJTM7P7EKy+PKFD53Fv6 tFha0hoP6UpOCJB2ZE3aVkOuA8e8zJ0rIwt8t53t3ZPjAjsreiFjUiMmvv21bz9kTsnq xliJ1ISaCQWGvb4d7BE+yiiuVA+0P6OhudhRnwO7/rJa10YS13aopyQJE5lP4HiXIdNU rgmZu1vtoFXPTLPStQCJceeGK5mrqSxGFYEBsXpDO5fi6jwfd7cO5HYcv+UoDoevWeM3 If5g== X-Gm-Message-State: AJIora8psmcXV9HAwuiLH0yCdAn75ZXcyaPfj8SShXWg0JVMJpEIV5QI LjsManDY5d9RlMCl2NxspXCk8rKoATnkwQ== X-Google-Smtp-Source: AGRyM1syCvkoM65NB7O+4/9AZzHgi6k/66P3Xg8nH/ToRSp+ANBXlBuAKKOv4tS6vWWn7KIVfqErdQ== X-Received: by 2002:a05:622a:1211:b0:31e:ba39:41e8 with SMTP id y17-20020a05622a121100b0031eba3941e8mr22777530qtx.189.1658172545549; Mon, 18 Jul 2022 12:29:05 -0700 (PDT) Received: from localhost ([2601:4c1:c100:1230:ab01:d009:465a:5ab1]) by smtp.gmail.com with ESMTPSA id dm1-20020a05620a1d4100b006b5c87e9eb1sm11659333qkb.102.2022.07.18.12.29.05 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 18 Jul 2022 12:29:05 -0700 (PDT) From: Yury Norov To: linux-kernel@vger.kernel.org, Alexander Lobakin , Alexei Starovoitov , Alexey Klimov , Andrew Morton , Andrii Nakryiko , Andy Shevchenko , Ben Segall , Christoph Lameter , Dan Williams , Daniel Borkmann , Daniel Bristot de Oliveira , Dennis Zhou , Dietmar Eggemann , Eric Dumazet , Frederic Weisbecker , Guenter Roeck , Ingo Molnar , Isabella Basso , John Fastabend , Josh Poimboeuf , Juergen Gross , Juri Lelli , KP Singh , Kees Cook , Martin KaFai Lau , Mel Gorman , Miroslav Benes , Nathan Chancellor , "Paul E . McKenney" , Peter Zijlstra , Randy Dunlap , Rasmus Villemoes , Sebastian Andrzej Siewior , Song Liu , Steven Rostedt , Tejun Heo , Thomas Gleixner , Valentin Schneider , Vincent Guittot , Vlastimil Babka , Yonghong Song , Yury Norov , linux-mm@kvack.org, netdev@vger.kernel.org, bpf@vger.kernel.org Subject: [PATCH 16/16] lib: create CONFIG_DEBUG_BITMAP parameter Date: Mon, 18 Jul 2022 12:28:44 -0700 Message-Id: <20220718192844.1805158-17-yury.norov@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220718192844.1805158-1-yury.norov@gmail.com> References: <20220718192844.1805158-1-yury.norov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org Create CONFIG_DEBUG_BITMAP parameter to let people use new option. Default is N. Signed-off-by: Yury Norov --- lib/Kconfig.debug | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/lib/Kconfig.debug b/lib/Kconfig.debug index 2e24db4bff19..cde1b5b7bb9d 100644 --- a/lib/Kconfig.debug +++ b/lib/Kconfig.debug @@ -533,6 +533,13 @@ config DEBUG_FORCE_WEAK_PER_CPU endmenu # "Compiler options" +config DEBUG_BITMAP + bool "Debug bitmaps" + help + Say Y here if you want to check bitmap functions parameters at + the runtime. Enable CONFIG_DEBUG_BITMAP only for debugging because + it may affect performance. + menu "Generic Kernel Debugging Instruments" config MAGIC_SYSRQ