From patchwork Fri Sep 23 13:25:21 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Valentin Schneider X-Patchwork-Id: 12986579 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5E76BECAAD8 for ; Fri, 23 Sep 2022 13:26:22 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231490AbiIWN0T (ORCPT ); Fri, 23 Sep 2022 09:26:19 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60492 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232355AbiIWN0E (ORCPT ); Fri, 23 Sep 2022 09:26:04 -0400 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7FC4D145CBA for ; Fri, 23 Sep 2022 06:25:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1663939557; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=3v7UUwoj2z6XyTFhjMkkxJBC5oQ9g5/RV9r887kLcjk=; b=QHcBH+zGrluJ8OwYL1j5obuUMPWVDkSc9LI0HCJOmugqV+5/6f0j9gAjxyzTAxzNu7Ebwt y2gF88ogPzBmkUAxxiMyosTpdM+PSweavVa8SqaMzK4PQN5km1mgN34DkZiKm0LoS3DaEe v9j+kbTl8rwNrK2i3G9EFP2TtEFIyZg= Received: from mail-wm1-f72.google.com (mail-wm1-f72.google.com [209.85.128.72]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_128_GCM_SHA256) id us-mta-606-158HFahzPOyxg81OqdWHvQ-1; Fri, 23 Sep 2022 09:25:54 -0400 X-MC-Unique: 158HFahzPOyxg81OqdWHvQ-1 Received: by mail-wm1-f72.google.com with SMTP id h187-20020a1c21c4000000b003b51369ff1bso736053wmh.3 for ; Fri, 23 Sep 2022 06:25:54 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date; bh=3v7UUwoj2z6XyTFhjMkkxJBC5oQ9g5/RV9r887kLcjk=; b=D5n9HhZNBSGYu6uutg75qs0nC5BJjYH2VuQuehl4ImbsPMUsh52dvb6byzqjKbziKx ZHJvZvoY1iuWLh3BnnxwrpWyRVqA8R1YR1LxpJHTaXl1sPVj+ZINGdll1P61fG22CtAx PSikNGVrYTnvpld3DUBdifhSjBrD/IiPhw26couPmyYoIoub9ViYKa7rW3k0mibniEDV j2l8X4rv/w6r3SBPYbIvknaNed4fWgBuoJ7pP9Fn6vE+w+BHS8lUgSpGoVwyUmGvxEh5 Gfhv4MA5Ez//iEihVVx1iVQfFgzTZRVxZAvQLdM+BRukujHOBCMNXnJb19346Tgk+qqM cgJg== X-Gm-Message-State: ACrzQf07GD+5PP4Y/f6Hjv6oxcYxI8cNlegcwtWo6mJxlnOdOJSUFJVO tdjDBCv+LXpXJ20YzT4MRVxvo7HEi8+VjkYcPcui0+7B8UHNkjBt8c1/6glw0xWJj6DUaAQxBvD b8Uq90HdS8VA4/svq+2QsPQ== X-Received: by 2002:a7b:ca46:0:b0:3b4:7ff1:4fcc with SMTP id m6-20020a7bca46000000b003b47ff14fccmr13250195wml.47.1663939551503; Fri, 23 Sep 2022 06:25:51 -0700 (PDT) X-Google-Smtp-Source: AMsMyM7G6JjOLCwT6f+/rhD9fsfQaNjJBH3cpQyFzkJ5Y39WRlDbzFautTo6dP0A7bA2xxxc1pNi1Q== X-Received: by 2002:a7b:ca46:0:b0:3b4:7ff1:4fcc with SMTP id m6-20020a7bca46000000b003b47ff14fccmr13250179wml.47.1663939551281; Fri, 23 Sep 2022 06:25:51 -0700 (PDT) Received: from vschneid.remote.csb ([185.11.37.247]) by smtp.gmail.com with ESMTPSA id q5-20020a5d6585000000b0022add5a6fb1sm7067306wru.30.2022.09.23.06.25.49 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 23 Sep 2022 06:25:50 -0700 (PDT) From: Valentin Schneider To: netdev@vger.kernel.org, linux-rdma@vger.kernel.org, linux-kernel@vger.kernel.org Cc: Saeed Mahameed , Leon Romanovsky , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Yury Norov , Andy Shevchenko , Rasmus Villemoes , Ingo Molnar , Peter Zijlstra , Vincent Guittot , Dietmar Eggemann , Steven Rostedt , Mel Gorman , Greg Kroah-Hartman , Heiko Carstens , Tony Luck , Jonathan Cameron , Gal Pressman , Tariq Toukan , Jesse Brandeburg Subject: [PATCH v4 1/7] lib/find_bit: Introduce find_next_andnot_bit() Date: Fri, 23 Sep 2022 14:25:21 +0100 Message-Id: <20220923132527.1001870-2-vschneid@redhat.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20220923132527.1001870-1-vschneid@redhat.com> References: <20220923132527.1001870-1-vschneid@redhat.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org In preparation of introducing for_each_cpu_andnot(), add a variant of find_next_bit() that negate the bits in @addr2 when ANDing them with the bits in @addr1. Signed-off-by: Valentin Schneider --- include/linux/find.h | 33 +++++++++++++++++++++++++++++++++ lib/find_bit.c | 9 +++++++++ 2 files changed, 42 insertions(+) diff --git a/include/linux/find.h b/include/linux/find.h index dead6f53a97b..e60b1ce89b29 100644 --- a/include/linux/find.h +++ b/include/linux/find.h @@ -12,6 +12,8 @@ unsigned long _find_next_bit(const unsigned long *addr1, unsigned long nbits, unsigned long start); unsigned long _find_next_and_bit(const unsigned long *addr1, const unsigned long *addr2, unsigned long nbits, unsigned long start); +unsigned long _find_next_andnot_bit(const unsigned long *addr1, const unsigned long *addr2, + unsigned long nbits, unsigned long start); unsigned long _find_next_zero_bit(const unsigned long *addr, unsigned long nbits, unsigned long start); extern unsigned long _find_first_bit(const unsigned long *addr, unsigned long size); @@ -86,6 +88,37 @@ unsigned long find_next_and_bit(const unsigned long *addr1, } #endif +#ifndef find_next_andnot_bit +/** + * find_next_andnot_bit - find the next set bit in *addr1 excluding all the bits + * in *addr2 + * @addr1: The first address to base the search on + * @addr2: The second address to base the search on + * @size: The bitmap size in bits + * @offset: The bitnumber to start searching at + * + * Returns the bit number for the next set bit + * If no bits are set, returns @size. + */ +static inline +unsigned long find_next_andnot_bit(const unsigned long *addr1, + const unsigned long *addr2, unsigned long size, + unsigned long offset) +{ + if (small_const_nbits(size)) { + unsigned long val; + + if (unlikely(offset >= size)) + return size; + + val = *addr1 & ~*addr2 & GENMASK(size - 1, offset); + return val ? __ffs(val) : size; + } + + return _find_next_andnot_bit(addr1, addr2, size, offset); +} +#endif + #ifndef find_next_zero_bit /** * find_next_zero_bit - find the next cleared bit in a memory region diff --git a/lib/find_bit.c b/lib/find_bit.c index d00ee23ab657..53b02405421b 100644 --- a/lib/find_bit.c +++ b/lib/find_bit.c @@ -120,6 +120,15 @@ unsigned long _find_next_and_bit(const unsigned long *addr1, const unsigned long EXPORT_SYMBOL(_find_next_and_bit); #endif +#ifndef find_next_andnot_bit +unsigned long _find_next_andnot_bit(const unsigned long *addr1, const unsigned long *addr2, + unsigned long nbits, unsigned long start) +{ + return FIND_NEXT_BIT(addr1[idx] & ~addr2[idx], /* nop */, nbits, start); +} +EXPORT_SYMBOL(_find_next_andnot_bit); +#endif + #ifndef find_next_zero_bit unsigned long _find_next_zero_bit(const unsigned long *addr, unsigned long nbits, unsigned long start) From patchwork Fri Sep 23 15:55:37 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Valentin Schneider X-Patchwork-Id: 12986721 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0C293C6FA90 for ; Fri, 23 Sep 2022 15:56:12 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231965AbiIWP4J (ORCPT ); Fri, 23 Sep 2022 11:56:09 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40124 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231837AbiIWP4D (ORCPT ); Fri, 23 Sep 2022 11:56:03 -0400 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 55C4F146FAB for ; Fri, 23 Sep 2022 08:56:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1663948561; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=f1rD50vdBgPCmvLBNPOPtby9Vrss+ZY8l15m1i9q/gc=; b=WOPM9PaW3S982zbzb4h8mOV835bdRI0ynZvkCvZeDDScYdLBbvSf+AJQDIHH2TnaKXTc+0 D27faiwbCgpDGzkCr3kUmMNE0EOMEvAnckr5wtIRZXqz5+AxFtPH2wzNR6X0GhjjL7IRi1 wVAaxkWEsqeYpJmYj8Lo+/opkfhHl1g= Received: from mail-wr1-f70.google.com (mail-wr1-f70.google.com [209.85.221.70]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_128_GCM_SHA256) id us-mta-519-FRW2emSDM9CGkYKLP6H07A-1; Fri, 23 Sep 2022 11:55:59 -0400 X-MC-Unique: FRW2emSDM9CGkYKLP6H07A-1 Received: by mail-wr1-f70.google.com with SMTP id c21-20020adfa315000000b0022adc2a2edaso130303wrb.0 for ; Fri, 23 Sep 2022 08:55:59 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date; bh=f1rD50vdBgPCmvLBNPOPtby9Vrss+ZY8l15m1i9q/gc=; b=ykzT5QhRc1qqZq35QL1CrxrTZpNyZANzkgYGgqbQCFTUpCv8OqlXxDCrtsM4R7HsNd EFeSMRcg0X3JRbQZfHoGdRJBNyCC3l/bT6BaW+npZCF8EqWxp1bkgavWee1KR1cRVo68 4RFG0BBSO1Kuc1TbWHpD0k5dWyd4mYrwpTMJufJCIv7zLbRTyiN27IzgefDjGW5uToPX muF/ThgpDsEKkJFScmbwQ/AWUpYdtuChXjCoLr/l5mgD5oIwB3PluhHFvQF89cv1y9Di sLqwgdrhNJ9QqdReZ7XoNfQzsdfLL1IOt2JY6gUzo0KKFqAlMGH3C9otU7VxIMiW2g/9 KRyw== X-Gm-Message-State: ACrzQf069MXlPIGmb7mm1t7n6jnSaETe8eT//2aabpYCUFej6ZfdePSG 9WJ9RMLtXLFsDUDrvLGCip0gTRa/YIyK6duFf8fB9T60vQTh0Oq09Y5v0etrPxFYh5D/DCoS5q4 J7o2sJ97uxK5vSG2rntQf3w== X-Received: by 2002:a5d:610a:0:b0:22a:2a9d:880 with SMTP id v10-20020a5d610a000000b0022a2a9d0880mr5436956wrt.22.1663948558614; Fri, 23 Sep 2022 08:55:58 -0700 (PDT) X-Google-Smtp-Source: AMsMyM7Qp9zZ4NuzIci/DpAgTQiB1AsWNZvgRAQooi24eczibkgAWqnA8TmIYsbQfk6ENbU93C431A== X-Received: by 2002:a5d:610a:0:b0:22a:2a9d:880 with SMTP id v10-20020a5d610a000000b0022a2a9d0880mr5436937wrt.22.1663948558409; Fri, 23 Sep 2022 08:55:58 -0700 (PDT) Received: from vschneid.remote.csb ([185.11.37.247]) by smtp.gmail.com with ESMTPSA id i7-20020a5d5587000000b0021badf3cb26sm9055429wrv.63.2022.09.23.08.55.57 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 23 Sep 2022 08:55:57 -0700 (PDT) From: Valentin Schneider To: netdev@vger.kernel.org, linux-rdma@vger.kernel.org, linux-kernel@vger.kernel.org Cc: Saeed Mahameed , Leon Romanovsky , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Yury Norov , Andy Shevchenko , Rasmus Villemoes , Ingo Molnar , Peter Zijlstra , Vincent Guittot , Dietmar Eggemann , Steven Rostedt , Mel Gorman , Greg Kroah-Hartman , Heiko Carstens , Tony Luck , Jonathan Cameron , Gal Pressman , Tariq Toukan , Jesse Brandeburg Subject: [PATCH v4 2/7] cpumask: Introduce for_each_cpu_andnot() Date: Fri, 23 Sep 2022 16:55:37 +0100 Message-Id: <20220923155542.1212814-1-vschneid@redhat.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20220923132527.1001870-1-vschneid@redhat.com> References: <20220923132527.1001870-1-vschneid@redhat.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org for_each_cpu_and() is very convenient as it saves having to allocate a temporary cpumask to store the result of cpumask_and(). The same issue applies to cpumask_andnot() which doesn't actually need temporary storage for iteration purposes. Following what has been done for for_each_cpu_and(), introduce for_each_cpu_andnot(). Signed-off-by: Valentin Schneider --- include/linux/cpumask.h | 39 +++++++++++++++++++++++++++++++++++++++ 1 file changed, 39 insertions(+) diff --git a/include/linux/cpumask.h b/include/linux/cpumask.h index 1b442fb2001f..4c69e338bb8c 100644 --- a/include/linux/cpumask.h +++ b/include/linux/cpumask.h @@ -238,6 +238,25 @@ unsigned int cpumask_next_and(int n, const struct cpumask *src1p, nr_cpumask_bits, n + 1); } +/** + * cpumask_next_andnot - get the next cpu in *src1p & ~*src2p + * @n: the cpu prior to the place to search (ie. return will be > @n) + * @src1p: the first cpumask pointer + * @src2p: the second cpumask pointer + * + * Returns >= nr_cpu_ids if no further cpus set in *src1p & ~*src2p + */ +static inline +unsigned int cpumask_next_andnot(int n, const struct cpumask *src1p, + const struct cpumask *src2p) +{ + /* -1 is a legal arg here. */ + if (n != -1) + cpumask_check(n); + return find_next_andnot_bit(cpumask_bits(src1p), cpumask_bits(src2p), + nr_cpumask_bits, n + 1); +} + /** * for_each_cpu - iterate over every cpu in a mask * @cpu: the (optionally unsigned) integer iterator @@ -317,6 +336,26 @@ unsigned int __pure cpumask_next_wrap(int n, const struct cpumask *mask, int sta (cpu) = cpumask_next_and((cpu), (mask1), (mask2)), \ (cpu) < nr_cpu_ids;) +/** + * for_each_cpu_andnot - iterate over every cpu present in one mask, excluding + * those present in another. + * @cpu: the (optionally unsigned) integer iterator + * @mask1: the first cpumask pointer + * @mask2: the second cpumask pointer + * + * This saves a temporary CPU mask in many places. It is equivalent to: + * struct cpumask tmp; + * cpumask_andnot(&tmp, &mask1, &mask2); + * for_each_cpu(cpu, &tmp) + * ... + * + * After the loop, cpu is >= nr_cpu_ids. + */ +#define for_each_cpu_andnot(cpu, mask1, mask2) \ + for ((cpu) = -1; \ + (cpu) = cpumask_next_andnot((cpu), (mask1), (mask2)), \ + (cpu) < nr_cpu_ids;) + /** * cpumask_any_but - return a "random" in a cpumask, but not this one. * @mask: the cpumask to search From patchwork Fri Sep 23 15:55:38 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Valentin Schneider X-Patchwork-Id: 12986720 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id D7062C6FA82 for ; Fri, 23 Sep 2022 15:56:10 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231851AbiIWP4I (ORCPT ); Fri, 23 Sep 2022 11:56:08 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40234 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231965AbiIWP4G (ORCPT ); Fri, 23 Sep 2022 11:56:06 -0400 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C6F79147687 for ; Fri, 23 Sep 2022 08:56:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1663948563; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=vikRyGRx44R+pUt2VG/nhewaVPQwv7B4yL3TDMhKXBE=; b=YVVABy+4r8lcGN1XtNPSh7WJ4Dksxz0mD1VwqJf81oOIiMJDAe9+qNjO0as2I1MwX1ydYc amgyO9Ypo3M4sF4Ay8QJfEEcH0H0LxdSiUyduZMMbdlJxsUV3ahSGzmP63TEjyxzYlWoGd 8EL73hatbuGLYNfWnfnwvdUKx6kBETU= Received: from mail-wr1-f72.google.com (mail-wr1-f72.google.com [209.85.221.72]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_128_GCM_SHA256) id us-mta-112-kg5ab3cMPmq_q4kQ-IeEHQ-1; Fri, 23 Sep 2022 11:56:01 -0400 X-MC-Unique: kg5ab3cMPmq_q4kQ-IeEHQ-1 Received: by mail-wr1-f72.google.com with SMTP id s5-20020adfa285000000b0022ad5c2771cso107109wra.18 for ; Fri, 23 Sep 2022 08:56:01 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date; bh=vikRyGRx44R+pUt2VG/nhewaVPQwv7B4yL3TDMhKXBE=; b=3A0fpKWO/abax9kK+fySGIMX3QhyuPXvQtPBDsKzuwMMznYuE5sZoUpHhLnepp2XT1 50Eab15x8nYie+kozznD2jYbsFJNRRReXKJfU54HaWTXnmcNfK3tZNiWcXw6d/kRnO1R ZvyT7Ee+Wv4NfHK3A/RkFMrIUiRCkP04iqNKvYnpys1ERfGIqcF5MO3ZLn8t55aGFmbv qtxYoWVOZw4BFEkG3BYsVZtygu+it8ymRZatVjD5JslpK+M1YAuzKdUfB7hhFhl/2YET O7exzrVy0g7WwwhgTFZf+6FTXEPX2U78UxVnL3BkZivptOO+1XkQKJRm98MWLInJTWIi taPA== X-Gm-Message-State: ACrzQf0nYCT5R+u5MDFuc3uJlBw+yIg95FGhSJr4sF1PfjQdfPhqu8zy BIlME5qTZ86UY77G3LP1sMtgFIXzHpKmsf4AoVylVE0yu2vFBtGCM+SwI+WaB8Mt4DqKTCd0NrB gYHR59nAuGVTAuFW+ZHsXWg== X-Received: by 2002:a05:6000:101:b0:228:de40:8898 with SMTP id o1-20020a056000010100b00228de408898mr5638919wrx.106.1663948560658; Fri, 23 Sep 2022 08:56:00 -0700 (PDT) X-Google-Smtp-Source: AMsMyM47DzwiN5PM9fuuQwSaApTf51dVQD8qwTfh15JBMFf32iFHSPpJD0mu7a8+o3i7y45MFlP0iA== X-Received: by 2002:a05:6000:101:b0:228:de40:8898 with SMTP id o1-20020a056000010100b00228de408898mr5638907wrx.106.1663948560504; Fri, 23 Sep 2022 08:56:00 -0700 (PDT) Received: from vschneid.remote.csb ([185.11.37.247]) by smtp.gmail.com with ESMTPSA id i7-20020a5d5587000000b0021badf3cb26sm9055429wrv.63.2022.09.23.08.55.58 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 23 Sep 2022 08:55:59 -0700 (PDT) From: Valentin Schneider To: netdev@vger.kernel.org, linux-rdma@vger.kernel.org, linux-kernel@vger.kernel.org Cc: Yury Norov , Saeed Mahameed , Leon Romanovsky , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Andy Shevchenko , Rasmus Villemoes , Ingo Molnar , Peter Zijlstra , Vincent Guittot , Dietmar Eggemann , Steven Rostedt , Mel Gorman , Greg Kroah-Hartman , Heiko Carstens , Tony Luck , Jonathan Cameron , Gal Pressman , Tariq Toukan , Jesse Brandeburg Subject: [PATCH v4 3/7] lib/test_cpumask: Add for_each_cpu_and(not) tests Date: Fri, 23 Sep 2022 16:55:38 +0100 Message-Id: <20220923155542.1212814-2-vschneid@redhat.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20220923132527.1001870-1-vschneid@redhat.com> References: <20220923132527.1001870-1-vschneid@redhat.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org Following the recent introduction of for_each_andnot(), add some tests to ensure for_each_cpu_and(not) results in the same as iterating over the result of cpumask_and(not)(). Suggested-by: Yury Norov Signed-off-by: Valentin Schneider --- lib/cpumask_kunit.c | 19 +++++++++++++++++++ 1 file changed, 19 insertions(+) diff --git a/lib/cpumask_kunit.c b/lib/cpumask_kunit.c index ecbeec72221e..d1fc6ece21f3 100644 --- a/lib/cpumask_kunit.c +++ b/lib/cpumask_kunit.c @@ -33,6 +33,19 @@ KUNIT_EXPECT_EQ_MSG((test), nr_cpu_ids - mask_weight, iter, MASK_MSG(mask)); \ } while (0) +#define EXPECT_FOR_EACH_CPU_OP_EQ(test, op, mask1, mask2) \ + do { \ + const cpumask_t *m1 = (mask1); \ + const cpumask_t *m2 = (mask2); \ + int weight; \ + int cpu, iter = 0; \ + cpumask_##op(&mask_tmp, m1, m2); \ + weight = cpumask_weight(&mask_tmp); \ + for_each_cpu_##op(cpu, mask1, mask2) \ + iter++; \ + KUNIT_EXPECT_EQ((test), weight, iter); \ + } while (0) + #define EXPECT_FOR_EACH_CPU_WRAP_EQ(test, mask) \ do { \ const cpumask_t *m = (mask); \ @@ -54,6 +67,7 @@ static cpumask_t mask_empty; static cpumask_t mask_all; +static cpumask_t mask_tmp; static void test_cpumask_weight(struct kunit *test) { @@ -101,10 +115,15 @@ static void test_cpumask_iterators(struct kunit *test) EXPECT_FOR_EACH_CPU_EQ(test, &mask_empty); EXPECT_FOR_EACH_CPU_NOT_EQ(test, &mask_empty); EXPECT_FOR_EACH_CPU_WRAP_EQ(test, &mask_empty); + EXPECT_FOR_EACH_CPU_OP_EQ(test, and, &mask_empty, &mask_empty); + EXPECT_FOR_EACH_CPU_OP_EQ(test, and, cpu_possible_mask, &mask_empty); + EXPECT_FOR_EACH_CPU_OP_EQ(test, andnot, &mask_empty, &mask_empty); EXPECT_FOR_EACH_CPU_EQ(test, cpu_possible_mask); EXPECT_FOR_EACH_CPU_NOT_EQ(test, cpu_possible_mask); EXPECT_FOR_EACH_CPU_WRAP_EQ(test, cpu_possible_mask); + EXPECT_FOR_EACH_CPU_OP_EQ(test, and, cpu_possible_mask, cpu_possible_mask); + EXPECT_FOR_EACH_CPU_OP_EQ(test, andnot, cpu_possible_mask, &mask_empty); } static void test_cpumask_iterators_builtin(struct kunit *test) From patchwork Fri Sep 23 15:55:39 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Valentin Schneider X-Patchwork-Id: 12986722 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 727FFC6FA8B for ; Fri, 23 Sep 2022 15:56:21 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232566AbiIWP4U (ORCPT ); Fri, 23 Sep 2022 11:56:20 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40296 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231693AbiIWP4H (ORCPT ); Fri, 23 Sep 2022 11:56:07 -0400 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A16AF14768D for ; Fri, 23 Sep 2022 08:56:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1663948564; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=9qKy0mrwb1ZaeVcnoJdtpT0vU2IyZpITeq2cNcv3elU=; b=LurPLWs6LsWsaRAZ2q62zl2TAJ78YCvwlgwJW4SxDxBp4BjVo0OV8NJK07TZrDjr5rbp3d e7yhlA/SsJ33l5OvrxlJxQ/UwYMq5qJHDmKAoI3+6BMjYUH0sGkFwF09Vt5x44kQcujYvi kizxAN4Wwya0m7BxQOLxEOpKxNFCEcw= Received: from mail-wr1-f71.google.com (mail-wr1-f71.google.com [209.85.221.71]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_128_GCM_SHA256) id us-mta-500-7IYWFg0tN1OHYEWZ3HT01g-1; Fri, 23 Sep 2022 11:56:03 -0400 X-MC-Unique: 7IYWFg0tN1OHYEWZ3HT01g-1 Received: by mail-wr1-f71.google.com with SMTP id l5-20020adfa385000000b0022a482f8285so119934wrb.5 for ; Fri, 23 Sep 2022 08:56:03 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date; bh=9qKy0mrwb1ZaeVcnoJdtpT0vU2IyZpITeq2cNcv3elU=; b=C4NEoS3xAFNtoW52Yp6jPRajncQ17vDEKso6frf4zkdBUTPsV2JZNP8JGZlq7gcYNj CJNp2UI2yEnWObTrvpuz8SjiHTuSpu/5WZVDhIzOyP6+wu5arJaMo0aj1GxvjLAtM2sk iXaKm5E5EARTNFMQnnWwjKLkYd+GOKhS2Ag8TBllCRrmvcCWxjFx+d+LJp5znLGm/xzz XB60m/qn+8qWv88R9VGelE51DuIuGDDCdadFUuYq7Ma8KNCcYpm1BMX6Bhw+RRaR9SEI LkV44jq6byx+16tyydIzCz6RaHg9BrPkWQorkZIumbgdj5tVmdSiEA1xp/t7YfU7IOK4 rQkQ== X-Gm-Message-State: ACrzQf3cWh4fNDRqDqihOQmaB098uFPsFjENdQ+ALyK2bpbXjYtdpUjL GpZmjHF1ezSMeBJe+N9uI/Sz92aBD/yLdILnPCqY4ygauvxbL5S/lW/WQa8f+UwOxHpBPVBcVJB 9cKH3Nj1NVw0IbZO8RO10+Q== X-Received: by 2002:a5d:62c8:0:b0:228:67d2:797b with SMTP id o8-20020a5d62c8000000b0022867d2797bmr5570910wrv.462.1663948562602; Fri, 23 Sep 2022 08:56:02 -0700 (PDT) X-Google-Smtp-Source: AMsMyM4aPbc4vrsR5Vfq6YATZvPVX52Oz7CjIB9xYibgKcC2wNByVZXJ2pv9DnxuyPBkjc5vbnX2Bw== X-Received: by 2002:a5d:62c8:0:b0:228:67d2:797b with SMTP id o8-20020a5d62c8000000b0022867d2797bmr5570899wrv.462.1663948562369; Fri, 23 Sep 2022 08:56:02 -0700 (PDT) Received: from vschneid.remote.csb ([185.11.37.247]) by smtp.gmail.com with ESMTPSA id i7-20020a5d5587000000b0021badf3cb26sm9055429wrv.63.2022.09.23.08.56.00 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 23 Sep 2022 08:56:01 -0700 (PDT) From: Valentin Schneider To: netdev@vger.kernel.org, linux-rdma@vger.kernel.org, linux-kernel@vger.kernel.org Cc: Yury Norov , Saeed Mahameed , Leon Romanovsky , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Andy Shevchenko , Rasmus Villemoes , Ingo Molnar , Peter Zijlstra , Vincent Guittot , Dietmar Eggemann , Steven Rostedt , Mel Gorman , Greg Kroah-Hartman , Heiko Carstens , Tony Luck , Jonathan Cameron , Gal Pressman , Tariq Toukan , Jesse Brandeburg Subject: [PATCH v4 4/7] sched/core: Merge cpumask_andnot()+for_each_cpu() into for_each_cpu_andnot() Date: Fri, 23 Sep 2022 16:55:39 +0100 Message-Id: <20220923155542.1212814-3-vschneid@redhat.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20220923132527.1001870-1-vschneid@redhat.com> References: <20220923132527.1001870-1-vschneid@redhat.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org This removes the second use of the sched_core_mask temporary mask. Suggested-by: Yury Norov Signed-off-by: Valentin Schneider --- kernel/sched/core.c | 5 +---- 1 file changed, 1 insertion(+), 4 deletions(-) diff --git a/kernel/sched/core.c b/kernel/sched/core.c index ee28253c9ac0..b4c3112b0095 100644 --- a/kernel/sched/core.c +++ b/kernel/sched/core.c @@ -360,10 +360,7 @@ static void __sched_core_flip(bool enabled) /* * Toggle the offline CPUs. */ - cpumask_copy(&sched_core_mask, cpu_possible_mask); - cpumask_andnot(&sched_core_mask, &sched_core_mask, cpu_online_mask); - - for_each_cpu(cpu, &sched_core_mask) + for_each_cpu_andnot(cpu, cpu_possible_mask, cpu_online_mask) cpu_rq(cpu)->core_enabled = enabled; cpus_read_unlock(); From patchwork Fri Sep 23 15:55:40 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Valentin Schneider X-Patchwork-Id: 12986723 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 25E54C6FA82 for ; Fri, 23 Sep 2022 15:56:34 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232743AbiIWP4b (ORCPT ); Fri, 23 Sep 2022 11:56:31 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40354 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232331AbiIWP4I (ORCPT ); Fri, 23 Sep 2022 11:56:08 -0400 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B8A51146FBE for ; Fri, 23 Sep 2022 08:56:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1663948567; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=kHBrvEPvwtrxok/newbrf2agVKLCITkG2Qf6CtamOLw=; b=TbcIWgCCgTefQjXAvmu4kZ46SwacUqEk+mk1jLiOOtE9UfbglZ+8FT6aMDukm6MrZDFsxI FCBSWP+4N5MM3LS+8js53z45fU6NPC1ZAhX1Ycz0VfXC/Yo4+oCkIXYW1FISH6Bx3MC7oP gGHt28PaR5m5ph4Dgal9LUDetQoE/y0= Received: from mail-wr1-f70.google.com (mail-wr1-f70.google.com [209.85.221.70]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_128_GCM_SHA256) id us-mta-232-qelISEC1MlO-kESRKZ-r9A-1; Fri, 23 Sep 2022 11:56:06 -0400 X-MC-Unique: qelISEC1MlO-kESRKZ-r9A-1 Received: by mail-wr1-f70.google.com with SMTP id a13-20020adfbc4d000000b00228631ddfafso117138wrh.9 for ; Fri, 23 Sep 2022 08:56:05 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date; bh=kHBrvEPvwtrxok/newbrf2agVKLCITkG2Qf6CtamOLw=; b=yPh2s6RkxVr76byF7o1daVIU5KvzNjpSKC5+LmAJz3ouFMYRUtM5Q5enLLykGNhHvd 0wSYu2u0anh2Nrh5dj7HOi3kiGXeK7gob73r50ypAgfyfWcy8OhpeTr/gmyneMInhuxd dCk+IliRr60id9c8OyySpbiZnusENFG/e4SBkMD7S7anrbycoKmY2pXE7051YE1oq0Z7 5+IvMMMVG4pnlpq05lQWXPNiv8d17u6KU/H4ociStwrbW+2RvmfpdLXmDmJQSYTAmkbh 2VGrTZcQkOBzRJ27clwbqUVSCeDH3PH0jbSN7h5BbQ1hL/KSimpvFW2rdaYRMziol6E7 bnkQ== X-Gm-Message-State: ACrzQf2MuLWFfoXCA14UchtiwdEvVpNXXB7IRjGkvSacE+WZdqy2Rmuw 964Lgm5MRnRZ0tq0tz1YoiUC6rMbhPUV0ojWoPhK3AufY0MyTwz1l5JRvF1t7mu2mZl2Set5pm2 aYXlUSHjXEL0cjC+jMCOFdw== X-Received: by 2002:a05:600c:1c22:b0:3b4:b2bc:15e4 with SMTP id j34-20020a05600c1c2200b003b4b2bc15e4mr6572332wms.69.1663948564543; Fri, 23 Sep 2022 08:56:04 -0700 (PDT) X-Google-Smtp-Source: AMsMyM4Cc+v1ZHKhGIhntGRkSRg5WAmD2mSxUj/B+KSYr9+DHgV96QzjwDqWCHxzt9g3noPgOYYFEA== X-Received: by 2002:a05:600c:1c22:b0:3b4:b2bc:15e4 with SMTP id j34-20020a05600c1c2200b003b4b2bc15e4mr6572323wms.69.1663948564353; Fri, 23 Sep 2022 08:56:04 -0700 (PDT) Received: from vschneid.remote.csb ([185.11.37.247]) by smtp.gmail.com with ESMTPSA id i7-20020a5d5587000000b0021badf3cb26sm9055429wrv.63.2022.09.23.08.56.02 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 23 Sep 2022 08:56:03 -0700 (PDT) From: Valentin Schneider To: netdev@vger.kernel.org, linux-rdma@vger.kernel.org, linux-kernel@vger.kernel.org Cc: Saeed Mahameed , Leon Romanovsky , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Yury Norov , Andy Shevchenko , Rasmus Villemoes , Ingo Molnar , Peter Zijlstra , Vincent Guittot , Dietmar Eggemann , Steven Rostedt , Mel Gorman , Greg Kroah-Hartman , Heiko Carstens , Tony Luck , Jonathan Cameron , Gal Pressman , Tariq Toukan , Jesse Brandeburg Subject: [PATCH v4 5/7] sched/topology: Introduce sched_numa_hop_mask() Date: Fri, 23 Sep 2022 16:55:40 +0100 Message-Id: <20220923155542.1212814-4-vschneid@redhat.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20220923132527.1001870-1-vschneid@redhat.com> References: <20220923132527.1001870-1-vschneid@redhat.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org Tariq has pointed out that drivers allocating IRQ vectors would benefit from having smarter NUMA-awareness - cpumask_local_spread() only knows about the local node and everything outside is in the same bucket. sched_domains_numa_masks is pretty much what we want to hand out (a cpumask of CPUs reachable within a given distance budget), introduce sched_numa_hop_mask() to export those cpumasks. Link: http://lore.kernel.org/r/20220728191203.4055-1-tariqt@nvidia.com Signed-off-by: Valentin Schneider --- include/linux/topology.h | 12 ++++++++++++ kernel/sched/topology.c | 31 +++++++++++++++++++++++++++++++ 2 files changed, 43 insertions(+) diff --git a/include/linux/topology.h b/include/linux/topology.h index 4564faafd0e1..3e91ae6d0ad5 100644 --- a/include/linux/topology.h +++ b/include/linux/topology.h @@ -245,5 +245,17 @@ static inline const struct cpumask *cpu_cpu_mask(int cpu) return cpumask_of_node(cpu_to_node(cpu)); } +#ifdef CONFIG_NUMA +extern const struct cpumask *sched_numa_hop_mask(int node, int hops); +#else +static inline const struct cpumask *sched_numa_hop_mask(int node, int hops) +{ + if (node == NUMA_NO_NODE && !hops) + return cpu_online_mask; + + return ERR_PTR(-EOPNOTSUPP); +} +#endif /* CONFIG_NUMA */ + #endif /* _LINUX_TOPOLOGY_H */ diff --git a/kernel/sched/topology.c b/kernel/sched/topology.c index 8739c2a5a54e..ee77706603c0 100644 --- a/kernel/sched/topology.c +++ b/kernel/sched/topology.c @@ -2067,6 +2067,37 @@ int sched_numa_find_closest(const struct cpumask *cpus, int cpu) return found; } +/** + * sched_numa_hop_mask() - Get the cpumask of CPUs at most @hops hops away. + * @node: The node to count hops from. + * @hops: Include CPUs up to that many hops away. 0 means local node. + * + * Requires rcu_lock to be held. Returned cpumask is only valid within that + * read-side section, copy it if required beyond that. + * + * Note that not all hops are equal in distance; see sched_init_numa() for how + * distances and masks are handled. + * + * Also note that this is a reflection of sched_domains_numa_masks, which may change + * during the lifetime of the system (offline nodes are taken out of the masks). + */ +const struct cpumask *sched_numa_hop_mask(int node, int hops) +{ + struct cpumask ***masks = rcu_dereference(sched_domains_numa_masks); + + if (node == NUMA_NO_NODE && !hops) + return cpu_online_mask; + + if (node >= nr_node_ids || hops >= sched_domains_numa_levels) + return ERR_PTR(-EINVAL); + + if (!masks) + return NULL; + + return masks[hops][node]; +} +EXPORT_SYMBOL_GPL(sched_numa_hop_mask); + #endif /* CONFIG_NUMA */ static int __sdt_alloc(const struct cpumask *cpu_map) From patchwork Fri Sep 23 15:55:41 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Valentin Schneider X-Patchwork-Id: 12986724 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id C0EEBECAAD8 for ; Fri, 23 Sep 2022 15:56:38 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232180AbiIWP4f (ORCPT ); Fri, 23 Sep 2022 11:56:35 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40798 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232550AbiIWP4T (ORCPT ); Fri, 23 Sep 2022 11:56:19 -0400 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C638D147693 for ; Fri, 23 Sep 2022 08:56:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1663948569; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=+vdqLCntBqaxf8FADpHkJLlOi77ldZqexu1XSAK/G7U=; b=Nqix3/gqX6Den5SbB7mbrnYbYd3jhwGksXezS92QMiXcRDBxCPBbqqjt5lRphWeTEPfxW6 HFB6ysQrKP0FT683sIxzYzlNWwtNS1g63pIJ9aroroY5PsHHR2j4/rd5Oegcc+/gqAR/uZ qpVE4+Scmk6fyGtP3mGIzLEbK83V1K8= Received: from mail-wm1-f70.google.com (mail-wm1-f70.google.com [209.85.128.70]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_128_GCM_SHA256) id us-mta-301-O1NOXulyPgmpNlfzVP4J6g-1; Fri, 23 Sep 2022 11:56:07 -0400 X-MC-Unique: O1NOXulyPgmpNlfzVP4J6g-1 Received: by mail-wm1-f70.google.com with SMTP id b5-20020a05600c4e0500b003b499f99aceso2921749wmq.1 for ; Fri, 23 Sep 2022 08:56:07 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date; bh=+vdqLCntBqaxf8FADpHkJLlOi77ldZqexu1XSAK/G7U=; b=lCt7bm6d9CbyHMPW8I6HQJlM9TAkwaDoTsHWkA1wFoZ1KLeK+sM7RbWkA9tkgXpsUw jCeiSMsPKBlakUyIn1UU/JAurdxfuCRG5L24qwIKx8WaHj2191vtZ3mo4I+DZmURA5sL IHIAYALRDv7cbk8OnX38q102RVk4OM8C6QGTMNJfL/7taC/RuE1OLgXnSK+hckZ3x4dF YlIQek67INNVYvRdcwqp+ILdNZl3J4kygy0tgHRWV5GPKxnIi+yNDwXNSBiXT1HXLvq+ ErS5A//grJ8JyKQfQEyBZkybw3MgARA/9sxq4ow/U5jJbcXgxNWlnfK0jUcAmR1gv1sZ rFwA== X-Gm-Message-State: ACrzQf0iJb5pr+7KHIDkcKSTFq5BIy8xzmi058+lJxr2J4vLdQnVXpKq 4o3VxMy1HzD44gP+2M1xlyaQxV4yrkJgpz+1sqclrv48l1UccwZw4qf7ms8nyKHqf1EZhPEpQmk PlkIiUOPy4NdRBtcJ1FJxdA== X-Received: by 2002:a7b:c457:0:b0:3b4:689d:b408 with SMTP id l23-20020a7bc457000000b003b4689db408mr13584478wmi.22.1663948566652; Fri, 23 Sep 2022 08:56:06 -0700 (PDT) X-Google-Smtp-Source: AMsMyM7tQjllyoAjaE0hgO29ZvO5RUX4Pw8HfukZFiwY7GUlODEt13p9thEkh19Yx2eyohjKDv4MTg== X-Received: by 2002:a7b:c457:0:b0:3b4:689d:b408 with SMTP id l23-20020a7bc457000000b003b4689db408mr13584449wmi.22.1663948566427; Fri, 23 Sep 2022 08:56:06 -0700 (PDT) Received: from vschneid.remote.csb ([185.11.37.247]) by smtp.gmail.com with ESMTPSA id i7-20020a5d5587000000b0021badf3cb26sm9055429wrv.63.2022.09.23.08.56.04 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 23 Sep 2022 08:56:05 -0700 (PDT) From: Valentin Schneider To: netdev@vger.kernel.org, linux-rdma@vger.kernel.org, linux-kernel@vger.kernel.org Cc: Saeed Mahameed , Leon Romanovsky , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Yury Norov , Andy Shevchenko , Rasmus Villemoes , Ingo Molnar , Peter Zijlstra , Vincent Guittot , Dietmar Eggemann , Steven Rostedt , Mel Gorman , Greg Kroah-Hartman , Heiko Carstens , Tony Luck , Jonathan Cameron , Gal Pressman , Tariq Toukan , Jesse Brandeburg Subject: [PATCH v4 6/7] sched/topology: Introduce for_each_numa_hop_cpu() Date: Fri, 23 Sep 2022 16:55:41 +0100 Message-Id: <20220923155542.1212814-5-vschneid@redhat.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20220923132527.1001870-1-vschneid@redhat.com> References: <20220923132527.1001870-1-vschneid@redhat.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org The recently introduced sched_numa_hop_mask() exposes cpumasks of CPUs reachable within a given distance budget, but this means each successive cpumask is a superset of the previous one. Code wanting to allocate one item per CPU (e.g. IRQs) at increasing distances would thus need to allocate a temporary cpumask to note which CPUs have already been visited. This can be prevented by leveraging for_each_cpu_andnot() - package all that logic into one ugl^D fancy macro. Signed-off-by: Valentin Schneider --- include/linux/topology.h | 37 +++++++++++++++++++++++++++++++++++++ 1 file changed, 37 insertions(+) diff --git a/include/linux/topology.h b/include/linux/topology.h index 3e91ae6d0ad5..7aa7e6a4c739 100644 --- a/include/linux/topology.h +++ b/include/linux/topology.h @@ -257,5 +257,42 @@ static inline const struct cpumask *sched_numa_hop_mask(int node, int hops) } #endif /* CONFIG_NUMA */ +/** + * for_each_numa_hop_cpu - iterate over CPUs by increasing NUMA distance, + * starting from a given node. + * @cpu: the iteration variable. + * @node: the NUMA node to start the search from. + * + * Requires rcu_lock to be held. + * Careful: this is a double loop, 'break' won't work as expected. + * + * + * Implementation notes: + * + * Providing it is valid, the mask returned by + * sched_numa_hop_mask(node, hops+1) + * is a superset of the one returned by + * sched_numa_hop_mask(node, hops) + * which may not be that useful for drivers that try to spread things out and + * want to visit a CPU not more than once. + * + * To accommodate for that, we use for_each_cpu_andnot() to iterate over the cpus + * of sched_numa_hop_mask(node, hops+1) with the CPUs of + * sched_numa_hop_mask(node, hops) removed, IOW we only iterate over CPUs + * a given distance away (rather than *up to* a given distance). + * + * hops=0 forces us to play silly games: we pass cpu_none_mask to + * for_each_cpu_andnot(), which turns it into for_each_cpu(). + */ +#define for_each_numa_hop_cpu(cpu, node) \ + for (struct { const struct cpumask *curr, *prev; int hops; } __v = \ + { sched_numa_hop_mask(node, 0), NULL, 0 }; \ + !IS_ERR_OR_NULL(__v.curr); \ + __v.hops++, \ + __v.prev = __v.curr, \ + __v.curr = sched_numa_hop_mask(node, __v.hops)) \ + for_each_cpu_andnot(cpu, \ + __v.curr, \ + __v.hops ? __v.prev : cpu_none_mask) #endif /* _LINUX_TOPOLOGY_H */ From patchwork Fri Sep 23 15:55:42 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Valentin Schneider X-Patchwork-Id: 12986738 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 71361C6FA82 for ; Fri, 23 Sep 2022 15:57:16 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232891AbiIWP5O (ORCPT ); Fri, 23 Sep 2022 11:57:14 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41396 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232797AbiIWP4f (ORCPT ); Fri, 23 Sep 2022 11:56:35 -0400 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 436D21476BD for ; Fri, 23 Sep 2022 08:56:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1663948572; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=95/R3vPHmElpIlZPV2xBiNtrbsY/TCs3HLgfyeLaW+0=; b=ZVX51Jpts3qsV3/st3ilYege9tHrAcPvf4+1M958k0v9K5WDUqff3i2lXOX3N6LBFcmjsl 5wVRju5DJ9A77a77EzeTXFaxTIqb64wGlEdiS/W+/1mtAfcPQON9zPS4Jo5aUvLx3B9LAQ jmZ6gPRre6wdgKV1U7WJNfVQdOHCKN0= Received: from mail-wr1-f72.google.com (mail-wr1-f72.google.com [209.85.221.72]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_128_GCM_SHA256) id us-mta-365-HC7Vp1BlNf298fg1wzC4Yw-1; Fri, 23 Sep 2022 11:56:10 -0400 X-MC-Unique: HC7Vp1BlNf298fg1wzC4Yw-1 Received: by mail-wr1-f72.google.com with SMTP id e2-20020adf9bc2000000b0022ae4ea0afcso117231wrc.8 for ; Fri, 23 Sep 2022 08:56:10 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date; bh=95/R3vPHmElpIlZPV2xBiNtrbsY/TCs3HLgfyeLaW+0=; b=2CKD0IQVQR1wYRjRN57EPxFXaQ+Y+G6Hvps97I/sorqPjABsBjH/bIOg7kbW5cd3Oy Ef08PYhHnhpNTpROxiuUDNtbcki+ZkYXc+tKvfU50ujCCRvPsbQsonCVS2Sa2me+h2VJ 2O3jRWLw2wLClOvLBzxVNd6WmDBBXPTfcBluuw6g+In3EeDISHVFRdiR1gjsNhlpEpQW OcU6BRYf8s3GaPAnoet1gwqNLszhJK0k2LK+gv7JxT6VowPaf2bJUghWdif3jFYs/s45 j7UrTDDfYIfSkdahHLL/yTRjlWY+FaXHtCXzZUPrfOoSqj15PG8YxJ9JLGgpZ2Vy2WjV EaDQ== X-Gm-Message-State: ACrzQf11ZiWiuR/iKpT37Y+/dBnlEQ0s+eHNwF4qCkkcnoI5BQyHz2Kx yHW+ic9p0DqMX7NvFxxwFSebUdnSoAXHAkDyfMJ8X4xszNxtAQC3XjjbUa80Jttta82olh+QbBM RJB0p54rDBNK6YDseVJaHgA== X-Received: by 2002:a5d:64a9:0:b0:22a:c3c3:4943 with SMTP id m9-20020a5d64a9000000b0022ac3c34943mr5966246wrp.655.1663948569007; Fri, 23 Sep 2022 08:56:09 -0700 (PDT) X-Google-Smtp-Source: AMsMyM4lJ/RLQiyAJ+NLTaw/AOAUvm3joxLkq/We+1pWqJZnNWVkzWqQSRNQ9NXeSCyxCxJP/HK4tA== X-Received: by 2002:a5d:64a9:0:b0:22a:c3c3:4943 with SMTP id m9-20020a5d64a9000000b0022ac3c34943mr5966220wrp.655.1663948568557; Fri, 23 Sep 2022 08:56:08 -0700 (PDT) Received: from vschneid.remote.csb ([185.11.37.247]) by smtp.gmail.com with ESMTPSA id i7-20020a5d5587000000b0021badf3cb26sm9055429wrv.63.2022.09.23.08.56.06 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 23 Sep 2022 08:56:07 -0700 (PDT) From: Valentin Schneider To: netdev@vger.kernel.org, linux-rdma@vger.kernel.org, linux-kernel@vger.kernel.org Cc: Tariq Toukan , Saeed Mahameed , Leon Romanovsky , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Yury Norov , Andy Shevchenko , Rasmus Villemoes , Ingo Molnar , Peter Zijlstra , Vincent Guittot , Dietmar Eggemann , Steven Rostedt , Mel Gorman , Greg Kroah-Hartman , Heiko Carstens , Tony Luck , Jonathan Cameron , Gal Pressman , Jesse Brandeburg Subject: [PATCH v4 7/7] net/mlx5e: Improve remote NUMA preferences used for the IRQ affinity hints Date: Fri, 23 Sep 2022 16:55:42 +0100 Message-Id: <20220923155542.1212814-6-vschneid@redhat.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20220923132527.1001870-1-vschneid@redhat.com> References: <20220923132527.1001870-1-vschneid@redhat.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org From: Tariq Toukan In the IRQ affinity hints, replace the binary NUMA preference (local / remote) with the improved for_each_numa_hop_cpu() API that minds the actual distances, so that remote NUMAs with short distance are preferred over farther ones. This has significant performance implications when using NUMA-aware allocated memory (follow [1] and derivatives for example). [1] drivers/net/ethernet/mellanox/mlx5/core/en_main.c :: mlx5e_open_channel() int cpu = cpumask_first(mlx5_comp_irq_get_affinity_mask(priv->mdev, ix)); Performance tests: TCP multi-stream, using 16 iperf3 instances pinned to 16 cores (with aRFS on). Active cores: 64,65,72,73,80,81,88,89,96,97,104,105,112,113,120,121 +-------------------------+-----------+------------------+------------------+ | | BW (Gbps) | TX side CPU util | RX side CPU util | +-------------------------+-----------+------------------+------------------+ | Baseline | 52.3 | 6.4 % | 17.9 % | +-------------------------+-----------+------------------+------------------+ | Applied on TX side only | 52.6 | 5.2 % | 18.5 % | +-------------------------+-----------+------------------+------------------+ | Applied on RX side only | 94.9 | 11.9 % | 27.2 % | +-------------------------+-----------+------------------+------------------+ | Applied on both sides | 95.1 | 8.4 % | 27.3 % | +-------------------------+-----------+------------------+------------------+ Bottleneck in RX side is released, reached linerate (~1.8x speedup). ~30% less cpu util on TX. * CPU util on active cores only. Setups details (similar for both sides): NIC: ConnectX6-DX dual port, 100 Gbps each. Single port used in the tests. $ lscpu Architecture: x86_64 CPU op-mode(s): 32-bit, 64-bit Byte Order: Little Endian CPU(s): 256 On-line CPU(s) list: 0-255 Thread(s) per core: 2 Core(s) per socket: 64 Socket(s): 2 NUMA node(s): 16 Vendor ID: AuthenticAMD CPU family: 25 Model: 1 Model name: AMD EPYC 7763 64-Core Processor Stepping: 1 CPU MHz: 2594.804 BogoMIPS: 4890.73 Virtualization: AMD-V L1d cache: 32K L1i cache: 32K L2 cache: 512K L3 cache: 32768K NUMA node0 CPU(s): 0-7,128-135 NUMA node1 CPU(s): 8-15,136-143 NUMA node2 CPU(s): 16-23,144-151 NUMA node3 CPU(s): 24-31,152-159 NUMA node4 CPU(s): 32-39,160-167 NUMA node5 CPU(s): 40-47,168-175 NUMA node6 CPU(s): 48-55,176-183 NUMA node7 CPU(s): 56-63,184-191 NUMA node8 CPU(s): 64-71,192-199 NUMA node9 CPU(s): 72-79,200-207 NUMA node10 CPU(s): 80-87,208-215 NUMA node11 CPU(s): 88-95,216-223 NUMA node12 CPU(s): 96-103,224-231 NUMA node13 CPU(s): 104-111,232-239 NUMA node14 CPU(s): 112-119,240-247 NUMA node15 CPU(s): 120-127,248-255 .. $ numactl -H .. node distances: node 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 0: 10 11 11 11 12 12 12 12 32 32 32 32 32 32 32 32 1: 11 10 11 11 12 12 12 12 32 32 32 32 32 32 32 32 2: 11 11 10 11 12 12 12 12 32 32 32 32 32 32 32 32 3: 11 11 11 10 12 12 12 12 32 32 32 32 32 32 32 32 4: 12 12 12 12 10 11 11 11 32 32 32 32 32 32 32 32 5: 12 12 12 12 11 10 11 11 32 32 32 32 32 32 32 32 6: 12 12 12 12 11 11 10 11 32 32 32 32 32 32 32 32 7: 12 12 12 12 11 11 11 10 32 32 32 32 32 32 32 32 8: 32 32 32 32 32 32 32 32 10 11 11 11 12 12 12 12 9: 32 32 32 32 32 32 32 32 11 10 11 11 12 12 12 12 10: 32 32 32 32 32 32 32 32 11 11 10 11 12 12 12 12 11: 32 32 32 32 32 32 32 32 11 11 11 10 12 12 12 12 12: 32 32 32 32 32 32 32 32 12 12 12 12 10 11 11 11 13: 32 32 32 32 32 32 32 32 12 12 12 12 11 10 11 11 14: 32 32 32 32 32 32 32 32 12 12 12 12 11 11 10 11 15: 32 32 32 32 32 32 32 32 12 12 12 12 11 11 11 10 $ cat /sys/class/net/ens5f0/device/numa_node 14 Affinity hints (127 IRQs): Before: 331: 00000000,00000000,00000000,00000000,00010000,00000000,00000000,00000000 332: 00000000,00000000,00000000,00000000,00020000,00000000,00000000,00000000 333: 00000000,00000000,00000000,00000000,00040000,00000000,00000000,00000000 334: 00000000,00000000,00000000,00000000,00080000,00000000,00000000,00000000 335: 00000000,00000000,00000000,00000000,00100000,00000000,00000000,00000000 336: 00000000,00000000,00000000,00000000,00200000,00000000,00000000,00000000 337: 00000000,00000000,00000000,00000000,00400000,00000000,00000000,00000000 338: 00000000,00000000,00000000,00000000,00800000,00000000,00000000,00000000 339: 00010000,00000000,00000000,00000000,00000000,00000000,00000000,00000000 340: 00020000,00000000,00000000,00000000,00000000,00000000,00000000,00000000 341: 00040000,00000000,00000000,00000000,00000000,00000000,00000000,00000000 342: 00080000,00000000,00000000,00000000,00000000,00000000,00000000,00000000 343: 00100000,00000000,00000000,00000000,00000000,00000000,00000000,00000000 344: 00200000,00000000,00000000,00000000,00000000,00000000,00000000,00000000 345: 00400000,00000000,00000000,00000000,00000000,00000000,00000000,00000000 346: 00800000,00000000,00000000,00000000,00000000,00000000,00000000,00000000 347: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000001 348: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000002 349: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000004 350: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000008 351: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000010 352: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000020 353: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000040 354: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000080 355: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000100 356: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000200 357: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000400 358: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000800 359: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00001000 360: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00002000 361: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00004000 362: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00008000 363: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00010000 364: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00020000 365: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00040000 366: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00080000 367: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00100000 368: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00200000 369: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00400000 370: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00800000 371: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,01000000 372: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,02000000 373: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,04000000 374: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,08000000 375: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,10000000 376: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,20000000 377: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,40000000 378: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,80000000 379: 00000000,00000000,00000000,00000000,00000000,00000000,00000001,00000000 380: 00000000,00000000,00000000,00000000,00000000,00000000,00000002,00000000 381: 00000000,00000000,00000000,00000000,00000000,00000000,00000004,00000000 382: 00000000,00000000,00000000,00000000,00000000,00000000,00000008,00000000 383: 00000000,00000000,00000000,00000000,00000000,00000000,00000010,00000000 384: 00000000,00000000,00000000,00000000,00000000,00000000,00000020,00000000 385: 00000000,00000000,00000000,00000000,00000000,00000000,00000040,00000000 386: 00000000,00000000,00000000,00000000,00000000,00000000,00000080,00000000 387: 00000000,00000000,00000000,00000000,00000000,00000000,00000100,00000000 388: 00000000,00000000,00000000,00000000,00000000,00000000,00000200,00000000 389: 00000000,00000000,00000000,00000000,00000000,00000000,00000400,00000000 390: 00000000,00000000,00000000,00000000,00000000,00000000,00000800,00000000 391: 00000000,00000000,00000000,00000000,00000000,00000000,00001000,00000000 392: 00000000,00000000,00000000,00000000,00000000,00000000,00002000,00000000 393: 00000000,00000000,00000000,00000000,00000000,00000000,00004000,00000000 394: 00000000,00000000,00000000,00000000,00000000,00000000,00008000,00000000 395: 00000000,00000000,00000000,00000000,00000000,00000000,00010000,00000000 396: 00000000,00000000,00000000,00000000,00000000,00000000,00020000,00000000 397: 00000000,00000000,00000000,00000000,00000000,00000000,00040000,00000000 398: 00000000,00000000,00000000,00000000,00000000,00000000,00080000,00000000 399: 00000000,00000000,00000000,00000000,00000000,00000000,00100000,00000000 400: 00000000,00000000,00000000,00000000,00000000,00000000,00200000,00000000 401: 00000000,00000000,00000000,00000000,00000000,00000000,00400000,00000000 402: 00000000,00000000,00000000,00000000,00000000,00000000,00800000,00000000 403: 00000000,00000000,00000000,00000000,00000000,00000000,01000000,00000000 404: 00000000,00000000,00000000,00000000,00000000,00000000,02000000,00000000 405: 00000000,00000000,00000000,00000000,00000000,00000000,04000000,00000000 406: 00000000,00000000,00000000,00000000,00000000,00000000,08000000,00000000 407: 00000000,00000000,00000000,00000000,00000000,00000000,10000000,00000000 408: 00000000,00000000,00000000,00000000,00000000,00000000,20000000,00000000 409: 00000000,00000000,00000000,00000000,00000000,00000000,40000000,00000000 410: 00000000,00000000,00000000,00000000,00000000,00000000,80000000,00000000 411: 00000000,00000000,00000000,00000000,00000000,00000001,00000000,00000000 412: 00000000,00000000,00000000,00000000,00000000,00000002,00000000,00000000 413: 00000000,00000000,00000000,00000000,00000000,00000004,00000000,00000000 414: 00000000,00000000,00000000,00000000,00000000,00000008,00000000,00000000 415: 00000000,00000000,00000000,00000000,00000000,00000010,00000000,00000000 416: 00000000,00000000,00000000,00000000,00000000,00000020,00000000,00000000 417: 00000000,00000000,00000000,00000000,00000000,00000040,00000000,00000000 418: 00000000,00000000,00000000,00000000,00000000,00000080,00000000,00000000 419: 00000000,00000000,00000000,00000000,00000000,00000100,00000000,00000000 420: 00000000,00000000,00000000,00000000,00000000,00000200,00000000,00000000 421: 00000000,00000000,00000000,00000000,00000000,00000400,00000000,00000000 422: 00000000,00000000,00000000,00000000,00000000,00000800,00000000,00000000 423: 00000000,00000000,00000000,00000000,00000000,00001000,00000000,00000000 424: 00000000,00000000,00000000,00000000,00000000,00002000,00000000,00000000 425: 00000000,00000000,00000000,00000000,00000000,00004000,00000000,00000000 426: 00000000,00000000,00000000,00000000,00000000,00008000,00000000,00000000 427: 00000000,00000000,00000000,00000000,00000000,00010000,00000000,00000000 428: 00000000,00000000,00000000,00000000,00000000,00020000,00000000,00000000 429: 00000000,00000000,00000000,00000000,00000000,00040000,00000000,00000000 430: 00000000,00000000,00000000,00000000,00000000,00080000,00000000,00000000 431: 00000000,00000000,00000000,00000000,00000000,00100000,00000000,00000000 432: 00000000,00000000,00000000,00000000,00000000,00200000,00000000,00000000 433: 00000000,00000000,00000000,00000000,00000000,00400000,00000000,00000000 434: 00000000,00000000,00000000,00000000,00000000,00800000,00000000,00000000 435: 00000000,00000000,00000000,00000000,00000000,01000000,00000000,00000000 436: 00000000,00000000,00000000,00000000,00000000,02000000,00000000,00000000 437: 00000000,00000000,00000000,00000000,00000000,04000000,00000000,00000000 438: 00000000,00000000,00000000,00000000,00000000,08000000,00000000,00000000 439: 00000000,00000000,00000000,00000000,00000000,10000000,00000000,00000000 440: 00000000,00000000,00000000,00000000,00000000,20000000,00000000,00000000 441: 00000000,00000000,00000000,00000000,00000000,40000000,00000000,00000000 442: 00000000,00000000,00000000,00000000,00000000,80000000,00000000,00000000 443: 00000000,00000000,00000000,00000000,00000001,00000000,00000000,00000000 444: 00000000,00000000,00000000,00000000,00000002,00000000,00000000,00000000 445: 00000000,00000000,00000000,00000000,00000004,00000000,00000000,00000000 446: 00000000,00000000,00000000,00000000,00000008,00000000,00000000,00000000 447: 00000000,00000000,00000000,00000000,00000010,00000000,00000000,00000000 448: 00000000,00000000,00000000,00000000,00000020,00000000,00000000,00000000 449: 00000000,00000000,00000000,00000000,00000040,00000000,00000000,00000000 450: 00000000,00000000,00000000,00000000,00000080,00000000,00000000,00000000 451: 00000000,00000000,00000000,00000000,00000100,00000000,00000000,00000000 452: 00000000,00000000,00000000,00000000,00000200,00000000,00000000,00000000 453: 00000000,00000000,00000000,00000000,00000400,00000000,00000000,00000000 454: 00000000,00000000,00000000,00000000,00000800,00000000,00000000,00000000 455: 00000000,00000000,00000000,00000000,00001000,00000000,00000000,00000000 456: 00000000,00000000,00000000,00000000,00002000,00000000,00000000,00000000 457: 00000000,00000000,00000000,00000000,00004000,00000000,00000000,00000000 After: 331: 00000000,00000000,00000000,00000000,00010000,00000000,00000000,00000000 332: 00000000,00000000,00000000,00000000,00020000,00000000,00000000,00000000 333: 00000000,00000000,00000000,00000000,00040000,00000000,00000000,00000000 334: 00000000,00000000,00000000,00000000,00080000,00000000,00000000,00000000 335: 00000000,00000000,00000000,00000000,00100000,00000000,00000000,00000000 336: 00000000,00000000,00000000,00000000,00200000,00000000,00000000,00000000 337: 00000000,00000000,00000000,00000000,00400000,00000000,00000000,00000000 338: 00000000,00000000,00000000,00000000,00800000,00000000,00000000,00000000 339: 00010000,00000000,00000000,00000000,00000000,00000000,00000000,00000000 340: 00020000,00000000,00000000,00000000,00000000,00000000,00000000,00000000 341: 00040000,00000000,00000000,00000000,00000000,00000000,00000000,00000000 342: 00080000,00000000,00000000,00000000,00000000,00000000,00000000,00000000 343: 00100000,00000000,00000000,00000000,00000000,00000000,00000000,00000000 344: 00200000,00000000,00000000,00000000,00000000,00000000,00000000,00000000 345: 00400000,00000000,00000000,00000000,00000000,00000000,00000000,00000000 346: 00800000,00000000,00000000,00000000,00000000,00000000,00000000,00000000 347: 00000000,00000000,00000000,00000000,00000001,00000000,00000000,00000000 348: 00000000,00000000,00000000,00000000,00000002,00000000,00000000,00000000 349: 00000000,00000000,00000000,00000000,00000004,00000000,00000000,00000000 350: 00000000,00000000,00000000,00000000,00000008,00000000,00000000,00000000 351: 00000000,00000000,00000000,00000000,00000010,00000000,00000000,00000000 352: 00000000,00000000,00000000,00000000,00000020,00000000,00000000,00000000 353: 00000000,00000000,00000000,00000000,00000040,00000000,00000000,00000000 354: 00000000,00000000,00000000,00000000,00000080,00000000,00000000,00000000 355: 00000000,00000000,00000000,00000000,00000100,00000000,00000000,00000000 356: 00000000,00000000,00000000,00000000,00000200,00000000,00000000,00000000 357: 00000000,00000000,00000000,00000000,00000400,00000000,00000000,00000000 358: 00000000,00000000,00000000,00000000,00000800,00000000,00000000,00000000 359: 00000000,00000000,00000000,00000000,00001000,00000000,00000000,00000000 360: 00000000,00000000,00000000,00000000,00002000,00000000,00000000,00000000 361: 00000000,00000000,00000000,00000000,00004000,00000000,00000000,00000000 362: 00000000,00000000,00000000,00000000,00008000,00000000,00000000,00000000 363: 00000000,00000000,00000000,00000000,01000000,00000000,00000000,00000000 364: 00000000,00000000,00000000,00000000,02000000,00000000,00000000,00000000 365: 00000000,00000000,00000000,00000000,04000000,00000000,00000000,00000000 366: 00000000,00000000,00000000,00000000,08000000,00000000,00000000,00000000 367: 00000000,00000000,00000000,00000000,10000000,00000000,00000000,00000000 368: 00000000,00000000,00000000,00000000,20000000,00000000,00000000,00000000 369: 00000000,00000000,00000000,00000000,40000000,00000000,00000000,00000000 370: 00000000,00000000,00000000,00000000,80000000,00000000,00000000,00000000 371: 00000001,00000000,00000000,00000000,00000000,00000000,00000000,00000000 372: 00000002,00000000,00000000,00000000,00000000,00000000,00000000,00000000 373: 00000004,00000000,00000000,00000000,00000000,00000000,00000000,00000000 374: 00000008,00000000,00000000,00000000,00000000,00000000,00000000,00000000 375: 00000010,00000000,00000000,00000000,00000000,00000000,00000000,00000000 376: 00000020,00000000,00000000,00000000,00000000,00000000,00000000,00000000 377: 00000040,00000000,00000000,00000000,00000000,00000000,00000000,00000000 378: 00000080,00000000,00000000,00000000,00000000,00000000,00000000,00000000 379: 00000100,00000000,00000000,00000000,00000000,00000000,00000000,00000000 380: 00000200,00000000,00000000,00000000,00000000,00000000,00000000,00000000 381: 00000400,00000000,00000000,00000000,00000000,00000000,00000000,00000000 382: 00000800,00000000,00000000,00000000,00000000,00000000,00000000,00000000 383: 00001000,00000000,00000000,00000000,00000000,00000000,00000000,00000000 384: 00002000,00000000,00000000,00000000,00000000,00000000,00000000,00000000 385: 00004000,00000000,00000000,00000000,00000000,00000000,00000000,00000000 386: 00008000,00000000,00000000,00000000,00000000,00000000,00000000,00000000 387: 01000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000 388: 02000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000 389: 04000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000 390: 08000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000 391: 10000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000 392: 20000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000 393: 40000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000 394: 80000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000 395: 00000000,00000000,00000000,00000000,00000000,00000001,00000000,00000000 396: 00000000,00000000,00000000,00000000,00000000,00000002,00000000,00000000 397: 00000000,00000000,00000000,00000000,00000000,00000004,00000000,00000000 398: 00000000,00000000,00000000,00000000,00000000,00000008,00000000,00000000 399: 00000000,00000000,00000000,00000000,00000000,00000010,00000000,00000000 400: 00000000,00000000,00000000,00000000,00000000,00000020,00000000,00000000 401: 00000000,00000000,00000000,00000000,00000000,00000040,00000000,00000000 402: 00000000,00000000,00000000,00000000,00000000,00000080,00000000,00000000 403: 00000000,00000000,00000000,00000000,00000000,00000100,00000000,00000000 404: 00000000,00000000,00000000,00000000,00000000,00000200,00000000,00000000 405: 00000000,00000000,00000000,00000000,00000000,00000400,00000000,00000000 406: 00000000,00000000,00000000,00000000,00000000,00000800,00000000,00000000 407: 00000000,00000000,00000000,00000000,00000000,00001000,00000000,00000000 408: 00000000,00000000,00000000,00000000,00000000,00002000,00000000,00000000 409: 00000000,00000000,00000000,00000000,00000000,00004000,00000000,00000000 410: 00000000,00000000,00000000,00000000,00000000,00008000,00000000,00000000 411: 00000000,00000000,00000000,00000000,00000000,00010000,00000000,00000000 412: 00000000,00000000,00000000,00000000,00000000,00020000,00000000,00000000 413: 00000000,00000000,00000000,00000000,00000000,00040000,00000000,00000000 414: 00000000,00000000,00000000,00000000,00000000,00080000,00000000,00000000 415: 00000000,00000000,00000000,00000000,00000000,00100000,00000000,00000000 416: 00000000,00000000,00000000,00000000,00000000,00200000,00000000,00000000 417: 00000000,00000000,00000000,00000000,00000000,00400000,00000000,00000000 418: 00000000,00000000,00000000,00000000,00000000,00800000,00000000,00000000 419: 00000000,00000000,00000000,00000000,00000000,01000000,00000000,00000000 420: 00000000,00000000,00000000,00000000,00000000,02000000,00000000,00000000 421: 00000000,00000000,00000000,00000000,00000000,04000000,00000000,00000000 422: 00000000,00000000,00000000,00000000,00000000,08000000,00000000,00000000 423: 00000000,00000000,00000000,00000000,00000000,10000000,00000000,00000000 424: 00000000,00000000,00000000,00000000,00000000,20000000,00000000,00000000 425: 00000000,00000000,00000000,00000000,00000000,40000000,00000000,00000000 426: 00000000,00000000,00000000,00000000,00000000,80000000,00000000,00000000 427: 00000000,00000001,00000000,00000000,00000000,00000000,00000000,00000000 428: 00000000,00000002,00000000,00000000,00000000,00000000,00000000,00000000 429: 00000000,00000004,00000000,00000000,00000000,00000000,00000000,00000000 430: 00000000,00000008,00000000,00000000,00000000,00000000,00000000,00000000 431: 00000000,00000010,00000000,00000000,00000000,00000000,00000000,00000000 432: 00000000,00000020,00000000,00000000,00000000,00000000,00000000,00000000 433: 00000000,00000040,00000000,00000000,00000000,00000000,00000000,00000000 434: 00000000,00000080,00000000,00000000,00000000,00000000,00000000,00000000 435: 00000000,00000100,00000000,00000000,00000000,00000000,00000000,00000000 436: 00000000,00000200,00000000,00000000,00000000,00000000,00000000,00000000 437: 00000000,00000400,00000000,00000000,00000000,00000000,00000000,00000000 438: 00000000,00000800,00000000,00000000,00000000,00000000,00000000,00000000 439: 00000000,00001000,00000000,00000000,00000000,00000000,00000000,00000000 440: 00000000,00002000,00000000,00000000,00000000,00000000,00000000,00000000 441: 00000000,00004000,00000000,00000000,00000000,00000000,00000000,00000000 442: 00000000,00008000,00000000,00000000,00000000,00000000,00000000,00000000 443: 00000000,00010000,00000000,00000000,00000000,00000000,00000000,00000000 444: 00000000,00020000,00000000,00000000,00000000,00000000,00000000,00000000 445: 00000000,00040000,00000000,00000000,00000000,00000000,00000000,00000000 446: 00000000,00080000,00000000,00000000,00000000,00000000,00000000,00000000 447: 00000000,00100000,00000000,00000000,00000000,00000000,00000000,00000000 448: 00000000,00200000,00000000,00000000,00000000,00000000,00000000,00000000 449: 00000000,00400000,00000000,00000000,00000000,00000000,00000000,00000000 450: 00000000,00800000,00000000,00000000,00000000,00000000,00000000,00000000 451: 00000000,01000000,00000000,00000000,00000000,00000000,00000000,00000000 452: 00000000,02000000,00000000,00000000,00000000,00000000,00000000,00000000 453: 00000000,04000000,00000000,00000000,00000000,00000000,00000000,00000000 454: 00000000,08000000,00000000,00000000,00000000,00000000,00000000,00000000 455: 00000000,10000000,00000000,00000000,00000000,00000000,00000000,00000000 456: 00000000,20000000,00000000,00000000,00000000,00000000,00000000,00000000 457: 00000000,40000000,00000000,00000000,00000000,00000000,00000000,00000000 Signed-off-by: Tariq Toukan --- drivers/net/ethernet/mellanox/mlx5/core/eq.c | 13 +++++++++++-- 1 file changed, 11 insertions(+), 2 deletions(-) diff --git a/drivers/net/ethernet/mellanox/mlx5/core/eq.c b/drivers/net/ethernet/mellanox/mlx5/core/eq.c index 229728c80233..4818cc0c9bc3 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/eq.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/eq.c @@ -812,6 +812,7 @@ static int comp_irqs_request(struct mlx5_core_dev *dev) int ncomp_eqs = table->num_comp_eqs; u16 *cpus; int ret; + int cpu; int i; ncomp_eqs = table->num_comp_eqs; @@ -830,8 +831,16 @@ static int comp_irqs_request(struct mlx5_core_dev *dev) ret = -ENOMEM; goto free_irqs; } - for (i = 0; i < ncomp_eqs; i++) - cpus[i] = cpumask_local_spread(i, dev->priv.numa_node); + + i = 0; + rcu_read_lock(); + for_each_numa_hop_cpu(cpu, dev->priv.numa_node) { + cpus[i] = cpu; + if (++i == ncomp_eqs) + goto spread_done; + } +spread_done: + rcu_read_unlock(); ret = mlx5_irqs_request_vectors(dev, cpus, ncomp_eqs, table->comp_irqs); kfree(cpus); if (ret < 0)