From patchwork Mon Aug 12 11:57:35 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Uros Bizjak X-Patchwork-Id: 13760452 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1CA3DC52D7C for ; Mon, 12 Aug 2024 12:00:08 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 56F0A6B008C; Mon, 12 Aug 2024 08:00:04 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 4F7236B0092; Mon, 12 Aug 2024 08:00:04 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 3705D6B009E; Mon, 12 Aug 2024 08:00:04 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 018C86B008C for ; Mon, 12 Aug 2024 08:00:03 -0400 (EDT) Received: from smtpin02.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id A13DE1A1D81 for ; Mon, 12 Aug 2024 12:00:03 +0000 (UTC) X-FDA: 82443449886.02.A84733C Received: from mail-ed1-f46.google.com (mail-ed1-f46.google.com [209.85.208.46]) by imf02.hostedemail.com (Postfix) with ESMTP id AAC3A80020 for ; Mon, 12 Aug 2024 12:00:00 +0000 (UTC) Authentication-Results: imf02.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=mnhqvuQ4; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf02.hostedemail.com: domain of ubizjak@gmail.com designates 209.85.208.46 as permitted sender) smtp.mailfrom=ubizjak@gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1723463989; a=rsa-sha256; cv=none; b=V7n5wrR+aHEXy89A2Zg/1GW8WVTfNYGro88bYJZO20Xv1ZqJ8ThzvY6YiSwx8rPFOD1Wja HftFmLOYDl8Dy00bO/ItbCfWIAFBrzbKmqb4i5DclsElOfDJKJiP/QYJqPLCtMBxxYNbzG qXw67XDHQAoUZzWDqkofnZolq/3TxGU= ARC-Authentication-Results: i=1; imf02.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=mnhqvuQ4; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf02.hostedemail.com: domain of ubizjak@gmail.com designates 209.85.208.46 as permitted sender) smtp.mailfrom=ubizjak@gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1723463989; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=fruDKXqvLpvOHXHlfsYMBztoBLfQBCYZJcuGnYhVG7c=; b=wyUL2bttKO7GGXvNEWSGMdaogt3wzNRh+daRk4OJiPydeQ/i2An6F0MEW+BGw4e9snr8Cm XaKixvjMINWWCE575b7APLh5WVxpjtkqnPIwCJOET7CkUNvWvnRJgNUxOJZQ7BFy+cHtmh 5jiDMWRWxR8C2VrMjHYF27vlfR6kV7c= Received: by mail-ed1-f46.google.com with SMTP id 4fb4d7f45d1cf-5bba25c1e15so5101523a12.2 for ; Mon, 12 Aug 2024 05:00:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1723463999; x=1724068799; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=fruDKXqvLpvOHXHlfsYMBztoBLfQBCYZJcuGnYhVG7c=; b=mnhqvuQ4i6NHom1fn685eb+gA3bgmMICXSjosDl2yseGTfEUk6QKrDAUR8EA90wsb2 QUG/11dROmGrgBUQx6sosG62nsAWrOjLpvmCBNFYytQqcQVIg8KL6Tu75VhvgEN1H64c 9s6XzVcXahQQaEsi73M7ELQWgYB8v/nskBj9u4MHU86tKH3UvXiE1YywQG8Uhu7KYD8Y XQoSI/fDPDH42/K1h5/zMYSjf8jdi6D7eSigEFTbVyVQ8TOTC154KD14IuZD34qtcTM/ PbD18FiTlYmo1FnIgpHx+HT3Wd0jmu0e5d0STtwk2L4MxBZrV/YgevQ2fLRiixtNRj1C 4Vtw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1723463999; x=1724068799; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=fruDKXqvLpvOHXHlfsYMBztoBLfQBCYZJcuGnYhVG7c=; b=vhWArl5XrQ1Y8+FC4gJhnPBpbXyOrP7N+rWWkfmQs1ebUgoZsqmG2/vkPpemKyUjmd P96AifrRM8MXK+vJG/WwbnOdOvXA16nhLEnc8kynpW7TLh8JOUHDTJwALsKMnl1qoAjk 8azGDWB20nm4XzZIffuTGpkY9kk0/6RUHDUBJKGua2HEnc1+E6KYjI0UyFn0i7u6niNv 3Dztp4RbHKfM6zWb7B9sJ8UCxRfbIAjlHzADmoXfUZbc8aDc0TFUuacRitRQhQZGqlX9 ooGl5opaT92DhYq3nriUyH91reOr6YHihwNCYdK3DvCoDx4bt2zti1j4m6Q3Oy/ghRm7 oBrQ== X-Gm-Message-State: AOJu0Yw0dHEUV1wX5MqO9vF0npLcRndcYb9SsymXGzkXbjfzJ536wVFn tHeKX0TQ1d+L1Z09mY9Ktg3JPLCRAlN+eAPOqbOHkF6C7JMl39zocTqvoeUmVLo= X-Google-Smtp-Source: AGHT+IHuOkc+I/HRzwKiheSTIvJj3Z6nmuPXhvaWwBFb6EcIiM2K0UMjtS7f0OP9qPL7ja6c4c3zXg== X-Received: by 2002:a05:6402:27d0:b0:57c:7471:a0dd with SMTP id 4fb4d7f45d1cf-5bd44c2fb29mr21508a12.12.1723463998274; Mon, 12 Aug 2024 04:59:58 -0700 (PDT) Received: from fedora.iskraemeco.si ([193.77.86.250]) by smtp.gmail.com with ESMTPSA id 4fb4d7f45d1cf-5bd1a5e04aesm2055970a12.76.2024.08.12.04.59.57 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 12 Aug 2024 04:59:57 -0700 (PDT) From: Uros Bizjak To: linux-mm@kvack.org, linux-kernel@vger.kernel.org Cc: Uros Bizjak , Dennis Zhou , Tejun Heo , Christoph Lameter , Andy Lutomirski , Ingo Molnar , Nadav Amit , Brian Gerst , Denys Vlasenko , "H. Peter Anvin" , Linus Torvalds , Peter Zijlstra , Thomas Gleixner , Borislav Petkov , Luc Van Oostenryck Subject: [RFC PATCH v2 1/4] percpu: Define pcpu_typeof() Date: Mon, 12 Aug 2024 13:57:35 +0200 Message-ID: <20240812115945.484051-2-ubizjak@gmail.com> X-Mailer: git-send-email 2.46.0 In-Reply-To: <20240812115945.484051-1-ubizjak@gmail.com> References: <20240812115945.484051-1-ubizjak@gmail.com> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Queue-Id: AAC3A80020 X-Rspamd-Server: rspam01 X-Stat-Signature: 9dan1b51ygchq81oqyghuy4w7pncqbwa X-HE-Tag: 1723464000-40558 X-HE-Meta: U2FsdGVkX19uORrbdYIiRiq+k034qcCxXjiEvBcl67VQrslahLIRzdf3mehGZxqnwVf5YMkI/glg6Ue4vhUfKC/wRDygYQwZ9mPCYPUDKbM4cnH8qNtOKAuzNsEZIN/pvL3nwjT0D2ZbbkjdFTbM0OPs+usvg8Y7KA7YHu+L5xLUeu+zd0LjXH16m3oyC+Akm2mFykah6QqybNn5UMNxtVKEu1FYZOih1uME3SFPEeRgsm4xtkCJuBbFswrLIQtCFS+SG5uWTVcrjq1Ihx0YOYa4Exhc2vgyBRDeqZT5JCnmjmesxOGViwMX4v7TRVMyRdGunkdBvccSYH1zaL4b3i1tyXs6unuSWocutH9WthG6XoxkgKddrsZomQmGTx4obUDPQXjQJFvCOq8UqL6YTepXfW+3impoUX/vIa/s2AEnKwEq7d36VYeayaM+jnpPwJdPmNi8+AEEtiz3vDE67HIOubJtNK8kddPXuaucWXsq0Hli0ansWsDgdoKQWDC9uptOKqAfl3edo+ISH3JAC72KdOW8YyWvuDsSI0LuSR0/b/syu7VncqaX33fpwIT1WGHSwNQFb5idDCg0fvYNY4BLXkux77nhVzuD0c/+aVp6/dsF8sJxh4b/t/u4urXn5PxFJzzyN8123xqNJkyB88b2Z34mkC3vBOe9Ts9C7xNUgWjst/mVHRqFzgUYNUrFk+WqcsNZoz6oTW0rGfk1BYw2uGcymbVvnIP+6BzhBnlb37TCn4QwoqOW9YK36s9XsJOFIDg2gPJUM3qF0ohaeur7RWehjoJgTVziac5MKZJczcF/9/BqWI3NLPoNzQsXeymFrOL6duEZYaV068o3bkceNTlNQ4CW7Y1Ze0gH9JtbcfcPPgSe3xBe4JjRviq01rZzee1qjgRilZz7ix/7R6ZVf9lIvzGxYCdr23ojN33bG6zDmp6HHc4xaxMEdvDWVHevjE3JFJWpH2tujsI ROqmRKZn IHzvmDq88KZnB0wQx+/rf88Ql2CL7Vt2Bj8NUfzI9cprs+vs555SFVrL5fY5p3QagsY0HfIq5wF+4hJjFMAAs1jaSrKLrqo6TmV5FY8VZ+uy2VlCqB86oZvPDBm3B/L55wNpm1eaS3h89ln3vpCsQ/gtc08QRNg/9UReLA+4HbFXSslN1ibLVMGzH2lPL3EIW4vO9lj6i+eyDLvfdpomnnAOcUz7n8bvLbyKMO8E4rGLhZhNK/Pve1+IA0UBUpil11NK8evkpLyQ09JFO0SYaTW+78IbrTD+kfO+p9+9DIb50znmKPLP8f4CIGbr9hJ9PKGUYkffWy9voTQrcZXKsMf1jTzI0PDc+pY5/DA0RuGaZymocMV5iNAAQw+N0/NxdlpAvsZYtAtpW3Ws2pokhJBlwXCMlsN1l7WS4qsaUDzPeaWjPfTpLxlo7ULLIXlwj2IgfuMUiJwCYeWDGjjEy5psTqOMZnyE8GrKBZID325qGzAXJkl/PwGy6T1nTctC4q5QizFg/tvhzKvTlGH/x7SR7lrqyvw2elZtvO5LMgfnYKKcU4SMhOZbCsEU9+NFyOxS+1Riy5e2VOEvWi0klrBDeqF9jzdf3oa67E4oSMk9kYP/IbO/Uj2sipYVxyM6V6kmJ9mXodmAmqNkMZg2FtkChJAwCbnNb4FUIuJ9slddYVMkk98clq+X9DqwNguoYH1bp8NDDMTSPkjrFMPMagd8NSAlrvRcSESkOyDr0Lsafxml/3E8+TsF0wSivt0L/XL1ZE6k8TgCofc9f/bi15JjfnZsSHHP7QK7I X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Define pcpu_typeof() to use __typeof_unqual__() as typeof operator when available, to return unqualified type of the expression. If an arch defines __percpu variables in their own named address space, then pcpu_typeof() returns unqualified type of the expression without named address space quialifier when CONFIG_CC_HAS_TYPEOF_UNQUAL is defined. Signed-off-by: Uros Bizjak Cc: Dennis Zhou Cc: Tejun Heo Cc: Christoph Lameter Cc: Andy Lutomirski Cc: Ingo Molnar Cc: Nadav Amit Cc: Brian Gerst Cc: Denys Vlasenko Cc: H. Peter Anvin Cc: Linus Torvalds Cc: Peter Zijlstra Cc: Thomas Gleixner Cc: Borislav Petkov Cc: Luc Van Oostenryck --- arch/x86/include/asm/percpu.h | 38 ++++++++++++++++++++++------------ include/asm-generic/percpu.h | 26 +++++++++++------------ include/linux/part_stat.h | 2 +- include/linux/percpu-defs.h | 39 ++++++++++++++++++++++++++++------- init/Kconfig | 3 +++ kernel/locking/percpu-rwsem.c | 2 +- 6 files changed, 75 insertions(+), 35 deletions(-) diff --git a/arch/x86/include/asm/percpu.h b/arch/x86/include/asm/percpu.h index c55a79d5feae..4ab4ad60d767 100644 --- a/arch/x86/include/asm/percpu.h +++ b/arch/x86/include/asm/percpu.h @@ -73,10 +73,14 @@ unsigned long tcp_ptr__ = raw_cpu_read_long(this_cpu_off); \ \ tcp_ptr__ += (__force unsigned long)(_ptr); \ - (typeof(*(_ptr)) __kernel __force *)tcp_ptr__; \ + (pcpu_typeof(*(_ptr)) __kernel __force *)tcp_ptr__; \ }) #else -#define arch_raw_cpu_ptr(_ptr) ({ BUILD_BUG(); (typeof(_ptr))0; }) +#define arch_raw_cpu_ptr(_ptr) \ +({ \ + BUILD_BUG(); \ + (pcpu_typeof(*(_ptr)) __kernel __force *)0; \ +}) #endif #define PER_CPU_VAR(var) %__percpu_seg:(var)__percpu_rel @@ -172,7 +176,7 @@ do { \ : [val] __pcpu_reg_##size("=", pfo_val__) \ : [var] "m" (__my_cpu_var(_var))); \ \ - (typeof(_var))(unsigned long) pfo_val__; \ + (pcpu_typeof(_var))(unsigned long) pfo_val__; \ }) #define __raw_cpu_write(size, qual, _var, _val) \ @@ -180,7 +184,7 @@ do { \ __pcpu_type_##size pto_val__ = __pcpu_cast_##size(_val); \ \ if (0) { \ - typeof(_var) pto_tmp__; \ + pcpu_typeof(_var) pto_tmp__; \ pto_tmp__ = (_val); \ (void)pto_tmp__; \ } \ @@ -193,7 +197,11 @@ do { \ * The generic per-CPU infrastrucutre is not suitable for * reading const-qualified variables. */ -#define __raw_cpu_read_const(pcp) ({ BUILD_BUG(); (typeof(pcp))0; }) +#define __raw_cpu_read_const(pcp) \ +({ \ + BUILD_BUG(); \ + (pcpu_typeof(pcp))0; \ +}) #endif /* CONFIG_USE_X86_SEG_SUPPORT */ @@ -205,7 +213,7 @@ do { \ : [val] __pcpu_reg_##size("=", pfo_val__) \ : [var] "i" (&(_var))); \ \ - (typeof(_var))(unsigned long) pfo_val__; \ + (pcpu_typeof(_var))(unsigned long) pfo_val__; \ }) #define percpu_unary_op(size, qual, op, _var) \ @@ -219,7 +227,7 @@ do { \ __pcpu_type_##size pto_val__ = __pcpu_cast_##size(_val); \ \ if (0) { \ - typeof(_var) pto_tmp__; \ + pcpu_typeof(_var) pto_tmp__; \ pto_tmp__ = (_val); \ (void)pto_tmp__; \ } \ @@ -239,7 +247,7 @@ do { \ (int)(val) : 0; \ \ if (0) { \ - typeof(var) pao_tmp__; \ + pcpu_typeof(var) pao_tmp__; \ pao_tmp__ = (val); \ (void)pao_tmp__; \ } \ @@ -263,7 +271,7 @@ do { \ : [tmp] __pcpu_reg_##size("+", paro_tmp__), \ [var] "+m" (__my_cpu_var(_var)) \ : : "memory"); \ - (typeof(_var))(unsigned long) (paro_tmp__ + _val); \ + (pcpu_typeof(_var))(unsigned long) (paro_tmp__ + _val); \ }) /* @@ -272,7 +280,7 @@ do { \ */ #define raw_percpu_xchg_op(_var, _nval) \ ({ \ - typeof(_var) pxo_old__ = raw_cpu_read(_var); \ + pcpu_typeof(_var) pxo_old__ = raw_cpu_read(_var); \ \ raw_cpu_write(_var, _nval); \ \ @@ -286,7 +294,7 @@ do { \ */ #define this_percpu_xchg_op(_var, _nval) \ ({ \ - typeof(_var) pxo_old__ = this_cpu_read(_var); \ + pcpu_typeof(_var) pxo_old__ = this_cpu_read(_var); \ \ do { } while (!this_cpu_try_cmpxchg(_var, &pxo_old__, _nval)); \ \ @@ -309,7 +317,7 @@ do { \ : [nval] __pcpu_reg_##size(, pco_new__) \ : "memory"); \ \ - (typeof(_var))(unsigned long) pco_old__; \ + (pcpu_typeof(_var))(unsigned long) pco_old__; \ }) #define percpu_try_cmpxchg_op(size, qual, _var, _ovalp, _nval) \ @@ -568,7 +576,11 @@ do { \ #else /* !CONFIG_X86_64: */ /* There is no generic 64-bit read stable operation for 32-bit targets. */ -#define this_cpu_read_stable_8(pcp) ({ BUILD_BUG(); (typeof(pcp))0; }) +#define this_cpu_read_stable_8(pcp) \ +({ \ + BUILD_BUG(); \ + (pcpu_typeof(pcp))0; \ +}) #define raw_cpu_read_long(pcp) raw_cpu_read_4(pcp) diff --git a/include/asm-generic/percpu.h b/include/asm-generic/percpu.h index 94cbd50cc870..c091e09046c5 100644 --- a/include/asm-generic/percpu.h +++ b/include/asm-generic/percpu.h @@ -74,7 +74,7 @@ do { \ #define raw_cpu_generic_add_return(pcp, val) \ ({ \ - typeof(pcp) *__p = raw_cpu_ptr(&(pcp)); \ + pcpu_typeof(pcp) *__p = raw_cpu_ptr(&(pcp)); \ \ *__p += val; \ *__p; \ @@ -82,8 +82,8 @@ do { \ #define raw_cpu_generic_xchg(pcp, nval) \ ({ \ - typeof(pcp) *__p = raw_cpu_ptr(&(pcp)); \ - typeof(pcp) __ret; \ + pcpu_typeof(pcp) *__p = raw_cpu_ptr(&(pcp)); \ + pcpu_typeof(pcp) __ret; \ __ret = *__p; \ *__p = nval; \ __ret; \ @@ -91,7 +91,7 @@ do { \ #define __cpu_fallback_try_cmpxchg(pcp, ovalp, nval, _cmpxchg) \ ({ \ - typeof(pcp) __val, __old = *(ovalp); \ + pcpu_typeof(pcp) __val, __old = *(ovalp); \ __val = _cmpxchg(pcp, __old, nval); \ if (__val != __old) \ *(ovalp) = __val; \ @@ -100,8 +100,8 @@ do { \ #define raw_cpu_generic_try_cmpxchg(pcp, ovalp, nval) \ ({ \ - typeof(pcp) *__p = raw_cpu_ptr(&(pcp)); \ - typeof(pcp) __val = *__p, ___old = *(ovalp); \ + pcpu_typeof(pcp) *__p = raw_cpu_ptr(&(pcp)); \ + pcpu_typeof(pcp) __val = *__p, ___old = *(ovalp); \ bool __ret; \ if (__val == ___old) { \ *__p = nval; \ @@ -115,14 +115,14 @@ do { \ #define raw_cpu_generic_cmpxchg(pcp, oval, nval) \ ({ \ - typeof(pcp) __old = (oval); \ + pcpu_typeof(pcp) __old = (oval); \ raw_cpu_generic_try_cmpxchg(pcp, &__old, nval); \ __old; \ }) #define __this_cpu_generic_read_nopreempt(pcp) \ ({ \ - typeof(pcp) ___ret; \ + pcpu_typeof(pcp) ___ret; \ preempt_disable_notrace(); \ ___ret = READ_ONCE(*raw_cpu_ptr(&(pcp))); \ preempt_enable_notrace(); \ @@ -131,7 +131,7 @@ do { \ #define __this_cpu_generic_read_noirq(pcp) \ ({ \ - typeof(pcp) ___ret; \ + pcpu_typeof(pcp) ___ret; \ unsigned long ___flags; \ raw_local_irq_save(___flags); \ ___ret = raw_cpu_generic_read(pcp); \ @@ -141,7 +141,7 @@ do { \ #define this_cpu_generic_read(pcp) \ ({ \ - typeof(pcp) __ret; \ + pcpu_typeof(pcp) __ret; \ if (__native_word(pcp)) \ __ret = __this_cpu_generic_read_nopreempt(pcp); \ else \ @@ -160,7 +160,7 @@ do { \ #define this_cpu_generic_add_return(pcp, val) \ ({ \ - typeof(pcp) __ret; \ + pcpu_typeof(pcp) __ret; \ unsigned long __flags; \ raw_local_irq_save(__flags); \ __ret = raw_cpu_generic_add_return(pcp, val); \ @@ -170,7 +170,7 @@ do { \ #define this_cpu_generic_xchg(pcp, nval) \ ({ \ - typeof(pcp) __ret; \ + pcpu_typeof(pcp) __ret; \ unsigned long __flags; \ raw_local_irq_save(__flags); \ __ret = raw_cpu_generic_xchg(pcp, nval); \ @@ -190,7 +190,7 @@ do { \ #define this_cpu_generic_cmpxchg(pcp, oval, nval) \ ({ \ - typeof(pcp) __ret; \ + pcpu_typeof(pcp) __ret; \ unsigned long __flags; \ raw_local_irq_save(__flags); \ __ret = raw_cpu_generic_cmpxchg(pcp, oval, nval); \ diff --git a/include/linux/part_stat.h b/include/linux/part_stat.h index ac8c44dd8237..e228e1087f0c 100644 --- a/include/linux/part_stat.h +++ b/include/linux/part_stat.h @@ -33,7 +33,7 @@ struct disk_stats { #define part_stat_read(part, field) \ ({ \ - typeof((part)->bd_stats->field) res = 0; \ + pcpu_typeof((part)->bd_stats->field) res = 0; \ unsigned int _cpu; \ for_each_possible_cpu(_cpu) \ res += per_cpu_ptr((part)->bd_stats, _cpu)->field; \ diff --git a/include/linux/percpu-defs.h b/include/linux/percpu-defs.h index 8efce7414fad..aa824c6edb82 100644 --- a/include/linux/percpu-defs.h +++ b/include/linux/percpu-defs.h @@ -220,6 +220,21 @@ do { \ (void)__vpp_verify; \ } while (0) +/* + * Define percpu_typeof() to use __typeof_unqual__() as typeof + * operator when available, to return unqualified type of the exp. + * + * If an arch defines __percpu variables in their own named address + * space, then pcpu_typeof() returns unqualified type of the + * expression without named address space qualifier when + * CONFIG_CC_HAS_TYPEOF_UNQUAL is defined. + */ +#ifdef CONFIG_CC_HAS_TYPEOF_UNQUAL +#define pcpu_typeof(exp) __typeof_unqual__(exp) +#else +#define pcpu_typeof(exp) __typeof__(exp) +#endif + #ifdef CONFIG_SMP /* @@ -228,7 +243,10 @@ do { \ * pointer value. The weird cast keeps both GCC and sparse happy. */ #define SHIFT_PERCPU_PTR(__p, __offset) \ - RELOC_HIDE((typeof(*(__p)) __kernel __force *)(__p), (__offset)) + uintptr_t ptr__ = (__force uintptr_t)(__p); \ + \ + RELOC_HIDE((pcpu_typeof(*(__p)) __kernel __force *)(ptr__), \ + (__offset)) #define per_cpu_ptr(ptr, cpu) \ ({ \ @@ -254,13 +272,20 @@ do { \ #else /* CONFIG_SMP */ -#define VERIFY_PERCPU_PTR(__p) \ +#define PERCPU_PTR(__p) \ ({ \ - __verify_pcpu_ptr(__p); \ - (typeof(*(__p)) __kernel __force *)(__p); \ + uintptr_t ptr__ = (__force uintptr_t)(__p); \ + \ + (pcpu_typeof(*(__p)) __kernel __force *)(ptr__); \ +}) + +#define per_cpu_ptr(ptr, cpu) \ +({ \ + __verify_pcpu_ptr(ptr); \ + (void)(cpu); \ + PERCPU_PTR(ptr); \ }) -#define per_cpu_ptr(ptr, cpu) ({ (void)(cpu); VERIFY_PERCPU_PTR(ptr); }) #define raw_cpu_ptr(ptr) per_cpu_ptr(ptr, 0) #define this_cpu_ptr(ptr) raw_cpu_ptr(ptr) @@ -315,7 +340,7 @@ static __always_inline void __this_cpu_preempt_check(const char *op) { } #define __pcpu_size_call_return(stem, variable) \ ({ \ - typeof(variable) pscr_ret__; \ + pcpu_typeof(variable) pscr_ret__; \ __verify_pcpu_ptr(&(variable)); \ switch(sizeof(variable)) { \ case 1: pscr_ret__ = stem##1(variable); break; \ @@ -330,7 +355,7 @@ static __always_inline void __this_cpu_preempt_check(const char *op) { } #define __pcpu_size_call_return2(stem, variable, ...) \ ({ \ - typeof(variable) pscr2_ret__; \ + pcpu_typeof(variable) pscr2_ret__; \ __verify_pcpu_ptr(&(variable)); \ switch(sizeof(variable)) { \ case 1: pscr2_ret__ = stem##1(variable, __VA_ARGS__); break; \ diff --git a/init/Kconfig b/init/Kconfig index 37260d17267e..a9a04d0683b6 100644 --- a/init/Kconfig +++ b/init/Kconfig @@ -873,6 +873,9 @@ config ARCH_WANT_BATCHED_UNMAP_TLB_FLUSH config CC_HAS_INT128 def_bool !$(cc-option,$(m64-flag) -D__SIZEOF_INT128__=0) && 64BIT +config CC_HAS_TYPEOF_UNQUAL + def_bool $(success,echo 'int foo (int a) { __typeof_unqual__(a) b = a; return b; }' | $(CC) -x c - -S -o /dev/null) + config CC_IMPLICIT_FALLTHROUGH string default "-Wimplicit-fallthrough=5" if CC_IS_GCC && $(cc-option,-Wimplicit-fallthrough=5) diff --git a/kernel/locking/percpu-rwsem.c b/kernel/locking/percpu-rwsem.c index 6083883c4fe0..8345238ba428 100644 --- a/kernel/locking/percpu-rwsem.c +++ b/kernel/locking/percpu-rwsem.c @@ -184,7 +184,7 @@ EXPORT_SYMBOL_GPL(__percpu_down_read); #define per_cpu_sum(var) \ ({ \ - typeof(var) __sum = 0; \ + pcpu_typeof(var) __sum = 0; \ int cpu; \ compiletime_assert_atomic_type(__sum); \ for_each_possible_cpu(cpu) \ From patchwork Mon Aug 12 11:57:36 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Uros Bizjak X-Patchwork-Id: 13760451 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 743A5C52D7F for ; Mon, 12 Aug 2024 12:00:05 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 184ED6B0098; Mon, 12 Aug 2024 08:00:04 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 10BB26B009A; Mon, 12 Aug 2024 08:00:04 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id EBAFA6B0098; Mon, 12 Aug 2024 08:00:03 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id CA5A46B008C for ; Mon, 12 Aug 2024 08:00:03 -0400 (EDT) Received: from smtpin16.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 75808A8A1B for ; Mon, 12 Aug 2024 12:00:03 +0000 (UTC) X-FDA: 82443449886.16.794D22B Received: from mail-ej1-f41.google.com (mail-ej1-f41.google.com [209.85.218.41]) by imf23.hostedemail.com (Postfix) with ESMTP id 91264140033 for ; Mon, 12 Aug 2024 12:00:01 +0000 (UTC) Authentication-Results: imf23.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=jfmgsZgA; spf=pass (imf23.hostedemail.com: domain of ubizjak@gmail.com designates 209.85.218.41 as permitted sender) smtp.mailfrom=ubizjak@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1723463933; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=efqfVk4wKUu5y2Eyqs3KUsdZhsPQcKGa1y+niD5IcF0=; b=wCPTf1l+B/YsK1vOBSItnOuksOqfqsH6orgMUDyq/lhRl8DB1s+G8TdjhTpFf5aIj0RMfp DPnjbXS6iTS9RaHVf4R28MwWt5sS91vC/3gt6a13alT648vcOP7cWIZaO+C3p11T4eCU78 s3GNmxDDNjtmXEugtxq5TEOPgyzq9Xc= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1723463933; a=rsa-sha256; cv=none; b=t5ZfWtdosUtRmGYAPm5p4AFbBv54zNw4rGYSwbCpAugTf/Xl44E1V3A9AIgJBRW30K2WBN m02i/sTwMPnARw+1NGpthrlUaRUu2fx8LZDGu0zEDECgBsDyHvrvaCwyJq8srZ1MBfhSgn nBEkJmnMjsJ6hQINdo12ZE/Z5QhCNN4= ARC-Authentication-Results: i=1; imf23.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=jfmgsZgA; spf=pass (imf23.hostedemail.com: domain of ubizjak@gmail.com designates 209.85.218.41 as permitted sender) smtp.mailfrom=ubizjak@gmail.com; dmarc=pass (policy=none) header.from=gmail.com Received: by mail-ej1-f41.google.com with SMTP id a640c23a62f3a-a7d638a1f27so145922566b.2 for ; Mon, 12 Aug 2024 05:00:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1723463999; x=1724068799; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=efqfVk4wKUu5y2Eyqs3KUsdZhsPQcKGa1y+niD5IcF0=; b=jfmgsZgARIXY3RuWcMO7oJLlRlYvcLYEx1i0yeHP5J8Z/s3V4rRM8vK1NLnS5B3+b8 wQnHgbHwifBpJtzZhuFFPVAj7QUJqAPPuMKOcfpOpvcQa/KWfd+dz4eJ9ZKbx++w6nVI mvCHsWObKw+PJ/Si0ZGEIIKgFFfvPENsrZfaycVjTQGEZzB4skvbeUJHhRmOUuB9BOv+ 5YvX/Pgddw3DpuP7nPqOIkClpGF5SDpDbvuAgmynlMt+P+FZSvxY1xSkxSIfi4C+1nAC ar/7lAzz6NwSeugApiNmJinCqRPFutZTH1141p0u6LDg5bXJHmgsjqjHgXBLyZXvkxaD 9drg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1723463999; x=1724068799; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=efqfVk4wKUu5y2Eyqs3KUsdZhsPQcKGa1y+niD5IcF0=; b=rqy6ZI/BtOoRhzov9trd72VK3TuwkT42pXArUVQzBAof9rVAoZFUnJCCFyY1r+xHi9 v0A08aZ9GfUrtdhARk52+jkyvsmSwbD+FfMqvBFCSM0HkcPWsPNhpBn7ifAwHLZTPzs2 UfPkRiz+eoirXXlVuu8bExYrMh9oCJh3+BCOH95G0G4CVZELdZ6Tfm/kShkLkhZrFJPs OQb6UAGyIh2gBoKktxxv6948OpUA32O+ZxFplfxWmb/WBw4C2xehswWvzvoCuMnro7Jy SkRFGFbGn5c8YNGL0yPglh2GU0QbuWZ3d7ohrfOns3lQNvD6whP+b74rxCnOAmzUO3gH p52w== X-Gm-Message-State: AOJu0Yy86PcZcI8DQ0qlqbEv5/eN95BDp9LaeJ+4HzO8PB5UVE1bDq+k i+Gu9Lt24rfp/ZcvLnO1QD2NQrq3XvoUcYR+KgIne/ugZCBAzgmbzVieR+fgHw4= X-Google-Smtp-Source: AGHT+IGiz6BT46LDWsI4Kfhh8bbjUUZbNV/Nqu7ImfCdRuAQ78vk+RWaGtts3dHDU6gbsnhG/C8MMg== X-Received: by 2002:a05:6402:2347:b0:57d:669:caf2 with SMTP id 4fb4d7f45d1cf-5bd44c66b08mr14452a12.25.1723463999358; Mon, 12 Aug 2024 04:59:59 -0700 (PDT) Received: from fedora.iskraemeco.si ([193.77.86.250]) by smtp.gmail.com with ESMTPSA id 4fb4d7f45d1cf-5bd1a5e04aesm2055970a12.76.2024.08.12.04.59.58 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 12 Aug 2024 04:59:58 -0700 (PDT) From: Uros Bizjak To: linux-mm@kvack.org, linux-kernel@vger.kernel.org Cc: Uros Bizjak , Dennis Zhou , Tejun Heo , Christoph Lameter , Andy Lutomirski , Ingo Molnar , Nadav Amit , Brian Gerst , Denys Vlasenko , "H. Peter Anvin" , Linus Torvalds , Peter Zijlstra , Thomas Gleixner , Borislav Petkov , Luc Van Oostenryck Subject: [RFC PATCH v2 2/4] percpu: Assorted fixes found by strict percpu address space checks Date: Mon, 12 Aug 2024 13:57:36 +0200 Message-ID: <20240812115945.484051-3-ubizjak@gmail.com> X-Mailer: git-send-email 2.46.0 In-Reply-To: <20240812115945.484051-1-ubizjak@gmail.com> References: <20240812115945.484051-1-ubizjak@gmail.com> MIME-Version: 1.0 X-Rspamd-Queue-Id: 91264140033 X-Stat-Signature: 53qqry68cegqkj4fdy5i3w1jnkzx54hi X-Rspamd-Server: rspam09 X-Rspam-User: X-HE-Tag: 1723464001-934196 X-HE-Meta: U2FsdGVkX18iA74+WGbVZjt0b6NjpNdlflehhOG6D2Wt43TMiWZSgKsM1LkBeZObqAhm0AIRCKdkbsMc79r/peb0cXknN/wBtRozplq8hAU/YKguBndnay+vtXEG5G32UCcBMdRm+ju8M72upOsGcIHCISSLU2/75qQkIP/+W9KKrSVT4rWSFtgZq1P72Ku7cOm9zDf2UHNcRSI6sPRKdUTHfBuHF0XLhZRpOsh17DXyYfuG4HPXFNtWL35BjpoqzrgOFbbgsaQXOkmJEeqosNqMdjWr3UFHTHaiyE0e87HFIw0zVvKZBcGjwzRb4BajrnWHrlgP8GlemaSRYvmRkQ/rHfe47L6zbBjfiWSyMDWGujuKP6Z5cXD1fdTjHOj2YZ4IMvo1BNGW0zG4NU0rqOGeA/YJ+p8sTOnTFzveYt3XL/UyVaFl1DfkA7UO7izU4DeMOE2HOzjDsM3bcDYgQ/ijaDM01QWG0gSiV+cIE3PyCmYCHIGNHz24LV/wiX7Vvga7z1RUeeIJHNTBtpP65cL5R0Ypsaj7x5fH/xxmT1zws5QryqHGzOP6fEJGHI7qke7OrleiVIf+nL9YQOO2nChxQW5DgdmQ+bjIqFRARcY/2F1fZ3OHSkUy4qATdJIfjJKynlXJmpYu5K+PbV40vPqHR1ucJGdOahnh4yTC1EXd/6B+ZsWPnjBmqtn8tdfx9htc7eiSkDcgA3NYQA8Zam8pg40tfxtn0L2SaDD/p/Olj19Uera8lX4QQPf8XKMzVXiO3IaWjoBWhVxXZRzz3SPVhhEDkX39B8gK4wcjS47DJRXl0Iwk6SLCb2ERUk6QjZ9UcxGzs+smIYW1fjwWnYNlNVYkQBI2kpQZDVL+oSFEvxlXh8HL5rkGMFnK9p3n6/Ct+CfGi5pyekxF/7Laz0Vrma7SgbOesSkrmHxHGze8usY2I6KuwJCRMiOTTNag70ji4LBQNvVdJxvIicb rH9ZC/KQ NnITI3tSYV2z+kIk7gI0ewl/CsqkwF48O6T90muK7jyGvxMlkStlhgtRALmaAmL9dk6kyq2Hi5jBl9rCg68Xqao+H3auT9bOrXlJnd9NxpNI4vLNAQE+0QMIJMOFxtSYdK6ELfaC2tHF+rcrLDI2cXARqsmnZ+WLDeckuWTniZO/twDBpTiG0C7vVqcR2sqfyh9mUMJWmhOseMeJllSc+C28vyhdZ6u+XWICNs0Y+KUuKrxv048sOeXaMIOePLhgp7LL/TiWIAtK+YjIMdok9dXW+vW+5N2JWAKV12kQzUXahNeHGZBw0j5rZRzJp3nTAr3HspRlf/xMcHxW6hPYV318h9lDryjMG8vklVr8AtnzrqnyRtK2PD1P/PZB8h7YA+AkUI00yTnFTngAVq6xZkdoDdZdZ3tT8G7P6D4GcLVD2wJZ2QH0yMH68j6vEYQcpgWWRBT0n947fB7OSLz4TCkYeu7FAlZB6n4+XAldPgvH2IBkbiuAqD28ggEifPkQKFilI3lo8zzlwNfF09XNVH83+yKQ1IocZb3LhpmVaxeukZX1CMnXOhNShrmB5u7Px/FA6yvQbPLZFTjCSj1uIlr2UJa8yICbXEIRhbJiTBsoA48c4I0i9rDU46mKri5ZmQp07hwMy90qgwJ7AxjHCyohY6T3gJ35Yz94WzpfyTcf9tyQdfoPCnHqVfcA8KCy6kr9DahAkd5lRYCVov9zIfwNB2gVeAy/m05TuCETd9V52a0c= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Assorted fixes to prevent defconfig build failures when strict percpu address space checks will be enabled. These show effeciveness of strict percpu address space checks. Signed-off-by: Uros Bizjak Cc: Dennis Zhou Cc: Tejun Heo Cc: Christoph Lameter Cc: Andy Lutomirski Cc: Ingo Molnar Cc: Nadav Amit Cc: Brian Gerst Cc: Denys Vlasenko Cc: H. Peter Anvin Cc: Linus Torvalds Cc: Peter Zijlstra Cc: Thomas Gleixner Cc: Borislav Petkov Cc: Luc Van Oostenryck --- arch/x86/kernel/head64.c | 3 ++- drivers/base/devres.c | 2 +- fs/aio.c | 2 +- include/linux/cleanup.h | 4 ++-- include/linux/prandom.h | 1 + kernel/events/hw_breakpoint.c | 4 ++-- kernel/workqueue.c | 2 +- lib/percpu_counter.c | 2 +- net/core/dev.c | 2 +- 9 files changed, 12 insertions(+), 10 deletions(-) diff --git a/arch/x86/kernel/head64.c b/arch/x86/kernel/head64.c index a817ed0724d1..f5d6ad351cc4 100644 --- a/arch/x86/kernel/head64.c +++ b/arch/x86/kernel/head64.c @@ -560,9 +560,10 @@ void early_setup_idt(void) void __head startup_64_setup_gdt_idt(void) { void *handler = NULL; + struct desc_struct *gdt = (struct desc_struct *)(uintptr_t)init_per_cpu_var(gdt_page.gdt); struct desc_ptr startup_gdt_descr = { - .address = (unsigned long)&RIP_REL_REF(init_per_cpu_var(gdt_page.gdt)), + .address = (unsigned long)&RIP_REL_REF(*gdt), .size = GDT_SIZE - 1, }; diff --git a/drivers/base/devres.c b/drivers/base/devres.c index a2ce0ead06a6..894f5a1e6d18 100644 --- a/drivers/base/devres.c +++ b/drivers/base/devres.c @@ -1231,6 +1231,6 @@ void devm_free_percpu(struct device *dev, void __percpu *pdata) * devm_free_pages() does. */ WARN_ON(devres_release(dev, devm_percpu_release, devm_percpu_match, - (__force void *)pdata)); + (__force void *)(uintptr_t)pdata)); } EXPORT_SYMBOL_GPL(devm_free_percpu); diff --git a/fs/aio.c b/fs/aio.c index 6066f64967b3..e8920178b50f 100644 --- a/fs/aio.c +++ b/fs/aio.c @@ -100,7 +100,7 @@ struct kioctx { unsigned long user_id; - struct __percpu kioctx_cpu *cpu; + struct kioctx_cpu __percpu *cpu; /* * For percpu reqs_available, number of slots we move to/from global diff --git a/include/linux/cleanup.h b/include/linux/cleanup.h index d9e613803df1..f7e1158cbacf 100644 --- a/include/linux/cleanup.h +++ b/include/linux/cleanup.h @@ -154,7 +154,7 @@ static inline class_##_name##_t class_##_name##ext##_constructor(_init_args) \ #define DEFINE_GUARD(_name, _type, _lock, _unlock) \ DEFINE_CLASS(_name, _type, if (_T) { _unlock; }, ({ _lock; _T; }), _type _T); \ static inline void * class_##_name##_lock_ptr(class_##_name##_t *_T) \ - { return *_T; } + { return (void *)(uintptr_t)*_T; } #define DEFINE_GUARD_COND(_name, _ext, _condlock) \ EXTEND_CLASS(_name, _ext, \ @@ -211,7 +211,7 @@ static inline void class_##_name##_destructor(class_##_name##_t *_T) \ \ static inline void *class_##_name##_lock_ptr(class_##_name##_t *_T) \ { \ - return _T->lock; \ + return (void *)(uintptr_t)_T->lock; \ } diff --git a/include/linux/prandom.h b/include/linux/prandom.h index f7f1e5251c67..f2ed5b72b3d6 100644 --- a/include/linux/prandom.h +++ b/include/linux/prandom.h @@ -10,6 +10,7 @@ #include #include +#include #include struct rnd_state { diff --git a/kernel/events/hw_breakpoint.c b/kernel/events/hw_breakpoint.c index 6c2cb4e4f48d..d82fe78f0658 100644 --- a/kernel/events/hw_breakpoint.c +++ b/kernel/events/hw_breakpoint.c @@ -849,7 +849,7 @@ register_wide_hw_breakpoint(struct perf_event_attr *attr, cpu_events = alloc_percpu(typeof(*cpu_events)); if (!cpu_events) - return (void __percpu __force *)ERR_PTR(-ENOMEM); + return (void __percpu __force *)(uintptr_t)ERR_PTR(-ENOMEM); cpus_read_lock(); for_each_online_cpu(cpu) { @@ -868,7 +868,7 @@ register_wide_hw_breakpoint(struct perf_event_attr *attr, return cpu_events; unregister_wide_hw_breakpoint(cpu_events); - return (void __percpu __force *)ERR_PTR(err); + return (void __percpu __force *)(uintptr_t)ERR_PTR(err); } EXPORT_SYMBOL_GPL(register_wide_hw_breakpoint); diff --git a/kernel/workqueue.c b/kernel/workqueue.c index 1745ca788ede..32729a2e93af 100644 --- a/kernel/workqueue.c +++ b/kernel/workqueue.c @@ -377,7 +377,7 @@ struct workqueue_struct { /* hot fields used during command issue, aligned to cacheline */ unsigned int flags ____cacheline_aligned; /* WQ: WQ_* flags */ - struct pool_workqueue __percpu __rcu **cpu_pwq; /* I: per-cpu pwqs */ + struct pool_workqueue __rcu * __percpu *cpu_pwq; /* I: per-cpu pwqs */ struct wq_node_nr_active *node_nr_active[]; /* I: per-node nr_active */ }; diff --git a/lib/percpu_counter.c b/lib/percpu_counter.c index 51bc5246986d..3d0613ac7e73 100644 --- a/lib/percpu_counter.c +++ b/lib/percpu_counter.c @@ -209,7 +209,7 @@ int __percpu_counter_init_many(struct percpu_counter *fbc, s64 amount, INIT_LIST_HEAD(&fbc[i].list); #endif fbc[i].count = amount; - fbc[i].counters = (void *)counters + (i * counter_size); + fbc[i].counters = (void __percpu *)counters + (i * counter_size); debug_percpu_counter_activate(&fbc[i]); } diff --git a/net/core/dev.c b/net/core/dev.c index 751d9b70e6ad..5cad88cf029c 100644 --- a/net/core/dev.c +++ b/net/core/dev.c @@ -10860,7 +10860,7 @@ noinline void netdev_core_stats_inc(struct net_device *dev, u32 offset) return; } - field = (__force unsigned long __percpu *)((__force void *)p + offset); + field = (unsigned long __percpu *)(void __percpu *)(p + offset); this_cpu_inc(*field); } EXPORT_SYMBOL_GPL(netdev_core_stats_inc); From patchwork Mon Aug 12 11:57:37 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Uros Bizjak X-Patchwork-Id: 13760453 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5797FC52D7C for ; Mon, 12 Aug 2024 12:00:11 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 6CFC86B0092; Mon, 12 Aug 2024 08:00:05 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 656A16B009A; Mon, 12 Aug 2024 08:00:05 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 433E16B009E; Mon, 12 Aug 2024 08:00:05 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 1C8A16B0092 for ; Mon, 12 Aug 2024 08:00:05 -0400 (EDT) Received: from smtpin13.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id C8C131A1D8C for ; Mon, 12 Aug 2024 12:00:04 +0000 (UTC) X-FDA: 82443449928.13.779A63C Received: from mail-ej1-f46.google.com (mail-ej1-f46.google.com [209.85.218.46]) by imf27.hostedemail.com (Postfix) with ESMTP id E2AEC40030 for ; Mon, 12 Aug 2024 12:00:02 +0000 (UTC) Authentication-Results: imf27.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b="PuA/uU0n"; spf=pass (imf27.hostedemail.com: domain of ubizjak@gmail.com designates 209.85.218.46 as permitted sender) smtp.mailfrom=ubizjak@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1723463934; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=JJLe5IUAL1/xG0v/qE20AqL4y9xZWwRXykIQj0IeARs=; b=OPiXDN5pFWqwd9NkEaz+KuFxV3AUOyki7eCSUSOaMduXXKNc3TDT9Knme0JWPeeQNxAo0+ Eqo2RkMwHRRnRONuEetYbmbQcXtqj9ZsWOgEyUx2IrkNY6dqDoripW7BaqCDMOFCKteeFe C0s8K4OVo7U2PRBF1LScABVJ+jqqfbI= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1723463934; a=rsa-sha256; cv=none; b=Te6UlLD+4FXR6MMd6lectwoF3Ym1m62GUu9eHd/J7Jr6KG+PGfWU0HlHPI04NZoRdB2zjf 5e/chyjatslqGUL70sxPEu+PfqDTW7CdbmCclZ5bfvZRAZRnyazmUJcceu840XhJx6ngmU dwMlb5M4FcU5FDL5ll3A32emfbYXUfU= ARC-Authentication-Results: i=1; imf27.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b="PuA/uU0n"; spf=pass (imf27.hostedemail.com: domain of ubizjak@gmail.com designates 209.85.218.46 as permitted sender) smtp.mailfrom=ubizjak@gmail.com; dmarc=pass (policy=none) header.from=gmail.com Received: by mail-ej1-f46.google.com with SMTP id a640c23a62f3a-a7ab63a388bso139964666b.1 for ; Mon, 12 Aug 2024 05:00:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1723464001; x=1724068801; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=JJLe5IUAL1/xG0v/qE20AqL4y9xZWwRXykIQj0IeARs=; b=PuA/uU0nzKHyPYZLI7v7KLmqKPhW68bWQScsHwIFYvCruYt1RnhkC8pJv3aVHbXus5 X3CNxMLNAbvbTyeVjBolJMdHKA2gcmo7DClccudJOdE222phlpwg/wvjhX3wZlNxS0NY rAxytcN+gaXWjoEpANURUeVch5KKS+XRMFtnWikPti97GXRP79ov/nb4zWLquGUMcZ// gXP0vThpYhC+WOLJtUytKj7q86PRVl26M5k6wCVqnoi1TYzpI2TjACanq0FBP90Rdxzn +44cvz3a27HBqVSfzdvENtQGYKj+JX/JNJ2vZByUpnxg2Fr8HihoZysqZjVZVFzY+xck nJyA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1723464001; x=1724068801; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=JJLe5IUAL1/xG0v/qE20AqL4y9xZWwRXykIQj0IeARs=; b=kkP48wvB0Yq2QHANvy/dx4IZGYFRq61JuHfL8mjaMnUxZuUc/sDxFrsTEX6SLtq71A 4ooHdIUntTqyPMbCmqnWp78/ekJztgm9g4jzZw6hg1jWkAGTKVI6v2K/eXi4rng0TfUQ HaXnr2gypiaCSzeunbEWFxb1v8++RRleUCzwsYTf31oNHRjy4rjOhhdD27WHHnj9oUG5 Bmv9J7yUWwbyJm/Dvrl7w7lIHFqCf5e9WrQ4SRT3F3cu6LFdwEQ7dMNhyE+wGx9CdrKt msPh61XCQon9ma42YKBjhgnu6rWcBxeZ1KHCfjkygBRZ4+aW8vJj9SbBXl5Yvspa7EmI TGoA== X-Gm-Message-State: AOJu0YyeCsMrrgm7zfK7Bxq08LhGumnyw2LVmcN8Hy1/tpLIYJXEhNBF V0FlcnNbPoT7rB5N9DSUVmGODBwX2bsfjvZEOiPYG+63UdxsArZ8KQrHk+ep65s= X-Google-Smtp-Source: AGHT+IHLf0wX3sTf+xZ73mLemXQoM3mULwR0RMBbNFiKu9Xs4bJxq0l9b69j/UaSYylfIRYCAo7wzA== X-Received: by 2002:a05:6402:3552:b0:5bb:9ae0:4a49 with SMTP id 4fb4d7f45d1cf-5bd44c66a3amr11213a12.30.1723464000709; Mon, 12 Aug 2024 05:00:00 -0700 (PDT) Received: from fedora.iskraemeco.si ([193.77.86.250]) by smtp.gmail.com with ESMTPSA id 4fb4d7f45d1cf-5bd1a5e04aesm2055970a12.76.2024.08.12.04.59.59 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 12 Aug 2024 05:00:00 -0700 (PDT) From: Uros Bizjak To: linux-mm@kvack.org, linux-kernel@vger.kernel.org Cc: Uros Bizjak , Dennis Zhou , Tejun Heo , Christoph Lameter , Andy Lutomirski , Ingo Molnar , Nadav Amit , Brian Gerst , Denys Vlasenko , "H. Peter Anvin" , Linus Torvalds , Peter Zijlstra , Thomas Gleixner , Borislav Petkov , Luc Van Oostenryck Subject: [RFC PATCH v2 3/4] percpu: Repurpose __percpu tag as a named address space qualifier Date: Mon, 12 Aug 2024 13:57:37 +0200 Message-ID: <20240812115945.484051-4-ubizjak@gmail.com> X-Mailer: git-send-email 2.46.0 In-Reply-To: <20240812115945.484051-1-ubizjak@gmail.com> References: <20240812115945.484051-1-ubizjak@gmail.com> MIME-Version: 1.0 X-Rspamd-Queue-Id: E2AEC40030 X-Stat-Signature: 7i5xtkeook7hnjmnu6ew1oqkk9dj5q1d X-Rspamd-Server: rspam09 X-Rspam-User: X-HE-Tag: 1723464002-41746 X-HE-Meta: U2FsdGVkX1/8kmiiGKzIpx0XXiQSuavO9poWUJYlVj4oR162JK687LFAW9ovQT0wVAvhla9OtQtJeDne6+4bOevUtCkVQ/6NpOoz4z1OU4041HTJYwSUjGvLa0s+Q7zxbD4mEnJ6+okwE3vzKNM1+RJUc/LDxddKFGqlGW4uOQdBlj0Tvi/wrE5zyCoRuBB5d5j1phQsLZu2x0+uWqXVKeO5rB8eHTqiJ1tLMCQOjthUstBPDXAkCmcX3a+3NEMYt7PFgKG8mAgYB5m/DNxqgcRJb9olRXbs4wXb5wLkh7skkrpFh84hNBNPAQwDIfjPlk8/zx59cxbl78bs+JRueeF6QX/0/yrr1bffk36I1ElF3g4l04gYh7ccPYTzQRKe2E1+AXSCeaHguTwJdI3cRYTsZJS4LlWv7ib/iRq+1/zJdsPBuU8Q2y/lzZ60FFES4UUG8SuNSrmZUPasvgTR48aYx8qa1dKAHOZg4D8jdkaP0h8kT5UKF0IPuAgpvRRhBZGrtdlX9z0EEGX8D43qZFjQ6rl5B5jugR9/1EhomLrceRV8YBO3+wIo/XwzC5HrBJYHVKIaFlwI3t28nA3Yf42M4QeR8MwlvwMs07JqrSt9nPbFIkQ9+PA3vzs9kdB/97TmwbXEjJOLK+5ATKDhxFbUmeXI8f64nk5xLpRaHy9lgEYUkQ8diLurn5s4BiIpOXo13b4YyjsA3lQSw7RBPeea5Ld4AKDOw7L71MTVXGEKAvHV3kx/PbtkGZtM0tQjZIAA/VhOeAsLvyIE9DyFjBs05c1BE7/AAn0uBFlaMmpDbnjCPRiZIduhTwglyoSXsWiXinJE1zB3qufiLS67qZGTbf4atf+1Ei9jZ6rO4iy6L8d/YymjyXqvlMi43bQ6SEyjmuJPEa3F/9LERoKboLQF6zmG3v9dVengjBJ+QBDeS0mCP5ps/sq3Po8WlEDDTEioJjSyk6K8X15TBKw mkPzgk+2 0Al+MXHXCy4DvcUAaouepB/j90HdMOoTtZrAUpZGyfdNJAFkJhXTz2Qqf+725v3+Alfpw+V5d4U05f705tVmG4Vy2OKeCWaZ3c+T9ZD6LJh+BoUOWkl/ZNLA2+Wr+qy7A7wRLG8gMyE8Yxr3tn4g+F4CUjxj07y46idRmxnX/MN/2lnYOP4ydnG/LFtpSlvJzZqyRZyasAVS3yt3kWxXjMIi+PRTMxFG28ILgtKG6WTEejok81sgchpBBuTRHNAhRJ+dwFSabnJIXnXAfRNj2qzmVyVNgDaD1tnER6hlUnYZyy9ZQQFfe4cOvaqa5RvGpUnsljeqf/M3t014mkIGADPcs9I6V15VUBXKhjWmv/W6QQc2hKboU7DkN3vTSngOt6XJJvxgK9T65JcDThEw5I+X+A1ge4caycfF7PLPWB0wSarHEaI4goyxV7ZZ+FYXoBnPK1yGdo1zei4vA+4jrVEtS3CDqoxWh+L1kNYfA9czcks7CWbJLPHt3qUHR/VHpiwNYAPFiZZgs0J0nGEUk8Pz599Mu5B4WMef8QM9K/1/FQpZopu+JIygAMDxbwavaoHPZwmySDvXd9gj8dnkVW7Cj6+CSTMIfmMdGU2sTAaW2G5XW17hLk6ZHh0+PrtNERwppnqZNGfUxs6Zkt+ETfzl7CEzRKqh2FNXH5nUo/fo996cmrT9KYUfUJjvV969eqiTZzRDR6lWAJlVkxW8fo3HBLZ5btMa45mqj0qp/WZZFTrlQkMxUUJtffVTRc0y+q8/ny4AZV5jdGEN80VBTJHzJkoLvQcAVxNBq X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: The patch introduces per_cpu_qual define and repurposes __percpu tag as a named address space qualifier using the new define. Arches can now conditionally define __per_cpu_qual as their named address space qualifier for percpu variables. Signed-off-by: Uros Bizjak Cc: Dennis Zhou Cc: Tejun Heo Cc: Christoph Lameter Cc: Andy Lutomirski Cc: Ingo Molnar Cc: Nadav Amit Cc: Brian Gerst Cc: Denys Vlasenko Cc: H. Peter Anvin Cc: Linus Torvalds Cc: Peter Zijlstra Cc: Thomas Gleixner Cc: Borislav Petkov Cc: Luc Van Oostenryck --- include/asm-generic/percpu.h | 15 +++++++++++++++ include/linux/compiler_types.h | 2 +- 2 files changed, 16 insertions(+), 1 deletion(-) diff --git a/include/asm-generic/percpu.h b/include/asm-generic/percpu.h index c091e09046c5..5f85d7ba8e2d 100644 --- a/include/asm-generic/percpu.h +++ b/include/asm-generic/percpu.h @@ -6,6 +6,21 @@ #include #include +/* + * per_cpu_qual is the qualifier for the percpu named address space. + * + * Most arches use generic named address space for percpu variables but + * some arches define percpu variables in different named address space + * (on the x86 arch, percpu variables may be declared as being relative + * to the %fs or %gs segments using __seg_fs or __seg_gs named address + * space qualifiers). + */ +#ifdef __per_cpu_qual +#define per_cpu_qual __per_cpu_qual +#else +#define per_cpu_qual /* nothing */ +#endif + #ifdef CONFIG_SMP /* diff --git a/include/linux/compiler_types.h b/include/linux/compiler_types.h index f14c275950b5..4c5917b97bc3 100644 --- a/include/linux/compiler_types.h +++ b/include/linux/compiler_types.h @@ -57,7 +57,7 @@ static inline void __chk_io_ptr(const volatile void __iomem *ptr) { } # define __user BTF_TYPE_TAG(user) # endif # define __iomem -# define __percpu BTF_TYPE_TAG(percpu) +# define __percpu per_cpu_qual BTF_TYPE_TAG(percpu) # define __rcu BTF_TYPE_TAG(rcu) # define __chk_user_ptr(x) (void)0 From patchwork Mon Aug 12 11:57:38 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Uros Bizjak X-Patchwork-Id: 13760454 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id E22DCC52D7C for ; Mon, 12 Aug 2024 12:00:14 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 4D1FC6B009E; Mon, 12 Aug 2024 08:00:08 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 482C96B009F; Mon, 12 Aug 2024 08:00:08 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 2AE216B00A0; Mon, 12 Aug 2024 08:00:08 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 0B2AD6B009E for ; Mon, 12 Aug 2024 08:00:08 -0400 (EDT) Received: from smtpin01.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id ACD78C1CEA for ; Mon, 12 Aug 2024 12:00:07 +0000 (UTC) X-FDA: 82443450054.01.852064F Received: from mail-lj1-f178.google.com (mail-lj1-f178.google.com [209.85.208.178]) by imf16.hostedemail.com (Postfix) with ESMTP id 8F17618000D for ; Mon, 12 Aug 2024 12:00:05 +0000 (UTC) Authentication-Results: imf16.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=MD3eaSLq; spf=pass (imf16.hostedemail.com: domain of ubizjak@gmail.com designates 209.85.208.178 as permitted sender) smtp.mailfrom=ubizjak@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1723463951; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=x3uJRiNLLgOcZsmmzORMPDWc5fHy9y5CyFJXSzVXE+M=; b=bqz3X3P3SaaEnFXQ2kPZpayGqk+O6E5fBMR1D+aNrFSMFnKYti3KYEkDuuFKfgJXRrpEvo XjXVsduZnud2pJutzYYoVaUYNJhC6rrFXBzI3gig+AGAUhzy8u1yEHyRXBHZEevfnQO87g ne1nJDldsHnMW6Pji9PLlGaxmFlHs9s= ARC-Authentication-Results: i=1; imf16.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=MD3eaSLq; spf=pass (imf16.hostedemail.com: domain of ubizjak@gmail.com designates 209.85.208.178 as permitted sender) smtp.mailfrom=ubizjak@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1723463951; a=rsa-sha256; cv=none; b=tSW1PVz583zpkszl7lHWe6H3BwquxKzbr9hmfgik6r98lVNbn2BEUKkMILiR0T/2+fgO8f juDt3A/fQNSO2xz4jbSk8xDbJ4Nfo4Fo7DbY/w3LeWm0KIm32Le4BjdEVLWLr2mi5z2dfX Nx3lFabCmlIODS7zMNb8MA+5T35cukw= Received: by mail-lj1-f178.google.com with SMTP id 38308e7fff4ca-2f15790b472so53121271fa.0 for ; Mon, 12 Aug 2024 05:00:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1723464003; x=1724068803; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=x3uJRiNLLgOcZsmmzORMPDWc5fHy9y5CyFJXSzVXE+M=; b=MD3eaSLqDKVEAkKjm8GtrAykzPJ/o0+QdCIvqybhhkx8dQmz86WbiHebKE5+w12YaR qGb1VrXGcmLyLh764X5eNnjpKVgx8xcY0ZpTD16IUfAis1tFp+0/AW7QrGYmtZdaqO4y B6uYqdx8fUBKsROEprR38QSzGJO1yziU0TZSYVbLj0NzwYRlor8bq2zBrIiGIqvxRoGN ZHgp52TO/8DAz4jvKdGB+i1kMFA6hXH1GOiDVlx4Wf/eLN9080YPDVn39T1D/xVBn9al xn4tmYw1gjXefZ9uMdSDqadvUhjsDLUL7MkFs5x8L6DEj526SyV94Egi+LjssqYNk/rf ot7Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1723464003; x=1724068803; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=x3uJRiNLLgOcZsmmzORMPDWc5fHy9y5CyFJXSzVXE+M=; b=TFsvR/Ttw47I0n2HoezRgASbNs53C5UgTn3jOteS6Ojm/ea1xh6Txtsmj4NbGWIPVj wVpumlMNWeYCi1ztcXhCwkRG8ePq9Hdq497vtmZb3ci7BMLulhUEPa3jLVI+CtMcyRyq 24yh0G01XZ5Dlm2uP/7UQXr4wdjVwFIXjZCllj8A6dYNdnk7263IR8Egr1tAg88bSdrn svcdKlbw4rMuH8u+VDtqg7+XW08U6rAwSI1jOpwuaZiAnhbHnAFw1nFaLlYk1p6pWoYQ Yd4YRIH5mZbmxgwrAbQ9sTJoxrdIIK0GEHOgpnudqKGR6CrPHEtWaLMldlTubfbRvP+t nq3A== X-Gm-Message-State: AOJu0Yy3Z10yl8Hq6BEd5xk3XUqoMjOV96fjaPzMcowXSP9ycnlCO0Ha KeKQb1hIOTHg9R24VOgHj9O907AYXkMc13/O4dcNQQmstx6Rt6JTIGt/p/Q+bRo= X-Google-Smtp-Source: AGHT+IHKlXKj0B7JY5kmC2Q2zZaBnmA29zjjoQCSZ3jusm9LFJTTfxdKLemTAAxWJakoGZgPOzvHiQ== X-Received: by 2002:a05:651c:545:b0:2ef:c281:54f7 with SMTP id 38308e7fff4ca-2f1a6cf52a4mr60754621fa.37.1723464002929; Mon, 12 Aug 2024 05:00:02 -0700 (PDT) Received: from fedora.iskraemeco.si ([193.77.86.250]) by smtp.gmail.com with ESMTPSA id 4fb4d7f45d1cf-5bd1a5e04aesm2055970a12.76.2024.08.12.05.00.00 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 12 Aug 2024 05:00:01 -0700 (PDT) From: Uros Bizjak To: linux-mm@kvack.org, linux-kernel@vger.kernel.org Cc: Uros Bizjak , Dennis Zhou , Tejun Heo , Christoph Lameter , Andy Lutomirski , Ingo Molnar , Nadav Amit , Brian Gerst , Denys Vlasenko , "H. Peter Anvin" , Linus Torvalds , Peter Zijlstra , Thomas Gleixner , Borislav Petkov , Luc Van Oostenryck Subject: [RFC PATCH v2 4/4] percpu/x86: Enable strict percpu checks via named AS qualifiers Date: Mon, 12 Aug 2024 13:57:38 +0200 Message-ID: <20240812115945.484051-5-ubizjak@gmail.com> X-Mailer: git-send-email 2.46.0 In-Reply-To: <20240812115945.484051-1-ubizjak@gmail.com> References: <20240812115945.484051-1-ubizjak@gmail.com> MIME-Version: 1.0 X-Stat-Signature: qc43may7wtocbo5h9o9ch91puq41o4o8 X-Rspam-User: X-Rspamd-Queue-Id: 8F17618000D X-Rspamd-Server: rspam02 X-HE-Tag: 1723464005-202441 X-HE-Meta: U2FsdGVkX1/uaxCZxbaZsTzovdPdzgMysiyR7iLF027QPpKPAtoHFhjz0vqnEMHa8AZLOEOcVDazfQkex4gQo39zxZcBivNpK3V8PLaYcZpFvE25sim5HV+TwofA9+J+mOyaeXHP9EkaNeNEe/ttCMBoBVF14IBAJtoexDwDiQGn7E1XhooQJhPxb/3SeTyXhsVGGTRpq2KIl+16ymCEN2VsNG5ltwMyuOFivLRhtplzQidsoOxz+G7TxmKhuLd/l8es26bM6OYUKszWyuq/Ri2TgiM8y6WbSLOczDIIu5gzYZl4z0uvprjGMX+BnEg0xblbFXUkxnhjOkizB7Ss7fxhS26GBD/yY2eByrpa0oEyixVPK5hoHqBbnf9JgzqoctZ1tIXW3hjgvnp/i2e5JLlOMqYZFIiRP8aXpDNeUxDqvThwMb/rshP++HHSrubWSTRp1ofqLC6k1v4hq8GB23EKd97OmTB5x3KPrauFgdsh+jVQMbudP/8IIuH3V1unwnn1Lk8wUMKI7woXf+96lP9nkJp1mBofzsczgG/9N0ChzFafguDpTa1nVR0rpaPHWxLjfnwi1rX9rtwR2jov4S5CFhio3+3vS/Fgb/pDnw95zJ1RzscJHGimf5CGQH/xj++IK8WfsKQwOpDo7g7hxR+bBTpfN7g5LPmFebxjiA/akk447LuE2ZbK7YMpvWpy3t10NROUkK3MMWEaoemuIU6fK6c9elzUv4nvkQA7qa4lgRgV5uF68KNOdnuxi7RY+AFhc4H2JIarbQfQwTxCMah8QazwTl3g6WPdHAMNyvoFsW8wAlWT0jRK4KeexlX+Gv2EO2k9ogKINZqoD7LlcnCKG2MM72ehwZH3iaGq/jx5KCuwMnJ42d6rHDJCxQe+YVrklfuhUf7rmaVeM/9eaT1ion/AOe106OFwep87LIoC8wN3nrmP9P8ZO+cRfymLxrWtY6md7KO1tIpHz3w sPNAC9O5 1UfXYEgHtUNBBtBVWXqxB2kVA2zhcGkNRA3sXSnLzmu8xa76/NZ4PUz6k10jbHMVMiErV3v4K3bomrnIgjGNoawShbPvc6y6LEk2A4/u3JNvhJlExuSCpfVSlFXEbGygqgW9YtZqoq8v+X1TS6vamef9OA+rQ0+CbYt3kefVhDkvltk1sVXyh3/iDgmcdP48mYQglj2ZPfaRlkiBik/U8EvyH8y6kAjkGZEjqfBurWdPiAg6ZaFXhOwqd+9Rjd5DJ3t1BzK35yuOOb01uerv04BqzgAh+b94b6Z5Nrb1MAqzmgdKUUqhbxHUE791shU+vHNs+SPBegVh1qXRAb2SPq4DlIO7/SKq8C7WlDPmniIwyKNqqW/B3+yA/zBltcOje20UJNNPBTzEOX3b/8FZv87cGbAEMVsSBbYtV036wx8NYyjpP0tY61CZMFM+ZASgEbPwTuvl133OyPkPHJ2qFgYq/TjdiEFqIpkIHvbU64NOsapBz5bmm1A2K0MO5JObEx6AJsUdjJPjCctDVqHf1Rl0XXia00HGyvRskjLGyRAX9lWMGnituHrBoU7Zv5UAzLcbnz+Rqqlft530QWEP4ot5QRGZzOwo3lDzLAMYZ+6syRd9O5stHumTdk7jE/DCAsceAQxkU2vUHP1+vfStw/MGGrwgzD5UHQbQBx45agGieoXAZIjQ7uRODbJHcqc2GR/NKvuqluHDdTuzjL08+DtuO8vAWQeiihv8duT8ADXGxjJ4= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Declare percpu variables in __seg_gs/__seg_fs named AS and keep them named AS qualified until they are dereferenced with percpu accessor. This approach enables various compiler check for cross-namespace variable assignments. Signed-off-by: Uros Bizjak Cc: Dennis Zhou Cc: Tejun Heo Cc: Christoph Lameter Cc: Andy Lutomirski Cc: Ingo Molnar Cc: Nadav Amit Cc: Brian Gerst Cc: Denys Vlasenko Cc: H. Peter Anvin Cc: Linus Torvalds Cc: Peter Zijlstra Cc: Thomas Gleixner Cc: Borislav Petkov Cc: Luc Van Oostenryck --- arch/x86/include/asm/percpu.h | 15 ++++++++++++--- 1 file changed, 12 insertions(+), 3 deletions(-) diff --git a/arch/x86/include/asm/percpu.h b/arch/x86/include/asm/percpu.h index 4ab4ad60d767..b54119398c4b 100644 --- a/arch/x86/include/asm/percpu.h +++ b/arch/x86/include/asm/percpu.h @@ -95,9 +95,18 @@ #endif /* CONFIG_SMP */ -#define __my_cpu_type(var) typeof(var) __percpu_seg_override -#define __my_cpu_ptr(ptr) (__my_cpu_type(*(ptr))*)(__force uintptr_t)(ptr) -#define __my_cpu_var(var) (*__my_cpu_ptr(&(var))) +#if defined(CONFIG_USE_X86_SEG_SUPPORT) && defined(CONFIG_CC_HAS_TYPEOF_UNQUAL) +# define __my_cpu_type(var) typeof(var) +# define __my_cpu_ptr(ptr) (ptr) +# define __my_cpu_var(var) (var) + +# define __per_cpu_qual __percpu_seg_override +#else +# define __my_cpu_type(var) typeof(var) __percpu_seg_override +# define __my_cpu_ptr(ptr) (__my_cpu_type(*(ptr))*)(__force uintptr_t)(ptr) +# define __my_cpu_var(var) (*__my_cpu_ptr(&(var))) +#endif + #define __percpu_arg(x) __percpu_prefix "%" #x #define __force_percpu_arg(x) __force_percpu_prefix "%" #x