From patchwork Tue Apr 7 03:10:42 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11477421 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 1FA5992A for ; Tue, 7 Apr 2020 03:10:46 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id D3C8C20769 for ; Tue, 7 Apr 2020 03:10:45 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="T056HuXr" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org D3C8C20769 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 9DDE98E0083; Mon, 6 Apr 2020 23:10:44 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 98D628E0062; Mon, 6 Apr 2020 23:10:44 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8CA778E0083; Mon, 6 Apr 2020 23:10:44 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0168.hostedemail.com [216.40.44.168]) by kanga.kvack.org (Postfix) with ESMTP id 7350E8E0062 for ; Mon, 6 Apr 2020 23:10:44 -0400 (EDT) Received: from smtpin01.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id 3F224A8C4 for ; Tue, 7 Apr 2020 03:10:44 +0000 (UTC) X-FDA: 76679581608.01.art05_660210c9a143 X-Spam-Summary: 13,1.2,0,41023112d3a1f90e,d41d8cd98f00b204,akpm@linux-foundation.org,,RULES_HIT:2:41:69:355:379:800:960:967:973:988:989:1260:1263:1345:1359:1381:1431:1437:1535:1605:1606:1730:1747:1777:1792:2194:2198:2199:2200:2393:2525:2553:2559:2563:2682:2685:2859:2898:2902:2933:2937:2939:2942:2945:2947:2951:2954:3022:3138:3139:3140:3141:3142:3865:3866:3867:3868:3870:3871:3872:3874:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:4117:4321:5007:6261:6653:6737:7576:7875:7904:8599:8957:9010:9012:9025:9389:9545:10008:10913:11026:11658:11914:12043:12048:12291:12296:12297:12438:12517:12519:12555:12679:12683:12783:12986:13172:13229:21080:21451:21627:21939:21972:21990:30029:30054:30070:30090,0,RBL:198.145.29.99:@linux-foundation.org:.lbl8.mailshell.net-62.2.0.100 64.100.201.201,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_rules:0:1:0,LFtime:124,LUA_SUMMARY:none X-HE-Tag: art05_660210c9a143 X-Filterd-Recvd-Size: 6843 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf43.hostedemail.com (Postfix) with ESMTP for ; Tue, 7 Apr 2020 03:10:43 +0000 (UTC) Received: from localhost.localdomain (c-73-231-172-41.hsd1.ca.comcast.net [73.231.172.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 95E24206B8; Tue, 7 Apr 2020 03:10:42 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1586229043; bh=+L3zYw4pehU5L4OgkeHb5v2q1Y3IAihsXIdnBEB0DLM=; h=Date:From:To:Subject:In-Reply-To:From; b=T056HuXraZJA9gljN5AbrZYgvg97bMYfLG9sQhulgIG6e16lV6ti5u3tesh1jnDCx aLGqkCxbEiq4vkyPqLR4tFtzJleeoFyGN9hQDByHxlMz1FB/2XSdd46PdFpTMbgBkC rWRfwr25Qmq453EUeugzoK7JGDjz929Qg/E8unWo= Date: Mon, 06 Apr 2020 20:10:42 -0700 From: Andrew Morton To: akpm@linux-foundation.org, chris@chris-wilson.co.uk, David.Laight@ACULAB.COM, elver@google.com, linux-mm@kvack.org, mark.rutland@arm.com, mm-commits@vger.kernel.org, paulmck@kernel.org, rdunlap@infradead.org, stable@vger.kernel.org, torvalds@linux-foundation.org Subject: [patch 125/166] lib/list: prevent compiler reloads inside 'safe' list iteration Message-ID: <20200407031042.8o-fYMox-%akpm@linux-foundation.org> In-Reply-To: <20200406200254.a69ebd9e08c4074e41ddebaf@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Chris Wilson Subject: lib/list: prevent compiler reloads inside 'safe' list iteration Instruct the compiler to read the next element in the list iteration once, and that it is not allowed to reload the value from the stale element later. This is important as during the course of the safe iteration, the stale element may be poisoned (unbeknownst to the compiler). This helps prevent kcsan warnings over 'unsafe' conduct in releasing the list elements during list_for_each_entry_safe() and friends. Link: http://lkml.kernel.org/r/20200310092119.14965-1-chris@chris-wilson.co.uk Signed-off-by: Chris Wilson Reviewed-by: Paul E. McKenney Cc: Randy Dunlap Cc: David Laight Cc: Mark Rutland Cc: Marco Elver Cc: Signed-off-by: Andrew Morton --- include/linux/list.h | 50 +++++++++++++++++++++++++++++------------ 1 file changed, 36 insertions(+), 14 deletions(-) --- a/include/linux/list.h~list-prevent-compiler-reloads-inside-safe-list-iteration +++ a/include/linux/list.h @@ -537,6 +537,17 @@ static inline void list_splice_tail_init list_entry((pos)->member.next, typeof(*(pos)), member) /** + * list_next_entry_safe - get the next element in list [once] + * @pos: the type * to cursor + * @member: the name of the list_head within the struct. + * + * Like list_next_entry() but prevents the compiler from reloading the + * next element. + */ +#define list_next_entry_safe(pos, member) \ + list_entry(READ_ONCE((pos)->member.next), typeof(*(pos)), member) + +/** * list_prev_entry - get the prev element in list * @pos: the type * to cursor * @member: the name of the list_head within the struct. @@ -545,6 +556,17 @@ static inline void list_splice_tail_init list_entry((pos)->member.prev, typeof(*(pos)), member) /** + * list_prev_entry_safe - get the prev element in list [once] + * @pos: the type * to cursor + * @member: the name of the list_head within the struct. + * + * Like list_prev_entry() but prevents the compiler from reloading the + * previous element. + */ +#define list_prev_entry_safe(pos, member) \ + list_entry(READ_ONCE((pos)->member.prev), typeof(*(pos)), member) + +/** * list_for_each - iterate over a list * @pos: the &struct list_head to use as a loop cursor. * @head: the head for your list. @@ -686,9 +708,9 @@ static inline void list_splice_tail_init */ #define list_for_each_entry_safe(pos, n, head, member) \ for (pos = list_first_entry(head, typeof(*pos), member), \ - n = list_next_entry(pos, member); \ + n = list_next_entry_safe(pos, member); \ &pos->member != (head); \ - pos = n, n = list_next_entry(n, member)) + pos = n, n = list_next_entry_safe(n, member)) /** * list_for_each_entry_safe_continue - continue list iteration safe against removal @@ -700,11 +722,11 @@ static inline void list_splice_tail_init * Iterate over list of given type, continuing after current point, * safe against removal of list entry. */ -#define list_for_each_entry_safe_continue(pos, n, head, member) \ - for (pos = list_next_entry(pos, member), \ - n = list_next_entry(pos, member); \ - &pos->member != (head); \ - pos = n, n = list_next_entry(n, member)) +#define list_for_each_entry_safe_continue(pos, n, head, member) \ + for (pos = list_next_entry(pos, member), \ + n = list_next_entry_safe(pos, member); \ + &pos->member != (head); \ + pos = n, n = list_next_entry_safe(n, member)) /** * list_for_each_entry_safe_from - iterate over list from current point safe against removal @@ -716,10 +738,10 @@ static inline void list_splice_tail_init * Iterate over list of given type from current point, safe against * removal of list entry. */ -#define list_for_each_entry_safe_from(pos, n, head, member) \ - for (n = list_next_entry(pos, member); \ - &pos->member != (head); \ - pos = n, n = list_next_entry(n, member)) +#define list_for_each_entry_safe_from(pos, n, head, member) \ + for (n = list_next_entry_safe(pos, member); \ + &pos->member != (head); \ + pos = n, n = list_next_entry_safe(n, member)) /** * list_for_each_entry_safe_reverse - iterate backwards over list safe against removal @@ -733,9 +755,9 @@ static inline void list_splice_tail_init */ #define list_for_each_entry_safe_reverse(pos, n, head, member) \ for (pos = list_last_entry(head, typeof(*pos), member), \ - n = list_prev_entry(pos, member); \ + n = list_prev_entry_safe(pos, member); \ &pos->member != (head); \ - pos = n, n = list_prev_entry(n, member)) + pos = n, n = list_prev_entry_safe(n, member)) /** * list_safe_reset_next - reset a stale list_for_each_entry_safe loop @@ -750,7 +772,7 @@ static inline void list_splice_tail_init * completing the current iteration of the loop body. */ #define list_safe_reset_next(pos, n, member) \ - n = list_next_entry(pos, member) + n = list_next_entry_safe(pos, member) /* * Double linked lists with a single pointer list head.