From patchwork Thu Oct 17 14:13:01 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Marco Elver X-Patchwork-Id: 11196103 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 199B814DB for ; Thu, 17 Oct 2019 14:13:44 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id CD6D0222C1 for ; Thu, 17 Oct 2019 14:13:43 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="AgCEZD8U" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org CD6D0222C1 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id E848C8E000A; Thu, 17 Oct 2019 10:13:40 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id DE4DA8E0003; Thu, 17 Oct 2019 10:13:40 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id CF9878E000A; Thu, 17 Oct 2019 10:13:40 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0034.hostedemail.com [216.40.44.34]) by kanga.kvack.org (Postfix) with ESMTP id AA0CB8E0003 for ; Thu, 17 Oct 2019 10:13:40 -0400 (EDT) Received: from smtpin16.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with SMTP id 43E348024E1E for ; Thu, 17 Oct 2019 14:13:40 +0000 (UTC) X-FDA: 76053469800.16.beef94_5a32129b30f34 X-Spam-Summary: 2,0,0,e69da42abf94fcc9,d41d8cd98f00b204,3eneoxqukcju3ak3g5dd5a3.1dba7cjm-bb9kz19.dg5@flex--elver.bounces.google.com,:elver@google.com:akiyks@gmail.com:stern@rowland.harvard.edu:glider@google.com:parri.andrea@gmail.com:andreyknvl@google.com:luto@kernel.org:ard.biesheuvel@linaro.org:arnd@arndb.de:boqun.feng@gmail.com:bp@alien8.de:dja@axtens.net:dlustig@nvidia.com:dave.hansen@linux.intel.com:dhowells@redhat.com:dvyukov@google.com:hpa@zytor.com:mingo@redhat.com:j.alglave@ucl.ac.uk:joel@joelfernandes.org:corbet@lwn.net:jpoimboe@redhat.com:luc.maranget@inria.fr:mark.rutland@arm.com:npiggin@gmail.com:paulmck@linux.ibm.com:peterz@infradead.org:tglx@linutronix.de:will@kernel.org:kasan-dev@googlegroups.com:linux-arch@vger.kernel.org:linux-doc@vger.kernel.org:linux-efi@vger.kernel.org:linux-kbuild@vger.kernel.org:linux-kernel@vger.kernel.org::x86@kernel.org,RULES_HIT:41:152:355:379:541:800:960:973:988:989:1260:1277:1313:1314:1345:1359:1431:1437:1516:1518:1535:1544:1593:1 594:1605 X-HE-Tag: beef94_5a32129b30f34 X-Filterd-Recvd-Size: 8025 Received: from mail-qt1-f202.google.com (mail-qt1-f202.google.com [209.85.160.202]) by imf11.hostedemail.com (Postfix) with ESMTP for ; Thu, 17 Oct 2019 14:13:39 +0000 (UTC) Received: by mail-qt1-f202.google.com with SMTP id d6so2436367qtn.2 for ; Thu, 17 Oct 2019 07:13:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=ip3P4i20fQEf4b5wirUedHF2574zJ+yd56tKHujm1bg=; b=AgCEZD8UVe6mWYXJb+txOu8iRAlpncqlSJC3w4AIm7e5AqUthlTntb/9CDvD2rB0dW a5FPKOjzCIVSVSZjUZMDO3Iad458Xn4cW7enROkYh/uBUhQdsH8m6IBNmVie1+ZoMIxm nctxfK0RkfHydReBwe5q1RhO6Tkl9ZNcOqRjsZODJWd5RADfElv2yDMaftj0eecg//jd oGXHVtKMwq+vS1cAJH58HLbXwzlMpZ3VpIFmGkphK3Z4W1pls2nsD9bqM8TtdMK0677B 4Tv6nEw0zI1BAjQLGfp6t0Hrhhb04f32p5H67aN0U+QsDvcmS3cySEvWhTnLlZN5+3lv qr4Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=ip3P4i20fQEf4b5wirUedHF2574zJ+yd56tKHujm1bg=; b=iN5cQECn3ZQMrYyXSh/+EP+YeqjcUIN6eqcIZq+v+HaZsQwvt0LSn9vxfqOM3wtdhX faCLhVbvkRlVRYzFp+gzeKr07I4DqgNwxJpEWbPshDfkDRhcYjYp0pr+9DM8z4TtckaO dbbxVaswLx5nh/tVSVS/ruNFaMcdXuKRfe7+oGC8O2uAcCbR6hC5k1//e9OrkbREjDzK HHuVkx3OqrCDNPiqee4e3cqydjoy1iJE96gvXayN3gRTIKDirYLQcxFlNV8qPckLd1/3 A28BipuDpFTqQl2b9PYaOpqzkSQ7fkVFvUc+AezRXVEl2umP9fz393AKin9jFz6OcC89 D9Nw== X-Gm-Message-State: APjAAAVMHLL9giFdvYfK5RLCoZA4Q2IdW6xRDYyJqyXZD5r3hhZwyW80 wo3pGoplGCtcgLt7gqKlAIW8CSawUg== X-Google-Smtp-Source: APXvYqzrcnGW1lI+Gxy2rLvKkrrRrqs3J9SSrFvhiW4X6PylUpt1XSF1crDZlHI52IRZwqkUyyn7a2hahw== X-Received: by 2002:a0c:ef85:: with SMTP id w5mr4041664qvr.159.1571321618815; Thu, 17 Oct 2019 07:13:38 -0700 (PDT) Date: Thu, 17 Oct 2019 16:13:01 +0200 In-Reply-To: <20191017141305.146193-1-elver@google.com> Message-Id: <20191017141305.146193-5-elver@google.com> Mime-Version: 1.0 References: <20191017141305.146193-1-elver@google.com> X-Mailer: git-send-email 2.23.0.866.gb869b98d4c-goog Subject: [PATCH v2 4/8] seqlock, kcsan: Add annotations for KCSAN From: Marco Elver To: elver@google.com Cc: akiyks@gmail.com, stern@rowland.harvard.edu, glider@google.com, parri.andrea@gmail.com, andreyknvl@google.com, luto@kernel.org, ard.biesheuvel@linaro.org, arnd@arndb.de, boqun.feng@gmail.com, bp@alien8.de, dja@axtens.net, dlustig@nvidia.com, dave.hansen@linux.intel.com, dhowells@redhat.com, dvyukov@google.com, hpa@zytor.com, mingo@redhat.com, j.alglave@ucl.ac.uk, joel@joelfernandes.org, corbet@lwn.net, jpoimboe@redhat.com, luc.maranget@inria.fr, mark.rutland@arm.com, npiggin@gmail.com, paulmck@linux.ibm.com, peterz@infradead.org, tglx@linutronix.de, will@kernel.org, kasan-dev@googlegroups.com, linux-arch@vger.kernel.org, linux-doc@vger.kernel.org, linux-efi@vger.kernel.org, linux-kbuild@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, x86@kernel.org X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Since seqlocks in the Linux kernel do not require the use of marked atomic accesses in critical sections, we teach KCSAN to assume such accesses are atomic. KCSAN currently also pretends that writes to `sequence` are atomic, although currently plain writes are used (their corresponding reads are READ_ONCE). Further, to avoid false positives in the absence of clear ending of a seqlock reader critical section (only when using the raw interface), KCSAN assumes a fixed number of accesses after start of a seqlock critical section are atomic. Signed-off-by: Marco Elver --- include/linux/seqlock.h | 44 +++++++++++++++++++++++++++++++++++++---- 1 file changed, 40 insertions(+), 4 deletions(-) diff --git a/include/linux/seqlock.h b/include/linux/seqlock.h index bcf4cf26b8c8..1e425831a7ed 100644 --- a/include/linux/seqlock.h +++ b/include/linux/seqlock.h @@ -37,8 +37,24 @@ #include #include #include +#include #include +/* + * The seqlock interface does not prescribe a precise sequence of read + * begin/retry/end. For readers, typically there is a call to + * read_seqcount_begin() and read_seqcount_retry(), however, there are more + * esoteric cases which do not follow this pattern. + * + * As a consequence, we take the following best-effort approach for *raw* usage + * of seqlocks under KCSAN: upon beginning a seq-reader critical section, + * pessimistically mark then next KCSAN_SEQLOCK_REGION_MAX memory accesses as + * atomics; if there is a matching read_seqcount_retry() call, no following + * memory operations are considered atomic. Non-raw usage of seqlocks is not + * affected. + */ +#define KCSAN_SEQLOCK_REGION_MAX 1000 + /* * Version using sequence counter only. * This can be used when code has its own mutex protecting the @@ -115,6 +131,7 @@ static inline unsigned __read_seqcount_begin(const seqcount_t *s) cpu_relax(); goto repeat; } + kcsan_atomic_next(KCSAN_SEQLOCK_REGION_MAX); return ret; } @@ -131,6 +148,7 @@ static inline unsigned raw_read_seqcount(const seqcount_t *s) { unsigned ret = READ_ONCE(s->sequence); smp_rmb(); + kcsan_atomic_next(KCSAN_SEQLOCK_REGION_MAX); return ret; } @@ -183,6 +201,7 @@ static inline unsigned raw_seqcount_begin(const seqcount_t *s) { unsigned ret = READ_ONCE(s->sequence); smp_rmb(); + kcsan_atomic_next(KCSAN_SEQLOCK_REGION_MAX); return ret & ~1; } @@ -202,7 +221,8 @@ static inline unsigned raw_seqcount_begin(const seqcount_t *s) */ static inline int __read_seqcount_retry(const seqcount_t *s, unsigned start) { - return unlikely(s->sequence != start); + kcsan_atomic_next(0); + return unlikely(READ_ONCE(s->sequence) != start); } /** @@ -225,6 +245,7 @@ static inline int read_seqcount_retry(const seqcount_t *s, unsigned start) static inline void raw_write_seqcount_begin(seqcount_t *s) { + kcsan_begin_atomic(true); s->sequence++; smp_wmb(); } @@ -233,6 +254,7 @@ static inline void raw_write_seqcount_end(seqcount_t *s) { smp_wmb(); s->sequence++; + kcsan_end_atomic(true); } /** @@ -262,18 +284,20 @@ static inline void raw_write_seqcount_end(seqcount_t *s) * * void write(void) * { - * Y = true; + * WRITE_ONCE(Y, true); * * raw_write_seqcount_barrier(seq); * - * X = false; + * WRITE_ONCE(X, false); * } */ static inline void raw_write_seqcount_barrier(seqcount_t *s) { + kcsan_begin_atomic(true); s->sequence++; smp_wmb(); s->sequence++; + kcsan_end_atomic(true); } static inline int raw_read_seqcount_latch(seqcount_t *s) @@ -398,7 +422,9 @@ static inline void write_seqcount_end(seqcount_t *s) static inline void write_seqcount_invalidate(seqcount_t *s) { smp_wmb(); + kcsan_begin_atomic(true); s->sequence+=2; + kcsan_end_atomic(true); } typedef struct { @@ -430,11 +456,21 @@ typedef struct { */ static inline unsigned read_seqbegin(const seqlock_t *sl) { - return read_seqcount_begin(&sl->seqcount); + unsigned ret = read_seqcount_begin(&sl->seqcount); + + kcsan_atomic_next(0); /* non-raw usage, assume closing read_seqretry */ + kcsan_begin_atomic(false); + return ret; } static inline unsigned read_seqretry(const seqlock_t *sl, unsigned start) { + /* + * Assume not nested: read_seqretry may be called multiple times when + * completing read critical section. + */ + kcsan_end_atomic(false); + return read_seqcount_retry(&sl->seqcount, start); }