From patchwork Mon Nov 4 14:27:41 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Marco Elver X-Patchwork-Id: 11225925 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id BB8861390 for ; Mon, 4 Nov 2019 14:29:09 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 7E2DE217F4 for ; Mon, 4 Nov 2019 14:29:09 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="XpBlnjzT" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 7E2DE217F4 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id D00FB6B0010; Mon, 4 Nov 2019 09:29:07 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id C5DED6B0266; Mon, 4 Nov 2019 09:29:07 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B75106B0269; Mon, 4 Nov 2019 09:29:07 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0063.hostedemail.com [216.40.44.63]) by kanga.kvack.org (Postfix) with ESMTP id 9C16D6B0010 for ; Mon, 4 Nov 2019 09:29:07 -0500 (EST) Received: from smtpin18.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with SMTP id 37D9E441C for ; Mon, 4 Nov 2019 14:29:07 +0000 (UTC) X-FDA: 76118827134.18.rake22_6caf8c4ad2017 X-Spam-Summary: 2,0,0,dbba9eecd6191017,d41d8cd98f00b204,3stxaxqukcbiw3dw9y66y3w.u64305cf-442dsu2.69y@flex--elver.bounces.google.com,:elver@google.com:akiyks@gmail.com:stern@rowland.harvard.edu:glider@google.com:parri.andrea@gmail.com:andreyknvl@google.com:luto@kernel.org:ard.biesheuvel@linaro.org:arnd@arndb.de:boqun.feng@gmail.com:bp@alien8.de:dja@axtens.net:dlustig@nvidia.com:dave.hansen@linux.intel.com:dhowells@redhat.com:dvyukov@google.com:hpa@zytor.com:mingo@redhat.com:j.alglave@ucl.ac.uk:joel@joelfernandes.org:corbet@lwn.net:jpoimboe@redhat.com:luc.maranget@inria.fr:mark.rutland@arm.com:npiggin@gmail.com:paulmck@kernel.org:peterz@infradead.org:tglx@linutronix.de:will@kernel.org:kasan-dev@googlegroups.com:linux-arch@vger.kernel.org:linux-doc@vger.kernel.org:linux-efi@vger.kernel.org:linux-kbuild@vger.kernel.org:linux-kernel@vger.kernel.org::x86@kernel.org,RULES_HIT:2:41:152:355:379:541:800:960:968:973:988:989:1260:1277:1313:1314:1345:1359:1431:1437:1516:1518:1535:1593:159 4:1605:1 X-HE-Tag: rake22_6caf8c4ad2017 X-Filterd-Recvd-Size: 9056 Received: from mail-vs1-f74.google.com (mail-vs1-f74.google.com [209.85.217.74]) by imf10.hostedemail.com (Postfix) with ESMTP for ; Mon, 4 Nov 2019 14:29:06 +0000 (UTC) Received: by mail-vs1-f74.google.com with SMTP id z20so2797588vsf.1 for ; Mon, 04 Nov 2019 06:29:06 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=PnlUsFEddsbKHz2GK+uTFYcWeErcIfLobMYOTI3aPFU=; b=XpBlnjzTP/JJfoBTfESYVVCyfq0Y28LdV7cBuhtg6jKBy/OVCuDgML+AEak4DEOggE 4FFwufuSvYhyjQrp6YEdsb+0R9547mXSl11TpqE3J5Gfy2l3SeOKD0jHn1XcAkiVjRsU zFnVpDB/HqpAYH+HlBDp/56aQqgVWbG1whyOoHDFClGVo0d7GgzNehqkJPuug6YEWUX4 GESn0vSshFS4E7Pnv2PBS6OtYbR8LbeNZq2cO3cLxo5KUh9JJuHlMVjnJo2PCOa88WDn E2GYi5/xuLjLSp4mmnif68wguqCwwnHfg6wwkzO66h3CRHGlahAtjZeHjE89Qv4ROYOz wGdg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=PnlUsFEddsbKHz2GK+uTFYcWeErcIfLobMYOTI3aPFU=; b=VFxuR8yRGegLi3xFL04je6AZfPCglHRkavAT7AFU8qUqwdvwsHvtPCaZ802Y2WkPfI pnBh/gcQpTOmWCeK/QhaE5NsY8cOC0GKpOvCJMHAz702g5gK4+kAjh3j+HZ6Zjsvtxru gdJoxKI6DUzAL5Iyl3p2w9/BUc9G2NSFapsddW55mmyauDyfpQjWgcK2plniu18l777R SEh58Y3+mVlxFZogJIgbn4NegdvAZxVemxADHrhpyjo0ET8AiNCRzWjphauap4v7qFLJ qmeN1aRpCC0MZhq0BgpFrEGKifS702sqFmPfOmVRQ1mh0l6gXgG1bAXjbvFpJeUWZunl 3Dzg== X-Gm-Message-State: APjAAAV8CDQC7uvXShK6i+p5G85BVxNEz+ShcGe5pXH3Hn+eWbZ76lHD G1Pw9VdARGgyLVv4CB/rSQAEu5vEUA== X-Google-Smtp-Source: APXvYqyxTcSSDIIgKVC9uqJiIVViZoCP8EgkGFOXLAR6swB3rAeT15fCv8Jgf2+FjQb1O2mN+n5pn+lMvg== X-Received: by 2002:a1f:3249:: with SMTP id y70mr4259895vky.31.1572877745747; Mon, 04 Nov 2019 06:29:05 -0800 (PST) Date: Mon, 4 Nov 2019 15:27:41 +0100 In-Reply-To: <20191104142745.14722-1-elver@google.com> Message-Id: <20191104142745.14722-6-elver@google.com> Mime-Version: 1.0 References: <20191104142745.14722-1-elver@google.com> X-Mailer: git-send-email 2.24.0.rc1.363.gb1bccd3e3d-goog Subject: [PATCH v3 5/9] seqlock, kcsan: Add annotations for KCSAN From: Marco Elver To: elver@google.com Cc: akiyks@gmail.com, stern@rowland.harvard.edu, glider@google.com, parri.andrea@gmail.com, andreyknvl@google.com, luto@kernel.org, ard.biesheuvel@linaro.org, arnd@arndb.de, boqun.feng@gmail.com, bp@alien8.de, dja@axtens.net, dlustig@nvidia.com, dave.hansen@linux.intel.com, dhowells@redhat.com, dvyukov@google.com, hpa@zytor.com, mingo@redhat.com, j.alglave@ucl.ac.uk, joel@joelfernandes.org, corbet@lwn.net, jpoimboe@redhat.com, luc.maranget@inria.fr, mark.rutland@arm.com, npiggin@gmail.com, paulmck@kernel.org, peterz@infradead.org, tglx@linutronix.de, will@kernel.org, kasan-dev@googlegroups.com, linux-arch@vger.kernel.org, linux-doc@vger.kernel.org, linux-efi@vger.kernel.org, linux-kbuild@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, x86@kernel.org X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Since seqlocks in the Linux kernel do not require the use of marked atomic accesses in critical sections, we teach KCSAN to assume such accesses are atomic. KCSAN currently also pretends that writes to `sequence` are atomic, although currently plain writes are used (their corresponding reads are READ_ONCE). Further, to avoid false positives in the absence of clear ending of a seqlock reader critical section (only when using the raw interface), KCSAN assumes a fixed number of accesses after start of a seqlock critical section are atomic. === Commentary on design around absence of clear begin/end markings === Seqlock usage via seqlock_t follows a predictable usage pattern, where clear critical section begin/end is enforced. With subtle special cases for readers needing to be flat atomic regions, e.g. because usage such as in: - fs/namespace.c:__legitimize_mnt - unbalanced read_seqretry - fs/dcache.c:d_walk - unbalanced need_seqretry But, anything directly accessing seqcount_t seems to be unpredictable. Filtering for usage of read_seqcount_retry not following 'do { .. } while (read_seqcount_retry(..));': $ git grep 'read_seqcount_retry' | grep -Ev 'while \(|seqlock.h|Doc|\* ' => about 1/3 of the total read_seqcount_retry usage. Just looking at fs/namei.c, we conclude that it is non-trivial to prescribe and migrate to an interface that would force clear begin/end seqlock markings for critical sections. As such, we concluded that the best design currently, is to simply ensure that KCSAN works well with the existing code. Signed-off-by: Marco Elver --- v3: * Remove comment from raw_seqcount_barrier that should have been in next patch. * Renamed kcsan_{nestable,flat}_atomic_{begin,end} * Elaborate why clear begin/end cannot be enforced easily. --- include/linux/seqlock.h | 40 ++++++++++++++++++++++++++++++++++++++-- 1 file changed, 38 insertions(+), 2 deletions(-) diff --git a/include/linux/seqlock.h b/include/linux/seqlock.h index bcf4cf26b8c8..61232bc223fd 100644 --- a/include/linux/seqlock.h +++ b/include/linux/seqlock.h @@ -37,8 +37,24 @@ #include #include #include +#include #include +/* + * The seqlock interface does not prescribe a precise sequence of read + * begin/retry/end. For readers, typically there is a call to + * read_seqcount_begin() and read_seqcount_retry(), however, there are more + * esoteric cases which do not follow this pattern. + * + * As a consequence, we take the following best-effort approach for raw usage + * via seqcount_t under KCSAN: upon beginning a seq-reader critical section, + * pessimistically mark then next KCSAN_SEQLOCK_REGION_MAX memory accesses as + * atomics; if there is a matching read_seqcount_retry() call, no following + * memory operations are considered atomic. Usage of seqlocks via seqlock_t + * interface is not affected. + */ +#define KCSAN_SEQLOCK_REGION_MAX 1000 + /* * Version using sequence counter only. * This can be used when code has its own mutex protecting the @@ -115,6 +131,7 @@ static inline unsigned __read_seqcount_begin(const seqcount_t *s) cpu_relax(); goto repeat; } + kcsan_atomic_next(KCSAN_SEQLOCK_REGION_MAX); return ret; } @@ -131,6 +148,7 @@ static inline unsigned raw_read_seqcount(const seqcount_t *s) { unsigned ret = READ_ONCE(s->sequence); smp_rmb(); + kcsan_atomic_next(KCSAN_SEQLOCK_REGION_MAX); return ret; } @@ -183,6 +201,7 @@ static inline unsigned raw_seqcount_begin(const seqcount_t *s) { unsigned ret = READ_ONCE(s->sequence); smp_rmb(); + kcsan_atomic_next(KCSAN_SEQLOCK_REGION_MAX); return ret & ~1; } @@ -202,7 +221,8 @@ static inline unsigned raw_seqcount_begin(const seqcount_t *s) */ static inline int __read_seqcount_retry(const seqcount_t *s, unsigned start) { - return unlikely(s->sequence != start); + kcsan_atomic_next(0); + return unlikely(READ_ONCE(s->sequence) != start); } /** @@ -225,6 +245,7 @@ static inline int read_seqcount_retry(const seqcount_t *s, unsigned start) static inline void raw_write_seqcount_begin(seqcount_t *s) { + kcsan_nestable_atomic_begin(); s->sequence++; smp_wmb(); } @@ -233,6 +254,7 @@ static inline void raw_write_seqcount_end(seqcount_t *s) { smp_wmb(); s->sequence++; + kcsan_nestable_atomic_end(); } /** @@ -271,9 +293,11 @@ static inline void raw_write_seqcount_end(seqcount_t *s) */ static inline void raw_write_seqcount_barrier(seqcount_t *s) { + kcsan_nestable_atomic_begin(); s->sequence++; smp_wmb(); s->sequence++; + kcsan_nestable_atomic_end(); } static inline int raw_read_seqcount_latch(seqcount_t *s) @@ -398,7 +422,9 @@ static inline void write_seqcount_end(seqcount_t *s) static inline void write_seqcount_invalidate(seqcount_t *s) { smp_wmb(); + kcsan_nestable_atomic_begin(); s->sequence+=2; + kcsan_nestable_atomic_end(); } typedef struct { @@ -430,11 +456,21 @@ typedef struct { */ static inline unsigned read_seqbegin(const seqlock_t *sl) { - return read_seqcount_begin(&sl->seqcount); + unsigned ret = read_seqcount_begin(&sl->seqcount); + + kcsan_atomic_next(0); /* non-raw usage, assume closing read_seqretry */ + kcsan_flat_atomic_begin(); + return ret; } static inline unsigned read_seqretry(const seqlock_t *sl, unsigned start) { + /* + * Assume not nested: read_seqretry may be called multiple times when + * completing read critical section. + */ + kcsan_flat_atomic_end(); + return read_seqcount_retry(&sl->seqcount, start); }