From patchwork Wed Jan 16 19:03:12 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ilya Dryomov X-Patchwork-Id: 10766829 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id C59D414E5 for ; Wed, 16 Jan 2019 19:03:40 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id B69FA2F4FF for ; Wed, 16 Jan 2019 19:03:40 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id AA8F82F501; Wed, 16 Jan 2019 19:03:40 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id AEE2C2F4F0 for ; Wed, 16 Jan 2019 19:03:39 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729509AbfAPTDi (ORCPT ); Wed, 16 Jan 2019 14:03:38 -0500 Received: from mail-wm1-f67.google.com ([209.85.128.67]:38118 "EHLO mail-wm1-f67.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729330AbfAPTDh (ORCPT ); Wed, 16 Jan 2019 14:03:37 -0500 Received: by mail-wm1-f67.google.com with SMTP id m22so3210352wml.3 for ; Wed, 16 Jan 2019 11:03:36 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id; bh=1KFcSq5RH2VzurKZGSLmjImDfLDfr7IwGlrb8u6r/i0=; b=lDgzaPoF8pA3U288EHUrauPhgeQ3eyaxVH4SG7dbdtmm85v7R0ybGuk4aBNTQHGzbR /WEBVuojvPrRW3aJJQjjDAiY32r8sscsFZG7QztYFkGkjPGSWRFbqDiBt9MdSzoSDpPy badsRzLVhohEO87wruitBlhm75WZ0ZRi2jGQ1wMv0ViqzU11evsbHgCS9rnYWeNdC65m pUy3CClg4E2mHH7O4uvkWt1haeGbGB2Bhfp12tIZBg8DZh0N5y4aaBpLz8fSfVmJZC1E 89aWxXOQGBx6YP2j3WrhPcrWIc4vjri+udBccrlaqoqlyG5xi20d/vdcWXQYDAtx7+A1 k/6w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id; bh=1KFcSq5RH2VzurKZGSLmjImDfLDfr7IwGlrb8u6r/i0=; b=JPxQhjsgl/EdzYk9j3IV8g3ev916wiUZ3aGxn4jbUsoBG7FRNJza8zXxnygdAWTZXb zniQ7syLNlIH7AEMbS0wXVjt13nUuXza4W79Qwx+Ps3zNyeapTj4Paslyl5Le2WCemaM oBIkvXwiNcBZ8L/5VC8X7s7AV6dGO2idAy1uU76ilVAza6nPfge9xGBbmUS0oauIqb8/ fBXBZEVVrX5v/cwsYiKwOd9ttl4vN3PfdvX9qC+T2dqdVlDGE6PtrUrsJtDE1GcuIMp/ yHImZvUGY1Ki8BmVnyhtd8ovZvpYzbI1MfiPL8VvrEw0xAqpzJt9ejnK1or7OCqi8QeM BmnA== X-Gm-Message-State: AJcUukfRPCrE5XBDP/jiDIWDQQY4dlPmxHZz/shrPICGGZyZzFfXnugs i80nrWDmepaYWkSIe684FPklPU+r X-Google-Smtp-Source: ALg8bN47vZ/SQ0pQmJ0lDYiaaFCdKM43GTrIDAyyPJArN8Ik1CCAsfoM9Dgc09ISwGTDG5kg3i+Kjg== X-Received: by 2002:a1c:ba89:: with SMTP id k131mr8564072wmf.85.1547665415110; Wed, 16 Jan 2019 11:03:35 -0800 (PST) Received: from orange.redhat.com (ip-89-102-68-132.net.upcbroadband.cz. [89.102.68.132]) by smtp.gmail.com with ESMTPSA id v1sm111260785wrw.90.2019.01.16.11.03.34 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Wed, 16 Jan 2019 11:03:34 -0800 (PST) From: Ilya Dryomov To: ceph-devel@vger.kernel.org Cc: Myungho Jung Subject: [PATCH] libceph: avoid KEEPALIVE_PENDING races in ceph_con_keepalive() Date: Wed, 16 Jan 2019 20:03:12 +0100 Message-Id: <20190116190312.26843-1-idryomov@gmail.com> X-Mailer: git-send-email 2.14.4 Sender: ceph-devel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: ceph-devel@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP con_fault() can transition the connection into STANDBY right after ceph_con_keepalive() clears STANDBY in clear_standby(): libceph user thread ceph-msgr worker ceph_con_keepalive() mutex_lock(&con->mutex) clear_standby(con) mutex_unlock(&con->mutex) mutex_lock(&con->mutex) con_fault() ... if KEEPALIVE_PENDING isn't set set state to STANDBY ... mutex_unlock(&con->mutex) set KEEPALIVE_PENDING set WRITE_PENDING This triggers warnings in clear_standby() when either ceph_con_send() or ceph_con_keepalive() get to clearing STANDBY next time. I don't see a reason to condition queue_con() call on the previous value of KEEPALIVE_PENDING, so move the setting of KEEPALIVE_PENDING into the critical section -- unlike WRITE_PENDING, KEEPALIVE_PENDING could have been a non-atomic flag. Reported-by: syzbot+acdeb633f6211ccdf886@syzkaller.appspotmail.com Signed-off-by: Ilya Dryomov Tested-by: Myungho Jung --- net/ceph/messenger.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/net/ceph/messenger.c b/net/ceph/messenger.c index d5718284db57..3661cdd927f1 100644 --- a/net/ceph/messenger.c +++ b/net/ceph/messenger.c @@ -3206,9 +3206,10 @@ void ceph_con_keepalive(struct ceph_connection *con) dout("con_keepalive %p\n", con); mutex_lock(&con->mutex); clear_standby(con); + con_flag_set(con, CON_FLAG_KEEPALIVE_PENDING); mutex_unlock(&con->mutex); - if (con_flag_test_and_set(con, CON_FLAG_KEEPALIVE_PENDING) == 0 && - con_flag_test_and_set(con, CON_FLAG_WRITE_PENDING) == 0) + + if (con_flag_test_and_set(con, CON_FLAG_WRITE_PENDING) == 0) queue_con(con); } EXPORT_SYMBOL(ceph_con_keepalive);