From patchwork Fri Jan 25 18:34:56 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bart Van Assche X-Patchwork-Id: 10781887 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 7C58B14E5 for ; Fri, 25 Jan 2019 18:35:47 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 6B2CA299B5 for ; Fri, 25 Jan 2019 18:35:47 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 5F4A22D40C; Fri, 25 Jan 2019 18:35:47 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.7 required=2.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id C611F299B5 for ; Fri, 25 Jan 2019 18:35:46 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727955AbfAYSfq (ORCPT ); Fri, 25 Jan 2019 13:35:46 -0500 Received: from com-out001.mailprotect.be ([83.217.72.83]:41727 "EHLO com-out001.mailprotect.be" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726689AbfAYSfq (ORCPT ); Fri, 25 Jan 2019 13:35:46 -0500 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=mailprotect.be; s=mail; h=Content-Transfer-Encoding:MIME-Version:References :In-Reply-To:Message-Id:Date:Subject:Cc:To:From:reply-to:sender:bcc: content-type; bh=lL01uxHK8llSqvPuKcOjVQW9cFx4eIVMlkl9Fg+a6fQ=; b=WliK3nTdW76J UZWW+ScetC6JIkN1qEkQ2rGevRQEmNf5kONsCXW9kFqacp58uxLPKGQMaKItTx50/EPWFtTO4Oe5h vhIYDlPtR7s9Xxr+QUHuzv0w1Akl6nV426izkP4QSeHUDBUESZTuyZGA5StIOyObKGq/+IZX4iycf +0AHQGaOT90hLnqy0V/ypgpFx6UsbRcMJIblEjMVrZA03LcWo9UQqAzgqC+Q58tyDVQbRLsjYzMgA /ZcMzeNmnzmXFu5EkizxHPVZ6M9ZAeLbcnhmFdW86g0OtZCV0Od+F+scq9oDGvvPNopr5BJh41JwN 09VDQMRO4v+TYweys12oEw==; Received: from smtp-auth.mailprotect.be ([178.208.39.155]) by com-mpt-out001.mailprotect.be with esmtp (Exim 4.89) (envelope-from ) id 1gn6KI-0009GE-BG; Fri, 25 Jan 2019 19:35:39 +0100 Received: from desktop-bart.svl.corp.google.com (unknown [104.133.8.89]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by smtp-auth.mailprotect.be (Postfix) with ESMTPSA id 5AA01C05E8; Fri, 25 Jan 2019 19:35:34 +0100 (CET) From: Bart Van Assche To: "Martin K . Petersen" Cc: Christoph Hellwig , target-devel@vger.kernel.org, Bart Van Assche , Nicholas Bellinger , Mike Christie , Hannes Reinecke , stable@vger.kernel.org Subject: [PATCH 14/16] target/iscsi: Avoid that iscsit_release_commands_from_conn() triggers a deadlock Date: Fri, 25 Jan 2019 10:34:56 -0800 Message-Id: <20190125183458.220477-15-bvanassche@acm.org> X-Mailer: git-send-email 2.20.1.495.gaa96b0ce6b-goog In-Reply-To: <20190125183458.220477-1-bvanassche@acm.org> References: <20190125183458.220477-1-bvanassche@acm.org> MIME-Version: 1.0 X-Originating-IP: 178.208.39.155 X-SpamExperts-Domain: mailprotect.be X-SpamExperts-Username: 178.208.39.128/27 Authentication-Results: mailprotect.be; auth=pass smtp.auth=178.208.39.128/27@mailprotect.be X-SpamExperts-Outgoing-Class: ham X-SpamExperts-Outgoing-Evidence: Combined (0.03) X-Recommended-Action: accept X-Filter-ID: EX5BVjFpneJeBchSMxfU5imjdJnrJZSiBidG5cRewel602E9L7XzfQH6nu9C/Fh9KJzpNe6xgvOx q3u0UDjvO1tLifGj39bI0bcPyaJsYTagCTS5H5Pie2sCQ3gGSzr06cM7U7xDGmYwNG93LVnxwxjk yGQYCd08t3lHobjxX/OigDGQOIiVUN5rGO3B5zfVpnsdsF0/474T9aStD4uEKVG9P9yTThxC66Ku Xb0HO3WlZa4xKvcBORT2AGmYYKfa2w4nwtOiKlIdftXY5+/HTLBPB4loq3vFboQ7gxpx2K5q243j OywFphhx6RtIX3/eBU519jxvHYi8yYUXqBwxztbjuU58k6ddZEUemp+2z71k67jEmAPDI71Tk0Cy k1CfTWUth/bIg1D/4cblfXFvIrwhcbIgt0qoJA65s+aV0TP5Yeia8HRRN1XgVEyTWHpXr3OnWExh FqqXKjnbUDBzcE+0HNyHhfqG0xcyQsZuCSP9hsa4s5F3MydIviotZvNHMvzpHD6s+1ao4iJ4PfBj 2N8Vdv96d+08iDufvobw6VUavl4snG+PRn+T8pp7vQV3rpLvOUh2w8gsgLX0uFSNx2AewJvG2VR5 mIGCzCE3N/asZm+FB8yckvFZiHWxnQEMkunODYtbvP72vAmOyjpz9UIDBHPtf27+c3J1HwicUV+G pS+C6BPAIEs8PiiVo3Eb3c7KClSBOi3EV0NWbVJ1KXxqWKxJiDvSXvZvXfo2UEYR9G8HIfDm0MPW TzQicTC9GplHcpVCCoX989hgB8R+yC9G9b3yazdC1eGXzxVaKphdoW8ELRMdd5tZtqdRHw0GmDVD n/TirW1Elu8hoWoocJTIqsriZLvaiGZnz91zoAMokdMyijyrNPLF2pjbdb3MosfYiA+A0wZZjhdL 0PpC7BEVniihuDwEGDcmr6e3OPQT0DVFoMDBp5PRUiQzcTDLKXYp8mSktA5RGpHjopTdfxOQd3Im gm6h3jLPIxOgGdnV6CvgdOXv8q3UV8vherSXctgzcDoFd+96Xw4QUNtTnccFhhwn2/Dj1pgXRdkG FXkx8yzZEloA4PGimr7c647Aza3muS2OYzrgPLHf0Rbu9udKEy7bP5gIt23U7fpAnNPRUBWWBSDR x5OeloKLHRs0g0z5EfZrz6E5xburfSl74HBEn95eECKi3gxjzXac2CIo7bSWHMwv14kxH53zEku4 6CxjyQf09M5rialBcVt4x9u3hHG5f/LZAZC2bUhjWf0= X-Report-Abuse-To: spam@com-mpt-mgt001.mailprotect.be Sender: target-devel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: target-devel@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP When using SCSI passthrough in combination with the iSCSI target driver then cmd->t_state_lock may be obtained from interrupt context. Hence, all code that obtains cmd->t_state_lock from thread context must disable interrupts first. This patch avoids that lockdep reports the following: WARNING: inconsistent lock state 4.18.0-dbg+ #1 Not tainted -------------------------------- inconsistent {HARDIRQ-ON-W} -> {IN-HARDIRQ-W} usage. iscsi_ttx/1800 [HC1[1]:SC0[2]:HE0:SE0] takes: 000000006e7b0ceb (&(&cmd->t_state_lock)->rlock){?...}, at: target_complete_cmd+0x47/0x2c0 [target_core_mod] {HARDIRQ-ON-W} state was registered at: lock_acquire+0xd2/0x260 _raw_spin_lock+0x32/0x50 iscsit_close_connection+0x97e/0x1020 [iscsi_target_mod] iscsit_take_action_for_connection_exit+0x108/0x200 [iscsi_target_mod] iscsi_target_rx_thread+0x180/0x190 [iscsi_target_mod] kthread+0x1cf/0x1f0 ret_from_fork+0x24/0x30 irq event stamp: 1281 hardirqs last enabled at (1279): [] __local_bh_enable_ip+0xa9/0x160 hardirqs last disabled at (1281): [] interrupt_entry+0xb5/0xd0 softirqs last enabled at (1278): [] lock_sock_nested+0x51/0xc0 softirqs last disabled at (1280): [] ip6_finish_output2+0x124/0xe40 [ipv6] other info that might help us debug this: Possible unsafe locking scenario: CPU0 ---- lock(&(&cmd->t_state_lock)->rlock); lock(&(&cmd->t_state_lock)->rlock); *** DEADLOCK *** 3 locks held by iscsi_ttx/1800: *0: 00000000c3b711b7 (sk_lock-AF_INET6){+.+.}, at: tcp_sendmsg+0x1e/0x50 *1: 00000000fa81046f (rcu_read_lock){....}, at: inet6_csk_xmit+0xc7/0x2e0 [ipv6] *2: 00000000c091d70d (rcu_read_lock_bh){....}, at: ip6_finish_output2+0x124/0xe40 [ipv6] stack backtrace: CPU: 0 PID: 1800 Comm: iscsi_ttx Not tainted 4.18.0-dbg+ #1 Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS 1.0.0-prebuilt.qemu-project.org 04/01/2014 Call Trace: dump_stack+0xa4/0xf5 print_usage_bug+0x25b/0x27b mark_lock+0x70f/0x7b0 __lock_acquire+0xbc2/0x1b50 lock_acquire+0xd2/0x260 _raw_spin_lock_irqsave+0x4a/0x60 target_complete_cmd+0x47/0x2c0 [target_core_mod] target_complete_cmd_with_length+0x70/0xa0 [target_core_mod] pscsi_req_done+0x335/0x530 [target_core_pscsi] __blk_mq_end_request+0xa5/0x140 scsi_end_request+0x112/0x320 [scsi_mod] scsi_io_completion+0x183/0xa30 [scsi_mod] scsi_finish_command+0x1c0/0x280 [scsi_mod] scsi_softirq_done+0x19a/0x230 [scsi_mod] __blk_mq_complete_request_remote+0x2f/0x40 flush_smp_call_function_queue+0x12a/0x220 generic_smp_call_function_single_interrupt+0x13/0x30 smp_call_function_single_interrupt+0x7a/0x350 call_function_single_interrupt+0xf/0x20 RIP: 0010:__asan_load4+0x1e/0x80 debug_lockdep_rcu_enabled+0x26/0x40 ip6_finish_output2+0x15a/0xe40 [ipv6] ip6_finish_output+0x308/0x440 [ipv6] ip6_output+0x11d/0x3b0 [ipv6] ip6_xmit+0x639/0xc50 [ipv6] inet6_csk_xmit+0x198/0x2e0 [ipv6] __tcp_transmit_skb+0xc1b/0x15b0 tcp_write_xmit+0x42e/0x1f20 __tcp_push_pending_frames+0x59/0x150 tcp_push+0x189/0x270 tcp_sendmsg_locked+0x7b9/0x1680 tcp_sendmsg+0x2c/0x50 inet_sendmsg+0x71/0x250 sock_sendmsg+0x4c/0x60 tx_data+0x12b/0x1f0 [iscsi_target_mod] iscsit_send_tx_data+0x77/0xe0 [iscsi_target_mod] iscsit_xmit_pdu+0x2c5/0x740 [iscsi_target_mod] iscsit_response_queue+0x941/0xd40 [iscsi_target_mod] iscsi_target_tx_thread+0x23b/0x350 [iscsi_target_mod] kthread+0x1cf/0x1f0 ret_from_fork+0x24/0x30 Fixes: 064cdd2d91c2 ("target: Fix race between iscsi-target connection shutdown + ABORT_TASK") Cc: Nicholas Bellinger Cc: Mike Christie Cc: Christoph Hellwig Cc: Hannes Reinecke Cc: Signed-off-by: Bart Van Assche --- drivers/target/iscsi/iscsi_target.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/drivers/target/iscsi/iscsi_target.c b/drivers/target/iscsi/iscsi_target.c index 7059ffdb0c20..408d77eb717c 100644 --- a/drivers/target/iscsi/iscsi_target.c +++ b/drivers/target/iscsi/iscsi_target.c @@ -4038,9 +4038,9 @@ static void iscsit_release_commands_from_conn(struct iscsi_conn *conn) struct se_cmd *se_cmd = &cmd->se_cmd; if (se_cmd->se_tfo != NULL) { - spin_lock(&se_cmd->t_state_lock); + spin_lock_irq(&se_cmd->t_state_lock); se_cmd->transport_state |= CMD_T_FABRIC_STOP; - spin_unlock(&se_cmd->t_state_lock); + spin_unlock_irq(&se_cmd->t_state_lock); } } spin_unlock_bh(&conn->cmd_lock);