From patchwork Wed Nov 13 22:05:07 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bart Van Assche X-Patchwork-Id: 11242781 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 820C6138C for ; Wed, 13 Nov 2019 22:05:19 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 535CA206E5 for ; Wed, 13 Nov 2019 22:05:19 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726335AbfKMWFR (ORCPT ); Wed, 13 Nov 2019 17:05:17 -0500 Received: from mail-pl1-f194.google.com ([209.85.214.194]:38283 "EHLO mail-pl1-f194.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726162AbfKMWFR (ORCPT ); Wed, 13 Nov 2019 17:05:17 -0500 Received: by mail-pl1-f194.google.com with SMTP id w8so1646017plq.5 for ; Wed, 13 Nov 2019 14:05:16 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Xl76pmWe9AcE/cdnE194As3UKSyvLWQyrFCc+s0s0jM=; b=s4kjet2gw+XcLvNDOi8IJXIUfjlVOb+W5SLpnavzavcCc/DalpOYHKjMugZnbEYyIL UPhaey7DRiTARwgM3fMgV5wDghhukc5RZLrqP2iAe3tdngH2nPZTr2PQvQGsm5iBKbsu BOIA9+ZbMWX9iGufpJuSc2kXbiDZ7HfQon5EWyG6C1qSPrpuJlTcJ12qhYZsKzmzIFPh SMLmEtmxMjlOt0xr/WSuen3mIN0QHOpMyvEVh78qAyFS36rHm5XkX+LVT7I+psayiXYH XEX7Teq8PnPbH8fYHR5bEEkgBk3H0u9hQ7BYiq+xYv6DUbpEV88ErMpMIz/qAFtS5ziJ jOkg== X-Gm-Message-State: APjAAAUW8uj0Zu8UNptwX6YRjoBbtOG7+rRSxhVCaRRtjxIqZ1k+qA8Q ER+do02TIuE4qeCi+U3oofU= X-Google-Smtp-Source: APXvYqyq4z2Q1wZ8L9Adey1T2Wkv3FbNBPwTw8okqkelCJx2vWHJPMrJl8qjMha19zlk8lUGt1v+kA== X-Received: by 2002:a17:902:8498:: with SMTP id c24mr3600043plo.223.1573682716423; Wed, 13 Nov 2019 14:05:16 -0800 (PST) Received: from desktop-bart.svl.corp.google.com ([2620:15c:2cd:202:4308:52a3:24b6:2c60]) by smtp.gmail.com with ESMTPSA id v23sm3843527pfm.175.2019.11.13.14.05.15 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 13 Nov 2019 14:05:15 -0800 (PST) From: Bart Van Assche To: "Martin K . Petersen" Cc: Mike Christie , Christoph Hellwig , target-devel@vger.kernel.org, Bart Van Assche , Roman Bolshakov Subject: [PATCH v4 1/2] target/core: Release SPC-2 reservations when closing a session Date: Wed, 13 Nov 2019 14:05:07 -0800 Message-Id: <20191113220508.198257-2-bvanassche@acm.org> X-Mailer: git-send-email 2.24.0.rc1.363.gb1bccd3e3d-goog In-Reply-To: <20191113220508.198257-1-bvanassche@acm.org> References: <20191113220508.198257-1-bvanassche@acm.org> MIME-Version: 1.0 Sender: target-devel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: target-devel@vger.kernel.org The SCSI specs require to release SPC-2 reservations when a session is closed. Make sure that the target core does this. Running the libiscsi tests triggers the KASAN complaint shown below. This patch fixes that use-after-free. BUG: KASAN: use-after-free in target_check_reservation+0x171/0x980 [target_core_mod] Read of size 8 at addr ffff88802ecd1878 by task iscsi_trx/17200 CPU: 0 PID: 17200 Comm: iscsi_trx Not tainted 5.4.0-rc1-dbg+ #1 Hardware name: Bochs Bochs, BIOS Bochs 01/01/2011 Call Trace: dump_stack+0x8a/0xd6 print_address_description.constprop.0+0x40/0x60 __kasan_report.cold+0x1b/0x34 kasan_report+0x16/0x20 __asan_load8+0x58/0x90 target_check_reservation+0x171/0x980 [target_core_mod] __target_execute_cmd+0xb1/0xf0 [target_core_mod] target_execute_cmd+0x22d/0x4d0 [target_core_mod] transport_generic_new_cmd+0x31f/0x5b0 [target_core_mod] transport_handle_cdb_direct+0x6f/0x90 [target_core_mod] iscsit_execute_cmd+0x381/0x3f0 [iscsi_target_mod] iscsit_sequence_cmd+0x13b/0x1f0 [iscsi_target_mod] iscsit_process_scsi_cmd+0x4c/0x130 [iscsi_target_mod] iscsit_get_rx_pdu+0x8e8/0x15f0 [iscsi_target_mod] iscsi_target_rx_thread+0x105/0x1b0 [iscsi_target_mod] kthread+0x1bc/0x210 ret_from_fork+0x24/0x30 Allocated by task 1079: save_stack+0x23/0x90 __kasan_kmalloc.constprop.0+0xcf/0xe0 kasan_slab_alloc+0x12/0x20 kmem_cache_alloc+0xfe/0x3a0 transport_alloc_session+0x29/0x80 [target_core_mod] iscsi_target_login_thread+0xceb/0x1920 [iscsi_target_mod] kthread+0x1bc/0x210 ret_from_fork+0x24/0x30 Freed by task 17193: save_stack+0x23/0x90 __kasan_slab_free+0x13a/0x190 kasan_slab_free+0x12/0x20 kmem_cache_free+0xc8/0x3e0 transport_free_session+0x179/0x2f0 [target_core_mod] transport_deregister_session+0x121/0x170 [target_core_mod] iscsit_close_session+0x12c/0x350 [iscsi_target_mod] iscsit_logout_post_handler+0x136/0x380 [iscsi_target_mod] iscsit_response_queue+0x8fa/0xc00 [iscsi_target_mod] iscsi_target_tx_thread+0x28e/0x390 [iscsi_target_mod] kthread+0x1bc/0x210 ret_from_fork+0x24/0x30 The buggy address belongs to the object at ffff88802ecd1860 which belongs to the cache se_sess_cache of size 352 The buggy address is located 24 bytes inside of 352-byte region [ffff88802ecd1860, ffff88802ecd19c0) The buggy address belongs to the page: page:ffffea0000bb3400 refcount:1 mapcount:0 mapping:ffff8880bef2ed00 index:0x0 compound_mapcount: 0 flags: 0x1000000000010200(slab|head) raw: 1000000000010200 dead000000000100 dead000000000122 ffff8880bef2ed00 raw: 0000000000000000 0000000080270027 00000001ffffffff 0000000000000000 page dumped because: kasan: bad access detected Memory state around the buggy address: ffff88802ecd1700: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb ffff88802ecd1780: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb >ffff88802ecd1800: fb fb fb fb fc fc fc fc fc fc fc fc fb fb fb fb ^ ffff88802ecd1880: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb ffff88802ecd1900: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb Reviewed-by: Roman Bolshakov Cc: Mike Christie Signed-off-by: Bart Van Assche --- drivers/target/target_core_transport.c | 15 +++++++++++++++ 1 file changed, 15 insertions(+) diff --git a/drivers/target/target_core_transport.c b/drivers/target/target_core_transport.c index ba7a01bcad7c..09f833c1de8a 100644 --- a/drivers/target/target_core_transport.c +++ b/drivers/target/target_core_transport.c @@ -584,6 +584,15 @@ void transport_free_session(struct se_session *se_sess) } EXPORT_SYMBOL(transport_free_session); +static int target_release_res(struct se_device *dev, void *data) +{ + struct se_session *sess = data; + + if (dev->reservation_holder == sess) + target_release_reservation(dev); + return 0; +} + void transport_deregister_session(struct se_session *se_sess) { struct se_portal_group *se_tpg = se_sess->se_tpg; @@ -600,6 +609,12 @@ void transport_deregister_session(struct se_session *se_sess) se_sess->fabric_sess_ptr = NULL; spin_unlock_irqrestore(&se_tpg->session_lock, flags); + /* + * Since the session is being removed, release SPC-2 + * reservations held by the session that is disappearing. + */ + target_for_each_device(target_release_res, se_sess); + pr_debug("TARGET_CORE[%s]: Deregistered fabric_sess\n", se_tpg->se_tpg_tfo->fabric_name); /* From patchwork Wed Nov 13 22:05:08 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bart Van Assche X-Patchwork-Id: 11242783 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 87AF514BC for ; Wed, 13 Nov 2019 22:05:20 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 5939E206E5 for ; Wed, 13 Nov 2019 22:05:20 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726363AbfKMWFS (ORCPT ); Wed, 13 Nov 2019 17:05:18 -0500 Received: from mail-pl1-f194.google.com ([209.85.214.194]:43338 "EHLO mail-pl1-f194.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726162AbfKMWFS (ORCPT ); Wed, 13 Nov 2019 17:05:18 -0500 Received: by mail-pl1-f194.google.com with SMTP id a18so1638221plm.10 for ; Wed, 13 Nov 2019 14:05:18 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Q4GJU83m8cGi/KjCzRthpYCCWp3mdguaaDVmNPxNaY4=; b=GFnmGyC+udJomQN2t68Q5fZxwoP92r+iYyBRJPv/Fab/bpEhgV4ERxRCnn41Xn9ibM IsReLGbl6gqkxJzA/NY+6LsFLxQ7jxN4KJ3BdYG8ZVqv/WQAiCEKTFtx69ZH5oc4OOdW YWGUA/wLy43DeJJflOql8bt75ppGchq8G5ndwDP3jdC29erCpPbev6TJfQQASSnLCJO1 1TFPdMMaPdmZd8jZGWVTGgIgHw75A2wseLPJwsMTufUOLuCKHkucL489Xb2Z+5OAHTQB cyKeoY8o03If8375yhwezW/usSXiXtQnTUWEHnuFfDAUOxE3qGRy19s1V4O87qbH9zEa n9jA== X-Gm-Message-State: APjAAAVTHFtoqs6ffJXiLDaX/xEN4RFYK9203W6rXEEb7X+ejaakAjU9 DzgtduU+f8oZSRaMrn1naog= X-Google-Smtp-Source: APXvYqwnh0jmNDgF8wrL9M3al5tjNGkmjI5Ux1g5UWg22po1fnl70x0Ve4HlLhZmsidvbogYQ6T+uQ== X-Received: by 2002:a17:902:a403:: with SMTP id p3mr5751288plq.275.1573682717902; Wed, 13 Nov 2019 14:05:17 -0800 (PST) Received: from desktop-bart.svl.corp.google.com ([2620:15c:2cd:202:4308:52a3:24b6:2c60]) by smtp.gmail.com with ESMTPSA id v23sm3843527pfm.175.2019.11.13.14.05.16 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 13 Nov 2019 14:05:16 -0800 (PST) From: Bart Van Assche To: "Martin K . Petersen" Cc: Mike Christie , Christoph Hellwig , target-devel@vger.kernel.org, Bart Van Assche , Roman Bolshakov Subject: [PATCH v4 2/2] target/iscsi: Wait for all commands to finish before freeing a session Date: Wed, 13 Nov 2019 14:05:08 -0800 Message-Id: <20191113220508.198257-3-bvanassche@acm.org> X-Mailer: git-send-email 2.24.0.rc1.363.gb1bccd3e3d-goog In-Reply-To: <20191113220508.198257-1-bvanassche@acm.org> References: <20191113220508.198257-1-bvanassche@acm.org> MIME-Version: 1.0 Sender: target-devel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: target-devel@vger.kernel.org The iSCSI target driver is the only target driver that does not wait for ongoing commands to finish before freeing a session. Make the iSCSI target driver wait for ongoing commands to finish before freeing a session. This patch fixes the following KASAN complaint: BUG: KASAN: use-after-free in __lock_acquire+0xb1a/0x2710 Read of size 8 at addr ffff8881154eca70 by task kworker/0:2/247 CPU: 0 PID: 247 Comm: kworker/0:2 Not tainted 5.4.0-rc1-dbg+ #6 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.12.0-1 04/01/2014 Workqueue: target_completion target_complete_ok_work [target_core_mod] Call Trace: dump_stack+0x8a/0xd6 print_address_description.constprop.0+0x40/0x60 __kasan_report.cold+0x1b/0x33 kasan_report+0x16/0x20 __asan_load8+0x58/0x90 __lock_acquire+0xb1a/0x2710 lock_acquire+0xd3/0x200 _raw_spin_lock_irqsave+0x43/0x60 target_release_cmd_kref+0x162/0x7f0 [target_core_mod] target_put_sess_cmd+0x2e/0x40 [target_core_mod] lio_check_stop_free+0x12/0x20 [iscsi_target_mod] transport_cmd_check_stop_to_fabric+0xd8/0xe0 [target_core_mod] target_complete_ok_work+0x1b0/0x790 [target_core_mod] process_one_work+0x549/0xa40 worker_thread+0x7a/0x5d0 kthread+0x1bc/0x210 ret_from_fork+0x24/0x30 Allocated by task 889: save_stack+0x23/0x90 __kasan_kmalloc.constprop.0+0xcf/0xe0 kasan_slab_alloc+0x12/0x20 kmem_cache_alloc+0xf6/0x360 transport_alloc_session+0x29/0x80 [target_core_mod] iscsi_target_login_thread+0xcd6/0x18f0 [iscsi_target_mod] kthread+0x1bc/0x210 ret_from_fork+0x24/0x30 Freed by task 1025: save_stack+0x23/0x90 __kasan_slab_free+0x13a/0x190 kasan_slab_free+0x12/0x20 kmem_cache_free+0x146/0x400 transport_free_session+0x179/0x2f0 [target_core_mod] transport_deregister_session+0x130/0x180 [target_core_mod] iscsit_close_session+0x12c/0x350 [iscsi_target_mod] iscsit_logout_post_handler+0x136/0x380 [iscsi_target_mod] iscsit_response_queue+0x8de/0xbe0 [iscsi_target_mod] iscsi_target_tx_thread+0x27f/0x370 [iscsi_target_mod] kthread+0x1bc/0x210 ret_from_fork+0x24/0x30 The buggy address belongs to the object at ffff8881154ec9c0 which belongs to the cache se_sess_cache of size 352 The buggy address is located 176 bytes inside of 352-byte region [ffff8881154ec9c0, ffff8881154ecb20) The buggy address belongs to the page: page:ffffea0004553b00 refcount:1 mapcount:0 mapping:ffff888101755400 index:0x0 compound_mapcount: 0 flags: 0x2fff000000010200(slab|head) raw: 2fff000000010200 dead000000000100 dead000000000122 ffff888101755400 raw: 0000000000000000 0000000080130013 00000001ffffffff 0000000000000000 page dumped because: kasan: bad access detected Memory state around the buggy address: ffff8881154ec900: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc ffff8881154ec980: fc fc fc fc fc fc fc fc fb fb fb fb fb fb fb fb >ffff8881154eca00: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb ^ ffff8881154eca80: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb ffff8881154ecb00: fb fb fb fb fc fc fc fc fc fc fc fc fc fc fc fc Reviewed-by: Roman Bolshakov Cc: Mike Christie Signed-off-by: Bart Van Assche --- drivers/target/iscsi/iscsi_target.c | 10 ++++++++-- include/scsi/iscsi_proto.h | 1 + 2 files changed, 9 insertions(+), 2 deletions(-) diff --git a/drivers/target/iscsi/iscsi_target.c b/drivers/target/iscsi/iscsi_target.c index 09e55ea0bf5d..7251a87bb576 100644 --- a/drivers/target/iscsi/iscsi_target.c +++ b/drivers/target/iscsi/iscsi_target.c @@ -1165,7 +1165,9 @@ int iscsit_setup_scsi_cmd(struct iscsi_conn *conn, struct iscsi_cmd *cmd, hdr->cmdsn, be32_to_cpu(hdr->data_length), payload_length, conn->cid); - target_get_sess_cmd(&cmd->se_cmd, true); + if (target_get_sess_cmd(&cmd->se_cmd, true) < 0) + return iscsit_add_reject_cmd(cmd, + ISCSI_REASON_WAITING_FOR_LOGOUT, buf); cmd->sense_reason = transport_lookup_cmd_lun(&cmd->se_cmd, scsilun_to_int(&hdr->lun)); @@ -2002,7 +2004,9 @@ iscsit_handle_task_mgt_cmd(struct iscsi_conn *conn, struct iscsi_cmd *cmd, conn->sess->se_sess, 0, DMA_NONE, TCM_SIMPLE_TAG, cmd->sense_buffer + 2); - target_get_sess_cmd(&cmd->se_cmd, true); + if (target_get_sess_cmd(&cmd->se_cmd, true) < 0) + return iscsit_add_reject_cmd(cmd, + ISCSI_REASON_WAITING_FOR_LOGOUT, buf); /* * TASK_REASSIGN for ERL=2 / connection stays inside of @@ -4230,6 +4234,8 @@ int iscsit_close_connection( * must wait until they have completed. */ iscsit_check_conn_usage_count(conn); + target_sess_cmd_list_set_waiting(sess->se_sess); + target_wait_for_sess_cmds(sess->se_sess); ahash_request_free(conn->conn_tx_hash); if (conn->conn_rx_hash) { diff --git a/include/scsi/iscsi_proto.h b/include/scsi/iscsi_proto.h index b71b5c4f418c..533f56733ba8 100644 --- a/include/scsi/iscsi_proto.h +++ b/include/scsi/iscsi_proto.h @@ -627,6 +627,7 @@ struct iscsi_reject { #define ISCSI_REASON_BOOKMARK_INVALID 9 #define ISCSI_REASON_BOOKMARK_NO_RESOURCES 10 #define ISCSI_REASON_NEGOTIATION_RESET 11 +#define ISCSI_REASON_WAITING_FOR_LOGOUT 12 /* Max. number of Key=Value pairs in a text message */ #define MAX_KEY_VALUE_PAIRS 8192