From patchwork Mon Sep 14 03:43:58 2015 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Nicholas A. Bellinger" X-Patchwork-Id: 7172971 Return-Path: X-Original-To: patchwork-linux-scsi@patchwork.kernel.org Delivered-To: patchwork-parsemail@patchwork2.web.kernel.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.136]) by patchwork2.web.kernel.org (Postfix) with ESMTP id A147FBEEC1 for ; Mon, 14 Sep 2015 03:50:40 +0000 (UTC) Received: from mail.kernel.org (localhost [127.0.0.1]) by mail.kernel.org (Postfix) with ESMTP id 72110206E0 for ; Mon, 14 Sep 2015 03:50:39 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 07147206DD for ; Mon, 14 Sep 2015 03:50:38 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752588AbbINDuV (ORCPT ); Sun, 13 Sep 2015 23:50:21 -0400 Received: from mail-oi0-f52.google.com ([209.85.218.52]:36261 "EHLO mail-oi0-f52.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753318AbbINDuU (ORCPT ); Sun, 13 Sep 2015 23:50:20 -0400 Received: by oibi136 with SMTP id i136so70408195oib.3 for ; Sun, 13 Sep 2015 20:50:19 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:from:to:cc:subject:date:message-id; bh=2Lgf5T6d60yGYL21BrM4Q+6HGhZtzbkQN6fdTYh2dj8=; b=FFl/E9B+Sx37J00jeN9TQA6ITNg4OPNGm3QfHpjf9kO9iqoFf+zTevQpW+12cSXgnB cc9mnVnA9V/DoWWQQDSvkrCWMytg6l1G4Sax9MJBkOUgcJKbV2EoX1fdLzk4bT1pqEuA EOCxW6uCzV/vZjNpWmUf9TXbx26IKrRxKcu/urb9WZLQy3cQ79QbwvVuMlbyLXQa9eEW emcd/bXkFNJVanWCWlt5cShrGof/t0gGtOhScLO1BFqNk6Xs35YgRjjypSxjgbH/Zl1O QS0qBhjvC6y0FGeYNQ6WgtMS9ggP6ts2Bf6vHgKKzQTlupyKZWAYofNZULzJ+D26kl+c Qt/A== X-Gm-Message-State: ALoCoQkHFlyimbUBJgEJGnv8NX8tkVriSUNZ/3r7VRhus91yIX826xednpWs6ndUwG0ETCsPCZJ1 X-Received: by 10.202.79.143 with SMTP id d137mr8905007oib.38.1442202619465; Sun, 13 Sep 2015 20:50:19 -0700 (PDT) Received: from localhost.localdomain (mail.linux-iscsi.org. [67.23.28.174]) by smtp.gmail.com with ESMTPSA id x142sm5590044oix.0.2015.09.13.20.50.18 (version=TLS1 cipher=RC4-SHA bits=128/128); Sun, 13 Sep 2015 20:50:18 -0700 (PDT) From: "Nicholas A. Bellinger" To: target-devel Cc: linux-scsi , Nicholas Bellinger , "Scott L. Lykens" , Christoph Hellwig , Hannes Reinecke , Lee Duncan Subject: [PATCH] target: Fix PR registration + APTPL RCU conversion regression Date: Mon, 14 Sep 2015 03:43:58 +0000 Message-Id: <1442202238-32708-1-git-send-email-nab@daterainc.com> X-Mailer: git-send-email 1.7.2.5 Sender: linux-scsi-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org X-Spam-Status: No, score=-6.9 required=5.0 tests=BAYES_00, RCVD_IN_DNSWL_HI, T_RP_MATCHES_RCVD, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on mail.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Nicholas Bellinger This patch fixes a v4.2+ regression introduced by commit 79dc9c9e86 where lookup of t10_pr_registration->pr_reg_deve and associated ->pr_kref get was missing from __core_scsi3_do_alloc_registration(), which is responsible for setting DEF_PR_REG_ACTIVE. This would result in REGISTER operations completing successfully, but subsequent core_scsi3_pr_seq_non_holder() checking would fail with !DEF_PR_REG_ACTIVE -> RESERVATION CONFLICT status. Update __core_scsi3_add_registration() to drop ->pr_kref reference after registration and any optional ALL_TG_PT=1 processing has completed. Update core_scsi3_decode_spec_i_port() to release the new parent local_pr_reg->pr_kref as well. Also, update __core_scsi3_check_aptpl_registration() to perform the same target_nacl_find_deve() lookup + ->pr_kref get, now that __core_scsi3_add_registration() expects to drop the reference. Finally, since there are cases when se_dev_entry->se_lun_acl can still be dereferenced in core_scsi3_lunacl_undepend_item() while holding ->pr_kref, go ahead and move explicit rcu_assign_pointer() NULL assignments within core_disable_device_list_for_node() until after orig->pr_comp finishes. Reported-by: Scott L. Lykens Cc: Scott L. Lykens Cc: Christoph Hellwig Cc: Hannes Reinecke Cc: Lee Duncan Signed-off-by: Nicholas Bellinger --- drivers/target/target_core_device.c | 5 +- drivers/target/target_core_pr.c | 91 +++++++++++++++++++++++++++---------- 2 files changed, 70 insertions(+), 26 deletions(-) diff --git a/drivers/target/target_core_device.c b/drivers/target/target_core_device.c index dcc424a..abf2076 100644 --- a/drivers/target/target_core_device.c +++ b/drivers/target/target_core_device.c @@ -427,8 +427,6 @@ void core_disable_device_list_for_node( hlist_del_rcu(&orig->link); clear_bit(DEF_PR_REG_ACTIVE, &orig->deve_flags); - rcu_assign_pointer(orig->se_lun, NULL); - rcu_assign_pointer(orig->se_lun_acl, NULL); orig->lun_flags = 0; orig->creation_time = 0; orig->attach_count--; @@ -439,6 +437,9 @@ void core_disable_device_list_for_node( kref_put(&orig->pr_kref, target_pr_kref_release); wait_for_completion(&orig->pr_comp); + rcu_assign_pointer(orig->se_lun, NULL); + rcu_assign_pointer(orig->se_lun_acl, NULL); + kfree_rcu(orig, rcu_head); core_scsi3_free_pr_reg_from_nacl(dev, nacl); diff --git a/drivers/target/target_core_pr.c b/drivers/target/target_core_pr.c index 5ab7100..e793311 100644 --- a/drivers/target/target_core_pr.c +++ b/drivers/target/target_core_pr.c @@ -618,7 +618,7 @@ static struct t10_pr_registration *__core_scsi3_do_alloc_registration( struct se_device *dev, struct se_node_acl *nacl, struct se_lun *lun, - struct se_dev_entry *deve, + struct se_dev_entry *dest_deve, u64 mapped_lun, unsigned char *isid, u64 sa_res_key, @@ -640,7 +640,29 @@ static struct t10_pr_registration *__core_scsi3_do_alloc_registration( INIT_LIST_HEAD(&pr_reg->pr_reg_atp_mem_list); atomic_set(&pr_reg->pr_res_holders, 0); pr_reg->pr_reg_nacl = nacl; - pr_reg->pr_reg_deve = deve; + /* + * For destination registrations for ALL_TG_PT=1 and SPEC_I_PT=1, + * the se_dev_entry->pr_ref will have been already obtained by + * core_get_se_deve_from_rtpi() or __core_scsi3_alloc_registration(). + * + * Otherwise, locate se_dev_entry now and obtain a reference until + * registration completes in __core_scsi3_add_registration(). + */ + if (dest_deve) { + pr_reg->pr_reg_deve = dest_deve; + } else { + rcu_read_lock(); + pr_reg->pr_reg_deve = target_nacl_find_deve(nacl, mapped_lun); + if (!pr_reg->pr_reg_deve) { + rcu_read_unlock(); + pr_err("Unable to locate PR deve %s mapped_lun: %llu\n", + nacl->initiatorname, mapped_lun); + kmem_cache_free(t10_pr_reg_cache, pr_reg); + return NULL; + } + kref_get(&pr_reg->pr_reg_deve->pr_kref); + rcu_read_unlock(); + } pr_reg->pr_res_mapped_lun = mapped_lun; pr_reg->pr_aptpl_target_lun = lun->unpacked_lun; pr_reg->tg_pt_sep_rtpi = lun->lun_rtpi; @@ -936,17 +958,29 @@ static int __core_scsi3_check_aptpl_registration( !(strcmp(pr_reg->pr_tport, t_port)) && (pr_reg->pr_reg_tpgt == tpgt) && (pr_reg->pr_aptpl_target_lun == target_lun)) { + /* + * Obtain the ->pr_reg_deve pointer + reference, that + * is released by __core_scsi3_add_registration() below. + */ + rcu_read_lock(); + pr_reg->pr_reg_deve = target_nacl_find_deve(nacl, mapped_lun); + if (!pr_reg->pr_reg_deve) { + pr_err("Unable to locate PR APTPL %s mapped_lun:" + " %llu\n", nacl->initiatorname, mapped_lun); + rcu_read_unlock(); + continue; + } + kref_get(&pr_reg->pr_reg_deve->pr_kref); + rcu_read_unlock(); pr_reg->pr_reg_nacl = nacl; pr_reg->tg_pt_sep_rtpi = lun->lun_rtpi; - list_del(&pr_reg->pr_reg_aptpl_list); spin_unlock(&pr_tmpl->aptpl_reg_lock); /* * At this point all of the pointers in *pr_reg will * be setup, so go ahead and add the registration. */ - __core_scsi3_add_registration(dev, nacl, pr_reg, 0, 0); /* * If this registration is the reservation holder, @@ -1044,18 +1078,11 @@ static void __core_scsi3_add_registration( __core_scsi3_dump_registration(tfo, dev, nacl, pr_reg, register_type); spin_unlock(&pr_tmpl->registration_lock); - - rcu_read_lock(); - deve = pr_reg->pr_reg_deve; - if (deve) - set_bit(DEF_PR_REG_ACTIVE, &deve->deve_flags); - rcu_read_unlock(); - /* * Skip extra processing for ALL_TG_PT=0 or REGISTER_AND_MOVE. */ if (!pr_reg->pr_reg_all_tg_pt || register_move) - return; + goto out; /* * Walk pr_reg->pr_reg_atp_list and add registrations for ALL_TG_PT=1 * allocated in __core_scsi3_alloc_registration() @@ -1075,19 +1102,31 @@ static void __core_scsi3_add_registration( __core_scsi3_dump_registration(tfo, dev, nacl_tmp, pr_reg_tmp, register_type); spin_unlock(&pr_tmpl->registration_lock); - + /* + * Drop configfs group dependency reference and deve->pr_kref + * obtained from __core_scsi3_alloc_registration() code. + */ rcu_read_lock(); deve = pr_reg_tmp->pr_reg_deve; - if (deve) + if (deve) { set_bit(DEF_PR_REG_ACTIVE, &deve->deve_flags); + core_scsi3_lunacl_undepend_item(deve); + pr_reg_tmp->pr_reg_deve = NULL; + } rcu_read_unlock(); - - /* - * Drop configfs group dependency reference from - * __core_scsi3_alloc_registration() - */ - core_scsi3_lunacl_undepend_item(pr_reg_tmp->pr_reg_deve); } +out: + /* + * Drop deve->pr_kref obtained in __core_scsi3_do_alloc_registration() + */ + rcu_read_lock(); + deve = pr_reg->pr_reg_deve; + if (deve) { + set_bit(DEF_PR_REG_ACTIVE, &deve->deve_flags); + kref_put(&deve->pr_kref, target_pr_kref_release); + pr_reg->pr_reg_deve = NULL; + } + rcu_read_unlock(); } static int core_scsi3_alloc_registration( @@ -1785,9 +1824,11 @@ core_scsi3_decode_spec_i_port( dest_node_acl->initiatorname, i_buf, (dest_se_deve) ? dest_se_deve->mapped_lun : 0); - if (!dest_se_deve) + if (!dest_se_deve) { + kref_put(&local_pr_reg->pr_reg_deve->pr_kref, + target_pr_kref_release); continue; - + } core_scsi3_lunacl_undepend_item(dest_se_deve); core_scsi3_nodeacl_undepend_item(dest_node_acl); core_scsi3_tpg_undepend_item(dest_tpg); @@ -1823,9 +1864,11 @@ out: kmem_cache_free(t10_pr_reg_cache, dest_pr_reg); - if (!dest_se_deve) + if (!dest_se_deve) { + kref_put(&local_pr_reg->pr_reg_deve->pr_kref, + target_pr_kref_release); continue; - + } core_scsi3_lunacl_undepend_item(dest_se_deve); core_scsi3_nodeacl_undepend_item(dest_node_acl); core_scsi3_tpg_undepend_item(dest_tpg);