From patchwork Tue Jun 6 19:38:43 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Martin Wilck X-Patchwork-Id: 13269682 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 83CC5C77B7A for ; Tue, 6 Jun 2023 19:39:46 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231970AbjFFTjM (ORCPT ); Tue, 6 Jun 2023 15:39:12 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60192 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232037AbjFFTix (ORCPT ); Tue, 6 Jun 2023 15:38:53 -0400 Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.220.29]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id F02C010EA; Tue, 6 Jun 2023 12:38:52 -0700 (PDT) Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id 94F6A1FD92; Tue, 6 Jun 2023 19:38:51 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1; t=1686080331; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=leWSNybkvgM5TIQBj/WwVrrY7dWbgHg5e1qUYq/uf14=; b=EAAvmzrNAsQWRMdlS/08G/MyNj7yO3Ovbr/0Z5+b5kWnP47Rk8Zh3DH4R575crn+vzSmtD q4HeWneAYjI62F62ycL7fayQjzZ0iW1y8Gzrjwy7dJtrNBwm4Pq83Tcih6aigh7Qha4Owb Xg3SK8KxCXnKbPKouHidmv8WecGKs6c= Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 4016A13A47; Tue, 6 Jun 2023 19:38:51 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id qM/tDUuLf2RaFwAAMHmgww (envelope-from ); Tue, 06 Jun 2023 19:38:51 +0000 From: mwilck@suse.com To: "Martin K. Petersen" , Christoph Hellwig , Ming Lei , Bart Van Assche Cc: James Bottomley , linux-scsi@vger.kernel.org, linux-block@vger.kernel.org, Hannes Reinecke Subject: [PATCH v2 1/3] bsg: increase number of devices Date: Tue, 6 Jun 2023 21:38:43 +0200 Message-Id: <20230606193845.9627-2-mwilck@suse.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20230606193845.9627-1-mwilck@suse.com> References: <20230606193845.9627-1-mwilck@suse.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org From: Hannes Reinecke Larger setups may need to allocate more than 32k bsg devices, so increase the number of devices to the full range of minor device numbers. Signed-off-by: Hannes Reinecke Reviewed-by: Christoph Hellwig --- block/bsg.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/block/bsg.c b/block/bsg.c index 7eca43f33d7f..c53f24243bf2 100644 --- a/block/bsg.c +++ b/block/bsg.c @@ -36,7 +36,7 @@ static inline struct bsg_device *to_bsg_device(struct inode *inode) } #define BSG_DEFAULT_CMDS 64 -#define BSG_MAX_DEVS 32768 +#define BSG_MAX_DEVS (1 << MINORBITS) static DEFINE_IDA(bsg_minor_ida); static struct class *bsg_class; From patchwork Tue Jun 6 19:38:44 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Martin Wilck X-Patchwork-Id: 13269683 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id C9A20C83003 for ; Tue, 6 Jun 2023 19:39:46 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229603AbjFFTjM (ORCPT ); Tue, 6 Jun 2023 15:39:12 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60204 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232094AbjFFTiy (ORCPT ); Tue, 6 Jun 2023 15:38:54 -0400 Received: from smtp-out2.suse.de (smtp-out2.suse.de [IPv6:2001:67c:2178:6::1d]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5B37210EC; Tue, 6 Jun 2023 12:38:53 -0700 (PDT) Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id F284C1FD94; Tue, 6 Jun 2023 19:38:51 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1; t=1686080332; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=qVIEkGqw5a+jSqUHhvJwqFvm7MGHdaBak6UofBYr9tc=; b=PrO7Lw0z3fEWibFaTiR5OrwxUNJnICasm5nPgrpv1IshqwYOy2tZ5HDcuuyl6dmmHjB2pC J0Mj0YCDro1oSGwNECCf5E2NOQ0Y9KjuGy6hloqnhQVJArgRoG7Oyxm7LSBFpggTWqiehM f8Ts775JlMNIkkkzXglb7/+6jXiz0w0= Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 9FDBF13776; Tue, 6 Jun 2023 19:38:51 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id MCl8JUuLf2RaFwAAMHmgww (envelope-from ); Tue, 06 Jun 2023 19:38:51 +0000 From: mwilck@suse.com To: "Martin K. Petersen" , Christoph Hellwig , Ming Lei , Bart Van Assche Cc: James Bottomley , linux-scsi@vger.kernel.org, linux-block@vger.kernel.org, Hannes Reinecke , Douglas Gilbert Subject: [PATCH v2 2/3] scsi: sg: increase number of devices Date: Tue, 6 Jun 2023 21:38:44 +0200 Message-Id: <20230606193845.9627-3-mwilck@suse.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20230606193845.9627-1-mwilck@suse.com> References: <20230606193845.9627-1-mwilck@suse.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org From: Hannes Reinecke Larger setups may need to allocate more than 32k sg devices, so increase the number of devices to the full range of minor device numbers. Signed-off-by: Hannes Reinecke Acked-by: Douglas Gilbert Reviewed-by: Christoph Hellwig --- drivers/scsi/sg.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/drivers/scsi/sg.c b/drivers/scsi/sg.c index 037f8c98a6d3..6c04cf941dac 100644 --- a/drivers/scsi/sg.c +++ b/drivers/scsi/sg.c @@ -71,7 +71,7 @@ static int sg_proc_init(void); #define SG_ALLOW_DIO_DEF 0 -#define SG_MAX_DEVS 32768 +#define SG_MAX_DEVS (1 << MINORBITS) /* SG_MAX_CDB_SIZE should be 260 (spc4r37 section 3.1.30) however the type * of sg_io_hdr::cmd_len can only represent 255. All SCSI commands greater From patchwork Tue Jun 6 19:38:45 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Martin Wilck X-Patchwork-Id: 13269685 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id E0811C7EE37 for ; Tue, 6 Jun 2023 19:39:46 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233505AbjFFTjN (ORCPT ); Tue, 6 Jun 2023 15:39:13 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60206 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232166AbjFFTiy (ORCPT ); Tue, 6 Jun 2023 15:38:54 -0400 Received: from smtp-out2.suse.de (smtp-out2.suse.de [IPv6:2001:67c:2178:6::1d]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id BD9A410D7; Tue, 6 Jun 2023 12:38:53 -0700 (PDT) Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id 53B5B1FD95; Tue, 6 Jun 2023 19:38:52 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1; t=1686080332; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=bXxH/xRJ6ryUI8Yc/IKZBzn1JvzrziF9j7uphU/kqjg=; b=tOvRyK5YbJRNuxdRMUelNSEY+Xb3gniZOVr3L78+JxY3Y5GE0G9jSGTcR2z6eA8zhtwe0C 1cbw7ZuGZrv2nKTXc2PlCElvJ5fhFQzYBiWkk4i3aUWdygch6VPXDy06PEl1ZRpuCSHhj3 8MXj/QwoRdpF09aG44k1X3zKTg/7zws= Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 0D69413776; Tue, 6 Jun 2023 19:38:52 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id QGQtAUyLf2RaFwAAMHmgww (envelope-from ); Tue, 06 Jun 2023 19:38:52 +0000 From: mwilck@suse.com To: "Martin K. Petersen" , Christoph Hellwig , Ming Lei , Bart Van Assche Cc: James Bottomley , linux-scsi@vger.kernel.org, linux-block@vger.kernel.org, Hannes Reinecke Subject: [PATCH v2 3/3] scsi: simplify scsi_stop_queue() Date: Tue, 6 Jun 2023 21:38:45 +0200 Message-Id: <20230606193845.9627-4-mwilck@suse.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20230606193845.9627-1-mwilck@suse.com> References: <20230606193845.9627-1-mwilck@suse.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org From: Hannes Reinecke scsi_target_block() calls scsi_stop_queue() for each scsi_device and scsi_stop_queue() calls blk_mq_wait_quiesce_done() for each LUN. As blk_mq_wait_quiesce_done() comes down to synchronize_rcu() for SCSI queues, this can cause substantial delay for scsi_target_block() on a target with a lot of logical units (we measured more than 100s delay for blocking a FC rport with 2048 LUNs). Simplify scsi_stop_queue(), which is only called in this code path, to never wait for the quiescing to finish. Rather call blk_mq_wait_quiesce_done() from scsi_target_block() after iterating over all devices. Also, move the call to scsi_stop_queue() in scsi_internal_device_block() out of the code section where the state_mutex is held. This patch uses the same basic idea as f983622ae605 ("scsi: core: Avoid calling synchronize_rcu() for each device in scsi_host_block()"). Signed-off-by: Hannes Reinecke Signed-off-by: Martin Wilck --- drivers/scsi/scsi_lib.c | 28 ++++++++++++++-------------- 1 file changed, 14 insertions(+), 14 deletions(-) diff --git a/drivers/scsi/scsi_lib.c b/drivers/scsi/scsi_lib.c index 25489fbd94c6..bc78bea62755 100644 --- a/drivers/scsi/scsi_lib.c +++ b/drivers/scsi/scsi_lib.c @@ -2726,24 +2726,18 @@ void scsi_start_queue(struct scsi_device *sdev) blk_mq_unquiesce_queue(sdev->request_queue); } -static void scsi_stop_queue(struct scsi_device *sdev, bool nowait) +static void scsi_stop_queue(struct scsi_device *sdev) { /* * The atomic variable of ->queue_stopped covers that * blk_mq_quiesce_queue* is balanced with blk_mq_unquiesce_queue. * * However, we still need to wait until quiesce is done - * in case that queue has been stopped. + * in case that queue has been stopped. This is done in + * scsi_target_block() for all devices of the target. */ - if (!cmpxchg(&sdev->queue_stopped, 0, 1)) { - if (nowait) - blk_mq_quiesce_queue_nowait(sdev->request_queue); - else - blk_mq_quiesce_queue(sdev->request_queue); - } else { - if (!nowait) - blk_mq_wait_quiesce_done(sdev->request_queue->tag_set); - } + if (!cmpxchg(&sdev->queue_stopped, 0, 1)) + blk_mq_quiesce_queue_nowait(sdev->request_queue); } /** @@ -2770,7 +2764,7 @@ int scsi_internal_device_block_nowait(struct scsi_device *sdev) * request queue. */ if (!ret) - scsi_stop_queue(sdev, true); + scsi_stop_queue(sdev); return ret; } EXPORT_SYMBOL_GPL(scsi_internal_device_block_nowait); @@ -2796,9 +2790,9 @@ static int scsi_internal_device_block(struct scsi_device *sdev) mutex_lock(&sdev->state_mutex); err = __scsi_internal_device_block_nowait(sdev); - if (err == 0) - scsi_stop_queue(sdev, false); mutex_unlock(&sdev->state_mutex); + if (err == 0) + scsi_stop_queue(sdev); return err; } @@ -2906,11 +2900,17 @@ target_block(struct device *dev, void *data) void scsi_target_block(struct device *dev) { + struct Scsi_Host *shost = dev_to_shost(dev); + if (scsi_is_target_device(dev)) starget_for_each_device(to_scsi_target(dev), NULL, device_block); else device_for_each_child(dev, NULL, target_block); + + /* Wait for ongoing scsi_queue_rq() calls to finish. */ + if (!WARN_ON_ONCE(!shost)) + blk_mq_wait_quiesce_done(&shost->tag_set); } EXPORT_SYMBOL_GPL(scsi_target_block);