From patchwork Fri May 6 10:57:55 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sebastian Andrzej Siewior X-Patchwork-Id: 12840997 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id B6712C433F5 for ; Fri, 6 May 2022 10:58:14 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1391102AbiEFLBz (ORCPT ); Fri, 6 May 2022 07:01:55 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:33104 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1391095AbiEFLBv (ORCPT ); Fri, 6 May 2022 07:01:51 -0400 Received: from galois.linutronix.de (Galois.linutronix.de [IPv6:2a0a:51c0:0:12e:550::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1758B183A1 for ; Fri, 6 May 2022 03:58:06 -0700 (PDT) From: Sebastian Andrzej Siewior DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1651834683; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=t51aPj6atti1TjwQSM11HjfqS0rAN9YXmURyukaAX5s=; b=W13TfLmet4ZK5qgd4vFKJVod65HTVP/jLyZtyl2Nplqxt4a0h/vIUntwk5mLBWk214A/Dc TB6W28B82ADHMol86YvXgQueaXBfsL9H9K5cxHnBmt9k0ASaFnxJYnJrwc3AoA/0YlC+M9 9ir24Y+pMcNI6/4IJgiXfpweyiPd/43wFTX54Cw6xn6MoDoKFVk4XtgoBhJLuRXdfIybtF 4x80pN0EiDqlzalQylyHMIgb2d15SKfdGwDzrBEoh69avtygBt2oLFhN0hX/6momNOMl7I AlvOB1m8OJMS6JO8tIArZ3N7Z+8jKn2rmVCG7YBYlzeVJOrG+SQEUUg+D5lCYQ== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1651834683; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=t51aPj6atti1TjwQSM11HjfqS0rAN9YXmURyukaAX5s=; b=m3ABj958awlvoW5uSpMwyS1BoY2bhNTeiLhrU+DdYvwQ4JAZqZGHCJJRJuUaDKHd/6fmmv Ii2qYK5Zmzk/sKCw== To: linux-scsi@vger.kernel.org Cc: "James E.J. Bottomley" , "Martin K. Petersen" , GR-QLogic-Storage-Upstream@marvell.com, Hannes Reinecke , Javed Hasan , Saurav Kashyap , Thomas Gleixner , Davidlohr Bueso , Davidlohr Bueso , Sebastian Andrzej Siewior Subject: [PATCH 1/4] scsi: fcoe: Add a local_lock to fcoe_percpu Date: Fri, 6 May 2022 12:57:55 +0200 Message-Id: <20220506105758.283887-2-bigeasy@linutronix.de> In-Reply-To: <20220506105758.283887-1-bigeasy@linutronix.de> References: <20220506105758.283887-1-bigeasy@linutronix.de> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org From: Davidlohr Bueso fcoe_get_paged_crc_eof() relies on the caller having preemption disabled to ensure the per-CPU fcoe_percpu context remains valid throughout the call. This is done by either holding spinlocks (such as bnx2fc_global_lock or qedf_global_lock) or the get_cpu() from fcoe_alloc_paged_crc_eof(). This last one breaks PREEMPT_RT semantics as there can be memory allocation and end up sleeping in atomic contexts. Introduce a local_lock_t to struct fcoe_percpu that will keep the non-RT case the same, mapping to preempt_disable/enable, while RT will use a per-CPU spinlock allowing the region to be preemptible but still maintain CPU locality. The other users of fcoe_percpu are already safe in this regard and do not require local_lock()ing. Signed-off-by: Davidlohr Bueso Acked-by: Sebastian Andrzej Siewior Signed-off-by: Sebastian Andrzej Siewior Link: https://lore.kernel.org/r/20211117025956.79616-3-dave@stgolabs.net --- drivers/scsi/fcoe/fcoe.c | 6 ++++-- include/scsi/libfcoe.h | 2 ++ 2 files changed, 6 insertions(+), 2 deletions(-) diff --git a/drivers/scsi/fcoe/fcoe.c b/drivers/scsi/fcoe/fcoe.c index 983e00135feac..af9b788823ac7 100644 --- a/drivers/scsi/fcoe/fcoe.c +++ b/drivers/scsi/fcoe/fcoe.c @@ -1453,9 +1453,10 @@ static int fcoe_alloc_paged_crc_eof(struct sk_buff *skb, int tlen) struct fcoe_percpu_s *fps; int rc; - fps = &get_cpu_var(fcoe_percpu); + local_lock(&fcoe_percpu.lock); + fps = this_cpu_ptr(&fcoe_percpu); rc = fcoe_get_paged_crc_eof(skb, tlen, fps); - put_cpu_var(fcoe_percpu); + local_unlock(&fcoe_percpu.lock); return rc; } @@ -2488,6 +2489,7 @@ static int __init fcoe_init(void) p = per_cpu_ptr(&fcoe_percpu, cpu); INIT_WORK(&p->work, fcoe_receive_work); skb_queue_head_init(&p->fcoe_rx_list); + local_lock_init(&p->lock); } /* Setup link change notification */ diff --git a/include/scsi/libfcoe.h b/include/scsi/libfcoe.h index 310e0dbffda99..279782156373a 100644 --- a/include/scsi/libfcoe.h +++ b/include/scsi/libfcoe.h @@ -14,6 +14,7 @@ #include #include #include +#include #include #include #include @@ -327,6 +328,7 @@ struct fcoe_percpu_s { struct sk_buff_head fcoe_rx_list; struct page *crc_eof_page; int crc_eof_offset; + local_lock_t lock; }; /** From patchwork Fri May 6 10:57:56 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sebastian Andrzej Siewior X-Patchwork-Id: 12840998 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id D24D8C433EF for ; Fri, 6 May 2022 10:58:16 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1391103AbiEFLB4 (ORCPT ); Fri, 6 May 2022 07:01:56 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:33108 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1391100AbiEFLBw (ORCPT ); Fri, 6 May 2022 07:01:52 -0400 Received: from galois.linutronix.de (Galois.linutronix.de [193.142.43.55]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D9B2722BEA for ; Fri, 6 May 2022 03:58:06 -0700 (PDT) From: Sebastian Andrzej Siewior DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1651834684; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=MGkD3iVME+BhlvIXayj/uprtMH0zghaRYvzGomdhdrI=; b=OSRWQcPQU4w2/mVRx6kSbwKF82GH6K9wlqBiYjIP/Hat2e56waI59UI0jGkbdkO/u/BhEf ebZpVTRgcr05PWW6V3ofSRy5MdsEHJSeptyk0ZJZ/ob8JiiS2CaPnMAz/reHH8ZE0QZKVk uenCsIb+wXpFSKvTWhDoj7sNJj6bWQAua07vpus++3VxGHggSMFqjaEC4tMIzZ33f7odAI jcBWuJcWK0YdRNOit3Kjaqx2r9fEr48icogG8ZDRrmeaVSdDPZmFufGDJYk81N5SU5pgPn DyqFodXvS7vVckS3htETjD3lpJ4QjsD2xJ1F03hl3etfY8wTUzL/7QcSp+R3PQ== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1651834684; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=MGkD3iVME+BhlvIXayj/uprtMH0zghaRYvzGomdhdrI=; b=CThwDiR7GlaRyygmdaSi4Oev1RBc3QiTiaSbnE2jLDuXUO1oqzjaTJe1BzMCrnApxwA+Vk L+S8RW0sEKejf8Dg== To: linux-scsi@vger.kernel.org Cc: "James E.J. Bottomley" , "Martin K. Petersen" , GR-QLogic-Storage-Upstream@marvell.com, Hannes Reinecke , Javed Hasan , Saurav Kashyap , Thomas Gleixner , Davidlohr Bueso , Sebastian Andrzej Siewior Subject: [PATCH 2/4] scsi: fcoe: Use per-CPU API to update per-CPU statistics. Date: Fri, 6 May 2022 12:57:56 +0200 Message-Id: <20220506105758.283887-3-bigeasy@linutronix.de> In-Reply-To: <20220506105758.283887-1-bigeasy@linutronix.de> References: <20220506105758.283887-1-bigeasy@linutronix.de> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org The per-CPU statistics (struct fc_stats) is updated by getting a stable per-CPU pointer via get_cpu() + per_cpu_ptr() and then performing the increment. This can be optimized by using this_cpu_*() which will do whatever is needed on the architecture to perform the update safe and efficient. The read out of the individual value (fc_get_host_stats()) should be done by using READ_ONCE() instead of a plain-C access. The difference is that READ_ONCE() will always perform a single access while the plain-C access can be splitt by the compiler into two loads if it appears beneficial. The usage of u64 has the side-effect that it is also 64bit wide on 32bit architectures and the read is always split into two loads. The can lead to strange values if the read happens during an update which alters both 32bit parts of the 64bit value. This can be circumvanted by either using a 32bit variables on 32bit architecures or extending the statistics with a sequence counter. Use this_cpu_*() API to update the statistics and READ_ONCE() to read it. Signed-off-by: Sebastian Andrzej Siewior Reviewed-by: Davidlohr Bueso --- drivers/scsi/bnx2fc/bnx2fc_fcoe.c | 22 +++++------------- drivers/scsi/bnx2fc/bnx2fc_io.c | 13 +++++------ drivers/scsi/fcoe/fcoe.c | 36 ++++++++++-------------------- drivers/scsi/fcoe/fcoe_ctlr.c | 26 ++++++++------------- drivers/scsi/fcoe/fcoe_transport.c | 6 ++--- drivers/scsi/libfc/fc_fcp.c | 29 +++++++++--------------- drivers/scsi/libfc/fc_lport.c | 30 ++++++++++++------------- drivers/scsi/qedf/qedf_main.c | 7 ++---- 8 files changed, 62 insertions(+), 107 deletions(-) diff --git a/drivers/scsi/bnx2fc/bnx2fc_fcoe.c b/drivers/scsi/bnx2fc/bnx2fc_fcoe.c index d295867a9b465..05ddbb9bb7d8a 100644 --- a/drivers/scsi/bnx2fc/bnx2fc_fcoe.c +++ b/drivers/scsi/bnx2fc/bnx2fc_fcoe.c @@ -273,7 +273,6 @@ static int bnx2fc_xmit(struct fc_lport *lport, struct fc_frame *fp) struct fcoe_port *port; struct fcoe_hdr *hp; struct bnx2fc_rport *tgt; - struct fc_stats *stats; u8 sof, eof; u32 crc; unsigned int hlen, tlen, elen; @@ -399,10 +398,8 @@ static int bnx2fc_xmit(struct fc_lport *lport, struct fc_frame *fp) } /*update tx stats */ - stats = per_cpu_ptr(lport->stats, get_cpu()); - stats->TxFrames++; - stats->TxWords += wlen; - put_cpu(); + this_cpu_inc(lport->stats->TxFrames); + this_cpu_add(lport->stats->TxWords, wlen); /* send down to lld */ fr_dev(fp) = lport; @@ -512,7 +509,6 @@ static void bnx2fc_recv_frame(struct sk_buff *skb) u32 fr_len, fr_crc; struct fc_lport *lport; struct fcoe_rcv_info *fr; - struct fc_stats *stats; struct fc_frame_header *fh; struct fcoe_crc_eof crc_eof; struct fc_frame *fp; @@ -543,10 +539,8 @@ static void bnx2fc_recv_frame(struct sk_buff *skb) skb_pull(skb, sizeof(struct fcoe_hdr)); fr_len = skb->len - sizeof(struct fcoe_crc_eof); - stats = per_cpu_ptr(lport->stats, get_cpu()); - stats->RxFrames++; - stats->RxWords += fr_len / FCOE_WORD_TO_BYTE; - put_cpu(); + this_cpu_inc(lport->stats->RxFrames); + this_cpu_add(lport->stats->RxWords, fr_len / FCOE_WORD_TO_BYTE); fp = (struct fc_frame *)skb; fc_frame_init(fp); @@ -633,9 +627,7 @@ static void bnx2fc_recv_frame(struct sk_buff *skb) fr_crc = le32_to_cpu(fr_crc(fp)); if (unlikely(fr_crc != ~crc32(~0, skb->data, fr_len))) { - stats = per_cpu_ptr(lport->stats, get_cpu()); - crc_err = (stats->InvalidCRCCount++); - put_cpu(); + crc_err = this_cpu_inc_return(lport->stats->InvalidCRCCount); if (crc_err < 5) printk(KERN_WARNING PFX "dropping frame with " "CRC error\n"); @@ -964,9 +956,7 @@ static void bnx2fc_indicate_netevent(void *context, unsigned long event, mutex_unlock(&lport->lp_mutex); fc_host_port_type(lport->host) = FC_PORTTYPE_UNKNOWN; - per_cpu_ptr(lport->stats, - get_cpu())->LinkFailureCount++; - put_cpu(); + this_cpu_inc(lport->stats->LinkFailureCount); fcoe_clean_pending_queue(lport); wait_for_upload = 1; } diff --git a/drivers/scsi/bnx2fc/bnx2fc_io.c b/drivers/scsi/bnx2fc/bnx2fc_io.c index 962454f2e2b16..6a1fc35b832ae 100644 --- a/drivers/scsi/bnx2fc/bnx2fc_io.c +++ b/drivers/scsi/bnx2fc/bnx2fc_io.c @@ -2032,7 +2032,6 @@ int bnx2fc_post_io_req(struct bnx2fc_rport *tgt, struct bnx2fc_interface *interface = port->priv; struct bnx2fc_hba *hba = interface->hba; struct fc_lport *lport = port->lport; - struct fc_stats *stats; int task_idx, index; u16 xid; @@ -2045,20 +2044,18 @@ int bnx2fc_post_io_req(struct bnx2fc_rport *tgt, io_req->data_xfer_len = scsi_bufflen(sc_cmd); bnx2fc_priv(sc_cmd)->io_req = io_req; - stats = per_cpu_ptr(lport->stats, get_cpu()); if (sc_cmd->sc_data_direction == DMA_FROM_DEVICE) { io_req->io_req_flags = BNX2FC_READ; - stats->InputRequests++; - stats->InputBytes += io_req->data_xfer_len; + this_cpu_inc(lport->stats->InputRequests); + this_cpu_add(lport->stats->InputBytes, io_req->data_xfer_len); } else if (sc_cmd->sc_data_direction == DMA_TO_DEVICE) { io_req->io_req_flags = BNX2FC_WRITE; - stats->OutputRequests++; - stats->OutputBytes += io_req->data_xfer_len; + this_cpu_inc(lport->stats->OutputRequests); + this_cpu_add(lport->stats->OutputBytes, io_req->data_xfer_len); } else { io_req->io_req_flags = 0; - stats->ControlRequests++; + this_cpu_inc(lport->stats->ControlRequests); } - put_cpu(); xid = io_req->xid; diff --git a/drivers/scsi/fcoe/fcoe.c b/drivers/scsi/fcoe/fcoe.c index af9b788823ac7..8bff94b5474cc 100644 --- a/drivers/scsi/fcoe/fcoe.c +++ b/drivers/scsi/fcoe/fcoe.c @@ -1434,8 +1434,7 @@ static int fcoe_rcv(struct sk_buff *skb, struct net_device *netdev, return NET_RX_SUCCESS; err: - per_cpu_ptr(lport->stats, get_cpu())->ErrorFrames++; - put_cpu(); + this_cpu_inc(lport->stats->ErrorFrames); err2: kfree_skb(skb); return NET_RX_DROP; @@ -1475,7 +1474,6 @@ static int fcoe_xmit(struct fc_lport *lport, struct fc_frame *fp) struct ethhdr *eh; struct fcoe_crc_eof *cp; struct sk_buff *skb; - struct fc_stats *stats; struct fc_frame_header *fh; unsigned int hlen; /* header length implies the version */ unsigned int tlen; /* trailer length */ @@ -1586,10 +1584,8 @@ static int fcoe_xmit(struct fc_lport *lport, struct fc_frame *fp) skb_shinfo(skb)->gso_size = 0; } /* update tx stats: regardless if LLD fails */ - stats = per_cpu_ptr(lport->stats, get_cpu()); - stats->TxFrames++; - stats->TxWords += wlen; - put_cpu(); + this_cpu_inc(lport->stats->TxFrames); + this_cpu_add(lport->stats->TxWords, wlen); /* send down to lld */ fr_dev(fp) = lport; @@ -1611,7 +1607,6 @@ static inline int fcoe_filter_frames(struct fc_lport *lport, struct fcoe_interface *fcoe; struct fc_frame_header *fh; struct sk_buff *skb = (struct sk_buff *)fp; - struct fc_stats *stats; /* * We only check CRC if no offload is available and if it is @@ -1641,11 +1636,8 @@ static inline int fcoe_filter_frames(struct fc_lport *lport, return 0; } - stats = per_cpu_ptr(lport->stats, get_cpu()); - stats->InvalidCRCCount++; - if (stats->InvalidCRCCount < 5) + if (this_cpu_inc_return(lport->stats->InvalidCRCCount) < 5) printk(KERN_WARNING "fcoe: dropping frame with CRC error\n"); - put_cpu(); return -EINVAL; } @@ -1658,7 +1650,6 @@ static void fcoe_recv_frame(struct sk_buff *skb) u32 fr_len; struct fc_lport *lport; struct fcoe_rcv_info *fr; - struct fc_stats *stats; struct fcoe_crc_eof crc_eof; struct fc_frame *fp; struct fcoe_hdr *hp; @@ -1686,9 +1677,11 @@ static void fcoe_recv_frame(struct sk_buff *skb) */ hp = (struct fcoe_hdr *) skb_network_header(skb); - stats = per_cpu_ptr(lport->stats, get_cpu()); if (unlikely(FC_FCOE_DECAPS_VER(hp) != FC_FCOE_VER)) { - if (stats->ErrorFrames < 5) + struct fc_stats *stats; + + stats = per_cpu_ptr(lport->stats, raw_smp_processor_id()); + if (READ_ONCE(stats->ErrorFrames) < 5) printk(KERN_WARNING "fcoe: FCoE version " "mismatch: The frame has " "version %x, but the " @@ -1701,8 +1694,8 @@ static void fcoe_recv_frame(struct sk_buff *skb) skb_pull(skb, sizeof(struct fcoe_hdr)); fr_len = skb->len - sizeof(struct fcoe_crc_eof); - stats->RxFrames++; - stats->RxWords += fr_len / FCOE_WORD_TO_BYTE; + this_cpu_inc(lport->stats->RxFrames); + this_cpu_add(lport->stats->RxWords, fr_len / FCOE_WORD_TO_BYTE); fp = (struct fc_frame *)skb; fc_frame_init(fp); @@ -1718,13 +1711,11 @@ static void fcoe_recv_frame(struct sk_buff *skb) goto drop; if (!fcoe_filter_frames(lport, fp)) { - put_cpu(); fc_exch_recv(lport, fp); return; } drop: - stats->ErrorFrames++; - put_cpu(); + this_cpu_inc(lport->stats->ErrorFrames); kfree_skb(skb); } @@ -1848,7 +1839,6 @@ static int fcoe_device_notification(struct notifier_block *notifier, struct net_device *netdev = netdev_notifier_info_to_dev(ptr); struct fcoe_ctlr *ctlr; struct fcoe_interface *fcoe; - struct fc_stats *stats; u32 link_possible = 1; u32 mfs; int rc = NOTIFY_OK; @@ -1922,9 +1912,7 @@ static int fcoe_device_notification(struct notifier_block *notifier, break; case FCOE_CTLR_ENABLED: case FCOE_CTLR_UNUSED: - stats = per_cpu_ptr(lport->stats, get_cpu()); - stats->LinkFailureCount++; - put_cpu(); + this_cpu_inc(lport->stats->LinkFailureCount); fcoe_clean_pending_queue(lport); } } diff --git a/drivers/scsi/fcoe/fcoe_ctlr.c b/drivers/scsi/fcoe/fcoe_ctlr.c index 558f3f4e18593..39e16eab47aad 100644 --- a/drivers/scsi/fcoe/fcoe_ctlr.c +++ b/drivers/scsi/fcoe/fcoe_ctlr.c @@ -824,22 +824,21 @@ static unsigned long fcoe_ctlr_age_fcfs(struct fcoe_ctlr *fip) unsigned long deadline; unsigned long sel_time = 0; struct list_head del_list; - struct fc_stats *stats; INIT_LIST_HEAD(&del_list); - stats = per_cpu_ptr(fip->lp->stats, get_cpu()); - list_for_each_entry_safe(fcf, next, &fip->fcfs, list) { deadline = fcf->time + fcf->fka_period + fcf->fka_period / 2; if (fip->sel_fcf == fcf) { if (time_after(jiffies, deadline)) { - stats->MissDiscAdvCount++; + u64 miss_cnt; + + miss_cnt = this_cpu_inc_return(fip->lp->stats->MissDiscAdvCount); printk(KERN_INFO "libfcoe: host%d: " "Missing Discovery Advertisement " "for fab %16.16llx count %lld\n", fip->lp->host->host_no, fcf->fabric_name, - stats->MissDiscAdvCount); + miss_cnt); } else if (time_after(next_timer, deadline)) next_timer = deadline; } @@ -855,7 +854,7 @@ static unsigned long fcoe_ctlr_age_fcfs(struct fcoe_ctlr *fip) */ list_del(&fcf->list); list_add(&fcf->list, &del_list); - stats->VLinkFailureCount++; + this_cpu_inc(fip->lp->stats->VLinkFailureCount); } else { if (time_after(next_timer, deadline)) next_timer = deadline; @@ -864,7 +863,6 @@ static unsigned long fcoe_ctlr_age_fcfs(struct fcoe_ctlr *fip) sel_time = fcf->time; } } - put_cpu(); list_for_each_entry_safe(fcf, next, &del_list, list) { /* Removes fcf from current list */ @@ -1142,7 +1140,6 @@ static void fcoe_ctlr_recv_els(struct fcoe_ctlr *fip, struct sk_buff *skb) struct fip_desc *desc; struct fip_encaps *els; struct fcoe_fcf *sel; - struct fc_stats *stats; enum fip_desc_type els_dtype = 0; u8 els_op; u8 sub; @@ -1286,10 +1283,8 @@ static void fcoe_ctlr_recv_els(struct fcoe_ctlr *fip, struct sk_buff *skb) fr_dev(fp) = lport; fr_encaps(fp) = els_dtype; - stats = per_cpu_ptr(lport->stats, get_cpu()); - stats->RxFrames++; - stats->RxWords += skb->len / FIP_BPW; - put_cpu(); + this_cpu_inc(lport->stats->RxFrames); + this_cpu_add(lport->stats->RxWords, skb->len / FIP_BPW); fc_exch_recv(lport, fp); return; @@ -1427,9 +1422,7 @@ static void fcoe_ctlr_recv_clr_vlink(struct fcoe_ctlr *fip, ntoh24(vp->fd_fc_id)); if (vn_port && (vn_port == lport)) { mutex_lock(&fip->ctlr_mutex); - per_cpu_ptr(lport->stats, - get_cpu())->VLinkFailureCount++; - put_cpu(); + this_cpu_inc(lport->stats->VLinkFailureCount); fcoe_ctlr_reset(fip); mutex_unlock(&fip->ctlr_mutex); } @@ -1457,8 +1450,7 @@ static void fcoe_ctlr_recv_clr_vlink(struct fcoe_ctlr *fip, * followed by physical port */ mutex_lock(&fip->ctlr_mutex); - per_cpu_ptr(lport->stats, get_cpu())->VLinkFailureCount++; - put_cpu(); + this_cpu_inc(lport->stats->VLinkFailureCount); fcoe_ctlr_reset(fip); mutex_unlock(&fip->ctlr_mutex); diff --git a/drivers/scsi/fcoe/fcoe_transport.c b/drivers/scsi/fcoe/fcoe_transport.c index 4d0e19e7c84b9..62341c6353a72 100644 --- a/drivers/scsi/fcoe/fcoe_transport.c +++ b/drivers/scsi/fcoe/fcoe_transport.c @@ -183,9 +183,9 @@ void __fcoe_get_lesb(struct fc_lport *lport, memset(lesb, 0, sizeof(*lesb)); for_each_possible_cpu(cpu) { stats = per_cpu_ptr(lport->stats, cpu); - lfc += stats->LinkFailureCount; - vlfc += stats->VLinkFailureCount; - mdac += stats->MissDiscAdvCount; + lfc += READ_ONCE(stats->LinkFailureCount); + vlfc += READ_ONCE(stats->VLinkFailureCount); + mdac += READ_ONCE(stats->MissDiscAdvCount); } lesb->lesb_link_fail = htonl(lfc); lesb->lesb_vlink_fail = htonl(vlfc); diff --git a/drivers/scsi/libfc/fc_fcp.c b/drivers/scsi/libfc/fc_fcp.c index bce90eb56c9ce..945adca5e72fd 100644 --- a/drivers/scsi/libfc/fc_fcp.c +++ b/drivers/scsi/libfc/fc_fcp.c @@ -143,8 +143,7 @@ static struct fc_fcp_pkt *fc_fcp_pkt_alloc(struct fc_lport *lport, gfp_t gfp) INIT_LIST_HEAD(&fsp->list); spin_lock_init(&fsp->scsi_pkt_lock); } else { - per_cpu_ptr(lport->stats, get_cpu())->FcpPktAllocFails++; - put_cpu(); + this_cpu_inc(lport->stats->FcpPktAllocFails); } return fsp; } @@ -266,8 +265,7 @@ static int fc_fcp_send_abort(struct fc_fcp_pkt *fsp) if (!fsp->seq_ptr) return -EINVAL; - per_cpu_ptr(fsp->lp->stats, get_cpu())->FcpPktAborts++; - put_cpu(); + this_cpu_inc(fsp->lp->stats->FcpPktAborts); fsp->state |= FC_SRB_ABORT_PENDING; rc = fc_seq_exch_abort(fsp->seq_ptr, 0); @@ -436,8 +434,7 @@ static inline struct fc_frame *fc_fcp_frame_alloc(struct fc_lport *lport, if (likely(fp)) return fp; - per_cpu_ptr(lport->stats, get_cpu())->FcpFrameAllocFails++; - put_cpu(); + this_cpu_inc(lport->stats->FcpFrameAllocFails); /* error case */ fc_fcp_can_queue_ramp_down(lport); shost_printk(KERN_ERR, lport->host, @@ -471,7 +468,6 @@ static void fc_fcp_recv_data(struct fc_fcp_pkt *fsp, struct fc_frame *fp) { struct scsi_cmnd *sc = fsp->cmd; struct fc_lport *lport = fsp->lp; - struct fc_stats *stats; struct fc_frame_header *fh; size_t start_offset; size_t offset; @@ -533,14 +529,12 @@ static void fc_fcp_recv_data(struct fc_fcp_pkt *fsp, struct fc_frame *fp) if (~crc != le32_to_cpu(fr_crc(fp))) { crc_err: - stats = per_cpu_ptr(lport->stats, get_cpu()); - stats->ErrorFrames++; + this_cpu_inc(lport->stats->ErrorFrames); /* per cpu count, not total count, but OK for limit */ - if (stats->InvalidCRCCount++ < FC_MAX_ERROR_CNT) + if (this_cpu_inc_return(lport->stats->InvalidCRCCount) < FC_MAX_ERROR_CNT) printk(KERN_WARNING "libfc: CRC error on data " "frame for port (%6.6x)\n", lport->port_id); - put_cpu(); /* * Assume the frame is total garbage. * We may have copied it over the good part @@ -1861,7 +1855,6 @@ int fc_queuecommand(struct Scsi_Host *shost, struct scsi_cmnd *sc_cmd) struct fc_fcp_pkt *fsp; int rval; int rc = 0; - struct fc_stats *stats; rval = fc_remote_port_chkready(rport); if (rval) { @@ -1913,20 +1906,18 @@ int fc_queuecommand(struct Scsi_Host *shost, struct scsi_cmnd *sc_cmd) /* * setup the data direction */ - stats = per_cpu_ptr(lport->stats, get_cpu()); if (sc_cmd->sc_data_direction == DMA_FROM_DEVICE) { fsp->req_flags = FC_SRB_READ; - stats->InputRequests++; - stats->InputBytes += fsp->data_len; + this_cpu_inc(lport->stats->InputRequests); + this_cpu_add(lport->stats->InputBytes, fsp->data_len); } else if (sc_cmd->sc_data_direction == DMA_TO_DEVICE) { fsp->req_flags = FC_SRB_WRITE; - stats->OutputRequests++; - stats->OutputBytes += fsp->data_len; + this_cpu_inc(lport->stats->OutputRequests); + this_cpu_add(lport->stats->OutputBytes, fsp->data_len); } else { fsp->req_flags = 0; - stats->ControlRequests++; + this_cpu_inc(lport->stats->ControlRequests); } - put_cpu(); /* * send it to the lower layer diff --git a/drivers/scsi/libfc/fc_lport.c b/drivers/scsi/libfc/fc_lport.c index 19cd4a95d354d..9c02c9523c4d4 100644 --- a/drivers/scsi/libfc/fc_lport.c +++ b/drivers/scsi/libfc/fc_lport.c @@ -308,21 +308,21 @@ struct fc_host_statistics *fc_get_host_stats(struct Scsi_Host *shost) stats = per_cpu_ptr(lport->stats, cpu); - fc_stats->tx_frames += stats->TxFrames; - fc_stats->tx_words += stats->TxWords; - fc_stats->rx_frames += stats->RxFrames; - fc_stats->rx_words += stats->RxWords; - fc_stats->error_frames += stats->ErrorFrames; - fc_stats->invalid_crc_count += stats->InvalidCRCCount; - fc_stats->fcp_input_requests += stats->InputRequests; - fc_stats->fcp_output_requests += stats->OutputRequests; - fc_stats->fcp_control_requests += stats->ControlRequests; - fcp_in_bytes += stats->InputBytes; - fcp_out_bytes += stats->OutputBytes; - fc_stats->fcp_packet_alloc_failures += stats->FcpPktAllocFails; - fc_stats->fcp_packet_aborts += stats->FcpPktAborts; - fc_stats->fcp_frame_alloc_failures += stats->FcpFrameAllocFails; - fc_stats->link_failure_count += stats->LinkFailureCount; + fc_stats->tx_frames += READ_ONCE(stats->TxFrames); + fc_stats->tx_words += READ_ONCE(stats->TxWords); + fc_stats->rx_frames += READ_ONCE(stats->RxFrames); + fc_stats->rx_words += READ_ONCE(stats->RxWords); + fc_stats->error_frames += READ_ONCE(stats->ErrorFrames); + fc_stats->invalid_crc_count += READ_ONCE(stats->InvalidCRCCount); + fc_stats->fcp_input_requests += READ_ONCE(stats->InputRequests); + fc_stats->fcp_output_requests += READ_ONCE(stats->OutputRequests); + fc_stats->fcp_control_requests += READ_ONCE(stats->ControlRequests); + fcp_in_bytes += READ_ONCE(stats->InputBytes); + fcp_out_bytes += READ_ONCE(stats->OutputBytes); + fc_stats->fcp_packet_alloc_failures += READ_ONCE(stats->FcpPktAllocFails); + fc_stats->fcp_packet_aborts += READ_ONCE(stats->FcpPktAborts); + fc_stats->fcp_frame_alloc_failures += READ_ONCE(stats->FcpFrameAllocFails); + fc_stats->link_failure_count += READ_ONCE(stats->LinkFailureCount); } fc_stats->fcp_input_megabytes = div_u64(fcp_in_bytes, 1000000); fc_stats->fcp_output_megabytes = div_u64(fcp_out_bytes, 1000000); diff --git a/drivers/scsi/qedf/qedf_main.c b/drivers/scsi/qedf/qedf_main.c index cbe01915da8c1..3d6b137314f3f 100644 --- a/drivers/scsi/qedf/qedf_main.c +++ b/drivers/scsi/qedf/qedf_main.c @@ -1067,7 +1067,6 @@ static int qedf_xmit(struct fc_lport *lport, struct fc_frame *fp) u32 crc; unsigned int hlen, tlen, elen; int wlen; - struct fc_stats *stats; struct fc_lport *tmp_lport; struct fc_lport *vn_port = NULL; struct qedf_rport *fcport; @@ -1215,10 +1214,8 @@ static int qedf_xmit(struct fc_lport *lport, struct fc_frame *fp) hp->fcoe_sof = sof; /*update tx stats */ - stats = per_cpu_ptr(lport->stats, get_cpu()); - stats->TxFrames++; - stats->TxWords += wlen; - put_cpu(); + this_cpu_inc(lport->stats->TxFrames); + this_cpu_add(lport->stats->TxWords, wlen); /* Get VLAN ID from skb for printing purposes */ __vlan_hwaccel_get_tag(skb, &vlan_tci); From patchwork Fri May 6 10:57:57 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sebastian Andrzej Siewior X-Patchwork-Id: 12840996 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6E5EDC433F5 for ; Fri, 6 May 2022 10:58:12 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1391101AbiEFLBx (ORCPT ); Fri, 6 May 2022 07:01:53 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:33106 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1391096AbiEFLBv (ORCPT ); Fri, 6 May 2022 07:01:51 -0400 Received: from galois.linutronix.de (Galois.linutronix.de [IPv6:2a0a:51c0:0:12e:550::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1B7EF1F614 for ; Fri, 6 May 2022 03:58:06 -0700 (PDT) From: Sebastian Andrzej Siewior DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1651834684; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=J7v1Eab4GLB095312EVi4rIlE+wlQX1zpsqmd0x9ybM=; b=Cb5ujYm5L443htQigITpl6UGitu5IHjn5vht87GJWbprdM/b6lJbwtlmDxz89T/Wz8KN+Q z5qWiQPLr6/0AsAaI4LU8cWJ7m5Z0O0MRVT7+dDTBn62fZzh3aLaDmgKfj1f/14lu22WUF fzAWmfHV8l024igzLfDS6WM5FiEy+ayZ9JGwJMSk1akZaR8gU7e97fjA+Cw6PBG05Cmtzj DFgwW88drSGuzeYn7l09RVfqvCjFTVvfI95sHZsHe2FACnfw755C/JBgz51wQca3GkObAi 20WW7VlFLnh6zzqpkjxyLdF4p6QHIY1N9QNcruid5HeeYAtRiLYxhkVL/LCf6w== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1651834684; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=J7v1Eab4GLB095312EVi4rIlE+wlQX1zpsqmd0x9ybM=; b=xhopbmP0VMnlwzROnbcyYAFdYvSx7w+EMur+vAIJjPMTfuXv5mDwr/fSVsZa+/vbmh41Ms La+Vs/4ebZxlaFCw== To: linux-scsi@vger.kernel.org Cc: "James E.J. Bottomley" , "Martin K. Petersen" , GR-QLogic-Storage-Upstream@marvell.com, Hannes Reinecke , Javed Hasan , Saurav Kashyap , Thomas Gleixner , Davidlohr Bueso , Davidlohr Bueso , Sebastian Andrzej Siewior Subject: [PATCH 3/4] scsi: libfc: Remove get_cpu() semantics in fc_exch_em_alloc() Date: Fri, 6 May 2022 12:57:57 +0200 Message-Id: <20220506105758.283887-4-bigeasy@linutronix.de> In-Reply-To: <20220506105758.283887-1-bigeasy@linutronix.de> References: <20220506105758.283887-1-bigeasy@linutronix.de> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org From: Davidlohr Bueso The get_cpu() in fc_exch_em_alloc() was introduced in: f018b73af6db ([SCSI] libfc, libfcoe, fcoe: use smp_processor_id() only when preempt disabled) for no other reason than to simply use smp_processor_id() without getting a warning, because everything is done with the pool->lock held anyway. However, get_cpu(), by disabling preemption, does not play well with PREEMPT_RT, particularly when acquiring a regular (and thus sleepable) spinlock. Therefore remove the get_cpu() and just use the unstable value as we will have CPU locality guarantees next by taking the lock. The window of migration, as noted by Sebastian, is small and even if it happens the result is correct. Signed-off-by: Davidlohr Bueso Acked-by: Sebastian Andrzej Siewior Signed-off-by: Sebastian Andrzej Siewior Link: https://lore.kernel.org/r/20211117025956.79616-2-dave@stgolabs.net --- drivers/scsi/libfc/fc_exch.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/drivers/scsi/libfc/fc_exch.c b/drivers/scsi/libfc/fc_exch.c index aa223db4cf53c..1d91c457527f3 100644 --- a/drivers/scsi/libfc/fc_exch.c +++ b/drivers/scsi/libfc/fc_exch.c @@ -825,10 +825,9 @@ static struct fc_exch *fc_exch_em_alloc(struct fc_lport *lport, } memset(ep, 0, sizeof(*ep)); - cpu = get_cpu(); + cpu = raw_smp_processor_id(); pool = per_cpu_ptr(mp->pool, cpu); spin_lock_bh(&pool->lock); - put_cpu(); /* peek cache of free slot */ if (pool->left != FC_XID_UNKNOWN) { From patchwork Fri May 6 10:57:58 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sebastian Andrzej Siewior X-Patchwork-Id: 12840995 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2E550C433EF for ; Fri, 6 May 2022 10:58:12 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1391099AbiEFLBw (ORCPT ); Fri, 6 May 2022 07:01:52 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:33094 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1391083AbiEFLBu (ORCPT ); Fri, 6 May 2022 07:01:50 -0400 Received: from galois.linutronix.de (Galois.linutronix.de [IPv6:2a0a:51c0:0:12e:550::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1683F64EF for ; Fri, 6 May 2022 03:58:06 -0700 (PDT) From: Sebastian Andrzej Siewior DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1651834684; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=az714Uw6U2hRUUDjtCNatC/A4A3mwY8ik7Gck4abaUE=; b=pYy7m/fRmT9GTSC8EOKLaog2HsBf+u8Og1eXgbxRrIIsa6xQNOey5naDUJn83EwuMcIw6R uHrQ+cb1yVFEVSWMfa9dwWRHDvPJzB39sdYSpdcZDA/86GWyuyA8xFAu0tuv7Byv8kTqjQ lrXe6WGramtrD/ZnFgB412ulh8pP6V7694rrUZBkz6dfO27Cu/mOqWf/d7peA+GSbV0IUz B2nl/mak/snI6u+2c15HY+H4L7nGZf9ay+7pxUF4YjgfOmBRUHqSOczvtrPnUBW5P405jk kErMTLN/LeglOgUYU54UrnF8TJi+2P+uZBdN3BOXY5H7y3vhQEnQwYpyd9lXCQ== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1651834684; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=az714Uw6U2hRUUDjtCNatC/A4A3mwY8ik7Gck4abaUE=; b=M3kMqvOwgm3rYeie4SJAafVOKjee7cY+QOxLnnFPCzIIEaOChdIeVf5CUR+n1H537k/+IQ hevussZJFh3XsBCQ== To: linux-scsi@vger.kernel.org Cc: "James E.J. Bottomley" , "Martin K. Petersen" , GR-QLogic-Storage-Upstream@marvell.com, Hannes Reinecke , Javed Hasan , Saurav Kashyap , Thomas Gleixner , Davidlohr Bueso , Sebastian Andrzej Siewior Subject: [PATCH 4/4] scsi: bnx2fc: Avoid using get_cpu() in bnx2fc_cmd_alloc(). Date: Fri, 6 May 2022 12:57:58 +0200 Message-Id: <20220506105758.283887-5-bigeasy@linutronix.de> In-Reply-To: <20220506105758.283887-1-bigeasy@linutronix.de> References: <20220506105758.283887-1-bigeasy@linutronix.de> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org Using get_cpu() leads to disabling preemption and in this context it is not possible to acquire the following spinlock_t on PREEMPT_RT because it becomes a sleeping lock. Commit 0ea5c27583e1c ("[SCSI] bnx2fc: common free list for cleanup commands") says that it is using get_cpu() as a fix in case the CPU is preempted. While this might be true, the important part is that it is now using the same CPU for locking and unlocking while previously it always relied on smp_processor_id(). The date structure itself is protected with a lock so it does not rely on CPU-local access. Replace get_cpu() with raw_smp_processor_id() to obtain the current CPU number which is used as an index for the per-CPU resource. Signed-off-by: Sebastian Andrzej Siewior Reviewed-by: Davidlohr Bueso --- drivers/scsi/bnx2fc/bnx2fc_io.c | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/drivers/scsi/bnx2fc/bnx2fc_io.c b/drivers/scsi/bnx2fc/bnx2fc_io.c index 6a1fc35b832ae..b42a9accb8320 100644 --- a/drivers/scsi/bnx2fc/bnx2fc_io.c +++ b/drivers/scsi/bnx2fc/bnx2fc_io.c @@ -472,7 +472,7 @@ struct bnx2fc_cmd *bnx2fc_cmd_alloc(struct bnx2fc_rport *tgt) u32 free_sqes; u32 max_sqes; u16 xid; - int index = get_cpu(); + int index = raw_smp_processor_id(); max_sqes = BNX2FC_SCSI_MAX_SQES; /* @@ -485,7 +485,6 @@ struct bnx2fc_cmd *bnx2fc_cmd_alloc(struct bnx2fc_rport *tgt) (tgt->num_active_ios.counter >= max_sqes) || (free_sqes + max_sqes <= BNX2FC_SQ_WQES_MAX)) { spin_unlock_bh(&cmd_mgr->free_list_lock[index]); - put_cpu(); return NULL; } @@ -498,7 +497,6 @@ struct bnx2fc_cmd *bnx2fc_cmd_alloc(struct bnx2fc_rport *tgt) atomic_inc(&tgt->num_active_ios); atomic_dec(&tgt->free_sqes); spin_unlock_bh(&cmd_mgr->free_list_lock[index]); - put_cpu(); INIT_LIST_HEAD(&io_req->link);