From patchwork Fri Apr 26 00:20:46 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jes Sorensen X-Patchwork-Id: 10917859 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id D75F4912 for ; Fri, 26 Apr 2019 00:21:18 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id C7A7528CFB for ; Fri, 26 Apr 2019 00:21:18 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id BB2E728D36; Fri, 26 Apr 2019 00:21:18 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 6777128D55 for ; Fri, 26 Apr 2019 00:21:18 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727284AbfDZAVR (ORCPT ); Thu, 25 Apr 2019 20:21:17 -0400 Received: from mail-yw1-f65.google.com ([209.85.161.65]:42221 "EHLO mail-yw1-f65.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726703AbfDZAVR (ORCPT ); Thu, 25 Apr 2019 20:21:17 -0400 Received: by mail-yw1-f65.google.com with SMTP id y131so565597ywa.9 for ; Thu, 25 Apr 2019 17:21:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=AcZ92YH0X8djThbWKLrf0MusGXZCJVm/wjWNJQyN0zw=; b=Pt5+uZZMse7LoMU02YGfkh4V6NNM64zWc5jb9uFADxiyv+MhXQSeea4tJhtJ6ateIU TjfIQ6AUYy80Fv714H8biQFs+ucUtcbeBPvuz1MywCKx4dsOPMz4uC3V9ieyDXvmz1Sn h5phx3a+1q72UokLDxMmnSrrL2lch65anF6vhF3VoMkSdfPx6vg6mgQobLFVap7mUjIo 57OC4A9R7JSvnbRZrNfEKM4TglkzXhq6bj5n9qa7EBfUhqtJ5VUOVaFcbkzO2LJoaFNS spF38xpGDA7T22sds3WQJssi3Z3sars1obRiC9DlffwiSN59xy/51ce9t6Xx9ffLchI0 5S6g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=AcZ92YH0X8djThbWKLrf0MusGXZCJVm/wjWNJQyN0zw=; b=jlIhG8Au3h83Q1mEsTXad6dO8xyyVWs7qlYWIIa2GaO+uI8g8+FtJMFS6t0i+F1Dr4 kTCItKNGrpfWwgtYcmrQwQMtS+ZC5bhYX9/E1Jc/rmgYuQZV5DHrz9DzVBsIGzVrYi/q 8o9pVTGe+auy+3PUImw0L5nl5HQ3pI0r1F2Cg4pRJirTw5FltEAU1rYbe2DmEwjHxTj+ 2KV8fkCAg8DGnRviiIQ+dk4AG5Pg5TnMjOEY5RCE6ngOCJmZjlALTL+kDBNjWlrgcAjL mOIanYvfhN9X6YXkbfvmtfaJIqA/1o4TciDTMitoDeAichpnBEwt6AJkcZClBHi+aJ6v +Yyw== X-Gm-Message-State: APjAAAXbwfUpKgwFEtD6rl5oHZG4eMvcovcGx3cwSoDK0p5vuoX2ghcK A13ucOFDHqEyh7iGiZVcdl3eCDr7 X-Google-Smtp-Source: APXvYqzqvHdaCYxCvoGRQIvKuzAd5308rO5l0Ekt+Tuq+WnwjTRHY+VJG9ytl+Uiqfq5csIFwXYFOw== X-Received: by 2002:a81:4cd5:: with SMTP id z204mr13469334ywa.318.1556238076159; Thu, 25 Apr 2019 17:21:16 -0700 (PDT) Received: from localhost ([2620:10d:c091:180::5484]) by smtp.gmail.com with ESMTPSA id 193sm8022184ywf.22.2019.04.25.17.21.15 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Thu, 25 Apr 2019 17:21:15 -0700 (PDT) From: Jes Sorensen X-Google-Original-From: Jes Sorensen To: linux-block@vger.kernel.org Cc: josef@toxicpanda.com, kernel-team@fb.com Subject: [PATCH 1/5] blk-stat: rename batch to time Date: Thu, 25 Apr 2019 20:20:46 -0400 Message-Id: <20190426002050.15499-2-Jes.Sorensen@gmail.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190426002050.15499-1-Jes.Sorensen@gmail.com> References: <20190426002050.15499-1-Jes.Sorensen@gmail.com> MIME-Version: 1.0 Sender: linux-block-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Josef Bacik Batch isn't at all related to what it's actually used for, which is the running sum of time spent doing IO. Rename it to time so it makes logical sense. Signed-off-by: Josef Bacik --- block/blk-stat.c | 6 +++--- include/linux/blk_types.h | 2 +- 2 files changed, 4 insertions(+), 4 deletions(-) diff --git a/block/blk-stat.c b/block/blk-stat.c index 696a04176e4d..8a40ac719ba7 100644 --- a/block/blk-stat.c +++ b/block/blk-stat.c @@ -21,7 +21,7 @@ void blk_rq_stat_init(struct blk_rq_stat *stat) { stat->min = -1ULL; stat->max = stat->nr_samples = stat->mean = 0; - stat->batch = 0; + stat->time = 0; } /* src is a per-cpu stat, mean isn't initialized */ @@ -33,7 +33,7 @@ void blk_rq_stat_sum(struct blk_rq_stat *dst, struct blk_rq_stat *src) dst->min = min(dst->min, src->min); dst->max = max(dst->max, src->max); - dst->mean = div_u64(src->batch + dst->mean * dst->nr_samples, + dst->mean = div_u64(src->time + dst->mean * dst->nr_samples, dst->nr_samples + src->nr_samples); dst->nr_samples += src->nr_samples; @@ -43,7 +43,7 @@ void blk_rq_stat_add(struct blk_rq_stat *stat, u64 value) { stat->min = min(stat->min, value); stat->max = max(stat->max, value); - stat->batch += value; + stat->time += value; stat->nr_samples++; } diff --git a/include/linux/blk_types.h b/include/linux/blk_types.h index 791fee35df88..df24baf455c2 100644 --- a/include/linux/blk_types.h +++ b/include/linux/blk_types.h @@ -446,7 +446,7 @@ struct blk_rq_stat { u64 min; u64 max; u32 nr_samples; - u64 batch; + u64 time; }; #endif /* __LINUX_BLK_TYPES_H */ From patchwork Fri Apr 26 00:20:47 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jes Sorensen X-Patchwork-Id: 10917861 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 514CE912 for ; Fri, 26 Apr 2019 00:21:24 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 41F5928CEF for ; Fri, 26 Apr 2019 00:21:24 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 35DBF28D54; Fri, 26 Apr 2019 00:21:24 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 1A47F28D51 for ; Fri, 26 Apr 2019 00:21:23 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727387AbfDZAVW (ORCPT ); Thu, 25 Apr 2019 20:21:22 -0400 Received: from mail-yb1-f196.google.com ([209.85.219.196]:36393 "EHLO mail-yb1-f196.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726703AbfDZAVW (ORCPT ); Thu, 25 Apr 2019 20:21:22 -0400 Received: by mail-yb1-f196.google.com with SMTP id e76so675359ybc.3 for ; Thu, 25 Apr 2019 17:21:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=+VnJO5Y4pDdKSbJZjy4lN6QclYi9c5ZuXMxvcyYdIjs=; b=OlpxipRwi1+2ScB4jzV3bi07wBHuQhPzlnPv6M3G5EmoxC5Wj06yV6Eo7ixlKUATPx C5zXX7xPKsjwnI2PCPRWWA+ZA28R8A8J0zq1GHRSzR3Mzmqaz70qgpImH+fI50PKoV6J hEQh635htBhquqTApNrBsCXjxlyxcy7zIaGuufgzqZphVjVHJCpbJtynwj5WgOE7YSpr 0qiZZSKBTWJHt6a7YMQZHut4JUmtXa2GD6MKCAtr3OFnULApGBjd5wpqFdjK3xuvsrVL 6YH5WOphUbL8HMEU3qQlP0AQUa+ZH0lLrkEe0HYstSddVwDqfnSBKcfitBR1r/cafoQa 4kGg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=+VnJO5Y4pDdKSbJZjy4lN6QclYi9c5ZuXMxvcyYdIjs=; b=TxiYRbUAGfgB8M+LKT9ptIdWl0Ry1VmrynAfc1ascQLUTysktZ6WySwb6jmfhq+xBv /VNbR31RIfTwgotbjOUrQkP4IS0B8rriAR3hw1A1hp8BAdDI8ZGo8rODc+KPr1HWtko7 oTQHjYonJbj1GmB3iqYjhQVS+77E9BioD3HA112sE7oGZN1gEA8dE36UlJe3xi1kZi+g 8mtz7fo2+eQI5QVHcJQakn5BGLANHs9fpZe9nwKmBRH1MYTQDHUWQppbMWN/jdGbhQwV OO+7U0Mk5T4s3u6K+PPI60kNQP5W1erFT2ds4fkO2JeZzVMm0V8JJNrOxkYfnO7a3nZ/ nOIw== X-Gm-Message-State: APjAAAUJIPBelVzbUw2VmdyJk0RK9iAOCMUwG7BYIBqh46jUfXQD4CtB nIoOghYQHcvXaT6QBV7Z2rUChqJm X-Google-Smtp-Source: APXvYqwtXtNX6vRnIE271WgyOLzGrVmEEnuqupirymIbrnD3S+wsIIN/FJooScAaNkykDpJq8XQ+fw== X-Received: by 2002:a25:3554:: with SMTP id c81mr33447317yba.498.1556238080606; Thu, 25 Apr 2019 17:21:20 -0700 (PDT) Received: from localhost ([2620:10d:c091:180::5484]) by smtp.gmail.com with ESMTPSA id h204sm25964134ywh.110.2019.04.25.17.21.19 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Thu, 25 Apr 2019 17:21:20 -0700 (PDT) From: Jes Sorensen X-Google-Original-From: Jes Sorensen To: linux-block@vger.kernel.org Cc: josef@toxicpanda.com, kernel-team@fb.com Subject: [PATCH 2/5] block: keep track of per-device io sizes in stats Date: Thu, 25 Apr 2019 20:20:47 -0400 Message-Id: <20190426002050.15499-3-Jes.Sorensen@gmail.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190426002050.15499-1-Jes.Sorensen@gmail.com> References: <20190426002050.15499-1-Jes.Sorensen@gmail.com> MIME-Version: 1.0 Sender: linux-block-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Josef Bacik In order to track things like bw we need to know the number of bytes done per device. Add a io_bytes field to the struct request and populate this at start time. Add a field to the blk_rq_stat to hold this information so that consumers of the blk_rq_stat stuff can use it. Signed-off-by: Josef Bacik --- block/blk-iolatency.c | 2 +- block/blk-mq.c | 4 +--- block/blk-stat.c | 14 ++++++++------ block/blk-stat.h | 2 +- block/blk-throttle.c | 3 ++- include/linux/blk_types.h | 1 + include/linux/blkdev.h | 5 ++--- 7 files changed, 16 insertions(+), 15 deletions(-) diff --git a/block/blk-iolatency.c b/block/blk-iolatency.c index 507212d75ee2..8e547958cc1b 100644 --- a/block/blk-iolatency.c +++ b/block/blk-iolatency.c @@ -218,7 +218,7 @@ static inline void latency_stat_record_time(struct iolatency_grp *iolat, stat->ps.missed++; stat->ps.total++; } else - blk_rq_stat_add(&stat->rqs, req_time); + blk_rq_stat_add(&stat->rqs, req_time, 0); put_cpu_ptr(stat); } diff --git a/block/blk-mq.c b/block/blk-mq.c index 65f628652b1e..e8cd2028745a 100644 --- a/block/blk-mq.c +++ b/block/blk-mq.c @@ -677,9 +677,7 @@ void blk_mq_start_request(struct request *rq) if (test_bit(QUEUE_FLAG_STATS, &q->queue_flags)) { rq->io_start_time_ns = ktime_get_ns(); -#ifdef CONFIG_BLK_DEV_THROTTLING_LOW - rq->throtl_size = blk_rq_sectors(rq); -#endif + rq->io_bytes = blk_rq_bytes(rq); rq->rq_flags |= RQF_STATS; rq_qos_issue(q, rq); } diff --git a/block/blk-stat.c b/block/blk-stat.c index 8a40ac719ba7..d718983bed5c 100644 --- a/block/blk-stat.c +++ b/block/blk-stat.c @@ -21,7 +21,7 @@ void blk_rq_stat_init(struct blk_rq_stat *stat) { stat->min = -1ULL; stat->max = stat->nr_samples = stat->mean = 0; - stat->time = 0; + stat->size = stat->time = 0; } /* src is a per-cpu stat, mean isn't initialized */ @@ -37,13 +37,15 @@ void blk_rq_stat_sum(struct blk_rq_stat *dst, struct blk_rq_stat *src) dst->nr_samples + src->nr_samples); dst->nr_samples += src->nr_samples; + dst->size += src->size; } -void blk_rq_stat_add(struct blk_rq_stat *stat, u64 value) +void blk_rq_stat_add(struct blk_rq_stat *stat, u64 time, u64 size) { - stat->min = min(stat->min, value); - stat->max = max(stat->max, value); - stat->time += value; + stat->min = min(stat->min, time); + stat->max = max(stat->max, time); + stat->time += time; + stat->size += size; stat->nr_samples++; } @@ -69,7 +71,7 @@ void blk_stat_add(struct request *rq, u64 now) continue; stat = &get_cpu_ptr(cb->cpu_stat)[bucket]; - blk_rq_stat_add(stat, value); + blk_rq_stat_add(stat, value, rq->io_bytes); put_cpu_ptr(cb->cpu_stat); } rcu_read_unlock(); diff --git a/block/blk-stat.h b/block/blk-stat.h index 17b47a86eefb..ea893c4a9af1 100644 --- a/block/blk-stat.h +++ b/block/blk-stat.h @@ -164,7 +164,7 @@ static inline void blk_stat_activate_msecs(struct blk_stat_callback *cb, mod_timer(&cb->timer, jiffies + msecs_to_jiffies(msecs)); } -void blk_rq_stat_add(struct blk_rq_stat *, u64); +void blk_rq_stat_add(struct blk_rq_stat *, u64, u64); void blk_rq_stat_sum(struct blk_rq_stat *, struct blk_rq_stat *); void blk_rq_stat_init(struct blk_rq_stat *); diff --git a/block/blk-throttle.c b/block/blk-throttle.c index 1b97a73d2fb1..b07e5feda553 100644 --- a/block/blk-throttle.c +++ b/block/blk-throttle.c @@ -2248,8 +2248,9 @@ void blk_throtl_stat_add(struct request *rq, u64 time_ns) { struct request_queue *q = rq->q; struct throtl_data *td = q->td; + sector_t size = rq->io_bytes >> SECTOR_SHIFT; - throtl_track_latency(td, rq->throtl_size, req_op(rq), time_ns >> 10); + throtl_track_latency(td, size, req_op(rq), time_ns >> 10); } void blk_throtl_bio_endio(struct bio *bio) diff --git a/include/linux/blk_types.h b/include/linux/blk_types.h index df24baf455c2..eb64361bc2a5 100644 --- a/include/linux/blk_types.h +++ b/include/linux/blk_types.h @@ -447,6 +447,7 @@ struct blk_rq_stat { u64 max; u32 nr_samples; u64 time; + u64 size; }; #endif /* __LINUX_BLK_TYPES_H */ diff --git a/include/linux/blkdev.h b/include/linux/blkdev.h index 317ab30d2904..a2291b1d1cb0 100644 --- a/include/linux/blkdev.h +++ b/include/linux/blkdev.h @@ -198,13 +198,12 @@ struct request { u64 start_time_ns; /* Time that I/O was submitted to the device. */ u64 io_start_time_ns; + /* Bytes submitted to the device. */ + unsigned int io_bytes; #ifdef CONFIG_BLK_WBT unsigned short wbt_flags; #endif -#ifdef CONFIG_BLK_DEV_THROTTLING_LOW - unsigned short throtl_size; -#endif /* * Number of scatter-gather DMA addr+len pairs after From patchwork Fri Apr 26 00:20:48 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jes Sorensen X-Patchwork-Id: 10917863 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 34C971515 for ; Fri, 26 Apr 2019 00:21:28 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 26D0928CF2 for ; Fri, 26 Apr 2019 00:21:28 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 1B50128D12; Fri, 26 Apr 2019 00:21:28 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 749CA28D15 for ; Fri, 26 Apr 2019 00:21:27 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726703AbfDZAV0 (ORCPT ); Thu, 25 Apr 2019 20:21:26 -0400 Received: from mail-yw1-f68.google.com ([209.85.161.68]:32802 "EHLO mail-yw1-f68.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726546AbfDZAV0 (ORCPT ); Thu, 25 Apr 2019 20:21:26 -0400 Received: by mail-yw1-f68.google.com with SMTP id q11so590288ywb.0 for ; Thu, 25 Apr 2019 17:21:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=iN/ceYqNGPSDkakr9QvuBsL28MVNsT42sKDm8nAFabI=; b=EeZOWhvLGBBj6VZjg3y080RQx1JZYxPiE0EQhnImIWQFsqMo6kwPwpvUH1vLBJ36CJ XAv6eJzZaJuAIUOQUNrFHHl8LbNjc00lWD2fl97ljhsqstRmJXjwDPTvH7zRPfnClznH KNRWRLVUSNzleYBdBx1g1R4XIMO2C7oMy1/jb9LIQwObLikx9CNGXd4aOqepnK0WGgBY DaujbDJx1JLGksQQRcGVkous/O6IWqUKOGcQfCcCcm308l4iGr3SHSXX9oDOIxGk+VIe Z3rHs5H22QKbaXVYddCDHpJd+WAeAlzYwHUMsvrjzfYBgKKExZZRBve32ppDZuHMSufq dBZw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=iN/ceYqNGPSDkakr9QvuBsL28MVNsT42sKDm8nAFabI=; b=PjwtiKTcIevtSgnae+xL/RWtDqanlgVeSqZUdq77mxdLlXa1vQQrwNbSWuQi6H3GMm kX9UdG5BKOerUx7ukEige3uhLrLen3anenxf+9DDshidWclLI9mc5rR4RhfKMJnix4LN UW1GdyI/xktsGdNPlxhUivWxQsmSzYXofZfd7hiqpmaPFLSvl+QW7ZJLDFRSPPQQ1ydL I4yGTgNBHgKxaLI3x0UGWbZI7DN3FblCDcALXZp3d8ZkdWe8OZzxYczNheKQD0S6sh2v i6BLi+wNKEGgpMnhqQCoXbhR9G966IRddpjhmvGtEZLqVyUv0mmydr/kg4/hcwfVB1kJ Bo+w== X-Gm-Message-State: APjAAAVGj2mM6JK6bhgOFZ99EBTZoZGyTNgmojr3xCvlFaOGlAgICoZI zB8vnS81jtXYhujKuNB+HEbAt1QB X-Google-Smtp-Source: APXvYqyEbKmWD9lQ4z6Gp8Ga9XdHXdwvlZ8z53SOz7rwew/RfTnQiiDCMVIAt42O0p6Fy8bxFg4AwQ== X-Received: by 2002:a81:388b:: with SMTP id f133mr4161383ywa.408.1556238085254; Thu, 25 Apr 2019 17:21:25 -0700 (PDT) Received: from localhost ([2620:10d:c091:180::5484]) by smtp.gmail.com with ESMTPSA id x126sm6775497ywa.13.2019.04.25.17.21.24 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Thu, 25 Apr 2019 17:21:24 -0700 (PDT) From: Jes Sorensen X-Google-Original-From: Jes Sorensen To: linux-block@vger.kernel.org Cc: josef@toxicpanda.com, kernel-team@fb.com Subject: [PATCH 3/5] Use blk-stat infrastructure to collect per queue device stats Date: Thu, 25 Apr 2019 20:20:48 -0400 Message-Id: <20190426002050.15499-4-Jes.Sorensen@gmail.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190426002050.15499-1-Jes.Sorensen@gmail.com> References: <20190426002050.15499-1-Jes.Sorensen@gmail.com> MIME-Version: 1.0 Sender: linux-block-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Jes Sorensen Put request bytes into 8 buckets of requests, for read, write, and discard. Enable stats by writing 1 to /sys/block//queue/histstat, disable by writing 0. Signed-off-by: Jes Sorensen --- block/blk-mq.c | 52 ++++++++++++++++++++++++++++++++++++++++++ block/blk-stat.h | 1 + block/blk-sysfs.c | 40 ++++++++++++++++++++++++++++++++ include/linux/blkdev.h | 7 ++++++ 4 files changed, 100 insertions(+) diff --git a/block/blk-mq.c b/block/blk-mq.c index e8cd2028745a..3b36e8e26feb 100644 --- a/block/blk-mq.c +++ b/block/blk-mq.c @@ -58,6 +58,52 @@ static int blk_mq_poll_stats_bkt(const struct request *rq) return bucket; } +/* + * 8 buckets for each of read, write, and discard + */ +static int blk_dev_stats_bkt(const struct request *rq) +{ + int grp, bucket; + + grp = op_stat_group(req_op(rq)); + + bucket = grp + 3*(ilog2(rq->io_bytes) - 9); + + if (bucket < 0) + return -1; + else if (bucket >= BLK_DEV_STATS_BKTS) + return grp + BLK_DEV_STATS_BKTS - 3; + + return bucket; +} + +/* + * Copy out the stats to their official location + */ +static void blk_dev_stats_cb(struct blk_stat_callback *cb) +{ + struct request_queue *q = cb->data; + int bucket; + + for (bucket = 0; bucket < BLK_DEV_STATS_BKTS; bucket++) { + if (cb->stat[bucket].nr_samples) { + q->dev_stat[bucket].size += + cb->stat[bucket].size; + q->dev_stat[bucket].nr_samples += + cb->stat[bucket].nr_samples; + } + } + + if (!blk_stat_is_active(cb)) + blk_stat_activate_msecs(cb, 100); +} + +void blk_dev_stats_free(struct request_queue *q) +{ + blk_stat_remove_callback(q, q->dev_cb); + blk_stat_free_callback(q->dev_cb); +} + /* * Check if any of the ctx, dispatch list or elevator * have pending work in this hardware queue. @@ -2851,6 +2897,12 @@ struct request_queue *blk_mq_init_allocated_queue(struct blk_mq_tag_set *set, if (!q->nr_hw_queues) goto err_hctxs; + q->dev_cb = blk_stat_alloc_callback(blk_dev_stats_cb, + blk_dev_stats_bkt, + BLK_DEV_STATS_BKTS, q); + if (!q->dev_cb) + goto err_hctxs; + INIT_WORK(&q->timeout_work, blk_mq_timeout_work); blk_queue_rq_timeout(q, set->timeout ? set->timeout : 30 * HZ); diff --git a/block/blk-stat.h b/block/blk-stat.h index ea893c4a9af1..7f0c8b737a9d 100644 --- a/block/blk-stat.h +++ b/block/blk-stat.h @@ -168,4 +168,5 @@ void blk_rq_stat_add(struct blk_rq_stat *, u64, u64); void blk_rq_stat_sum(struct blk_rq_stat *, struct blk_rq_stat *); void blk_rq_stat_init(struct blk_rq_stat *); +void blk_dev_stats_free(struct request_queue *q); #endif diff --git a/block/blk-sysfs.c b/block/blk-sysfs.c index 422327089e0f..394760754bfc 100644 --- a/block/blk-sysfs.c +++ b/block/blk-sysfs.c @@ -530,6 +530,37 @@ static ssize_t queue_dax_show(struct request_queue *q, char *page) return queue_var_show(blk_queue_dax(q), page); } +static ssize_t queue_histstat_show(struct request_queue *q, char *page) +{ + return queue_var_show(test_bit(QUEUE_FLAG_HISTSTATS, + &q->queue_flags), page); +} + +static ssize_t queue_histstat_store(struct request_queue *q, const char *page, + size_t size) +{ + unsigned long histstat_on; + ssize_t ret; + + ret = queue_var_store(&histstat_on, page, size); + if (ret < 0) + return ret; + + if (histstat_on) { + if (!blk_queue_flag_test_and_set(QUEUE_FLAG_HISTSTATS, q)) + blk_stat_add_callback(q, q->dev_cb); + if (!blk_stat_is_active(q->dev_cb)) + blk_stat_activate_msecs(q->dev_cb, 100); + } else { + if (test_bit(QUEUE_FLAG_HISTSTATS, &q->queue_flags)) { + blk_stat_remove_callback(q, q->dev_cb); + blk_queue_flag_clear(QUEUE_FLAG_HISTSTATS, q); + } + } + + return ret; +} + static struct queue_sysfs_entry queue_requests_entry = { .attr = {.name = "nr_requests", .mode = 0644 }, .show = queue_requests_show, @@ -728,6 +759,12 @@ static struct queue_sysfs_entry throtl_sample_time_entry = { }; #endif +static struct queue_sysfs_entry queue_histstat_entry = { + .attr = {.name = "histstat", .mode = 0644 }, + .show = queue_histstat_show, + .store = queue_histstat_store, +}; + static struct attribute *default_attrs[] = { &queue_requests_entry.attr, &queue_ra_entry.attr, @@ -767,6 +804,7 @@ static struct attribute *default_attrs[] = { #ifdef CONFIG_BLK_DEV_THROTTLING_LOW &throtl_sample_time_entry.attr, #endif + &queue_histstat_entry.attr, NULL, }; @@ -837,6 +875,8 @@ static void __blk_release_queue(struct work_struct *work) { struct request_queue *q = container_of(work, typeof(*q), release_work); + blk_dev_stats_free(q); + if (test_bit(QUEUE_FLAG_POLL_STATS, &q->queue_flags)) blk_stat_remove_callback(q, q->poll_cb); blk_stat_free_callback(q->poll_cb); diff --git a/include/linux/blkdev.h b/include/linux/blkdev.h index a2291b1d1cb0..54f7718cd6a2 100644 --- a/include/linux/blkdev.h +++ b/include/linux/blkdev.h @@ -53,6 +53,9 @@ struct blk_stat_callback; /* Doing classic polling */ #define BLK_MQ_POLL_CLASSIC -1 +/* Must be consistent with blk_part_stats_bkt() */ +#define BLK_DEV_STATS_BKTS (3 * 8) + /* * Maximum number of blkcg policies allowed to be registered concurrently. * Defined here to simplify include dependency. @@ -478,6 +481,9 @@ struct request_queue { struct blk_stat_callback *poll_cb; struct blk_rq_stat poll_stat[BLK_MQ_POLL_STATS_BKTS]; + struct blk_stat_callback *dev_cb; + struct blk_rq_stat dev_stat[BLK_DEV_STATS_BKTS]; + struct timer_list timeout; struct work_struct timeout_work; @@ -593,6 +599,7 @@ struct request_queue { #define QUEUE_FLAG_SCSI_PASSTHROUGH 23 /* queue supports SCSI commands */ #define QUEUE_FLAG_QUIESCED 24 /* queue has been quiesced */ #define QUEUE_FLAG_PCI_P2PDMA 25 /* device supports PCI p2p requests */ +#define QUEUE_FLAG_HISTSTATS 26 /* Hist stats enabled if set */ #define QUEUE_FLAG_MQ_DEFAULT ((1 << QUEUE_FLAG_IO_STAT) | \ (1 << QUEUE_FLAG_SAME_COMP)) From patchwork Fri Apr 26 00:20:49 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jes Sorensen X-Patchwork-Id: 10917865 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 5CD76933 for ; Fri, 26 Apr 2019 00:21:32 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 4D38328A72 for ; Fri, 26 Apr 2019 00:21:32 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 41D5828D0A; Fri, 26 Apr 2019 00:21:32 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id DC23F28D0A for ; Fri, 26 Apr 2019 00:21:31 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727771AbfDZAVb (ORCPT ); Thu, 25 Apr 2019 20:21:31 -0400 Received: from mail-yb1-f193.google.com ([209.85.219.193]:44441 "EHLO mail-yb1-f193.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726546AbfDZAVb (ORCPT ); Thu, 25 Apr 2019 20:21:31 -0400 Received: by mail-yb1-f193.google.com with SMTP id u187so654521ybg.11 for ; Thu, 25 Apr 2019 17:21:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=gK5nqmvoIh7e/MtccfmFy5xTemV1dhtFwOFBM32mQJ8=; b=NmQ0yPRFEa8XFIAf6cg9dJrjZlwW3LefpNKC9mU208ZUqBV9CyNxuAaDogcuE9tuvU 4eXf7Mbr988P+Jiyc//MkM0OueESee+XEK1ljsK0V0RJ2k5MCiTG+zwCHQ2kOF6GotaZ qASRNsL9KoXn5lghdwsryXWZCYcc8rX4KTgEGMyPSvNPc/CMjh1wtUJnuriDTkOdScls 3fZi33F0c6CwR8UHchPLWNB4/t7fSS3e8EtSYMpG0hC+gAZk4BrGNc3CKX1L1BZqJpex cqEV2JSg/n5tF0eDObZlgD37B80G55hAYN5pXjnPJEM6OjMhfP9UXLjw2M2Bcl8GdtAh t/gg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=gK5nqmvoIh7e/MtccfmFy5xTemV1dhtFwOFBM32mQJ8=; b=Wn3PoDetGBBoStcsMdEhMg9BFPy/DjSevn4alQCiSboYjKuWUpGZ2JzUFq/yxsOe++ bc3iXThYc/xsUSQ2gRQe8nSN8CZ+I2Pyj2QnMtu4YluWr7TnyGMUovxy1PZEjm9GXLhz zmSPx/ukDAriDBUEbrbzc3mTfnI4bGHlBwzluICy8wHoxIAbKJr76LmWLf9hG0CaAYZk AveUqYGpSiFt8gd7IOYOmZqbDhT4k9aYG2bVIZhUdMp70oStRxV5dULXFL2czjfHNL9a BZXyP78X7JfvT/dGI8vz1L64+wn/nJunvghvMZ5C3x6KdJR70FLKFqncVm3NT8+Bzjsp Lidg== X-Gm-Message-State: APjAAAXR0p9uaoMa3Yj0gsPeDCyO/me7/8jeI0daZpwY4jfUTEMPFrHa CNhVWgas689UNMr/VqpPWhARvCAe X-Google-Smtp-Source: APXvYqz4Ct+01ZEYVr8uDnnaRJx/OsfGc3NdbkF10o88qSKVfLltTbOd3ja6x2OHrHXRVAgGN8ovdg== X-Received: by 2002:a25:d289:: with SMTP id j131mr10101538ybg.289.1556238089927; Thu, 25 Apr 2019 17:21:29 -0700 (PDT) Received: from localhost ([2620:10d:c091:180::5484]) by smtp.gmail.com with ESMTPSA id n63sm813143ywf.61.2019.04.25.17.21.29 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Thu, 25 Apr 2019 17:21:29 -0700 (PDT) From: Jes Sorensen X-Google-Original-From: Jes Sorensen To: linux-block@vger.kernel.org Cc: josef@toxicpanda.com, kernel-team@fb.com Subject: [PATCH 4/5] Export block dev stats to sysfs Date: Thu, 25 Apr 2019 20:20:49 -0400 Message-Id: <20190426002050.15499-5-Jes.Sorensen@gmail.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190426002050.15499-1-Jes.Sorensen@gmail.com> References: <20190426002050.15499-1-Jes.Sorensen@gmail.com> MIME-Version: 1.0 Sender: linux-block-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Jes Sorensen This exports bytes of read/write/discard per bucket to sysfs Signed-off-by: Jes Sorensen --- block/blk-sysfs.c | 23 +++++++++++++++++++++++ 1 file changed, 23 insertions(+) diff --git a/block/blk-sysfs.c b/block/blk-sysfs.c index 394760754bfc..f54d91b54b61 100644 --- a/block/blk-sysfs.c +++ b/block/blk-sysfs.c @@ -530,6 +530,23 @@ static ssize_t queue_dax_show(struct request_queue *q, char *page) return queue_var_show(blk_queue_dax(q), page); } +static ssize_t queue_stat_show(struct request_queue *q, char *p) +{ + char name[3][8] = {"read", "write", "discard"}; + int bkt, off, i; + + off = 0; + for (i = 0; i < 3; i++) { + off += sprintf(p + off, "%s: ", name[i]); + for (bkt = 0; bkt < (BLK_DEV_STATS_BKTS / 3); bkt++) { + off += sprintf(p + off, "%llu ", + q->dev_stat[i + 3 * bkt].size); + } + off += sprintf(p + off, "\n"); + } + return off; +} + static ssize_t queue_histstat_show(struct request_queue *q, char *page) { return queue_var_show(test_bit(QUEUE_FLAG_HISTSTATS, @@ -759,6 +776,11 @@ static struct queue_sysfs_entry throtl_sample_time_entry = { }; #endif +static struct queue_sysfs_entry queue_stat_entry = { + .attr = {.name = "stat", .mode = 0444 }, + .show = queue_stat_show, +}; + static struct queue_sysfs_entry queue_histstat_entry = { .attr = {.name = "histstat", .mode = 0644 }, .show = queue_histstat_show, @@ -804,6 +826,7 @@ static struct attribute *default_attrs[] = { #ifdef CONFIG_BLK_DEV_THROTTLING_LOW &throtl_sample_time_entry.attr, #endif + &queue_stat_entry.attr, &queue_histstat_entry.attr, NULL, }; From patchwork Fri Apr 26 00:20:50 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jes Sorensen X-Patchwork-Id: 10917867 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 42A8E912 for ; Fri, 26 Apr 2019 00:21:37 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 356FE28CED for ; Fri, 26 Apr 2019 00:21:37 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 28A3228D49; Fri, 26 Apr 2019 00:21:37 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id C47F828CED for ; Fri, 26 Apr 2019 00:21:36 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727925AbfDZAVg (ORCPT ); Thu, 25 Apr 2019 20:21:36 -0400 Received: from mail-yw1-f66.google.com ([209.85.161.66]:40380 "EHLO mail-yw1-f66.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726546AbfDZAVg (ORCPT ); Thu, 25 Apr 2019 20:21:36 -0400 Received: by mail-yw1-f66.google.com with SMTP id t79so571766ywc.7 for ; Thu, 25 Apr 2019 17:21:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=iJAKFIUwXmdu8PAlVtviwz/Fyb0DGbl6JsI8sXHzYrc=; b=n419d2H/edMvMTnPWCA0jHqd1XLaO+L6cCzlr11ST5P3AGokTBIiEJEiFRQPCRzyBI Ll2tmFFL5C022dyR5kmWmA3igMoZJTRaEz+B80CRCTApr1t8c2Z8BYnzWhwkpAj5waiy bFjhyAdbUoH27IZcvImUOxdqEFLtJA4ihxrllkYlZELOljV7DIkSGqPFpkFrF2UHHr1M CatFCW7NVbqaXymfRqJ3iXAArxjfyGeX0RbMPieraAIupJwHVDyMXGCWKiCWyEMOb8vp w4P8MOdn8iAj2Qhea3/sJMMQDk9XgCbDo6hXHop49tVBxtvU4hiauuRobqLVvgZYH6mL pH0w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=iJAKFIUwXmdu8PAlVtviwz/Fyb0DGbl6JsI8sXHzYrc=; b=hLK8t1IL/92ItdrTGk54TTWUneg+NRLxPcqV2tCwjk40+hi5J5VTa991z7zoV7Ipon bAoK21wcrOasUzXeFZ/eowy0JrIaVuRKqsWVKw6UyGVTf8lw+V/NeRD02vBa4cMbsw8A W6Bucx5Sxe1tHVppenDjt5FHZb6YL18XhzKYbCNWIsCWL1O5JeYX8i6nxCSoh7Pmweiv 6SHZCNwpBG1Utv7mXMgbsjwfDKpuPV3SYyNOXQxEfkoCgEqGn9Y1+QG/29HY1s3zg3Ev rFf+wq5Oo9XfH9wCKNZ1E+xpdeJ7tQ5CHeGMavnYdA8ixWF4BeSRFUmn6Dw2tVXM80Ts BoyA== X-Gm-Message-State: APjAAAWXJPaL8rEOl9e5kzuExTNf3bNzhNahXhxnJ3+nL4K1snlu/M8T GVvDfymO+QpzDD5yU0GmVBK7kwpy X-Google-Smtp-Source: APXvYqxARVlO37HbaMq8P+LJkxVN7wKO4l+N/TVNqnIP2L7mBKZKfrZI3xCNzNouTJ8+Giit4mT+TQ== X-Received: by 2002:a81:9913:: with SMTP id q19mr32242105ywg.215.1556238094765; Thu, 25 Apr 2019 17:21:34 -0700 (PDT) Received: from localhost ([2620:10d:c091:180::5484]) by smtp.gmail.com with ESMTPSA id v144sm10066822ywc.37.2019.04.25.17.21.34 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Thu, 25 Apr 2019 17:21:34 -0700 (PDT) From: Jes Sorensen X-Google-Original-From: Jes Sorensen To: linux-block@vger.kernel.org Cc: josef@toxicpanda.com, kernel-team@fb.com Subject: [PATCH 5/5] Expand block stats to export number of of requests per bucket Date: Thu, 25 Apr 2019 20:20:50 -0400 Message-Id: <20190426002050.15499-6-Jes.Sorensen@gmail.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190426002050.15499-1-Jes.Sorensen@gmail.com> References: <20190426002050.15499-1-Jes.Sorensen@gmail.com> MIME-Version: 1.0 Sender: linux-block-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Jes Sorensen Signed-off-by: Jes Sorensen --- block/blk-sysfs.c | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/block/blk-sysfs.c b/block/blk-sysfs.c index f54d91b54b61..04d1938d1aa9 100644 --- a/block/blk-sysfs.c +++ b/block/blk-sysfs.c @@ -543,6 +543,14 @@ static ssize_t queue_stat_show(struct request_queue *q, char *p) q->dev_stat[i + 3 * bkt].size); } off += sprintf(p + off, "\n"); + + off += sprintf(p + off, "%s reqs: ", name[i]); + for (bkt = 0; bkt < (BLK_DEV_STATS_BKTS / 3); bkt++) { + off += sprintf(p + off, "%u ", + q->dev_stat[i + 3 * bkt].nr_samples); + } + + off += sprintf(p + off, "\n"); } return off; }