From patchwork Thu Sep 15 21:16:49 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jens Axboe X-Patchwork-Id: 9334755 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 50B3E60839 for ; Thu, 15 Sep 2016 21:16:59 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 3BF9A29BCD for ; Thu, 15 Sep 2016 21:16:59 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 30D9129BD3; Thu, 15 Sep 2016 21:16:59 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.8 required=2.0 tests=BAYES_00,DKIM_SIGNED, RCVD_IN_DNSWL_HI,T_DKIM_INVALID autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 77A4F29BCD for ; Thu, 15 Sep 2016 21:16:58 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1754063AbcIOVQ5 (ORCPT ); Thu, 15 Sep 2016 17:16:57 -0400 Received: from mx0b-00082601.pphosted.com ([67.231.153.30]:57948 "EHLO mx0a-00082601.pphosted.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1752388AbcIOVQ4 (ORCPT ); Thu, 15 Sep 2016 17:16:56 -0400 Received: from pps.filterd (m0089730.ppops.net [127.0.0.1]) by m0089730.ppops.net (8.16.0.17/8.16.0.17) with SMTP id u8FL8sY9010989 for ; Thu, 15 Sep 2016 14:16:55 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=fb.com; h=to : from : subject : message-id : date : mime-version : content-type : content-transfer-encoding; s=facebook; bh=T/E22BUS2rUh/LFR+mq8g3F4m6nUhpJ/pihHDqv5AD0=; b=SKc2MNLI2t4NuhtHy3t9PWxa1EUeNLXN2IM7d5ZIi5mcnQyb6oXTbgwhJWF6/chqYlyB 5XWOKa0mkc1qzBOIB6fNWBeeP0rFd17LVz17wLuLYWT2OSzy6e+DYeo3oLxEdOiChuJj tGk3FK8TPkMwP0rGdNFcPk5j5/2QmuvhMso= Received: from maileast.thefacebook.com ([199.201.65.23]) by m0089730.ppops.net with ESMTP id 25g2cgrfqw-1 (version=TLSv1 cipher=ECDHE-RSA-AES256-SHA bits=256 verify=NOT) for ; Thu, 15 Sep 2016 14:16:55 -0700 Received: from NAM02-CY1-obe.outbound.protection.outlook.com (192.168.183.28) by o365-in.thefacebook.com (192.168.177.25) with Microsoft SMTP Server (TLS) id 14.3.294.0; Thu, 15 Sep 2016 17:16:55 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fb.onmicrosoft.com; s=selector1-fb-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=ar59/4iKQ/JA4rqy72KUO0Zpnuqp9vhNhYqiZT7/t9g=; b=TI5TKsDEopWRBpILESJgLVPrt+yWD9uBkXDjzIKm7Vx+Sdhrx3ZFKzT8yk/V4ih5yWtyjIpSxHebNEuKROSi6CYssRfCD3A0Rk08n1IeXqtUcYiOsHEztAlQyfuG+LPcNIak2Q9aMShsjemDg8s1MdeUzAjbCzStiuW3xxzShoE= Received: from [192.168.1.153] (216.160.245.98) by DM5PR15MB1194.namprd15.prod.outlook.com (10.173.209.12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384) id 15.1.619.10; Thu, 15 Sep 2016 21:16:53 +0000 To: "linux-block@vger.kernel.org" From: Jens Axboe Subject: [PATCH] blk-mq: account higher order dispatch Message-ID: <80fca3d5-7c21-bce1-11db-2010a459a774@fb.com> Date: Thu, 15 Sep 2016 15:16:49 -0600 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101 Thunderbird/45.2.0 MIME-Version: 1.0 X-Originating-IP: [216.160.245.98] X-ClientProxiedBy: DM3PR12CA0013.namprd12.prod.outlook.com (10.164.12.151) To DM5PR15MB1194.namprd15.prod.outlook.com (10.173.209.12) X-MS-Office365-Filtering-Correlation-Id: 464c4a64-065d-407e-92c7-08d3ddad9de5 X-Microsoft-Exchange-Diagnostics: 1; DM5PR15MB1194; 2:bAnugSBthq4TRzXbr2Y2Qg4Z0zToEuxS2+HsJqU6HyAn50cEpikKrxrNBbu2S8pmRvFyuqhBTZ1wqyKCrhYb+NAgk1pxVXwa4e/D8n3Jzg+zsRlvLPdek9tFpSI3QM05ncrNQLLX7drTW76B02ywsVCgOd78FPuj/EJ5KXkFct4PYPOhwW9q2Ufjwe/pPZ0p; 3:GkezsOvSTRHeRXdBF6wEtHmTmJqJ+Y82f+z6E6uIjdcPF7XtrBaRQmbzzC0IwGmXMhwGjcwVRBOjrIYcvOVroU7ana1vxy/RnqHq/iMnclPL+q/7GlxQ4+tMwMm617y1 X-Microsoft-Antispam: UriScan:;BCL:0;PCL:0;RULEID:;SRVR:DM5PR15MB1194; X-Microsoft-Exchange-Diagnostics: 1; DM5PR15MB1194; 25:cviL9HOnjWEa38tYwJgGrURL57iZinAm1m2N0NtkzxGK/Fjg6Y2iVzPUQgTAU7hG1FLOs51sCkOEZfEjLjcQ5Mz6bRqz4vCVEioTjXKoUzpF7EXQ2CVvPbbWDzcRD1XDCiwlKlNWoDBhrsax2UktFwpWXkidWYgd8sY/8qIyXmdNrfPbjVIPZ8fBojxqT+fCizcFjmZmCnz25XSYhnjwdTZ+rbA9H4KHzJI1z502RBycKosQRBRNxQb7ZCzbaFMWhyl7ORerLwpduLGULBZ48hVJ9+HTn+95qyVqMkEroZgUrHgym8H5OqJU3lBHgGF/m60p7QyDuC9cCO4X3I5ZDxzBZeMVOelD4kxhycdBkk85kR1AA1ILSJqc7IRCg38paaoPuCXX0BaTxm5/LUlvHsFrBlHKVNaYOmqTQBlHvYuGBqZ6lw3KRYaakNu0YZm6fThvwjnPnpu0v2DFEYcmuISKwaUwCfgha26sHIkOdRL6CdCSLDPDHexuMNLYSa2JhA6ZfsJOLcOzyb81XPLTFc3vBGOsACz0ymg1S7KDA+lJ1vEBn6cWPsoXJmmC5tTK75+1qElVn/Obr6OldW99CA5kOnPI2n2n59fsKgYNdxQ8Omman7pldwOp6xdDd9irm2wC1EOE5sBV4vvGYByloi5MKtlQUBmUWb2ENimyaa646RvrcwFgIifWlCHAQUzeNGmFMico38BRbGCw1WxhJrS6+RIXZEB41m26ByurxSI= X-Microsoft-Exchange-Diagnostics: 1; DM5PR15MB1194; 31:Ohu/HW0cbinbMO+VkfT1QfrqWSgj6XYa+vSShkECirnC6r5a4hPDcdjcVMCYLr+kWnqBGbZv5UFGxJB6z278o+dV1q0h5EG4bZaI1OBhFuxbpIwK+sx1Rjuha65P3OQyARSQdVWWOEf4kmT9WRSISMXb/hdvR5YoclGSqvToYY5xlBYXntUULIUewLEIx35lxHosI7eB4MtC20uB/mEOJJVEIx7N0g/+AoYK60+iC8E=; 20:W1T9itJcrVDEaPux8hkqpqTjMxp2F3eAAlIPqGB4HhNnp3mjYxVfGHwYC5S0ubZsNFpJ0RQFf/WuzTc+AVjcgHI2SCJzh9gcWwmbFkvP5ndi7NaFk84Ken09NkjZI+ngXl7BwOfSm9l8srfErhJYKXU9aevmauOW5GBl0JSJYPE=; 4:LWuZJ8q0maS4eAgUUVZ5oJrAJuiVlvmrkDQ2+j0TfdvBr5k/mC4gLG9HQE9QCzRGgV8xPdOxVVA05l1tX3QCHlHuEryBGWPWBvQsk367Ov1c2pNLAlQumufYKvxt5PAm//pOghJTbxAvEhNPAK03TT7PgedR52UNPeRQ0HBOj2t+pb7E80VfzuDxXF5TIWCC1y3yogl5XwuUQyqbl3v7uRQ/tDriWpGgwQrZjPqGO+j7bSBpLUpcEL8lkcHA3eKdl83HwO1w6ReKz/lEMXh6KZ/qZorCjWY3H9GvIChZigSvH61mMD5fLCKunwXsP3Z7RvFd/jeELiJK7j+2v7B525qFBqM7rSEH8xkZrNTSe8NF3z7hhNSii3LDEZwBgVJafbvRKxo+OvWT9StkcsUBdfGqeuD1IiuTfu6OZqUUhZziWrb1GiA0PYdG1hg53Nvg X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:(67672495146484); X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(6040176)(601004)(2401047)(5005006)(8121501046)(3002001)(10201501046); SRVR:DM5PR15MB1194; BCL:0; PCL:0; RULEID:; SRVR:DM5PR15MB1194; X-Forefront-PRVS: 0066D63CE6 X-Forefront-Antispam-Report: SFV:NSPM; SFS:(10019020)(4630300001)(6049001)(6009001)(7916002)(199003)(189002)(50986999)(54356999)(106356001)(97736004)(47776003)(36756003)(105586002)(81156014)(23676002)(2351001)(81166006)(6116002)(65956001)(50466002)(189998001)(8676002)(66066001)(65806001)(15650500001)(230700001)(101416001)(586003)(229853001)(3846002)(64126003)(31696002)(4001350100001)(117156001)(5660300001)(77096005)(7846002)(575784001)(19580395003)(305945005)(110136003)(92566002)(107886002)(42186005)(2906002)(19580405001)(33646002)(2501003)(68736007)(7736002)(65826007)(450100001)(31686004)(86362001)(83506001); DIR:OUT; SFP:1102; SCL:1; SRVR:DM5PR15MB1194; H:[192.168.1.153]; FPR:; SPF:None; PTR:InfoNoRecords; A:1; MX:1; LANG:en; Received-SPF: None (protection.outlook.com: fb.com does not designate permitted sender hosts) X-Microsoft-Exchange-Diagnostics: =?utf-8?B?MTtETTVQUjE1TUIxMTk0OzIzOkUxMEMwVTFBbVZxTm4wOThVSkJHbHZqTkZv?= =?utf-8?B?Rll4ZFJBSENNNEhWbWFxUXVsejVJWEdFb053Uzcvb2RBdUNEWmRFQWdKc2pm?= =?utf-8?B?dnREVVhJS3ZjeHBtL3NsRVZlc0RwajNBeDBEZ0hia0ZmRjVJOGRlSXVzQkZY?= =?utf-8?B?SkRHdWJXSUp2MXhjOWQxeU1Ld0YzSlQ3UHpITm95RklWZ2tnRlVGYkFiUDh2?= =?utf-8?B?UGNhV3ZJQy83czRMNlF0ckl5R0ZPSG5DWVo1T3dhdndKN0lhUDhhUnRnWEUr?= =?utf-8?B?cDk1OUZ3akdxUGVmVkZod0c3R281Qnh4dDlIeU9WVW9iWTlLMUFkekVESmlw?= =?utf-8?B?OWY4N0JmcTZ0NCtYYXVtNmRVZldqMEdCRjRHeUVTbzkvM3pUdjZPN0psN2x0?= =?utf-8?B?eUp5eE5BT21HT2p6OG9mdGI2am0waExmNjVhZ3pyZVU4NkNGVGVhNXJUUGw4?= =?utf-8?B?b2QwT0pRVWQwUElKTGNCbUZ3SStJRFdFaGZxZklyY05heUFydWJtb0lBV0Rs?= =?utf-8?B?ejNhcUQydUJLdUU1R0NYRUhiSDVYd0E3bzY0VUNvYkRrcm5rdTNLYWRNS1R4?= =?utf-8?B?dG1KZUdtOFhkbFc5QThtRUtQOStzUHdDL3poRWpjNlFvWS80dkdjeWM0ZG0w?= =?utf-8?B?aWZhWll1MHQxSlBtLytvYXQ0RGVhbThOTlFpekFGTDNlWmMyeE9VYi8wUlFP?= =?utf-8?B?ejkyaytOL0ZpZWlBUklGOUVUN3I1MlZNclcxNHZhL3BGbkZQZFY0ajcwbHo1?= =?utf-8?B?cDN3ZEU5eGJyaC9BNGZoSUJIVklKQXNCRTFXSm03dkZBWjVCU3JUSnJ6Y0ow?= =?utf-8?B?Z2ZWVmwrRDVYdWZJYlo1OUFSRGtDUVZ4MHU5cDRrdE91WFhiVmhPRXU5SnRU?= =?utf-8?B?K0srT0VEQVBKSmpqY2lsU3NCZlVqbDlGR0RlUFJFRHJ0enF6V3VHSEdSUGxJ?= =?utf-8?B?czRFSjh2S0lUTzBHMTVUY3QxM2ROWDRpVFE4cTN2SnZSNVFwQXZZU05WVDdD?= =?utf-8?B?QVQyajhYcVdJZU1aZWZqSHhzUHJ2eEdmYUlJUnFKZmlyZmFwNThvdmc2VXQ1?= =?utf-8?B?Sk1aZlZleGg2V1JtR2gzRTkwbDcxaldxaU5yME9PRWV0d05OT3VWcE5lRTFx?= =?utf-8?B?Wi9pZ1RPQmNYMTUxcnA3RHZHVEIwTzdCS1F2Z1NkOUI4b01GbVNhKytVd2p0?= =?utf-8?B?KzJ1a09YWno4ZHBISWxFWkZDTGZ0QjdFelhFaGJkKzcxSDlDV000VmcyeVRF?= =?utf-8?B?eXI3eHEzUjhvYTFQN2lZTk91cTZib2pwSk5rUmx6bWNCaTUxQlI0dGJaa0hN?= =?utf-8?B?WlFYd204bTVwbTZTeGJwUnRDclNJbTEyaXRQQWlhMjl1eDgyUTRFVVNkS25R?= =?utf-8?B?NURqbElzUlp3TFNlNTlCZEg0SmVqb3VWR2dGclBkQ0htVjJBbk85VjlSVGY4?= =?utf-8?B?MnBqOWUxanJrRThBdnZkSForU0ZvenU3YUR3T21xS0pzMlNmajNTVkRMTDdw?= =?utf-8?B?Y0JsQ3E1TDl0YitIOGRvMk9MU2lzMndrRzlKbHE2ak1qWDRFcmpkQUE5ZCs5?= =?utf-8?B?bUtmaE9pbkZKMXlYOVVwaSs3NW1teHJ2RlZQTVU3WFhYdzN3QXA0QThYVitq?= =?utf-8?B?T2ZtaFd5MjdTNWVnWnc1V0o0SEphaFRlOC90Mmo3RHJSeGJSUWNnL2dkazNS?= =?utf-8?B?TTUraXdGekpiK3BhdjQySEt2Q2JhWTBvMHFUZWFIUlJaanN5ZFpPa2xaWE85?= =?utf-8?B?ZkhlbWE5SitMZEcrblpFRWlzc0Zya3krM1dNa3ZGcGduWGxZYjdmNzNJQlpj?= =?utf-8?B?cElvT2oxR3BwU29yY0syd1g0K3pSNDZqS0JtTVVOOEpOQ0E9PQ==?= X-Microsoft-Exchange-Diagnostics: 1; DM5PR15MB1194; 6:03asNo0TIAOheFvxk9l3QzVR9d3ut/UOrQi5ayIws83OXMzenwEJ00rdg/8CDtNaz9NdG1stL14+HPm06j76n0WUrpDoGvOuW7Xbtkq89MCz67UR6u9LIcCCZLN8y4YKOdjDFpgUbIHsvKreyB93R8DjR/xgkGsl78nxpr+5dvLoWtNOD3nOQsm+NlmkNiy263uFAqHsaNFEfatgtyJFO8BNr8n35FXUOGOUF+PGPRKh1bqJqDzdilQ8z2gp8WCqqUDUZyDTlxRZjnikHzhoHIAq6rlojDehPGzPz42czQI=; 5:k0gMvbEAJ8tZ1nV7n/4HVsgBI4k3w9eniv3qH/PlyFrryYRfVqviw12YDd5kCcjLJprKXMvUES94aZwsLFWHls+GV1MbAKVErZ25apljDfcY7Lfk8LAHev4IgyKEvun5frKHfA2UDSJZpr271n++QQ==; 24:UtdUWENmaRAyGEz4A5+UJCuLJ9Aqh+FsWJ34ic+4j5VmU1f+/fSYEhpl3IOnvOxgCGUkKQkqgv0d9j4hTIzRdj5lxSt3TlkDon0JSeJwL40=; 7:EuxjKxRNxT1kQlPOE9OG7tpErw2jZGbE2rmev2fsgb/Qdx357GALV+PWwBZjQazfGER89cC91EpUcfK3gER2aVcdrFv9I5JB30gr09JrHviWGnF3XKApIBhp5p/zpqGgtvOZ2k4+R6cR+4J2rtsqHzmmmpEmc5YDbWpW8XyjdM2WTpj1+y1ekU1oXu8upkWHUUb0eDm/LIJQsaCcTl5I6uKV6Uyaf0YDDdana3VfxKr0Wb9OBM+5W6fXXl1pJ2H3 SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-Microsoft-Exchange-Diagnostics: 1; DM5PR15MB1194; 20:3g+rSaoOXB6sLK+SYDDmf7V8bCiO7nHtatAJS4tUSgNcmBIHMMfswuFdaGYd9w0N/uTDu/0Q3bGmnx/gCEMCYsLOm0ZpW9dNo66sFDa6g0hkaOp9KLenG5xv5L0m3XRrcyGfHhDUyPNiA4fJLFEd27UxURqESJDBQ2kzBR/T4Aw= X-MS-Exchange-CrossTenant-OriginalArrivalTime: 15 Sep 2016 21:16:53.3347 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM5PR15MB1194 X-OriginatorOrg: fb.com X-Proofpoint-Spam-Reason: safe X-FB-Internal: Safe X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:, , definitions=2016-09-15_10:, , signatures=0 Sender: linux-block-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP We currently account a '0' dispatch, and anything above that still falls below the range set by BLK_MQ_MAX_DISPATCH_ORDER. If we dispatch more, we don't account it. Change the last bucket to be inclusive of anything above the range we track, and have the sysfs file reflect that by including a '+' in the output: $ cat /sys/block/nvme0n1/mq/0/dispatched 0 1006 1 20229 2 1 4 0 8 0 16 0 32+ 0 Signed-off-by: Jens Axboe * Note that this function currently has various problems around ordering @@ -890,10 +898,7 @@ static void __blk_mq_run_hw_queue(struct blk_mq_hw_ctx *hctx) dptr = &driver_list; } - if (!queued) - hctx->dispatched[0]++; - else if (queued < (1 << (BLK_MQ_MAX_DISPATCH_ORDER - 1))) - hctx->dispatched[ilog2(queued) + 1]++; + hctx->dispatched[queued_to_index(queued)]++; /* * Any items that need requeuing? Stuff them into hctx->dispatch, Reviewed-by: Omar Sandoval diff --git a/block/blk-mq-sysfs.c b/block/blk-mq-sysfs.c index 885a4a59acd9..39840c37b7ce 100644 --- a/block/blk-mq-sysfs.c +++ b/block/blk-mq-sysfs.c @@ -208,12 +208,14 @@ static ssize_t blk_mq_hw_sysfs_dispatched_show(struct blk_mq_hw_ctx *hctx, page += sprintf(page, "%8u\t%lu\n", 0U, hctx->dispatched[0]); - for (i = 1; i < BLK_MQ_MAX_DISPATCH_ORDER; i++) { - unsigned long d = 1U << (i - 1); + for (i = 1; i < BLK_MQ_MAX_DISPATCH_ORDER - 1; i++) { + unsigned int d = 1U << (i - 1); - page += sprintf(page, "%8lu\t%lu\n", d, hctx->dispatched[i]); + page += sprintf(page, "%8u\t%lu\n", d, hctx->dispatched[i]); } + page += sprintf(page, "%8u+\t%lu\n", 1U << (i - 1), + hctx->dispatched[i]); return page - start_page; } diff --git a/block/blk-mq.c b/block/blk-mq.c index 4b8034e54478..fb4b6795ec57 100644 --- a/block/blk-mq.c +++ b/block/blk-mq.c @@ -802,6 +802,14 @@ static void flush_busy_ctxs(struct blk_mq_hw_ctx *hctx, struct list_head *list) } } +static inline unsigned int queued_to_index(unsigned int queued) +{ + if (!queued) + return 0; + + return min(BLK_MQ_MAX_DISPATCH_ORDER - 1, ilog2(queued) + 1); +} + /* * Run this hardware queue, pulling any software queues mapped to it in.