From patchwork Mon Aug 27 05:56:39 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "jianchao.wang" X-Patchwork-Id: 10576389 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 29A1415A7 for ; Mon, 27 Aug 2018 05:56:06 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 0016129661 for ; Mon, 27 Aug 2018 05:56:05 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id E8B422966B; Mon, 27 Aug 2018 05:56:05 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 8DFBF29661 for ; Mon, 27 Aug 2018 05:56:05 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726809AbeH0JlM (ORCPT ); Mon, 27 Aug 2018 05:41:12 -0400 Received: from userp2130.oracle.com ([156.151.31.86]:54282 "EHLO userp2130.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726785AbeH0JlM (ORCPT ); Mon, 27 Aug 2018 05:41:12 -0400 Received: from pps.filterd (userp2130.oracle.com [127.0.0.1]) by userp2130.oracle.com (8.16.0.22/8.16.0.22) with SMTP id w7R5s4JS022493; Mon, 27 Aug 2018 05:56:00 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=to : from : subject : message-id : date : mime-version : content-type : content-transfer-encoding; s=corp-2018-07-02; bh=eWKX0Le5hwBUX3DM/R5OqDHKU3rWcDHmSomnPJUmdzw=; b=AkN6pRtQSm058GU6coqFc+tIYLmYNjMFZeCR7/t+tyYWl21uGebS6KDrRpie/N+MPGf8 8BIcGwQkv0cjxQUic5xIrc3z2GAZZHiNu5kTfNwnmcLfnRGOXFuoDodRcvY1o/9nsT3v c2XAuW2HUQLPduJPuh3xL6q7re6CbJIMWqXc7Gt+ZH5jcJOkGb8hQBkRlnilyyjJIhfg 7tDShi0RADLK+BvBDHD4F7N51aCD0lvmi3zpAZ27fLS05y67j8wCRCKa801mu5jGdUgg iyNwVAeKuVAGoSm11EaXAHNx79BpgpoNIMsEBc4MD+8pczQDuV33l8W0d1DFVKx8n4PY yg== Received: from aserv0022.oracle.com (aserv0022.oracle.com [141.146.126.234]) by userp2130.oracle.com with ESMTP id 2m2xhtbujc-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 27 Aug 2018 05:56:00 +0000 Received: from userv0121.oracle.com (userv0121.oracle.com [156.151.31.72]) by aserv0022.oracle.com (8.14.4/8.14.4) with ESMTP id w7R5twVi009655 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 27 Aug 2018 05:55:59 GMT Received: from abhmp0002.oracle.com (abhmp0002.oracle.com [141.146.116.8]) by userv0121.oracle.com (8.14.4/8.13.8) with ESMTP id w7R5twi5013367; Mon, 27 Aug 2018 05:55:58 GMT Received: from [10.182.70.180] (/10.182.70.180) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Sun, 26 Aug 2018 22:55:58 -0700 To: Ming Lei , "linux-block@vger.kernel.org" From: "jianchao.wang" Subject: No protection on the hctx->dispatch_busy Message-ID: <306399af-99d9-ed45-bf3b-75908ff9187c@oracle.com> Date: Mon, 27 Aug 2018 13:56:39 +0800 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Thunderbird/52.9.1 MIME-Version: 1.0 Content-Language: en-US X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=8997 signatures=668707 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=0 malwarescore=0 phishscore=0 bulkscore=0 spamscore=0 mlxscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1807170000 definitions=main-1808270064 Sender: linux-block-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Hi Ming Currently, blk_mq_update_dispatch_busy is hooked in blk_mq_dispatch_rq_list and __blk_mq_issue_directly. blk_mq_update_dispatch_busy could be invoked on multiple cpus concurrently. But there is not any protection on the hctx->dispatch_busy. We cannot ensure the update on the dispatch_busy atomically. Look at the test result after applied the debug patch below: fio-1761 [000] .... 227.246251: blk_mq_update_dispatch_busy.part.50: old 0 ewma 2 cur 2 fio-1766 [004] .... 227.246252: blk_mq_update_dispatch_busy.part.50: old 2 ewma 1 cur 1 fio-1755 [000] .... 227.246366: blk_mq_update_dispatch_busy.part.50: old 1 ewma 0 cur 0 fio-1754 [003] .... 227.266050: blk_mq_update_dispatch_busy.part.50: old 2 ewma 3 cur 3 fio-1763 [007] .... 227.266050: blk_mq_update_dispatch_busy.part.50: old 0 ewma 2 cur 2 fio-1761 [000] .... 227.266051: blk_mq_update_dispatch_busy.part.50: old 3 ewma 2 cur 2 fio-1766 [004] .... 227.266051: blk_mq_update_dispatch_busy.part.50: old 3 ewma 2 cur 2 fio-1760 [005] .... 227.266165: blk_mq_update_dispatch_busy.part.50: old 2 ewma 1 cur 1 Is it expected ? Thanks Jianchao --- a/block/blk-mq.c +++ b/block/blk-mq.c @@ -1088,11 +1088,12 @@ static bool blk_mq_mark_tag_wait(struct blk_mq_hw_ctx *hctx, static void blk_mq_update_dispatch_busy(struct blk_mq_hw_ctx *hctx, bool busy) { unsigned int ewma; + unsigned int old; if (hctx->queue->elevator) return; - ewma = hctx->dispatch_busy; + old = ewma = hctx->dispatch_busy; if (!ewma && !busy) return; @@ -1103,6 +1104,8 @@ static void blk_mq_update_dispatch_busy(struct blk_mq_hw_ctx *hctx, bool busy) ewma /= BLK_MQ_DISPATCH_BUSY_EWMA_WEIGHT; hctx->dispatch_busy = ewma; + + trace_printk("old %u ewma %u cur %u\n", old, ewma, READ_ONCE(hctx->dispatch_busy)); }