From patchwork Tue Oct 23 12:32:13 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jia Guo X-Patchwork-Id: 10653179 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 79AF313B5 for ; Tue, 23 Oct 2018 12:33:07 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 6137A294CE for ; Tue, 23 Oct 2018 12:33:07 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 54F8E294F0; Tue, 23 Oct 2018 12:33:07 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.2 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from aserp2120.oracle.com (aserp2120.oracle.com [141.146.126.78]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id CE97D294CE for ; Tue, 23 Oct 2018 12:33:06 +0000 (UTC) Received: from pps.filterd (aserp2120.oracle.com [127.0.0.1]) by aserp2120.oracle.com (8.16.0.22/8.16.0.22) with SMTP id w9NCJJHQ008197; Tue, 23 Oct 2018 12:32:56 GMT Received: from userv0021.oracle.com (userv0021.oracle.com [156.151.31.71]) by aserp2120.oracle.com with ESMTP id 2n7vapvtwt-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 23 Oct 2018 12:32:56 +0000 Received: from oss.oracle.com (oss-old-reserved.oracle.com [137.254.22.2]) by userv0021.oracle.com (8.14.4/8.14.4) with ESMTP id w9NCWq68024156 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Tue, 23 Oct 2018 12:32:53 GMT Received: from localhost ([127.0.0.1] helo=lb-oss.oracle.com) by oss.oracle.com with esmtp (Exim 4.63) (envelope-from ) id 1gEvrg-0001s3-H5; Tue, 23 Oct 2018 05:32:52 -0700 Received: from userv0021.oracle.com ([156.151.31.71]) by oss.oracle.com with esmtp (Exim 4.63) (envelope-from ) id 1gEvra-0001rk-Fc for ocfs2-devel@oss.oracle.com; Tue, 23 Oct 2018 05:32:46 -0700 Received: from userp2040.oracle.com (userp2040.oracle.com [156.151.31.90]) by userv0021.oracle.com (8.14.4/8.14.4) with ESMTP id w9NCWj9F023796 (version=TLSv1/SSLv3 cipher=AES256-GCM-SHA384 bits=256 verify=OK) for ; Tue, 23 Oct 2018 12:32:45 GMT Received: from pps.filterd (userp2040.oracle.com [127.0.0.1]) by userp2040.oracle.com (8.16.0.22/8.16.0.22) with SMTP id w9NCJHmM046755 for ; Tue, 23 Oct 2018 12:32:45 GMT Received: from huawei.com (szxga06-in.huawei.com [45.249.212.32]) by userp2040.oracle.com with ESMTP id 2na1xphy8t-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO) for ; Tue, 23 Oct 2018 12:32:45 +0000 Received: from DGGEMS414-HUB.china.huawei.com (unknown [172.30.72.58]) by Forcepoint Email with ESMTP id 7B42B840D938B; Tue, 23 Oct 2018 20:32:38 +0800 (CST) Received: from [10.177.218.160] (10.177.218.160) by smtp.huawei.com (10.3.19.214) with Microsoft SMTP Server id 14.3.408.0; Tue, 23 Oct 2018 20:32:30 +0800 From: Jia Guo To: "mark@fasheh.com" , "jlbec@evilplan.org" , "junxiao.bi@oracle.com" , "jiangqi903@gmail.com" , "akpm@linux-foundation.org" Message-ID: <20d062b0-c91b-dbdb-c2af-b826d9a773a5@huawei.com> Date: Tue, 23 Oct 2018 20:32:13 +0800 User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:52.0) Gecko/20100101 Thunderbird/52.6.0 MIME-Version: 1.0 Content-Language: en-US X-Originating-IP: [10.177.218.160] X-CFilter-Loop: Reflected X-CLX-Shades: MLX X-CLX-Response: 1TFkXGx0TEQpMehcbHxkRCllNF2dmchEKWUkXGnEaEBp3BhsfE3EZEhAadwY YGgYaEQpZXhdobmYRCklGF0VYS0lGT3VaWEVOX0leQ0VEGXVPSxEKQ04XB2IZU0Fvf2Z6aXxkbF 8YTGMebRtETUZOHxxIQlJIRxgRClhcFx8EGgQbHxsHSBpIHBNJTB8FGxoEGxoaBB4SBBsQGx4aH xoRCl5ZF35oZkdjEQpNXBceHRkRCkxaF2loaUJNXREKTU4XaBEKQ1oXHh8EGB4TBBgbGAQZGBEK Ql4XGxEKRF4XGxoRCkRJFxwRCkJGF2cTbWAbW2VCH359EQpCXBcaEQpCRRdjQWASWUIdZBhHGRE KQk4XbEJIWVMaTWV4eB0RCkJMF2twWktIRgVdU24SEQpCbBdgbRNsARpBaE4cexEKQkAXbgFsZX NBWEJCfn0RCkJYF2J9b3kBTxgZcHB7EQpaWBcYEQpwZxdpcERPf1phZ39yXhAaEQpwaBdtf09iZ G8fExJLEhAaEQpwaBdvHFNlS3pGaGFmXxAaEQpwaBdgbVpucHtFRR14WhAaEQpwaBdkE0VQY0Fw b119cBAaEQpwaBdpQFlbRh8YZmZoGBAaEQpwbBdgehl+HERea1oBGRAcGhEKbX4XGhEKWE0XSxE g X-PDR: PASS X-Source-IP: 45.249.212.32 X-ServerName: szxga06-in.huawei.com X-Proofpoint-SPF-Result: pass X-Proofpoint-SPF-Record: v=spf1 ip4:45.249.212.32 ip4:45.249.212.35 ip4:119.145.14.93 ip4:58.251.152.93 ip4:194.213.3.17 ip4:206.16.17.72 ip4:45.249.212.255 ip4:45.249.212.187/29 ip4:45.249.212.191 ip4:185.176.76.210 ~all X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=9054 signatures=668683 X-Proofpoint-DMARC-Record: none X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 priorityscore=153 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=179 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1807170000 definitions=main-1810230106 X-Spam: Clean Cc: "ocfs2-devel@oss.oracle.com" Subject: [Ocfs2-devel] [PATCH] ocfs2: Optimize the reading of heartbeat data X-BeenThere: ocfs2-devel@oss.oracle.com X-Mailman-Version: 2.1.9 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: ocfs2-devel-bounces@oss.oracle.com Errors-To: ocfs2-devel-bounces@oss.oracle.com X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=9054 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=0 malwarescore=0 phishscore=0 bulkscore=0 spamscore=0 mlxscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1807170000 definitions=main-1810230106 X-Virus-Scanned: ClamAV using ClamSMTP Reading heartbeat data from lowest node rather than from zero, in cases where the node is not defined from zero, can reduce the number of sectors read. Here is a simple test data obtained with 'iostat -dmx dm-5 2', with two nodes in the cluster, node number 10, 20, respectively. Before optimization: Device: rrqm/s wrqm/s r/s w/s rMB/s wMB/s avgrq-sz avgqu-sz await r_await w_await svctm %util dm-5 0.00 0.00 0.50 0.50 0.01 0.00 11.00 0.00 1.00 1.00 1.00 1.50 0.15 After the optimization: Device: rrqm/s wrqm/s r/s w/s rMB/s wMB/s avgrq-sz avgqu-sz await r_await w_await svctm %util dm-5 0.00 0.00 0.50 0.50 0.00 0.00 6.00 0.00 0.50 1.00 0.00 0.50 0.05 Signed-off-by: Jia Guo Reviewed-by: Jun Piao Reviewed-by: Yiwen Jiang --- cluster/heartbeat.c | 23 +++++++++++++++-------- 1 file changed, 15 insertions(+), 8 deletions(-) diff --git a/cluster/heartbeat.c b/cluster/heartbeat.c index 9b2ed62..a9cf984 100644 --- a/cluster/heartbeat.c +++ b/cluster/heartbeat.c @@ -582,17 +582,18 @@ bail: } static int o2hb_read_slots(struct o2hb_region *reg, - unsigned int max_slots) + unsigned int min_slot, + unsigned int max_slot) { - unsigned int current_slot=0; + unsigned int current_slot = min_slot; int status; struct o2hb_bio_wait_ctxt wc; struct bio *bio; o2hb_bio_wait_init(&wc); - while(current_slot < max_slots) { - bio = o2hb_setup_one_bio(reg, &wc, ¤t_slot, max_slots, + while(current_slot < max_slot) { + bio = o2hb_setup_one_bio(reg, &wc, ¤t_slot, max_slot, REQ_OP_READ, 0); if (IS_ERR(bio)) { status = PTR_ERR(bio); @@ -1093,9 +1094,14 @@ static int o2hb_highest_node(unsigned long *nodes, int numbits) return find_last_bit(nodes, numbits); } +static int o2hb_lowest_node(unsigned long *nodes, int numbits) +{ + return find_first_bit(nodes, numbits); +} + static int o2hb_do_disk_heartbeat(struct o2hb_region *reg) { - int i, ret, highest_node; + int i, ret, highest_node, lowest_node; int membership_change = 0, own_slot_ok = 0; unsigned long configured_nodes[BITS_TO_LONGS(O2NM_MAX_NODES)]; unsigned long live_node_bitmap[BITS_TO_LONGS(O2NM_MAX_NODES)]; @@ -1120,7 +1126,8 @@ static int o2hb_do_disk_heartbeat(struct o2hb_region *reg) } highest_node = o2hb_highest_node(configured_nodes, O2NM_MAX_NODES); - if (highest_node >= O2NM_MAX_NODES) { + lowest_node = o2hb_lowest_node(configured_nodes, O2NM_MAX_NODES); + if (highest_node >= O2NM_MAX_NODES || lowest_node >= O2NM_MAX_NODES) { mlog(ML_NOTICE, "o2hb: No configured nodes found!\n"); ret = -EINVAL; goto bail; @@ -1130,7 +1137,7 @@ static int o2hb_do_disk_heartbeat(struct o2hb_region *reg) * yet. Of course, if the node definitions have holes in them * then we're reading an empty slot anyway... Consider this * best-effort. */ - ret = o2hb_read_slots(reg, highest_node + 1); + ret = o2hb_read_slots(reg, lowest_node, highest_node + 1); if (ret < 0) { mlog_errno(ret); goto bail; @@ -1801,7 +1808,7 @@ static int o2hb_populate_slot_data(struct o2hb_region *reg) struct o2hb_disk_slot *slot; struct o2hb_disk_heartbeat_block *hb_block; - ret = o2hb_read_slots(reg, reg->hr_blocks); + ret = o2hb_read_slots(reg, 0, reg->hr_blocks); if (ret) goto out;