From patchwork Fri Jan 19 23:07:37 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Khazhy Kumykov X-Patchwork-Id: 10177753 X-Patchwork-Delegate: snitzer@redhat.com Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id E1FF360224 for ; Mon, 22 Jan 2018 09:03:28 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id CA07B27C05 for ; Mon, 22 Jan 2018 09:03:28 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id BC8BF27EE2; Mon, 22 Jan 2018 09:03:28 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.8 required=2.0 tests=BAYES_00, DKIM_ADSP_CUSTOM_MED, DKIM_SIGNED,RCVD_IN_DNSWL_HI,T_DKIM_INVALID,T_TVD_MIME_NO_HEADERS autolearn=ham version=3.3.1 Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 4FACD27C05 for ; Mon, 22 Jan 2018 09:03:27 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id A02245D675; Mon, 22 Jan 2018 09:03:26 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.21]) by smtp.corp.redhat.com (Postfix) with ESMTPS id D4B359065; Mon, 22 Jan 2018 09:03:25 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id C53823FAE7; Mon, 22 Jan 2018 09:03:22 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id w0JN8OgS022681 for ; Fri, 19 Jan 2018 18:08:24 -0500 Received: by smtp.corp.redhat.com (Postfix) id DC72B60BEE; Fri, 19 Jan 2018 23:08:24 +0000 (UTC) Delivered-To: dm-devel@redhat.com Received: from mx1.redhat.com (ext-mx02.extmail.prod.ext.phx2.redhat.com [10.5.110.26]) by smtp.corp.redhat.com (Postfix) with ESMTPS id D270860BE3 for ; Fri, 19 Jan 2018 23:08:22 +0000 (UTC) Received: from mail-pf0-f194.google.com (mail-pf0-f194.google.com [209.85.192.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 223B785A04 for ; Fri, 19 Jan 2018 23:08:21 +0000 (UTC) Received: by mail-pf0-f194.google.com with SMTP id p1so2456831pfh.4 for ; Fri, 19 Jan 2018 15:08:21 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id; bh=TH1NRbMovH2nxgFT7RYEyrl9ww74Y882sj2SWs4tqWk=; b=flNniSRH9b2I3acmI49wvYJgc0S91ifTiPm++VVf14KPcePlBS+H2gTZtDjM0M4jAn JqTNHFC4rpou6NyZZZQfa8PkmeAErYM5i1zQsKfSlGFjWAw5LcdtAi8CSUx9Z1PDAUAw rI7hNJWFCaM83772qnH/gUSOlMxEyO+9JxHVCcB/4RCoAkxbWVyqFLrx1/+EAgNq0htI 96acvn5r0Q/WzLRng68Bfl5FZksvKdFQViYmpPtB3Rgf7NP68lGQPA37+laoVT65X2et PGVz/pKXiNIQunZbD7X1T7+7sZnpTWWNvRbOU2FkLpoEEG8RxXhzeI7cQaam9f2es6RB uNmA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id; bh=TH1NRbMovH2nxgFT7RYEyrl9ww74Y882sj2SWs4tqWk=; b=A2CTY69BzTQaMbLmSkWVmInGxHQ1yqQGByvqjEJQeNb70X9RZpevxIeS8+L5Pj97sk hKtX1Tm7ka6KvYvYVutaMbinugKI/XLPlvrxnAax4L40luxPz2WktOju5Rqrer6j30RP wJNoB4jPPneKJpRkRCgSG9sCqYiMoHYdgk8dcp1WSJmHqVvNBtOXJ6nL45dlPQavwCkT YScyCuRUxYGtnRDrM6lj7Cc9aV0gFNQSzjrlovAOAuLu1Y3+0orfgTvdQC5KFEBOdzyG mRke/RA5vl6xF4z6/mv9K/EW47bN9aYk/SspIyy0udNxPRXdL9mJna5fmwCsrLa0sQ+f U6Qg== X-Gm-Message-State: AKwxyteuAUgsGKBlJ3pzY7/H7FZdpuhnTJPqJJZjU6k50Kbzt81k1FGh nCo9QVh76aXH7UawFXzQpkrv5g== X-Google-Smtp-Source: AH8x225CJgVkmDPbz6Vj9f5z1fYHb0qdkH7juF4vmUNah4KcafpcNUQNmMsLZBzZop2+lzQEeDDopA== X-Received: by 2002:a17:902:7003:: with SMTP id y3-v6mr33568plk.449.1516403300177; Fri, 19 Jan 2018 15:08:20 -0800 (PST) Received: from khazhy.svl.corp.google.com ([100.116.144.80]) by smtp.gmail.com with ESMTPSA id s14sm17062079pgf.16.2018.01.19.15.08.19 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Fri, 19 Jan 2018 15:08:19 -0800 (PST) From: Khazhismel Kumykov To: agk@redhat.com, snitzer@redhat.com, dm-devel@redhat.com Date: Fri, 19 Jan 2018 15:07:37 -0800 Message-Id: <20180119230737.133596-1-khazhy@google.com> X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.26]); Fri, 19 Jan 2018 23:08:21 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.26]); Fri, 19 Jan 2018 23:08:21 +0000 (UTC) for IP:'209.85.192.194' DOMAIN:'mail-pf0-f194.google.com' HELO:'mail-pf0-f194.google.com' FROM:'khazhy@google.com' RCPT:'' X-RedHat-Spam-Score: 1.884 * (DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, MIME_HEADER_CTYPE_ONLY, RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H2, SPF_PASS, T_RP_MATCHES_RCVD, T_TVD_MIME_NO_HEADERS) 209.85.192.194 mail-pf0-f194.google.com 209.85.192.194 mail-pf0-f194.google.com X-Scanned-By: MIMEDefang 2.78 on 10.5.110.26 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 X-loop: dm-devel@redhat.com X-Mailman-Approved-At: Mon, 22 Jan 2018 04:03:21 -0500 Cc: linux-kernel@vger.kernel.org, Khazhismel Kumykov Subject: [dm-devel] [PATCH] dm mpath selector: more evenly distribute ties X-BeenThere: dm-devel@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: device-mapper development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Sender: dm-devel-bounces@redhat.com Errors-To: dm-devel-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.39]); Mon, 22 Jan 2018 09:03:27 +0000 (UTC) X-Virus-Scanned: ClamAV using ClamSMTP Move the last used path to the end of the list (least preferred) so that ties are more evenly distributed. For example, in case with three paths with one that is slower than others, the remaining two would be unevenly used if they tie. This is due to the rotation not being a truely fair distribution. Illustrated: paths a, b, c, 'c' has 1 outstanding IO, a and b are 'tied' Three possible rotations: (a, b, c) -> best path 'a' (b, c, a) -> best path 'b' (c, a, b) -> best path 'a' (a, b, c) -> best path 'a' (b, c, a) -> best path 'b' (c, a, b) -> best path 'a' ... So 'a' is used 2x more than 'b', although they should be used evenly. With this change, the most recently used path is always the least preferred, removing this bias resulting in even distribution. (a, b, c) -> best path 'a' (b, c, a) -> best path 'b' (c, a, b) -> best path 'a' (c, b, a) -> best path 'b' ... Signed-off-by: Khazhismel Kumykov Reviewed-by: Martin Wilck --- drivers/md/dm-queue-length.c | 6 +++--- drivers/md/dm-service-time.c | 6 +++--- 2 files changed, 6 insertions(+), 6 deletions(-) diff --git a/drivers/md/dm-queue-length.c b/drivers/md/dm-queue-length.c index 23f178641794..969c4f1a3633 100644 --- a/drivers/md/dm-queue-length.c +++ b/drivers/md/dm-queue-length.c @@ -195,9 +195,6 @@ static struct dm_path *ql_select_path(struct path_selector *ps, size_t nr_bytes) if (list_empty(&s->valid_paths)) goto out; - /* Change preferred (first in list) path to evenly balance. */ - list_move_tail(s->valid_paths.next, &s->valid_paths); - list_for_each_entry(pi, &s->valid_paths, list) { if (!best || (atomic_read(&pi->qlen) < atomic_read(&best->qlen))) @@ -210,6 +207,9 @@ static struct dm_path *ql_select_path(struct path_selector *ps, size_t nr_bytes) if (!best) goto out; + /* Move most recently used to least preferred to evenly balance. */ + list_move_tail(&best->list, &s->valid_paths); + ret = best->path; out: spin_unlock_irqrestore(&s->lock, flags); diff --git a/drivers/md/dm-service-time.c b/drivers/md/dm-service-time.c index 7b8642045c55..f006a9005593 100644 --- a/drivers/md/dm-service-time.c +++ b/drivers/md/dm-service-time.c @@ -282,9 +282,6 @@ static struct dm_path *st_select_path(struct path_selector *ps, size_t nr_bytes) if (list_empty(&s->valid_paths)) goto out; - /* Change preferred (first in list) path to evenly balance. */ - list_move_tail(s->valid_paths.next, &s->valid_paths); - list_for_each_entry(pi, &s->valid_paths, list) if (!best || (st_compare_load(pi, best, nr_bytes) < 0)) best = pi; @@ -292,6 +289,9 @@ static struct dm_path *st_select_path(struct path_selector *ps, size_t nr_bytes) if (!best) goto out; + /* Move most recently used to least preferred to evenly balance. */ + list_move_tail(&best->list, &s->valid_paths); + ret = best->path; out: spin_unlock_irqrestore(&s->lock, flags);