From patchwork Wed Jul 8 00:20:22 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Damien Le Moal X-Patchwork-Id: 11650495 X-Patchwork-Delegate: snitzer@redhat.com Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id D0C03739 for ; Wed, 8 Jul 2020 00:21:45 +0000 (UTC) Received: from us-smtp-delivery-1.mimecast.com (us-smtp-2.mimecast.com [207.211.31.81]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 832FA2067D for ; Wed, 8 Jul 2020 00:21:44 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 832FA2067D Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=wdc.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=dm-devel-bounces@redhat.com Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-117-F0BSRi1tNbWNnk4Nil_AsQ-1; Tue, 07 Jul 2020 20:21:41 -0400 X-MC-Unique: F0BSRi1tNbWNnk4Nil_AsQ-1 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 2268D107ACF2; Wed, 8 Jul 2020 00:21:37 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.20]) by smtp.corp.redhat.com (Postfix) with ESMTPS id B8A8D5C1D0; Wed, 8 Jul 2020 00:21:36 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id EA880180954D; Wed, 8 Jul 2020 00:21:35 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.rdu2.redhat.com [10.11.54.5]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id 0680LYSB030883 for ; Tue, 7 Jul 2020 20:21:34 -0400 Received: by smtp.corp.redhat.com (Postfix) id 29EBF7AEF; Wed, 8 Jul 2020 00:21:34 +0000 (UTC) Delivered-To: dm-devel@redhat.com Received: from mimecast-mx02.redhat.com (mimecast04.extmail.prod.ext.rdu2.redhat.com [10.11.55.20]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 251DA82864 for ; Wed, 8 Jul 2020 00:21:32 +0000 (UTC) Received: from us-smtp-1.mimecast.com (us-smtp-1.mimecast.com [207.211.31.81]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 1519E101A525 for ; Wed, 8 Jul 2020 00:21:32 +0000 (UTC) Received: from esa3.hgst.iphmx.com (esa3.hgst.iphmx.com [216.71.153.141]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-189-i0NjXa61PTCjJ2mljYXCnA-2; Tue, 07 Jul 2020 20:21:30 -0400 X-MC-Unique: i0NjXa61PTCjJ2mljYXCnA-2 IronPort-SDR: Xp7a/0lAEwTKpXhcPtjcXXTlmCOo0yTnuzXCVMO7I7VsNqJ9YYdw3WPPiJJhiG3oLV8zOrVTD7 zdQ30Mw8gAGNxhSie+oybPGALI1nC3sACnqDIiBzSkT2NjID8R4K1u74HbwqqPQZNBtlp3D2ee rnD6s+ncMdgsVp+zErIbD+boecSZBKg/0x9KnLANYX/7mxQ729d5E54wOQ4LbJ2pO+dyrewXZG 9EsYzzEFvwO4aqwCPgOQuySMJ8J5Z9wT6RKs06+92j1t+h7CQ4DHjOBzMHaUJt9EPiu4dwBF4G qOk= X-IronPort-AV: E=Sophos;i="5.75,325,1589212800"; d="scan'208";a="146187471" Received: from h199-255-45-15.hgst.com (HELO uls-op-cesaep02.wdc.com) ([199.255.45.15]) by ob1.hgst.iphmx.com with ESMTP; 08 Jul 2020 08:20:26 +0800 IronPort-SDR: WsAao1qumCH8ATBIfh3P17c7xigLiyPCFXr7DUTsLHykHM892DmASZy5hmLa9kVBYzEVDZL4ir 8/8lit0Dzlc2ZXAMgy1mCPNVW/2Hj1raY= Received: from uls-op-cesaip01.wdc.com ([10.248.3.36]) by uls-op-cesaep02.wdc.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 07 Jul 2020 17:08:32 -0700 IronPort-SDR: LyzTzeBUN7oULFPiMOYImKxkrFkjdKlnHdIOWMtZMo99KfyvIoGgcKlFQ6AyRF5gOt50yKG3Zd qd7dyayDT9xQ== WDCIronportException: Internal Received: from washi.fujisawa.hgst.com ([10.149.53.254]) by uls-op-cesaip01.wdc.com with ESMTP; 07 Jul 2020 17:20:25 -0700 From: Damien Le Moal To: dm-devel@redhat.com, Mike Snitzer Date: Wed, 8 Jul 2020 09:20:22 +0900 Message-Id: <20200708002023.738147-2-damien.lemoal@wdc.com> In-Reply-To: <20200708002023.738147-1-damien.lemoal@wdc.com> References: <20200708002023.738147-1-damien.lemoal@wdc.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.11.54.5 X-loop: dm-devel@redhat.com Cc: Shin'ichiro Kawasaki , Damien Le Moal Subject: [dm-devel] [PATCH 1/2] dm zoned: Fix zone reclaim trigger X-BeenThere: dm-devel@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: device-mapper development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: dm-devel-bounces@redhat.com Errors-To: dm-devel-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=dm-devel-bounces@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Triggerring reclaim only based on the percentage of unmapped cache zones can fail to detect cases where reclaim is needed, e.g. if the target has only 2 or 3 cache zones and only one unmapped cache zone, the percentage of free cache zone is higher than DMZ_RECLAIM_LOW_UNMAP_ZONES (30%) and reclaim does not trigger. This problem, combined with the fact that dmz_schedule_reclaim() is called from dmz_handle_bio() without the map lock held leads to a race between zone allocation and dmz_should_reclaim() result. Depending on the workload applied, this race can lead to the write path forever waiting for a free zone without reclaim being triggerred. Fix this by moving dmz_schedule_reclaim() inside dmz_alloc_zone() under the map lock, checking the need for zone reclaim whenever a new data or buffer zone needs to be allocated. Also fix dmz_reclaim_percentage() to always return 0 if the number of unmapped cache (or random) zone is less than or equal to 1. Suggested-by: Shin'ichiro Kawasaki Signed-off-by: Damien Le Moal Reviewed-by: Hannes Reinecke --- drivers/md/dm-zoned-metadata.c | 9 ++++++++- drivers/md/dm-zoned-reclaim.c | 2 ++ drivers/md/dm-zoned-target.c | 10 +--------- 3 files changed, 11 insertions(+), 10 deletions(-) diff --git a/drivers/md/dm-zoned-metadata.c b/drivers/md/dm-zoned-metadata.c index 5cf6f5f552e0..b298fefb022e 100644 --- a/drivers/md/dm-zoned-metadata.c +++ b/drivers/md/dm-zoned-metadata.c @@ -2217,8 +2217,15 @@ struct dm_zone *dmz_alloc_zone(struct dmz_metadata *zmd, unsigned int dev_idx, { struct list_head *list; struct dm_zone *zone; - int i = 0; + int i; + + /* Schedule reclaim to ensure free zones are available */ + if (!(flags & DMZ_ALLOC_RECLAIM)) { + for (i = 0; i < zmd->nr_devs; i++) + dmz_schedule_reclaim(zmd->dev[i].reclaim); + } + i = 0; again: if (flags & DMZ_ALLOC_CACHE) list = &zmd->unmap_cache_list; diff --git a/drivers/md/dm-zoned-reclaim.c b/drivers/md/dm-zoned-reclaim.c index dd1eebf6e50f..9c6e264465bc 100644 --- a/drivers/md/dm-zoned-reclaim.c +++ b/drivers/md/dm-zoned-reclaim.c @@ -456,6 +456,8 @@ static unsigned int dmz_reclaim_percentage(struct dmz_reclaim *zrc) nr_zones = dmz_nr_rnd_zones(zmd, zrc->dev_idx); nr_unmap = dmz_nr_unmap_rnd_zones(zmd, zrc->dev_idx); } + if (nr_unmap <= 1) + return 0; return nr_unmap * 100 / nr_zones; } diff --git a/drivers/md/dm-zoned-target.c b/drivers/md/dm-zoned-target.c index cf915009c306..42aa5139df7c 100644 --- a/drivers/md/dm-zoned-target.c +++ b/drivers/md/dm-zoned-target.c @@ -400,15 +400,7 @@ static void dmz_handle_bio(struct dmz_target *dmz, struct dm_chunk_work *cw, dm_per_bio_data(bio, sizeof(struct dmz_bioctx)); struct dmz_metadata *zmd = dmz->metadata; struct dm_zone *zone; - int i, ret; - - /* - * Write may trigger a zone allocation. So make sure the - * allocation can succeed. - */ - if (bio_op(bio) == REQ_OP_WRITE) - for (i = 0; i < dmz->nr_ddevs; i++) - dmz_schedule_reclaim(dmz->dev[i].reclaim); + int ret; dmz_lock_metadata(zmd); From patchwork Wed Jul 8 00:20:23 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Damien Le Moal X-Patchwork-Id: 11650497 X-Patchwork-Delegate: snitzer@redhat.com Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 5A96C739 for ; Wed, 8 Jul 2020 00:21:56 +0000 (UTC) Received: from us-smtp-1.mimecast.com (us-smtp-delivery-1.mimecast.com [207.211.31.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 0C7D42067D for ; Wed, 8 Jul 2020 00:21:55 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 0C7D42067D Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=wdc.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=dm-devel-bounces@redhat.com Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-377-dfvk3uhaNEeSM0uIxhWj2g-1; Tue, 07 Jul 2020 20:21:53 -0400 X-MC-Unique: dfvk3uhaNEeSM0uIxhWj2g-1 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 75D6310059AF; Wed, 8 Jul 2020 00:21:48 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.20]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 0BC78C006C; Wed, 8 Jul 2020 00:21:48 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id D63E71809561; Wed, 8 Jul 2020 00:21:47 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id 0680LZLK030889 for ; Tue, 7 Jul 2020 20:21:35 -0400 Received: by smtp.corp.redhat.com (Postfix) id 541742026D69; Wed, 8 Jul 2020 00:21:35 +0000 (UTC) Delivered-To: dm-devel@redhat.com Received: from mimecast-mx02.redhat.com (mimecast05.extmail.prod.ext.rdu2.redhat.com [10.11.55.21]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 4FCAF2026D5D for ; Wed, 8 Jul 2020 00:21:33 +0000 (UTC) Received: from us-smtp-1.mimecast.com (us-smtp-delivery-1.mimecast.com [207.211.31.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 157A080013F for ; Wed, 8 Jul 2020 00:21:33 +0000 (UTC) Received: from esa3.hgst.iphmx.com (esa3.hgst.iphmx.com [216.71.153.141]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-189-m5aGAESAOpiu-JSYsoRppQ-3; Tue, 07 Jul 2020 20:21:31 -0400 X-MC-Unique: m5aGAESAOpiu-JSYsoRppQ-3 IronPort-SDR: 7y1UoH0MSSfV3VcIl0YI+Q6kxaDZewCIpojLfADJ6dY1QTGEPqvQq7cHZ5nfbOEjUoSgTGDcqL HI/v10JXlfg0nlDCbEn2Hxe+GjGLmYBS+rNxpfCQkpRwGgVwwKvbquimkoYxrijFcYIDDsPkIY qYF/YTpfXO7rUMHMRFhU4chSyBX5eu/MDZdU9gt2Aw6QgOTluIBuT/k07+ekYaFXqzuQI/T4K4 ghUX2yqDio5a9zXRUo9il76jIq1vBfI04ce75G7z73CB2oWN92K6QFsZ1CJXVvfNm4o6hd7lIH X40= X-IronPort-AV: E=Sophos;i="5.75,325,1589212800"; d="scan'208";a="146187473" Received: from h199-255-45-15.hgst.com (HELO uls-op-cesaep02.wdc.com) ([199.255.45.15]) by ob1.hgst.iphmx.com with ESMTP; 08 Jul 2020 08:20:27 +0800 IronPort-SDR: jTAm7rL0u89J7TW5U94xz0Fp2CE0qFP/KPupI3Yuxfm8bENDQsAfyS4BLeCO56qSivYAboHtE/ 4LBKLOWPJ4+bp2KQk2Tn8d9UsiLt0CYJE= Received: from uls-op-cesaip01.wdc.com ([10.248.3.36]) by uls-op-cesaep02.wdc.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 07 Jul 2020 17:08:33 -0700 IronPort-SDR: bHbn4XD38LR2C3wkP/xeAgtjhowVfhjZ70s2tx3q+R4lmRJmdnlHgttLnuP+7Lfd9C/bZKpZCI 6dJUNStkWw5w== WDCIronportException: Internal Received: from washi.fujisawa.hgst.com ([10.149.53.254]) by uls-op-cesaip01.wdc.com with ESMTP; 07 Jul 2020 17:20:26 -0700 From: Damien Le Moal To: dm-devel@redhat.com, Mike Snitzer Date: Wed, 8 Jul 2020 09:20:23 +0900 Message-Id: <20200708002023.738147-3-damien.lemoal@wdc.com> In-Reply-To: <20200708002023.738147-1-damien.lemoal@wdc.com> References: <20200708002023.738147-1-damien.lemoal@wdc.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.78 on 10.11.54.4 X-loop: dm-devel@redhat.com Cc: Shin'ichiro Kawasaki , Damien Le Moal Subject: [dm-devel] [PATCH 2/2] dm zoned: Remove set but unused variables X-BeenThere: dm-devel@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: device-mapper development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: dm-devel-bounces@redhat.com Errors-To: dm-devel-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=dm-devel-bounces@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com In dmz_reclaim_work(), the variables nr_unmap_rnd and nr_rnd are set but unused. Remove them. Fixes: f97809aec589 ("dm zoned: per-device reclaim") Signed-off-by: Damien Le Moal Reviewed-by: Hannes Reinecke --- drivers/md/dm-zoned-reclaim.c | 5 +---- 1 file changed, 1 insertion(+), 4 deletions(-) diff --git a/drivers/md/dm-zoned-reclaim.c b/drivers/md/dm-zoned-reclaim.c index 9c6e264465bc..9c0ecc9568a4 100644 --- a/drivers/md/dm-zoned-reclaim.c +++ b/drivers/md/dm-zoned-reclaim.c @@ -503,7 +503,7 @@ static void dmz_reclaim_work(struct work_struct *work) { struct dmz_reclaim *zrc = container_of(work, struct dmz_reclaim, work.work); struct dmz_metadata *zmd = zrc->metadata; - unsigned int p_unmap, nr_unmap_rnd = 0, nr_rnd = 0; + unsigned int p_unmap; int ret; if (dmz_dev_is_dying(zmd)) @@ -529,9 +529,6 @@ static void dmz_reclaim_work(struct work_struct *work) zrc->kc_throttle.throttle = min(75U, 100U - p_unmap / 2); } - nr_unmap_rnd = dmz_nr_unmap_rnd_zones(zmd, zrc->dev_idx); - nr_rnd = dmz_nr_rnd_zones(zmd, zrc->dev_idx); - DMDEBUG("(%s/%u): Reclaim (%u): %s, %u%% free zones (%u/%u cache %u/%u random)", dmz_metadata_label(zmd), zrc->dev_idx, zrc->kc_throttle.throttle,