From patchwork Tue Mar 26 10:29:20 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: tada keisuke X-Patchwork-Id: 13603948 Received: from mo-csw-fb.securemx.jp (mo-csw-fb1120.securemx.jp [210.130.202.128]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id BD5455C99; Tue, 26 Mar 2024 11:29:47 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=210.130.202.128 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1711452590; cv=none; b=o5yyO2MlNPdKNJ9eKxRg0Z6dyEj3z+Ly4xf1p9MzZRe1KO11Se/At1O5ihOYmNywxn9xGjQvFYPKsbCYM1PE/ivp5cIBUIMXchvAGE90NwX5tO1dMZRP8TGW5ucwihCr45me5Sx7dkfKiCz2I9U6CbmYj9itHjpIYZvHumLXpmA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1711452590; c=relaxed/simple; bh=TcCxixfsR8/nanhTm/t47/mH9dljdW3na3A6R3j2mz8=; h=From:To:CC:Subject:Date:Message-ID:Content-Type:MIME-Version; b=L+838s/97JNLp+kvI0iXPaYcH9svNOXpvXbqwiRBTShdyi4zPvMbW70nk7/1p9oaXzCiN/u8TXwxCqZH8k94UsF3MeS5Wf2JcR24Ku4+PFrX6eTgCrRxDWBfAyBXaklYjGvifDQs/liYe0iKAWGoZf4jBRFdMuk8QM7AyJjbKEU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=kioxia.com; spf=pass smtp.mailfrom=kioxia.com; arc=none smtp.client-ip=210.130.202.128 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=kioxia.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=kioxia.com Received: by mo-csw-fb.securemx.jp (mx-mo-csw-fb1120) id 42QATlDx671697; Tue, 26 Mar 2024 19:29:47 +0900 Received: by mo-csw.securemx.jp (mx-mo-csw1120) id 42QATNdA2208940; Tue, 26 Mar 2024 19:29:23 +0900 X-Iguazu-Qid: 2rWgHjhWsbaT7nes8G X-Iguazu-QSIG: v=2; s=0; t=1711448962; q=2rWgHjhWsbaT7nes8G; m=guZWhoTu9ab33iQ8ONREOi9dm7r4TbvlNifoLkJPiZ8= Received: from CNN1EMTA01.test.kioxia.com ([202.248.33.144]) by relay.securemx.jp (mx-mr1123) id 42QATLmS2317580 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NOT); Tue, 26 Mar 2024 19:29:22 +0900 Received: from Switcher-Post_Send (gateway [10.232.20.1]) by CNN1EMTA01.test.kioxia.com (Postfix) with ESMTP id DCF9F2F015; Tue, 26 Mar 2024 19:29:21 +0900 (JST) Received: from CNN1ESTR03.kioxia.com (localhost [127.0.0.1]) by Switcher-Post_Send (Postfix) with ESMTP id 59DC61938D7203; Tue, 26 Mar 2024 19:07:55 +0900 (JST) Received: from localhost [127.0.0.1] by CNN1ESTR03.kioxia.com with ESMTP id 0002VAAAAAA0122M; Tue, 26 Mar 2024 19:07:55 +0900 Received: from CNN1EXMB01.r1.kioxia.com (CNN1EXMB01.r1.kioxia.com [10.232.20.150]) by Switcher-Pre_Send (Postfix) with ESMTP id 4E7D5A035DC28; Tue, 26 Mar 2024 19:07:55 +0900 (JST) Received: from CNN1EXMB03.r1.kioxia.com (10.232.20.152) by CNN1EXMB01.r1.kioxia.com (10.232.20.150) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Tue, 26 Mar 2024 19:29:21 +0900 Received: from CNN1EXMB03.r1.kioxia.com ([10.13.100.22]) by CNN1EXMB03.r1.kioxia.com ([10.13.100.22]) with mapi id 15.01.2507.035; Tue, 26 Mar 2024 19:29:21 +0900 From: tada keisuke To: "song@kernel.org" , "yukuai3@huawei.com" CC: "linux-raid@vger.kernel.org" , "linux-kernel@vger.kernel.org" Subject: [PATCH 01/11] md: add infra for active_aligned_reads changes Thread-Topic: [PATCH 01/11] md: add infra for active_aligned_reads changes Thread-Index: Adp/Y7Mh/ylXUWuPROyIG62nOILkrg== Date: Tue, 26 Mar 2024 10:29:20 +0000 Message-ID: <6682020683594800921ca2567e479c3c@kioxia.com> Accept-Language: ja-JP, en-US Content-Language: ja-JP X-MS-Has-Attach: X-MS-TNEF-Correlator: x-tm-as-product-ver: ISME-14.0.0.2080-9.0.1002-28274.000 x-tm-as-result: No-10--1.684500-8.000000 x-tmase-matchedrid: biL4YBBTMFamagT1k9kBppTQgFTHgkhZKQNhMboqZlrHhoBEyuHcbYUl JlxlTIT+ewNtMWg+aOa12HagvbwDji/7QU2czuUNA9lly13c/gE2SSdNEgI3sEz5vzLEGq8DTTC YELjM+CVvF9oQ/B54yFo4rfBbwEOR23JrSYCBAceJI2OHymiAOytc78OfBHnLGTX5zLczNzU5d9 n04fLNZlepMMW3nEo1fpJkaQdvaIEGzl9AHE05zaSRA9wkcoxP6i+mRcc5QVe/WXZS/HqJ2gtuK BGekqUpdvpxIsTHHHaRTpSQiv9X7Yp2iWs90+63zEVb/qxkUfgJB7rruKhZCxODU2P2dcmMcJ2G 9s1in4jGthxghyWxGxuDt74+KZNqnbcWxYm4IwyqYm5C4G8JX8NZ1S8JnLvUgup8jVhIwn/G61L W3xarQYiiduc0eATAz+phZh9BMadJA4+6YYiS7R/BXqwE9HSW x-tm-as-user-approved-sender: No x-tm-as-user-blocked-sender: No x-tmase-result: 10--1.684500-8.000000 x-tmase-version: ISME-14.0.0.2080-9.0.1002-28274.000 x-tm-snts-smtp: 6CD62EF26E8749BC01D0D36931D1CB24239EABFA7879A738542FB1070EA382F52000:8 Precedence: bulk X-Mailing-List: linux-raid@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-CrossPremisesHeadersFilteredBySendConnector: CNN1EXMB01.r1.kioxia.com X-OrganizationHeadersPreserved: CNN1EXMB01.r1.kioxia.com Prepare to smoothly change the type of active_aligned_reads from atomic_t to percpu_ref. Signed-off-by: Keisuke TADA Signed-off-by: Toshifumi OHTAKE --- drivers/md/raid5.c | 12 ++++++------ drivers/md/raid5.h | 20 ++++++++++++++++++++ 2 files changed, 26 insertions(+), 6 deletions(-) diff --git a/drivers/md/raid5.c b/drivers/md/raid5.c index f2e3c3e2d879..fd121629603c 100644 --- a/drivers/md/raid5.c +++ b/drivers/md/raid5.c @@ -5436,7 +5436,7 @@ static void raid5_align_endio(struct bio *bi) if (!error) { bio_endio(raid_bi); - if (atomic_dec_and_test(&conf->active_aligned_reads)) + if (active_aligned_reads_dec_and_test(conf)) wake_up(&conf->wait_for_quiescent); return; } @@ -5500,7 +5500,7 @@ static int raid5_read_one_chunk(struct mddev *mddev, struct bio *raid_bio) did_inc = false; if (conf->quiesce == 0) { - atomic_inc(&conf->active_aligned_reads); + active_aligned_reads_inc(conf); did_inc = true; } /* need a memory barrier to detect the race with raid5_quiesce() */ @@ -5508,12 +5508,12 @@ static int raid5_read_one_chunk(struct mddev *mddev, struct bio *raid_bio) /* quiesce is in progress, so we need to undo io activation and wait * for it to finish */ - if (did_inc && atomic_dec_and_test(&conf->active_aligned_reads)) + if (did_inc && active_aligned_reads_dec_and_test(conf)) wake_up(&conf->wait_for_quiescent); spin_lock_irq(&conf->device_lock); wait_event_lock_irq(conf->wait_for_quiescent, conf->quiesce == 0, conf->device_lock); - atomic_inc(&conf->active_aligned_reads); + active_aligned_reads_inc(conf); spin_unlock_irq(&conf->device_lock); } @@ -6609,7 +6609,7 @@ static int retry_aligned_read(struct r5conf *conf, struct bio *raid_bio, bio_endio(raid_bio); - if (atomic_dec_and_test(&conf->active_aligned_reads)) + if (active_aligned_reads_dec_and_test(conf)) wake_up(&conf->wait_for_quiescent); return handled; } @@ -8620,7 +8620,7 @@ static void raid5_quiesce(struct mddev *mddev, int quiesce) smp_store_release(&conf->quiesce, 2); wait_event_cmd(conf->wait_for_quiescent, atomic_read(&conf->active_stripes) == 0 && - atomic_read(&conf->active_aligned_reads) == 0, + active_aligned_reads_is_zero(conf), unlock_all_device_hash_locks_irq(conf), lock_all_device_hash_locks_irq(conf)); conf->quiesce = 1; diff --git a/drivers/md/raid5.h b/drivers/md/raid5.h index 9b5a7dc3f2a0..5bd6bb3540c5 100644 --- a/drivers/md/raid5.h +++ b/drivers/md/raid5.h @@ -696,6 +696,26 @@ struct r5conf { struct r5pending_data *next_pending_data; }; +static inline void active_aligned_reads_inc(struct r5conf *conf) +{ + atomic_inc(&conf->active_aligned_reads); +} + +static inline void active_aligned_reads_dec(struct r5conf *conf) +{ + atomic_dec(&conf->active_aligned_reads); +} + +static inline bool active_aligned_reads_is_zero(struct r5conf *conf) +{ + return atomic_read(&conf->active_aligned_reads) == 0; +} + +static inline bool active_aligned_reads_dec_and_test(struct r5conf *conf) +{ + return atomic_dec_and_test(&conf->active_aligned_reads); +} + #if PAGE_SIZE == DEFAULT_STRIPE_SIZE #define RAID5_STRIPE_SIZE(conf) STRIPE_SIZE #define RAID5_STRIPE_SHIFT(conf) STRIPE_SHIFT From patchwork Tue Mar 26 10:29:24 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: tada keisuke X-Patchwork-Id: 13604021 Received: from mo-csw-fb.securemx.jp (mo-csw-fb1122.securemx.jp [210.130.202.130]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 480156E61B; Tue, 26 Mar 2024 11:58:55 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=210.130.202.130 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1711454338; cv=none; b=VRNC58PY6PcIs9pw469CRbWNl3t8M4mWEGsW70ADox538McG6c7AXjQZ7cXGEJgc5XwZ/akgBxm3EEDgNuiekBw6D0SqCYXrosUKBvVY0hbfq0uiwpwtvEEf5GAjG5/fn49JzdTcENk2F6dFBDfZjgaN7so339Sl2dUwGwYvP0c= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1711454338; c=relaxed/simple; bh=5YbkyWsQ0uZHG1ms6mH5//QUt/ohtWoPJA4T42CHas8=; h=From:To:CC:Subject:Date:Message-ID:Content-Type:MIME-Version; b=NheEGj5rGRKrP2GWYRpsmwiJCH8Sk+ZguTiVutFITnkZ7GlyukIAFtdXSx8kqNmONVwKJzyCBkfaiwpY2CffydF3tyWMFhUXh3Yss4vU8W5tOWYGwlzkLoHaFA7sHw7iOaJpqe+3zg3wJmulsJgHMFgWEfk/tmiuCwogry/IXkQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=kioxia.com; spf=pass smtp.mailfrom=kioxia.com; arc=none smtp.client-ip=210.130.202.130 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=kioxia.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=kioxia.com Received: by mo-csw-fb.securemx.jp (mx-mo-csw-fb1122) id 42QATmLP899005; Tue, 26 Mar 2024 19:29:48 +0900 Received: by mo-csw.securemx.jp (mx-mo-csw1122) id 42QATQ7N2610782; Tue, 26 Mar 2024 19:29:27 +0900 X-Iguazu-Qid: 2rWgdkpBNplhHZOjqa X-Iguazu-QSIG: v=2; s=0; t=1711448966; q=2rWgdkpBNplhHZOjqa; m=4NV5EUN3RkLeead42rcWfaw0EiUveXLZmPpDSgfT9a0= Received: from CNN1EMTA02.test.kioxia.com ([202.248.33.144]) by relay.securemx.jp (mx-mr1121) id 42QATPit2222002 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NOT); Tue, 26 Mar 2024 19:29:25 +0900 Received: from Switcher-Post_Send (gateway [10.232.20.1]) by CNN1EMTA02.test.kioxia.com (Postfix) with ESMTP id 94ADB2F02B; Tue, 26 Mar 2024 19:29:25 +0900 (JST) Received: from CNN1ESTR04.kioxia.com (localhost [127.0.0.1]) by Switcher-Post_Send (Postfix) with ESMTP id 5C5F41900001E2; Tue, 26 Mar 2024 19:16:59 +0900 (JST) Received: from localhost [127.0.0.1] by CNN1ESTR04.kioxia.com with ESMTP id 0003VAAAAAA01LJ3; Tue, 26 Mar 2024 19:16:59 +0900 Received: from CNN1EXMB02.r1.kioxia.com (CNN1EXMB02.r1.kioxia.com [10.232.20.151]) by Switcher-Pre_Send (Postfix) with ESMTP id 50CA0A29B6838; Tue, 26 Mar 2024 19:16:59 +0900 (JST) Received: from CNN1EXMB03.r1.kioxia.com (10.232.20.152) by CNN1EXMB02.r1.kioxia.com (10.232.20.151) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Tue, 26 Mar 2024 19:29:24 +0900 Received: from CNN1EXMB03.r1.kioxia.com ([10.13.100.22]) by CNN1EXMB03.r1.kioxia.com ([10.13.100.22]) with mapi id 15.01.2507.035; Tue, 26 Mar 2024 19:29:24 +0900 From: tada keisuke To: "song@kernel.org" , "yukuai3@huawei.com" CC: "linux-raid@vger.kernel.org" , "linux-kernel@vger.kernel.org" Subject: [PATCH 02/11] md: add infra for nr_pending changes Thread-Topic: [PATCH 02/11] md: add infra for nr_pending changes Thread-Index: Adp/Y+DFDYJIM6FzQL2o0BFQHP3a9w== Date: Tue, 26 Mar 2024 10:29:24 +0000 Message-ID: Accept-Language: ja-JP, en-US Content-Language: ja-JP X-MS-Has-Attach: X-MS-TNEF-Correlator: x-tm-as-product-ver: ISME-14.0.0.2080-9.0.1002-28274.000 x-tm-as-result: No-10--13.824200-8.000000 x-tmase-matchedrid: MCb2X8FnAXSmagT1k9kBppTQgFTHgkhZKQNhMboqZlrHhoBEyuHcbXkK 0FnHivpUuH0SzH4JWPa12HagvbwDji/7QU2czuUNA9lly13c/gFr2qJoNIuCjRjNpP+VAaW2cwh qloOj04o3haSYSnhyTlXcW9CR1Vd5FIetOq2+pVDfSQNpZkETVE+crEA4+nhZWm66nNAKC7PQGF lKOB0iJLlW2fEUZEj7eIXhTXDYCVBppBeolGl6TAihQ5NZCXsSsBSA1tuZVSbo8w4N7te92p/p6 742jA2BiFSRT7HBaXD/4j+p8jFGbrkRMDcY9eUDAD5jSg1rFtBNLPQl0QAltLs3Yh2IOCYzlmGO ZvTvMRc1+ahTbhS+5eZRkGfolSWMUaNA17Pg5uoOeRRGICV9PcnlJe2gk8vIOmcd2FDUExunXhF 0LKDIJa4M7Qf5HIbfgt5d7sz8fUvsR/15S/KqDguw+MVcHJpKGYkyyELuGKTfc2Xd6VJ+yjQEU0 gh8OM4V2J5gVT3U4/lwxij6XpX1ZAHY36XNwNS585VzGMOFzA9wJeM2pSaRcK21zBg2KlfqmoKk D3hCYzK8lk/8zvlR3V70/lvA49+OjEcXv1b3B2uyIJB9xNRP3JDHY0OwPoQ x-tm-as-user-approved-sender: No x-tm-as-user-blocked-sender: No x-tmase-result: 10--13.824200-8.000000 x-tmase-version: ISME-14.0.0.2080-9.0.1002-28274.000 x-tm-snts-smtp: 4B845472E1C5B0B9685C82E220B3AE4073CD807F9BA7765DA8A106E1825EEC0A2000:8 Precedence: bulk X-Mailing-List: linux-raid@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-CrossPremisesHeadersFilteredBySendConnector: CNN1EXMB02.r1.kioxia.com X-OrganizationHeadersPreserved: CNN1EXMB02.r1.kioxia.com Prepare to smoothly change the type of nr_pending from atomic_t to percpu_ref. Signed-off-by: Keisuke TADA Signed-off-by: Toshifumi OHTAKE --- drivers/md/md-bitmap.c | 2 +- drivers/md/md.c | 6 +++--- drivers/md/md.h | 32 +++++++++++++++++++++++++++- drivers/md/raid1.c | 20 ++++++++--------- drivers/md/raid10.c | 46 ++++++++++++++++++++-------------------- drivers/md/raid5-cache.c | 4 ++-- drivers/md/raid5.c | 20 ++++++++--------- 7 files changed, 80 insertions(+), 50 deletions(-) diff --git a/drivers/md/md-bitmap.c b/drivers/md/md-bitmap.c index 059afc24c08b..6d49a6e4cd2f 100644 --- a/drivers/md/md-bitmap.c +++ b/drivers/md/md-bitmap.c @@ -190,7 +190,7 @@ static struct md_rdev *next_active_rdev(struct md_rdev *rdev, struct mddev *mdde if (rdev->raid_disk >= 0 && !test_bit(Faulty, &rdev->flags)) { /* this is a usable devices */ - atomic_inc(&rdev->nr_pending); + nr_pending_inc(rdev); rcu_read_unlock(); return rdev; } diff --git a/drivers/md/md.c b/drivers/md/md.c index 7d7b982e369c..df868b315b45 100644 --- a/drivers/md/md.c +++ b/drivers/md/md.c @@ -575,7 +575,7 @@ static void submit_flushes(struct work_struct *ws) !test_bit(Faulty, &rdev->flags)) { struct bio *bi; - atomic_inc(&rdev->nr_pending); + nr_pending_inc(rdev); rcu_read_unlock(); bi = bio_alloc_bioset(rdev->bdev, 0, REQ_OP_WRITE | REQ_PREFLUSH, @@ -1050,7 +1050,7 @@ void md_super_write(struct mddev *mddev, struct md_rdev *rdev, | REQ_PREFLUSH | REQ_FUA, GFP_NOIO, &mddev->sync_set); - atomic_inc(&rdev->nr_pending); + nr_pending_inc(rdev); bio->bi_iter.bi_sector = sector; __bio_add_page(bio, page, size, 0); @@ -9255,7 +9255,7 @@ static bool rdev_removeable(struct md_rdev *rdev) return false; /* There are still inflight io, don't remove this rdev. */ - if (atomic_read(&rdev->nr_pending)) + if (nr_pending_is_not_zero(rdev)) return false; /* diff --git a/drivers/md/md.h b/drivers/md/md.h index 67e50c44f4b5..b990be0981bc 100644 --- a/drivers/md/md.h +++ b/drivers/md/md.h @@ -211,6 +211,36 @@ enum flag_bits { Nonrot, /* non-rotational device (SSD) */ }; +static inline void nr_pending_inc(struct md_rdev *rdev) +{ + atomic_inc(&rdev->nr_pending); +} + +static inline void nr_pending_dec(struct md_rdev *rdev) +{ + atomic_dec(&rdev->nr_pending); +} + +static inline bool nr_pending_is_zero(struct md_rdev *rdev) +{ + return atomic_read(&rdev->nr_pending) == 0; +} + +static inline bool nr_pending_is_not_zero(struct md_rdev *rdev) +{ + return atomic_read(&rdev->nr_pending) != 0; +} + +static inline unsigned int nr_pending_read(struct md_rdev *rdev) +{ + return atomic_read(&rdev->nr_pending); +} + +static inline bool nr_pending_dec_and_test(struct md_rdev *rdev) +{ + return atomic_dec_and_test(&rdev->nr_pending); +} + static inline int is_badblock(struct md_rdev *rdev, sector_t s, int sectors, sector_t *first_bad, int *bad_sectors) { @@ -845,7 +875,7 @@ static inline bool is_rdev_broken(struct md_rdev *rdev) static inline void rdev_dec_pending(struct md_rdev *rdev, struct mddev *mddev) { int faulty = test_bit(Faulty, &rdev->flags); - if (atomic_dec_and_test(&rdev->nr_pending) && faulty) { + if (nr_pending_dec_and_test(rdev) && faulty) { set_bit(MD_RECOVERY_NEEDED, &mddev->recovery); md_wakeup_thread(mddev->thread); } diff --git a/drivers/md/raid1.c b/drivers/md/raid1.c index 7b8a71ca66dd..9cf56bc1340f 100644 --- a/drivers/md/raid1.c +++ b/drivers/md/raid1.c @@ -581,7 +581,7 @@ static void update_read_sectors(struct r1conf *conf, int disk, { struct raid1_info *info = &conf->mirrors[disk]; - atomic_inc(&info->rdev->nr_pending); + nr_pending_inc(info->rdev); if (info->next_seq_sect != this_sector) info->seq_start = this_sector; info->next_seq_sect = this_sector + len; @@ -784,7 +784,7 @@ static int choose_best_rdev(struct r1conf *conf, struct r1bio *r1_bio) if (ctl.readable_disks++ == 1) set_bit(R1BIO_FailFast, &r1_bio->state); - pending = atomic_read(&rdev->nr_pending); + pending = nr_pending_read(rdev); dist = abs(r1_bio->sector - conf->mirrors[disk].head_position); /* Don't change to another disk for sequential reads */ @@ -1495,7 +1495,7 @@ static void raid1_write_request(struct mddev *mddev, struct bio *bio, write_behind = true; if (rdev && unlikely(test_bit(Blocked, &rdev->flags))) { - atomic_inc(&rdev->nr_pending); + nr_pending_inc(rdev); blocked_rdev = rdev; break; } @@ -1506,7 +1506,7 @@ static void raid1_write_request(struct mddev *mddev, struct bio *bio, continue; } - atomic_inc(&rdev->nr_pending); + nr_pending_inc(rdev); if (test_bit(WriteErrorSeen, &rdev->flags)) { sector_t first_bad; int bad_sectors; @@ -1879,7 +1879,7 @@ static bool raid1_remove_conf(struct r1conf *conf, int disk) struct md_rdev *rdev = info->rdev; if (!rdev || test_bit(In_sync, &rdev->flags) || - atomic_read(&rdev->nr_pending)) + nr_pending_is_not_zero(rdev)) return false; /* Only remove non-faulty devices if recovery is not possible. */ @@ -1987,7 +1987,7 @@ static int raid1_remove_disk(struct mddev *mddev, struct md_rdev *rdev) struct md_rdev *repl = conf->mirrors[conf->raid_disks + number].rdev; freeze_array(conf, 0); - if (atomic_read(&repl->nr_pending)) { + if (nr_pending_is_not_zero(repl)) { /* It means that some queued IO of retry_list * hold repl. Thus, we cannot set replacement * as NULL, avoiding rdev NULL pointer @@ -2403,7 +2403,7 @@ static void fix_read_error(struct r1conf *conf, struct r1bio *r1_bio) (!test_bit(Faulty, &rdev->flags) && rdev->recovery_offset >= sect + s)) && rdev_has_badblock(rdev, sect, s) == 0) { - atomic_inc(&rdev->nr_pending); + nr_pending_inc(rdev); if (sync_page_io(rdev, sect, s<<9, conf->tmppage, REQ_OP_READ, false)) success = 1; @@ -2433,7 +2433,7 @@ static void fix_read_error(struct r1conf *conf, struct r1bio *r1_bio) rdev = conf->mirrors[d].rdev; if (rdev && !test_bit(Faulty, &rdev->flags)) { - atomic_inc(&rdev->nr_pending); + nr_pending_inc(rdev); r1_sync_page_io(rdev, sect, s, conf->tmppage, REQ_OP_WRITE); rdev_dec_pending(rdev, mddev); @@ -2447,7 +2447,7 @@ static void fix_read_error(struct r1conf *conf, struct r1bio *r1_bio) rdev = conf->mirrors[d].rdev; if (rdev && !test_bit(Faulty, &rdev->flags)) { - atomic_inc(&rdev->nr_pending); + nr_pending_inc(rdev); if (r1_sync_page_io(rdev, sect, s, conf->tmppage, REQ_OP_READ)) { atomic_add(s, &rdev->corrected_errors); @@ -2909,7 +2909,7 @@ static sector_t raid1_sync_request(struct mddev *mddev, sector_t sector_nr, } } if (rdev && bio->bi_end_io) { - atomic_inc(&rdev->nr_pending); + nr_pending_inc(rdev); bio->bi_iter.bi_sector = sector_nr + rdev->data_offset; bio_set_dev(bio, rdev->bdev); if (test_bit(FailFast, &rdev->flags)) diff --git a/drivers/md/raid10.c b/drivers/md/raid10.c index b0fd3005f5c1..d8c4bf608767 100644 --- a/drivers/md/raid10.c +++ b/drivers/md/raid10.c @@ -808,7 +808,7 @@ static struct md_rdev *read_balance(struct r10conf *conf, nonrot = bdev_nonrot(rdev->bdev); has_nonrot_disk |= nonrot; - pending = atomic_read(&rdev->nr_pending); + pending = nr_pending_read(rdev); if (min_pending > pending && nonrot) { min_pending = pending; best_pending_slot = slot; @@ -849,7 +849,7 @@ static struct md_rdev *read_balance(struct r10conf *conf, } if (slot >= 0) { - atomic_inc(&rdev->nr_pending); + nr_pending_inc(rdev); r10_bio->read_slot = slot; } else rdev = NULL; @@ -1296,12 +1296,12 @@ static void wait_blocked_dev(struct mddev *mddev, struct r10bio *r10_bio) rdev = conf->mirrors[i].rdev; rrdev = conf->mirrors[i].replacement; if (rdev && unlikely(test_bit(Blocked, &rdev->flags))) { - atomic_inc(&rdev->nr_pending); + nr_pending_inc(rdev); blocked_rdev = rdev; break; } if (rrdev && unlikely(test_bit(Blocked, &rrdev->flags))) { - atomic_inc(&rrdev->nr_pending); + nr_pending_inc(rrdev); blocked_rdev = rrdev; break; } @@ -1322,7 +1322,7 @@ static void wait_blocked_dev(struct mddev *mddev, struct r10bio *r10_bio) * Mustn't write here until the bad block * is acknowledged */ - atomic_inc(&rdev->nr_pending); + nr_pending_inc(rdev); set_bit(BlockedBadBlocks, &rdev->flags); blocked_rdev = rdev; break; @@ -1467,11 +1467,11 @@ static void raid10_write_request(struct mddev *mddev, struct bio *bio, } if (rdev) { r10_bio->devs[i].bio = bio; - atomic_inc(&rdev->nr_pending); + nr_pending_inc(rdev); } if (rrdev) { r10_bio->devs[i].repl_bio = bio; - atomic_inc(&rrdev->nr_pending); + nr_pending_inc(rrdev); } } @@ -1731,11 +1731,11 @@ static int raid10_handle_discard(struct mddev *mddev, struct bio *bio) if (rdev) { r10_bio->devs[disk].bio = bio; - atomic_inc(&rdev->nr_pending); + nr_pending_inc(rdev); } if (rrdev) { r10_bio->devs[disk].repl_bio = bio; - atomic_inc(&rrdev->nr_pending); + nr_pending_inc(rrdev); } } @@ -2155,7 +2155,7 @@ static int raid10_remove_disk(struct mddev *mddev, struct md_rdev *rdev) return 0; if (test_bit(In_sync, &rdev->flags) || - atomic_read(&rdev->nr_pending)) { + nr_pending_is_not_zero(rdev)) { err = -EBUSY; goto abort; } @@ -2394,7 +2394,7 @@ static void sync_request_write(struct mddev *mddev, struct r10bio *r10_bio) bio_copy_data(tbio, fbio); - atomic_inc(&conf->mirrors[d].rdev->nr_pending); + nr_pending_inc(conf->mirrors[d].rdev); atomic_inc(&r10_bio->remaining); md_sync_acct(conf->mirrors[d].rdev->bdev, bio_sectors(tbio)); @@ -2552,12 +2552,12 @@ static void recovery_request_write(struct mddev *mddev, struct r10bio *r10_bio) */ d = r10_bio->devs[1].devnum; if (wbio->bi_end_io) { - atomic_inc(&conf->mirrors[d].rdev->nr_pending); + nr_pending_inc(conf->mirrors[d].rdev); md_sync_acct(conf->mirrors[d].rdev->bdev, bio_sectors(wbio)); submit_bio_noacct(wbio); } if (wbio2) { - atomic_inc(&conf->mirrors[d].replacement->nr_pending); + nr_pending_inc(conf->mirrors[d].replacement); md_sync_acct(conf->mirrors[d].replacement->bdev, bio_sectors(wbio2)); submit_bio_noacct(wbio2); @@ -2633,7 +2633,7 @@ static void fix_read_error(struct r10conf *conf, struct mddev *mddev, struct r10 rdev_has_badblock(rdev, r10_bio->devs[sl].addr + sect, s) == 0) { - atomic_inc(&rdev->nr_pending); + nr_pending_inc(rdev); success = sync_page_io(rdev, r10_bio->devs[sl].addr + sect, @@ -2682,7 +2682,7 @@ static void fix_read_error(struct r10conf *conf, struct mddev *mddev, struct r10 !test_bit(In_sync, &rdev->flags)) continue; - atomic_inc(&rdev->nr_pending); + nr_pending_inc(rdev); if (r10_sync_page_io(rdev, r10_bio->devs[sl].addr + sect, @@ -2714,7 +2714,7 @@ static void fix_read_error(struct r10conf *conf, struct mddev *mddev, struct r10 !test_bit(In_sync, &rdev->flags)) continue; - atomic_inc(&rdev->nr_pending); + nr_pending_inc(rdev); switch (r10_sync_page_io(rdev, r10_bio->devs[sl].addr + sect, @@ -3342,9 +3342,9 @@ static sector_t raid10_sync_request(struct mddev *mddev, sector_t sector_nr, continue; } if (mrdev) - atomic_inc(&mrdev->nr_pending); + nr_pending_inc(mrdev); if (mreplace) - atomic_inc(&mreplace->nr_pending); + nr_pending_inc(mreplace); r10_bio = raid10_alloc_init_r10buf(conf); r10_bio->state = 0; @@ -3413,7 +3413,7 @@ static sector_t raid10_sync_request(struct mddev *mddev, sector_t sector_nr, bio->bi_iter.bi_sector = from_addr + rdev->data_offset; bio_set_dev(bio, rdev->bdev); - atomic_inc(&rdev->nr_pending); + nr_pending_inc(rdev); /* and we write to 'i' (if not in_sync) */ for (k=0; kcopies; k++) @@ -3601,7 +3601,7 @@ static sector_t raid10_sync_request(struct mddev *mddev, sector_t sector_nr, continue; } } - atomic_inc(&rdev->nr_pending); + nr_pending_inc(rdev); atomic_inc(&r10_bio->remaining); bio->bi_next = biolist; biolist = bio; @@ -3617,7 +3617,7 @@ static sector_t raid10_sync_request(struct mddev *mddev, sector_t sector_nr, if (rdev == NULL || test_bit(Faulty, &rdev->flags)) continue; - atomic_inc(&rdev->nr_pending); + nr_pending_inc(rdev); /* Need to set up for writing to the replacement */ bio = r10_bio->devs[i].repl_bio; @@ -4918,7 +4918,7 @@ static void reshape_request_write(struct mddev *mddev, struct r10bio *r10_bio) if (!rdev || test_bit(Faulty, &rdev->flags)) continue; - atomic_inc(&rdev->nr_pending); + nr_pending_inc(rdev); md_sync_acct_bio(b, r10_bio->sectors); atomic_inc(&r10_bio->remaining); b->bi_next = NULL; @@ -4998,7 +4998,7 @@ static int handle_reshape_read_error(struct mddev *mddev, goto failed; addr = r10b->devs[slot].addr + idx * PAGE_SIZE; - atomic_inc(&rdev->nr_pending); + nr_pending_inc(rdev); success = sync_page_io(rdev, addr, s << 9, diff --git a/drivers/md/raid5-cache.c b/drivers/md/raid5-cache.c index 874874fe4fa1..1253466666e4 100644 --- a/drivers/md/raid5-cache.c +++ b/drivers/md/raid5-cache.c @@ -1892,7 +1892,7 @@ r5l_recovery_replay_one_stripe(struct r5conf *conf, /* in case device is broken */ rdev = conf->disks[disk_index].rdev; if (rdev) { - atomic_inc(&rdev->nr_pending); + nr_pending_inc(rdev); sync_page_io(rdev, sh->sector, PAGE_SIZE, sh->dev[disk_index].page, REQ_OP_WRITE, false); @@ -1900,7 +1900,7 @@ r5l_recovery_replay_one_stripe(struct r5conf *conf, } rrdev = conf->disks[disk_index].replacement; if (rrdev) { - atomic_inc(&rrdev->nr_pending); + nr_pending_inc(rrdev); sync_page_io(rrdev, sh->sector, PAGE_SIZE, sh->dev[disk_index].page, REQ_OP_WRITE, false); diff --git a/drivers/md/raid5.c b/drivers/md/raid5.c index fd121629603c..118dfed9695f 100644 --- a/drivers/md/raid5.c +++ b/drivers/md/raid5.c @@ -1200,11 +1200,11 @@ static void ops_run_io(struct stripe_head *sh, struct stripe_head_state *s) if (rdev && test_bit(Faulty, &rdev->flags)) rdev = NULL; if (rdev) - atomic_inc(&rdev->nr_pending); + nr_pending_inc(rdev); if (rrdev && test_bit(Faulty, &rrdev->flags)) rrdev = NULL; if (rrdev) - atomic_inc(&rrdev->nr_pending); + nr_pending_inc(rrdev); /* We have already checked bad blocks for reads. Now * need to check for writes. We never accept write errors @@ -1232,7 +1232,7 @@ static void ops_run_io(struct stripe_head *sh, struct stripe_head_state *s) * will dec nr_pending, we must * increment it first. */ - atomic_inc(&rdev->nr_pending); + nr_pending_inc(rdev); md_wait_for_blocked_rdev(rdev, conf->mddev); } else { /* Acknowledged bad block - skip the write */ @@ -3629,7 +3629,7 @@ handle_failed_stripe(struct r5conf *conf, struct stripe_head *sh, if (rdev && test_bit(In_sync, &rdev->flags) && !test_bit(Faulty, &rdev->flags)) - atomic_inc(&rdev->nr_pending); + nr_pending_inc(rdev); else rdev = NULL; if (rdev) { @@ -4730,7 +4730,7 @@ static void analyse_stripe(struct stripe_head *sh, struct stripe_head_state *s) set_bit(BlockedBadBlocks, &rdev->flags); s->blocked_rdev = rdev; - atomic_inc(&rdev->nr_pending); + nr_pending_inc(rdev); } } clear_bit(R5_Insync, &dev->flags); @@ -4768,7 +4768,7 @@ static void analyse_stripe(struct stripe_head *sh, struct stripe_head_state *s) clear_bit(R5_Insync, &dev->flags); if (rdev2 && !test_bit(Faulty, &rdev2->flags)) { s->handle_bad_blocks = 1; - atomic_inc(&rdev2->nr_pending); + nr_pending_inc(rdev2); } else clear_bit(R5_WriteError, &dev->flags); } @@ -4779,7 +4779,7 @@ static void analyse_stripe(struct stripe_head *sh, struct stripe_head_state *s) if (rdev2 && !test_bit(Faulty, &rdev2->flags)) { s->handle_bad_blocks = 1; - atomic_inc(&rdev2->nr_pending); + nr_pending_inc(rdev2); } else clear_bit(R5_MadeGood, &dev->flags); } @@ -4788,7 +4788,7 @@ static void analyse_stripe(struct stripe_head *sh, struct stripe_head_state *s) if (rdev2 && !test_bit(Faulty, &rdev2->flags)) { s->handle_bad_blocks = 1; - atomic_inc(&rdev2->nr_pending); + nr_pending_inc(rdev2); } else clear_bit(R5_MadeGoodRepl, &dev->flags); } @@ -5479,7 +5479,7 @@ static int raid5_read_one_chunk(struct mddev *mddev, struct bio *raid_bio) return 0; } - atomic_inc(&rdev->nr_pending); + nr_pending_inc(rdev); if (rdev_has_badblock(rdev, sector, bio_sectors(raid_bio))) { rdev_dec_pending(rdev, mddev); @@ -8170,7 +8170,7 @@ static int raid5_remove_disk(struct mddev *mddev, struct md_rdev *rdev) clear_bit(In_sync, &rdev->flags); if (test_bit(In_sync, &rdev->flags) || - atomic_read(&rdev->nr_pending)) { + nr_pending_is_not_zero(rdev)) { err = -EBUSY; goto abort; } From patchwork Tue Mar 26 10:29:28 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: tada keisuke X-Patchwork-Id: 13603944 Received: from mo-csw-fb.securemx.jp (mo-csw-fb1120.securemx.jp [210.130.202.128]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 831D7819; Tue, 26 Mar 2024 11:18:14 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=210.130.202.128 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1711451896; cv=none; b=qIVDTq3tO3rtOy+gGERpzQ+4zaaA0Fj7/qFLXDK68RyvGTiygwpI+tImPTPkl94lWhkTfCrSt1WkSsDQgE5iUIXK7oAmRSjygrUQcEkVu/uwzBYf3qNxd00vXiZdZ4SFDyo1YnDWrL0V+YDkF6O2HREoGinTRjaeHqiClEUWE0U= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1711451896; c=relaxed/simple; bh=RXxVwB++RfdUZm7ZxVwICwWyDkmPGasgS6OIKXv04vw=; h=From:To:CC:Subject:Date:Message-ID:Content-Type:MIME-Version; b=NRLTKPR2MfDeelrxbz43qb5pPj8TppKcP08MnUTHaI/2gLFGe4aAt7kK5qPyodaULyzYhQ+KnzEZgMoALn/UwiWMuKWTAvHKEXm6/Wo5k9mk+DRJN1QMVtThUQGrIxTy2tL7PpG4KeQN2TvaLqLPweIhQyQ7yflrYkebig7O7ug= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=kioxia.com; spf=pass smtp.mailfrom=kioxia.com; arc=none smtp.client-ip=210.130.202.128 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=kioxia.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=kioxia.com Received: by mo-csw-fb.securemx.jp (mx-mo-csw-fb1120) id 42QATs4b671700; Tue, 26 Mar 2024 19:29:54 +0900 Received: by mo-csw.securemx.jp (mx-mo-csw1122) id 42QATVpK2610903; Tue, 26 Mar 2024 19:29:31 +0900 X-Iguazu-Qid: 2rWglQcuMYmTG5ydZz X-Iguazu-QSIG: v=2; s=0; t=1711448970; q=2rWglQcuMYmTG5ydZz; m=qMipVdnTgwUQK90bAw3dCRXuz+LfD9mEaB3kpyrVuPQ= Received: from CNN1EMTA01.test.kioxia.com ([202.248.33.144]) by relay.securemx.jp (mx-mr1121) id 42QATTe02222074 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NOT); Tue, 26 Mar 2024 19:29:30 +0900 Received: from Switcher-Post_Send (gateway [10.232.20.1]) by CNN1EMTA01.test.kioxia.com (Postfix) with ESMTP id A16ED2F015; Tue, 26 Mar 2024 19:29:29 +0900 (JST) Received: from CNN1ESTR04.kioxia.com (localhost [127.0.0.1]) by Switcher-Post_Send (Postfix) with ESMTP id 697D01900001E2; Tue, 26 Mar 2024 19:17:03 +0900 (JST) Received: from localhost [127.0.0.1] by CNN1ESTR04.kioxia.com with ESMTP id 0003VAAAAAA01M15; Tue, 26 Mar 2024 19:17:03 +0900 Received: from CNN1EXMB03.r1.kioxia.com (CNN1EXMB03.r1.kioxia.com [10.232.20.152]) by Switcher-Pre_Send (Postfix) with ESMTP id 5E55EA29B6838; Tue, 26 Mar 2024 19:17:03 +0900 (JST) Received: from CNN1EXMB03.r1.kioxia.com (10.232.20.152) by CNN1EXMB03.r1.kioxia.com (10.232.20.152) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Tue, 26 Mar 2024 19:29:28 +0900 Received: from CNN1EXMB03.r1.kioxia.com ([10.13.100.22]) by CNN1EXMB03.r1.kioxia.com ([10.13.100.22]) with mapi id 15.01.2507.035; Tue, 26 Mar 2024 19:29:28 +0900 From: tada keisuke To: "song@kernel.org" , "yukuai3@huawei.com" CC: "linux-raid@vger.kernel.org" , "linux-kernel@vger.kernel.org" Subject: [PATCH 03/11] md: workaround for inconsistency of config state in takeover Thread-Topic: [PATCH 03/11] md: workaround for inconsistency of config state in takeover Thread-Index: Adp/ZBwVAi9WlQnMTJC02G7SaeeULA== Date: Tue, 26 Mar 2024 10:29:28 +0000 Message-ID: Accept-Language: ja-JP, en-US Content-Language: ja-JP X-MS-Has-Attach: X-MS-TNEF-Correlator: x-tm-as-product-ver: ISME-14.0.0.2080-9.0.1002-28274.000 x-tm-as-result: No-10--9.492700-8.000000 x-tmase-matchedrid: 13nEecSBt9GmagT1k9kBpgrcxrzwsv5uXPK9y3z82GtXGTbsQqHbkkog 3hT9f/oCJ2spf+DCeJzmn3xyPJAJoh2P280ZiGmRupDIC9422DoWUg/hm489yO+sdQufriAS4aS bGEJzu4sESV4es7KJNx8GSwQM8TG4+qIQ7qDJ9P+AwosDBeuPwZki3iIBA3o/rhnpK05RvESFFj ccLWe/lssDZ/QWaNswKDX7UR/i5drv4tNJ1H3OMBJo06V2lHTeBGvINcfHqhe52DlOQbIerXeYf bSrjau6JciLWd+sUhmbKItl61J/yZkw8KdMzN86KrauXd3MZDWkkQPcJHKMTx+KTZxSwIuuqgWO R64FI8BOK/uOxPTRF+tu2l1+ba2/A2TBCP7SkgN+3BndfXUhXQ== x-tm-as-user-approved-sender: No x-tm-as-user-blocked-sender: No x-tmase-result: 10--9.492700-8.000000 x-tmase-version: ISME-14.0.0.2080-9.0.1002-28274.000 x-tm-snts-smtp: BED65E495F77F5F10E61A2D081FCFEEEED0BF200A2CBF3F097802F4240FA06132000:8 Precedence: bulk X-Mailing-List: linux-raid@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-CrossPremisesHeadersFilteredBySendConnector: CNN1EXMB03.r1.kioxia.com X-OrganizationHeadersPreserved: CNN1EXMB03.r1.kioxia.com This patch depends on patch 01. An inconsistency occurs between config state in takeover and percpu_ref state. Differentiate percpu_ref state setting for takeover and other to avoid the inconsistency. Therefore, add percpu_ref state setting to match config state in takeover. Signed-off-by: Keisuke TADA Signed-off-by: Toshifumi OHTAKE --- drivers/md/raid5.c | 39 +++++++++++++++++++++++++++++++-------- 1 file changed, 31 insertions(+), 8 deletions(-) diff --git a/drivers/md/raid5.c b/drivers/md/raid5.c index 118dfed9695f..c7186ebcd299 100644 --- a/drivers/md/raid5.c +++ b/drivers/md/raid5.c @@ -7405,7 +7405,8 @@ static unsigned long raid5_cache_count(struct shrinker *shrink, return max_stripes - min_stripes; } -static struct r5conf *setup_conf(struct mddev *mddev) + +static struct r5conf *setup_conf(struct mddev *mddev, bool quiesce) { struct r5conf *conf; int raid_disk, memory, max_disks; @@ -7416,6 +7417,7 @@ static struct r5conf *setup_conf(struct mddev *mddev) int group_cnt; struct r5worker_group *new_group; int ret = -ENOMEM; + unsigned int percpu_ref_init_flags; if (mddev->new_level != 5 && mddev->new_level != 4 @@ -7486,6 +7488,10 @@ static struct r5conf *setup_conf(struct mddev *mddev) init_llist_head(&conf->released_stripes); atomic_set(&conf->active_stripes, 0); atomic_set(&conf->preread_active_stripes, 0); + if (quiesce) + percpu_ref_init_flags = PERCPU_REF_ALLOW_REINIT | PERCPU_REF_INIT_DEAD; + else + percpu_ref_init_flags = PERCPU_REF_ALLOW_REINIT; atomic_set(&conf->active_aligned_reads, 0); spin_lock_init(&conf->pending_bios_lock); conf->batch_bio_dispatch = true; @@ -7666,6 +7672,23 @@ static struct r5conf *setup_conf(struct mddev *mddev) return ERR_PTR(ret); } +static struct r5conf *setup_conf_for_run(struct mddev *mddev) +{ + return setup_conf(mddev, false); +} + +static struct r5conf *setup_conf_for_takeover(struct mddev *mddev) +{ + struct r5conf *conf; + bool quiesce = false; + + if (mddev->level == 4 || mddev->level == 5 || mddev->level == 6) { + conf = mddev->private; + quiesce = false; + } + return setup_conf(mddev, quiesce); +} + static int only_parity(int raid_disk, int algo, int raid_disks, int max_degraded) { switch (algo) { @@ -7893,7 +7916,7 @@ static int raid5_run(struct mddev *mddev) } if (mddev->private == NULL) - conf = setup_conf(mddev); + conf = setup_conf_for_run(mddev); else conf = mddev->private; @@ -8661,7 +8684,7 @@ static void *raid45_takeover_raid0(struct mddev *mddev, int level) /* make sure it will be not marked as dirty */ mddev->recovery_cp = MaxSector; - return setup_conf(mddev); + return setup_conf_for_takeover(mddev); } static void *raid5_takeover_raid1(struct mddev *mddev) @@ -8689,7 +8712,7 @@ static void *raid5_takeover_raid1(struct mddev *mddev) mddev->new_layout = ALGORITHM_LEFT_SYMMETRIC; mddev->new_chunk_sectors = chunksect; - ret = setup_conf(mddev); + ret = setup_conf_for_takeover(mddev); if (!IS_ERR(ret)) mddev_clear_unsupported_flags(mddev, UNSUPPORTED_MDDEV_FLAGS); @@ -8726,7 +8749,7 @@ static void *raid5_takeover_raid6(struct mddev *mddev) mddev->new_layout = new_layout; mddev->delta_disks = -1; mddev->raid_disks -= 1; - return setup_conf(mddev); + return setup_conf_for_takeover(mddev); } static int raid5_check_reshape(struct mddev *mddev) @@ -8804,7 +8827,7 @@ static void *raid5_takeover(struct mddev *mddev) if (mddev->level == 4) { mddev->new_layout = ALGORITHM_PARITY_N; mddev->new_level = 5; - return setup_conf(mddev); + return setup_conf_for_takeover(mddev); } if (mddev->level == 6) return raid5_takeover_raid6(mddev); @@ -8824,7 +8847,7 @@ static void *raid4_takeover(struct mddev *mddev) mddev->layout == ALGORITHM_PARITY_N) { mddev->new_layout = 0; mddev->new_level = 4; - return setup_conf(mddev); + return setup_conf_for_takeover(mddev); } return ERR_PTR(-EINVAL); } @@ -8874,7 +8897,7 @@ static void *raid6_takeover(struct mddev *mddev) mddev->new_layout = new_layout; mddev->delta_disks = 1; mddev->raid_disks += 1; - return setup_conf(mddev); + return setup_conf_for_takeover(mddev); } static int raid5_change_consistency_policy(struct mddev *mddev, const char *buf) From patchwork Tue Mar 26 10:29:31 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: tada keisuke X-Patchwork-Id: 13604035 Received: from mo-csw-fb.securemx.jp (mo-csw-fb1120.securemx.jp [210.130.202.128]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 350511804F; Tue, 26 Mar 2024 12:15:16 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=210.130.202.128 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1711455319; cv=none; b=oxF6tWHuoR16cSGlg2JMt6KNxF3C3mnyMBf8lKGZ+4VlFugx8K65NG6PLV44lBjaJD8AfuV6yzpXvBEY7Dmmi18b6M+NLKHlsry2YmQcdFabVt4+Xob1/yN07mW3NdKDn4YN0WYxcYFCOijhv9Z+ghhMHckYrAYGCRAe2zBQjuM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1711455319; c=relaxed/simple; bh=7GKEcD8lRU98zTCcjYmD6skL4kJppLGyorf7w2xZPcQ=; h=From:To:CC:Subject:Date:Message-ID:Content-Type:MIME-Version; b=AeL5/MOBnoAsp1YCNnRHIOCrevWpvSxfsOJBdBNvVN0XYVWIx8kQ/urrMd7K24XuaQj/kE5ho+WHSYO5E1QijZ5+bule3mhA/Bvm7xVQ3PzLVB4G9XQSOEndeQeltBim1+RAfpJyF52SWKlT2XJ0Fw5NeKuE1CtjKm4nSqfhiRo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=kioxia.com; spf=pass smtp.mailfrom=kioxia.com; arc=none smtp.client-ip=210.130.202.128 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=kioxia.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=kioxia.com Received: by mo-csw-fb.securemx.jp (mx-mo-csw-fb1120) id 42QATmXK671698; Tue, 26 Mar 2024 19:29:48 +0900 Received: by mo-csw.securemx.jp (mx-mo-csw1120) id 42QATXKa2209272; Tue, 26 Mar 2024 19:29:33 +0900 X-Iguazu-Qid: 2rWgQ8FBVvU2Je3XjI X-Iguazu-QSIG: v=2; s=0; t=1711448972; q=2rWgQ8FBVvU2Je3XjI; m=CbOMDN9j+hbog7p/aMHarUVt7cTJKkxEtg07Zdph4ek= Received: from CNN1EMTA03.test.kioxia.com ([202.248.33.144]) by relay.securemx.jp (mx-mr1123) id 42QATWjE2317794 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NOT); Tue, 26 Mar 2024 19:29:32 +0900 Received: from Switcher-Post_Send (gateway [10.232.20.1]) by CNN1EMTA03.test.kioxia.com (Postfix) with ESMTP id 0C4A232440; Tue, 26 Mar 2024 19:29:32 +0900 (JST) Received: from CNN1ESTR03.kioxia.com (localhost [127.0.0.1]) by Switcher-Post_Send (Postfix) with ESMTP id 86A391938D7203; Tue, 26 Mar 2024 19:08:05 +0900 (JST) Received: from localhost [127.0.0.1] by CNN1ESTR03.kioxia.com with ESMTP id 0002VAAAAAA012AM; Tue, 26 Mar 2024 19:08:05 +0900 Received: from CNN1EXMB02.r1.kioxia.com (CNN1EXMB02.r1.kioxia.com [10.232.20.151]) by Switcher-Pre_Send (Postfix) with ESMTP id 7B8ABA035DC28; Tue, 26 Mar 2024 19:08:05 +0900 (JST) Received: from CNN1EXMB03.r1.kioxia.com (10.232.20.152) by CNN1EXMB02.r1.kioxia.com (10.232.20.151) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Tue, 26 Mar 2024 19:29:31 +0900 Received: from CNN1EXMB03.r1.kioxia.com ([10.13.100.22]) by CNN1EXMB03.r1.kioxia.com ([10.13.100.22]) with mapi id 15.01.2507.035; Tue, 26 Mar 2024 19:29:31 +0900 From: tada keisuke To: "song@kernel.org" , "yukuai3@huawei.com" CC: "linux-raid@vger.kernel.org" , "linux-kernel@vger.kernel.org" Subject: [PATCH 04/11] md: minimize execution of zero check for nr_pending Thread-Topic: [PATCH 04/11] md: minimize execution of zero check for nr_pending Thread-Index: Adp/ZCwIEaSNdvzYSU+n+frmE556SA== Date: Tue, 26 Mar 2024 10:29:31 +0000 Message-ID: <5319159bd74f4c4c9979c748b216b2f8@kioxia.com> Accept-Language: ja-JP, en-US Content-Language: ja-JP X-MS-Has-Attach: X-MS-TNEF-Correlator: x-tm-as-product-ver: ISME-14.0.0.2080-9.0.1002-28274.000 x-tm-as-result: No-10--5.659600-8.000000 x-tmase-matchedrid: 5V0fm+sVFB2magT1k9kBpgrcxrzwsv5uXPK9y3z82GtXGTbsQqHbkmzE TQ7/otyjd+FThkXYawcVNmOCBj98YKZY4PxfRMWETuctSpiuWyUUi4Ehat05499RlPzeVuQQB4x q6g7kDalvF9oQ/B54yGPKQgbIX4yuFdV6KRtQicSkkQPcJHKMT+ovpkXHOUFXv1l2Uvx6idpqHX ONfTwSQsRB0bsfrpPIFT1PBs+MVY3ocuK7+JBlJC5g7R642iP7Sqy+Sk/9UVNHtFfvh1y31yo9L wbibKKfG/fqr4pTSlM= x-tm-as-user-approved-sender: No x-tm-as-user-blocked-sender: No x-tmase-result: 10--5.659600-8.000000 x-tmase-version: ISME-14.0.0.2080-9.0.1002-28274.000 x-tm-snts-smtp: 9B2049F3415A909983D745DE68787A79943D7C77A482092E0C8205982E682B572000:8 Precedence: bulk X-Mailing-List: linux-raid@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-CrossPremisesHeadersFilteredBySendConnector: CNN1EXMB02.r1.kioxia.com X-OrganizationHeadersPreserved: CNN1EXMB02.r1.kioxia.com This patch depends on patch 02. The Faulty flag is required for the if statement to be true. A rdev in Faulty state does not issue I/O commands. So, nr_pending is also not increased. Therefore, decrement and zero check of nr_pending can be separated and still be consistent. Signed-off-by: Keisuke TADA Signed-off-by: Toshifumi OHTAKE --- drivers/md/md.h | 14 ++++++-------- 1 file changed, 6 insertions(+), 8 deletions(-) diff --git a/drivers/md/md.h b/drivers/md/md.h index b990be0981bc..ed7e36212d58 100644 --- a/drivers/md/md.h +++ b/drivers/md/md.h @@ -236,11 +236,6 @@ static inline unsigned int nr_pending_read(struct md_rdev *rdev) return atomic_read(&rdev->nr_pending); } -static inline bool nr_pending_dec_and_test(struct md_rdev *rdev) -{ - return atomic_dec_and_test(&rdev->nr_pending); -} - static inline int is_badblock(struct md_rdev *rdev, sector_t s, int sectors, sector_t *first_bad, int *bad_sectors) { @@ -875,9 +870,12 @@ static inline bool is_rdev_broken(struct md_rdev *rdev) static inline void rdev_dec_pending(struct md_rdev *rdev, struct mddev *mddev) { int faulty = test_bit(Faulty, &rdev->flags); - if (nr_pending_dec_and_test(rdev) && faulty) { - set_bit(MD_RECOVERY_NEEDED, &mddev->recovery); - md_wakeup_thread(mddev->thread); + nr_pending_dec(rdev); + if (faulty) { + if (nr_pending_is_zero(rdev)) { + set_bit(MD_RECOVERY_NEEDED, &mddev->recovery); + md_wakeup_thread(mddev->thread); + } } } From patchwork Tue Mar 26 10:29:34 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: tada keisuke X-Patchwork-Id: 13603945 Received: from mo-csw-fb.securemx.jp (mo-csw-fb1120.securemx.jp [210.130.202.128]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 46D8E548F4; Tue, 26 Mar 2024 11:18:48 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=210.130.202.128 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1711451931; cv=none; b=uFG2c121192t+WYZYEQC/nGCLxZ5x/2Z118/8IU4IzIKNxEMdsqgS52DjRGdWKpSegE08Kdfb2HG43H+KEHZmT9SonyJ84Inr8lfYpJQiQi9HTyMHVqn5mkb+nNrbTEHjJGAWwgvW21go9YQmUyO+cEiDXI/YWGm9a6OfxVYwnU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1711451931; c=relaxed/simple; bh=KufBODuLQhqwQMLcQD/3FVI3mBLOX9NAtVALwxNOPBk=; h=From:To:CC:Subject:Date:Message-ID:Content-Type:MIME-Version; b=E9whNq5b1kVUTKPBl1gnhA2Ntofq1wmV4flfVKbYyps0ritmYL0Wn2xxS5xlpUfPz+w5iHyWgUSzyYWXc3tnuVKPfR2AXGZjFoygFCWnYgtmT9Q5qwIgK0ZvGkyw/QIioCiwG7aiU5v95WWBuTi1aIXCG7dTMhacJZbCYLwkq7E= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=kioxia.com; spf=pass smtp.mailfrom=kioxia.com; arc=none smtp.client-ip=210.130.202.128 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=kioxia.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=kioxia.com Received: by mo-csw-fb.securemx.jp (mx-mo-csw-fb1120) id 42QAToCX671699; Tue, 26 Mar 2024 19:29:50 +0900 Received: by mo-csw.securemx.jp (mx-mo-csw1122) id 42QATaOw2611071; Tue, 26 Mar 2024 19:29:36 +0900 X-Iguazu-Qid: 2rWgHjhWsbaU6kJ3f0 X-Iguazu-QSIG: v=2; s=0; t=1711448975; q=2rWgHjhWsbaU6kJ3f0; m=5FYyzYhBQls1LZjbj4dr6Qe9A2WGNlbNbcjLFfjbYJ4= Received: from CNN1EMTA01.test.kioxia.com ([202.248.33.144]) by relay.securemx.jp (mx-mr1122) id 42QATZKB2453832 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NOT); Tue, 26 Mar 2024 19:29:35 +0900 Received: from Switcher-Post_Send (gateway [10.232.20.1]) by CNN1EMTA01.test.kioxia.com (Postfix) with ESMTP id 33B652F015; Tue, 26 Mar 2024 19:29:35 +0900 (JST) Received: from CNN1ESTR03.kioxia.com (localhost [127.0.0.1]) by Switcher-Post_Send (Postfix) with ESMTP id A4F681938D7227; Tue, 26 Mar 2024 19:08:08 +0900 (JST) Received: from localhost [127.0.0.1] by CNN1ESTR03.kioxia.com with ESMTP id 0002VAAAAAA012CM; Tue, 26 Mar 2024 19:08:08 +0900 Received: from CNN1EXMB04.r1.kioxia.com (CNN1EXMB04.r1.kioxia.com [10.232.20.153]) by Switcher-Pre_Send (Postfix) with ESMTP id 99C84A035DC28; Tue, 26 Mar 2024 19:08:08 +0900 (JST) Received: from CNN1EXMB03.r1.kioxia.com (10.232.20.152) by CNN1EXMB04.r1.kioxia.com (10.232.20.153) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Tue, 26 Mar 2024 19:29:34 +0900 Received: from CNN1EXMB03.r1.kioxia.com ([10.13.100.22]) by CNN1EXMB03.r1.kioxia.com ([10.13.100.22]) with mapi id 15.01.2507.035; Tue, 26 Mar 2024 19:29:34 +0900 From: tada keisuke To: "song@kernel.org" , "yukuai3@huawei.com" CC: "linux-raid@vger.kernel.org" , "linux-kernel@vger.kernel.org" Subject: [PATCH 05/11] md: match the type of variables to percpu_ref Thread-Topic: [PATCH 05/11] md: match the type of variables to percpu_ref Thread-Index: Adp/ZDpSRCvn7gYcTuKr3Qsn+dny/g== Date: Tue, 26 Mar 2024 10:29:34 +0000 Message-ID: Accept-Language: ja-JP, en-US Content-Language: ja-JP X-MS-Has-Attach: X-MS-TNEF-Correlator: x-tm-as-product-ver: ISME-14.0.0.2080-9.0.1002-28274.000 x-tm-as-result: No-10--5.106300-8.000000 x-tmase-matchedrid: 5V0fm+sVFB2magT1k9kBpgrcxrzwsv5uXPK9y3z82GsPdZmyx8xAa+K5 fhRxt1aADpxsntCj+iajrLRBxwnZS7VQ6XPWwtdyEXjPIvKd74BMkOX0UoduuRFBDiQWqOMkJwY Oi+iMxY3U5iXDpe6YeP6mFiC4o7QUdJMdJS+tDywZgmFGHqyx6+DDEwfnBUy6lwV2iaAfSWeDGx /OQ1GV8t0H8LFZNFG7gaH1JgrOSO7lXdnkMuZ1VhpbbFGOduqkezVlI2TOQlzhJbWRfawt93FGC DH6TcAeMm05OGx1HAU= x-tm-as-user-approved-sender: No x-tm-as-user-blocked-sender: No x-tmase-result: 10--5.106300-8.000000 x-tmase-version: ISME-14.0.0.2080-9.0.1002-28274.000 x-tm-snts-smtp: 91D0754E4EDB280E8754A76B207E8AEA70593F419E63FA6610B8F89FB2973A632000:8 Precedence: bulk X-Mailing-List: linux-raid@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-CrossPremisesHeadersFilteredBySendConnector: CNN1EXMB04.r1.kioxia.com X-OrganizationHeadersPreserved: CNN1EXMB04.r1.kioxia.com This patch depends on patch 04. The counter size of percpu_ref is different from that of atomic_t. This is the only place where the value of nr_pending is read. The others are used for zero check and type has no effect. Signed-off-by: Keisuke TADA Signed-off-by: Toshifumi OHTAKE --- drivers/md/md.h | 4 ++-- drivers/md/raid1.c | 4 ++-- drivers/md/raid10.c | 4 ++-- 3 files changed, 6 insertions(+), 6 deletions(-) diff --git a/drivers/md/md.h b/drivers/md/md.h index ed7e36212d58..724439400ed1 100644 --- a/drivers/md/md.h +++ b/drivers/md/md.h @@ -231,9 +231,9 @@ static inline bool nr_pending_is_not_zero(struct md_rdev *rdev) return atomic_read(&rdev->nr_pending) != 0; } -static inline unsigned int nr_pending_read(struct md_rdev *rdev) +static inline unsigned long nr_pending_read(struct md_rdev *rdev) { - return atomic_read(&rdev->nr_pending); + return (unsigned long)atomic_read(&rdev->nr_pending); } static inline int is_badblock(struct md_rdev *rdev, sector_t s, int sectors, diff --git a/drivers/md/raid1.c b/drivers/md/raid1.c index 9cf56bc1340f..43cc36fec7a2 100644 --- a/drivers/md/raid1.c +++ b/drivers/md/raid1.c @@ -751,7 +751,7 @@ static bool rdev_readable(struct md_rdev *rdev, struct r1bio *r1_bio) struct read_balance_ctl { sector_t closest_dist; int closest_dist_disk; - int min_pending; + long min_pending; int min_pending_disk; int sequential_disk; int readable_disks; @@ -771,7 +771,7 @@ static int choose_best_rdev(struct r1conf *conf, struct r1bio *r1_bio) for (disk = 0 ; disk < conf->raid_disks * 2 ; disk++) { struct md_rdev *rdev; sector_t dist; - unsigned int pending; + unsigned long pending; if (r1_bio->bios[disk] == IO_BLOCKED) continue; diff --git a/drivers/md/raid10.c b/drivers/md/raid10.c index d8c4bf608767..cd5094c4df50 100644 --- a/drivers/md/raid10.c +++ b/drivers/md/raid10.c @@ -733,7 +733,7 @@ static struct md_rdev *read_balance(struct r10conf *conf, int do_balance; int best_dist_slot, best_pending_slot; bool has_nonrot_disk = false; - unsigned int min_pending; + unsigned long min_pending; struct geom *geo = &conf->geo; raid10_find_phys(conf, r10_bio); @@ -753,7 +753,7 @@ static struct md_rdev *read_balance(struct r10conf *conf, sector_t first_bad; int bad_sectors; sector_t dev_sector; - unsigned int pending; + unsigned long pending; bool nonrot; if (r10_bio->devs[slot].bio == IO_BLOCKED) From patchwork Tue Mar 26 10:29:37 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: tada keisuke X-Patchwork-Id: 13603940 Received: from mo-csw-fb.securemx.jp (mo-csw-fb1120.securemx.jp [210.130.202.128]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0A467134A8; Tue, 26 Mar 2024 11:16:48 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=210.130.202.128 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1711451811; cv=none; b=TCDgxGj3KD0UA6mjYn/eAcMVaoq7vjdRZyNlad+PoUjxdKtK05bl8eunLlLq8WB2J7yRtqUgNuIOrxeero6flgyyPT7X0ms+lrHonmM2AisOSKuM9O17SMFVoJZh+twdnZKZOpwyzc9mn5OLjifR+e/yNhHyQGCrl5U8pmexh4E= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1711451811; c=relaxed/simple; bh=xSEzLoXG7ilM1etL8aaSD1TDnPKgw15ps6p0vsfPTXo=; h=From:To:CC:Subject:Date:Message-ID:Content-Type:MIME-Version; b=bQc5j7FOAycvyI/ea16OtSaWE84mU2mFfMcLXwYZKhNMD5HntyRw5IAMOOJfj5dWZO2P8x+JH6rOLS1nhYXIwhG5zr+LC2FCQ6ymadJs+OZqi0O5EAg9yApRLEmIdYpDFcy1MPFu54iwZHTBkb95wHHtgeiKfr0ikh2k9Glmcqo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=kioxia.com; spf=pass smtp.mailfrom=kioxia.com; arc=none smtp.client-ip=210.130.202.128 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=kioxia.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=kioxia.com Received: by mo-csw-fb.securemx.jp (mx-mo-csw-fb1120) id 42QATssb671701; Tue, 26 Mar 2024 19:29:54 +0900 Received: by mo-csw.securemx.jp (mx-mo-csw1122) id 42QATdJk2611184; Tue, 26 Mar 2024 19:29:39 +0900 X-Iguazu-Qid: 2rWg9o4uinxHarSWO2 X-Iguazu-QSIG: v=2; s=0; t=1711448978; q=2rWg9o4uinxHarSWO2; m=VTooRhJM/Sss0dbxVP1M9CwhVAaK2S3XeIYq6RBLwrU= Received: from CNN1EMTA03.test.kioxia.com ([202.248.33.144]) by relay.securemx.jp (mx-mr1122) id 42QATcWM2454001 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NOT); Tue, 26 Mar 2024 19:29:38 +0900 Received: from Switcher-Post_Send (gateway [10.232.20.1]) by CNN1EMTA03.test.kioxia.com (Postfix) with ESMTP id 3A26C32440; Tue, 26 Mar 2024 19:29:38 +0900 (JST) Received: from CNN1ESTR03.kioxia.com (localhost [127.0.0.1]) by Switcher-Post_Send (Postfix) with ESMTP id AB3561938D7224; Tue, 26 Mar 2024 19:08:11 +0900 (JST) Received: from localhost [127.0.0.1] by CNN1ESTR03.kioxia.com with ESMTP id 0002VAAAAAA012FG; Tue, 26 Mar 2024 19:08:11 +0900 Received: from CNN1EXMB01.r1.kioxia.com (CNN1EXMB01.r1.kioxia.com [10.232.20.150]) by Switcher-Pre_Send (Postfix) with ESMTP id A0015A035DC28; Tue, 26 Mar 2024 19:08:11 +0900 (JST) Received: from CNN1EXMB03.r1.kioxia.com (10.232.20.152) by CNN1EXMB01.r1.kioxia.com (10.232.20.150) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Tue, 26 Mar 2024 19:29:37 +0900 Received: from CNN1EXMB03.r1.kioxia.com ([10.13.100.22]) by CNN1EXMB03.r1.kioxia.com ([10.13.100.22]) with mapi id 15.01.2507.035; Tue, 26 Mar 2024 19:29:37 +0900 From: tada keisuke To: "song@kernel.org" , "yukuai3@huawei.com" CC: "linux-raid@vger.kernel.org" , "linux-kernel@vger.kernel.org" Subject: [PATCH 06/11] md: avoid conflicts in active_aligned_reads operations Thread-Topic: [PATCH 06/11] md: avoid conflicts in active_aligned_reads operations Thread-Index: Adp/ZElvNupNLKs/Rxilvj3/IfhIIg== Date: Tue, 26 Mar 2024 10:29:37 +0000 Message-ID: Accept-Language: ja-JP, en-US Content-Language: ja-JP X-MS-Has-Attach: X-MS-TNEF-Correlator: x-tm-as-product-ver: ISME-14.0.0.2080-9.0.1002-28274.000 x-tm-as-result: No-10--10.964700-8.000000 x-tmase-matchedrid: +gTBWF+Z0ROmagT1k9kBpgrcxrzwsv5uXPK9y3z82GsPdZmyx8xAa4Ul JlxlTIT+SMhOCdNodgNTbKunqk1vfbVQ6XPWwtdyEXjPIvKd74BMkOX0UoduufLIj08N6FsEuJE UmdYPGSVzy5czsqHK1z4BLF18v1Ji23JrSYCBAcfN+qWlu2ZxaFr2hZxjCLzqMoh6scCF9jF96t 9wDfdERVvLgfDdKWEFEn91NheE+HWip6xhGipIWazGfgakLdja0NaTZUEzRZzC9DxHKrnk8luWw Soz2HVyH7iboQD+7I4VaGy/EyU2eafccBtB84hSnXdphQTSK/K/yN2q8U674odzPqBXhP/S4Mmu En4nwqo1W+3DxdRobxDflJk00ivp0FQxfjRHOzi4u3nS+3EEDuDDEwfnBUy6lwV2iaAfSWeDGx/ OQ1GV8t0H8LFZNFG7gaH1JgrOSO4aMUyeC0staN0RHUNUuP2opWu923L0/V6a+gqq4Q/lRG0nH4 VYvyBw3HbxZLSwb85agnrZz8DBG+ulxyHOcPoH x-tm-as-user-approved-sender: No x-tm-as-user-blocked-sender: No x-tmase-result: 10--10.964700-8.000000 x-tmase-version: ISME-14.0.0.2080-9.0.1002-28274.000 x-tm-snts-smtp: BD34B6522F5962555E5B84274C8D74DA3A9F433931466499FF82D0D1F3A5AB512000:8 Precedence: bulk X-Mailing-List: linux-raid@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-CrossPremisesHeadersFilteredBySendConnector: CNN1EXMB01.r1.kioxia.com X-OrganizationHeadersPreserved: CNN1EXMB01.r1.kioxia.com This patch depends on patch 03. Change the type of active_aligned_reads from atomic_t to percpu_ref in percpu mode. Signed-off-by: Keisuke TADA Signed-off-by: Toshifumi OHTAKE --- drivers/md/raid5.c | 26 ++++++++++++++++++-------- drivers/md/raid5.h | 13 ++++--------- 2 files changed, 22 insertions(+), 17 deletions(-) diff --git a/drivers/md/raid5.c b/drivers/md/raid5.c index c7186ebcd299..99f42bc02231 100644 --- a/drivers/md/raid5.c +++ b/drivers/md/raid5.c @@ -5436,8 +5436,7 @@ static void raid5_align_endio(struct bio *bi) if (!error) { bio_endio(raid_bi); - if (active_aligned_reads_dec_and_test(conf)) - wake_up(&conf->wait_for_quiescent); + active_aligned_reads_dec(conf); return; } @@ -5508,8 +5507,8 @@ static int raid5_read_one_chunk(struct mddev *mddev, struct bio *raid_bio) /* quiesce is in progress, so we need to undo io activation and wait * for it to finish */ - if (did_inc && active_aligned_reads_dec_and_test(conf)) - wake_up(&conf->wait_for_quiescent); + if (did_inc) + active_aligned_reads_dec(conf); spin_lock_irq(&conf->device_lock); wait_event_lock_irq(conf->wait_for_quiescent, conf->quiesce == 0, conf->device_lock); @@ -6609,8 +6608,7 @@ static int retry_aligned_read(struct r5conf *conf, struct bio *raid_bio, bio_endio(raid_bio); - if (active_aligned_reads_dec_and_test(conf)) - wake_up(&conf->wait_for_quiescent); + active_aligned_reads_dec(conf); return handled; } @@ -7324,6 +7322,7 @@ static void free_conf(struct r5conf *conf) { int i; + percpu_ref_exit(&conf->active_aligned_reads); log_exit(conf); shrinker_free(conf->shrinker); @@ -7405,6 +7404,12 @@ static unsigned long raid5_cache_count(struct shrinker *shrink, return max_stripes - min_stripes; } +static void percpu_wakeup_handle_req_active(struct percpu_ref *r) +{ + struct r5conf *conf = container_of(r, struct r5conf, active_aligned_reads); + + wake_up(&conf->wait_for_quiescent); +} static struct r5conf *setup_conf(struct mddev *mddev, bool quiesce) { @@ -7492,7 +7497,10 @@ static struct r5conf *setup_conf(struct mddev *mddev, bool quiesce) percpu_ref_init_flags = PERCPU_REF_ALLOW_REINIT | PERCPU_REF_INIT_DEAD; else percpu_ref_init_flags = PERCPU_REF_ALLOW_REINIT; - atomic_set(&conf->active_aligned_reads, 0); + ret = percpu_ref_init(&conf->active_aligned_reads, percpu_wakeup_handle_req_active, + percpu_ref_init_flags, GFP_KERNEL); + if (ret) + goto abort; spin_lock_init(&conf->pending_bios_lock); conf->batch_bio_dispatch = true; rdev_for_each(rdev, mddev) { @@ -7684,7 +7692,7 @@ static struct r5conf *setup_conf_for_takeover(struct mddev *mddev) if (mddev->level == 4 || mddev->level == 5 || mddev->level == 6) { conf = mddev->private; - quiesce = false; + quiesce = percpu_ref_is_dying(&conf->active_aligned_reads); } return setup_conf(mddev, quiesce); } @@ -8641,6 +8649,7 @@ static void raid5_quiesce(struct mddev *mddev, int quiesce) * quiesce started and reverts to slow (locked) path. */ smp_store_release(&conf->quiesce, 2); + percpu_ref_kill(&conf->active_aligned_reads); wait_event_cmd(conf->wait_for_quiescent, atomic_read(&conf->active_stripes) == 0 && active_aligned_reads_is_zero(conf), @@ -8653,6 +8662,7 @@ static void raid5_quiesce(struct mddev *mddev, int quiesce) } else { /* re-enable writes */ lock_all_device_hash_locks_irq(conf); + percpu_ref_reinit(&conf->active_aligned_reads); conf->quiesce = 0; wake_up(&conf->wait_for_quiescent); wake_up(&conf->wait_for_overlap); diff --git a/drivers/md/raid5.h b/drivers/md/raid5.h index 5bd6bb3540c5..c4ab418e2084 100644 --- a/drivers/md/raid5.h +++ b/drivers/md/raid5.h @@ -621,7 +621,7 @@ struct r5conf { unsigned int retry_read_offset; /* sector offset into retry_read_aligned */ struct bio *retry_read_aligned_list; /* aligned bios retry list */ atomic_t preread_active_stripes; /* stripes with scheduled io */ - atomic_t active_aligned_reads; + struct percpu_ref active_aligned_reads; atomic_t pending_full_writes; /* full write backlog */ int bypass_count; /* bypassed prereads */ int bypass_threshold; /* preread nice */ @@ -698,22 +698,17 @@ struct r5conf { static inline void active_aligned_reads_inc(struct r5conf *conf) { - atomic_inc(&conf->active_aligned_reads); + percpu_ref_get(&conf->active_aligned_reads); } static inline void active_aligned_reads_dec(struct r5conf *conf) { - atomic_dec(&conf->active_aligned_reads); + percpu_ref_put(&conf->active_aligned_reads); } static inline bool active_aligned_reads_is_zero(struct r5conf *conf) { - return atomic_read(&conf->active_aligned_reads) == 0; -} - -static inline bool active_aligned_reads_dec_and_test(struct r5conf *conf) -{ - return atomic_dec_and_test(&conf->active_aligned_reads); + return percpu_ref_is_zero(&conf->active_aligned_reads); } #if PAGE_SIZE == DEFAULT_STRIPE_SIZE From patchwork Tue Mar 26 10:29:41 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: tada keisuke X-Patchwork-Id: 13603916 Received: from mo-csw-fb.securemx.jp (mo-csw-fb1121.securemx.jp [210.130.202.129]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id DFBE01B59A; Tue, 26 Mar 2024 11:05:28 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=210.130.202.129 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1711451131; cv=none; b=EicYdT6okheoVHvOMqRXHUbD+F3L50gzHGVibH3lZKIIn6gsGZbtezYrb68xhLMBBN7zZmlOMZ3OfIFzYdcieQNmHjIaQXIkE8JKNRVfP7CaN2y/krJ48CCg2++WIx+gEwc41ynFFgJ9o2hDRLrK7eDbbXAOIxRccnhrS4A3ChI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1711451131; c=relaxed/simple; bh=pWwwp25BjYeP7PCVUAYK9bGOXwcU5GgFSs/6zOVf1kI=; h=From:To:CC:Subject:Date:Message-ID:Content-Type:MIME-Version; b=HKUxp4qmyOKyUcnUHn8J2qiqw65BQifhinxNZ22z6WIs2PG4adfOSFuYumhwnqPtmHnd2jdKeanbZCBeauzkNbIJ0t7Ik1sYJatE3oDREyK+UwVP9lyg7E93jXBxZiTyMqeDbKVv1RKWV07LfsddsaiQG2jM2wse+kqxoRG5BdY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=kioxia.com; spf=pass smtp.mailfrom=kioxia.com; arc=none smtp.client-ip=210.130.202.129 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=kioxia.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=kioxia.com Received: by mo-csw-fb.securemx.jp (mx-mo-csw-fb1121) id 42QATtd4846052; Tue, 26 Mar 2024 19:29:55 +0900 Received: by mo-csw.securemx.jp (mx-mo-csw1122) id 42QAThJH2611311; Tue, 26 Mar 2024 19:29:43 +0900 X-Iguazu-Qid: 2rWgWcEMw7gzoPQ1DM X-Iguazu-QSIG: v=2; s=0; t=1711448982; q=2rWgWcEMw7gzoPQ1DM; m=PaWt10EU8bzrpSXl5RYsZTwdoaLSopBY8IGYGc3YOyI= Received: from CNN1EMTA01.test.kioxia.com ([202.248.33.144]) by relay.securemx.jp (mx-mr1121) id 42QATf9m2222317 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NOT); Tue, 26 Mar 2024 19:29:42 +0900 Received: from Switcher-Post_Send (gateway [10.232.20.1]) by CNN1EMTA01.test.kioxia.com (Postfix) with ESMTP id D19152F015; Tue, 26 Mar 2024 19:29:41 +0900 (JST) Received: from CNN1ESTR04.kioxia.com (localhost [127.0.0.1]) by Switcher-Post_Send (Postfix) with ESMTP id 996031900001E2; Tue, 26 Mar 2024 19:17:15 +0900 (JST) Received: from localhost [127.0.0.1] by CNN1ESTR04.kioxia.com with ESMTP id 0003VAAAAAA01M89; Tue, 26 Mar 2024 19:17:15 +0900 Received: from CNN1EXMB01.r1.kioxia.com (CNN1EXMB01.r1.kioxia.com [10.232.20.150]) by Switcher-Pre_Send (Postfix) with ESMTP id 8DFC6A29B6838; Tue, 26 Mar 2024 19:17:15 +0900 (JST) Received: from CNN1EXMB03.r1.kioxia.com (10.232.20.152) by CNN1EXMB01.r1.kioxia.com (10.232.20.150) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Tue, 26 Mar 2024 19:29:41 +0900 Received: from CNN1EXMB03.r1.kioxia.com ([10.13.100.22]) by CNN1EXMB03.r1.kioxia.com ([10.13.100.22]) with mapi id 15.01.2507.035; Tue, 26 Mar 2024 19:29:41 +0900 From: tada keisuke To: "song@kernel.org" , "yukuai3@huawei.com" CC: "linux-raid@vger.kernel.org" , "linux-kernel@vger.kernel.org" Subject: [PATCH 07/11] md: change the type of nr_pending from atomic_t to percpu_ref Thread-Topic: [PATCH 07/11] md: change the type of nr_pending from atomic_t to percpu_ref Thread-Index: Adp/ZHDlM7a9GiVuROS72GmNvZS6yA== Date: Tue, 26 Mar 2024 10:29:41 +0000 Message-ID: Accept-Language: ja-JP, en-US Content-Language: ja-JP X-MS-Has-Attach: X-MS-TNEF-Correlator: x-tm-as-product-ver: ISME-14.0.0.2080-9.0.1002-28274.000 x-tm-as-result: No-10--11.170200-8.000000 x-tmase-matchedrid: Sx6Z8sZ1/uymagT1k9kBpgrcxrzwsv5uXPK9y3z82GsPdZmyx8xAa4Ul JlxlTIT+uzW0mXOtDblgTs7PNu5YyogrKnsJ1GRgTuctSpiuWyUUi4Ehat05499RlPzeVuQQbo+ zo8QXL7dce6BqICgMNO++yHYM+maWCEeKzjvXhRNT3j6uyvUYPHO1oaQ51CuNyPRAwD/3abY9/h S2mXmf7hyQILeil6Pygimqy60vioomCg+Lak4DCfQxpA7auLwMsnK72MaPSqc4UXcfFIawgC4TI zFTm9BjfS0Ip2eEHnzWRN8STJpl3PoLR4+zsDTta4jJ6gfgIy2la73bcvT9XuBggTcklL0jRIHt QC2du/mubi7Hj6ORU1qe1Xx4uHiGxYVzI3UCCaY= x-tm-as-user-approved-sender: No x-tm-as-user-blocked-sender: No x-tmase-result: 10--11.170200-8.000000 x-tmase-version: ISME-14.0.0.2080-9.0.1002-28274.000 x-tm-snts-smtp: ED4621EFA559CECE17F037732FB8301002A8F9A433FA24094DEBA62C9E64F8112000:8 Precedence: bulk X-Mailing-List: linux-raid@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-CrossPremisesHeadersFilteredBySendConnector: CNN1EXMB01.r1.kioxia.com X-OrganizationHeadersPreserved: CNN1EXMB01.r1.kioxia.com This patch depends on patch 05. Change the type of nr_pending from atomic_t to percpu_ref. Additionally, temporarily switch from percpu mode to atomic mode. This is to reduce the amount of changes from patches 8 to 10. Patches 8 to 10 assume that nr_pending is run in percpu mode. Finally, switch from atomic mode to percpu mode in patch 11. Signed-off-by: Keisuke TADA Signed-off-by: Toshifumi OHTAKE --- drivers/md/md.c | 13 ++++++++++++- drivers/md/md.h | 18 +++++++++--------- 2 files changed, 21 insertions(+), 10 deletions(-) diff --git a/drivers/md/md.c b/drivers/md/md.c index df868b315b45..30fbba38ea58 100644 --- a/drivers/md/md.c +++ b/drivers/md/md.c @@ -998,6 +998,7 @@ void md_rdev_clear(struct md_rdev *rdev) put_page(rdev->bb_page); rdev->bb_page = NULL; } + percpu_ref_exit(&rdev->nr_pending); badblocks_exit(&rdev->badblocks); } EXPORT_SYMBOL_GPL(md_rdev_clear); @@ -3720,8 +3721,14 @@ static const struct kobj_type rdev_ktype = { .default_groups = rdev_default_groups, }; +static void percpu_wakeup_handle_req_pending(struct percpu_ref *ref) +{ +} + int md_rdev_init(struct md_rdev *rdev) { + int ret; + rdev->desc_nr = -1; rdev->saved_raid_disk = -1; rdev->raid_disk = -1; @@ -3732,7 +3739,11 @@ int md_rdev_init(struct md_rdev *rdev) rdev->last_read_error = 0; rdev->sb_loaded = 0; rdev->bb_page = NULL; - atomic_set(&rdev->nr_pending, 0); + ret = percpu_ref_init(&rdev->nr_pending, percpu_wakeup_handle_req_pending, + PERCPU_REF_ALLOW_REINIT, GFP_KERNEL); + WARN_ON(ret); + percpu_ref_switch_to_atomic_sync(&rdev->nr_pending); + nr_pending_dec(rdev); atomic_set(&rdev->read_errors, 0); atomic_set(&rdev->corrected_errors, 0); diff --git a/drivers/md/md.h b/drivers/md/md.h index 724439400ed1..c69e3a0bc4ca 100644 --- a/drivers/md/md.h +++ b/drivers/md/md.h @@ -107,10 +107,10 @@ struct md_rdev { */ }; - atomic_t nr_pending; /* number of pending requests. - * only maintained for arrays that - * support hot removal - */ + struct percpu_ref nr_pending; /* number of pending requests. + * only maintained for arrays that + * support hot removal + */ atomic_t read_errors; /* number of consecutive read errors that * we have tried to ignore. */ @@ -213,27 +213,27 @@ enum flag_bits { static inline void nr_pending_inc(struct md_rdev *rdev) { - atomic_inc(&rdev->nr_pending); + percpu_ref_get(&rdev->nr_pending); } static inline void nr_pending_dec(struct md_rdev *rdev) { - atomic_dec(&rdev->nr_pending); + percpu_ref_put(&rdev->nr_pending); } static inline bool nr_pending_is_zero(struct md_rdev *rdev) { - return atomic_read(&rdev->nr_pending) == 0; + return percpu_ref_is_zero(&rdev->nr_pending); } static inline bool nr_pending_is_not_zero(struct md_rdev *rdev) { - return atomic_read(&rdev->nr_pending) != 0; + return !percpu_ref_is_zero(&rdev->nr_pending); } static inline unsigned long nr_pending_read(struct md_rdev *rdev) { - return (unsigned long)atomic_read(&rdev->nr_pending); + return atomic_long_read(&rdev->nr_pending.data->count); } static inline int is_badblock(struct md_rdev *rdev, sector_t s, int sectors, From patchwork Tue Mar 26 10:29:44 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: tada keisuke X-Patchwork-Id: 13603867 Received: from mo-csw.securemx.jp (mo-csw1120.securemx.jp [210.130.202.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id EB5FE6BFBC; Tue, 26 Mar 2024 10:29:55 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=210.130.202.131 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1711448998; cv=none; b=GJDOE0c+stQ5wQDwsy5kUaN2MU9zicprwNwNtyqJmoC8h/LniEILEOJs5LO4Cj243BXsmXW2h3ZAxGZYXVe9Qu6u0OWjDBCgyDzJr21bIk6FTTBjAYfUv7MRWO8F8/G3Hln9tHV2b4YUUIuBxbqqFt/cabe4OR20+l5+No6k/So= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1711448998; c=relaxed/simple; bh=EUex2mfZ7Bm7MxE6kDtOUOglovPxx3drI3WWB33h0AQ=; h=From:To:CC:Subject:Date:Message-ID:Content-Type:MIME-Version; b=c2U2LjOSPvuZoOnG3amSGRGPfKR+A8eFeWaHoI3MkoitN5fc/Od+f8KhG9aEIHN+6f2UrvUSlC+gs++U4zv8fgOBOUGV4vZtOqvudgijo0PNHN3hreAiaBms60rYbZo3Ia+d0ozWSoYxnDwzvMZmBqMgC4KMiJ5s6StmBaUQWYw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=kioxia.com; spf=pass smtp.mailfrom=kioxia.com; arc=none smtp.client-ip=210.130.202.131 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=kioxia.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=kioxia.com Received: by mo-csw.securemx.jp (mx-mo-csw1120) id 42QATkOn2209605; Tue, 26 Mar 2024 19:29:46 +0900 X-Iguazu-Qid: 2rWgsWh2CpWREdGp19 X-Iguazu-QSIG: v=2; s=0; t=1711448985; q=2rWgsWh2CpWREdGp19; m=uexFo6Dzx4m6jZkL12YDgItO1rwd2NhgrNEo1Lwarlk= Received: from CNN1EMTA03.test.kioxia.com ([202.248.33.144]) by relay.securemx.jp (mx-mr1120) id 42QATiOa2928036 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NOT); Tue, 26 Mar 2024 19:29:45 +0900 Received: from Switcher-Post_Send (gateway [10.232.20.1]) by CNN1EMTA03.test.kioxia.com (Postfix) with ESMTP id D5AD532440; Tue, 26 Mar 2024 19:29:44 +0900 (JST) Received: from CNN1ESTR04.kioxia.com (localhost [127.0.0.1]) by Switcher-Post_Send (Postfix) with ESMTP id 9DB131900001E2; Tue, 26 Mar 2024 19:17:18 +0900 (JST) Received: from localhost [127.0.0.1] by CNN1ESTR04.kioxia.com with ESMTP id 0003VAAAAAA01M8E; Tue, 26 Mar 2024 19:17:18 +0900 Received: from CNN1EXMB02.r1.kioxia.com (CNN1EXMB02.r1.kioxia.com [10.232.20.151]) by Switcher-Pre_Send (Postfix) with ESMTP id 9277BA29B6838; Tue, 26 Mar 2024 19:17:18 +0900 (JST) Received: from CNN1EXMB03.r1.kioxia.com (10.232.20.152) by CNN1EXMB02.r1.kioxia.com (10.232.20.151) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Tue, 26 Mar 2024 19:29:44 +0900 Received: from CNN1EXMB03.r1.kioxia.com ([10.13.100.22]) by CNN1EXMB03.r1.kioxia.com ([10.13.100.22]) with mapi id 15.01.2507.035; Tue, 26 Mar 2024 19:29:44 +0900 From: tada keisuke To: "song@kernel.org" , "yukuai3@huawei.com" CC: "linux-raid@vger.kernel.org" , "linux-kernel@vger.kernel.org" Subject: [PATCH 08/11] md: add atomic mode switching in RAID 1/10 Thread-Topic: [PATCH 08/11] md: add atomic mode switching in RAID 1/10 Thread-Index: Adp/ZICXwOfGlnukSn+3jpdX/zMiPQ== Date: Tue, 26 Mar 2024 10:29:44 +0000 Message-ID: <9060d1a27d424db099b60f291555cd2c@kioxia.com> Accept-Language: ja-JP, en-US Content-Language: ja-JP X-MS-Has-Attach: X-MS-TNEF-Correlator: x-tm-as-product-ver: ISME-14.0.0.2080-9.0.1002-28274.000 x-tm-as-result: No-10--6.422500-8.000000 x-tmase-matchedrid: biL4YBBTMFamagT1k9kBppTQgFTHgkhZKQNhMboqZlrGZbX3v50WrzPn 4yDW5yf3V6Ny+iZr9hf2Rn4xkq0qEkDek9Bnh47Tuce7gFxhKa3BOVz0Jwcxl6vCrG0TnfVULPJ tWpbJjY391wFzAlGcPpNywEy6SR9HMX9mM1cyCSMjRwcsjqWGAvA5Umnv0zLsuqWf6Nh7tmEphT mCSWz6qUjuhwwkGBHrOR5CVdmNzBKLb+PIurKEXancG1vVsfbQM+pky3DTsZ2qagqQPeEJjJ4CI KY/Hg3AcmfM3DjaQLHEQdG7H66TyBU9TwbPjFWN6HLiu/iQZSQZOoCi/3UCsIlztRJHyQ1ROA1J CisWh+jAWfSqfUCswndMV9iIuCaQ x-tm-as-user-approved-sender: No x-tm-as-user-blocked-sender: No x-tmase-result: 10--6.422500-8.000000 x-tmase-version: ISME-14.0.0.2080-9.0.1002-28274.000 x-tm-snts-smtp: 749F8F7BD728F02F15D30D7A42B633AE9604D3D59524E77EA174EED40124F0DA2000:8 Precedence: bulk X-Mailing-List: linux-raid@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-CrossPremisesHeadersFilteredBySendConnector: CNN1EXMB02.r1.kioxia.com X-OrganizationHeadersPreserved: CNN1EXMB02.r1.kioxia.com This patch depends on patch 07. All rdevs running in RAID 1/10 switch nr_pending to atomic mode. The value of nr_pending is read in a normal operation (read_balance()). Therefore, nr_pending must always be consistent. Signed-off-by: Keisuke TADA Signed-off-by: Toshifumi OHTAKE --- drivers/md/md.h | 14 ++++++++++++++ drivers/md/raid1.c | 7 +++++++ drivers/md/raid10.c | 4 ++++ 3 files changed, 25 insertions(+) diff --git a/drivers/md/md.h b/drivers/md/md.h index c69e3a0bc4ca..ee84c4b5ee87 100644 --- a/drivers/md/md.h +++ b/drivers/md/md.h @@ -236,6 +236,20 @@ static inline unsigned long nr_pending_read(struct md_rdev *rdev) return atomic_long_read(&rdev->nr_pending.data->count); } +static inline bool nr_pending_is_percpu_mode(struct md_rdev *rdev) +{ + unsigned long __percpu *percpu_count; + + return __ref_is_percpu(&rdev->nr_pending, &percpu_count); +} + +static inline bool nr_pending_is_atomic_mode(struct md_rdev *rdev) +{ + unsigned long __percpu *percpu_count; + + return !__ref_is_percpu(&rdev->nr_pending, &percpu_count); +} + static inline int is_badblock(struct md_rdev *rdev, sector_t s, int sectors, sector_t *first_bad, int *bad_sectors) { diff --git a/drivers/md/raid1.c b/drivers/md/raid1.c index 43cc36fec7a2..146eda6c4961 100644 --- a/drivers/md/raid1.c +++ b/drivers/md/raid1.c @@ -784,6 +784,7 @@ static int choose_best_rdev(struct r1conf *conf, struct r1bio *r1_bio) if (ctl.readable_disks++ == 1) set_bit(R1BIO_FailFast, &r1_bio->state); + WARN_ON_ONCE(nr_pending_is_percpu_mode(rdev)); pending = nr_pending_read(rdev); dist = abs(r1_bio->sector - conf->mirrors[disk].head_position); @@ -1930,6 +1931,7 @@ static int raid1_add_disk(struct mddev *mddev, struct md_rdev *rdev) if (err) return err; + percpu_ref_switch_to_atomic_sync(&rdev->nr_pending); raid1_add_conf(conf, rdev, mirror, false); /* As all devices are equivalent, we don't need a full recovery * if this was recently any drive of the array @@ -1949,6 +1951,7 @@ static int raid1_add_disk(struct mddev *mddev, struct md_rdev *rdev) set_bit(Replacement, &rdev->flags); raid1_add_conf(conf, rdev, repl_slot, true); err = 0; + percpu_ref_switch_to_atomic_sync(&rdev->nr_pending); conf->fullsync = 1; } @@ -3208,6 +3211,7 @@ static void raid1_free(struct mddev *mddev, void *priv); static int raid1_run(struct mddev *mddev) { struct r1conf *conf; + struct md_rdev *rdev; int i; int ret; @@ -3269,6 +3273,9 @@ static int raid1_run(struct mddev *mddev) /* * Ok, everything is just fine now */ + rdev_for_each(rdev, mddev) { + percpu_ref_switch_to_atomic_sync(&rdev->nr_pending); + } rcu_assign_pointer(mddev->thread, conf->thread); rcu_assign_pointer(conf->thread, NULL); mddev->private = conf; diff --git a/drivers/md/raid10.c b/drivers/md/raid10.c index cd5094c4df50..eebbf9185deb 100644 --- a/drivers/md/raid10.c +++ b/drivers/md/raid10.c @@ -808,6 +808,7 @@ static struct md_rdev *read_balance(struct r10conf *conf, nonrot = bdev_nonrot(rdev->bdev); has_nonrot_disk |= nonrot; + WARN_ON_ONCE(nr_pending_is_percpu_mode(rdev)); pending = nr_pending_read(rdev); if (min_pending > pending && nonrot) { min_pending = pending; @@ -2113,6 +2114,7 @@ static int raid10_add_disk(struct mddev *mddev, struct md_rdev *rdev) p->recovery_disabled = mddev->recovery_disabled - 1; rdev->raid_disk = mirror; err = 0; + percpu_ref_switch_to_atomic_sync(&rdev->nr_pending); if (rdev->saved_raid_disk != mirror) conf->fullsync = 1; WRITE_ONCE(p->rdev, rdev); @@ -2127,6 +2129,7 @@ static int raid10_add_disk(struct mddev *mddev, struct md_rdev *rdev) err = mddev_stack_new_rdev(mddev, rdev); if (err) return err; + percpu_ref_switch_to_atomic_sync(&rdev->nr_pending); conf->fullsync = 1; WRITE_ONCE(p->replacement, rdev); } @@ -4028,6 +4031,7 @@ static int raid10_run(struct mddev *mddev) rdev_for_each(rdev, mddev) { long long diff; + percpu_ref_switch_to_atomic_sync(&rdev->nr_pending); disk_idx = rdev->raid_disk; if (disk_idx < 0) continue; From patchwork Tue Mar 26 10:29:47 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: tada keisuke X-Patchwork-Id: 13603868 Received: from mo-csw.securemx.jp (mo-csw1121.securemx.jp [210.130.202.132]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1E5F080C18; Tue, 26 Mar 2024 10:29:59 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=210.130.202.132 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1711449001; cv=none; b=tzcqo+H9cn34+JTYz79kqihT6EuIHAU9XBiQTewnlOxwZNWpwFiBOiMdJwPliAXlczRNE7F1kIfFYIN3fAR+zMshptsjPM4+Ob76eE8285NmreLnXgIeUQkkv0nTzpTgFiKkCNNmY/8tPd8mBR4QjewDuaB/pnRxlk7QLGspkIg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1711449001; c=relaxed/simple; bh=4QZLmSbm1MLXhr0qJmXOhyAfwyRRkmgsv34pized0xw=; h=From:To:CC:Subject:Date:Message-ID:Content-Type:MIME-Version; b=F663h/EKqV4kv2BL9NrdTMzgvU23VscWrbyJddcJBTtiUw/II5Mx++bD9NM1jqMRvD2MPfUd2OoRJ62xCNVwi6Bvwf2s00CHTvWgCr6UJJ5KHq5TcKy8+TOFlRgpN/fRCfi3pLWyRrTtHkoE0Q3qIN5vAeR327lKbEoCiJmwUew= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=kioxia.com; spf=pass smtp.mailfrom=kioxia.com; arc=none smtp.client-ip=210.130.202.132 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=kioxia.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=kioxia.com Received: by mo-csw.securemx.jp (mx-mo-csw1121) id 42QATnt32378260; Tue, 26 Mar 2024 19:29:49 +0900 X-Iguazu-Qid: 2rWh11C4gmtKNHpm0e X-Iguazu-QSIG: v=2; s=0; t=1711448988; q=2rWh11C4gmtKNHpm0e; m=wHAkRlTPynDlSN95zkryN8yupJQcpu7OLqPlrTvtLmA= Received: from CNN1EMTA02.test.kioxia.com ([202.248.33.144]) by relay.securemx.jp (mx-mr1121) id 42QATlth2222415 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NOT); Tue, 26 Mar 2024 19:29:48 +0900 Received: from Switcher-Post_Send (gateway [10.232.20.1]) by CNN1EMTA02.test.kioxia.com (Postfix) with ESMTP id D9D6C2F028; Tue, 26 Mar 2024 19:29:47 +0900 (JST) Received: from CNN1ESTR03.kioxia.com (localhost [127.0.0.1]) by Switcher-Post_Send (Postfix) with ESMTP id 5E8261938D7224; Tue, 26 Mar 2024 19:08:21 +0900 (JST) Received: from localhost [127.0.0.1] by CNN1ESTR03.kioxia.com with ESMTP id 0002VAAAAAA012HK; Tue, 26 Mar 2024 19:08:21 +0900 Received: from CNN1EXMB03.r1.kioxia.com (CNN1EXMB03.r1.kioxia.com [10.232.20.152]) by Switcher-Pre_Send (Postfix) with ESMTP id 532FCA035DC28; Tue, 26 Mar 2024 19:08:21 +0900 (JST) Received: from CNN1EXMB03.r1.kioxia.com (10.232.20.152) by CNN1EXMB03.r1.kioxia.com (10.232.20.152) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Tue, 26 Mar 2024 19:29:47 +0900 Received: from CNN1EXMB03.r1.kioxia.com ([10.13.100.22]) by CNN1EXMB03.r1.kioxia.com ([10.13.100.22]) with mapi id 15.01.2507.035; Tue, 26 Mar 2024 19:29:47 +0900 From: tada keisuke To: "song@kernel.org" , "yukuai3@huawei.com" CC: "linux-raid@vger.kernel.org" , "linux-kernel@vger.kernel.org" Subject: [PATCH 09/11] md: add atomic mode switching when removing disk Thread-Topic: [PATCH 09/11] md: add atomic mode switching when removing disk Thread-Index: Adp/ZIq7EnQH8Y/+QBWXhs+ct+FbWA== Date: Tue, 26 Mar 2024 10:29:47 +0000 Message-ID: Accept-Language: ja-JP, en-US Content-Language: ja-JP X-MS-Has-Attach: X-MS-TNEF-Correlator: x-tm-as-product-ver: ISME-14.0.0.2080-9.0.1002-28274.000 x-tm-as-result: No-10--5.650100-8.000000 x-tmase-matchedrid: 5V0fm+sVFB2magT1k9kBppTQgFTHgkhZKQNhMboqZlrGZbX3v50Wrzos Lf1kH5Cf9KYkbDev+bE52p2V9wXr31gyUUCO8mz8gnMtC97jHVTBOVz0Jwcxl6vCrG0TnfVUg9x e4gtUJtop7xmDsiWI7eSkRJ+6d2nBY+Xr17aLI9mkkQPcJHKMT+ovpkXHOUFXv1l2Uvx6idqu65 UDD0aDgsRB0bsfrpPIFT1PBs+MVY3ocuK7+JBlJKynSBuJdXzfLdINTQrDOxQdkq5G9RbsCnc6Y nBhjO9btRkNxJ2JyUw= x-tm-as-user-approved-sender: No x-tm-as-user-blocked-sender: No x-tmase-result: 10--5.650100-8.000000 x-tmase-version: ISME-14.0.0.2080-9.0.1002-28274.000 x-tm-snts-smtp: C84AFBBFA731F091C7A1F930C85F5F89732CE1C1BAF081C83AA8472DD88FC65F2000:8 Precedence: bulk X-Mailing-List: linux-raid@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-CrossPremisesHeadersFilteredBySendConnector: CNN1EXMB03.r1.kioxia.com X-OrganizationHeadersPreserved: CNN1EXMB03.r1.kioxia.com This patch depends on patch 08. To minimize the number of execution of atomic mode switching, not only adding atomic mode switching, but also remove judgment is divided into two stages. Latency is minimized because only rdev to be removed is switched to atomic mode, not all rdevs. Signed-off-by: Keisuke TADA Signed-off-by: Toshifumi OHTAKE --- drivers/md/md.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/drivers/md/md.c b/drivers/md/md.c index 30fbba38ea58..8c0c48a3a585 100644 --- a/drivers/md/md.c +++ b/drivers/md/md.c @@ -9277,8 +9277,10 @@ static bool rdev_removeable(struct md_rdev *rdev) return false; /* Fautly rdev is not used, it's safe to remove it. */ - if (test_bit(Faulty, &rdev->flags)) + if (test_bit(Faulty, &rdev->flags)) { + percpu_ref_switch_to_atomic_sync(&rdev->nr_pending); return true; + } /* Journal disk can only be removed if it's faulty. */ if (test_bit(Journal, &rdev->flags)) From patchwork Tue Mar 26 10:29:50 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: tada keisuke X-Patchwork-Id: 13603869 Received: from mo-csw.securemx.jp (mo-csw1120.securemx.jp [210.130.202.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id F0C8B5C90B; Tue, 26 Mar 2024 10:30:00 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=210.130.202.131 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1711449002; cv=none; b=H/IGWutCypr92MPaGrcNLvdTjcS4SRdCcgePK9RmgADhpqnBxMuLqwSh3f+J3GeExwJgppBwE5FNruLRmCBUSykWO2Ub9yOiWVrAhf7UApzwHfalb65xS9wwhEYaycbVMobED/5XRRXzfv7MIyLYsQZyp33wBgnkxYXGhkWEiuU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1711449002; c=relaxed/simple; bh=M0BjFWay9RW56DcTyI8VhWYWKwj3d1u/51hugxS02FI=; h=From:To:CC:Subject:Date:Message-ID:Content-Type:MIME-Version; b=pfXebLl8pmUIPAL7Xra5mbhBT1qy0aEe5c0105x6IgANNZ+xsq75XSMGJwOlzF6t8D9EA/9td7miFT1fs/WHbwaWpSjdBPTeTtA/X3sCELKheWCjDMgtSZ7A+PFtykhqUnSoI6w6M3NmMOc4pYgXFQQ9s6ErBGqi/67ZjEr/MT8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=kioxia.com; spf=pass smtp.mailfrom=kioxia.com; arc=none smtp.client-ip=210.130.202.131 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=kioxia.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=kioxia.com Received: by mo-csw.securemx.jp (mx-mo-csw1120) id 42QATq9K2209751; Tue, 26 Mar 2024 19:29:52 +0900 X-Iguazu-Qid: 2rWg9o4uinxIZo6huv X-Iguazu-QSIG: v=2; s=0; t=1711448991; q=2rWg9o4uinxIZo6huv; m=8Tw/BXHkkoj6j3cmE/J7d5JyB0KYQsLU2Fgyl+o/4Zw= Received: from CNN1EMTA03.test.kioxia.com ([202.248.33.144]) by relay.securemx.jp (mx-mr1120) id 42QATpWK2928151 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NOT); Tue, 26 Mar 2024 19:29:51 +0900 Received: from Switcher-Post_Send (gateway [10.232.20.1]) by CNN1EMTA03.test.kioxia.com (Postfix) with ESMTP id 066B432440; Tue, 26 Mar 2024 19:29:51 +0900 (JST) Received: from CNN1ESTR03.kioxia.com (localhost [127.0.0.1]) by Switcher-Post_Send (Postfix) with ESMTP id 77A981938D7224; Tue, 26 Mar 2024 19:08:24 +0900 (JST) Received: from localhost [127.0.0.1] by CNN1ESTR03.kioxia.com with ESMTP id 0002VAAAAAA012JI; Tue, 26 Mar 2024 19:08:24 +0900 Received: from CNN1EXMB02.r1.kioxia.com (CNN1EXMB02.r1.kioxia.com [10.232.20.151]) by Switcher-Pre_Send (Postfix) with ESMTP id 6C44DA035DC28; Tue, 26 Mar 2024 19:08:24 +0900 (JST) Received: from CNN1EXMB03.r1.kioxia.com (10.232.20.152) by CNN1EXMB02.r1.kioxia.com (10.232.20.151) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Tue, 26 Mar 2024 19:29:50 +0900 Received: from CNN1EXMB03.r1.kioxia.com ([10.13.100.22]) by CNN1EXMB03.r1.kioxia.com ([10.13.100.22]) with mapi id 15.01.2507.035; Tue, 26 Mar 2024 19:29:50 +0900 From: tada keisuke To: "song@kernel.org" , "yukuai3@huawei.com" CC: "linux-raid@vger.kernel.org" , "linux-kernel@vger.kernel.org" Subject: [PATCH 10/11] md: add atomic mode switching when I/O completion Thread-Topic: [PATCH 10/11] md: add atomic mode switching when I/O completion Thread-Index: Adp/ZJXld54bLap0S/C/8U6V7u3sKA== Date: Tue, 26 Mar 2024 10:29:50 +0000 Message-ID: <2b9cfb893d5b40478182a8f93d0463a8@kioxia.com> Accept-Language: ja-JP, en-US Content-Language: ja-JP X-MS-Has-Attach: X-MS-TNEF-Correlator: x-tm-as-product-ver: ISME-14.0.0.2080-9.0.1002-28274.000 x-tm-as-result: No-10--8.850000-8.000000 x-tmase-matchedrid: WBjkmIX+tHCmagT1k9kBppTQgFTHgkhZKQNhMboqZlrGZbX3v50WrzPn 4yDW5yf3GSNOWaKP4s2t2un08Qrrt3EWVXH5btzQTuctSpiuWyUUi4Ehat054zdlsYL2g/87GyQ B1oR95AqsIJcbcumfxGPKQgbIX4yuhcAT8+NWWH8jRwcsjqWGAh90OsLQe4vlyIKHzIGoT62k16 eVRb8rL5dhlAAXYT7M7YL9OFIRb909S7tGIPO8TQ55FEYgJX09SKtHZwIquEv+Aw16GgqpO+m57 3tusZjQTqumDQCn8tgrsSQ3gQbaomwp3fcRm9P4HWBogl1oa71ftuJwrFEhTY2j49Ftap9Eymsk /wUE4hq54HgM8GZx2SMFB0oG7nHS31EhHiJb6BsE+431M+6R1X3P4jWY9z7S59+FXOE8fLM= x-tm-as-user-approved-sender: No x-tm-as-user-blocked-sender: No x-tmase-result: 10--8.850000-8.000000 x-tmase-version: ISME-14.0.0.2080-9.0.1002-28274.000 x-tm-snts-smtp: 72FE3221312B6410F6B4AE800173FE40C97901191C27C1544272DB2841FBE76B2000:8 Precedence: bulk X-Mailing-List: linux-raid@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-CrossPremisesHeadersFilteredBySendConnector: CNN1EXMB02.r1.kioxia.com X-OrganizationHeadersPreserved: CNN1EXMB02.r1.kioxia.com This patch depends on patch 09. If percpu mode in softirq context, switch to atomic mode in delayed execution. Move from softirq context to a context where RCUs are available and switch to atomic mode. This patch completes the addition of atomic mode switching. Signed-off-by: Keisuke TADA Signed-off-by: Toshifumi OHTAKE --- drivers/md/md.c | 16 ++++++++++++++++ drivers/md/md.h | 6 ++++++ 2 files changed, 22 insertions(+) diff --git a/drivers/md/md.c b/drivers/md/md.c index 8c0c48a3a585..1c7b774dbb48 100644 --- a/drivers/md/md.c +++ b/drivers/md/md.c @@ -9566,6 +9566,7 @@ void md_check_recovery(struct mddev *mddev) return; if ( ! ( (mddev->sb_flags & ~ (1<recovery) || test_bit(MD_RECOVERY_NEEDED, &mddev->recovery) || test_bit(MD_RECOVERY_DONE, &mddev->recovery) || (mddev->external == 0 && mddev->safemode == 1) || @@ -9576,6 +9577,21 @@ void md_check_recovery(struct mddev *mddev) if (mddev_trylock(mddev)) { bool try_set_sync = mddev->safemode != 0; + struct md_rdev *fault_rdev; + bool faulty_some = false; + + rdev_for_each(fault_rdev, mddev) { + if (fault_rdev->raid_disk >= 0 && + test_bit(Faulty, &fault_rdev->flags) && + nr_pending_is_percpu_mode(fault_rdev)) { + percpu_ref_switch_to_atomic_sync(&fault_rdev->nr_pending); + faulty_some = true; + } + } + if (faulty_some) { + clear_bit(MD_RECOVERY_PERCPU, &mddev->recovery); + goto unlock; + } if (!mddev->external && mddev->safemode == 1) mddev->safemode = 0; diff --git a/drivers/md/md.h b/drivers/md/md.h index ee84c4b5ee87..15e10205b578 100644 --- a/drivers/md/md.h +++ b/drivers/md/md.h @@ -605,6 +605,7 @@ enum recovery_flags { MD_RECOVERY_FROZEN, /* User request to abort, and not restart, any action */ MD_RECOVERY_ERROR, /* sync-action interrupted because io-error */ MD_RECOVERY_WAIT, /* waiting for pers->start() to finish */ + MD_RECOVERY_PERCPU, /* nr_pending when faulty needs to be switched to atomic */ MD_RESYNCING_REMOTE, /* remote node is running resync thread */ }; @@ -886,6 +887,11 @@ static inline void rdev_dec_pending(struct md_rdev *rdev, struct mddev *mddev) int faulty = test_bit(Faulty, &rdev->flags); nr_pending_dec(rdev); if (faulty) { + if (nr_pending_is_percpu_mode(rdev)) { + set_bit(MD_RECOVERY_PERCPU, &mddev->recovery); + md_wakeup_thread(mddev->thread); + return; + } if (nr_pending_is_zero(rdev)) { set_bit(MD_RECOVERY_NEEDED, &mddev->recovery); md_wakeup_thread(mddev->thread); From patchwork Tue Mar 26 10:29:53 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: tada keisuke X-Patchwork-Id: 13603870 Received: from mo-csw.securemx.jp (mo-csw1121.securemx.jp [210.130.202.132]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0A8FD8286F; Tue, 26 Mar 2024 10:30:02 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=210.130.202.132 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1711449004; cv=none; b=n4iSljv8Gccm2zxUK5JpXHc61jJ3XcZRPL9hIed6P7Cj+Z7ZF56qclEE6eFscF2b9SMCm7t42dDkE0UTImOnmQ0Xoufa5T5GwY/FAIK+yckhCpMdKE9jDzZCFUkYYGxq4DYQzI7L58O8E2Pi99VQZVQg96bQDvmWixHADdu/Xx0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1711449004; c=relaxed/simple; bh=v2jL1hy4xGTvomfW5Zhh9PpQnxZHZB2SqnKYDTHsHco=; h=From:To:CC:Subject:Date:Message-ID:Content-Type:MIME-Version; b=XSTxa5WKvwIMB/6U9Z6TVBL2ROqEaZJ/oLbPV9B9NbvyqQDTZKZkio16vEEEqgq2eE/HGEQ3IJi0Ct2p/ZZGN2XIf9IPUYT1ZucsYGRGW4QBMKQwRgBaNffIBhNLHOA9Q+BgEH9Kh5WVdFArvMyesXGvH5hdzWDlNphpbYgz2kw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=kioxia.com; spf=pass smtp.mailfrom=kioxia.com; arc=none smtp.client-ip=210.130.202.132 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=kioxia.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=kioxia.com Received: by mo-csw.securemx.jp (mx-mo-csw1121) id 42QATt6i2378481; Tue, 26 Mar 2024 19:29:55 +0900 X-Iguazu-Qid: 2rWh11C4gmtKu6UUN6 X-Iguazu-QSIG: v=2; s=0; t=1711448995; q=2rWh11C4gmtKu6UUN6; m=NZKyFdp6DPl6ZkM+TxeUnhEP8FgEa1YAjoiLGCcwQkY= Received: from CNN1EMTA02.test.kioxia.com ([202.248.33.144]) by relay.securemx.jp (mx-mr1120) id 42QATsUr2928218 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NOT); Tue, 26 Mar 2024 19:29:54 +0900 Received: from Switcher-Post_Send (gateway [10.232.20.1]) by CNN1EMTA02.test.kioxia.com (Postfix) with ESMTP id 736292F028; Tue, 26 Mar 2024 19:29:54 +0900 (JST) Received: from CNN1ESTR03.kioxia.com (localhost [127.0.0.1]) by Switcher-Post_Send (Postfix) with ESMTP id E4EF81938D7224; Tue, 26 Mar 2024 19:08:27 +0900 (JST) Received: from localhost [127.0.0.1] by CNN1ESTR03.kioxia.com with ESMTP id 0002VAAAAAA012LG; Tue, 26 Mar 2024 19:08:27 +0900 Received: from CNN1EXMB04.r1.kioxia.com (CNN1EXMB04.r1.kioxia.com [10.232.20.153]) by Switcher-Pre_Send (Postfix) with ESMTP id D9CBDA035DC28; Tue, 26 Mar 2024 19:08:27 +0900 (JST) Received: from CNN1EXMB03.r1.kioxia.com (10.232.20.152) by CNN1EXMB04.r1.kioxia.com (10.232.20.153) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Tue, 26 Mar 2024 19:29:53 +0900 Received: from CNN1EXMB03.r1.kioxia.com ([10.13.100.22]) by CNN1EXMB03.r1.kioxia.com ([10.13.100.22]) with mapi id 15.01.2507.035; Tue, 26 Mar 2024 19:29:53 +0900 From: tada keisuke To: "song@kernel.org" , "yukuai3@huawei.com" CC: "linux-raid@vger.kernel.org" , "linux-kernel@vger.kernel.org" Subject: [PATCH 11/11] md: avoid conflicts in nr_pending operations Thread-Topic: [PATCH 11/11] md: avoid conflicts in nr_pending operations Thread-Index: Adp/ZKBKW6uU4n/lRH2pDrm0xZHe5Q== Date: Tue, 26 Mar 2024 10:29:53 +0000 Message-ID: <22c6696bf0594cd69cad3a2002695402@kioxia.com> Accept-Language: ja-JP, en-US Content-Language: ja-JP X-MS-Has-Attach: X-MS-TNEF-Correlator: x-tm-as-product-ver: ISME-14.0.0.2080-9.0.1002-28274.000 x-tm-as-result: No-10--9.404500-8.000000 x-tmase-matchedrid: k7uoLACGqMqmagT1k9kBpgrcxrzwsv5uXPK9y3z82GsPdZmyx8xAa4Ul JlxlTIT+tPwtczFrCbS12HagvbwDji/7QU2czuUNA9lly13c/gFezmeoa8MJ86G5P8xDq4BXoK5 B6+qXU7BvF9oQ/B54yN5WE7eMYlPKhrD3pNcSx1aqFx2c/3V5cbs5i+al07BJUrMN6XtW1EtvXF TtvgJmGhSu3n/oQBOZmsmH6sEd2eHyAV/ow31xqxMxKDqgAFSzuoYFb0nRiqNULhp+W/a9mTiRi c6D7updJciLWd+sUhmbKItl61J/ySKveQ4wmYdMijciq9xw8kXCttcwYNipX6pqCpA94QmM0iSE K8EapKcB2Iz/+kAhduXu7VHCH84vZipFGUz6bng9AUz4NAIVDw== x-tm-as-user-approved-sender: No x-tm-as-user-blocked-sender: No x-tmase-result: 10--9.404500-8.000000 x-tmase-version: ISME-14.0.0.2080-9.0.1002-28274.000 x-tm-snts-smtp: 3DD78E085F0BFC6C32DCF8BE2822A0836075F333BA941AC969AEB0C3F5A3F29B2000:8 Precedence: bulk X-Mailing-List: linux-raid@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-CrossPremisesHeadersFilteredBySendConnector: CNN1EXMB04.r1.kioxia.com X-OrganizationHeadersPreserved: CNN1EXMB04.r1.kioxia.com This patch depends on patch 10. Change nr_pending from atomic mode to percpu mode. Signed-off-by: Keisuke TADA Signed-off-by: Toshifumi OHTAKE --- drivers/md/md.c | 5 +++-- drivers/md/raid1.c | 6 ++++++ drivers/md/raid10.c | 6 ++++++ 3 files changed, 15 insertions(+), 2 deletions(-) diff --git a/drivers/md/md.c b/drivers/md/md.c index 1c7b774dbb48..7cbf199bd168 100644 --- a/drivers/md/md.c +++ b/drivers/md/md.c @@ -3742,7 +3742,6 @@ int md_rdev_init(struct md_rdev *rdev) ret = percpu_ref_init(&rdev->nr_pending, percpu_wakeup_handle_req_pending, PERCPU_REF_ALLOW_REINIT, GFP_KERNEL); WARN_ON(ret); - percpu_ref_switch_to_atomic_sync(&rdev->nr_pending); nr_pending_dec(rdev); atomic_set(&rdev->read_errors, 0); atomic_set(&rdev->corrected_errors, 0); @@ -9266,7 +9265,7 @@ static bool rdev_removeable(struct md_rdev *rdev) return false; /* There are still inflight io, don't remove this rdev. */ - if (nr_pending_is_not_zero(rdev)) + if (nr_pending_is_atomic_mode(rdev) && nr_pending_is_not_zero(rdev)) return false; /* @@ -9364,6 +9363,8 @@ static int remove_and_add_spares(struct mddev *mddev, rdev->raid_disk = -1; removed++; } + if (mddev->pers->level != 1 && mddev->pers->level != 10) + percpu_ref_switch_to_percpu(&rdev->nr_pending); } if (removed && mddev->kobj.sd) diff --git a/drivers/md/raid1.c b/drivers/md/raid1.c index 146eda6c4961..dd721b9c19b9 100644 --- a/drivers/md/raid1.c +++ b/drivers/md/raid1.c @@ -2006,6 +2006,7 @@ static int raid1_remove_disk(struct mddev *mddev, struct md_rdev *rdev) conf->mirrors[conf->raid_disks + number].rdev = NULL; unfreeze_array(conf); } + percpu_ref_switch_to_percpu(&rdev->nr_pending); clear_bit(WantReplacement, &rdev->flags); err = md_integrity_register(mddev); @@ -3298,6 +3299,11 @@ static int raid1_run(struct mddev *mddev) static void raid1_free(struct mddev *mddev, void *priv) { struct r1conf *conf = priv; + struct md_rdev *rdev; + + rdev_for_each(rdev, mddev) { + percpu_ref_switch_to_percpu(&rdev->nr_pending); + } mempool_exit(&conf->r1bio_pool); kfree(conf->mirrors); diff --git a/drivers/md/raid10.c b/drivers/md/raid10.c index eebbf9185deb..e285481464b9 100644 --- a/drivers/md/raid10.c +++ b/drivers/md/raid10.c @@ -2180,6 +2180,7 @@ static int raid10_remove_disk(struct mddev *mddev, struct md_rdev *rdev) clear_bit(Replacement, &p->replacement->flags); WRITE_ONCE(p->replacement, NULL); } + percpu_ref_switch_to_percpu(&rdev->nr_pending); clear_bit(WantReplacement, &rdev->flags); err = md_integrity_register(mddev); @@ -4172,6 +4173,11 @@ static int raid10_run(struct mddev *mddev) static void raid10_free(struct mddev *mddev, void *priv) { + struct md_rdev *rdev; + + rdev_for_each(rdev, mddev) { + percpu_ref_switch_to_percpu(&rdev->nr_pending); + } raid10_free_conf(priv); }