From patchwork Mon Nov 18 11:41:52 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Yu Kuai X-Patchwork-Id: 13878459 Received: from dggsgout12.his.huawei.com (dggsgout12.his.huawei.com [45.249.212.56]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B2F21194A63; Mon, 18 Nov 2024 11:44:16 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=45.249.212.56 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731930260; cv=none; b=mky4ei1Ag8VelKBUUr6GPMlRT56Eus25tCU4X03ESLUWp6DSuziBVga92ZL0/z3/NUBONCce3VOnt4beqPUVaUbWtlTBe40Zny3Kazg62aadXP7xxV6UVlYo8HqEBxGWhXMQyS71jTsp1h2ViWNhgNxfauOCRBsXEYIcR/UihKw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731930260; c=relaxed/simple; bh=j+L80QTVaLAsIQmV5Hx5cn3KWXUNzn+m2TCxIwARhgw=; h=From:To:Cc:Subject:Date:Message-Id:MIME-Version:Content-Type; b=D0CaG1UwjQvZl81PZBZ3WiwCb5flK9SLoyouQYivOfJrVt1K3kqduL2V7XrJ5bcdMU4uYV6qtsNggoEjBzAomAvEvLbDJUg1D/TcrQNCxAML66B1I5tSH8NlWS6USUGfqcoUbG/2n1E5roCpEE/HmpOGDnw/ZCrvrLF0Lrd6Jr8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=huaweicloud.com; spf=pass smtp.mailfrom=huaweicloud.com; arc=none smtp.client-ip=45.249.212.56 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=huaweicloud.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=huaweicloud.com Received: from mail.maildlp.com (unknown [172.19.93.142]) by dggsgout12.his.huawei.com (SkyGuard) with ESMTP id 4XsQkf40K5z4f3jJ5; Mon, 18 Nov 2024 19:43:54 +0800 (CST) Received: from mail02.huawei.com (unknown [10.116.40.128]) by mail.maildlp.com (Postfix) with ESMTP id 029FB1A0359; Mon, 18 Nov 2024 19:44:13 +0800 (CST) Received: from huaweicloud.com (unknown [10.175.104.67]) by APP4 (Coremail) with SMTP id gCh0CgBHI4eKKDtn_9+KCA--.2699S4; Mon, 18 Nov 2024 19:44:12 +0800 (CST) From: Yu Kuai To: song@kernel.org, xni@redhat.com Cc: linux-raid@vger.kernel.org, linux-kernel@vger.kernel.org, yukuai3@huawei.com, yukuai1@huaweicloud.com, yi.zhang@huawei.com, yangerkun@huawei.com Subject: [PATCH md-6.13 0/5] md/md-bitmap: move bitmap_{start,end}write to md upper layer Date: Mon, 18 Nov 2024 19:41:52 +0800 Message-Id: <20241118114157.355749-1-yukuai1@huaweicloud.com> X-Mailer: git-send-email 2.39.2 Precedence: bulk X-Mailing-List: linux-raid@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-CM-TRANSID: gCh0CgBHI4eKKDtn_9+KCA--.2699S4 X-Coremail-Antispam: 1UD129KBjvJXoWxZrW8XF1UCr4kZrW3Xr4ruFg_yoW5Kw1xpF 47Jry5Jw18Gr47Jr47Jr1UJryrJr18Jr1UXr17Jr1UXw1UJw4UtrnrJr1UJr1DJr1UCr1U Jr1UAr1UGr1UJwUanT9S1TB71UUUUU7qnTZGkaVYY2UrUUUUjbIjqfuFe4nvWSU5nxnvy2 9KBjDU0xBIdaVrnRJUUUkG14x267AKxVW8JVW5JwAFc2x0x2IEx4CE42xK8VAvwI8IcIk0 rVWrJVCq3wAFIxvE14AKwVWUJVWUGwA2ocxC64kIII0Yj41l84x0c7CEw4AK67xGY2AK02 1l84ACjcxK6xIIjxv20xvE14v26F1j6w1UM28EF7xvwVC0I7IYx2IY6xkF7I0E14v26F4j 6r4UJwA2z4x0Y4vEx4A2jsIE14v26rxl6s0DM28EF7xvwVC2z280aVCY1x0267AKxVW0oV Cq3wAS0I0E0xvYzxvE52x082IY62kv0487Mc02F40EFcxC0VAKzVAqx4xG6I80ewAv7VC0 I7IYx2IY67AKxVWUJVWUGwAv7VC2z280aVAFwI0_Jr0_Gr1lOx8S6xCaFVCjc4AY6r1j6r 4UM4x0Y48IcxkI7VAKI48JM4x0x7Aq67IIx4CEVc8vx2IErcIFxwCY1x0262kKe7AKxVWU AVWUtwCF04k20xvY0x0EwIxGrwCFx2IqxVCFs4IE7xkEbVWUJVW8JwC20s026c02F40E14 v26r1j6r18MI8I3I0E7480Y4vE14v26r106r1rMI8E67AF67kF1VAFwI0_Jw0_GFylIxkG c2Ij64vIr41lIxAIcVC0I7IYx2IY67AKxVWUJVWUCwCI42IY6xIIjxv20xvEc7CjxVAFwI 0_Jr0_Gr1lIxAIcVCF04k26cxKx2IYs7xG6r1j6r1xMIIF0xvEx4A2jsIE14v26r1j6r4U MIIF0xvEx4A2jsIEc7CjxVAFwI0_Gr0_Gr1UYxBIdaVFxhVjvjDU0xZFpf9x0JUBVbkUUU UU= X-CM-SenderInfo: 51xn3trlr6x35dzhxuhorxvhhfrp/ From: Yu Kuai While reviewing raid5 code, it's found that bitmap operations can be optimized. For example, for a 4 disks raid5, with chunksize=8k, if user issue a IO (0 + 48k) to the array: ┌────────────────────────────────────────────────────────────┐ │chunk 0 │ │ ┌────────────┬─────────────┬─────────────┬────────────┼ │ sh0 │A0: 0 + 4k │A1: 8k + 4k │A2: 16k + 4k │A3: P │ │ ┼────────────┼─────────────┼─────────────┼────────────┼ │ sh1 │B0: 4k + 4k │B1: 12k + 4k │B2: 20k + 4k │B3: P │ ┼──────┴────────────┴─────────────┴─────────────┴────────────┼ │chunk 1 │ │ ┌────────────┬─────────────┬─────────────┬────────────┤ │ sh2 │C0: 24k + 4k│C1: 32k + 4k │C2: P │C3: 40k + 4k│ │ ┼────────────┼─────────────┼─────────────┼────────────┼ │ sh3 │D0: 28k + 4k│D1: 36k + 4k │D2: P │D3: 44k + 4k│ └──────┴────────────┴─────────────┴─────────────┴────────────┘ Before this set, 4 stripe head will be used, and each sh will attach bio for 3 disks, and each attached bio will trigger bitmap_startwrite() once, which means total 12 times. - 3 times (0 + 4k), for (A0, A1 and A2) - 3 times (4 + 4k), for (B0, B1 and B2) - 3 times (8 + 4k), for (C0, C1 and C3) - 3 times (12 + 4k), for (D0, D1 and D3) After this set, md upper layer will calculate that IO range (0 + 48k) is corresponding to the bitmap (0 + 16k), and call bitmap_startwrite() just once. Noted that this patch will align bitmap ranges to the chunks, for example, if user issue a IO (0 + 4k) to array: - Before this set, 1 time (0 + 4k), for A0; - After this set, 1 time (0 + 8k) for chunk 0; Usually, one bitmap bit will represent more than one disk chunk, and this doesn't have any difference. And even if user really created a array that one chunk contain multiple bits, the overhead is that more data will be recovered after power failure. Yu Kuai (5): md/md-bitmap: factor behind write counters out from bitmap_{start/end}write() md/md-bitmap: remove the last parameter for bimtap_ops->endwrite() md: add a new callback pers->bitmap_sector() md/raid5: implement pers->bitmap_sector() md/md-bitmap: move bitmap_{start, end}write to md upper layer drivers/md/md-bitmap.c | 74 ++++++++++++++++++++++++---------------- drivers/md/md-bitmap.h | 7 ++-- drivers/md/md.c | 29 ++++++++++++++++ drivers/md/md.h | 5 +++ drivers/md/raid1.c | 11 +++--- drivers/md/raid10.c | 6 ---- drivers/md/raid5-cache.c | 4 --- drivers/md/raid5.c | 48 ++++++++++++-------------- 8 files changed, 109 insertions(+), 75 deletions(-)