From patchwork Fri Jun 7 13:17:40 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naohiro Aota X-Patchwork-Id: 10981835 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 7A73014E5 for ; Fri, 7 Jun 2019 13:19:23 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 6B28F28BA0 for ; Fri, 7 Jun 2019 13:19:23 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 5FA2E28BBC; Fri, 7 Jun 2019 13:19:23 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.7 required=2.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 13C6D28BA5 for ; Fri, 7 Jun 2019 13:19:21 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728261AbfFGNSD (ORCPT ); Fri, 7 Jun 2019 09:18:03 -0400 Received: from esa2.hgst.iphmx.com ([68.232.143.124]:56451 "EHLO esa2.hgst.iphmx.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727467AbfFGNSD (ORCPT ); Fri, 7 Jun 2019 09:18:03 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=wdc.com; i=@wdc.com; q=dns/txt; s=dkim.wdc.com; t=1559913510; x=1591449510; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=mcN9WtvQEWdujXnIA7oYPdYlHv79AOG60zJ58+aPVaw=; b=SyUKUFgIU2Is/+yrEKogBLJmIgZUpPSi0feq0SPTJ83/FGXHqN8v3J/N YQPm1YVwiScLQA9XgzfBGap9cmg5X9nBgUvvJy6V44tSe3euq1joDYU+C O/JOhFY9xRKIC7H5y34o+deBpWzXnN8NYkPiqvaWZRdEOwehvx4KZY7hC E/nUgKVWXWz47zvfUWf5MbhLul63PUe+Bya0xD68a0TCui/JCEf7AEp+T 0zqJZUjRt4MIuVL6IJXJqgYUzuloI7oeQl7mfpbDiwbvccU3o882yrnWv JXNka0rUVYSTGT+bactc9sU/sDTxESMXYuPe8q4aCQE0kto1/yOtp70WK Q==; X-IronPort-AV: E=Sophos;i="5.63,563,1557158400"; d="scan'208";a="209674976" Received: from uls-op-cesaip02.wdc.com (HELO uls-op-cesaep02.wdc.com) ([199.255.45.15]) by ob1.hgst.iphmx.com with ESMTP; 07 Jun 2019 21:18:29 +0800 IronPort-SDR: SBbqHnJA5jmRXmt2EGP55K+bxw8dMC9uCdUJLI4j3vysUOhgbXvgMS/IK7yOzPmBxZAyDY67f5 sl7GVsmQIZ8OoXvhfvZ9HMs0TPNPpVogJM/kLy7GKw577ZxD288HlM1Gac8IVzsmUPOjtJyaa3 yaGHMt7PHLaDP03vVZijbhznoHxMaH9AU5CpRU7uxDDLqEsVogoWSUdAYswUDCD4RSQiUxWtZd b0FBTpCqu6PXBtlnUxIqMMcmOwAkprin+L8FhmLYT0oXp1/EM/ybJRGgBx/q9oQz6rnr2/MkLi ptOz4neXnvgm7GY5ogaGco/1 Received: from uls-op-cesaip02.wdc.com ([10.248.3.37]) by uls-op-cesaep02.wdc.com with ESMTP; 07 Jun 2019 05:55:20 -0700 IronPort-SDR: uCGEZ0Cr8PiYRuFSJSdPJVQ2ygdQWLtl5ZOTaxWaNxZ5uUJ8wPn9mI/bAShPxilfe9967eTrdh Njp/lHbIUK8ZLUe+QpPApZlrUxiltbLJr9XQ9PWMpcdDoopi0m9RPDOS6ik5uU/Qd8SMiLfB/1 my/12itXS3Mw7U9MS3ZeOhdKSzW4HFrZXNBKM1jg6e4EqRJoVjpTxe32tyD8lvGQyM9UCniG95 /4D6Q9wDaX8Kw9C9LlLdeZOgj+mGoQOZ2IxvXgUHbn3leqrg79tB/1KNp4ozbsAtReuxsgE3hz 2WI= Received: from naota.dhcp.fujisawa.hgst.com (HELO naota.fujisawa.hgst.com) ([10.149.53.115]) by uls-op-cesaip02.wdc.com with ESMTP; 07 Jun 2019 06:18:01 -0700 From: Naohiro Aota To: linux-btrfs@vger.kernel.org, David Sterba Cc: Chris Mason , Josef Bacik , Qu Wenruo , Nikolay Borisov , linux-kernel@vger.kernel.org, Hannes Reinecke , linux-fsdevel@vger.kernel.org, Damien Le Moal , =?utf-8?q?Matias_Bj=C3=B8rling?= , Johannes Thumshirn , Bart Van Assche , Naohiro Aota Subject: [PATCH 01/12] btrfs-progs: build: Check zoned block device support Date: Fri, 7 Jun 2019 22:17:40 +0900 Message-Id: <20190607131751.5359-1-naohiro.aota@wdc.com> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20190607131025.31996-1-naohiro.aota@wdc.com> References: <20190607131025.31996-1-naohiro.aota@wdc.com> MIME-Version: 1.0 Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP If the kernel supports zoned block devices, the file /usr/include/linux/blkzoned.h will be present. Check this and define BTRFS_ZONED if the file is present. If it present, enables HMZONED feature, if not disable it. Signed-off-by: Damien Le Moal Signed-off-by: Naohiro Aota --- configure.ac | 13 +++++++++++++ 1 file changed, 13 insertions(+) diff --git a/configure.ac b/configure.ac index cf792eb5488b..c637f72a8fe6 100644 --- a/configure.ac +++ b/configure.ac @@ -206,6 +206,18 @@ else AC_DEFINE([HAVE_OWN_FIEMAP_EXTENT_SHARED_DEFINE], [0], [We did not define FIEMAP_EXTENT_SHARED]) fi +AC_CHECK_HEADER(linux/blkzoned.h, [blkzoned_found=yes], [blkzoned_found=no]) +AC_ARG_ENABLE([zoned], + AS_HELP_STRING([--disable-zoned], [disable zoned block device support]), + [], [enable_zoned=$blkzoned_found] +) + +AS_IF([test "x$enable_zoned" = xyes], [ + AC_CHECK_HEADER(linux/blkzoned.h, [], + [AC_MSG_ERROR([Couldn't find linux/blkzoned.h])]) + AC_DEFINE([BTRFS_ZONED], [1], [enable zoned block device support]) +]) + dnl Define _LIBS= and _CFLAGS= by pkg-config dnl dnl The default PKG_CHECK_MODULES() action-if-not-found is end the @@ -307,6 +319,7 @@ AC_MSG_RESULT([ btrfs-restore zstd: ${enable_zstd} Python bindings: ${enable_python} Python interpreter: ${PYTHON} + zoned device: ${enable_zoned} Type 'make' to compile. ]) From patchwork Fri Jun 7 13:17:41 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naohiro Aota X-Patchwork-Id: 10981831 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 0392F15E6 for ; Fri, 7 Jun 2019 13:19:19 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id E7EB72879D for ; Fri, 7 Jun 2019 13:19:18 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id DC50328B5B; Fri, 7 Jun 2019 13:19:18 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.7 required=2.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 731FB28B7A for ; Fri, 7 Jun 2019 13:19:18 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728619AbfFGNSG (ORCPT ); Fri, 7 Jun 2019 09:18:06 -0400 Received: from esa2.hgst.iphmx.com ([68.232.143.124]:56451 "EHLO esa2.hgst.iphmx.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727467AbfFGNSF (ORCPT ); Fri, 7 Jun 2019 09:18:05 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=wdc.com; i=@wdc.com; q=dns/txt; s=dkim.wdc.com; t=1559913513; x=1591449513; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=Yc969BoWhmiS9qHR4q79BQ4HSMy5w/pXrnIo0UWQJmY=; b=UkkXvQUzN7tsWlwtELDMqlg+UgWuY986h2iNKpdCv8ySchRj3fpDb//V RMN+Ml6hqP4/x6uGol8aTHUApEwm33xkAohSIxWLfC9LAgEnOU4nlOTor HYLsLr6UhCaYadQFV5U6seZ8PsLycn5h1Z3SG2edeL3T4DDhz7DDXxRYT GOiA33eFN0k/w+L3rC+YCA2oRgejABXCS54uLTzDjBikb0MLtFRPEel5g wkFJzffoohUhZDDCnMFbgQyldWBC9A3gF51KQRWxJ2rqbUQvJzSv0fOxl gxpOcuUSf161ZzYBj2k/VKM1IgcgrfV+kfVK7+geXvkvLjVOkSr5UcLRv w==; X-IronPort-AV: E=Sophos;i="5.63,563,1557158400"; d="scan'208";a="209674981" Received: from uls-op-cesaip02.wdc.com (HELO uls-op-cesaep02.wdc.com) ([199.255.45.15]) by ob1.hgst.iphmx.com with ESMTP; 07 Jun 2019 21:18:32 +0800 IronPort-SDR: rmtvv5nKXeyRyZxuWjbCVuAKeQ1mrf7MA2JIu3qYWO6NfbWid60j3nK929FRtGwwsYR6v/piXJ qch08UWSWH53cN5LEHX/1Kp6Mo1YEi8MiLcPsk3s3tRhPhtYPXJEIrbceZD2QjiRxYFytzezA0 5es02aPxo0Fw+D++kZhqp6MNKd9MJuG9JqpiD+y0Q8oQtNsBG/ZjJOAgfN3p3DeGQZdUfQ7J4+ egUCSKM5QR4ohvB5xlf3ZfEaoWRo776Bn2wcJJ5XR4jX3aCwQH17S5oaL81+TeYk8vTSeq7kdp 3YQEwJKCBQesSTy4+KTig/LS Received: from uls-op-cesaip02.wdc.com ([10.248.3.37]) by uls-op-cesaep02.wdc.com with ESMTP; 07 Jun 2019 05:55:22 -0700 IronPort-SDR: nIMyNf0cdHsV8clYR97sR6yXmzQaSw1ZOwFyLEPpECQq4biZuhMO0Yv/Q61ZpBW8ELsOnf0S8y OEwLWN0J73Ef1CzjUM3ajO9XtRg/y+r017FHj+JrQ1GEkkVZsdoR2MsA4Di571aBB6zej8qt9u F1GQNr/rBL738AotV5qHDshK0NKHNlDHU3Qa4BgCpEXuu1DZs5yOIdP4cz5BR9TdizAMaupWYC pVNtp9ckZYBLrAsBWtNneR9Ss37NfgtRlu5KTkz1wwEsJGaQk+yZdBFKQkWaZpS2aUWGzvXXry Z5I= Received: from naota.dhcp.fujisawa.hgst.com (HELO naota.fujisawa.hgst.com) ([10.149.53.115]) by uls-op-cesaip02.wdc.com with ESMTP; 07 Jun 2019 06:18:03 -0700 From: Naohiro Aota To: linux-btrfs@vger.kernel.org, David Sterba Cc: Chris Mason , Josef Bacik , Qu Wenruo , Nikolay Borisov , linux-kernel@vger.kernel.org, Hannes Reinecke , linux-fsdevel@vger.kernel.org, Damien Le Moal , =?utf-8?q?Matias_Bj=C3=B8rling?= , Johannes Thumshirn , Bart Van Assche , Naohiro Aota Subject: [PATCH 02/12] btrfs-progs: utils: Introduce queue_param Date: Fri, 7 Jun 2019 22:17:41 +0900 Message-Id: <20190607131751.5359-2-naohiro.aota@wdc.com> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20190607131751.5359-1-naohiro.aota@wdc.com> References: <20190607131025.31996-1-naohiro.aota@wdc.com> <20190607131751.5359-1-naohiro.aota@wdc.com> MIME-Version: 1.0 Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Introduce the queue_param function to get a device request queue parameter and this function to test if the device is an SSD in is_ssd(). Signed-off-by: Damien Le Moal [Naohiro] fixed error return value Signed-off-by: Naohiro Aota --- mkfs/main.c | 40 ++-------------------------------------- utils.c | 46 ++++++++++++++++++++++++++++++++++++++++++++++ utils.h | 1 + 3 files changed, 49 insertions(+), 38 deletions(-) diff --git a/mkfs/main.c b/mkfs/main.c index b442e6e40c37..93c0b71c864e 100644 --- a/mkfs/main.c +++ b/mkfs/main.c @@ -404,49 +404,13 @@ static int zero_output_file(int out_fd, u64 size) static int is_ssd(const char *file) { - blkid_probe probe; - char wholedisk[PATH_MAX]; - char sysfs_path[PATH_MAX]; - dev_t devno; - int fd; char rotational; int ret; - probe = blkid_new_probe_from_filename(file); - if (!probe) + ret = queue_param(file, "rotational", &rotational, 1); + if (ret < 1) return 0; - /* Device number of this disk (possibly a partition) */ - devno = blkid_probe_get_devno(probe); - if (!devno) { - blkid_free_probe(probe); - return 0; - } - - /* Get whole disk name (not full path) for this devno */ - ret = blkid_devno_to_wholedisk(devno, - wholedisk, sizeof(wholedisk), NULL); - if (ret) { - blkid_free_probe(probe); - return 0; - } - - snprintf(sysfs_path, PATH_MAX, "/sys/block/%s/queue/rotational", - wholedisk); - - blkid_free_probe(probe); - - fd = open(sysfs_path, O_RDONLY); - if (fd < 0) { - return 0; - } - - if (read(fd, &rotational, 1) < 1) { - close(fd); - return 0; - } - close(fd); - return rotational == '0'; } diff --git a/utils.c b/utils.c index c6cdc8f01dc1..7d5a1f3b7f8d 100644 --- a/utils.c +++ b/utils.c @@ -65,6 +65,52 @@ static unsigned short rand_seed[3]; struct btrfs_config bconf; +/* + * Get a device request queue parameter. + */ +int queue_param(const char *file, const char *param, char *buf, size_t len) +{ + blkid_probe probe; + char wholedisk[PATH_MAX]; + char sysfs_path[PATH_MAX]; + dev_t devno; + int fd; + int ret; + + probe = blkid_new_probe_from_filename(file); + if (!probe) + return 0; + + /* Device number of this disk (possibly a partition) */ + devno = blkid_probe_get_devno(probe); + if (!devno) { + blkid_free_probe(probe); + return 0; + } + + /* Get whole disk name (not full path) for this devno */ + ret = blkid_devno_to_wholedisk(devno, + wholedisk, sizeof(wholedisk), NULL); + if (ret) { + blkid_free_probe(probe); + return 0; + } + + snprintf(sysfs_path, PATH_MAX, "/sys/block/%s/queue/%s", + wholedisk, param); + + blkid_free_probe(probe); + + fd = open(sysfs_path, O_RDONLY); + if (fd < 0) + return 0; + + len = read(fd, buf, len); + close(fd); + + return len; +} + /* * Discard the given range in one go */ diff --git a/utils.h b/utils.h index 7c5eb798557d..47321f62c8e0 100644 --- a/utils.h +++ b/utils.h @@ -121,6 +121,7 @@ int get_label(const char *btrfs_dev, char *label); int set_label(const char *btrfs_dev, const char *label); char *__strncpy_null(char *dest, const char *src, size_t n); +int queue_param(const char *file, const char *param, char *buf, size_t len); int is_block_device(const char *file); int is_mount_point(const char *file); int is_path_exist(const char *file); From patchwork Fri Jun 7 13:17:42 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naohiro Aota X-Patchwork-Id: 10981793 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 793CD14E5 for ; Fri, 7 Jun 2019 13:18:14 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 5EEB828B5B for ; Fri, 7 Jun 2019 13:18:14 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 52A8A28B36; Fri, 7 Jun 2019 13:18:14 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.7 required=2.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id CD28028B5B for ; Fri, 7 Jun 2019 13:18:13 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728759AbfFGNSI (ORCPT ); Fri, 7 Jun 2019 09:18:08 -0400 Received: from esa2.hgst.iphmx.com ([68.232.143.124]:56451 "EHLO esa2.hgst.iphmx.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728689AbfFGNSI (ORCPT ); Fri, 7 Jun 2019 09:18:08 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=wdc.com; i=@wdc.com; q=dns/txt; s=dkim.wdc.com; t=1559913516; x=1591449516; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=kE1dxgEdKiS9vm8kRxNFlOdjtYNADwAQaRBEWkOjG7I=; b=e8v+X/GAePuNSn+9kWOQk9oCn3mz16Ocx5kkdi7fauImTBsq2+IH7VyG BPTl+CCdBolwub3SKsOoZOKYeRcMKj0Z18MazRyHJOLvPs0wuQTns6H+T 1j27sKYQcTcShDvfUGYgWKci7hrvVNSJxEsv53lXHM9y/HYuxEqlUIVEV WjwYmOLm49irb1syqEOijhI3gerDf3+Wr01q7Xh62hxTsIAr1F5vH0d4a ZtNacjQr9Nw7fStlgPkp158QwvK8EisKkZ63bE47XRoAOzBphAKEx3ys4 7/DOCdEzlOTkzbPwio/Z5yjGQrd2SYcu2LTH4fytOeM5qhaX8W/HdRfkG g==; X-IronPort-AV: E=Sophos;i="5.63,563,1557158400"; d="scan'208";a="209674986" Received: from uls-op-cesaip02.wdc.com (HELO uls-op-cesaep02.wdc.com) ([199.255.45.15]) by ob1.hgst.iphmx.com with ESMTP; 07 Jun 2019 21:18:36 +0800 IronPort-SDR: wh1Fat/eLjraEeHlm//cqBN4HcCeQnXpbsg57H6RSXleGlXvzwW/MCo6+TAmh9OvXMYfdWrEO9 9UjvzCX1d6WsD8IJF45GstmBqnmypbOuOEAWmUptajqGiMtEvF/kXE2pjidkKuX6BCNYMETqY+ fAjdypuj9NFWvIpoWm2TrnfHz+kWDEGW6sN5+NSiJuoC/tCy4v1mGQchmxExluDvpLLyaGLE+g KKuAeF7CBbb5fydqHOQbmNYCiGh5/8OJ/J4NF1DPWDCNGpHMDiirz09LqVPPDuj3znxZYnYGBs QR/TeZUzJhteoJN8SyKaZIKw Received: from uls-op-cesaip02.wdc.com ([10.248.3.37]) by uls-op-cesaep02.wdc.com with ESMTP; 07 Jun 2019 05:55:24 -0700 IronPort-SDR: yz4Fwogp9nNtP6CLFtFcpZ88nZKTt+wJImrrhc8x8wAKFFBYsOLVrVR7kjJY/1AcKL3io1l2/z u1Exda8OB9QDtbKEayGOAx25XX4m4sPhkSIp/u41UYCaSZ81DFUKbsKSS95rjPQY36fnwm6zEw dcyCnGkNXJzkuc0qJNdC/CijPiKn2iPkYnt+dygPN1u0dZpC5z1qQhPL1DGPh8hHAnnAoTaOS/ E1dugsZNcoiHTxmM5f1O+hrrQPbIfPBAL9yOUpJsF2TdQqVS5sucHohTVeU01FChaRncE+NEfU 6fU= Received: from naota.dhcp.fujisawa.hgst.com (HELO naota.fujisawa.hgst.com) ([10.149.53.115]) by uls-op-cesaip02.wdc.com with ESMTP; 07 Jun 2019 06:18:06 -0700 From: Naohiro Aota To: linux-btrfs@vger.kernel.org, David Sterba Cc: Chris Mason , Josef Bacik , Qu Wenruo , Nikolay Borisov , linux-kernel@vger.kernel.org, Hannes Reinecke , linux-fsdevel@vger.kernel.org, Damien Le Moal , =?utf-8?q?Matias_Bj=C3=B8rling?= , Johannes Thumshirn , Bart Van Assche , Naohiro Aota Subject: [PATCH 03/12] btrfs-progs: add new HMZONED feature flag Date: Fri, 7 Jun 2019 22:17:42 +0900 Message-Id: <20190607131751.5359-3-naohiro.aota@wdc.com> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20190607131751.5359-1-naohiro.aota@wdc.com> References: <20190607131025.31996-1-naohiro.aota@wdc.com> <20190607131751.5359-1-naohiro.aota@wdc.com> MIME-Version: 1.0 Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP With this feature enabled, a zoned block device aware btrfs allocates block groups aligned to the device zones and always write in sequential zones at the zone write pointer position. Enabling this feature also force disable conversion from ext4 volumes. Note: this flag can be moved to COMPAT_RO, so that older kernel can read but not write zoned block devices formatted with btrfs. Signed-off-by: Naohiro Aota --- cmds-inspect-dump-super.c | 3 ++- ctree.h | 4 +++- fsfeatures.c | 8 ++++++++ fsfeatures.h | 2 +- libbtrfsutil/btrfs.h | 2 ++ 5 files changed, 16 insertions(+), 3 deletions(-) diff --git a/cmds-inspect-dump-super.c b/cmds-inspect-dump-super.c index d62f0932556c..ff3c0aa262c8 100644 --- a/cmds-inspect-dump-super.c +++ b/cmds-inspect-dump-super.c @@ -229,7 +229,8 @@ static struct readable_flag_entry incompat_flags_array[] = { DEF_INCOMPAT_FLAG_ENTRY(RAID56), DEF_INCOMPAT_FLAG_ENTRY(SKINNY_METADATA), DEF_INCOMPAT_FLAG_ENTRY(NO_HOLES), - DEF_INCOMPAT_FLAG_ENTRY(METADATA_UUID) + DEF_INCOMPAT_FLAG_ENTRY(METADATA_UUID), + DEF_INCOMPAT_FLAG_ENTRY(HMZONED) }; static const int incompat_flags_num = sizeof(incompat_flags_array) / sizeof(struct readable_flag_entry); diff --git a/ctree.h b/ctree.h index 76f52b1c9b08..9f79686690e0 100644 --- a/ctree.h +++ b/ctree.h @@ -492,6 +492,7 @@ struct btrfs_super_block { #define BTRFS_FEATURE_INCOMPAT_SKINNY_METADATA (1ULL << 8) #define BTRFS_FEATURE_INCOMPAT_NO_HOLES (1ULL << 9) #define BTRFS_FEATURE_INCOMPAT_METADATA_UUID (1ULL << 10) +#define BTRFS_FEATURE_INCOMPAT_HMZONED (1ULL << 11) #define BTRFS_FEATURE_COMPAT_SUPP 0ULL @@ -515,7 +516,8 @@ struct btrfs_super_block { BTRFS_FEATURE_INCOMPAT_MIXED_GROUPS | \ BTRFS_FEATURE_INCOMPAT_SKINNY_METADATA | \ BTRFS_FEATURE_INCOMPAT_NO_HOLES | \ - BTRFS_FEATURE_INCOMPAT_METADATA_UUID) + BTRFS_FEATURE_INCOMPAT_METADATA_UUID | \ + BTRFS_FEATURE_INCOMPAT_HMZONED) /* * A leaf is full of items. offset and size tell us where to find diff --git a/fsfeatures.c b/fsfeatures.c index 7f3ef03b8452..c4904ce8baf5 100644 --- a/fsfeatures.c +++ b/fsfeatures.c @@ -86,6 +86,14 @@ static const struct btrfs_fs_feature { VERSION_TO_STRING2(4,0), NULL, 0, "no explicit hole extents for files" }, +#ifdef BTRFS_ZONED + { "hmzoned", BTRFS_FEATURE_INCOMPAT_HMZONED, + "hmzoned", + NULL, 0, + NULL, 0, + NULL, 0, + "support Host-Managed Zoned devices" }, +#endif /* Keep this one last */ { "list-all", BTRFS_FEATURE_LIST_ALL, NULL } }; diff --git a/fsfeatures.h b/fsfeatures.h index 3cc9452a3327..0918ee1aa113 100644 --- a/fsfeatures.h +++ b/fsfeatures.h @@ -25,7 +25,7 @@ | BTRFS_FEATURE_INCOMPAT_SKINNY_METADATA) /* - * Avoid multi-device features (RAID56) and mixed block groups + * Avoid multi-device features (RAID56), mixed block groups, and hmzoned device */ #define BTRFS_CONVERT_ALLOWED_FEATURES \ (BTRFS_FEATURE_INCOMPAT_MIXED_BACKREF \ diff --git a/libbtrfsutil/btrfs.h b/libbtrfsutil/btrfs.h index 944d50132456..5c415240f74c 100644 --- a/libbtrfsutil/btrfs.h +++ b/libbtrfsutil/btrfs.h @@ -268,6 +268,8 @@ struct btrfs_ioctl_fs_info_args { #define BTRFS_FEATURE_INCOMPAT_RAID56 (1ULL << 7) #define BTRFS_FEATURE_INCOMPAT_SKINNY_METADATA (1ULL << 8) #define BTRFS_FEATURE_INCOMPAT_NO_HOLES (1ULL << 9) +/* Missing */ +#define BTRFS_FEATURE_INCOMPAT_HMZONED (1ULL << 11) struct btrfs_ioctl_feature_flags { __u64 compat_flags; From patchwork Fri Jun 7 13:17:43 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naohiro Aota X-Patchwork-Id: 10981823 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id AA7856C5 for ; Fri, 7 Jun 2019 13:19:07 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 996EA28B36 for ; Fri, 7 Jun 2019 13:19:07 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 8E2C728BA0; Fri, 7 Jun 2019 13:19:07 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.7 required=2.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id E0DCD28B7A for ; Fri, 7 Jun 2019 13:19:06 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728932AbfFGNSO (ORCPT ); Fri, 7 Jun 2019 09:18:14 -0400 Received: from esa2.hgst.iphmx.com ([68.232.143.124]:56467 "EHLO esa2.hgst.iphmx.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728689AbfFGNSO (ORCPT ); Fri, 7 Jun 2019 09:18:14 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=wdc.com; i=@wdc.com; q=dns/txt; s=dkim.wdc.com; t=1559913526; x=1591449526; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=M6mauCdJ2eXa3K/DCJMkHatGsXpK+p82lDihf8eCgSA=; b=QFKpKlaFP4fS3NYikUsvzBk30Ga5M2I4P/5XX+Jx1gvHW12ZnDcY7XRe olIuWnoLXdfgZR/PlIDEyrlZOuv8Q6/kkvDiIBjutD0DvQanq2h7NR352 6opo2w/7NWtNlb1Ye73LRSImQgBkG8+2GDZllOrcfeWwVLJnsYAo3Mo9Z fQxHJGaolS6UF95rPmaTk/g+zJLq52xM1MMrpCOX1KYDV2lwkJJ+H7xK0 qipfqASdkHEd+/S+3k12hMOiUiVYrsS8MB3QKxpJVZZHAn0cXGK1Xlmam gCelDa/xUgsvMBq3oSsUApkLZW/fqz3IxGqU9dt2xd89to+zp5JAv+TnT g==; X-IronPort-AV: E=Sophos;i="5.63,563,1557158400"; d="scan'208";a="209674990" Received: from uls-op-cesaip02.wdc.com (HELO uls-op-cesaep02.wdc.com) ([199.255.45.15]) by ob1.hgst.iphmx.com with ESMTP; 07 Jun 2019 21:18:42 +0800 IronPort-SDR: QQ3FctQPCqPR6J700W1rkoNVT4AUn6w5kfbx5qaB0LCfptC1ZJurp8YMnV029nlCbYJWpqlCqb yeYWoPcE0zDTceyeU4ZyrYCHcFCQJSEnaaEQ/I/MvNqBDm8siYDN1qWEpfrEeN5B81Tgii0zUE UDIySWIYpF5XFQU7uC+aFIfkdCmbBTVuR/XSN/hCUHpQBxzJevrYY7lW9Kg+eq5MsGTHVq59bd 8YDRPYJg9jOz/Ud5u45RxiSzGrqyCCAdWut1XfvqePHvMcQtA8AqW0j8oztPTXU734qjdxxdDI Rgcxsm2thran3BS3SDPbRJXS Received: from uls-op-cesaip02.wdc.com ([10.248.3.37]) by uls-op-cesaep02.wdc.com with ESMTP; 07 Jun 2019 05:55:26 -0700 IronPort-SDR: rGqJ6sX/PeEuL+gtp71J29Fhvsd8uI4iMQw2RAr9mrtmVnzvJGfbnzmu11Hfll60Te3ik2RUBd QMtjqiyAPKcwbchhSxq5SPDNu/wNV3Xn3+eIMFi07Tf2CjXrrsCJjCcabdhoNNYWt2pw7ED2Az VOmf7XMuguuLaYeAhaz42cydhfeWneMLfuJgRtVoIkZvxyyl4/OPkVU1GBcndUe41gLwTZhJpo +Tw/a5w2K+yqOiMA+eL2uXzVIMv47xAh5zYey71kWaS9vK6QYg7ERfANbP5X4vMB9imiLn/xfY ulc= Received: from naota.dhcp.fujisawa.hgst.com (HELO naota.fujisawa.hgst.com) ([10.149.53.115]) by uls-op-cesaip02.wdc.com with ESMTP; 07 Jun 2019 06:18:08 -0700 From: Naohiro Aota To: linux-btrfs@vger.kernel.org, David Sterba Cc: Chris Mason , Josef Bacik , Qu Wenruo , Nikolay Borisov , linux-kernel@vger.kernel.org, Hannes Reinecke , linux-fsdevel@vger.kernel.org, Damien Le Moal , =?utf-8?q?Matias_Bj=C3=B8rling?= , Johannes Thumshirn , Bart Van Assche , Naohiro Aota Subject: [PATCH 04/12] btrfs-progs: Introduce zone block device helper functions Date: Fri, 7 Jun 2019 22:17:43 +0900 Message-Id: <20190607131751.5359-4-naohiro.aota@wdc.com> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20190607131751.5359-1-naohiro.aota@wdc.com> References: <20190607131025.31996-1-naohiro.aota@wdc.com> <20190607131751.5359-1-naohiro.aota@wdc.com> MIME-Version: 1.0 Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP This patch introduce several zone related functions: btrfs_get_zones() to get zone information from the specified device and put the information in zinfo, and zone_is_random_write() to check if a zone accept random writes. Signed-off-by: Naohiro Aota --- utils.c | 194 ++++++++++++++++++++++++++++++++++++++++++++++++++++++ utils.h | 16 +++++ volumes.h | 28 ++++++++ 3 files changed, 238 insertions(+) diff --git a/utils.c b/utils.c index 7d5a1f3b7f8d..d50304b1be80 100644 --- a/utils.c +++ b/utils.c @@ -359,6 +359,200 @@ out: return ret; } +enum btrfs_zoned_model zoned_model(const char *file) +{ + char model[32]; + int ret; + + ret = queue_param(file, "zoned", model, sizeof(model)); + if (ret <= 0) + return ZONED_NONE; + + if (strncmp(model, "host-aware", 10) == 0) + return ZONED_HOST_AWARE; + if (strncmp(model, "host-managed", 12) == 0) + return ZONED_HOST_MANAGED; + + return ZONED_NONE; +} + +size_t zone_size(const char *file) +{ + char chunk[32]; + int ret; + + ret = queue_param(file, "chunk_sectors", chunk, sizeof(chunk)); + if (ret <= 0) + return 0; + + return strtoul((const char *)chunk, NULL, 10) << 9; +} + +#ifdef BTRFS_ZONED +int zone_is_random_write(struct btrfs_zone_info *zinfo, u64 bytenr) +{ + unsigned int zno; + + if (zinfo->model == ZONED_NONE) + return 1; + + zno = bytenr / zinfo->zone_size; + + /* + * Only sequential write required zones on host-managed + * devices cannot be written randomly. + */ + return zinfo->zones[zno].type != BLK_ZONE_TYPE_SEQWRITE_REQ; +} + +#define BTRFS_REPORT_NR_ZONES 8192 + +static int btrfs_get_zones(int fd, const char *file, u64 block_count, + struct btrfs_zone_info *zinfo) +{ + size_t zone_bytes = zone_size(file); + size_t rep_size; + u64 sector = 0; + struct blk_zone_report *rep; + struct blk_zone *zone; + unsigned int i, n = 0; + int ret; + + /* + * Zones are guaranteed (by the kernel) to be a power of 2 number of + * sectors. Check this here and make sure that zones are not too + * small. + */ + if (!zone_bytes || (zone_bytes & (zone_bytes - 1))) { + error("ERROR: Illegal zone size %zu (not a power of 2)\n", + zone_bytes); + exit(1); + } + if (zone_bytes < BTRFS_MKFS_SYSTEM_GROUP_SIZE) { + error("ERROR: Illegal zone size %zu (smaller than %d)\n", + zone_bytes, + BTRFS_MKFS_SYSTEM_GROUP_SIZE); + exit(1); + } + + /* Allocate the zone information array */ + zinfo->zone_size = zone_bytes; + zinfo->nr_zones = block_count / zone_bytes; + if (block_count & (zone_bytes - 1)) + zinfo->nr_zones++; + zinfo->zones = calloc(zinfo->nr_zones, sizeof(struct blk_zone)); + if (!zinfo->zones) { + error("No memory for zone information\n"); + exit(1); + } + + /* Allocate a zone report */ + rep_size = sizeof(struct blk_zone_report) + + sizeof(struct blk_zone) * BTRFS_REPORT_NR_ZONES; + rep = malloc(rep_size); + if (!rep) { + error("No memory for zones report\n"); + exit(1); + } + + /* Get zone information */ + zone = (struct blk_zone *)(rep + 1); + while (n < zinfo->nr_zones) { + + memset(rep, 0, rep_size); + rep->sector = sector; + rep->nr_zones = BTRFS_REPORT_NR_ZONES; + + ret = ioctl(fd, BLKREPORTZONE, rep); + if (ret != 0) { + error("ioctl BLKREPORTZONE failed (%s)\n", + strerror(errno)); + exit(1); + } + + if (!rep->nr_zones) + break; + + for (i = 0; i < rep->nr_zones; i++) { + if (n >= zinfo->nr_zones) + break; + memcpy(&zinfo->zones[n], &zone[i], + sizeof(struct blk_zone)); + sector = zone[i].start + zone[i].len; + n++; + } + + } + + /* + * We need at least one random write zone (a conventional zone or + * a sequential write preferred zone on a host-aware device). + */ + if (!zone_is_random_write(zinfo, 0)) { + error("ERROR: No conventional zone at block 0\n"); + exit(1); + } + + zinfo->nr_zones = n; + + free(rep); + + return 0; +} + +#endif + +int btrfs_get_zone_info(int fd, const char *file, int hmzoned, + struct btrfs_zone_info *zinfo) +{ + struct stat st; + int ret; + + memset(zinfo, 0, sizeof(struct btrfs_zone_info)); + + ret = fstat(fd, &st); + if (ret < 0) { + error("unable to stat %s\n", file); + return 1; + } + + if (!S_ISBLK(st.st_mode)) + return 0; + + /* Check zone model */ + zinfo->model = zoned_model(file); + if (zinfo->model == ZONED_NONE) + return 0; + + if (zinfo->model == ZONED_HOST_MANAGED && !hmzoned) { + error("%s: host-managed zoned block device (enable zone block device support with -O hmzoned)\n", + file); + return -1; + } + + if (!hmzoned) { + /* Treat host-aware devices as regular devices */ + zinfo->model = ZONED_NONE; + return 0; + } + +#ifdef BTRFS_ZONED + /* Get zone information */ + ret = btrfs_get_zones(fd, file, btrfs_device_size(fd, &st), zinfo); + if (ret != 0) + return ret; +#else + error("%s: Unsupported host-%s zoned block device\n", + file, zinfo->model == ZONED_HOST_MANAGED ? "managed" : "aware"); + if (zinfo->model == ZONED_HOST_MANAGED) + return -1; + + printf("%s: heandling host-aware block device as a regular disk\n", + file); +#endif + return 0; +} + int btrfs_prepare_device(int fd, const char *file, u64 *block_count_ret, u64 max_block_count, unsigned opflags) { diff --git a/utils.h b/utils.h index 47321f62c8e0..f5e5c5bec66a 100644 --- a/utils.h +++ b/utils.h @@ -69,6 +69,7 @@ void units_set_base(unsigned *units, unsigned base); #define PREP_DEVICE_ZERO_END (1U << 0) #define PREP_DEVICE_DISCARD (1U << 1) #define PREP_DEVICE_VERBOSE (1U << 2) +#define PREP_DEVICE_HMZONED (1U << 3) #define SEEN_FSID_HASH_SIZE 256 struct seen_fsid { @@ -78,10 +79,25 @@ struct seen_fsid { int fd; }; +struct btrfs_zone_info; + +enum btrfs_zoned_model zoned_model(const char *file); +size_t zone_size(const char *file); int btrfs_make_root_dir(struct btrfs_trans_handle *trans, struct btrfs_root *root, u64 objectid); int btrfs_prepare_device(int fd, const char *file, u64 *block_count_ret, u64 max_block_count, unsigned opflags); +int btrfs_get_zone_info(int fd, const char *file, int hmzoned, + struct btrfs_zone_info *zinfo); +#ifdef BTRFS_ZONED +int zone_is_random_write(struct btrfs_zone_info *zinfo, u64 bytenr); +#else +static inline int zone_is_random_write(struct btrfs_zone_info *zinfo, + u64 bytenr) +{ + return 1; +} +#endif int btrfs_add_to_fsid(struct btrfs_trans_handle *trans, struct btrfs_root *root, int fd, const char *path, u64 block_count, u32 io_width, u32 io_align, diff --git a/volumes.h b/volumes.h index dbe9d3dea647..c9262ceaea93 100644 --- a/volumes.h +++ b/volumes.h @@ -22,12 +22,40 @@ #include "kerncompat.h" #include "ctree.h" +#ifdef BTRFS_ZONED +#include +#else +struct blk_zone { + int dummy; +}; +#endif + +/* + * Zoned block device models. + */ +enum btrfs_zoned_model { + ZONED_NONE = 0, + ZONED_HOST_AWARE, + ZONED_HOST_MANAGED, +}; + +/* + * Zone information for a zoned block device. + */ +struct btrfs_zone_info { + enum btrfs_zoned_model model; + size_t zone_size; + struct blk_zone *zones; + unsigned int nr_zones; +}; + #define BTRFS_STRIPE_LEN SZ_64K struct btrfs_device { struct list_head dev_list; struct btrfs_root *dev_root; struct btrfs_fs_devices *fs_devices; + struct btrfs_zone_info zinfo; u64 total_ios; From patchwork Fri Jun 7 13:17:44 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naohiro Aota X-Patchwork-Id: 10981825 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id BF05A14E5 for ; Fri, 7 Jun 2019 13:19:13 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id AEA1D28BA0 for ; Fri, 7 Jun 2019 13:19:13 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 9EEB328BBE; Fri, 7 Jun 2019 13:19:13 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.7 required=2.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 3946D28B91 for ; Fri, 7 Jun 2019 13:19:13 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729443AbfFGNTI (ORCPT ); Fri, 7 Jun 2019 09:19:08 -0400 Received: from esa2.hgst.iphmx.com ([68.232.143.124]:56467 "EHLO esa2.hgst.iphmx.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728817AbfFGNSO (ORCPT ); Fri, 7 Jun 2019 09:18:14 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=wdc.com; i=@wdc.com; q=dns/txt; s=dkim.wdc.com; t=1559913527; x=1591449527; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=QZ/1/rNX3/3n7cc4vvS1dkFdK6HEV83hxR0tcxXdqV8=; b=gbnTZDa94dvjfbCoCSQ7ir/4EqqeBTtYJtiONx9rJZ4t59lPz7WUaTFC VpsNPn4G+VtlNoO5MuNuWNHxU6mMYQzxlAAElCmf6HT2zOkk5NiQZCKQ9 tCXA7kSE9MxTdy841smkU7Mb7utXVl5KvWlWuaLAGXdMEddVZMSDWyJIO udJO9VC8ot0vo6jIzMWvCGqR0RE3tNYsR2ycFD/eR5Z5ifJeUEFhtyn4K QcuivAV0w7mab2nYn/B9kzptJ/AQl4dAHKjku8Ugm6kGlfc+fDmIz+W1I N+gZpfoWQQMsfENI/EGVRxUONvAS3bBxDlSVX03CcOFiJFrrN942krN9/ w==; X-IronPort-AV: E=Sophos;i="5.63,563,1557158400"; d="scan'208";a="209674998" Received: from uls-op-cesaip02.wdc.com (HELO uls-op-cesaep02.wdc.com) ([199.255.45.15]) by ob1.hgst.iphmx.com with ESMTP; 07 Jun 2019 21:18:43 +0800 IronPort-SDR: 977dDIxHZD/vuTaoSyqWmLjV7G2jG2z4l/2DNIG8RIji2BhvI5gZMlrFfBfUXmbbrMzqVQx1WV w5rsaEv5oQqRvTUAUI8EsFKS3DYsECyZeZArPATuGZspwPdLKyyhkhEmEMBl/91xbs5298PgME iFnLA5iu2FNv5MERSqcCh+xdGdBWf6NfB0U2XAD5oaXTTzW6Ll69/VXRREQ2j0Q700SNoA5Xsy wXzPgir7MW1NOChBgJ9YO4NX2/QMXDYHqWxvMo6pw6uspX9+LHOWMYVr8Ng1nfyzYM24UglhRU jTzDyGMDwxshYfDyN/hKbdkz Received: from uls-op-cesaip02.wdc.com ([10.248.3.37]) by uls-op-cesaep02.wdc.com with ESMTP; 07 Jun 2019 05:55:29 -0700 IronPort-SDR: FzKJnxM8QyGDUNjyTLrZeFo/pnqqCte9zBMugrZeI2BgIKRGZx6g2NJMGMeIx2ounariG6vD/F kifFaoiCi+s3h/Ewp+1DliFFHKsV43ftPvPtdNVX5h9XFdA/ZL0GMIVXNzmAZdlhiM+rW13MN3 D1ACy4kUyr+RWhBmEfLq7ETLtJdnjQiY3EyQ4BFhPUAyM30xpwTskmobAUW6bA/nanT6U2XUTW 7oqN9G9NcbBFGNGsGRuEszEplcTCEbDCPEcRwSUOr47xnWRDcSg1D6ZUM80zUmPyF6DwZLfbSd 7Qs= Received: from naota.dhcp.fujisawa.hgst.com (HELO naota.fujisawa.hgst.com) ([10.149.53.115]) by uls-op-cesaip02.wdc.com with ESMTP; 07 Jun 2019 06:18:10 -0700 From: Naohiro Aota To: linux-btrfs@vger.kernel.org, David Sterba Cc: Chris Mason , Josef Bacik , Qu Wenruo , Nikolay Borisov , linux-kernel@vger.kernel.org, Hannes Reinecke , linux-fsdevel@vger.kernel.org, Damien Le Moal , =?utf-8?q?Matias_Bj=C3=B8rling?= , Johannes Thumshirn , Bart Van Assche , Naohiro Aota Subject: [PATCH 05/12] btrfs-progs: load and check zone information Date: Fri, 7 Jun 2019 22:17:44 +0900 Message-Id: <20190607131751.5359-5-naohiro.aota@wdc.com> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20190607131751.5359-1-naohiro.aota@wdc.com> References: <20190607131025.31996-1-naohiro.aota@wdc.com> <20190607131751.5359-1-naohiro.aota@wdc.com> MIME-Version: 1.0 Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP This patch checks if a device added to btrfs is a zoned block device. If it is, load zones information and the zone size for the device. For a btrfs volume composed of multiple zoned block devices, all devices must have the same zone size. Signed-off-by: Naohiro Aota --- utils.c | 10 ++++++++++ volumes.c | 18 ++++++++++++++++++ volumes.h | 3 +++ 3 files changed, 31 insertions(+) diff --git a/utils.c b/utils.c index d50304b1be80..a26fe7a5743c 100644 --- a/utils.c +++ b/utils.c @@ -250,6 +250,16 @@ int btrfs_add_to_fsid(struct btrfs_trans_handle *trans, goto out; } + ret = btrfs_get_zone_info(fd, path, fs_info->fs_devices->hmzoned, + &device->zinfo); + if (ret) + goto out; + if (device->zinfo.zone_size != fs_info->fs_devices->zone_size) { + error("Device zone size differ\n"); + ret = -EINVAL; + goto out; + } + disk_super = (struct btrfs_super_block *)buf; dev_item = &disk_super->dev_item; diff --git a/volumes.c b/volumes.c index 3a91b43b378b..f6d1b1e9dc7f 100644 --- a/volumes.c +++ b/volumes.c @@ -168,6 +168,8 @@ static int device_list_add(const char *path, u64 found_transid = btrfs_super_generation(disk_super); bool metadata_uuid = (btrfs_super_incompat_flags(disk_super) & BTRFS_FEATURE_INCOMPAT_METADATA_UUID); + int hmzoned = btrfs_super_incompat_flags(disk_super) & + BTRFS_FEATURE_INCOMPAT_HMZONED; if (metadata_uuid) fs_devices = find_fsid(disk_super->fsid, @@ -257,6 +259,8 @@ static int device_list_add(const char *path, if (fs_devices->lowest_devid > devid) { fs_devices->lowest_devid = devid; } + if (hmzoned) + fs_devices->hmzoned = 1; *fs_devices_ret = fs_devices; return 0; } @@ -327,6 +331,8 @@ int btrfs_open_devices(struct btrfs_fs_devices *fs_devices, int flags) struct btrfs_device *device; int ret; + fs_devices->zone_size = 0; + list_for_each_entry(device, &fs_devices->devices, dev_list) { if (!device->name) { printk("no name for device %llu, skip it now\n", device->devid); @@ -350,6 +356,18 @@ int btrfs_open_devices(struct btrfs_fs_devices *fs_devices, int flags) device->fd = fd; if (flags & O_RDWR) device->writeable = 1; + + ret = btrfs_get_zone_info(fd, device->name, fs_devices->hmzoned, + &device->zinfo); + if (ret != 0) + goto fail; + if (!fs_devices->zone_size) { + fs_devices->zone_size = device->zinfo.zone_size; + } else if (device->zinfo.zone_size != fs_devices->zone_size) { + fprintf(stderr, "Device zone size differ\n"); + ret = -EINVAL; + goto fail; + } } return 0; fail: diff --git a/volumes.h b/volumes.h index c9262ceaea93..6ec83fe43cfe 100644 --- a/volumes.h +++ b/volumes.h @@ -115,6 +115,9 @@ struct btrfs_fs_devices { int seeding; struct btrfs_fs_devices *seed; + + u64 zone_size; + unsigned int hmzoned:1; }; struct btrfs_bio_stripe { From patchwork Fri Jun 7 13:17:45 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naohiro Aota X-Patchwork-Id: 10981817 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 13EC46C5 for ; Fri, 7 Jun 2019 13:19:02 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 0529E2879D for ; Fri, 7 Jun 2019 13:19:02 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id ED19128BFA; Fri, 7 Jun 2019 13:19:01 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.7 required=2.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=unavailable version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id A21AA28BBE for ; Fri, 7 Jun 2019 13:19:01 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729086AbfFGNSS (ORCPT ); Fri, 7 Jun 2019 09:18:18 -0400 Received: from esa2.hgst.iphmx.com ([68.232.143.124]:56474 "EHLO esa2.hgst.iphmx.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728861AbfFGNSP (ORCPT ); Fri, 7 Jun 2019 09:18:15 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=wdc.com; i=@wdc.com; q=dns/txt; s=dkim.wdc.com; t=1559913527; x=1591449527; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=USN6rEALIg6DXGN8tesjcKQiWrb+oKgfNOsJQAIlJpY=; b=j/ia6CjyvdTRUluv0mkHo0/NcHrheAm7T55UJ2zK7eLBoQHhg2qgWPXQ Q9d2964eZYh07qg1zXQ7cjndHOOQDixjol4ven8lV2JsjhyAqr6LVdWQv vzJwQH/I4yVTd+vXvoRPZnoD2QDiGlxFDuH0mnBztwrP3PD9Q7i/lAwfu tBwOruI/UOkCxotifENbW3Hm5CuJfZCnf8NI1n6Id7Vlf6Vx9YAi//U95 3+8xavkW4DllnljEg4q7zW2GBnHqt3a/pYTL2zHQLcLMXrtVR2zJdvaZ4 LMzgZhK3oKpUYeTKc12rdD8IBnUcDDwb724D+0xsHY/Hs1H5ZwQfLgf7l w==; X-IronPort-AV: E=Sophos;i="5.63,563,1557158400"; d="scan'208";a="209675005" Received: from uls-op-cesaip02.wdc.com (HELO uls-op-cesaep02.wdc.com) ([199.255.45.15]) by ob1.hgst.iphmx.com with ESMTP; 07 Jun 2019 21:18:46 +0800 IronPort-SDR: ZFFN9wfnrckh/kPPFPSJhuRuDuH+D4SPRhQxe4ksJpeHmp8G0EF/1qjbh32KeaTBbHlGj0wlhk 811nLrAd0swD8++b6t7Rdq91JDdJV1dx+t6Q2cHQVO8Wdu24seYJLuiJUNqCs6TQ/0+ifI+7Q4 RVnNNkflPOYTKzHuafJzdILp1Ah2WjEgj5pB9BkixWNXattMziZTGHTGKfHaA7oSzvVoMigz+m korJZxdfeQhHnXiwYl0j91Ju6voUCHJ5q7mGmccJ3/I5jnNPRfAi0N+HfuOa27Qyc4FCj3ohz+ ootl4IMpe7d7PxslvS6zxQI9 Received: from uls-op-cesaip02.wdc.com ([10.248.3.37]) by uls-op-cesaep02.wdc.com with ESMTP; 07 Jun 2019 05:55:31 -0700 IronPort-SDR: ilYLTBqhqUPHW7ACJRfxQ87dXLApIggwra3dceiCoshC2+EcCZNFHJRbIuZLRuHRTKVFNDqr6O +HtTWQEN5GRP5SKvGCOBoQBaAIXKXgFJ26vexpOzQKinWoBUSI5mRYimRGhowuQlHnYx9OEY2K /YD6rtJG5fy6VYhApsElYuFLB/5DGl2e6aIajyn8hLNCKvdqSOqXrxmP5KSNC9zRrePGRJjLEr CvieLuLgiHTSzVS6YuNK781rlQ0nt9HM0iViCi293vFQ5u1K1/Stxfrv7KQGtLwb5sHhJLQnAA 6no= Received: from naota.dhcp.fujisawa.hgst.com (HELO naota.fujisawa.hgst.com) ([10.149.53.115]) by uls-op-cesaip02.wdc.com with ESMTP; 07 Jun 2019 06:18:12 -0700 From: Naohiro Aota To: linux-btrfs@vger.kernel.org, David Sterba Cc: Chris Mason , Josef Bacik , Qu Wenruo , Nikolay Borisov , linux-kernel@vger.kernel.org, Hannes Reinecke , linux-fsdevel@vger.kernel.org, Damien Le Moal , =?utf-8?q?Matias_Bj=C3=B8rling?= , Johannes Thumshirn , Bart Van Assche , Naohiro Aota Subject: [PATCH 06/12] btrfs-progs: avoid writing super block to sequential zones Date: Fri, 7 Jun 2019 22:17:45 +0900 Message-Id: <20190607131751.5359-6-naohiro.aota@wdc.com> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20190607131751.5359-1-naohiro.aota@wdc.com> References: <20190607131025.31996-1-naohiro.aota@wdc.com> <20190607131751.5359-1-naohiro.aota@wdc.com> MIME-Version: 1.0 Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP It is not possible to write a super block copy in sequential write required zones as this prevents in-place updates required for super blocks. This patch limits super block possible locations to zones accepting random writes. In particular, the zone containing the first block of the device or partition being formatted must accept random writes. Signed-off-by: Naohiro Aota --- disk-io.c | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/disk-io.c b/disk-io.c index 151eb3b5a278..74a4346cbca7 100644 --- a/disk-io.c +++ b/disk-io.c @@ -1609,6 +1609,7 @@ static int write_dev_supers(struct btrfs_fs_info *fs_info, struct btrfs_super_block *sb, struct btrfs_device *device) { + struct btrfs_zone_info *zinfo = &device->zinfo; u64 bytenr; u32 crc; int i, ret; @@ -1631,6 +1632,14 @@ static int write_dev_supers(struct btrfs_fs_info *fs_info, BTRFS_SUPER_INFO_SIZE - BTRFS_CSUM_SIZE); btrfs_csum_final(crc, &sb->csum[0]); + if (!zone_is_random_write(zinfo, fs_info->super_bytenr)) { + errno = -EIO; + error( + "failed to write super block for devid %llu: require random write zone: %m", + device->devid); + return -EIO; + } + /* * super_copy is BTRFS_SUPER_INFO_SIZE bytes and is * zero filled, we can use it directly @@ -1659,6 +1668,8 @@ static int write_dev_supers(struct btrfs_fs_info *fs_info, bytenr = btrfs_sb_offset(i); if (bytenr + BTRFS_SUPER_INFO_SIZE > device->total_bytes) break; + if (!zone_is_random_write(zinfo, bytenr)) + continue; btrfs_set_super_bytenr(sb, bytenr); From patchwork Fri Jun 7 13:17:46 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naohiro Aota X-Patchwork-Id: 10981819 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id F181A14E5 for ; Fri, 7 Jun 2019 13:19:03 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id E17B5288F7 for ; Fri, 7 Jun 2019 13:19:03 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id D63B728BBC; Fri, 7 Jun 2019 13:19:03 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.7 required=2.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 4E77F288F7 for ; Fri, 7 Jun 2019 13:19:03 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729047AbfFGNSR (ORCPT ); Fri, 7 Jun 2019 09:18:17 -0400 Received: from esa2.hgst.iphmx.com ([68.232.143.124]:56479 "EHLO esa2.hgst.iphmx.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728689AbfFGNSR (ORCPT ); Fri, 7 Jun 2019 09:18:17 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=wdc.com; i=@wdc.com; q=dns/txt; s=dkim.wdc.com; t=1559913530; x=1591449530; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=R36dF+7nZi2WAiLjfSgK/nhq+RPyynPLz4Nk77Mne08=; b=ORCf4r26XpD8cowvkyz5uSHJSYmlO6YJd7h0z3ixDRrbKcZOPxadKw30 TWeowZGuHUGhJk8xWV2XtOHPX8ZK+rovWI2R+oOPwQm0OdMMbM86mBZDa SDyoraSJwRw/+NpuDBLWxeyNp+oo4gUMOTosLXTZ0xwkfZ0d7TvLn+h0v 3+opRaEKyA+cD2NmzOshggI+mCKi66ZaJt9ogGI7vtTwEYFGu1qbJZtur 7rIOvmzLnAl61ET6cCBOUbOz/6OH0rrdjZbsixQlm3IIclLrLvVenPnWK lW7X9F7R56Jwk2QMHrC1y+ACpLp12CtlnBHyi5sMoAka+24c/VsiP+TGa g==; X-IronPort-AV: E=Sophos;i="5.63,563,1557158400"; d="scan'208";a="209675011" Received: from uls-op-cesaip02.wdc.com (HELO uls-op-cesaep02.wdc.com) ([199.255.45.15]) by ob1.hgst.iphmx.com with ESMTP; 07 Jun 2019 21:18:49 +0800 IronPort-SDR: TSp0czc6XgGVtJw41zWmdI3JZINXupRRMQDuZssPw6thlG57X5DbJxChz4v+pmnCZSxv22XWYQ GL3zzmhztjOZKpmsjYa2H7O+cEEGfU8SSQZsY1uGMozbw26mvzsULf8hftWfqZGtcbXR5VCmwn z9gYa5tQtxcHFnM2VF6khiSe9kF3ZTdNH6vnXGPzNwpz7AgpZDffjSc+TngpIkpKD+BkcznDFh uLwYzEX9CYuiUhvq9+5uslwB1lz2Csgh8YT8JUlgLZZdzq35B3Q9Hs10qaVSRkWIonP3Xmu7Xy Owy4FF6bQKcu39VeNqMWvC08 Received: from uls-op-cesaip02.wdc.com ([10.248.3.37]) by uls-op-cesaep02.wdc.com with ESMTP; 07 Jun 2019 05:55:33 -0700 IronPort-SDR: yGleyCxwzuT+RVjVyFlh1DV8xHaWda2b8IxHb2Ai7z3KXVw3UUCtkZEnVt+LbdM5v4QdGZQTVN bfufgj2QUAkWFxAYAveIHT5pJnW+4tdoZrHu9XBpPZeYh62AArkzIVyU82XO4gM2Q0fdbwmvKS nYVgsCOMycZ/BKvtkk5GTr8NNdrw4ufQKB5gs0QE+uAUboNzVGEZXsJ+yXfJr3wBuAjg5QyvJ+ wo/MnNyKsU+oDSHwlhngOmcwx6/x2RZsmbwSfGcVp1SrYbTLPF/JEHpXoUWLvboP/26CQWtFH/ 9fA= Received: from naota.dhcp.fujisawa.hgst.com (HELO naota.fujisawa.hgst.com) ([10.149.53.115]) by uls-op-cesaip02.wdc.com with ESMTP; 07 Jun 2019 06:18:15 -0700 From: Naohiro Aota To: linux-btrfs@vger.kernel.org, David Sterba Cc: Chris Mason , Josef Bacik , Qu Wenruo , Nikolay Borisov , linux-kernel@vger.kernel.org, Hannes Reinecke , linux-fsdevel@vger.kernel.org, Damien Le Moal , =?utf-8?q?Matias_Bj=C3=B8rling?= , Johannes Thumshirn , Bart Van Assche , Naohiro Aota Subject: [PATCH 07/12] btrfs-progs: support discarding zoned device Date: Fri, 7 Jun 2019 22:17:46 +0900 Message-Id: <20190607131751.5359-7-naohiro.aota@wdc.com> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20190607131751.5359-1-naohiro.aota@wdc.com> References: <20190607131025.31996-1-naohiro.aota@wdc.com> <20190607131751.5359-1-naohiro.aota@wdc.com> MIME-Version: 1.0 Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP All zones of zoned block devices should be reset before writing. Support this by considering zone reset as a special case of block discard and block zeroing. Of note is that only zones accepting random writes can be zeroed. Signed-off-by: Naohiro Aota --- utils.c | 94 +++++++++++++++++++++++++++++++++++++++++++++++++++++---- 1 file changed, 88 insertions(+), 6 deletions(-) diff --git a/utils.c b/utils.c index a26fe7a5743c..c375b32953f7 100644 --- a/utils.c +++ b/utils.c @@ -123,6 +123,37 @@ static int discard_range(int fd, u64 start, u64 len) return 0; } +/* + * Discard blocks in the zones of a zoned block device. + * Process this with zone size granularity so that blocks in + * conventional zones are discarded using discard_range and + * blocks in sequential zones are discarded though a zone reset. + */ +static int discard_zones(int fd, struct btrfs_zone_info *zinfo) +{ +#ifdef BTRFS_ZONED + unsigned int i; + + /* Zone size granularity */ + for (i = 0; i < zinfo->nr_zones; i++) { + if (zinfo->zones[i].type == BLK_ZONE_TYPE_CONVENTIONAL) { + discard_range(fd, zinfo->zones[i].start << 9, + zinfo->zone_size); + } else if (zinfo->zones[i].cond != BLK_ZONE_COND_EMPTY) { + struct blk_zone_range range = { + zinfo->zones[i].start, + zinfo->zone_size >> 9 }; + if (ioctl(fd, BLKRESETZONE, &range) < 0) + return errno; + } + } + + return 0; +#else + return -EIO; +#endif +} + /* * Discard blocks in the given range in 1G chunks, the process is interruptible */ @@ -205,8 +236,38 @@ static int zero_blocks(int fd, off_t start, size_t len) #define ZERO_DEV_BYTES SZ_2M +static int zero_zone_blocks(int fd, struct btrfs_zone_info *zinfo, + off_t start, size_t len) +{ + size_t zone_len = zinfo->zone_size; + off_t ofst = start; + size_t count; + int ret; + + /* Make sure that zero_blocks does not write sequential zones */ + while (len > 0) { + + /* Limit zero_blocks to a single zone */ + count = min_t(size_t, len, zone_len); + if (count > zone_len - (ofst & (zone_len - 1))) + count = zone_len - (ofst & (zone_len - 1)); + + if (zone_is_random_write(zinfo, ofst)) { + ret = zero_blocks(fd, ofst, count); + if (ret != 0) + return ret; + } + + len -= count; + ofst += count; + } + + return 0; +} + /* don't write outside the device by clamping the region to the device size */ -static int zero_dev_clamped(int fd, off_t start, ssize_t len, u64 dev_size) +static int zero_dev_clamped(int fd, struct btrfs_zone_info *zinfo, + off_t start, ssize_t len, u64 dev_size) { off_t end = max(start, start + len); @@ -219,6 +280,9 @@ static int zero_dev_clamped(int fd, off_t start, ssize_t len, u64 dev_size) start = min_t(u64, start, dev_size); end = min_t(u64, end, dev_size); + if (zinfo->model != ZONED_NONE) + return zero_zone_blocks(fd, zinfo, start, end - start); + return zero_blocks(fd, start, end - start); } @@ -566,6 +630,7 @@ int btrfs_get_zone_info(int fd, const char *file, int hmzoned, int btrfs_prepare_device(int fd, const char *file, u64 *block_count_ret, u64 max_block_count, unsigned opflags) { + struct btrfs_zone_info zinfo; u64 block_count; struct stat st; int i, ret; @@ -584,13 +649,30 @@ int btrfs_prepare_device(int fd, const char *file, u64 *block_count_ret, if (max_block_count) block_count = min(block_count, max_block_count); + ret = btrfs_get_zone_info(fd, file, opflags & PREP_DEVICE_HMZONED, + &zinfo); + if (ret < 0) + return 1; + if (opflags & PREP_DEVICE_DISCARD) { /* * We intentionally ignore errors from the discard ioctl. It * is not necessary for the mkfs functionality but just an - * optimization. + * optimization. However, we cannot ignore zone discard (reset) + * errors for a zoned block device as this could result in the + * inability to write to non-empty sequential zones of the + * device. */ - if (discard_range(fd, 0, 0) == 0) { + if (zinfo.model != ZONED_NONE) { + printf("Resetting device zones %s (%u zones) ...\n", + file, zinfo.nr_zones); + if (discard_zones(fd, &zinfo)) { + fprintf(stderr, + "ERROR: failed to reset device '%s' zones\n", + file); + return 1; + } + } else if (discard_range(fd, 0, 0) == 0) { if (opflags & PREP_DEVICE_VERBOSE) printf("Performing full device TRIM %s (%s) ...\n", file, pretty_size(block_count)); @@ -598,12 +680,12 @@ int btrfs_prepare_device(int fd, const char *file, u64 *block_count_ret, } } - ret = zero_dev_clamped(fd, 0, ZERO_DEV_BYTES, block_count); + ret = zero_dev_clamped(fd, &zinfo, 0, ZERO_DEV_BYTES, block_count); for (i = 0 ; !ret && i < BTRFS_SUPER_MIRROR_MAX; i++) - ret = zero_dev_clamped(fd, btrfs_sb_offset(i), + ret = zero_dev_clamped(fd, &zinfo, btrfs_sb_offset(i), BTRFS_SUPER_INFO_SIZE, block_count); if (!ret && (opflags & PREP_DEVICE_ZERO_END)) - ret = zero_dev_clamped(fd, block_count - ZERO_DEV_BYTES, + ret = zero_dev_clamped(fd, &zinfo, block_count - ZERO_DEV_BYTES, ZERO_DEV_BYTES, block_count); if (ret < 0) { From patchwork Fri Jun 7 13:17:47 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naohiro Aota X-Patchwork-Id: 10981813 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id B958A6C5 for ; Fri, 7 Jun 2019 13:18:53 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id AA86928BA5 for ; Fri, 7 Jun 2019 13:18:53 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 9F1A628C02; Fri, 7 Jun 2019 13:18:53 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.7 required=2.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 16C9E28BA5 for ; Fri, 7 Jun 2019 13:18:53 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729362AbfFGNSs (ORCPT ); Fri, 7 Jun 2019 09:18:48 -0400 Received: from esa2.hgst.iphmx.com ([68.232.143.124]:56479 "EHLO esa2.hgst.iphmx.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729103AbfFGNSS (ORCPT ); Fri, 7 Jun 2019 09:18:18 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=wdc.com; i=@wdc.com; q=dns/txt; s=dkim.wdc.com; t=1559913533; x=1591449533; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=mupKjZhti+RJeT5nPjQ1XsnqAXflA8i/lJpWsbDUs5g=; b=q8ust0ihOp3oa1FRY4OOh0eNdCI2fpU69P3F5W171ZA7MHgUpA7X8W74 oLRJ93xnAz+MjVdC/RW4Dbm7VJVLaNCA05vQJY5+o9y1mH1e9GAdwEQ6h jOtay/WzqvUf+EfJ/mMd18scRurQOf9DULxUXIJVIx/loEvH0CTecS/qD vpPtMNXOq2dpAOVZmGHbHvaKDL0ZvczFF1Kj73zQbDjdtynCtuKmOoz/Q ai6BxHJf+JeGtmTx+Sjsf6Gf+QdMtPTEqAt6Z+bwKSzDGReglA0qtDQNc /HJeXaqVU1KPm2sZqBcr228TjKPR7Fbm0PYrduydFb0JK4Fa/S4NbNeSa Q==; X-IronPort-AV: E=Sophos;i="5.63,563,1557158400"; d="scan'208";a="209675016" Received: from uls-op-cesaip02.wdc.com (HELO uls-op-cesaep02.wdc.com) ([199.255.45.15]) by ob1.hgst.iphmx.com with ESMTP; 07 Jun 2019 21:18:53 +0800 IronPort-SDR: QL7PSejTigskEbVuyqF3ANhyCwH4qzVUVF3d19D7iq8Kcxjrw7GYB+ulxgmzLHDyag5rzPGi/5 V/E8END2kTUagitlIY13yFx/fRyXxq8T4kHzz6uKZ7pbOQzlNyED4sQh9wEVCHoogq9Ofm8PIZ EEgQTnUU0cUWfyDTM2hukuiFMZExdUocW6u42eX1BKNKUc5xtbVUFfO6Y8oIN4BtSJpXC+vCF7 KtFQsyoW0UT4nF4SLxuSESWmDNyIi4fogfZvPPLyEC0qCbnSceSwMIaGCxxMYXpN18eyIQx1CI tWFq097EcXeHtBRBgUELnkRb Received: from uls-op-cesaip02.wdc.com ([10.248.3.37]) by uls-op-cesaep02.wdc.com with ESMTP; 07 Jun 2019 05:55:35 -0700 IronPort-SDR: 9McBzwsmq/zufQcdeMvV/PRCGlDdl9a1nWqwfshZWXCEIRHBBCf+LvqkEO6pMUZKolgqHv+YS6 +kQp3uUEVlBkLbCAp5qqM/VSPsq5B1aF2OZbdBlo92ktrrZdA5bIusVhUBglfr8jWk/WYoggi+ ZUMrMm6j268gw7VxbMjegABSrkyZNdwVgfCde9UvGwFPYUPTOcl+78+vabUpN65eN8L6nHGnpm M84zKrlOXuOSD2z+3ycyNeIjzrSZpY/NXF8CFw2CrV2LYi+c0J8ebL2mxqU5S6hj397Nf2Pt7Q bhQ= Received: from naota.dhcp.fujisawa.hgst.com (HELO naota.fujisawa.hgst.com) ([10.149.53.115]) by uls-op-cesaip02.wdc.com with ESMTP; 07 Jun 2019 06:18:17 -0700 From: Naohiro Aota To: linux-btrfs@vger.kernel.org, David Sterba Cc: Chris Mason , Josef Bacik , Qu Wenruo , Nikolay Borisov , linux-kernel@vger.kernel.org, Hannes Reinecke , linux-fsdevel@vger.kernel.org, Damien Le Moal , =?utf-8?q?Matias_Bj=C3=B8rling?= , Johannes Thumshirn , Bart Van Assche , Naohiro Aota Subject: [PATCH 08/12] btrfs-progs: volume: align chunk allocation to zones Date: Fri, 7 Jun 2019 22:17:47 +0900 Message-Id: <20190607131751.5359-8-naohiro.aota@wdc.com> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20190607131751.5359-1-naohiro.aota@wdc.com> References: <20190607131025.31996-1-naohiro.aota@wdc.com> <20190607131751.5359-1-naohiro.aota@wdc.com> MIME-Version: 1.0 Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP To facilitate support for zoned block devices in the extent buffer allocation, a zoned block device chunk is always aligned to a zone of the device. With this, the zone write pointer location simply becomes a hint to allocate new buffers. Signed-off-by: Naohiro Aota --- volumes.c | 79 ++++++++++++++++++++++++++++++++++++++++++++++++++++--- 1 file changed, 75 insertions(+), 4 deletions(-) diff --git a/volumes.c b/volumes.c index f6d1b1e9dc7f..64b42643390b 100644 --- a/volumes.c +++ b/volumes.c @@ -399,6 +399,34 @@ int btrfs_scan_one_device(int fd, const char *path, return ret; } +/* zone size is ensured to be power of 2 */ +static u64 btrfs_zone_align(struct btrfs_zone_info *zinfo, u64 val) +{ + if (zinfo && zinfo->zone_size) + return (val + zinfo->zone_size - 1) & ~(zinfo->zone_size - 1); + return val; +} + +static bool check_dev_zone(struct btrfs_zone_info *zinfo, u64 physical, + u64 num_bytes) +{ + u64 zone_size = zinfo->zone_size; + int zone_is_random; + + WARN_ON(!IS_ALIGNED(num_bytes, zone_size)); + zone_is_random = zone_is_random_write(zinfo, physical); + + while (num_bytes) { + if (zone_is_random != zone_is_random_write(zinfo, physical)) + return false; + + physical += zone_size; + num_bytes -= zone_size; + } + + return true; +} + /* * find_free_dev_extent_start - find free space in the specified device * @device: the device which we search the free space in @@ -428,6 +456,7 @@ static int find_free_dev_extent_start(struct btrfs_device *device, struct btrfs_root *root = device->dev_root; struct btrfs_dev_extent *dev_extent; struct btrfs_path *path; + struct btrfs_zone_info *zinfo = &device->zinfo; u64 hole_size; u64 max_hole_start; u64 max_hole_size; @@ -445,6 +474,7 @@ static int find_free_dev_extent_start(struct btrfs_device *device, */ min_search_start = max(root->fs_info->alloc_start, (u64)SZ_1M); search_start = max(search_start, min_search_start); + search_start = btrfs_zone_align(zinfo, search_start); path = btrfs_alloc_path(); if (!path) @@ -497,6 +527,18 @@ static int find_free_dev_extent_start(struct btrfs_device *device, goto next; if (key.offset > search_start) { + if (zinfo && zinfo->zone_size) { + while (key.offset > search_start) { + hole_size = key.offset - search_start; + if (hole_size < num_bytes) + break; + if (check_dev_zone(zinfo, search_start, + num_bytes)) + break; + search_start += zinfo->zone_size; + } + } + hole_size = key.offset - search_start; /* @@ -527,7 +569,8 @@ static int find_free_dev_extent_start(struct btrfs_device *device, extent_end = key.offset + btrfs_dev_extent_length(l, dev_extent); if (extent_end > search_start) - search_start = extent_end; + search_start = btrfs_zone_align(&device->zinfo, + extent_end); next: path->slots[0]++; cond_resched(); @@ -539,6 +582,18 @@ next: * search_end may be smaller than search_start. */ if (search_end > search_start) { + if (zinfo && zinfo->zone_size) { + while (search_end > search_start) { + hole_size = search_end - search_start; + if (hole_size < num_bytes) + break; + if (check_dev_zone(zinfo, search_start, + num_bytes)) + break; + search_start += zinfo->zone_size; + } + } + hole_size = search_end - search_start; if (hole_size > max_hole_size) { @@ -582,6 +637,9 @@ int btrfs_insert_dev_extent(struct btrfs_trans_handle *trans, struct extent_buffer *leaf; struct btrfs_key key; + /* Align to zone for a zoned block device */ + start = btrfs_zone_align(&device->zinfo, start); + path = btrfs_alloc_path(); if (!path) return -ENOMEM; @@ -1065,9 +1123,15 @@ int btrfs_alloc_chunk(struct btrfs_trans_handle *trans, btrfs_super_stripesize(info->super_copy)); } - /* we don't want a chunk larger than 10% of the FS */ - percent_max = div_factor(btrfs_super_total_bytes(info->super_copy), 1); - max_chunk_size = min(percent_max, max_chunk_size); + if (info->fs_devices->hmzoned) { + /* Zoned mode uses zone aligned chunks */ + calc_size = info->fs_devices->zone_size; + max_chunk_size = calc_size * num_stripes; + } else { + /* we don't want a chunk larger than 10% of the FS */ + percent_max = div_factor(btrfs_super_total_bytes(info->super_copy), 1); + max_chunk_size = min(percent_max, max_chunk_size); + } again: if (chunk_bytes_by_type(type, calc_size, num_stripes, sub_stripes) > @@ -1147,7 +1211,9 @@ again: *num_bytes = chunk_bytes_by_type(type, calc_size, num_stripes, sub_stripes); index = 0; + dev_offset = 0; while(index < num_stripes) { + size_t zone_size = device->zinfo.zone_size; struct btrfs_stripe *stripe; BUG_ON(list_empty(&private_devs)); cur = private_devs.next; @@ -1158,11 +1224,16 @@ again: (index == num_stripes - 1)) list_move_tail(&device->dev_list, dev_list); + if (device->zinfo.zone_size) + calc_size = device->zinfo.zone_size; + ret = btrfs_alloc_dev_extent(trans, device, key.offset, calc_size, &dev_offset); if (ret < 0) goto out_chunk_map; + WARN_ON(zone_size && !IS_ALIGNED(dev_offset, zone_size)); + device->bytes_used += calc_size; ret = btrfs_update_device(trans, device); if (ret < 0) From patchwork Fri Jun 7 13:17:48 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naohiro Aota X-Patchwork-Id: 10981797 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 215C914E5 for ; Fri, 7 Jun 2019 13:18:24 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 11F2028B7A for ; Fri, 7 Jun 2019 13:18:24 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 066BB28BA5; Fri, 7 Jun 2019 13:18:24 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.7 required=2.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id B720628BFA for ; Fri, 7 Jun 2019 13:18:22 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729176AbfFGNSV (ORCPT ); Fri, 7 Jun 2019 09:18:21 -0400 Received: from esa2.hgst.iphmx.com ([68.232.143.124]:56479 "EHLO esa2.hgst.iphmx.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729165AbfFGNSV (ORCPT ); Fri, 7 Jun 2019 09:18:21 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=wdc.com; i=@wdc.com; q=dns/txt; s=dkim.wdc.com; t=1559913536; x=1591449536; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=e4NWtweua7TkhXD0M6sRjN0MzNkFoyAD3U+9vqG8Tws=; b=QIZAnzZRmQxsijP03Gg6EUtjIdqHHPp+PdurIHpl9FWAtCZYzKVq9BY5 cPmgM2Kit7M6zw2QgCEGUugZaialuotl9c+SU0VIdwKdCApqyJUA5NGUd Ku9Nwgzpste4b5L4iaOcKSejXq7HUOvmdl7VbKYllk/tvXJwLiDlpRmCL mwBtoByP8z6PmvrQ+vb8LD/R4oRKpMpEw976Aqs7xsQsXKapvpT6FrPm+ 6I/I5shhr0VLCTOiGrwYypf04JWhyH4Rcbaa8TdVom2zAATOBUkbM49lS 1yfbBbr1In5CxFDWc/n2sXQV5+LAnkhg206hs1+OUVvhaiuYfOiWxurPJ Q==; X-IronPort-AV: E=Sophos;i="5.63,563,1557158400"; d="scan'208";a="209675022" Received: from uls-op-cesaip02.wdc.com (HELO uls-op-cesaep02.wdc.com) ([199.255.45.15]) by ob1.hgst.iphmx.com with ESMTP; 07 Jun 2019 21:18:56 +0800 IronPort-SDR: ZthmMGTQGMOOYvRdROOzND4Fk0u68/P+08GEijdd3cBzqxsEYWJQv3iXqTqVaumJoY1OorHC2f e64k2hQqg3JEQ4DEl3W+mp12euvp5aq1r4a3JLCsUJNsJZZhhWDmfR3KdxSWc1a56quZgxSxuy pA5bfB+yj8s8cCwD3ojbmMM9PEktrYB76MtDYHJIpV01YFq3nY2DQVcDevf6CoZJJA79SoWUAE o8NYAcjg+3nksv2Fo1zoA4kakQ7Lnw6d6PeKgD/SbuIZfndSZxLDwlTPA0UBTI0hdGnH9yF/lp YwPGDEgt4VBmWdyelObfnzph Received: from uls-op-cesaip02.wdc.com ([10.248.3.37]) by uls-op-cesaep02.wdc.com with ESMTP; 07 Jun 2019 05:55:38 -0700 IronPort-SDR: LOv5t6VjMVSb81F17hV+/uTNSWRclkqc+qgAuEhG7YA3jTskaMuADsO0/qngMWl6R9M8e/t+a5 ddwJfDeXemyhN5nlxe8dwnH3ZJ544imnAgaGqXh7hKYfaU0akHJNHCO1wK6PEcd29MNSxRu63K fmO78Z7rxgguxszfkLi4eu4CSHar61UgYGhKf3H5QHUEbVQf3OEqaObpFZZwYopC2Q/SMk7wF/ wHAqVpIhTqdMh1t7zML2pbxQ3fI5aOTUYRZjGY+DddwDHeuywjVjjT4dKFz8kOdmB+RXBs6vPL CNA= Received: from naota.dhcp.fujisawa.hgst.com (HELO naota.fujisawa.hgst.com) ([10.149.53.115]) by uls-op-cesaip02.wdc.com with ESMTP; 07 Jun 2019 06:18:19 -0700 From: Naohiro Aota To: linux-btrfs@vger.kernel.org, David Sterba Cc: Chris Mason , Josef Bacik , Qu Wenruo , Nikolay Borisov , linux-kernel@vger.kernel.org, Hannes Reinecke , linux-fsdevel@vger.kernel.org, Damien Le Moal , =?utf-8?q?Matias_Bj=C3=B8rling?= , Johannes Thumshirn , Bart Van Assche , Naohiro Aota Subject: [PATCH 09/12] btrfs-progs: do sequential allocation Date: Fri, 7 Jun 2019 22:17:48 +0900 Message-Id: <20190607131751.5359-9-naohiro.aota@wdc.com> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20190607131751.5359-1-naohiro.aota@wdc.com> References: <20190607131025.31996-1-naohiro.aota@wdc.com> <20190607131751.5359-1-naohiro.aota@wdc.com> MIME-Version: 1.0 Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Ensures that block allocation in sequential write required zones is always done sequentially using an allocation pointer which is the zone write pointer plus the number of blocks already allocated but not yet written. For conventional zones, the legacy behavior is used. Signed-off-by: Naohiro Aota --- ctree.h | 17 +++++ extent-tree.c | 186 ++++++++++++++++++++++++++++++++++++++++++++++++++ transaction.c | 16 +++++ 3 files changed, 219 insertions(+) diff --git a/ctree.h b/ctree.h index 9f79686690e0..2e828bf1250e 100644 --- a/ctree.h +++ b/ctree.h @@ -1068,15 +1068,32 @@ struct btrfs_space_info { struct list_head list; }; +/* Block group allocation types */ +enum btrfs_alloc_type { + + /* Regular first fit allocation */ + BTRFS_ALLOC_FIT = 0, + + /* + * Sequential allocation: this is for HMZONED mode and + * will result in ignoring free space before a block + * group allocation offset. + */ + BTRFS_ALLOC_SEQ = 1, +}; + struct btrfs_block_group_cache { struct cache_extent cache; struct btrfs_key key; struct btrfs_block_group_item item; struct btrfs_space_info *space_info; struct btrfs_free_space_ctl *free_space_ctl; + enum btrfs_alloc_type alloc_type; u64 bytes_super; u64 pinned; u64 flags; + u64 alloc_offset; + u64 write_offset; int cached; int ro; /* diff --git a/extent-tree.c b/extent-tree.c index e62ee8c2ba13..528c6875c8fb 100644 --- a/extent-tree.c +++ b/extent-tree.c @@ -251,6 +251,14 @@ again: if (cache->ro || !block_group_bits(cache, data)) goto new_group; + if (cache->alloc_type == BTRFS_ALLOC_SEQ) { + if (cache->key.offset - cache->alloc_offset < num) + goto new_group; + *start_ret = cache->key.objectid + cache->alloc_offset; + cache->alloc_offset += num; + return 0; + } + while(1) { ret = find_first_extent_bit(&root->fs_info->free_space_cache, last, &start, &end, EXTENT_DIRTY); @@ -277,6 +285,7 @@ out: (unsigned long long)search_start); return -ENOENT; } + printf("nospace\n"); return -ENOSPC; new_group: @@ -3039,6 +3048,176 @@ error: return ret; } +#ifdef BTRFS_ZONED +static int +btrfs_get_block_group_alloc_offset(struct btrfs_fs_info *fs_info, + struct btrfs_block_group_cache *cache) +{ + struct btrfs_device *device; + struct btrfs_mapping_tree *map_tree = &fs_info->mapping_tree; + struct cache_extent *ce; + struct map_lookup *map; + u64 logical = cache->key.objectid; + u64 length = cache->key.offset; + u64 physical = 0; + int ret = 0; + int i; + u64 zone_size = fs_info->fs_devices->zone_size; + u64 *alloc_offsets = NULL; + + if (!btrfs_fs_incompat(fs_info, HMZONED)) + return 0; + + /* Sanity check */ + if (!IS_ALIGNED(length, zone_size)) { + fprintf(stderr, "unaligned block group at %llu", logical); + return -EIO; + } + + /* Get the chunk mapping */ + ce = search_cache_extent(&map_tree->cache_tree, logical); + if (!ce) { + fprintf(stderr, "failed to find block group at %llu", logical); + return -ENOENT; + } + map = container_of(ce, struct map_lookup, ce); + + /* + * Get the zone type: if the group is mapped to a non-sequential zone, + * there is no need for the allocation offset (fit allocation is OK). + */ + device = map->stripes[0].dev; + physical = map->stripes[0].physical; + if (!zone_is_random_write(&device->zinfo, physical)) + cache->alloc_type = BTRFS_ALLOC_SEQ; + + /* check block group mapping */ + alloc_offsets = calloc(map->num_stripes, sizeof(*alloc_offsets)); + for (i = 0; i < map->num_stripes; i++) { + int is_sequential; + struct blk_zone zone; + + device = map->stripes[i].dev; + physical = map->stripes[i].physical; + + is_sequential = !zone_is_random_write(&device->zinfo, physical); + if ((is_sequential && cache->alloc_type != BTRFS_ALLOC_SEQ) || + (!is_sequential && cache->alloc_type == BTRFS_ALLOC_SEQ)) { + fprintf(stderr, + "found block group of mixed zone types"); + ret = -EIO; + goto out; + } + + if (!is_sequential) + continue; + + WARN_ON(!IS_ALIGNED(physical, zone_size)); + zone = device->zinfo.zones[physical / zone_size]; + + /* + * The group is mapped to a sequential zone. Get the zone write + * pointer to determine the allocation offset within the zone. + */ + switch (zone.cond) { + case BLK_ZONE_COND_OFFLINE: + case BLK_ZONE_COND_READONLY: + fprintf(stderr, "Offline/readonly zone %llu", + physical / fs_info->fs_devices->zone_size); + ret = -EIO; + goto out; + case BLK_ZONE_COND_EMPTY: + alloc_offsets[i] = 0; + break; + case BLK_ZONE_COND_FULL: + alloc_offsets[i] = zone_size; + break; + default: + /* Partially used zone */ + alloc_offsets[i] = ((zone.wp - zone.start) << 9); + break; + } + } + + if (cache->alloc_type != BTRFS_ALLOC_SEQ) + goto out; + + switch (map->type & BTRFS_BLOCK_GROUP_PROFILE_MASK) { + case 0: /* single */ + case BTRFS_BLOCK_GROUP_DUP: + case BTRFS_BLOCK_GROUP_RAID1: + for (i = 1; i < map->num_stripes; i++) { + if (alloc_offsets[i] != alloc_offsets[0]) { + fprintf(stderr, + "zones' write pointers mismatch\n"); + ret = -EIO; + goto out; + } + } + cache->alloc_offset = alloc_offsets[0]; + break; + case BTRFS_BLOCK_GROUP_RAID0: + cache->alloc_offset = alloc_offsets[0]; + for (i = 1; i < map->num_stripes; i++) { + cache->alloc_offset += alloc_offsets[i]; + if (alloc_offsets[0] < alloc_offsets[i]) { + fprintf(stderr, + "zones' write pointers mismatch\n"); + ret = -EIO; + goto out; + } + } + break; + case BTRFS_BLOCK_GROUP_RAID10: + cache->alloc_offset = 0; + for (i = 0; i < map->num_stripes / map->sub_stripes; i++) { + int j; + int base; + + base = i*map->sub_stripes; + for (j = 1; j < map->sub_stripes; j++) { + if (alloc_offsets[base] != + alloc_offsets[base+j]) { + fprintf(stderr, + "zones' write pointer mismatch\n"); + ret = -EIO; + goto out; + } + } + + if (alloc_offsets[0] < alloc_offsets[base]) { + fprintf(stderr, + "zones' write pointer mismatch\n"); + ret = -EIO; + goto out; + } + cache->alloc_offset += alloc_offsets[base]; + } + break; + case BTRFS_BLOCK_GROUP_RAID5: + case BTRFS_BLOCK_GROUP_RAID6: + /* RAID5/6 is not supported yet */ + default: + fprintf(stderr, "Unsupported profile %llu\n", + map->type & BTRFS_BLOCK_GROUP_PROFILE_MASK); + ret = -EINVAL; + goto out; + } + +out: + cache->write_offset = cache->alloc_offset; + free(alloc_offsets); + return ret; +} +#else +static int +btrfs_get_block_group_alloc_offset(struct btrfs_fs_info *fs_info, + struct btrfs_block_group_cache *cache) +{ + return 0; +} +#endif + int btrfs_read_block_groups(struct btrfs_root *root) { struct btrfs_path *path; @@ -3122,6 +3301,10 @@ int btrfs_read_block_groups(struct btrfs_root *root) BUG_ON(ret); cache->space_info = space_info; + ret = btrfs_get_block_group_alloc_offset(info, cache); + if (ret) + goto error; + /* use EXTENT_LOCKED to prevent merging */ set_extent_bits(block_group_cache, found_key.objectid, found_key.objectid + found_key.offset - 1, @@ -3151,6 +3334,9 @@ btrfs_add_block_group(struct btrfs_fs_info *fs_info, u64 bytes_used, u64 type, cache->key.objectid = chunk_offset; cache->key.offset = size; + ret = btrfs_get_block_group_alloc_offset(fs_info, cache); + BUG_ON(ret); + cache->key.type = BTRFS_BLOCK_GROUP_ITEM_KEY; btrfs_set_block_group_used(&cache->item, bytes_used); btrfs_set_block_group_chunk_objectid(&cache->item, diff --git a/transaction.c b/transaction.c index 138e10f0d6cc..39a52732bc71 100644 --- a/transaction.c +++ b/transaction.c @@ -129,16 +129,32 @@ int __commit_transaction(struct btrfs_trans_handle *trans, { u64 start; u64 end; + u64 next = 0; struct btrfs_fs_info *fs_info = root->fs_info; struct extent_buffer *eb; struct extent_io_tree *tree = &fs_info->extent_cache; + struct btrfs_block_group_cache *bg = NULL; int ret; while(1) { +again: ret = find_first_extent_bit(tree, 0, &start, &end, EXTENT_DIRTY); if (ret) break; + bg = btrfs_lookup_first_block_group(fs_info, start); + BUG_ON(!bg); + if (bg->alloc_type == BTRFS_ALLOC_SEQ && + bg->key.objectid + bg->write_offset < start) { + next = bg->key.objectid + bg->write_offset; + BUG_ON(next + fs_info->nodesize > start); + eb = btrfs_find_create_tree_block(fs_info, next); + btrfs_mark_buffer_dirty(eb); + free_extent_buffer(eb); + goto again; + } + if (bg->alloc_type == BTRFS_ALLOC_SEQ) + bg->write_offset += (end + 1 - start); while(start <= end) { eb = find_first_extent_buffer(tree, start); BUG_ON(!eb || eb->start != start); From patchwork Fri Jun 7 13:17:49 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naohiro Aota X-Patchwork-Id: 10981803 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 78C6F15E6 for ; Fri, 7 Jun 2019 13:18:36 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 67F9428B7A for ; Fri, 7 Jun 2019 13:18:36 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 5BF9628BBC; Fri, 7 Jun 2019 13:18:36 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.7 required=2.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id AD69028C04 for ; Fri, 7 Jun 2019 13:18:35 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729226AbfFGNSY (ORCPT ); Fri, 7 Jun 2019 09:18:24 -0400 Received: from esa2.hgst.iphmx.com ([68.232.143.124]:56479 "EHLO esa2.hgst.iphmx.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729213AbfFGNSX (ORCPT ); Fri, 7 Jun 2019 09:18:23 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=wdc.com; i=@wdc.com; q=dns/txt; s=dkim.wdc.com; t=1559913540; x=1591449540; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=iPnY+83avGj3yJ6GvVf5GDtpfB2HkmMtBeQYtYkFtCA=; b=dhkfs/yCkxli/vSL4lI561yxGXW8nsuaC6bbtH2QtgpvlfnzTdandgN1 WMchXRpHwv/LR29QLhwmyzZ4U79bWv+ESrU1bGyP5NhquBDBL1cdiJdlV iNSgJG7H9CeKCjqDez415Ov5XjYEXi0T6HyQYnQTxhfD3E2a+MfWI3f8p WVSZV+ppqJ2Quw74RFwPrwK/NF62p1rk+lIL4vyaT+oz4N9WZ2Lh5J0PK jRbaq9AzocK0LsqYTG2S7vgT/5W1GIUziE0KYnsg4YR7tPzla11UYCVu1 OxLHPLJw420bNGBZ4g3uenMdFomE2i452HPkIfU7iEDmBQUk6bnfjlGK5 w==; X-IronPort-AV: E=Sophos;i="5.63,563,1557158400"; d="scan'208";a="209675029" Received: from uls-op-cesaip02.wdc.com (HELO uls-op-cesaep02.wdc.com) ([199.255.45.15]) by ob1.hgst.iphmx.com with ESMTP; 07 Jun 2019 21:19:00 +0800 IronPort-SDR: vJUeiJMjh7t5JmiMPjQ+lH1OhAaVFWIC4Vvx6UM4KWCDwySbrj7DSmgKksbZALyvz5TDpbBAad OXIqeKbmme67tlZbxEx1tHaPj37QOTDLLZb+OgoKRM1DvQenVz8spBIlMU4k8MvcI3FryjRXo9 SKaE707xwV4PI1xSSN3EKz+qr4iEA0Pkl9Roay/fkOajNh6WZaCNlAnC8kigMnE9kFY+83L+zV ARVA5On+/mUA6fPQGYjX06PhzSSQOcNsNzz+cD5LYRLwK+f0Jp7so+T4davJqpnrBcJUlx9hfn KpcTE+6xPtcn5ySF+DMPmW18 Received: from uls-op-cesaip02.wdc.com ([10.248.3.37]) by uls-op-cesaep02.wdc.com with ESMTP; 07 Jun 2019 05:55:40 -0700 IronPort-SDR: zYF9CF8Fzm1lHiV3A3KmeqXQyik2i6J5T3nJ/725Knt6b6kkQXFan4nNmEL9aMiLdPiCO17qeh CV/7PmRS3oGke77YGXDan9iyuDX62Mnanh70ID+EhhKXKOwLKS4ctRdlCgMz66lqsSfdxiXvYO V3Mz2QE35cSRsMIbO7Nzua8It+xVqCDWLZVlo/n79y3VKupDcjOpAJHB9OOp0MseBXpQLvuC4e fx46fUkwzdf16Jh91P7AmT8GdZwUMgpI+AIZgJwfydQig4+mGpnpxUpFJjrfsDY4F/1e7u///w S/Y= Received: from naota.dhcp.fujisawa.hgst.com (HELO naota.fujisawa.hgst.com) ([10.149.53.115]) by uls-op-cesaip02.wdc.com with ESMTP; 07 Jun 2019 06:18:21 -0700 From: Naohiro Aota To: linux-btrfs@vger.kernel.org, David Sterba Cc: Chris Mason , Josef Bacik , Qu Wenruo , Nikolay Borisov , linux-kernel@vger.kernel.org, Hannes Reinecke , linux-fsdevel@vger.kernel.org, Damien Le Moal , =?utf-8?q?Matias_Bj=C3=B8rling?= , Johannes Thumshirn , Bart Van Assche , Naohiro Aota Subject: [PATCH 10/12] btrfs-progs: mkfs: Zoned block device support Date: Fri, 7 Jun 2019 22:17:49 +0900 Message-Id: <20190607131751.5359-10-naohiro.aota@wdc.com> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20190607131751.5359-1-naohiro.aota@wdc.com> References: <20190607131025.31996-1-naohiro.aota@wdc.com> <20190607131751.5359-1-naohiro.aota@wdc.com> MIME-Version: 1.0 Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP This patch makes the size of the temporary system group chunk equal to the device zone size. It also enables PREP_DEVICE_HMZONED if the user enables the HMZONED feature. Enabling HMZONED feature is done using option "-O hmzoned". This feature is incompatible for now with source directory setup. Signed-off-by: Naohiro Aota --- mkfs/common.c | 12 +++++++----- mkfs/common.h | 1 + mkfs/main.c | 45 +++++++++++++++++++++++++++++++++++++++------ 3 files changed, 47 insertions(+), 11 deletions(-) diff --git a/mkfs/common.c b/mkfs/common.c index f7e3badcf2b9..12af54c1d886 100644 --- a/mkfs/common.c +++ b/mkfs/common.c @@ -152,6 +152,7 @@ int make_btrfs(int fd, struct btrfs_mkfs_config *cfg) int skinny_metadata = !!(cfg->features & BTRFS_FEATURE_INCOMPAT_SKINNY_METADATA); u64 num_bytes; + u64 system_group_size; buf = malloc(sizeof(*buf) + max(cfg->sectorsize, cfg->nodesize)); if (!buf) @@ -312,12 +313,14 @@ int make_btrfs(int fd, struct btrfs_mkfs_config *cfg) btrfs_set_item_offset(buf, btrfs_item_nr(nritems), itemoff); btrfs_set_item_size(buf, btrfs_item_nr(nritems), item_size); + system_group_size = (cfg->features & BTRFS_FEATURE_INCOMPAT_HMZONED) ? + cfg->zone_size : BTRFS_MKFS_SYSTEM_GROUP_SIZE; + dev_item = btrfs_item_ptr(buf, nritems, struct btrfs_dev_item); btrfs_set_device_id(buf, dev_item, 1); btrfs_set_device_generation(buf, dev_item, 0); btrfs_set_device_total_bytes(buf, dev_item, num_bytes); - btrfs_set_device_bytes_used(buf, dev_item, - BTRFS_MKFS_SYSTEM_GROUP_SIZE); + btrfs_set_device_bytes_used(buf, dev_item, system_group_size); btrfs_set_device_io_align(buf, dev_item, cfg->sectorsize); btrfs_set_device_io_width(buf, dev_item, cfg->sectorsize); btrfs_set_device_sector_size(buf, dev_item, cfg->sectorsize); @@ -345,7 +348,7 @@ int make_btrfs(int fd, struct btrfs_mkfs_config *cfg) btrfs_set_item_size(buf, btrfs_item_nr(nritems), item_size); chunk = btrfs_item_ptr(buf, nritems, struct btrfs_chunk); - btrfs_set_chunk_length(buf, chunk, BTRFS_MKFS_SYSTEM_GROUP_SIZE); + btrfs_set_chunk_length(buf, chunk, system_group_size); btrfs_set_chunk_owner(buf, chunk, BTRFS_EXTENT_TREE_OBJECTID); btrfs_set_chunk_stripe_len(buf, chunk, BTRFS_STRIPE_LEN); btrfs_set_chunk_type(buf, chunk, BTRFS_BLOCK_GROUP_SYSTEM); @@ -411,8 +414,7 @@ int make_btrfs(int fd, struct btrfs_mkfs_config *cfg) (unsigned long)btrfs_dev_extent_chunk_tree_uuid(dev_extent), BTRFS_UUID_SIZE); - btrfs_set_dev_extent_length(buf, dev_extent, - BTRFS_MKFS_SYSTEM_GROUP_SIZE); + btrfs_set_dev_extent_length(buf, dev_extent, system_group_size); nritems++; btrfs_set_header_bytenr(buf, cfg->blocks[MKFS_DEV_TREE]); diff --git a/mkfs/common.h b/mkfs/common.h index 28912906d0a9..d0e4c7b2c906 100644 --- a/mkfs/common.h +++ b/mkfs/common.h @@ -53,6 +53,7 @@ struct btrfs_mkfs_config { u64 features; /* Size of the filesystem in bytes */ u64 num_bytes; + u64 zone_size; /* Output fields, set during creation */ diff --git a/mkfs/main.c b/mkfs/main.c index 93c0b71c864e..cbfd45bee836 100644 --- a/mkfs/main.c +++ b/mkfs/main.c @@ -61,8 +61,12 @@ static int create_metadata_block_groups(struct btrfs_root *root, int mixed, u64 bytes_used; u64 chunk_start = 0; u64 chunk_size = 0; + u64 system_group_size = 0; int ret; + system_group_size = fs_info->fs_devices->hmzoned ? + fs_info->fs_devices->zone_size : BTRFS_MKFS_SYSTEM_GROUP_SIZE; + trans = btrfs_start_transaction(root, 1); BUG_ON(IS_ERR(trans)); bytes_used = btrfs_super_bytes_used(fs_info->super_copy); @@ -75,8 +79,8 @@ static int create_metadata_block_groups(struct btrfs_root *root, int mixed, ret = btrfs_make_block_group(trans, fs_info, bytes_used, BTRFS_BLOCK_GROUP_SYSTEM, BTRFS_BLOCK_RESERVED_1M_FOR_SUPER, - BTRFS_MKFS_SYSTEM_GROUP_SIZE); - allocation->system += BTRFS_MKFS_SYSTEM_GROUP_SIZE; + system_group_size); + allocation->system += system_group_size; if (ret) return ret; @@ -761,6 +765,7 @@ int main(int argc, char **argv) int metadata_profile_opt = 0; int discard = 1; int ssd = 0; + int hmzoned = 0; int force_overwrite = 0; char *source_dir = NULL; bool source_dir_set = false; @@ -774,6 +779,7 @@ int main(int argc, char **argv) u64 features = BTRFS_MKFS_DEFAULT_FEATURES; struct mkfs_allocation allocation = { 0 }; struct btrfs_mkfs_config mkfs_cfg; + u64 system_group_size; while(1) { int c; @@ -896,6 +902,8 @@ int main(int argc, char **argv) if (dev_cnt == 0) print_usage(1); + hmzoned = features & BTRFS_FEATURE_INCOMPAT_HMZONED; + if (source_dir_set && dev_cnt > 1) { error("the option -r is limited to a single device"); goto error; @@ -905,6 +913,11 @@ int main(int argc, char **argv) goto error; } + if (source_dir_set && hmzoned) { + error("The -r and hmzoned feature are incompatible\n"); + exit(1); + } + if (*fs_uuid) { uuid_t dummy_uuid; @@ -936,6 +949,16 @@ int main(int argc, char **argv) file = argv[optind++]; ssd = is_ssd(file); + if (hmzoned) { + if (zoned_model(file) == ZONED_NONE) { + error("%s: not a zoned block device\n", file); + exit(1); + } + if (!zone_size(file)) { + error("%s: zone size undefined\n", file); + exit(1); + } + } /* * Set default profiles according to number of added devices. @@ -1087,7 +1110,8 @@ int main(int argc, char **argv) ret = btrfs_prepare_device(fd, file, &dev_block_count, block_count, (zero_end ? PREP_DEVICE_ZERO_END : 0) | (discard ? PREP_DEVICE_DISCARD : 0) | - (verbose ? PREP_DEVICE_VERBOSE : 0)); + (verbose ? PREP_DEVICE_VERBOSE : 0) | + (hmzoned ? PREP_DEVICE_HMZONED : 0)); if (ret) goto error; if (block_count && block_count > dev_block_count) { @@ -1098,9 +1122,11 @@ int main(int argc, char **argv) } /* To create the first block group and chunk 0 in make_btrfs */ - if (dev_block_count < BTRFS_MKFS_SYSTEM_GROUP_SIZE) { + system_group_size = hmzoned ? + zone_size(file) : BTRFS_MKFS_SYSTEM_GROUP_SIZE; + if (dev_block_count < system_group_size) { error("device is too small to make filesystem, must be at least %llu", - (unsigned long long)BTRFS_MKFS_SYSTEM_GROUP_SIZE); + (unsigned long long)system_group_size); goto error; } @@ -1116,6 +1142,7 @@ int main(int argc, char **argv) mkfs_cfg.sectorsize = sectorsize; mkfs_cfg.stripesize = stripesize; mkfs_cfg.features = features; + mkfs_cfg.zone_size = zone_size(file); ret = make_btrfs(fd, &mkfs_cfg); if (ret) { @@ -1126,6 +1153,7 @@ int main(int argc, char **argv) fs_info = open_ctree_fs_info(file, 0, 0, 0, OPEN_CTREE_WRITES | OPEN_CTREE_TEMPORARY_SUPER); + if (!fs_info) { error("open ctree failed"); goto error; @@ -1199,7 +1227,8 @@ int main(int argc, char **argv) block_count, (verbose ? PREP_DEVICE_VERBOSE : 0) | (zero_end ? PREP_DEVICE_ZERO_END : 0) | - (discard ? PREP_DEVICE_DISCARD : 0)); + (discard ? PREP_DEVICE_DISCARD : 0) | + (hmzoned ? PREP_DEVICE_HMZONED : 0)); if (ret) { goto error; } @@ -1296,6 +1325,10 @@ raid_groups: btrfs_group_profile_str(metadata_profile), pretty_size(allocation.system)); printf("SSD detected: %s\n", ssd ? "yes" : "no"); + printf("Zoned device: %s\n", hmzoned ? "yes" : "no"); + if (hmzoned) + printf("Zone size: %s\n", + pretty_size(fs_info->fs_devices->zone_size)); btrfs_parse_features_to_string(features_buf, features); printf("Incompat features: %s", features_buf); printf("\n"); From patchwork Fri Jun 7 13:17:50 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naohiro Aota X-Patchwork-Id: 10981807 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 774426C5 for ; Fri, 7 Jun 2019 13:18:40 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 685F628BBE for ; Fri, 7 Jun 2019 13:18:40 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 5CE2128BBC; Fri, 7 Jun 2019 13:18:40 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.7 required=2.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 07DC128BBE for ; Fri, 7 Jun 2019 13:18:40 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729329AbfFGNSf (ORCPT ); Fri, 7 Jun 2019 09:18:35 -0400 Received: from esa2.hgst.iphmx.com ([68.232.143.124]:56479 "EHLO esa2.hgst.iphmx.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729237AbfFGNSZ (ORCPT ); Fri, 7 Jun 2019 09:18:25 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=wdc.com; i=@wdc.com; q=dns/txt; s=dkim.wdc.com; t=1559913543; x=1591449543; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=7YLox0tNG9lHpTIY6mw5AUho1M9ivC0mOCOT+Ptqtag=; b=N9ghDEItGra4M3hn4J1YEmpfaYc9wUP+5be/71BtOpMut9cZp2kdLYR8 QqoYsaGt8Mym7ZQtDnwBTA7gggHMUxuUbb23vKufW3LGIcwIcg4DpLtxu NXR/PB8gGX75sGB+1sOWcpTRm5XtwdxkpM3IXjxcEGQ7rLBaZ0ls2J3nv VfYZTw4pInOwWS8g3R5WcpMW/+JSFUOw4R3iQ0bazCXFxRFyj7KdtwG5w zomhklQHEoE31PqXn7vCPFfXDViyYyrxKQCIeVw9o91fz6jSgTPuH+5aX eTFfd7knC17TwshWbd9xozsIEYEBnjupYDGp+SoQYr//UMsUXoj/k3qAs A==; X-IronPort-AV: E=Sophos;i="5.63,563,1557158400"; d="scan'208";a="209675031" Received: from uls-op-cesaip02.wdc.com (HELO uls-op-cesaep02.wdc.com) ([199.255.45.15]) by ob1.hgst.iphmx.com with ESMTP; 07 Jun 2019 21:19:03 +0800 IronPort-SDR: fhh+Eb7eEBM7K27cHL19ZtzYge3s7ME3DC4xsAzr9bGw1moJGYwXL4Qt77pXNniZMZKS00OXNF /aZwVyFzJTX80pwg3bnHoLTAkyJUa4W+diwCANjnRTYWXNfmfvCsqqP66N+qeXpe8zeZsK0H2m 1rmmdAf8+gEXukIQfG0B5IkYhyQ7wPS+/PAEvpxLIe3RcUt9NRYQkpKQvo2Ny1fKruWsdYXodi 6Pglo1qLc3ZjNw8a45yOnOtch3scUq4k+Y5gXhOB5q8tf6Hw14cUop5B96IpCBI3dmOX7tHHQh 800aOelsOYlUlvZhjYqPVtgK Received: from uls-op-cesaip02.wdc.com ([10.248.3.37]) by uls-op-cesaep02.wdc.com with ESMTP; 07 Jun 2019 05:55:42 -0700 IronPort-SDR: vbwS6gnqfZa3FR8uJ/1d5LALS/dMBFOW13K5BAVI3QDC6EPc2VxU7W6/CnqRw6a4WWOb/Y7a6S PVIYYUylJJJMchov5wzsmU7EzajGiChKTaApdkirYb0Vk5YGJbni87vTI0RY3GbyOkzwaVo6Gv D3hO8Yr65KVJTPGoMdxCofVZ7G7g+dEmzlOkcWJWncZIOW649RCU4c6FwF0n0jBULMSakAnz8c s/UzpElvM+dcJenD57r9GSaXZ7/2qsUnusp3KTMqfjF71QZ55ytwp9wFuEX/i6tbQfY/efjuF4 3PU= Received: from naota.dhcp.fujisawa.hgst.com (HELO naota.fujisawa.hgst.com) ([10.149.53.115]) by uls-op-cesaip02.wdc.com with ESMTP; 07 Jun 2019 06:18:24 -0700 From: Naohiro Aota To: linux-btrfs@vger.kernel.org, David Sterba Cc: Chris Mason , Josef Bacik , Qu Wenruo , Nikolay Borisov , linux-kernel@vger.kernel.org, Hannes Reinecke , linux-fsdevel@vger.kernel.org, Damien Le Moal , =?utf-8?q?Matias_Bj=C3=B8rling?= , Johannes Thumshirn , Bart Van Assche , Naohiro Aota Subject: [PATCH 11/12] btrfs-progs: device-add: support HMZONED device Date: Fri, 7 Jun 2019 22:17:50 +0900 Message-Id: <20190607131751.5359-11-naohiro.aota@wdc.com> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20190607131751.5359-1-naohiro.aota@wdc.com> References: <20190607131025.31996-1-naohiro.aota@wdc.com> <20190607131751.5359-1-naohiro.aota@wdc.com> MIME-Version: 1.0 Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP This patch check if the target file system is flagged as HMZONED. If it is, the device to be added is flagged PREP_DEVICE_HMZONED. Also add checks to prevent mixing non-zoned devices and zoned devices. Signed-off-by: Naohiro Aota --- cmds-device.c | 29 +++++++++++++++++++++++++++-- 1 file changed, 27 insertions(+), 2 deletions(-) diff --git a/cmds-device.c b/cmds-device.c index e3e30b6d5ded..86ffb1a2a5c2 100644 --- a/cmds-device.c +++ b/cmds-device.c @@ -57,6 +57,9 @@ static int cmd_device_add(int argc, char **argv) int discard = 1; int force = 0; int last_dev; + int res; + int hmzoned; + struct btrfs_ioctl_feature_flags feature_flags; optind = 0; while (1) { @@ -92,12 +95,33 @@ static int cmd_device_add(int argc, char **argv) if (fdmnt < 0) return 1; + res = ioctl(fdmnt, BTRFS_IOC_GET_FEATURES, &feature_flags); + if (res) { + error("error getting feature flags '%s': %m", mntpnt); + return 1; + } + hmzoned = feature_flags.incompat_flags & BTRFS_FEATURE_INCOMPAT_HMZONED; + for (i = optind; i < last_dev; i++){ struct btrfs_ioctl_vol_args ioctl_args; - int devfd, res; + int devfd; u64 dev_block_count = 0; char *path; + if (hmzoned && zoned_model(argv[i]) == ZONED_NONE) { + error("cannot add non-zoned device to HMZONED file system '%s'", + argv[i]); + ret++; + continue; + } + + if (!hmzoned && zoned_model(argv[i]) == ZONED_HOST_MANAGED) { + error("cannot add host managed zoned device to non-HMZONED file system '%s'", + argv[i]); + ret++; + continue; + } + res = test_dev_for_mkfs(argv[i], force); if (res) { ret++; @@ -113,7 +137,8 @@ static int cmd_device_add(int argc, char **argv) res = btrfs_prepare_device(devfd, argv[i], &dev_block_count, 0, PREP_DEVICE_ZERO_END | PREP_DEVICE_VERBOSE | - (discard ? PREP_DEVICE_DISCARD : 0)); + (discard ? PREP_DEVICE_DISCARD : 0) | + (hmzoned ? PREP_DEVICE_HMZONED : 0)); close(devfd); if (res) { ret++; From patchwork Fri Jun 7 13:17:51 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naohiro Aota X-Patchwork-Id: 10981799 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 070C86C5 for ; Fri, 7 Jun 2019 13:18:34 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id ECAFD28BBE for ; Fri, 7 Jun 2019 13:18:33 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id E140928C02; Fri, 7 Jun 2019 13:18:33 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.7 required=2.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 7E75F28BBE for ; Fri, 7 Jun 2019 13:18:33 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729287AbfFGNS2 (ORCPT ); Fri, 7 Jun 2019 09:18:28 -0400 Received: from esa2.hgst.iphmx.com ([68.232.143.124]:56479 "EHLO esa2.hgst.iphmx.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729274AbfFGNS1 (ORCPT ); Fri, 7 Jun 2019 09:18:27 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=wdc.com; i=@wdc.com; q=dns/txt; s=dkim.wdc.com; t=1559913547; x=1591449547; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=7OJCfl7mJcrjPMSDFaEobu5+R6OlEI3WvKTSWthUVKA=; b=cat6pKsmsGvICKvIRFSiNmsIDjb+2tYCRhFMsW/psD4i/JBUD6ASRYWc MrSq6ihFSsV1A/Sx15fDJ7A2UeCW+l4xPXh1CEjYHvOZCJdUfngtIk3+a yHV8K+EC/kcjbPfrQSXksOVkfNWO0TJLjVEoj2ZnWJzm8RpcUFccVq+m8 /bCFXK6E7J8/oedlREQTvr07WJQiGRYJ37eRQLOHvMVnxHSo5CLR1mV92 PALQtANBD+sb0cFHBFHgFjcSFSbZWhlNefj46fteFZZuJg2KUPXL6/MB4 9vIZJphdEMF4ImHluYERcOESMrQrEITidGId6Ubu0GBvo0EfFISPqLR9B g==; X-IronPort-AV: E=Sophos;i="5.63,563,1557158400"; d="scan'208";a="209675034" Received: from uls-op-cesaip02.wdc.com (HELO uls-op-cesaep02.wdc.com) ([199.255.45.15]) by ob1.hgst.iphmx.com with ESMTP; 07 Jun 2019 21:19:06 +0800 IronPort-SDR: xgi4EgEqZ/nvDaA/Y4QrDRNhQ2jkmUpl8UrzS1+ryZ+f+egYcyXt5uTbFk5mOfYb0VFvecmc/Q 1TlSlKMNZARr1/4JLsIGsAEULTWcjY+0++VKgRDa+b/94VEZzaV2hzbhGO3LGiGxKjogrX+YyJ XDUDT8/aU7ov8/HeWxsgJD73Yg0ZHPmqAQ/EnU2w6C+dszcmRDMtmTvOrZkhali59ZfwRTxNYj DOqy2leTzYKjOMlwmwbbwYcv26xlc2sVOSjdTAKQlZjPIDfuTLlj/7JY/nRakdQwSgNIMxZP1y +o/rGJKTp4Bin2wUa+mgLrEN Received: from uls-op-cesaip02.wdc.com ([10.248.3.37]) by uls-op-cesaep02.wdc.com with ESMTP; 07 Jun 2019 05:55:45 -0700 IronPort-SDR: EGR3OeGqPSc8rQAD7I2QT16I633TuWGjdQODiLRNAhQQB0qJp685MdKHrdeEnUgxVpoX3sjoGN XubpuDY4bDd/9XP1+/gJRVFWr8gzq0PSYCMxUhy+YEK/tGNV2sB7RDEPxT5kgl5/mrZ4UhfpAZ OWUqG6ADmh7hYLzNP6pqy67y/g8eermho3CKWM3F5PGlCHl4ivONBiAtcQEwGwaLWszTy3E0Cq wyUj+avjKxf6f0sSmVT8RU2fheYM+ZaWzwDMzBCupHQvIdJ7PbxV7Nt81ZJDgHPYUPKkT3sSsa zms= Received: from naota.dhcp.fujisawa.hgst.com (HELO naota.fujisawa.hgst.com) ([10.149.53.115]) by uls-op-cesaip02.wdc.com with ESMTP; 07 Jun 2019 06:18:26 -0700 From: Naohiro Aota To: linux-btrfs@vger.kernel.org, David Sterba Cc: Chris Mason , Josef Bacik , Qu Wenruo , Nikolay Borisov , linux-kernel@vger.kernel.org, Hannes Reinecke , linux-fsdevel@vger.kernel.org, Damien Le Moal , =?utf-8?q?Matias_Bj=C3=B8rling?= , Johannes Thumshirn , Bart Van Assche , Naohiro Aota Subject: [PATCH 12/12] btrfs-progs: introduce support for dev-place HMZONED device Date: Fri, 7 Jun 2019 22:17:51 +0900 Message-Id: <20190607131751.5359-12-naohiro.aota@wdc.com> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20190607131751.5359-1-naohiro.aota@wdc.com> References: <20190607131025.31996-1-naohiro.aota@wdc.com> <20190607131751.5359-1-naohiro.aota@wdc.com> MIME-Version: 1.0 Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP This patch check if the target file system is flagged as HMZONED. If it is, the device to be added is flagged PREP_DEVICE_HMZONED. Also add checks to prevent mixing non-zoned devices and zoned devices. Signed-off-by: Naohiro Aota --- cmds-replace.c | 12 +++++++++++- 1 file changed, 11 insertions(+), 1 deletion(-) diff --git a/cmds-replace.c b/cmds-replace.c index 713d200938d4..c752ceaadb77 100644 --- a/cmds-replace.c +++ b/cmds-replace.c @@ -116,6 +116,7 @@ static const char *const cmd_replace_start_usage[] = { static int cmd_replace_start(int argc, char **argv) { + struct btrfs_ioctl_feature_flags feature_flags; struct btrfs_ioctl_dev_replace_args start_args = {0}; struct btrfs_ioctl_dev_replace_args status_args = {0}; int ret; @@ -123,6 +124,7 @@ static int cmd_replace_start(int argc, char **argv) int c; int fdmnt = -1; int fddstdev = -1; + int hmzoned; char *path; char *srcdev; char *dstdev = NULL; @@ -163,6 +165,13 @@ static int cmd_replace_start(int argc, char **argv) if (fdmnt < 0) goto leave_with_error; + ret = ioctl(fdmnt, BTRFS_IOC_GET_FEATURES, &feature_flags); + if (ret) { + error("ioctl(GET_FEATURES) on '%s' returns error: %m", path); + goto leave_with_error; + } + hmzoned = feature_flags.incompat_flags & BTRFS_FEATURE_INCOMPAT_HMZONED; + /* check for possible errors before backgrounding */ status_args.cmd = BTRFS_IOCTL_DEV_REPLACE_CMD_STATUS; status_args.result = BTRFS_IOCTL_DEV_REPLACE_RESULT_NO_RESULT; @@ -257,7 +266,8 @@ static int cmd_replace_start(int argc, char **argv) strncpy((char *)start_args.start.tgtdev_name, dstdev, BTRFS_DEVICE_PATH_NAME_MAX); ret = btrfs_prepare_device(fddstdev, dstdev, &dstdev_block_count, 0, - PREP_DEVICE_ZERO_END | PREP_DEVICE_VERBOSE); + PREP_DEVICE_ZERO_END | PREP_DEVICE_VERBOSE | + (hmzoned ? PREP_DEVICE_HMZONED | PREP_DEVICE_DISCARD : 0)); if (ret) goto leave_with_error;