From patchwork Thu Aug 9 18:10:54 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naohiro Aota X-Patchwork-Id: 10561681 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id C8C9D13B4 for ; Thu, 9 Aug 2018 18:12:54 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id B445428306 for ; Thu, 9 Aug 2018 18:12:54 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id A7635283A6; Thu, 9 Aug 2018 18:12:54 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.8 required=2.0 tests=BAYES_00,DKIM_SIGNED, MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI,T_DKIM_INVALID autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 3C4522580E for ; Thu, 9 Aug 2018 18:12:53 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727107AbeHIUhL (ORCPT ); Thu, 9 Aug 2018 16:37:11 -0400 Received: from mail-pf1-f193.google.com ([209.85.210.193]:45171 "EHLO mail-pf1-f193.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726744AbeHIUhK (ORCPT ); Thu, 9 Aug 2018 16:37:10 -0400 Received: by mail-pf1-f193.google.com with SMTP id i26-v6so3204767pfo.12; Thu, 09 Aug 2018 11:11:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:from:to:cc:subject:date:message-id:in-reply-to:references; bh=As0lKa5+Tv1HOY4/e5U6uHnlYo/L2iz6KBMvgOghx1Y=; b=ZCmVuOg/8mUD/pHRWoLO2xWqalxH6ZPuvTCZH7XOdigAWIVj/2e/lmiF4WzUOOT+1f MKNLo9fyl0HMqJcX+spvYlpVsOj4L4A3+fnSvq/cIs2BmSS3y2ndxF5hjmntkcYBhdl1 S4oPoU8FnHU4otOsvZtGZWzqX2OR7pdSx6/zZ7ecTjZ/thsfkkZ8Q2p9sG8UhPMR/9gI cRHMVReU2bA4/TddrH/JBeEBn0SRRh9pEf5i6mK22etQObTQumQJnPbucz5ewp4JMlYV 2SQyVV3BX/5vAybol4xNgl9gTy5w+0ZxwbK+xtH1jTTZ7wKv5VefIyxzGwwcAVuwuMMJ KlwA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:from:to:cc:subject:date:message-id :in-reply-to:references; bh=As0lKa5+Tv1HOY4/e5U6uHnlYo/L2iz6KBMvgOghx1Y=; b=VuHONOtvh616urvewvb8mDLwDEKPBFseJOMeDCfCpnikTjcUSToB6zuc9j1gh/v/ar R55NmhcLvaY55EQJq8jQfC7Y3+ztKt2tOAG64iB20UI8aQDweZhwL/iD6y+0cDy14FkW iGQVDIYtpi0gheJbN3JVAB5x8FApJ4q3XgTOMVE70eVQw6tUAfM3UzM+6jm6uzxQ1pVA Ebusp915//PbzHTXvCxk5ViEVZb2SyvwIT0KzUakQNGF/bUwLyIUxM64l9uTOJNd/+NP SKW9+GWrkckGt05m2V6e6AjKuQXK4ddDhL7Z0AJAky6HTsz9PHtmL5oOodnDEju/sGgv BqLw== X-Gm-Message-State: AOUpUlG8LcHrruVpXMrHEWsUFxdXQ36HY0Y2LAMmFzkbiIbVEw+4OTzF V98jtk1eXgoeS0onNQF1A5o= X-Google-Smtp-Source: AA+uWPyIbjyaTx1S5zcMuNHJC26GI+Pj9ruSSvDqGqrOYJuqqKUDHKFWzYLSHJ1NaI9pf8NDc6cbfA== X-Received: by 2002:a62:1f8c:: with SMTP id l12-v6mr3482696pfj.143.1533838270075; Thu, 09 Aug 2018 11:11:10 -0700 (PDT) Received: from localhost (h101-111-148-072.catv02.itscom.jp. [101.111.148.72]) by smtp.gmail.com with ESMTPSA id i125-v6sm7017445pgc.16.2018.08.09.11.11.09 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 09 Aug 2018 11:11:09 -0700 (PDT) From: Naohiro Aota To: David Sterba , linux-btrfs@vger.kernel.org Cc: Chris Mason , Josef Bacik , linux-kernel@vger.kernel.org, Hannes Reinecke , Damien Le Moal , Bart Van Assche , Matias Bjorling , Naohiro Aota Subject: [RFC PATCH 01/12] btrfs-progs: build: Check zoned block device support Date: Fri, 10 Aug 2018 03:10:54 +0900 Message-Id: <20180809181105.12856-1-naota@elisp.net> X-Mailer: git-send-email 2.18.0 In-Reply-To: <20180809180450.5091-1-naota@elisp.net> References: <20180809180450.5091-1-naota@elisp.net> Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP If the kernel supports zoned block devices, the file /usr/include/linux/blkzoned.h will be present. Check this and define BTRFS_ZONED if the file is present. If it present, enables HMZONED feature, if not disable it. Signed-off-by: Damien Le Moal Signed-off-by: Naohiro Aota --- configure.ac | 13 +++++++++++++ 1 file changed, 13 insertions(+) diff --git a/configure.ac b/configure.ac index df02f206..616d62a1 100644 --- a/configure.ac +++ b/configure.ac @@ -207,6 +207,18 @@ else AC_DEFINE([HAVE_OWN_FIEMAP_EXTENT_SHARED_DEFINE], [0], [We did not define FIEMAP_EXTENT_SHARED]) fi +AC_CHECK_HEADER(linux/blkzoned.h, [blkzoned_found=yes], [blkzoned_found=no]) +AC_ARG_ENABLE([zoned], + AS_HELP_STRING([--disable-zoned], [disable zoned block device support]), + [], [enable_zoned=$blkzoned_found] +) + +AS_IF([test "x$enable_zoned" = xyes], [ + AC_CHECK_HEADER(linux/blkzoned.h, [], + [AC_MSG_ERROR([Couldn't find linux/blkzoned.h])]) + AC_DEFINE([BTRFS_ZONED], [1], [enable zoned block device support]) +]) + dnl Define _LIBS= and _CFLAGS= by pkg-config dnl dnl The default PKG_CHECK_MODULES() action-if-not-found is end the @@ -308,6 +320,7 @@ AC_MSG_RESULT([ btrfs-restore zstd: ${enable_zstd} Python bindings: ${enable_python} Python interpreter: ${PYTHON} + zoned device: ${enable_zoned} Type 'make' to compile. ]) From patchwork Thu Aug 9 18:10:55 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naohiro Aota X-Patchwork-Id: 10561679 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id A6B2A13B4 for ; Thu, 9 Aug 2018 18:12:50 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 908F82B7FB for ; Thu, 9 Aug 2018 18:12:50 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 824F12B814; Thu, 9 Aug 2018 18:12:50 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.8 required=2.0 tests=BAYES_00,DKIM_SIGNED, MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI,T_DKIM_INVALID autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 162272B7FB for ; Thu, 9 Aug 2018 18:12:50 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727166AbeHIUhO (ORCPT ); Thu, 9 Aug 2018 16:37:14 -0400 Received: from mail-pf1-f193.google.com ([209.85.210.193]:42310 "EHLO mail-pf1-f193.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726744AbeHIUhN (ORCPT ); Thu, 9 Aug 2018 16:37:13 -0400 Received: by mail-pf1-f193.google.com with SMTP id l9-v6so3208849pff.9; Thu, 09 Aug 2018 11:11:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:from:to:cc:subject:date:message-id:in-reply-to:references; bh=iFWTvH5MPUVacum5DjJN+Ab+qJD2T9tCP90fDGvByTY=; b=KjF9ok55YzqvtxHShY9FQqeeM80Q6/niMME0dlEZNCNsdmjOxZhy9EuhesYXlBteyG 40TWYLZSbrQt1lDZZRKg4xgUaD3rAyozBK/6qNLRrdVW94SpxWlKGsgeoKziv7gfj7eX MGPVkPULLrPFnHR9AA4k0a8E7gU5IYuO3rPOl7d5zRPyNZVVG7FizYuk6zMr/1UAIAag 2MFzOxZT3NN1vUxjDVEVkZE/QxF27UOWizn3Utal0+8fYG59YJ/ArBA9W09szM7G18GL COsWp+HRhbgRMI4eeecoSKA5HbGsB/rZuKQDK9qVVbVtjjysbhdNV0V1Zi/ODkEaqjNw CWBA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:from:to:cc:subject:date:message-id :in-reply-to:references; bh=iFWTvH5MPUVacum5DjJN+Ab+qJD2T9tCP90fDGvByTY=; b=NvleE656Y2R7lcsxLdzNfGD0mMIg8zJBH/ulkmFJg/eHeaKFdMC4RUpgX8n5n7Gjuj 7gwbeEtpZLKqd9UzD6A5Np0WSo+gJiDiZzkooRjoK7kMqtWH7NmBkt+e5ugtBvpP81O6 QUky5NE4+U7lFwuRCwcYJWWA0BFS6BN+361RkXRlw1AvEKQ8TdsqGwnhGcO0OTGp2MqP Y9MjcttxSUGMbF6z7MgfjcyFp+8XQrZDT0TbBp70hZFcj6YlAwKYbhbuLSs6YFtucNQj gPeiTo2w26LhSuNtPdI43zGIJFifPWzvDUvwy6DGUF9/VzhchU+eXkDyGM34e4XkNfsA saDA== X-Gm-Message-State: AOUpUlG9oG/gWYoaaFnjY1IB9vj5doGSwq28gie2tPGubJCXo/rQX8p1 TTLppWi21ZIaLvM83DQMank= X-Google-Smtp-Source: AA+uWPxkmkWqnq13HLCK3XmWcV1mQ9Z2NYKg6s5tUh0azi9ZEXbkhDfoocwQFdfEZJue9zPkJROIiw== X-Received: by 2002:a63:fc0a:: with SMTP id j10-v6mr3135496pgi.1.1533838273018; Thu, 09 Aug 2018 11:11:13 -0700 (PDT) Received: from localhost (h101-111-148-072.catv02.itscom.jp. [101.111.148.72]) by smtp.gmail.com with ESMTPSA id z11-v6sm16325417pff.162.2018.08.09.11.11.12 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 09 Aug 2018 11:11:12 -0700 (PDT) From: Naohiro Aota To: David Sterba , linux-btrfs@vger.kernel.org Cc: Chris Mason , Josef Bacik , linux-kernel@vger.kernel.org, Hannes Reinecke , Damien Le Moal , Bart Van Assche , Matias Bjorling , Naohiro Aota Subject: [RFC PATCH 02/12] btrfs-progs: utils: Introduce queue_param Date: Fri, 10 Aug 2018 03:10:55 +0900 Message-Id: <20180809181105.12856-2-naota@elisp.net> X-Mailer: git-send-email 2.18.0 In-Reply-To: <20180809181105.12856-1-naota@elisp.net> References: <20180809180450.5091-1-naota@elisp.net> <20180809181105.12856-1-naota@elisp.net> Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Introduce the queue_param function to get a device request queue parameter and this function to test if the device is an SSD in is_ssd(). Signed-off-by: Damien Le Moal [Naohiro] fixed error return value Signed-off-by: Naohiro Aota --- mkfs/main.c | 40 ++-------------------------------------- utils.c | 46 ++++++++++++++++++++++++++++++++++++++++++++++ utils.h | 1 + 3 files changed, 49 insertions(+), 38 deletions(-) diff --git a/mkfs/main.c b/mkfs/main.c index b76462a7..83969b4b 100644 --- a/mkfs/main.c +++ b/mkfs/main.c @@ -435,49 +435,13 @@ static int zero_output_file(int out_fd, u64 size) static int is_ssd(const char *file) { - blkid_probe probe; - char wholedisk[PATH_MAX]; - char sysfs_path[PATH_MAX]; - dev_t devno; - int fd; char rotational; int ret; - probe = blkid_new_probe_from_filename(file); - if (!probe) + ret = queue_param(file, "rotational", &rotational, 1); + if (ret < 1) return 0; - /* Device number of this disk (possibly a partition) */ - devno = blkid_probe_get_devno(probe); - if (!devno) { - blkid_free_probe(probe); - return 0; - } - - /* Get whole disk name (not full path) for this devno */ - ret = blkid_devno_to_wholedisk(devno, - wholedisk, sizeof(wholedisk), NULL); - if (ret) { - blkid_free_probe(probe); - return 0; - } - - snprintf(sysfs_path, PATH_MAX, "/sys/block/%s/queue/rotational", - wholedisk); - - blkid_free_probe(probe); - - fd = open(sysfs_path, O_RDONLY); - if (fd < 0) { - return 0; - } - - if (read(fd, &rotational, 1) < 1) { - close(fd); - return 0; - } - close(fd); - return rotational == '0'; } diff --git a/utils.c b/utils.c index d4395b1f..2212692c 100644 --- a/utils.c +++ b/utils.c @@ -65,6 +65,52 @@ static unsigned short rand_seed[3]; struct btrfs_config bconf; +/* + * Get a device request queue parameter. + */ +int queue_param(const char *file, const char *param, char *buf, size_t len) +{ + blkid_probe probe; + char wholedisk[PATH_MAX]; + char sysfs_path[PATH_MAX]; + dev_t devno; + int fd; + int ret; + + probe = blkid_new_probe_from_filename(file); + if (!probe) + return 0; + + /* Device number of this disk (possibly a partition) */ + devno = blkid_probe_get_devno(probe); + if (!devno) { + blkid_free_probe(probe); + return 0; + } + + /* Get whole disk name (not full path) for this devno */ + ret = blkid_devno_to_wholedisk(devno, + wholedisk, sizeof(wholedisk), NULL); + if (ret) { + blkid_free_probe(probe); + return 0; + } + + snprintf(sysfs_path, PATH_MAX, "/sys/block/%s/queue/%s", + wholedisk, param); + + blkid_free_probe(probe); + + fd = open(sysfs_path, O_RDONLY); + if (fd < 0) + return 0; + + len = read(fd, buf, len); + close(fd); + + return len; +} + /* * Discard the given range in one go */ diff --git a/utils.h b/utils.h index b6c00cfa..ac333095 100644 --- a/utils.h +++ b/utils.h @@ -120,6 +120,7 @@ int get_label(const char *btrfs_dev, char *label); int set_label(const char *btrfs_dev, const char *label); char *__strncpy_null(char *dest, const char *src, size_t n); +int queue_param(const char *file, const char *param, char *buf, size_t len); int is_block_device(const char *file); int is_mount_point(const char *file); int is_path_exist(const char *file); From patchwork Thu Aug 9 18:10:56 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naohiro Aota X-Patchwork-Id: 10561677 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 16ED113BB for ; Thu, 9 Aug 2018 18:12:44 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 033982B7FB for ; Thu, 9 Aug 2018 18:12:44 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id EAC522B814; Thu, 9 Aug 2018 18:12:43 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.8 required=2.0 tests=BAYES_00,DKIM_SIGNED, MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI,T_DKIM_INVALID autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 799F22B7FB for ; Thu, 9 Aug 2018 18:12:43 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727221AbeHIUhR (ORCPT ); Thu, 9 Aug 2018 16:37:17 -0400 Received: from mail-pl0-f66.google.com ([209.85.160.66]:43089 "EHLO mail-pl0-f66.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726882AbeHIUhQ (ORCPT ); Thu, 9 Aug 2018 16:37:16 -0400 Received: by mail-pl0-f66.google.com with SMTP id x6-v6so2870909plv.10; Thu, 09 Aug 2018 11:11:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:from:to:cc:subject:date:message-id:in-reply-to:references; bh=kM+SE/GanJfI/Ih58/7lkGHceeH98C7iNNYae6MdgJg=; b=Z1XIe6n1fc4wP/Mgd54n7clah+FZowPBAoU/23bYMsdQiiKBNILFO7kkkSbPwopHo4 T8a4MSKnbIR4t62E8VGtUnucT2QekgMElMxjfkRk2qADp5H1dj2JjbqVWdPWiTPolUL2 tMlOSLACPs5xMDNX+ul21b5CyURDb4rJygSPVsioGbKBr9T411/z7eHLg5Z2CU9rFqf1 6tQKRfPe2M5af8wx1anfVH6Xjzr/w1LHApwgOl9s7HR7KzD0I1twE6EMEdbHrbp2PcTh Z6GKMW9L8wzqh4EJi4oLDtVWG5bzS0uTC7AwHZmD2EiSk4GAfFzftsUdjtUt4sKKkjkS OrBQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:from:to:cc:subject:date:message-id :in-reply-to:references; bh=kM+SE/GanJfI/Ih58/7lkGHceeH98C7iNNYae6MdgJg=; b=h9gXxOeQHML9S+JBoxcILqMR8waiQq8bUkRjGILnOPCKwgQJhLay3EiaSJmciX3MwB cbEPgJseweKRoiyYf5NJAvHxOEhPhwbYZPQ3hn4iirGR2ZBNKUzcTBfjQb978TxFQepF zcsshu4QL5pf3BVUQy0TV5FSWQHqQ2eNNFKu6mIflREufINUKau0JF4I1Iw2mokrBsu+ H6+/A5yu2l5gg1ICgPddYNjGN+hansQAnSTTUfuOhsOn8f7GpoEqXypHGgO5AWkT/xhq SVYWf88/a+gSF9Bww6QrsSwhXFFF5r9zHx6CppiMEIVvsr5KJzZsI1MadFWIcg3ErOi/ aZSA== X-Gm-Message-State: AOUpUlFFeFPpBjQug5ybOHsmtAG6UjnoOVfo7ditw7uxNoqo0GfhTSPy aIWYvuPoFt8Rqplx1K2+Kjk= X-Google-Smtp-Source: AA+uWPydeaX1uhq+jJ+sHsyiskd+4NFLI7HQ1kkmYZikKMlReQWloKs9Z0OrQDCbk+6P7HfoBRpEwg== X-Received: by 2002:a17:902:8482:: with SMTP id c2-v6mr3012007plo.45.1533838275970; Thu, 09 Aug 2018 11:11:15 -0700 (PDT) Received: from localhost (h101-111-148-072.catv02.itscom.jp. [101.111.148.72]) by smtp.gmail.com with ESMTPSA id w70-v6sm13850086pgd.18.2018.08.09.11.11.14 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 09 Aug 2018 11:11:15 -0700 (PDT) From: Naohiro Aota To: David Sterba , linux-btrfs@vger.kernel.org Cc: Chris Mason , Josef Bacik , linux-kernel@vger.kernel.org, Hannes Reinecke , Damien Le Moal , Bart Van Assche , Matias Bjorling , Naohiro Aota Subject: [RFC PATCH 03/12] btrfs-progs: add new HMZONED feature flag Date: Fri, 10 Aug 2018 03:10:56 +0900 Message-Id: <20180809181105.12856-3-naota@elisp.net> X-Mailer: git-send-email 2.18.0 In-Reply-To: <20180809181105.12856-1-naota@elisp.net> References: <20180809180450.5091-1-naota@elisp.net> <20180809181105.12856-1-naota@elisp.net> Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP With this feature enabled, a zoned block device aware btrfs allocates block groups aligned to the device zones and always write in sequential zones at the zone write pointer position. Enabling this feature also force disable conversion from ext4 volumes. Note: this flag can be moved to COMPAT_RO, so that older kernel can read but not write zoned block devices formatted with btrfs. Signed-off-by: Naohiro Aota --- cmds-inspect-dump-super.c | 3 ++- ctree.h | 4 +++- fsfeatures.c | 8 ++++++++ fsfeatures.h | 2 +- libbtrfsutil/btrfs.h | 1 + 5 files changed, 15 insertions(+), 3 deletions(-) diff --git a/cmds-inspect-dump-super.c b/cmds-inspect-dump-super.c index e965267c..4e365ead 100644 --- a/cmds-inspect-dump-super.c +++ b/cmds-inspect-dump-super.c @@ -228,7 +228,8 @@ static struct readable_flag_entry incompat_flags_array[] = { DEF_INCOMPAT_FLAG_ENTRY(EXTENDED_IREF), DEF_INCOMPAT_FLAG_ENTRY(RAID56), DEF_INCOMPAT_FLAG_ENTRY(SKINNY_METADATA), - DEF_INCOMPAT_FLAG_ENTRY(NO_HOLES) + DEF_INCOMPAT_FLAG_ENTRY(NO_HOLES), + DEF_INCOMPAT_FLAG_ENTRY(HMZONED) }; static const int incompat_flags_num = sizeof(incompat_flags_array) / sizeof(struct readable_flag_entry); diff --git a/ctree.h b/ctree.h index 4719962d..6d805ecd 100644 --- a/ctree.h +++ b/ctree.h @@ -489,6 +489,7 @@ struct btrfs_super_block { #define BTRFS_FEATURE_INCOMPAT_RAID56 (1ULL << 7) #define BTRFS_FEATURE_INCOMPAT_SKINNY_METADATA (1ULL << 8) #define BTRFS_FEATURE_INCOMPAT_NO_HOLES (1ULL << 9) +#define BTRFS_FEATURE_INCOMPAT_HMZONED (1ULL << 10) #define BTRFS_FEATURE_COMPAT_SUPP 0ULL @@ -509,7 +510,8 @@ struct btrfs_super_block { BTRFS_FEATURE_INCOMPAT_RAID56 | \ BTRFS_FEATURE_INCOMPAT_MIXED_GROUPS | \ BTRFS_FEATURE_INCOMPAT_SKINNY_METADATA | \ - BTRFS_FEATURE_INCOMPAT_NO_HOLES) + BTRFS_FEATURE_INCOMPAT_NO_HOLES | \ + BTRFS_FEATURE_INCOMPAT_HMZONED) /* * A leaf is full of items. offset and size tell us where to find diff --git a/fsfeatures.c b/fsfeatures.c index 7d85d60f..53396dd4 100644 --- a/fsfeatures.c +++ b/fsfeatures.c @@ -86,6 +86,14 @@ static const struct btrfs_fs_feature { VERSION_TO_STRING2(4,0), NULL, 0, "no explicit hole extents for files" }, +#ifdef BTRFS_ZONED + { "hmzoned", BTRFS_FEATURE_INCOMPAT_HMZONED, + "hmzoned", + NULL, 0, + NULL, 0, + NULL, 0, + "support Host-Managed Zoned devices" }, +#endif /* Keep this one last */ { "list-all", BTRFS_FEATURE_LIST_ALL, NULL } }; diff --git a/fsfeatures.h b/fsfeatures.h index 3cc9452a..0918ee1a 100644 --- a/fsfeatures.h +++ b/fsfeatures.h @@ -25,7 +25,7 @@ | BTRFS_FEATURE_INCOMPAT_SKINNY_METADATA) /* - * Avoid multi-device features (RAID56) and mixed block groups + * Avoid multi-device features (RAID56), mixed block groups, and hmzoned device */ #define BTRFS_CONVERT_ALLOWED_FEATURES \ (BTRFS_FEATURE_INCOMPAT_MIXED_BACKREF \ diff --git a/libbtrfsutil/btrfs.h b/libbtrfsutil/btrfs.h index c293f6bf..c6a60fbc 100644 --- a/libbtrfsutil/btrfs.h +++ b/libbtrfsutil/btrfs.h @@ -268,6 +268,7 @@ struct btrfs_ioctl_fs_info_args { #define BTRFS_FEATURE_INCOMPAT_RAID56 (1ULL << 7) #define BTRFS_FEATURE_INCOMPAT_SKINNY_METADATA (1ULL << 8) #define BTRFS_FEATURE_INCOMPAT_NO_HOLES (1ULL << 9) +#define BTRFS_FEATURE_INCOMPAT_HMZONED (1ULL << 10) struct btrfs_ioctl_feature_flags { __u64 compat_flags; From patchwork Thu Aug 9 18:10:57 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naohiro Aota X-Patchwork-Id: 10561675 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id A096313BB for ; Thu, 9 Aug 2018 18:12:41 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 8BD562B7FB for ; Thu, 9 Aug 2018 18:12:41 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 803912B814; Thu, 9 Aug 2018 18:12:41 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.8 required=2.0 tests=BAYES_00,DKIM_SIGNED, MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI,T_DKIM_INVALID autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id D47E62B7FB for ; Thu, 9 Aug 2018 18:12:40 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727244AbeHIUhT (ORCPT ); Thu, 9 Aug 2018 16:37:19 -0400 Received: from mail-pf1-f194.google.com ([209.85.210.194]:32955 "EHLO mail-pf1-f194.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726882AbeHIUhT (ORCPT ); Thu, 9 Aug 2018 16:37:19 -0400 Received: by mail-pf1-f194.google.com with SMTP id d4-v6so3233190pfn.0; Thu, 09 Aug 2018 11:11:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:from:to:cc:subject:date:message-id:in-reply-to:references; bh=sSiVVi3rqDaN9cOwbVZla/jtHoG6CbWIGbpqxT3FMYw=; b=JpkozzayIBkTZoq7nEMnOnb+AqMkjCfJTNzra59vHZtsFBReoyU6ZOKpEvzez8ttwq DlzZ+Zxsn2Z3EUSRZLV5BFZRGa0zwHcoESo4lvNeDgMj7C7U1PJanAdrQN2tYvszhAV9 Pde0DpaU7pYgxAvaPow/IzmIrbUeYZf9sRycvOEy2xS9/KlM5MkGZdx3BS7UtFqMooDQ GxUJc7/BC6Js6U5hFn022HWSDgSxCCGi7M3y095iwfSrXTh+3hmzQjMGCwtcoafP44qj 4254PSSGJ0/sv0g1IbTP7FCcKdZm2S0qYdrQdsZkITSd0MuxZbqiUG/ygdxGXhDiJNZ+ NvTQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:from:to:cc:subject:date:message-id :in-reply-to:references; bh=sSiVVi3rqDaN9cOwbVZla/jtHoG6CbWIGbpqxT3FMYw=; b=jlSUv2p7yy8dLtgVr1Qw7xMG+ZzTc3Zq1DzqewLE3EQoybaSX4376p1+bYt/TbTP/q YJ2JxN80J72VJ5PwGxfZfJKPdXOXU8K1vQ4cGHuzgC2LsjVY8E6jD3r7TogSJOKY3HMe xpoe80bW9NeGjzgC+8tCUlyOfICYc6TGTAiBkgVZMUJVriRL/abnsJ6QSAs2eIpinYEL spfFO5yOOewQo0Vs5jcny65unkW8cJGwdm7oHwCVjlLIkly0qMORr1GaKxrvlJO9/l4G 5KiWhGGY5r+oQHfTOsJLSY0NxWFOvRdZjeIOj32jgOpK03YUfeIgo593PiMVeyj2Sk2b /I/w== X-Gm-Message-State: AOUpUlFu0L6zSfwvGAFWUQdaYwZY8uquMgID9TAkL2raeH+iMQw8Jz6Q gsvuJgSEOLtTID6jseTq7VE= X-Google-Smtp-Source: AA+uWPzndkqIa8Of0ZBquZi6yK+iLdlTGgRwCvgpmBifMjQzp7+eswSn6xSVmXNWd5q75r+2KfJLuA== X-Received: by 2002:a63:2647:: with SMTP id m68-v6mr3173234pgm.60.1533838278334; Thu, 09 Aug 2018 11:11:18 -0700 (PDT) Received: from localhost (h101-111-148-072.catv02.itscom.jp. [101.111.148.72]) by smtp.gmail.com with ESMTPSA id k64-v6sm16081815pfc.160.2018.08.09.11.11.17 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 09 Aug 2018 11:11:17 -0700 (PDT) From: Naohiro Aota To: David Sterba , linux-btrfs@vger.kernel.org Cc: Chris Mason , Josef Bacik , linux-kernel@vger.kernel.org, Hannes Reinecke , Damien Le Moal , Bart Van Assche , Matias Bjorling , Naohiro Aota Subject: [RFC PATCH 04/12] btrfs-progs: Introduce zone block device helper functions Date: Fri, 10 Aug 2018 03:10:57 +0900 Message-Id: <20180809181105.12856-4-naota@elisp.net> X-Mailer: git-send-email 2.18.0 In-Reply-To: <20180809181105.12856-1-naota@elisp.net> References: <20180809180450.5091-1-naota@elisp.net> <20180809181105.12856-1-naota@elisp.net> Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP This patch introduce several zone related functions: btrfs_get_zones() to get zone information from the specified device and put the information in zinfo, and zone_is_random_write() to check if a zone accept random writes. Signed-off-by: Naohiro Aota --- utils.c | 194 ++++++++++++++++++++++++++++++++++++++++++++++++++++++ utils.h | 16 +++++ volumes.h | 28 ++++++++ 3 files changed, 238 insertions(+) diff --git a/utils.c b/utils.c index 2212692c..71fc044a 100644 --- a/utils.c +++ b/utils.c @@ -359,6 +359,200 @@ out: return ret; } +enum btrfs_zoned_model zoned_model(const char *file) +{ + char model[32]; + int ret; + + ret = queue_param(file, "zoned", model, sizeof(model)); + if (ret <= 0) + return ZONED_NONE; + + if (strncmp(model, "host-aware", 10) == 0) + return ZONED_HOST_AWARE; + if (strncmp(model, "host-managed", 12) == 0) + return ZONED_HOST_MANAGED; + + return ZONED_NONE; +} + +size_t zone_size(const char *file) +{ + char chunk[32]; + int ret; + + ret = queue_param(file, "chunk_sectors", chunk, sizeof(chunk)); + if (ret <= 0) + return 0; + + return strtoul((const char *)chunk, NULL, 10) << 9; +} + +#ifdef BTRFS_ZONED +int zone_is_random_write(struct btrfs_zone_info *zinfo, u64 bytenr) +{ + unsigned int zno; + + if (zinfo->model == ZONED_NONE) + return 1; + + zno = bytenr / zinfo->zone_size; + + /* + * Only sequential write required zones on host-managed + * devices cannot be written randomly. + */ + return zinfo->zones[zno].type != BLK_ZONE_TYPE_SEQWRITE_REQ; +} + +#define BTRFS_REPORT_NR_ZONES 8192 + +static int btrfs_get_zones(int fd, const char *file, u64 block_count, + struct btrfs_zone_info *zinfo) +{ + size_t zone_bytes = zone_size(file); + size_t rep_size; + u64 sector = 0; + struct blk_zone_report *rep; + struct blk_zone *zone; + unsigned int i, n = 0; + int ret; + + /* + * Zones are guaranteed (by the kernel) to be a power of 2 number of + * sectors. Check this here and make sure that zones are not too + * small. + */ + if (!zone_bytes || (zone_bytes & (zone_bytes - 1))) { + error("ERROR: Illegal zone size %zu (not a power of 2)\n", + zone_bytes); + exit(1); + } + if (zone_bytes < BTRFS_MKFS_SYSTEM_GROUP_SIZE) { + error("ERROR: Illegal zone size %zu (smaller than %d)\n", + zone_bytes, + BTRFS_MKFS_SYSTEM_GROUP_SIZE); + exit(1); + } + + /* Allocate the zone information array */ + zinfo->zone_size = zone_bytes; + zinfo->nr_zones = block_count / zone_bytes; + if (block_count & (zone_bytes - 1)) + zinfo->nr_zones++; + zinfo->zones = calloc(zinfo->nr_zones, sizeof(struct blk_zone)); + if (!zinfo->zones) { + error("No memory for zone information\n"); + exit(1); + } + + /* Allocate a zone report */ + rep_size = sizeof(struct blk_zone_report) + + sizeof(struct blk_zone) * BTRFS_REPORT_NR_ZONES; + rep = malloc(rep_size); + if (!rep) { + error("No memory for zones report\n"); + exit(1); + } + + /* Get zone information */ + zone = (struct blk_zone *)(rep + 1); + while (n < zinfo->nr_zones) { + + memset(rep, 0, rep_size); + rep->sector = sector; + rep->nr_zones = BTRFS_REPORT_NR_ZONES; + + ret = ioctl(fd, BLKREPORTZONE, rep); + if (ret != 0) { + error("ioctl BLKREPORTZONE failed (%s)\n", + strerror(errno)); + exit(1); + } + + if (!rep->nr_zones) + break; + + for (i = 0; i < rep->nr_zones; i++) { + if (n >= zinfo->nr_zones) + break; + memcpy(&zinfo->zones[n], &zone[i], + sizeof(struct blk_zone)); + sector = zone[i].start + zone[i].len; + n++; + } + + } + + /* + * We need at least one random write zone (a conventional zone or + * a sequential write preferred zone on a host-aware device). + */ + if (!zone_is_random_write(zinfo, 0)) { + error("ERROR: No conventional zone at block 0\n"); + exit(1); + } + + zinfo->nr_zones = n; + + free(rep); + + return 0; +} + +#endif + +int btrfs_get_zone_info(int fd, const char *file, int hmzoned, + struct btrfs_zone_info *zinfo) +{ + struct stat st; + int ret; + + memset(zinfo, 0, sizeof(struct btrfs_zone_info)); + + ret = fstat(fd, &st); + if (ret < 0) { + error("unable to stat %s\n", file); + return 1; + } + + if (!S_ISBLK(st.st_mode)) + return 0; + + /* Check zone model */ + zinfo->model = zoned_model(file); + if (zinfo->model == ZONED_NONE) + return 0; + + if (zinfo->model == ZONED_HOST_MANAGED && !hmzoned) { + error("%s: host-managed zoned block device (enable zone block device support with -O hmzoned)\n", + file); + return -1; + } + + if (!hmzoned) { + /* Treat host-aware devices as regular devices */ + zinfo->model = ZONED_NONE; + return 0; + } + +#ifdef BTRFS_ZONED + /* Get zone information */ + ret = btrfs_get_zones(fd, file, btrfs_device_size(fd, &st), zinfo); + if (ret != 0) + return ret; +#else + error("%s: Unsupported host-%s zoned block device\n", + file, zinfo->model == ZONED_HOST_MANAGED ? "managed" : "aware"); + if (zinfo->model == ZONED_HOST_MANAGED) + return -1; + + printf("%s: heandling host-aware block device as a regular disk\n", + file); +#endif + return 0; +} + int btrfs_prepare_device(int fd, const char *file, u64 *block_count_ret, u64 max_block_count, unsigned opflags) { diff --git a/utils.h b/utils.h index ac333095..47f6b101 100644 --- a/utils.h +++ b/utils.h @@ -68,6 +68,7 @@ void units_set_base(unsigned *units, unsigned base); #define PREP_DEVICE_ZERO_END (1U << 0) #define PREP_DEVICE_DISCARD (1U << 1) #define PREP_DEVICE_VERBOSE (1U << 2) +#define PREP_DEVICE_HMZONED (1U << 3) #define SEEN_FSID_HASH_SIZE 256 struct seen_fsid { @@ -77,10 +78,25 @@ struct seen_fsid { int fd; }; +struct btrfs_zone_info; + +enum btrfs_zoned_model zoned_model(const char *file); +size_t zone_size(const char *file); int btrfs_make_root_dir(struct btrfs_trans_handle *trans, struct btrfs_root *root, u64 objectid); int btrfs_prepare_device(int fd, const char *file, u64 *block_count_ret, u64 max_block_count, unsigned opflags); +int btrfs_get_zone_info(int fd, const char *file, int hmzoned, + struct btrfs_zone_info *zinfo); +#ifdef BTRFS_ZONED +int zone_is_random_write(struct btrfs_zone_info *zinfo, u64 bytenr); +#else +static inline int zone_is_random_write(struct btrfs_zone_info *zinfo, + u64 bytenr) +{ + return 1; +} +#endif int btrfs_add_to_fsid(struct btrfs_trans_handle *trans, struct btrfs_root *root, int fd, const char *path, u64 block_count, u32 io_width, u32 io_align, diff --git a/volumes.h b/volumes.h index b4ea93f0..bad688e5 100644 --- a/volumes.h +++ b/volumes.h @@ -22,12 +22,40 @@ #include "kerncompat.h" #include "ctree.h" +#ifdef BTRFS_ZONED +#include +#else +struct blk_zone { + int dummy; +}; +#endif + +/* + * Zoned block device models. + */ +enum btrfs_zoned_model { + ZONED_NONE = 0, + ZONED_HOST_AWARE, + ZONED_HOST_MANAGED, +}; + +/* + * Zone information for a zoned block device. + */ +struct btrfs_zone_info { + enum btrfs_zoned_model model; + size_t zone_size; + struct blk_zone *zones; + unsigned int nr_zones; +}; + #define BTRFS_STRIPE_LEN SZ_64K struct btrfs_device { struct list_head dev_list; struct btrfs_root *dev_root; struct btrfs_fs_devices *fs_devices; + struct btrfs_zone_info zinfo; u64 total_ios; From patchwork Thu Aug 9 18:10:58 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naohiro Aota X-Patchwork-Id: 10561673 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id F2AF113B4 for ; Thu, 9 Aug 2018 18:12:35 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id DF68C2B7FB for ; Thu, 9 Aug 2018 18:12:35 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id D3BE92B814; Thu, 9 Aug 2018 18:12:35 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.8 required=2.0 tests=BAYES_00,DKIM_SIGNED, MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI,T_DKIM_INVALID autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 7093E2B7FB for ; Thu, 9 Aug 2018 18:12:35 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727296AbeHIUhW (ORCPT ); Thu, 9 Aug 2018 16:37:22 -0400 Received: from mail-pf1-f177.google.com ([209.85.210.177]:34879 "EHLO mail-pf1-f177.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726882AbeHIUhW (ORCPT ); Thu, 9 Aug 2018 16:37:22 -0400 Received: by mail-pf1-f177.google.com with SMTP id p12-v6so3228298pfh.2; Thu, 09 Aug 2018 11:11:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:from:to:cc:subject:date:message-id:in-reply-to:references; bh=vEgVkYKjembb7998doR0VHcTdu4uilIeK4V2QI2Qgx4=; b=tJdf5T0KC1vyfXTtLja5K0aym0G+xycjFXFhvpvA/UkLBLPJ1QsZe+vxXfCP5Z6RjW Y2YWT+qbYJuQnX0buPciVnKIaXpQSi6sVXRLg6iOO84ebyYThkvR3wMNPkuN1Zom07s4 jl5ctIsiS++VKWh4y18ctDbQuBpDeppeG+uNMIyr+55ze6IfnvzBcrPq7rCvLnVclViG 4azPNF58ol1yvPyZikE8ttnIuFWSAmu6EtVaTOgWBBpKv7/XCXDNjjDhywfkZSCMjdys thBSsw8U+y412/NgT5h6MiB0c6OeTsNy8gQxlUUiN+M31VOn8nlnCNdik0tJwaPpDsyl 3n3A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:from:to:cc:subject:date:message-id :in-reply-to:references; bh=vEgVkYKjembb7998doR0VHcTdu4uilIeK4V2QI2Qgx4=; b=s9cDBqfsud2XrTsjPuxAi/jF262R72l6xmRlYqfXr0vSBFNm1IZF8oKY/S/ciTL1qG LREvAmo7nAAPxyFTLmXPgWhu/UTpLF5gZzMn0GSe8k+XKWobAHg9Emjwu9xPRk40OCtt VMIJQ68VXRmIVqYSe2wPIKlZPE+9VXboqqJNVWdOH9th+J0mBSZbr7M3nlJMMiblbrIT iGcOhgQYtOo7Y3DcDcqpvz3fHDw/PxA7gtX/EugWiI+k8rpZ7ijx+HkdCb5W2C3Qb4Mi bnwAbrlOK3y7sp1YgA+PHrjD6bD7huY8Q+pVSUOYgNM2SY+wr3S2RuG74d+EVZ1n7yRY 9bBg== X-Gm-Message-State: AOUpUlGJZMgcrLlmHUVfS4zWOvtJMiAQ/z7kf969vkp/0GMkpz1c/qUf sA5YCtnmhvtKlZ5GsGn201fQ8APSDb4= X-Google-Smtp-Source: AA+uWPwYqci3E8ZLxZUMAa3To3x5DUp2vi3v19eKUaRwR3dkP1R9xE+wqL1HSwlhroLv9WtVITGVnw== X-Received: by 2002:a62:998:: with SMTP id 24-v6mr3386794pfj.99.1533838281287; Thu, 09 Aug 2018 11:11:21 -0700 (PDT) Received: from localhost (h101-111-148-072.catv02.itscom.jp. [101.111.148.72]) by smtp.gmail.com with ESMTPSA id 87-v6sm15442992pfn.103.2018.08.09.11.11.20 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 09 Aug 2018 11:11:20 -0700 (PDT) From: Naohiro Aota To: David Sterba , linux-btrfs@vger.kernel.org Cc: Chris Mason , Josef Bacik , linux-kernel@vger.kernel.org, Hannes Reinecke , Damien Le Moal , Bart Van Assche , Matias Bjorling , Naohiro Aota Subject: [RFC PATCH 05/12] btrfs-progs: load and check zone information Date: Fri, 10 Aug 2018 03:10:58 +0900 Message-Id: <20180809181105.12856-5-naota@elisp.net> X-Mailer: git-send-email 2.18.0 In-Reply-To: <20180809181105.12856-1-naota@elisp.net> References: <20180809180450.5091-1-naota@elisp.net> <20180809181105.12856-1-naota@elisp.net> Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP This patch checks if a device added to btrfs is a zoned block device. If it is, load zones information and the zone size for the device. For a btrfs volume composed of multiple zoned block devices, all devices must have the same zone size. Signed-off-by: Naohiro Aota --- utils.c | 10 ++++++++++ volumes.c | 18 ++++++++++++++++++ volumes.h | 3 +++ 3 files changed, 31 insertions(+) diff --git a/utils.c b/utils.c index 71fc044a..a2172a82 100644 --- a/utils.c +++ b/utils.c @@ -250,6 +250,16 @@ int btrfs_add_to_fsid(struct btrfs_trans_handle *trans, goto out; } + ret = btrfs_get_zone_info(fd, path, fs_info->fs_devices->hmzoned, + &device->zinfo); + if (ret) + goto out; + if (device->zinfo.zone_size != fs_info->fs_devices->zone_size) { + error("Device zone size differ\n"); + ret = -EINVAL; + goto out; + } + disk_super = (struct btrfs_super_block *)buf; dev_item = &disk_super->dev_item; diff --git a/volumes.c b/volumes.c index d81b348e..2ec27cd7 100644 --- a/volumes.c +++ b/volumes.c @@ -160,6 +160,8 @@ static int device_list_add(const char *path, struct btrfs_device *device; struct btrfs_fs_devices *fs_devices; u64 found_transid = btrfs_super_generation(disk_super); + int hmzoned = btrfs_super_incompat_flags(disk_super) & + BTRFS_FEATURE_INCOMPAT_HMZONED; fs_devices = find_fsid(disk_super->fsid); if (!fs_devices) { @@ -237,6 +239,8 @@ static int device_list_add(const char *path, if (fs_devices->lowest_devid > devid) { fs_devices->lowest_devid = devid; } + if (hmzoned) + fs_devices->hmzoned = 1; *fs_devices_ret = fs_devices; return 0; } @@ -307,6 +311,8 @@ int btrfs_open_devices(struct btrfs_fs_devices *fs_devices, int flags) struct btrfs_device *device; int ret; + fs_devices->zone_size = 0; + list_for_each_entry(device, &fs_devices->devices, dev_list) { if (!device->name) { printk("no name for device %llu, skip it now\n", device->devid); @@ -330,6 +336,18 @@ int btrfs_open_devices(struct btrfs_fs_devices *fs_devices, int flags) device->fd = fd; if (flags & O_RDWR) device->writeable = 1; + + ret = btrfs_get_zone_info(fd, device->name, fs_devices->hmzoned, + &device->zinfo); + if (ret != 0) + goto fail; + if (!fs_devices->zone_size) { + fs_devices->zone_size = device->zinfo.zone_size; + } else if (device->zinfo.zone_size != fs_devices->zone_size) { + fprintf(stderr, "Device zone size differ\n"); + ret = -EINVAL; + goto fail; + } } return 0; fail: diff --git a/volumes.h b/volumes.h index bad688e5..36a6f44b 100644 --- a/volumes.h +++ b/volumes.h @@ -111,6 +111,9 @@ struct btrfs_fs_devices { int seeding; struct btrfs_fs_devices *seed; + + u64 zone_size; + unsigned int hmzoned:1; }; struct btrfs_bio_stripe { From patchwork Thu Aug 9 18:10:59 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naohiro Aota X-Patchwork-Id: 10561671 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 3190613BB for ; Thu, 9 Aug 2018 18:12:34 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 1D5342B7FB for ; Thu, 9 Aug 2018 18:12:34 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 1176B2B814; Thu, 9 Aug 2018 18:12:34 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.8 required=2.0 tests=BAYES_00,DKIM_SIGNED, MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI,T_DKIM_INVALID autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 913C62B7FB for ; Thu, 9 Aug 2018 18:12:33 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727325AbeHIUhZ (ORCPT ); Thu, 9 Aug 2018 16:37:25 -0400 Received: from mail-pg1-f196.google.com ([209.85.215.196]:40697 "EHLO mail-pg1-f196.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726882AbeHIUhZ (ORCPT ); Thu, 9 Aug 2018 16:37:25 -0400 Received: by mail-pg1-f196.google.com with SMTP id x5-v6so3110314pgp.7; Thu, 09 Aug 2018 11:11:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:from:to:cc:subject:date:message-id:in-reply-to:references; bh=NbmpInz5/joKYbzaioP/45OSGEcNKQUls8Zih/tMAyA=; b=PpDfi8/3zCXQAWOTly/hgj2fVILbBAR/v1v/oAR4VkoW0JyMySKDDJbh8tSSx7A5tP GWfBTZqXIm0Rr2sw3REdv6oX8lY0uG4U9ho4clFIn+7GYon4E0RvWSaETIG6H+iRLJR4 bu/uD9us01N1nxqsV/U6DZWi745nP2Qu/PUs/j9xxxivZ2Oxfe2LHL55UpSqvLb8fR9U /ZEGQ3gbkgg6CfdLjKLLfNxrC9rdnFz+K7KGQ6QfLF+OYweCbj3n6tnsBN/CMOOFRdlJ jI+Eo7ybOGqs76KmBmycTK2SCGSL7H5RDVpoKnlp+aFmzyo0BWeb5+UlMifP4L0RgXz2 YuuQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:from:to:cc:subject:date:message-id :in-reply-to:references; bh=NbmpInz5/joKYbzaioP/45OSGEcNKQUls8Zih/tMAyA=; b=j43Tu96wsGPFNjfHI1JMN0YvsxyDGIQ1SWE0eW5I5YbCuhudZ5zwWcsr1sGqio8GW8 XPGsqg053VWe11Rsj7OZo4csPE2Z42gVi6v96NQ34pjSM5+Xmq46jPbt3HKuxU6S00Z/ VKUr88De/ttBSr4m6PaKwy6Kt6sbnHY+/uo/y0EBbuF7jjLanar0ka72IjSogZDu45Ae Bt1SP/fOJE9hPwZIruvrpjOktSoEq5yB825oUdIAHQ+Vcjo5qiUJaGfFhYlTDoDOJmrt AGg1Yue9enGBQqqayOmnT11dks0a9nN7G02ecSwRfCuf5izUvHzzU1536H9KI48/M2qU rlKw== X-Gm-Message-State: AOUpUlGEIFIFzT7d3EjUJxm7u8heCTtcy/t9wHFhfmN262omwmTt4jVB iJP2bfLEbAysc4nJc4diMlA= X-Google-Smtp-Source: AA+uWPw9TNS57HDQ/KEKkC4gS5NfT7obFwEDNPSStVTnwWkjjfhNU5gxk525k+iu+x1Qchubc0jnBQ== X-Received: by 2002:a63:4386:: with SMTP id q128-v6mr3130277pga.353.1533838284236; Thu, 09 Aug 2018 11:11:24 -0700 (PDT) Received: from localhost (h101-111-148-072.catv02.itscom.jp. [101.111.148.72]) by smtp.gmail.com with ESMTPSA id e14-v6sm9778335pff.128.2018.08.09.11.11.23 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 09 Aug 2018 11:11:23 -0700 (PDT) From: Naohiro Aota To: David Sterba , linux-btrfs@vger.kernel.org Cc: Chris Mason , Josef Bacik , linux-kernel@vger.kernel.org, Hannes Reinecke , Damien Le Moal , Bart Van Assche , Matias Bjorling , Naohiro Aota Subject: [RFC PATCH 06/12] btrfs-progs: avoid writing super block to sequential zones Date: Fri, 10 Aug 2018 03:10:59 +0900 Message-Id: <20180809181105.12856-6-naota@elisp.net> X-Mailer: git-send-email 2.18.0 In-Reply-To: <20180809181105.12856-1-naota@elisp.net> References: <20180809180450.5091-1-naota@elisp.net> <20180809181105.12856-1-naota@elisp.net> Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP It is not possible to write a super block copy in sequential write required zones as this prevents in-place updates required for super blocks. This patch limits super block possible locations to zones accepting random writes. In particular, the zone containing the first block of the device or partition being formatted must accept random writes. Signed-off-by: Naohiro Aota --- disk-io.c | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/disk-io.c b/disk-io.c index 26e4f6e9..127d8cf4 100644 --- a/disk-io.c +++ b/disk-io.c @@ -1523,6 +1523,7 @@ static int write_dev_supers(struct btrfs_fs_info *fs_info, struct btrfs_super_block *sb, struct btrfs_device *device) { + struct btrfs_zone_info *zinfo = &device->zinfo; u64 bytenr; u32 crc; int i, ret; @@ -1534,6 +1535,11 @@ static int write_dev_supers(struct btrfs_fs_info *fs_info, BTRFS_SUPER_INFO_SIZE - BTRFS_CSUM_SIZE); btrfs_csum_final(crc, &sb->csum[0]); + if (!zone_is_random_write(zinfo, fs_info->super_bytenr)) { + ret = -EIO; + goto write_err; + } + /* * super_copy is BTRFS_SUPER_INFO_SIZE bytes and is * zero filled, we can use it directly @@ -1550,6 +1556,8 @@ static int write_dev_supers(struct btrfs_fs_info *fs_info, bytenr = btrfs_sb_offset(i); if (bytenr + BTRFS_SUPER_INFO_SIZE > device->total_bytes) break; + if (!zone_is_random_write(zinfo, bytenr)) + continue; btrfs_set_super_bytenr(sb, bytenr); From patchwork Thu Aug 9 18:11:00 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naohiro Aota X-Patchwork-Id: 10561653 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id B1EE613B4 for ; Thu, 9 Aug 2018 18:11:33 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 9DAFA2B846 for ; Thu, 9 Aug 2018 18:11:33 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 9B9022B808; Thu, 9 Aug 2018 18:11:33 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.8 required=2.0 tests=BAYES_00,DKIM_SIGNED, MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI,T_DKIM_INVALID autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 3BF132B848 for ; Thu, 9 Aug 2018 18:11:31 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727383AbeHIUh3 (ORCPT ); Thu, 9 Aug 2018 16:37:29 -0400 Received: from mail-pg1-f194.google.com ([209.85.215.194]:33691 "EHLO mail-pg1-f194.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726976AbeHIUh2 (ORCPT ); Thu, 9 Aug 2018 16:37:28 -0400 Received: by mail-pg1-f194.google.com with SMTP id r5-v6so3125789pgv.0; Thu, 09 Aug 2018 11:11:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:from:to:cc:subject:date:message-id:in-reply-to:references; bh=oOCMaRWDGBx2P16zaSrQG44JuqG6jiFD4BQxMhQSjxw=; b=iZJL5vB6KST4yVzlHcHE97tAkN+RPilFSnhA2GWM8teM3KW3d4055Il42rq3yNP4dr WvIRunldBSaikTFjBHA/bu198kYf/NBkdop8U0qZHek6LVo/6a6TOhE/v+o1+sH3Bt2p TmT9vwi/vRrhEEwji/7o4OJnK/l6HwF1bNz9K915Z2rL0b53oZpOshYSWGl+SqtqmmWe NFqHLWpq69n5avfUM5sU62kSgiJ59rSHgHKstm5Ko+nGwFXMKQNazo1fsItpZvfrQLdp oipNVSNrxgJWH/Fe6KwOgMUo4ORzIMn3lLOyl5NF6AdTzcZ065wEv+NuaK5GOu+K2/L4 wClw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:from:to:cc:subject:date:message-id :in-reply-to:references; bh=oOCMaRWDGBx2P16zaSrQG44JuqG6jiFD4BQxMhQSjxw=; b=Q6Skb5gDfz0KHpCFCdEbAMYKpk+h60LUAxSEtPXGfyuPJUI01vxN9ceQp1RB1LF4Zq nmdIq/PVQJBgYhSIww1trIHI/KmFrBK3TU5Y9O8VX0u97Mwchu12JZsV1Dpx41JGpdyD 7WlVmTV2cufeinDvSzHWhGySG4W/H17J9NSoYvk/7DYAOdqYMGjGtnMRplb2kPGUQ182 DHN6o1IJ2nsERNZQHfMajvL5XiISyc27aKZzLeokY8gn4DXBbXCYBVVu2NmlETlJfVSx 6Y90cPAW7ZLkoIMMpiF7OKft7aY5D00DxsLEq6hlmVEsxKwXk7gRoRy3cnn6Uz/kbNY+ MvbQ== X-Gm-Message-State: AOUpUlEyNlvQjbxeftnidG++lV2uweGlYU+yOk/bLqVwR/4Y5kT9lO5i TvBSc+owMPZP+xnpEo2Y6Mc= X-Google-Smtp-Source: AA+uWPwhel7H6qKI+Qy1ZTiSaPVGP0PYYdO9ijDjjxUOTZnx8ogPNV/2HtcHFW1sNZCz9MkDwS3E2g== X-Received: by 2002:a62:67c2:: with SMTP id t63-v6mr3455435pfj.204.1533838287195; Thu, 09 Aug 2018 11:11:27 -0700 (PDT) Received: from localhost (h101-111-148-072.catv02.itscom.jp. [101.111.148.72]) by smtp.gmail.com with ESMTPSA id v30-v6sm12506966pgn.80.2018.08.09.11.11.26 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 09 Aug 2018 11:11:26 -0700 (PDT) From: Naohiro Aota To: David Sterba , linux-btrfs@vger.kernel.org Cc: Chris Mason , Josef Bacik , linux-kernel@vger.kernel.org, Hannes Reinecke , Damien Le Moal , Bart Van Assche , Matias Bjorling , Naohiro Aota Subject: [RFC PATCH 07/12] btrfs-progs: support discarding zoned device Date: Fri, 10 Aug 2018 03:11:00 +0900 Message-Id: <20180809181105.12856-7-naota@elisp.net> X-Mailer: git-send-email 2.18.0 In-Reply-To: <20180809181105.12856-1-naota@elisp.net> References: <20180809180450.5091-1-naota@elisp.net> <20180809181105.12856-1-naota@elisp.net> Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP All zones of zoned block devices should be reset before writing. Support this by considering zone reset as a special case of block discard and block zeroing. Of note is that only zones accepting random writes can be zeroed. Signed-off-by: Naohiro Aota --- utils.c | 94 +++++++++++++++++++++++++++++++++++++++++++++++++++++---- 1 file changed, 88 insertions(+), 6 deletions(-) diff --git a/utils.c b/utils.c index a2172a82..79a45d92 100644 --- a/utils.c +++ b/utils.c @@ -123,6 +123,37 @@ static int discard_range(int fd, u64 start, u64 len) return 0; } +/* + * Discard blocks in the zones of a zoned block device. + * Process this with zone size granularity so that blocks in + * conventional zones are discarded using discard_range and + * blocks in sequential zones are discarded though a zone reset. + */ +static int discard_zones(int fd, struct btrfs_zone_info *zinfo) +{ +#ifdef BTRFS_ZONED + unsigned int i; + + /* Zone size granularity */ + for (i = 0; i < zinfo->nr_zones; i++) { + if (zinfo->zones[i].type == BLK_ZONE_TYPE_CONVENTIONAL) { + discard_range(fd, zinfo->zones[i].start << 9, + zinfo->zone_size); + } else if (zinfo->zones[i].cond != BLK_ZONE_COND_EMPTY) { + struct blk_zone_range range = { + zinfo->zones[i].start, + zinfo->zone_size >> 9 }; + if (ioctl(fd, BLKRESETZONE, &range) < 0) + return errno; + } + } + + return 0; +#else + return -EIO; +#endif +} + /* * Discard blocks in the given range in 1G chunks, the process is interruptible */ @@ -205,8 +236,38 @@ static int zero_blocks(int fd, off_t start, size_t len) #define ZERO_DEV_BYTES SZ_2M +static int zero_zone_blocks(int fd, struct btrfs_zone_info *zinfo, + off_t start, size_t len) +{ + size_t zone_len = zinfo->zone_size; + off_t ofst = start; + size_t count; + int ret; + + /* Make sure that zero_blocks does not write sequential zones */ + while (len > 0) { + + /* Limit zero_blocks to a single zone */ + count = min_t(size_t, len, zone_len); + if (count > zone_len - (ofst & (zone_len - 1))) + count = zone_len - (ofst & (zone_len - 1)); + + if (zone_is_random_write(zinfo, ofst)) { + ret = zero_blocks(fd, ofst, count); + if (ret != 0) + return ret; + } + + len -= count; + ofst += count; + } + + return 0; +} + /* don't write outside the device by clamping the region to the device size */ -static int zero_dev_clamped(int fd, off_t start, ssize_t len, u64 dev_size) +static int zero_dev_clamped(int fd, struct btrfs_zone_info *zinfo, + off_t start, ssize_t len, u64 dev_size) { off_t end = max(start, start + len); @@ -219,6 +280,9 @@ static int zero_dev_clamped(int fd, off_t start, ssize_t len, u64 dev_size) start = min_t(u64, start, dev_size); end = min_t(u64, end, dev_size); + if (zinfo->model != ZONED_NONE) + return zero_zone_blocks(fd, zinfo, start, end - start); + return zero_blocks(fd, start, end - start); } @@ -566,6 +630,7 @@ int btrfs_get_zone_info(int fd, const char *file, int hmzoned, int btrfs_prepare_device(int fd, const char *file, u64 *block_count_ret, u64 max_block_count, unsigned opflags) { + struct btrfs_zone_info zinfo; u64 block_count; struct stat st; int i, ret; @@ -584,13 +649,30 @@ int btrfs_prepare_device(int fd, const char *file, u64 *block_count_ret, if (max_block_count) block_count = min(block_count, max_block_count); + ret = btrfs_get_zone_info(fd, file, opflags & PREP_DEVICE_HMZONED, + &zinfo); + if (ret < 0) + return 1; + if (opflags & PREP_DEVICE_DISCARD) { /* * We intentionally ignore errors from the discard ioctl. It * is not necessary for the mkfs functionality but just an - * optimization. + * optimization. However, we cannot ignore zone discard (reset) + * errors for a zoned block device as this could result in the + * inability to write to non-empty sequential zones of the + * device. */ - if (discard_range(fd, 0, 0) == 0) { + if (zinfo.model != ZONED_NONE) { + printf("Resetting device zones %s (%u zones) ...\n", + file, zinfo.nr_zones); + if (discard_zones(fd, &zinfo)) { + fprintf(stderr, + "ERROR: failed to reset device '%s' zones\n", + file); + return 1; + } + } else if (discard_range(fd, 0, 0) == 0) { if (opflags & PREP_DEVICE_VERBOSE) printf("Performing full device TRIM %s (%s) ...\n", file, pretty_size(block_count)); @@ -598,12 +680,12 @@ int btrfs_prepare_device(int fd, const char *file, u64 *block_count_ret, } } - ret = zero_dev_clamped(fd, 0, ZERO_DEV_BYTES, block_count); + ret = zero_dev_clamped(fd, &zinfo, 0, ZERO_DEV_BYTES, block_count); for (i = 0 ; !ret && i < BTRFS_SUPER_MIRROR_MAX; i++) - ret = zero_dev_clamped(fd, btrfs_sb_offset(i), + ret = zero_dev_clamped(fd, &zinfo, btrfs_sb_offset(i), BTRFS_SUPER_INFO_SIZE, block_count); if (!ret && (opflags & PREP_DEVICE_ZERO_END)) - ret = zero_dev_clamped(fd, block_count - ZERO_DEV_BYTES, + ret = zero_dev_clamped(fd, &zinfo, block_count - ZERO_DEV_BYTES, ZERO_DEV_BYTES, block_count); if (ret < 0) { From patchwork Thu Aug 9 18:11:01 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naohiro Aota X-Patchwork-Id: 10561655 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id E947613BB for ; Thu, 9 Aug 2018 18:11:34 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id D5CDA2B853 for ; Thu, 9 Aug 2018 18:11:34 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id C9BAF2B863; Thu, 9 Aug 2018 18:11:34 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.8 required=2.0 tests=BAYES_00,DKIM_SIGNED, MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI,T_DKIM_INVALID autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id F23982B853 for ; Thu, 9 Aug 2018 18:11:32 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727393AbeHIUhc (ORCPT ); Thu, 9 Aug 2018 16:37:32 -0400 Received: from mail-pl0-f65.google.com ([209.85.160.65]:46176 "EHLO mail-pl0-f65.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727390AbeHIUhb (ORCPT ); Thu, 9 Aug 2018 16:37:31 -0400 Received: by mail-pl0-f65.google.com with SMTP id t17-v6so2870611ply.13; Thu, 09 Aug 2018 11:11:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:from:to:cc:subject:date:message-id:in-reply-to:references; bh=JDhFAs3xXtk7n4+Cx+CXUjqd+12covKCPBG+j/hN8kA=; b=oRKjI1LWzVGSSDlmA4Rsf9pKC+gnscv4KBYFqVpOzby5qnFjUXup5ARyijIZdXM/i8 jsGbL2BhbGVcBMpV2W6br5KfnXGIYMTBxdFNG6A7PUYPpqDTUvWgttvM94zlWZlQqHj+ zKIP5XMIXfKoVCJPa+Z2P+kDAoCPyRDAR8S0pHWArRYjketO1ohJXIV3Ic8izaWtuhqc G9Glh8XvAf/rBKIUArWkSi/ALKd/415JjEJrBlo7/WlpPZxcDfD5dkAsdosI5rCqNPdw jRgmyayoHc8njdc1f8TN/HXSeTFy2zZtBaPGF6ZzWu6FT/9CvSMBySiF3x0QgqPbQ8h0 DCdA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:from:to:cc:subject:date:message-id :in-reply-to:references; bh=JDhFAs3xXtk7n4+Cx+CXUjqd+12covKCPBG+j/hN8kA=; b=MnNstHgRtPZ2wJPxh4T8Dz76kH7vjnx1r26iJdxM0bXKk81qWYVdKqm6l4SD16z1z9 k0vOW0IDUzfRRPfGWn14kbPByYr/mLk07q8rUJNFdmLyMbWfROelfiockPAL6+fyvtkK abIMeioM1qs6ZghgX56ieWj0XmPfp3n2vjvLGGHBqKJk0G0Hy+zJrpwsXIE7D1gDzqQo G2B++R16tWQAeu+pEoBFlwDRSTqjgeEpDJEY1DwWDxN2rU7t/rqO7z93F1uDKgsrhU4f 6fniy6UNbnzZL05aHZPKGSLfqeVGUl6g3P79hwcEEznWzOduLsB3rB4a6rJbUxme3A+9 Laxg== X-Gm-Message-State: AOUpUlFyQqWbvNogmcXUGSntSg+KjJmw9M/ADmqVZ15DvyL+BZBNkfXg qAcW/w28IXfwmMPJ9nqrbm+LERkEcW8= X-Google-Smtp-Source: AA+uWPxCIUGIzB0OZbI/B6vtTKDykdOOGurcsYy465ZZcm+UQx3BBFCs51Uk5qsGpXg2i+nDP+94GQ== X-Received: by 2002:a17:902:bc41:: with SMTP id t1-v6mr3057166plz.26.1533838290093; Thu, 09 Aug 2018 11:11:30 -0700 (PDT) Received: from localhost (h101-111-148-072.catv02.itscom.jp. [101.111.148.72]) by smtp.gmail.com with ESMTPSA id 11-v6sm19059377pfq.43.2018.08.09.11.11.29 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 09 Aug 2018 11:11:29 -0700 (PDT) From: Naohiro Aota To: David Sterba , linux-btrfs@vger.kernel.org Cc: Chris Mason , Josef Bacik , linux-kernel@vger.kernel.org, Hannes Reinecke , Damien Le Moal , Bart Van Assche , Matias Bjorling , Naohiro Aota Subject: [RFC PATCH 08/12] btrfs-progs: volume: align chunk allocation to zones Date: Fri, 10 Aug 2018 03:11:01 +0900 Message-Id: <20180809181105.12856-8-naota@elisp.net> X-Mailer: git-send-email 2.18.0 In-Reply-To: <20180809181105.12856-1-naota@elisp.net> References: <20180809180450.5091-1-naota@elisp.net> <20180809181105.12856-1-naota@elisp.net> Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP To facilitate support for zoned block devices in the extent buffer allocation, a zoned block device chunk is always aligned to a zone of the device. With this, the zone write pointer location simply becomes a hint to allocate new buffers. Signed-off-by: Naohiro Aota --- volumes.c | 34 ++++++++++++++++++++++++++++++---- 1 file changed, 30 insertions(+), 4 deletions(-) diff --git a/volumes.c b/volumes.c index 2ec27cd7..ba3b45d2 100644 --- a/volumes.c +++ b/volumes.c @@ -379,6 +379,14 @@ int btrfs_scan_one_device(int fd, const char *path, return ret; } +/* zone size is ensured to be power of 2 */ +static u64 btrfs_zone_align(struct btrfs_zone_info *zinfo, u64 val) +{ + if (zinfo && zinfo->zone_size) + return (val + zinfo->zone_size - 1) & ~(zinfo->zone_size - 1); + return val; +} + /* * find_free_dev_extent_start - find free space in the specified device * @device: the device which we search the free space in @@ -425,6 +433,7 @@ static int find_free_dev_extent_start(struct btrfs_device *device, */ min_search_start = max(root->fs_info->alloc_start, (u64)SZ_1M); search_start = max(search_start, min_search_start); + search_start = btrfs_zone_align(&device->zinfo, search_start); path = btrfs_alloc_path(); if (!path) @@ -507,7 +516,8 @@ static int find_free_dev_extent_start(struct btrfs_device *device, extent_end = key.offset + btrfs_dev_extent_length(l, dev_extent); if (extent_end > search_start) - search_start = extent_end; + search_start = btrfs_zone_align(&device->zinfo, + extent_end); next: path->slots[0]++; cond_resched(); @@ -560,6 +570,9 @@ static int btrfs_alloc_dev_extent(struct btrfs_trans_handle *trans, struct extent_buffer *leaf; struct btrfs_key key; + /* Align to zone for a zoned block device */ + *start = btrfs_zone_align(&device->zinfo, *start); + path = btrfs_alloc_path(); if (!path) return -ENOMEM; @@ -1030,9 +1043,15 @@ int btrfs_alloc_chunk(struct btrfs_trans_handle *trans, btrfs_super_stripesize(info->super_copy)); } - /* we don't want a chunk larger than 10% of the FS */ - percent_max = div_factor(btrfs_super_total_bytes(info->super_copy), 1); - max_chunk_size = min(percent_max, max_chunk_size); + if (info->fs_devices->hmzoned) { + /* Zoned mode uses zone aligned chunks */ + calc_size = info->fs_devices->zone_size; + max_chunk_size = calc_size * num_stripes; + } else { + /* we don't want a chunk larger than 10% of the FS */ + percent_max = div_factor(btrfs_super_total_bytes(info->super_copy), 1); + max_chunk_size = min(percent_max, max_chunk_size); + } again: if (chunk_bytes_by_type(type, calc_size, num_stripes, sub_stripes) > @@ -1112,7 +1131,9 @@ again: *num_bytes = chunk_bytes_by_type(type, calc_size, num_stripes, sub_stripes); index = 0; + dev_offset = 0; while(index < num_stripes) { + size_t zone_size = device->zinfo.zone_size; struct btrfs_stripe *stripe; BUG_ON(list_empty(&private_devs)); cur = private_devs.next; @@ -1123,11 +1144,16 @@ again: (index == num_stripes - 1)) list_move_tail(&device->dev_list, dev_list); + if (device->zinfo.zone_size) + calc_size = device->zinfo.zone_size; + ret = btrfs_alloc_dev_extent(trans, device, key.offset, calc_size, &dev_offset, 0); if (ret < 0) goto out_chunk_map; + WARN_ON(zone_size && !IS_ALIGNED(dev_offset, zone_size)); + device->bytes_used += calc_size; ret = btrfs_update_device(trans, device); if (ret < 0) From patchwork Thu Aug 9 18:11:02 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naohiro Aota X-Patchwork-Id: 10561657 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 776A513BB for ; Thu, 9 Aug 2018 18:11:37 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 613722B82F for ; Thu, 9 Aug 2018 18:11:37 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 546592B837; Thu, 9 Aug 2018 18:11:37 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.8 required=2.0 tests=BAYES_00,DKIM_SIGNED, MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI,T_DKIM_INVALID autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 7915C2B808 for ; Thu, 9 Aug 2018 18:11:36 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727461AbeHIUhf (ORCPT ); Thu, 9 Aug 2018 16:37:35 -0400 Received: from mail-pf1-f194.google.com ([209.85.210.194]:40218 "EHLO mail-pf1-f194.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727390AbeHIUhe (ORCPT ); Thu, 9 Aug 2018 16:37:34 -0400 Received: by mail-pf1-f194.google.com with SMTP id e13-v6so3211303pff.7; Thu, 09 Aug 2018 11:11:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:from:to:cc:subject:date:message-id:in-reply-to:references; bh=pJlZGWei2gsWFEDFHY/yR33U2XNBivlD3ZxVWnBZt0A=; b=bXNGfIDb7rbruBoDL+SS0iWYo1VQNfjYsLBGm+sqxM4pXMBQjlvYIdMSEyzx/WYq1P B0N+/pXNtR1hgthJcOzydh8eoMWLW+VdYUFIW/QrrhXBw6umYbwQKpYfbTcUp8H1zaK0 JN9cukC4aZvJSk8a2K3e7oKmbq0WjfmdX3e0Kvcb8L7Tw5zD+2vUKYprDNxSNUNSC0M6 i4JCcCjjaCeN8C/jyoDSIDt3LdE6wPLow3md6ARyu5mRFxUBK8YjG5Lm5UW3/iewVf6r FLsEX9O4PSKjNUAcfgdzVDhzNu5OfW9KjsmVtiBpuDdcnNsqjFN1OpeTqnsGu2I1F9Pq 0e+w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:from:to:cc:subject:date:message-id :in-reply-to:references; bh=pJlZGWei2gsWFEDFHY/yR33U2XNBivlD3ZxVWnBZt0A=; b=llvlkVofb9VEWyFwlDYvX/26CJNpWheUnJikUzyfwxbtoaPj+1kIcD9L41+mx8i81K 4Qa9mnW6FXF+lDqb0taa7JNMXr2h1RDGuuFuGdyQEvqZXetJYWyJVK9yytwinDoj9gJg vXWIoYr8jsGDcPg0B/mQW2ztgeA5YwCKrxg6z18oT8yz8mcKrVysknJwjlK6kgw8ek1q 8ftHIsZoqjiMCnWnk9p1zKmlkj0EuqcysBgPCuEgF38KYiZilbKlozKg7H4KNtQmMQkt gCMFglGV04lWIitfLqMr/o99VhFl3ZiyUQ8Ztbv38twKmiGDnDS7oA14xjtOcsRxEFo0 XICg== X-Gm-Message-State: AOUpUlGyl8MwOuQ0NemO0mFtgJRQ60jwUTpuTfIgoYfhVLVxUluvVM+R o2Z09WWDg5nuIebHZ9NvYag= X-Google-Smtp-Source: AA+uWPwa9VIUNkyJsN1XCji9UOY6SsnlEpk+B9+LlzED8kMPEpKvME7BsaRqBgEtzJdlpm/TJ5DMbg== X-Received: by 2002:a63:c20:: with SMTP id b32-v6mr3050616pgl.400.1533838293034; Thu, 09 Aug 2018 11:11:33 -0700 (PDT) Received: from localhost (h101-111-148-072.catv02.itscom.jp. [101.111.148.72]) by smtp.gmail.com with ESMTPSA id t19-v6sm19889786pfk.182.2018.08.09.11.11.32 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 09 Aug 2018 11:11:32 -0700 (PDT) From: Naohiro Aota To: David Sterba , linux-btrfs@vger.kernel.org Cc: Chris Mason , Josef Bacik , linux-kernel@vger.kernel.org, Hannes Reinecke , Damien Le Moal , Bart Van Assche , Matias Bjorling , Naohiro Aota Subject: [RFC PATCH 09/12] btrfs-progs: mkfs: Zoned block device support Date: Fri, 10 Aug 2018 03:11:02 +0900 Message-Id: <20180809181105.12856-9-naota@elisp.net> X-Mailer: git-send-email 2.18.0 In-Reply-To: <20180809181105.12856-1-naota@elisp.net> References: <20180809180450.5091-1-naota@elisp.net> <20180809181105.12856-1-naota@elisp.net> Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP This patch makes the size of the temporary system group chunk equal to the device zone size. It also enables PREP_DEVICE_HMZONED if the user enables the HMZONED feature. Enabling HMZONED feature is done using option "-O hmzoned". This feature is incompatible for now with source directory setup. Signed-off-by: Naohiro Aota --- mkfs/common.c | 12 +++++++----- mkfs/common.h | 1 + mkfs/main.c | 45 +++++++++++++++++++++++++++++++++++++++------ 3 files changed, 47 insertions(+), 11 deletions(-) diff --git a/mkfs/common.c b/mkfs/common.c index 0ace262b..d01402c8 100644 --- a/mkfs/common.c +++ b/mkfs/common.c @@ -152,6 +152,7 @@ int make_btrfs(int fd, struct btrfs_mkfs_config *cfg) int skinny_metadata = !!(cfg->features & BTRFS_FEATURE_INCOMPAT_SKINNY_METADATA); u64 num_bytes; + u64 system_group_size; buf = malloc(sizeof(*buf) + max(cfg->sectorsize, cfg->nodesize)); if (!buf) @@ -312,12 +313,14 @@ int make_btrfs(int fd, struct btrfs_mkfs_config *cfg) btrfs_set_item_offset(buf, btrfs_item_nr(nritems), itemoff); btrfs_set_item_size(buf, btrfs_item_nr(nritems), item_size); + system_group_size = (cfg->features & BTRFS_FEATURE_INCOMPAT_HMZONED) ? + cfg->zone_size : BTRFS_MKFS_SYSTEM_GROUP_SIZE; + dev_item = btrfs_item_ptr(buf, nritems, struct btrfs_dev_item); btrfs_set_device_id(buf, dev_item, 1); btrfs_set_device_generation(buf, dev_item, 0); btrfs_set_device_total_bytes(buf, dev_item, num_bytes); - btrfs_set_device_bytes_used(buf, dev_item, - BTRFS_MKFS_SYSTEM_GROUP_SIZE); + btrfs_set_device_bytes_used(buf, dev_item, system_group_size); btrfs_set_device_io_align(buf, dev_item, cfg->sectorsize); btrfs_set_device_io_width(buf, dev_item, cfg->sectorsize); btrfs_set_device_sector_size(buf, dev_item, cfg->sectorsize); @@ -345,7 +348,7 @@ int make_btrfs(int fd, struct btrfs_mkfs_config *cfg) btrfs_set_item_size(buf, btrfs_item_nr(nritems), item_size); chunk = btrfs_item_ptr(buf, nritems, struct btrfs_chunk); - btrfs_set_chunk_length(buf, chunk, BTRFS_MKFS_SYSTEM_GROUP_SIZE); + btrfs_set_chunk_length(buf, chunk, system_group_size); btrfs_set_chunk_owner(buf, chunk, BTRFS_EXTENT_TREE_OBJECTID); btrfs_set_chunk_stripe_len(buf, chunk, BTRFS_STRIPE_LEN); btrfs_set_chunk_type(buf, chunk, BTRFS_BLOCK_GROUP_SYSTEM); @@ -411,8 +414,7 @@ int make_btrfs(int fd, struct btrfs_mkfs_config *cfg) (unsigned long)btrfs_dev_extent_chunk_tree_uuid(dev_extent), BTRFS_UUID_SIZE); - btrfs_set_dev_extent_length(buf, dev_extent, - BTRFS_MKFS_SYSTEM_GROUP_SIZE); + btrfs_set_dev_extent_length(buf, dev_extent, system_group_size); nritems++; btrfs_set_header_bytenr(buf, cfg->blocks[MKFS_DEV_TREE]); diff --git a/mkfs/common.h b/mkfs/common.h index 28912906..d0e4c7b2 100644 --- a/mkfs/common.h +++ b/mkfs/common.h @@ -53,6 +53,7 @@ struct btrfs_mkfs_config { u64 features; /* Size of the filesystem in bytes */ u64 num_bytes; + u64 zone_size; /* Output fields, set during creation */ diff --git a/mkfs/main.c b/mkfs/main.c index 83969b4b..f940eba1 100644 --- a/mkfs/main.c +++ b/mkfs/main.c @@ -60,8 +60,12 @@ static int create_metadata_block_groups(struct btrfs_root *root, int mixed, u64 bytes_used; u64 chunk_start = 0; u64 chunk_size = 0; + u64 system_group_size = 0; int ret; + system_group_size = fs_info->fs_devices->hmzoned ? + fs_info->fs_devices->zone_size : BTRFS_MKFS_SYSTEM_GROUP_SIZE; + trans = btrfs_start_transaction(root, 1); BUG_ON(IS_ERR(trans)); bytes_used = btrfs_super_bytes_used(fs_info->super_copy); @@ -74,8 +78,8 @@ static int create_metadata_block_groups(struct btrfs_root *root, int mixed, ret = btrfs_make_block_group(trans, fs_info, bytes_used, BTRFS_BLOCK_GROUP_SYSTEM, BTRFS_BLOCK_RESERVED_1M_FOR_SUPER, - BTRFS_MKFS_SYSTEM_GROUP_SIZE); - allocation->system += BTRFS_MKFS_SYSTEM_GROUP_SIZE; + system_group_size); + allocation->system += system_group_size; if (ret) return ret; @@ -700,6 +704,7 @@ int main(int argc, char **argv) int metadata_profile_opt = 0; int discard = 1; int ssd = 0; + int hmzoned = 0; int force_overwrite = 0; char *source_dir = NULL; bool source_dir_set = false; @@ -713,6 +718,7 @@ int main(int argc, char **argv) u64 features = BTRFS_MKFS_DEFAULT_FEATURES; struct mkfs_allocation allocation = { 0 }; struct btrfs_mkfs_config mkfs_cfg; + u64 system_group_size; while(1) { int c; @@ -835,6 +841,8 @@ int main(int argc, char **argv) if (dev_cnt == 0) print_usage(1); + hmzoned = features & BTRFS_FEATURE_INCOMPAT_HMZONED; + if (source_dir_set && dev_cnt > 1) { error("the option -r is limited to a single device"); goto error; @@ -844,6 +852,11 @@ int main(int argc, char **argv) goto error; } + if (source_dir_set && hmzoned) { + error("The -r and hmzoned feature are incompatible\n"); + exit(1); + } + if (*fs_uuid) { uuid_t dummy_uuid; @@ -875,6 +888,16 @@ int main(int argc, char **argv) file = argv[optind++]; ssd = is_ssd(file); + if (hmzoned) { + if (zoned_model(file) == ZONED_NONE) { + error("%s: not a zoned block device\n", file); + exit(1); + } + if (!zone_size(file)) { + error("%s: zone size undefined\n", file); + exit(1); + } + } /* * Set default profiles according to number of added devices. @@ -1026,7 +1049,8 @@ int main(int argc, char **argv) ret = btrfs_prepare_device(fd, file, &dev_block_count, block_count, (zero_end ? PREP_DEVICE_ZERO_END : 0) | (discard ? PREP_DEVICE_DISCARD : 0) | - (verbose ? PREP_DEVICE_VERBOSE : 0)); + (verbose ? PREP_DEVICE_VERBOSE : 0) | + (hmzoned ? PREP_DEVICE_HMZONED : 0)); if (ret) goto error; if (block_count && block_count > dev_block_count) { @@ -1037,9 +1061,11 @@ int main(int argc, char **argv) } /* To create the first block group and chunk 0 in make_btrfs */ - if (dev_block_count < BTRFS_MKFS_SYSTEM_GROUP_SIZE) { + system_group_size = hmzoned ? + zone_size(file) : BTRFS_MKFS_SYSTEM_GROUP_SIZE; + if (dev_block_count < system_group_size) { error("device is too small to make filesystem, must be at least %llu", - (unsigned long long)BTRFS_MKFS_SYSTEM_GROUP_SIZE); + (unsigned long long)system_group_size); goto error; } @@ -1055,6 +1081,7 @@ int main(int argc, char **argv) mkfs_cfg.sectorsize = sectorsize; mkfs_cfg.stripesize = stripesize; mkfs_cfg.features = features; + mkfs_cfg.zone_size = zone_size(file); ret = make_btrfs(fd, &mkfs_cfg); if (ret) { @@ -1064,6 +1091,7 @@ int main(int argc, char **argv) fs_info = open_ctree_fs_info(file, 0, 0, 0, OPEN_CTREE_WRITES | OPEN_CTREE_TEMPORARY_SUPER); + if (!fs_info) { error("open ctree failed"); goto error; @@ -1137,7 +1165,8 @@ int main(int argc, char **argv) block_count, (verbose ? PREP_DEVICE_VERBOSE : 0) | (zero_end ? PREP_DEVICE_ZERO_END : 0) | - (discard ? PREP_DEVICE_DISCARD : 0)); + (discard ? PREP_DEVICE_DISCARD : 0) | + (hmzoned ? PREP_DEVICE_HMZONED : 0)); if (ret) { goto error; } @@ -1234,6 +1263,10 @@ raid_groups: btrfs_group_profile_str(metadata_profile), pretty_size(allocation.system)); printf("SSD detected: %s\n", ssd ? "yes" : "no"); + printf("Zoned device: %s\n", hmzoned ? "yes" : "no"); + if (hmzoned) + printf("Zone size: %s\n", + pretty_size(fs_info->fs_devices->zone_size)); btrfs_parse_features_to_string(features_buf, features); printf("Incompat features: %s", features_buf); printf("\n"); From patchwork Thu Aug 9 18:11:03 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naohiro Aota X-Patchwork-Id: 10561659 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id B47B713B4 for ; Thu, 9 Aug 2018 18:11:39 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id A10092B837 for ; Thu, 9 Aug 2018 18:11:39 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 949902B861; Thu, 9 Aug 2018 18:11:39 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.8 required=2.0 tests=BAYES_00,DKIM_SIGNED, MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI,T_DKIM_INVALID autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 271032B7FB for ; Thu, 9 Aug 2018 18:11:39 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727513AbeHIUhh (ORCPT ); Thu, 9 Aug 2018 16:37:37 -0400 Received: from mail-pl0-f65.google.com ([209.85.160.65]:38694 "EHLO mail-pl0-f65.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727390AbeHIUhh (ORCPT ); Thu, 9 Aug 2018 16:37:37 -0400 Received: by mail-pl0-f65.google.com with SMTP id u11-v6so2881103plq.5; Thu, 09 Aug 2018 11:11:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:from:to:cc:subject:date:message-id:in-reply-to:references; bh=hE4t/T5xBCjUgD1IFsfSM9RZtqZj7metO9MTlR+zsb8=; b=Xk+K3KwUHXcywGM+K3ut6NnKXWYEqvXnqXIz2VvWDycdVMxNYfdkSGfQATZ0aIMiKp 1k0TWycLBJmcxbU5pLVJkEy85akmJPaupu1JfdhFso09KTcootzUISgUI57Pl9LLwJdX q31Vb+tun4nLl7X7AAnTf2kUd4bkYc7TdT68qEulTDDrF2BOiDYWqHzUaROTspX+ElJR k5c8aCbaTDBlIciziy0Wnpkiwv1BTJYkeBBDPgOF3N+9/LIMtaEkJIAQfNtMg+5r1pzA SwJVI1jUQHAUwKQAmVZPQlZzr5i5SBLdXR1EfjSblxNBDhs0V4iJ/1gTque3BsnjG5ia sqUg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:from:to:cc:subject:date:message-id :in-reply-to:references; bh=hE4t/T5xBCjUgD1IFsfSM9RZtqZj7metO9MTlR+zsb8=; b=Ssa2V18CEL/BAurzSuCRsX6aSkcrrCpcii7Bq1cxvKtk5cN8UzNZ1N/vFlpuo6fAUT edLFIEioCNnndsp1YlynB8PTgpOw0amp5XEQt99SDT9b3CJMzh5Fadslj18L408oGxLs 2MsX42B6fNVz7d4tAkFeuFLGxBhzl7gMsgrJhSUsfkHJGdmRmb+ea87cgurXfgGRLCHI VV8kgoPT2vMR9P7SP3gz/rSqlA244eYaGV48sEUkCM1tCeQI9un75m7rrT/Znb+nd2kv 0hYWTj5p8aJxj6gcWtYkN19YY/uYEkFguMWtwDx7CbAz/yKSM1CuCRdCBTMTcHXpUDUc fOdg== X-Gm-Message-State: AOUpUlFxe2vU8BeVals/9pn2DVzuCHBC5Cdwr3JWq8XI8LXuj9GXJ+6T aqFrFUXxxCCCNJwS2fzwDmw= X-Google-Smtp-Source: AA+uWPx0Ok9f+uiGer0s+0tAOTlYBfKqeTl23atym9n4jQS7pJsLD94drVD1gdLgtwrGLPL80nqmnA== X-Received: by 2002:a17:902:6105:: with SMTP id t5-v6mr3025295plj.92.1533838295955; Thu, 09 Aug 2018 11:11:35 -0700 (PDT) Received: from localhost (h101-111-148-072.catv02.itscom.jp. [101.111.148.72]) by smtp.gmail.com with ESMTPSA id t12-v6sm14913213pgg.72.2018.08.09.11.11.34 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 09 Aug 2018 11:11:35 -0700 (PDT) From: Naohiro Aota To: David Sterba , linux-btrfs@vger.kernel.org Cc: Chris Mason , Josef Bacik , linux-kernel@vger.kernel.org, Hannes Reinecke , Damien Le Moal , Bart Van Assche , Matias Bjorling , Naohiro Aota Subject: [RFC PATCH 10/12] btrfs-progs: device-add: support HMZONED device Date: Fri, 10 Aug 2018 03:11:03 +0900 Message-Id: <20180809181105.12856-10-naota@elisp.net> X-Mailer: git-send-email 2.18.0 In-Reply-To: <20180809181105.12856-1-naota@elisp.net> References: <20180809180450.5091-1-naota@elisp.net> <20180809181105.12856-1-naota@elisp.net> Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP This patch check if the target file system is flagged as HMZONED. If it is, the device to be added is flagged PREP_DEVICE_HMZONED. Also add checks to prevent mixing non-zoned devices and zoned devices. Signed-off-by: Naohiro Aota --- cmds-device.c | 29 +++++++++++++++++++++++++++-- 1 file changed, 27 insertions(+), 2 deletions(-) diff --git a/cmds-device.c b/cmds-device.c index 2a05f70a..10696bf7 100644 --- a/cmds-device.c +++ b/cmds-device.c @@ -56,6 +56,9 @@ static int cmd_device_add(int argc, char **argv) int discard = 1; int force = 0; int last_dev; + int res; + int hmzoned; + struct btrfs_ioctl_feature_flags feature_flags; optind = 0; while (1) { @@ -91,12 +94,33 @@ static int cmd_device_add(int argc, char **argv) if (fdmnt < 0) return 1; + res = ioctl(fdmnt, BTRFS_IOC_GET_FEATURES, &feature_flags); + if (res) { + error("error getting feature flags '%s': %m", mntpnt); + return 1; + } + hmzoned = feature_flags.incompat_flags & BTRFS_FEATURE_INCOMPAT_HMZONED; + for (i = optind; i < last_dev; i++){ struct btrfs_ioctl_vol_args ioctl_args; - int devfd, res; + int devfd; u64 dev_block_count = 0; char *path; + if (hmzoned && zoned_model(argv[i]) == ZONED_NONE) { + error("cannot add non-zoned device to HMZONED file system '%s'", + argv[i]); + ret++; + continue; + } + + if (!hmzoned && zoned_model(argv[i]) == ZONED_HOST_MANAGED) { + error("cannot add host managed zoned device to non-HMZONED file system '%s'", + argv[i]); + ret++; + continue; + } + res = test_dev_for_mkfs(argv[i], force); if (res) { ret++; @@ -112,7 +136,8 @@ static int cmd_device_add(int argc, char **argv) res = btrfs_prepare_device(devfd, argv[i], &dev_block_count, 0, PREP_DEVICE_ZERO_END | PREP_DEVICE_VERBOSE | - (discard ? PREP_DEVICE_DISCARD : 0)); + (discard ? PREP_DEVICE_DISCARD : 0) | + (hmzoned ? PREP_DEVICE_HMZONED : 0)); close(devfd); if (res) { ret++; From patchwork Thu Aug 9 18:11:04 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naohiro Aota X-Patchwork-Id: 10561667 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id AD17C13B4 for ; Thu, 9 Aug 2018 18:12:07 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 985802B7E6 for ; Thu, 9 Aug 2018 18:12:07 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 8C9A82B7FB; Thu, 9 Aug 2018 18:12:07 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.8 required=2.0 tests=BAYES_00,DKIM_SIGNED, MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI,T_DKIM_INVALID autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 338852B7E6 for ; Thu, 9 Aug 2018 18:12:07 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727549AbeHIUhm (ORCPT ); Thu, 9 Aug 2018 16:37:42 -0400 Received: from mail-pg1-f195.google.com ([209.85.215.195]:41401 "EHLO mail-pg1-f195.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727537AbeHIUhk (ORCPT ); Thu, 9 Aug 2018 16:37:40 -0400 Received: by mail-pg1-f195.google.com with SMTP id z8-v6so3112964pgu.8; Thu, 09 Aug 2018 11:11:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:from:to:cc:subject:date:message-id:in-reply-to:references; bh=K9pEvg/1xtQMCMUm7eNfeyd82C7DFAaxfDmz/G7l5JI=; b=lvhmBoUvi1sgETLgmrkk5/eJG+QjX7O20Kipqn2vLQt1OaFSYGpF0I5Sok1tWryjmT ivKJvmWenEck4SeICntWYPAjfgXXfjMPV15a1hHUPvTlL5qX0sZ9JlYfR+3zVph8G0A4 JQ0wJr4ewwYUA7YAHaRZd8trcMbFCNY+bll1/FE/07yQ1OXFFMWb7N5e5A1bXJPQQY8j aS+YSYNZhbe1TfuwRh5GVmQKmSwxY3Lxkn0I1xzDZ7Q5tVk6SHB/p6LGEKmOHZJ1jRBL 91XzmtQPS2Vaw7wnt/hIHiEk1K2HjUhVQYozCxc9Wgon1FOwfIaiOfRIOCOicWY8Z55x 7kvA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:from:to:cc:subject:date:message-id :in-reply-to:references; bh=K9pEvg/1xtQMCMUm7eNfeyd82C7DFAaxfDmz/G7l5JI=; b=QJ1e3rX6uzWnttnO+MSZYGJNZskSyHv8hgBMUKTHARB9mlw/+sEtt2+cN/i3lTAreT x+m8xdfLHCsUb24stLmFWAYsqjLKYixvdkpA9Gjgzr8RIndlhVpgnNMZswOdSfBNzbqU lzNptP1/r70Xb+KKCyxsprRw9uzUO0p/nLcjlzun2Bw0mz4jnzW1N/a9OdbqKj93XAqa IHsR3i4iNYm+Z24krnBcY107fqHb7RT+sS+L2WbR4ImBqIryEdk+CsLMtGCMqIMk4o9m +DS1Mz2cL0UK0wU2d+D7wnJtaIgR7M+/2L6QV3OUlkLo/Ao5uyND5XiRv9b8TDvGFmdS K7zg== X-Gm-Message-State: AOUpUlFPvOPlY8XePLT4eXC8AQg6lcpGDT1iKkjoVgik2dt9Zii1hBQ2 7tdr25IVa88DebBCkuOPjhw= X-Google-Smtp-Source: AA+uWPzaTMXOmHRmVsxbO/K96mYFCFENRcXwFP4B/ytP2Oh9PKVkbRvrQyAM8a5KxKjQanghoVoIQQ== X-Received: by 2002:a63:1360:: with SMTP id 32-v6mr3200393pgt.159.1533838298870; Thu, 09 Aug 2018 11:11:38 -0700 (PDT) Received: from localhost (h101-111-148-072.catv02.itscom.jp. [101.111.148.72]) by smtp.gmail.com with ESMTPSA id t12-v6sm14913273pgg.72.2018.08.09.11.11.37 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 09 Aug 2018 11:11:38 -0700 (PDT) From: Naohiro Aota To: David Sterba , linux-btrfs@vger.kernel.org Cc: Chris Mason , Josef Bacik , linux-kernel@vger.kernel.org, Hannes Reinecke , Damien Le Moal , Bart Van Assche , Matias Bjorling , Naohiro Aota Subject: [RFC PATCH 11/12] btrfs-progs: replace: disable in HMZONED device Date: Fri, 10 Aug 2018 03:11:04 +0900 Message-Id: <20180809181105.12856-11-naota@elisp.net> X-Mailer: git-send-email 2.18.0 In-Reply-To: <20180809181105.12856-1-naota@elisp.net> References: <20180809180450.5091-1-naota@elisp.net> <20180809181105.12856-1-naota@elisp.net> Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP As show in the kernel patches, device replace feature needs more works to complete. Disable the feature for now. Signed-off-by: Naohiro Aota --- cmds-replace.c | 15 +++++++++++++++ 1 file changed, 15 insertions(+) diff --git a/cmds-replace.c b/cmds-replace.c index 1fa80284..642fbd4b 100644 --- a/cmds-replace.c +++ b/cmds-replace.c @@ -116,6 +116,7 @@ static const char *const cmd_replace_start_usage[] = { static int cmd_replace_start(int argc, char **argv) { + struct btrfs_ioctl_feature_flags feature_flags; struct btrfs_ioctl_dev_replace_args start_args = {0}; struct btrfs_ioctl_dev_replace_args status_args = {0}; int ret; @@ -123,6 +124,7 @@ static int cmd_replace_start(int argc, char **argv) int c; int fdmnt = -1; int fddstdev = -1; + int hmzoned; char *path; char *srcdev; char *dstdev = NULL; @@ -200,6 +202,13 @@ static int cmd_replace_start(int argc, char **argv) goto leave_with_error; } + ret = ioctl(fdmnt, BTRFS_IOC_GET_FEATURES, &feature_flags); + if (ret) { + error("error getting feature flags '%s': %m", path); + return 1; + } + hmzoned = feature_flags.incompat_flags & BTRFS_FEATURE_INCOMPAT_HMZONED; + if (string_is_numerical(srcdev)) { struct btrfs_ioctl_fs_info_args fi_args; struct btrfs_ioctl_dev_info_args *di_args = NULL; @@ -238,6 +247,12 @@ static int cmd_replace_start(int argc, char **argv) goto leave_with_error; } + if (hmzoned) { + error("cannot replace device on HMZONED file system '%s'", + dstdev); + goto leave_with_error; + } + ret = test_dev_for_mkfs(dstdev, force_using_targetdev); if (ret) goto leave_with_error; From patchwork Thu Aug 9 18:11:05 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naohiro Aota X-Patchwork-Id: 10561661 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id DC13E13BB for ; Thu, 9 Aug 2018 18:11:46 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id C63E72B7C5 for ; Thu, 9 Aug 2018 18:11:46 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id BA3062B805; Thu, 9 Aug 2018 18:11:46 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.8 required=2.0 tests=BAYES_00,DKIM_SIGNED, MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI,T_DKIM_INVALID autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 010962B7E6 for ; Thu, 9 Aug 2018 18:11:45 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727640AbeHIUhp (ORCPT ); Thu, 9 Aug 2018 16:37:45 -0400 Received: from mail-pg1-f195.google.com ([209.85.215.195]:44684 "EHLO mail-pg1-f195.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727588AbeHIUho (ORCPT ); Thu, 9 Aug 2018 16:37:44 -0400 Received: by mail-pg1-f195.google.com with SMTP id r1-v6so3104087pgp.11; Thu, 09 Aug 2018 11:11:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:from:to:cc:subject:date:message-id:in-reply-to:references; bh=Pr49YW5PHK+o+EJjtD+vL6jwy1YFuuBE4eYOx9iZcVM=; b=o7UpqvyKt16Xgj42Ctrm2uAd591IBxA5HJx/nHKceRwk4b2pTO3RJPsyBUisrMtAhr 1V9VFm9QjB0zTNtaO9BIwCDopfAQs0nT3W6dRs/wKoxOUHs6Q6StRxf90QomhSFKHkiF O9McV9ncdyv0xM1iJcVLE5Fh/d0Dlr4Sf8XegTdh14gdob1JhkhUyFgE1El4qSgsnI6a 7py+9sLCPTl+LioKqP2UtdaamFsHcdpVUFX91qxuEysDgK18SxtlxIOTobwqY0A0UvfA WCtSVH9yIFcFUjsDNMQSnfayRbLhuE9/rvFBkBZ0/gvmLLnj70HgS5PexnCygIg6kgrc cAyw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:from:to:cc:subject:date:message-id :in-reply-to:references; bh=Pr49YW5PHK+o+EJjtD+vL6jwy1YFuuBE4eYOx9iZcVM=; b=Q42OkKCU+fjksZK7Mnqtr0EKG8toxFyfh1kY6yTl3c6WJK9V3/Vql3njzT1+ukjhdB EnWN1QEtpqe3zHzX3wxGGW4kwAD02t9pVAa4C82Mv7GcjpiemKYljT06keJvMPN+jLcJ UPhNOHId8ep0CgTw1Wn4Q4BYfN9uxDpOk5ePKh594Cg+VrYlBdL2sGHyy+IK9E9kijK+ 4gNRWuE2K8DyNBcztwbxYXcpFIJy/DiE4+Az/PIO755uFxUDrK9TG1XYg8i35P8gq3ea ziivr/CHm1w4qhdOJKQQhGUNzbcyS7Rx9eAIAfBOs3JT7cx6xucEqX2mlu7WTPblWn+7 5wew== X-Gm-Message-State: AOUpUlF0tYWYGczV3iyckLGwlDrknd1F4z1WKSfr2R6VL5Qbqxg587U7 P8AiYF87eh8UwhrkXyf1w1s= X-Google-Smtp-Source: AA+uWPyKPyHLTp0Za9vYTqCGPl52hXNTW4UWKvofYyHI2TM1zcSxPbVQ8I7ktJexoFCIrLZaVq8VPA== X-Received: by 2002:a63:e914:: with SMTP id i20-v6mr3179830pgh.10.1533838301913; Thu, 09 Aug 2018 11:11:41 -0700 (PDT) Received: from localhost (h101-111-148-072.catv02.itscom.jp. [101.111.148.72]) by smtp.gmail.com with ESMTPSA id z4-v6sm16536853pfl.11.2018.08.09.11.11.40 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 09 Aug 2018 11:11:41 -0700 (PDT) From: Naohiro Aota To: David Sterba , linux-btrfs@vger.kernel.org Cc: Chris Mason , Josef Bacik , linux-kernel@vger.kernel.org, Hannes Reinecke , Damien Le Moal , Bart Van Assche , Matias Bjorling , Naohiro Aota Subject: [RFC PATCH 12/12] btrfs-progs: do sequential allocation Date: Fri, 10 Aug 2018 03:11:05 +0900 Message-Id: <20180809181105.12856-12-naota@elisp.net> X-Mailer: git-send-email 2.18.0 In-Reply-To: <20180809181105.12856-1-naota@elisp.net> References: <20180809180450.5091-1-naota@elisp.net> <20180809181105.12856-1-naota@elisp.net> Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Ensures that block allocation in sequential write required zones is always done sequentially using an allocation pointer which is the zone write pointer plus the number of blocks already allocated but not yet written. For conventional zones, the legacy behavior is used. Signed-off-by: Naohiro Aota --- ctree.h | 17 +++++ extent-tree.c | 186 ++++++++++++++++++++++++++++++++++++++++++++++++++ transaction.c | 16 +++++ 3 files changed, 219 insertions(+) diff --git a/ctree.h b/ctree.h index 6d805ecd..5324f7b9 100644 --- a/ctree.h +++ b/ctree.h @@ -1062,15 +1062,32 @@ struct btrfs_space_info { struct list_head list; }; +/* Block group allocation types */ +enum btrfs_alloc_type { + + /* Regular first fit allocation */ + BTRFS_ALLOC_FIT = 0, + + /* + * Sequential allocation: this is for HMZONED mode and + * will result in ignoring free space before a block + * group allocation offset. + */ + BTRFS_ALLOC_SEQ = 1, +}; + struct btrfs_block_group_cache { struct cache_extent cache; struct btrfs_key key; struct btrfs_block_group_item item; struct btrfs_space_info *space_info; struct btrfs_free_space_ctl *free_space_ctl; + enum btrfs_alloc_type alloc_type; u64 bytes_super; u64 pinned; u64 flags; + u64 alloc_offset; + u64 write_offset; int cached; int ro; }; diff --git a/extent-tree.c b/extent-tree.c index 5d49af5a..01660864 100644 --- a/extent-tree.c +++ b/extent-tree.c @@ -256,6 +256,14 @@ again: if (cache->ro || !block_group_bits(cache, data)) goto new_group; + if (cache->alloc_type == BTRFS_ALLOC_SEQ) { + if (cache->key.offset - cache->alloc_offset < num) + goto new_group; + *start_ret = cache->key.objectid + cache->alloc_offset; + cache->alloc_offset += num; + return 0; + } + while(1) { ret = find_first_extent_bit(&root->fs_info->free_space_cache, last, &start, &end, EXTENT_DIRTY); @@ -282,6 +290,7 @@ out: (unsigned long long)search_start); return -ENOENT; } + printf("nospace\n"); return -ENOSPC; new_group: @@ -3143,6 +3152,176 @@ error: return ret; } +#ifdef BTRFS_ZONED +static int +btrfs_get_block_group_alloc_offset(struct btrfs_fs_info *fs_info, + struct btrfs_block_group_cache *cache) +{ + struct btrfs_device *device; + struct btrfs_mapping_tree *map_tree = &fs_info->mapping_tree; + struct cache_extent *ce; + struct map_lookup *map; + u64 logical = cache->key.objectid; + u64 length = cache->key.offset; + u64 physical = 0; + int ret = 0; + int i; + u64 zone_size = fs_info->fs_devices->zone_size; + u64 *alloc_offsets = NULL; + + if (!btrfs_fs_incompat(fs_info, HMZONED)) + return 0; + + /* Sanity check */ + if (!IS_ALIGNED(length, zone_size)) { + fprintf(stderr, "unaligned block group at %llu", logical); + return -EIO; + } + + /* Get the chunk mapping */ + ce = search_cache_extent(&map_tree->cache_tree, logical); + if (!ce) { + fprintf(stderr, "failed to find block group at %llu", logical); + return -ENOENT; + } + map = container_of(ce, struct map_lookup, ce); + + /* + * Get the zone type: if the group is mapped to a non-sequential zone, + * there is no need for the allocation offset (fit allocation is OK). + */ + device = map->stripes[0].dev; + physical = map->stripes[0].physical; + if (!zone_is_random_write(&device->zinfo, physical)) + cache->alloc_type = BTRFS_ALLOC_SEQ; + + /* check block group mapping */ + alloc_offsets = calloc(map->num_stripes, sizeof(*alloc_offsets)); + for (i = 0; i < map->num_stripes; i++) { + int is_sequential; + struct blk_zone zone; + + device = map->stripes[i].dev; + physical = map->stripes[i].physical; + + is_sequential = !zone_is_random_write(&device->zinfo, physical); + if ((is_sequential && cache->alloc_type != BTRFS_ALLOC_SEQ) || + (!is_sequential && cache->alloc_type == BTRFS_ALLOC_SEQ)) { + fprintf(stderr, + "found block group of mixed zone types"); + ret = -EIO; + goto out; + } + + if (!is_sequential) + continue; + + WARN_ON(!IS_ALIGNED(physical, zone_size)); + zone = device->zinfo.zones[physical / zone_size]; + + /* + * The group is mapped to a sequential zone. Get the zone write + * pointer to determine the allocation offset within the zone. + */ + switch (zone.cond) { + case BLK_ZONE_COND_OFFLINE: + case BLK_ZONE_COND_READONLY: + fprintf(stderr, "Offline/readonly zone %llu", + physical / fs_info->fs_devices->zone_size); + ret = -EIO; + goto out; + case BLK_ZONE_COND_EMPTY: + alloc_offsets[i] = 0; + break; + case BLK_ZONE_COND_FULL: + alloc_offsets[i] = zone_size; + break; + default: + /* Partially used zone */ + alloc_offsets[i] = ((zone.wp - zone.start) << 9); + break; + } + } + + if (cache->alloc_type != BTRFS_ALLOC_SEQ) + goto out; + + switch (map->type & BTRFS_BLOCK_GROUP_PROFILE_MASK) { + case 0: /* single */ + case BTRFS_BLOCK_GROUP_DUP: + case BTRFS_BLOCK_GROUP_RAID1: + for (i = 1; i < map->num_stripes; i++) { + if (alloc_offsets[i] != alloc_offsets[0]) { + fprintf(stderr, + "zones' write pointers mismatch\n"); + ret = -EIO; + goto out; + } + } + cache->alloc_offset = alloc_offsets[0]; + break; + case BTRFS_BLOCK_GROUP_RAID0: + cache->alloc_offset = alloc_offsets[0]; + for (i = 1; i < map->num_stripes; i++) { + cache->alloc_offset += alloc_offsets[i]; + if (alloc_offsets[0] < alloc_offsets[i]) { + fprintf(stderr, + "zones' write pointers mismatch\n"); + ret = -EIO; + goto out; + } + } + break; + case BTRFS_BLOCK_GROUP_RAID10: + cache->alloc_offset = 0; + for (i = 0; i < map->num_stripes / map->sub_stripes; i++) { + int j; + int base; + + base = i*map->sub_stripes; + for (j = 1; j < map->sub_stripes; j++) { + if (alloc_offsets[base] != + alloc_offsets[base+j]) { + fprintf(stderr, + "zones' write pointer mismatch\n"); + ret = -EIO; + goto out; + } + } + + if (alloc_offsets[0] < alloc_offsets[base]) { + fprintf(stderr, + "zones' write pointer mismatch\n"); + ret = -EIO; + goto out; + } + cache->alloc_offset += alloc_offsets[base]; + } + break; + case BTRFS_BLOCK_GROUP_RAID5: + case BTRFS_BLOCK_GROUP_RAID6: + /* RAID5/6 is not supported yet */ + default: + fprintf(stderr, "Unsupported profile %llu\n", + map->type & BTRFS_BLOCK_GROUP_PROFILE_MASK); + ret = -EINVAL; + goto out; + } + +out: + cache->write_offset = cache->alloc_offset; + free(alloc_offsets); + return ret; +} +#else +static int +btrfs_get_block_group_alloc_offset(struct btrfs_fs_info *fs_info, + struct btrfs_block_group_cache *cache) +{ + return 0; +} +#endif + int btrfs_read_block_groups(struct btrfs_root *root) { struct btrfs_path *path; @@ -3226,6 +3405,10 @@ int btrfs_read_block_groups(struct btrfs_root *root) BUG_ON(ret); cache->space_info = space_info; + ret = btrfs_get_block_group_alloc_offset(info, cache); + if (ret) + goto error; + /* use EXTENT_LOCKED to prevent merging */ set_extent_bits(block_group_cache, found_key.objectid, found_key.objectid + found_key.offset - 1, @@ -3255,6 +3438,9 @@ btrfs_add_block_group(struct btrfs_fs_info *fs_info, u64 bytes_used, u64 type, cache->key.objectid = chunk_offset; cache->key.offset = size; + ret = btrfs_get_block_group_alloc_offset(fs_info, cache); + BUG_ON(ret); + cache->key.type = BTRFS_BLOCK_GROUP_ITEM_KEY; btrfs_set_block_group_used(&cache->item, bytes_used); btrfs_set_block_group_chunk_objectid(&cache->item, diff --git a/transaction.c b/transaction.c index ecafbb15..0e49b8b7 100644 --- a/transaction.c +++ b/transaction.c @@ -115,16 +115,32 @@ int __commit_transaction(struct btrfs_trans_handle *trans, { u64 start; u64 end; + u64 next = 0; struct btrfs_fs_info *fs_info = root->fs_info; struct extent_buffer *eb; struct extent_io_tree *tree = &fs_info->extent_cache; + struct btrfs_block_group_cache *bg = NULL; int ret; while(1) { +again: ret = find_first_extent_bit(tree, 0, &start, &end, EXTENT_DIRTY); if (ret) break; + bg = btrfs_lookup_first_block_group(fs_info, start); + BUG_ON(!bg); + if (bg->alloc_type == BTRFS_ALLOC_SEQ && + bg->key.objectid + bg->write_offset < start) { + next = bg->key.objectid + bg->write_offset; + BUG_ON(next + fs_info->nodesize > start); + eb = btrfs_find_create_tree_block(fs_info, next); + btrfs_mark_buffer_dirty(eb); + free_extent_buffer(eb); + goto again; + } + if (bg->alloc_type == BTRFS_ALLOC_SEQ) + bg->write_offset += (end + 1 - start); while(start <= end) { eb = find_first_extent_buffer(tree, start); BUG_ON(!eb || eb->start != start);