From patchwork Mon Jan 28 21:24:27 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dennis Zhou X-Patchwork-Id: 10784807 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id E241C13B4 for ; Mon, 28 Jan 2019 21:24:50 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id CCB852B279 for ; Mon, 28 Jan 2019 21:24:50 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id BB6AB2B288; Mon, 28 Jan 2019 21:24:50 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 92CC72B279 for ; Mon, 28 Jan 2019 21:24:49 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728232AbfA1VYr (ORCPT ); Mon, 28 Jan 2019 16:24:47 -0500 Received: from mail-yb1-f193.google.com ([209.85.219.193]:41497 "EHLO mail-yb1-f193.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728111AbfA1VYr (ORCPT ); Mon, 28 Jan 2019 16:24:47 -0500 Received: by mail-yb1-f193.google.com with SMTP id n6so1916052ybg.8; Mon, 28 Jan 2019 13:24:46 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=urZHYaHVYTvaWvnMKkIGbrK6fCowBa9M7UNu2RVTVI8=; b=p/QRTCCL/QGTnFucc6pVxApqjerSboKfViaOdy5hYjE2pZwK7TGJXThqmkNz8bo3nP RCkc7OeDbNt3M0FClF47Y2m+SpUJSrOi8GQhshPgdtqIqevCZSxKzGtta78dHfxgch6x XNvlpoIiU1w4Xo7FQZ2bszVv0KBCZ39nhtu5OOddJI8wUb/ldMgxq97qDP+hTd1AGrE6 lJE0JczYKQnvh3fTozUNaSrw+qlJbj5U+Q4bUg50Xy67az+RobqaZiTkr7RoHi/P3E5t 2dkr8xCaeGKNOck/V0pugCWX0jg1ZNP1gxyovW9rOIrJX1OXNrRwb6BqS0m3HjO0AyjH qtoA== X-Gm-Message-State: AJcUukdfB2ijFKjBrIwAGf66IKAsyM8xrQenPHa7Tu6PP+ZEcGoDGNBd 4DDVDSTcfjnk3lWQRPs04iQ= X-Google-Smtp-Source: ALg8bN7gRS0+yO9J9n6aHbHUlmqnzBOIFagxyXJANSwcT7qja+wzR4Lg2kSaJJY6VCUz0yzIVev1wQ== X-Received: by 2002:a25:6ec5:: with SMTP id j188mr21995242ybc.129.1548710686109; Mon, 28 Jan 2019 13:24:46 -0800 (PST) Received: from dennisz-mbp.thefacebook.com ([199.201.65.135]) by smtp.gmail.com with ESMTPSA id a7sm24170415ywb.18.2019.01.28.13.24.44 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Mon, 28 Jan 2019 13:24:45 -0800 (PST) From: Dennis Zhou To: David Sterba , Josef Bacik , Chris Mason , Omar Sandoval , Nick Terrell Cc: kernel-team@fb.com, linux-btrfs@vger.kernel.org, linux-kernel@vger.kernel.org, Dennis Zhou Subject: [PATCH 01/11] btrfs: add macros for compression type and level Date: Mon, 28 Jan 2019 16:24:27 -0500 Message-Id: <20190128212437.11597-2-dennis@kernel.org> X-Mailer: git-send-email 2.13.5 In-Reply-To: <20190128212437.11597-1-dennis@kernel.org> References: <20190128212437.11597-1-dennis@kernel.org> Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP It is very easy to miss places that rely on a certain bitshifting for decyphering the type_level overloading. Make macros handle this instead. Signed-off-by: Dennis Zhou Reviewed-by: Nikolay Borisov Reviewed-by: Josef Bacik --- fs/btrfs/compression.c | 2 +- fs/btrfs/compression.h | 3 +++ fs/btrfs/zlib.c | 2 +- 3 files changed, 5 insertions(+), 2 deletions(-) diff --git a/fs/btrfs/compression.c b/fs/btrfs/compression.c index 548057630b69..586f95ac0aea 100644 --- a/fs/btrfs/compression.c +++ b/fs/btrfs/compression.c @@ -1036,9 +1036,9 @@ int btrfs_compress_pages(unsigned int type_level, struct address_space *mapping, unsigned long *total_in, unsigned long *total_out) { + int type = BTRFS_COMPRESS_TYPE(type_level); struct list_head *workspace; int ret; - int type = type_level & 0xF; workspace = find_workspace(type); diff --git a/fs/btrfs/compression.h b/fs/btrfs/compression.h index ddda9b80bf20..69a9197dadc3 100644 --- a/fs/btrfs/compression.h +++ b/fs/btrfs/compression.h @@ -25,6 +25,9 @@ #define BTRFS_ZLIB_DEFAULT_LEVEL 3 +#define BTRFS_COMPRESS_TYPE(type_level) (type_level & 0xF) +#define BTRFS_COMPRESS_LEVEL(type_level) ((type_level & 0xF0) >> 4) + struct compressed_bio { /* number of bios pending for this compressed extent */ refcount_t pending_bios; diff --git a/fs/btrfs/zlib.c b/fs/btrfs/zlib.c index 970ff3e35bb3..1480b3eee306 100644 --- a/fs/btrfs/zlib.c +++ b/fs/btrfs/zlib.c @@ -393,7 +393,7 @@ static int zlib_decompress(struct list_head *ws, unsigned char *data_in, static void zlib_set_level(struct list_head *ws, unsigned int type) { struct workspace *workspace = list_entry(ws, struct workspace, list); - unsigned level = (type & 0xF0) >> 4; + unsigned int level = BTRFS_COMPRESS_LEVEL(type); if (level > 9) level = 9; From patchwork Mon Jan 28 21:24:28 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dennis Zhou X-Patchwork-Id: 10784827 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id C8E2613B5 for ; Mon, 28 Jan 2019 21:25:55 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id B81102B279 for ; Mon, 28 Jan 2019 21:25:55 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id ACA172B288; Mon, 28 Jan 2019 21:25:55 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 491412B279 for ; Mon, 28 Jan 2019 21:25:55 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728574AbfA1VZy (ORCPT ); Mon, 28 Jan 2019 16:25:54 -0500 Received: from mail-yb1-f194.google.com ([209.85.219.194]:39603 "EHLO mail-yb1-f194.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728236AbfA1VYs (ORCPT ); Mon, 28 Jan 2019 16:24:48 -0500 Received: by mail-yb1-f194.google.com with SMTP id s17so7322686ybp.6; Mon, 28 Jan 2019 13:24:48 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=/WAqu0DX/+VHR8/AR+fAyEvYgy44PedCWuNMUiZ3ZwQ=; b=d9MVOrVNgSX2INSgNVifINeWCR6i0NxJLibRPOyo4/eMoenFfEIFSybljD/QP1XZ0D RnF2FMxIrDxRMVkRd7xV/dKDglq91Cf/wNnMK5MIzuA1OfDwD2+U7i0jn9BIrXQTAsMJ 3QV3LMt9F6Ht3+HQ8hdLgx8Shr7uIJNk1mlwVbpO2SACLK/OTjCkfIEBsQ96vv7tftnP s61ufYwwlOe2oLK+BJyjd/oFR1qkP7K18W8IVVL4sl63A/aaL5YlBRka1XhONjocx8yb 35TaHJXS8TFwv9xBIunYTMkw0Cb/HE5gCfHgpNDpIiUn72fx1dIOzLqUOu4OXao/WNE+ IA9Q== X-Gm-Message-State: AHQUAua1EnbjB5KWOo5t1VKmTuhroDNqlWOcplL+log5B+6o8zjccA5n gjD+Js98cgIQgSoxYt8plGu1vv7Feyc= X-Google-Smtp-Source: AHgI3IaVb36rwZpvhgoQDBTd7ET85JdVrNzWf6IindOBp+706Jv0fJWHaNbLuBj4zYOpe7fEkk7+Cw== X-Received: by 2002:a25:414f:: with SMTP id o76mr6477935yba.159.1548710687582; Mon, 28 Jan 2019 13:24:47 -0800 (PST) Received: from dennisz-mbp.thefacebook.com ([199.201.65.135]) by smtp.gmail.com with ESMTPSA id a7sm24170415ywb.18.2019.01.28.13.24.46 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Mon, 28 Jan 2019 13:24:46 -0800 (PST) From: Dennis Zhou To: David Sterba , Josef Bacik , Chris Mason , Omar Sandoval , Nick Terrell Cc: kernel-team@fb.com, linux-btrfs@vger.kernel.org, linux-kernel@vger.kernel.org, Dennis Zhou Subject: [PATCH 02/11] btrfs: rename workspaces_list to workspace_manager Date: Mon, 28 Jan 2019 16:24:28 -0500 Message-Id: <20190128212437.11597-3-dennis@kernel.org> X-Mailer: git-send-email 2.13.5 In-Reply-To: <20190128212437.11597-1-dennis@kernel.org> References: <20190128212437.11597-1-dennis@kernel.org> Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP This is in preparation for zstd compression levels. As each level will require different sized workspaces, workspaces_list is no longer a really fitting name. Signed-off-by: Dennis Zhou Reviewed-by: Nikolay Borisov Reviewed-by: Josef Bacik --- fs/btrfs/compression.c | 46 +++++++++++++++++++++--------------------- 1 file changed, 23 insertions(+), 23 deletions(-) diff --git a/fs/btrfs/compression.c b/fs/btrfs/compression.c index 586f95ac0aea..aced261984e2 100644 --- a/fs/btrfs/compression.c +++ b/fs/btrfs/compression.c @@ -769,7 +769,7 @@ static struct list_head *alloc_heuristic_ws(void) return ERR_PTR(-ENOMEM); } -struct workspaces_list { +struct workspace_manager { struct list_head idle_ws; spinlock_t ws_lock; /* Number of free workspaces */ @@ -780,9 +780,9 @@ struct workspaces_list { wait_queue_head_t ws_wait; }; -static struct workspaces_list btrfs_comp_ws[BTRFS_COMPRESS_TYPES]; +static struct workspace_manager wsm[BTRFS_COMPRESS_TYPES]; -static struct workspaces_list btrfs_heuristic_ws; +static struct workspace_manager btrfs_heuristic_ws; static const struct btrfs_compress_op * const btrfs_compress_op[] = { &btrfs_zlib_compress, @@ -811,10 +811,10 @@ void __init btrfs_init_compress(void) } for (i = 0; i < BTRFS_COMPRESS_TYPES; i++) { - INIT_LIST_HEAD(&btrfs_comp_ws[i].idle_ws); - spin_lock_init(&btrfs_comp_ws[i].ws_lock); - atomic_set(&btrfs_comp_ws[i].total_ws, 0); - init_waitqueue_head(&btrfs_comp_ws[i].ws_wait); + INIT_LIST_HEAD(&wsm[i].idle_ws); + spin_lock_init(&wsm[i].ws_lock); + atomic_set(&wsm[i].total_ws, 0); + init_waitqueue_head(&wsm[i].ws_wait); /* * Preallocate one workspace for each compression type so @@ -824,9 +824,9 @@ void __init btrfs_init_compress(void) if (IS_ERR(workspace)) { pr_warn("BTRFS: cannot preallocate compression workspace, will try later\n"); } else { - atomic_set(&btrfs_comp_ws[i].total_ws, 1); - btrfs_comp_ws[i].free_ws = 1; - list_add(workspace, &btrfs_comp_ws[i].idle_ws); + atomic_set(&wsm[i].total_ws, 1); + wsm[i].free_ws = 1; + list_add(workspace, &wsm[i].idle_ws); } } } @@ -856,11 +856,11 @@ static struct list_head *__find_workspace(int type, bool heuristic) ws_wait = &btrfs_heuristic_ws.ws_wait; free_ws = &btrfs_heuristic_ws.free_ws; } else { - idle_ws = &btrfs_comp_ws[idx].idle_ws; - ws_lock = &btrfs_comp_ws[idx].ws_lock; - total_ws = &btrfs_comp_ws[idx].total_ws; - ws_wait = &btrfs_comp_ws[idx].ws_wait; - free_ws = &btrfs_comp_ws[idx].free_ws; + idle_ws = &wsm[idx].idle_ws; + ws_lock = &wsm[idx].ws_lock; + total_ws = &wsm[idx].total_ws; + ws_wait = &wsm[idx].ws_wait; + free_ws = &wsm[idx].free_ws; } again: @@ -952,11 +952,11 @@ static void __free_workspace(int type, struct list_head *workspace, ws_wait = &btrfs_heuristic_ws.ws_wait; free_ws = &btrfs_heuristic_ws.free_ws; } else { - idle_ws = &btrfs_comp_ws[idx].idle_ws; - ws_lock = &btrfs_comp_ws[idx].ws_lock; - total_ws = &btrfs_comp_ws[idx].total_ws; - ws_wait = &btrfs_comp_ws[idx].ws_wait; - free_ws = &btrfs_comp_ws[idx].free_ws; + idle_ws = &wsm[idx].idle_ws; + ws_lock = &wsm[idx].ws_lock; + total_ws = &wsm[idx].total_ws; + ws_wait = &wsm[idx].ws_wait; + free_ws = &wsm[idx].free_ws; } spin_lock(ws_lock); @@ -998,11 +998,11 @@ static void free_workspaces(void) } for (i = 0; i < BTRFS_COMPRESS_TYPES; i++) { - while (!list_empty(&btrfs_comp_ws[i].idle_ws)) { - workspace = btrfs_comp_ws[i].idle_ws.next; + while (!list_empty(&wsm[i].idle_ws)) { + workspace = wsm[i].idle_ws.next; list_del(workspace); btrfs_compress_op[i]->free_workspace(workspace); - atomic_dec(&btrfs_comp_ws[i].total_ws); + atomic_dec(&wsm[i].total_ws); } } } From patchwork Mon Jan 28 21:24:29 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dennis Zhou X-Patchwork-Id: 10784809 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 034E213B4 for ; Mon, 28 Jan 2019 21:24:54 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id E5D2E2B279 for ; Mon, 28 Jan 2019 21:24:53 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id D9CB02B288; Mon, 28 Jan 2019 21:24:53 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 20E542B279 for ; Mon, 28 Jan 2019 21:24:53 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728293AbfA1VYv (ORCPT ); Mon, 28 Jan 2019 16:24:51 -0500 Received: from mail-yw1-f66.google.com ([209.85.161.66]:44115 "EHLO mail-yw1-f66.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728274AbfA1VYu (ORCPT ); Mon, 28 Jan 2019 16:24:50 -0500 Received: by mail-yw1-f66.google.com with SMTP id b63so7334597ywc.11; Mon, 28 Jan 2019 13:24:49 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=mffI2+SI9kSKwDwOcuUxQxRb/or/4TAhqInDe+WLfoQ=; b=KG3biWfV3l6CMi2Dr5rDBs9f065iu5BDP9c5SPotVH44LBsY4w0Ufq7clSMKZgHFsk ucXgMWrkluKDsyyDL9bOITZMZckRJUAoEFZIyTZ7cAesGP8sinJoAOUykgfy3wjJNdLi YeZ7XXzuaJFQzpYd89qtLjgd2jPm9E6qioYxsyy4I0FXnUZq+1IbOnf1IRLho1GTTMFD 54OWs4PWBOj5Lo1swKTRukJViXnMUhUGjjYbzOTyYROfpfo5er1G63RcW1h+08i7WM/7 Am+bhRGzJkQjWm+6wbYfEKJLW2V77zw2h9z8YPUeMKaJJKMSM8MOMPPx5rqpAUZbeys2 i5sA== X-Gm-Message-State: AJcUukciiiX7cYWnaupN5pIoouGr+2zxBZ38cLmG8zoDyT9bWX3IeetG NTD/AqkzqYtZd4ApHZfrMWJJDY41BtI= X-Google-Smtp-Source: ALg8bN6gVOdA/3XbYurcL0Ff54OZDwOi+x5cP6ahzHIe5UNI32JMkp+Ie3vj96kDSSfj/Mu08Q46sw== X-Received: by 2002:a0d:c182:: with SMTP id c124mr22508227ywd.190.1548710688956; Mon, 28 Jan 2019 13:24:48 -0800 (PST) Received: from dennisz-mbp.thefacebook.com ([199.201.65.135]) by smtp.gmail.com with ESMTPSA id a7sm24170415ywb.18.2019.01.28.13.24.47 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Mon, 28 Jan 2019 13:24:48 -0800 (PST) From: Dennis Zhou To: David Sterba , Josef Bacik , Chris Mason , Omar Sandoval , Nick Terrell Cc: kernel-team@fb.com, linux-btrfs@vger.kernel.org, linux-kernel@vger.kernel.org, Dennis Zhou Subject: [PATCH 03/11] btrfs: manage heuristic workspace as index 0 Date: Mon, 28 Jan 2019 16:24:29 -0500 Message-Id: <20190128212437.11597-4-dennis@kernel.org> X-Mailer: git-send-email 2.13.5 In-Reply-To: <20190128212437.11597-1-dennis@kernel.org> References: <20190128212437.11597-1-dennis@kernel.org> Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP While the heuristic workspaces aren't really compression workspaces, they use the same interface for managing them. So rather than branching, let's just handle them once again as the index 0 compression type. Signed-off-by: Dennis Zhou Reviewed-by: Nikolay Borisov albeit one minor nit Reviewed-by: Josef Bacik --- fs/btrfs/compression.c | 107 +++++++++++----------------------------- fs/btrfs/compression.h | 3 +- fs/btrfs/ioctl.c | 2 +- fs/btrfs/tree-checker.c | 4 +- 4 files changed, 33 insertions(+), 83 deletions(-) diff --git a/fs/btrfs/compression.c b/fs/btrfs/compression.c index aced261984e2..bda7e8d2cbc7 100644 --- a/fs/btrfs/compression.c +++ b/fs/btrfs/compression.c @@ -37,6 +37,8 @@ const char* btrfs_compress_type2str(enum btrfs_compression_type type) case BTRFS_COMPRESS_ZSTD: case BTRFS_COMPRESS_NONE: return btrfs_compress_types[type]; + default: + return NULL; } return NULL; @@ -769,6 +771,11 @@ static struct list_head *alloc_heuristic_ws(void) return ERR_PTR(-ENOMEM); } +const struct btrfs_compress_op btrfs_heuristic_compress = { + .alloc_workspace = alloc_heuristic_ws, + .free_workspace = free_heuristic_ws, +}; + struct workspace_manager { struct list_head idle_ws; spinlock_t ws_lock; @@ -782,9 +789,8 @@ struct workspace_manager { static struct workspace_manager wsm[BTRFS_COMPRESS_TYPES]; -static struct workspace_manager btrfs_heuristic_ws; - static const struct btrfs_compress_op * const btrfs_compress_op[] = { + &btrfs_heuristic_compress, &btrfs_zlib_compress, &btrfs_lzo_compress, &btrfs_zstd_compress, @@ -795,21 +801,6 @@ void __init btrfs_init_compress(void) struct list_head *workspace; int i; - INIT_LIST_HEAD(&btrfs_heuristic_ws.idle_ws); - spin_lock_init(&btrfs_heuristic_ws.ws_lock); - atomic_set(&btrfs_heuristic_ws.total_ws, 0); - init_waitqueue_head(&btrfs_heuristic_ws.ws_wait); - - workspace = alloc_heuristic_ws(); - if (IS_ERR(workspace)) { - pr_warn( - "BTRFS: cannot preallocate heuristic workspace, will try later\n"); - } else { - atomic_set(&btrfs_heuristic_ws.total_ws, 1); - btrfs_heuristic_ws.free_ws = 1; - list_add(workspace, &btrfs_heuristic_ws.idle_ws); - } - for (i = 0; i < BTRFS_COMPRESS_TYPES; i++) { INIT_LIST_HEAD(&wsm[i].idle_ws); spin_lock_init(&wsm[i].ws_lock); @@ -837,11 +828,10 @@ void __init btrfs_init_compress(void) * Preallocation makes a forward progress guarantees and we do not return * errors. */ -static struct list_head *__find_workspace(int type, bool heuristic) +static struct list_head *find_workspace(int type) { struct list_head *workspace; int cpus = num_online_cpus(); - int idx = type - 1; unsigned nofs_flag; struct list_head *idle_ws; spinlock_t *ws_lock; @@ -849,19 +839,11 @@ static struct list_head *__find_workspace(int type, bool heuristic) wait_queue_head_t *ws_wait; int *free_ws; - if (heuristic) { - idle_ws = &btrfs_heuristic_ws.idle_ws; - ws_lock = &btrfs_heuristic_ws.ws_lock; - total_ws = &btrfs_heuristic_ws.total_ws; - ws_wait = &btrfs_heuristic_ws.ws_wait; - free_ws = &btrfs_heuristic_ws.free_ws; - } else { - idle_ws = &wsm[idx].idle_ws; - ws_lock = &wsm[idx].ws_lock; - total_ws = &wsm[idx].total_ws; - ws_wait = &wsm[idx].ws_wait; - free_ws = &wsm[idx].free_ws; - } + idle_ws = &wsm[type].idle_ws; + ws_lock = &wsm[type].ws_lock; + total_ws = &wsm[type].total_ws; + ws_wait = &wsm[type].ws_wait; + free_ws = &wsm[type].free_ws; again: spin_lock(ws_lock); @@ -892,10 +874,7 @@ static struct list_head *__find_workspace(int type, bool heuristic) * context of btrfs_compress_bio/btrfs_compress_pages */ nofs_flag = memalloc_nofs_save(); - if (heuristic) - workspace = alloc_heuristic_ws(); - else - workspace = btrfs_compress_op[idx]->alloc_workspace(); + workspace = btrfs_compress_op[type]->alloc_workspace(); memalloc_nofs_restore(nofs_flag); if (IS_ERR(workspace)) { @@ -926,38 +905,23 @@ static struct list_head *__find_workspace(int type, bool heuristic) return workspace; } -static struct list_head *find_workspace(int type) -{ - return __find_workspace(type, false); -} - /* * put a workspace struct back on the list or free it if we have enough * idle ones sitting around */ -static void __free_workspace(int type, struct list_head *workspace, - bool heuristic) +static void free_workspace(int type, struct list_head *workspace) { - int idx = type - 1; struct list_head *idle_ws; spinlock_t *ws_lock; atomic_t *total_ws; wait_queue_head_t *ws_wait; int *free_ws; - if (heuristic) { - idle_ws = &btrfs_heuristic_ws.idle_ws; - ws_lock = &btrfs_heuristic_ws.ws_lock; - total_ws = &btrfs_heuristic_ws.total_ws; - ws_wait = &btrfs_heuristic_ws.ws_wait; - free_ws = &btrfs_heuristic_ws.free_ws; - } else { - idle_ws = &wsm[idx].idle_ws; - ws_lock = &wsm[idx].ws_lock; - total_ws = &wsm[idx].total_ws; - ws_wait = &wsm[idx].ws_wait; - free_ws = &wsm[idx].free_ws; - } + idle_ws = &wsm[type].idle_ws; + ws_lock = &wsm[type].ws_lock; + total_ws = &wsm[type].total_ws; + ws_wait = &wsm[type].ws_wait; + free_ws = &wsm[type].free_ws; spin_lock(ws_lock); if (*free_ws <= num_online_cpus()) { @@ -968,20 +932,12 @@ static void __free_workspace(int type, struct list_head *workspace, } spin_unlock(ws_lock); - if (heuristic) - free_heuristic_ws(workspace); - else - btrfs_compress_op[idx]->free_workspace(workspace); + btrfs_compress_op[type]->free_workspace(workspace); atomic_dec(total_ws); wake: cond_wake_up(ws_wait); } -static void free_workspace(int type, struct list_head *ws) -{ - return __free_workspace(type, ws, false); -} - /* * cleanup function for module exit */ @@ -990,13 +946,6 @@ static void free_workspaces(void) struct list_head *workspace; int i; - while (!list_empty(&btrfs_heuristic_ws.idle_ws)) { - workspace = btrfs_heuristic_ws.idle_ws.next; - list_del(workspace); - free_heuristic_ws(workspace); - atomic_dec(&btrfs_heuristic_ws.total_ws); - } - for (i = 0; i < BTRFS_COMPRESS_TYPES; i++) { while (!list_empty(&wsm[i].idle_ws)) { workspace = wsm[i].idle_ws.next; @@ -1042,8 +991,8 @@ int btrfs_compress_pages(unsigned int type_level, struct address_space *mapping, workspace = find_workspace(type); - btrfs_compress_op[type - 1]->set_level(workspace, type_level); - ret = btrfs_compress_op[type-1]->compress_pages(workspace, mapping, + btrfs_compress_op[type]->set_level(workspace, type_level); + ret = btrfs_compress_op[type]->compress_pages(workspace, mapping, start, pages, out_pages, total_in, total_out); @@ -1072,7 +1021,7 @@ static int btrfs_decompress_bio(struct compressed_bio *cb) int type = cb->compress_type; workspace = find_workspace(type); - ret = btrfs_compress_op[type - 1]->decompress_bio(workspace, cb); + ret = btrfs_compress_op[type]->decompress_bio(workspace, cb); free_workspace(type, workspace); return ret; @@ -1091,7 +1040,7 @@ int btrfs_decompress(int type, unsigned char *data_in, struct page *dest_page, workspace = find_workspace(type); - ret = btrfs_compress_op[type-1]->decompress(workspace, data_in, + ret = btrfs_compress_op[type]->decompress(workspace, data_in, dest_page, start_byte, srclen, destlen); @@ -1512,7 +1461,7 @@ static void heuristic_collect_sample(struct inode *inode, u64 start, u64 end, */ int btrfs_compress_heuristic(struct inode *inode, u64 start, u64 end) { - struct list_head *ws_list = __find_workspace(0, true); + struct list_head *ws_list = find_workspace(0); struct heuristic_ws *ws; u32 i; u8 byte; @@ -1581,7 +1530,7 @@ int btrfs_compress_heuristic(struct inode *inode, u64 start, u64 end) } out: - __free_workspace(0, ws_list, true); + free_workspace(0, ws_list); return ret; } diff --git a/fs/btrfs/compression.h b/fs/btrfs/compression.h index 69a9197dadc3..53a8b9e93217 100644 --- a/fs/btrfs/compression.h +++ b/fs/btrfs/compression.h @@ -97,7 +97,7 @@ enum btrfs_compression_type { BTRFS_COMPRESS_ZLIB = 1, BTRFS_COMPRESS_LZO = 2, BTRFS_COMPRESS_ZSTD = 3, - BTRFS_COMPRESS_TYPES = 3, + BTRFS_COMPRESS_TYPES = 4, }; struct btrfs_compress_op { @@ -125,6 +125,7 @@ struct btrfs_compress_op { void (*set_level)(struct list_head *ws, unsigned int type); }; +extern const struct btrfs_compress_op btrfs_heuristic_compress; extern const struct btrfs_compress_op btrfs_zlib_compress; extern const struct btrfs_compress_op btrfs_lzo_compress; extern const struct btrfs_compress_op btrfs_zstd_compress; diff --git a/fs/btrfs/ioctl.c b/fs/btrfs/ioctl.c index 9c8e1734429c..20081465a451 100644 --- a/fs/btrfs/ioctl.c +++ b/fs/btrfs/ioctl.c @@ -1410,7 +1410,7 @@ int btrfs_defrag_file(struct inode *inode, struct file *file, return -EINVAL; if (do_compress) { - if (range->compress_type > BTRFS_COMPRESS_TYPES) + if (range->compress_type >= BTRFS_COMPRESS_TYPES) return -EINVAL; if (range->compress_type) compress_type = range->compress_type; diff --git a/fs/btrfs/tree-checker.c b/fs/btrfs/tree-checker.c index a62e1e837a89..c88e146d8e99 100644 --- a/fs/btrfs/tree-checker.c +++ b/fs/btrfs/tree-checker.c @@ -133,9 +133,9 @@ static int check_extent_data_item(struct btrfs_fs_info *fs_info, * Support for new compression/encryption must introduce incompat flag, * and must be caught in open_ctree(). */ - if (btrfs_file_extent_compression(leaf, fi) > BTRFS_COMPRESS_TYPES) { + if (btrfs_file_extent_compression(leaf, fi) >= BTRFS_COMPRESS_TYPES) { file_extent_err(fs_info, leaf, slot, - "invalid compression for file extent, have %u expect range [0, %u]", + "invalid compression for file extent, have %u expect range [0, %u)", btrfs_file_extent_compression(leaf, fi), BTRFS_COMPRESS_TYPES); return -EUCLEAN; From patchwork Mon Jan 28 21:24:30 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dennis Zhou X-Patchwork-Id: 10784825 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id B724413BF for ; Mon, 28 Jan 2019 21:25:50 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id A57B92B279 for ; Mon, 28 Jan 2019 21:25:50 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 9A06D2B288; Mon, 28 Jan 2019 21:25:50 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 45C082B279 for ; Mon, 28 Jan 2019 21:25:50 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728545AbfA1VZo (ORCPT ); Mon, 28 Jan 2019 16:25:44 -0500 Received: from mail-yb1-f195.google.com ([209.85.219.195]:45459 "EHLO mail-yb1-f195.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728111AbfA1VYv (ORCPT ); Mon, 28 Jan 2019 16:24:51 -0500 Received: by mail-yb1-f195.google.com with SMTP id t11so2165190ybi.12; Mon, 28 Jan 2019 13:24:50 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=V9/6QDtWudym7hvxLzXEVyuXETxb6FE0Jk6cpZNUt5M=; b=jSe29YILpeXOWAC91sVqt35NDpDGKkW+KThhEl3Aqxz9xOS5X8MAPcXC9wN0F/W32+ qt4Cr0Sxjhkinci3HY9onNEGd52V8jEPaH+u22k0Be37egQDD73RKLsv/Ke1+M0HMrGW tyMbKKiMMfMs7ohh6DDpMdYjHmGUJrk1M1yiOgvDuNBRNxHmF6m6FVkRlGeQfwU2VKYs s7HZVgmmwVFCD8ZU4WBrpKnZKdWCKGpWj2gwg6tKoGPDxzP/9X6moZwPXHoEjiLeAttp UuSrdf0ncDqY4h3cWQVshvhBywWRYArP3ZEFQEe09FmY3Q0Njw4cqJ0VUNgE+INWCbP5 uO2A== X-Gm-Message-State: AJcUukclGPK6bMksn6Dk2n0GQK6nYF6ZBT9eOJZY02kjylAkeKdibBSn 3A5eX869mGVnrqjqnYJfYac= X-Google-Smtp-Source: ALg8bN7KZDdZErYD7mTbh9im250KZSnbK4Opl2eppGF1IYPoGva8TiMYQd8rEuR/STISn1knirlSpg== X-Received: by 2002:a25:2f92:: with SMTP id v140mr22633996ybv.257.1548710690360; Mon, 28 Jan 2019 13:24:50 -0800 (PST) Received: from dennisz-mbp.thefacebook.com ([199.201.65.135]) by smtp.gmail.com with ESMTPSA id a7sm24170415ywb.18.2019.01.28.13.24.48 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Mon, 28 Jan 2019 13:24:49 -0800 (PST) From: Dennis Zhou To: David Sterba , Josef Bacik , Chris Mason , Omar Sandoval , Nick Terrell Cc: kernel-team@fb.com, linux-btrfs@vger.kernel.org, linux-kernel@vger.kernel.org, Dennis Zhou Subject: [PATCH 04/11] btrfs: unify compression ops with workspace_manager Date: Mon, 28 Jan 2019 16:24:30 -0500 Message-Id: <20190128212437.11597-5-dennis@kernel.org> X-Mailer: git-send-email 2.13.5 In-Reply-To: <20190128212437.11597-1-dennis@kernel.org> References: <20190128212437.11597-1-dennis@kernel.org> Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Make the workspace_manager own the interface operations rather than managing index-paired arrays for the workspace_manager and compression operations. Signed-off-by: Dennis Zhou Reviewed-by: Nikolay Borisov Reviewed-by: Josef Bacik --- fs/btrfs/compression.c | 11 +++++++---- 1 file changed, 7 insertions(+), 4 deletions(-) diff --git a/fs/btrfs/compression.c b/fs/btrfs/compression.c index bda7e8d2cbc7..b7e986e16640 100644 --- a/fs/btrfs/compression.c +++ b/fs/btrfs/compression.c @@ -777,6 +777,7 @@ const struct btrfs_compress_op btrfs_heuristic_compress = { }; struct workspace_manager { + const struct btrfs_compress_op *ops; struct list_head idle_ws; spinlock_t ws_lock; /* Number of free workspaces */ @@ -802,6 +803,8 @@ void __init btrfs_init_compress(void) int i; for (i = 0; i < BTRFS_COMPRESS_TYPES; i++) { + wsm[i].ops = btrfs_compress_op[i]; + INIT_LIST_HEAD(&wsm[i].idle_ws); spin_lock_init(&wsm[i].ws_lock); atomic_set(&wsm[i].total_ws, 0); @@ -811,7 +814,7 @@ void __init btrfs_init_compress(void) * Preallocate one workspace for each compression type so * we can guarantee forward progress in the worst case */ - workspace = btrfs_compress_op[i]->alloc_workspace(); + workspace = wsm[i].ops->alloc_workspace(); if (IS_ERR(workspace)) { pr_warn("BTRFS: cannot preallocate compression workspace, will try later\n"); } else { @@ -874,7 +877,7 @@ static struct list_head *find_workspace(int type) * context of btrfs_compress_bio/btrfs_compress_pages */ nofs_flag = memalloc_nofs_save(); - workspace = btrfs_compress_op[type]->alloc_workspace(); + workspace = wsm[type].ops->alloc_workspace(); memalloc_nofs_restore(nofs_flag); if (IS_ERR(workspace)) { @@ -932,7 +935,7 @@ static void free_workspace(int type, struct list_head *workspace) } spin_unlock(ws_lock); - btrfs_compress_op[type]->free_workspace(workspace); + wsm[type].ops->free_workspace(workspace); atomic_dec(total_ws); wake: cond_wake_up(ws_wait); @@ -950,7 +953,7 @@ static void free_workspaces(void) while (!list_empty(&wsm[i].idle_ws)) { workspace = wsm[i].idle_ws.next; list_del(workspace); - btrfs_compress_op[i]->free_workspace(workspace); + wsm[i].ops->free_workspace(workspace); atomic_dec(&wsm[i].total_ws); } } From patchwork Mon Jan 28 21:24:31 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dennis Zhou X-Patchwork-Id: 10784821 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 4F17C13B4 for ; Mon, 28 Jan 2019 21:25:34 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 3E4202B279 for ; Mon, 28 Jan 2019 21:25:34 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 329302B288; Mon, 28 Jan 2019 21:25:34 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id C5C462B279 for ; Mon, 28 Jan 2019 21:25:33 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728343AbfA1VYy (ORCPT ); Mon, 28 Jan 2019 16:24:54 -0500 Received: from mail-yb1-f194.google.com ([209.85.219.194]:36597 "EHLO mail-yb1-f194.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728316AbfA1VYw (ORCPT ); Mon, 28 Jan 2019 16:24:52 -0500 Received: by mail-yb1-f194.google.com with SMTP id 64so2823085ybf.3; Mon, 28 Jan 2019 13:24:52 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=UiLjqvhWknXTEAjd/auwTYxNB5iT2kzVZ+BxEPsbB3w=; b=XkQ2oMmLQBulRrWUEJLqxCBMmAbzhT+M3UUGPkYtyMBDIIJEhKfdxK2/ET/AxiHFwT CmEyNPkfOipSDQnoPaHqIOqwNSPidxksoR1MS6BuoVmlBiO9wazAmm42KaYljwLcvRSb mxKm+SRwq1yB6qmzzOCl0GO9r4FCxc8Sum3VJ5XMTel22tRKXInaIj3dCKw2UIXQmvtU YQcUipuhU33G9vp2+n1EwkqlxXmCViNHOUHRHTJvQNtvgYU41cb7PLHED7UyfvW88orN 9HSooDK9hnYTX5t/R3Qd/nCE0v3sVpiG/30HGskHemPfn7HK3F/Jr8vXe1kRkwBmZ6so SNUg== X-Gm-Message-State: AJcUukeFSeZipqm5k0hTB4GQtZ6iJa9XX2XADNCEbu2fPfDSh28MflCv 6UbO6lD8Ep3D9HLOF9uPvwA= X-Google-Smtp-Source: ALg8bN6JnvRQ/s9r7F8Np2jL0Xi5rbLTJUww7EbJvyaP38sYkULwZbH/3hztamjAdzCprLvq772qoQ== X-Received: by 2002:a25:c304:: with SMTP id t4mr23111882ybf.359.1548710691713; Mon, 28 Jan 2019 13:24:51 -0800 (PST) Received: from dennisz-mbp.thefacebook.com ([199.201.65.135]) by smtp.gmail.com with ESMTPSA id a7sm24170415ywb.18.2019.01.28.13.24.50 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Mon, 28 Jan 2019 13:24:50 -0800 (PST) From: Dennis Zhou To: David Sterba , Josef Bacik , Chris Mason , Omar Sandoval , Nick Terrell Cc: kernel-team@fb.com, linux-btrfs@vger.kernel.org, linux-kernel@vger.kernel.org, Dennis Zhou Subject: [PATCH 05/11] btrfs: add helper methods for workspace manager init and cleanup Date: Mon, 28 Jan 2019 16:24:31 -0500 Message-Id: <20190128212437.11597-6-dennis@kernel.org> X-Mailer: git-send-email 2.13.5 In-Reply-To: <20190128212437.11597-1-dennis@kernel.org> References: <20190128212437.11597-1-dennis@kernel.org> Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Workspace manager init and cleanup code is open coded inside a for loop over the compression types. This forces each compression type to rely on the same workspace manager implementation. This patch creates helper methods that will be the generic implementation for btrfs workspace management. Signed-off-by: Dennis Zhou Reviewed-by: Nikolay Borisov Reviewed-by: Josef Bacik --- fs/btrfs/compression.c | 81 ++++++++++++++++++++++-------------------- 1 file changed, 42 insertions(+), 39 deletions(-) diff --git a/fs/btrfs/compression.c b/fs/btrfs/compression.c index b7e986e16640..63fa3eaeeacc 100644 --- a/fs/btrfs/compression.c +++ b/fs/btrfs/compression.c @@ -797,31 +797,41 @@ static const struct btrfs_compress_op * const btrfs_compress_op[] = { &btrfs_zstd_compress, }; -void __init btrfs_init_compress(void) +static void btrfs_init_workspace_manager(int type) { + struct workspace_manager *wsman = &wsm[type]; struct list_head *workspace; - int i; - for (i = 0; i < BTRFS_COMPRESS_TYPES; i++) { - wsm[i].ops = btrfs_compress_op[i]; + wsman->ops = btrfs_compress_op[type]; - INIT_LIST_HEAD(&wsm[i].idle_ws); - spin_lock_init(&wsm[i].ws_lock); - atomic_set(&wsm[i].total_ws, 0); - init_waitqueue_head(&wsm[i].ws_wait); + INIT_LIST_HEAD(&wsman->idle_ws); + spin_lock_init(&wsman->ws_lock); + atomic_set(&wsman->total_ws, 0); + init_waitqueue_head(&wsman->ws_wait); - /* - * Preallocate one workspace for each compression type so - * we can guarantee forward progress in the worst case - */ - workspace = wsm[i].ops->alloc_workspace(); - if (IS_ERR(workspace)) { - pr_warn("BTRFS: cannot preallocate compression workspace, will try later\n"); - } else { - atomic_set(&wsm[i].total_ws, 1); - wsm[i].free_ws = 1; - list_add(workspace, &wsm[i].idle_ws); - } + /* + * Preallocate one workspace for each compression type so + * we can guarantee forward progress in the worst case + */ + workspace = wsman->ops->alloc_workspace(); + if (IS_ERR(workspace)) { + pr_warn("BTRFS: cannot preallocate compression workspace, will try later\n"); + } else { + atomic_set(&wsman->total_ws, 1); + wsman->free_ws = 1; + list_add(workspace, &wsman->idle_ws); + } +} + +static void btrfs_cleanup_workspace_manager(struct workspace_manager *wsman) +{ + struct list_head *ws; + + while (!list_empty(&wsman->idle_ws)) { + ws = wsman->idle_ws.next; + list_del(ws); + wsman->ops->free_workspace(ws); + atomic_dec(&wsman->total_ws); } } @@ -941,24 +951,6 @@ static void free_workspace(int type, struct list_head *workspace) cond_wake_up(ws_wait); } -/* - * cleanup function for module exit - */ -static void free_workspaces(void) -{ - struct list_head *workspace; - int i; - - for (i = 0; i < BTRFS_COMPRESS_TYPES; i++) { - while (!list_empty(&wsm[i].idle_ws)) { - workspace = wsm[i].idle_ws.next; - list_del(workspace); - wsm[i].ops->free_workspace(workspace); - atomic_dec(&wsm[i].total_ws); - } - } -} - /* * Given an address space and start and length, compress the bytes into @pages * that are allocated on demand. @@ -1051,9 +1043,20 @@ int btrfs_decompress(int type, unsigned char *data_in, struct page *dest_page, return ret; } +void __init btrfs_init_compress(void) +{ + int i; + + for (i = 0; i < BTRFS_COMPRESS_TYPES; i++) + btrfs_init_workspace_manager(i); +} + void __cold btrfs_exit_compress(void) { - free_workspaces(); + int i; + + for (i = 0; i < BTRFS_COMPRESS_TYPES; i++) + btrfs_cleanup_workspace_manager(&wsm[i]); } /* From patchwork Mon Jan 28 21:24:32 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dennis Zhou X-Patchwork-Id: 10784823 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 5BEC613B4 for ; Mon, 28 Jan 2019 21:25:40 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 4B58E2B279 for ; Mon, 28 Jan 2019 21:25:40 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 3F8782B288; Mon, 28 Jan 2019 21:25:40 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id AFDD02B279 for ; Mon, 28 Jan 2019 21:25:39 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728515AbfA1VZd (ORCPT ); Mon, 28 Jan 2019 16:25:33 -0500 Received: from mail-yw1-f68.google.com ([209.85.161.68]:46082 "EHLO mail-yw1-f68.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728329AbfA1VYy (ORCPT ); Mon, 28 Jan 2019 16:24:54 -0500 Received: by mail-yw1-f68.google.com with SMTP id t13so7329678ywe.13; Mon, 28 Jan 2019 13:24:53 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=LjkQC3v3bDQeODs64cW3hycjplTBkfWbIT2+EqSQ+uc=; b=tONn/PICYMbK2k5JNfZdvCPZjdaV/bh9prVdNubhwIVwgzUgK8h3JUzC5UURIuoTb/ IGRKZ7EA9HvetkNdcmzGNVCagipReuMC4mBWEw9yJc5bqW4QkFhj7h3fiiGgKK89MUQ+ rX9obWjcrbrMGJ1eqe5p5xwcMhAslo+R7RhpY6IW0Y8WIuUVMmQU6oqixBz8OvvPlusE uwu0c8sm7gnBA1WESb2wrrm6Pfokh+7ImI0Lc1OEHxOuLLAvnLA+qAFeEZpKZqVNkP8b PM1dcRD5uOATRuZnpiuSmLULA66ngkvABvkZPxapiKWmZYbkokLBJv7fidY2eyUiMJMk Cg5A== X-Gm-Message-State: AJcUukdvQmi+mLYDlUtaD92a6c+YodVkSd9TC/BImkUKp7tv/MginoEe ubwTvv4oNqKlXrtqZ71nDFg= X-Google-Smtp-Source: ALg8bN4krUaaFh3/FYbayZmTR5wPbx/Lm0BpztA0iqE4JGNhILMBbLMdeQY1QlfnL0vFXwT+C2wt0g== X-Received: by 2002:a81:b61a:: with SMTP id u26mr22289727ywh.250.1548710693154; Mon, 28 Jan 2019 13:24:53 -0800 (PST) Received: from dennisz-mbp.thefacebook.com ([199.201.65.135]) by smtp.gmail.com with ESMTPSA id a7sm24170415ywb.18.2019.01.28.13.24.51 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Mon, 28 Jan 2019 13:24:52 -0800 (PST) From: Dennis Zhou To: David Sterba , Josef Bacik , Chris Mason , Omar Sandoval , Nick Terrell Cc: kernel-team@fb.com, linux-btrfs@vger.kernel.org, linux-kernel@vger.kernel.org, Dennis Zhou Subject: [PATCH 06/11] btrfs: add compression interface in (get/put)_workspace() Date: Mon, 28 Jan 2019 16:24:32 -0500 Message-Id: <20190128212437.11597-7-dennis@kernel.org> X-Mailer: git-send-email 2.13.5 In-Reply-To: <20190128212437.11597-1-dennis@kernel.org> References: <20190128212437.11597-1-dennis@kernel.org> Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP There are two levels of workspace management. First, alloc()/free() which are responsible for actually creating and destroy workspaces. Second, at a higher level, get()/put() which is the compression code asking for a workspace from a workspace_manager. The compression code shouldn't really care how it gets a workspace, but that it got a workspace. This adds get_workspace() and put_workspace() to be the higher level interface which is responsible for indexing into the appropriate compression type. It also introduces btrfs_put_workspace() and btrfs_get_workspace() to be the generic implementations of the higher interface. Signed-off-by: Dennis Zhou Reviewed-by: Josef Bacik --- fs/btrfs/compression.c | 57 +++++++++++++++++++++++++----------------- 1 file changed, 34 insertions(+), 23 deletions(-) diff --git a/fs/btrfs/compression.c b/fs/btrfs/compression.c index 63fa3eaeeacc..2e748d8785f0 100644 --- a/fs/btrfs/compression.c +++ b/fs/btrfs/compression.c @@ -841,7 +841,7 @@ static void btrfs_cleanup_workspace_manager(struct workspace_manager *wsman) * Preallocation makes a forward progress guarantees and we do not return * errors. */ -static struct list_head *find_workspace(int type) +static struct list_head *btrfs_get_workspace(struct workspace_manager *wsman) { struct list_head *workspace; int cpus = num_online_cpus(); @@ -852,11 +852,11 @@ static struct list_head *find_workspace(int type) wait_queue_head_t *ws_wait; int *free_ws; - idle_ws = &wsm[type].idle_ws; - ws_lock = &wsm[type].ws_lock; - total_ws = &wsm[type].total_ws; - ws_wait = &wsm[type].ws_wait; - free_ws = &wsm[type].free_ws; + idle_ws = &wsman->idle_ws; + ws_lock = &wsman->ws_lock; + total_ws = &wsman->total_ws; + ws_wait = &wsman->ws_wait; + free_ws = &wsman->free_ws; again: spin_lock(ws_lock); @@ -887,7 +887,7 @@ static struct list_head *find_workspace(int type) * context of btrfs_compress_bio/btrfs_compress_pages */ nofs_flag = memalloc_nofs_save(); - workspace = wsm[type].ops->alloc_workspace(); + workspace = wsman->ops->alloc_workspace(); memalloc_nofs_restore(nofs_flag); if (IS_ERR(workspace)) { @@ -918,11 +918,17 @@ static struct list_head *find_workspace(int type) return workspace; } +static struct list_head *get_workspace(int type) +{ + return btrfs_get_workspace(&wsm[type]); +} + /* * put a workspace struct back on the list or free it if we have enough * idle ones sitting around */ -static void free_workspace(int type, struct list_head *workspace) +static void btrfs_put_workspace(struct workspace_manager *wsman, + struct list_head *ws) { struct list_head *idle_ws; spinlock_t *ws_lock; @@ -930,27 +936,32 @@ static void free_workspace(int type, struct list_head *workspace) wait_queue_head_t *ws_wait; int *free_ws; - idle_ws = &wsm[type].idle_ws; - ws_lock = &wsm[type].ws_lock; - total_ws = &wsm[type].total_ws; - ws_wait = &wsm[type].ws_wait; - free_ws = &wsm[type].free_ws; + idle_ws = &wsman->idle_ws; + ws_lock = &wsman->ws_lock; + total_ws = &wsman->total_ws; + ws_wait = &wsman->ws_wait; + free_ws = &wsman->free_ws; spin_lock(ws_lock); if (*free_ws <= num_online_cpus()) { - list_add(workspace, idle_ws); + list_add(ws, idle_ws); (*free_ws)++; spin_unlock(ws_lock); goto wake; } spin_unlock(ws_lock); - wsm[type].ops->free_workspace(workspace); + wsman->ops->free_workspace(ws); atomic_dec(total_ws); wake: cond_wake_up(ws_wait); } +static void put_workspace(int type, struct list_head *ws) +{ + return btrfs_put_workspace(&wsm[type], ws); +} + /* * Given an address space and start and length, compress the bytes into @pages * that are allocated on demand. @@ -984,14 +995,14 @@ int btrfs_compress_pages(unsigned int type_level, struct address_space *mapping, struct list_head *workspace; int ret; - workspace = find_workspace(type); + workspace = get_workspace(type); btrfs_compress_op[type]->set_level(workspace, type_level); ret = btrfs_compress_op[type]->compress_pages(workspace, mapping, start, pages, out_pages, total_in, total_out); - free_workspace(type, workspace); + put_workspace(type, workspace); return ret; } @@ -1015,9 +1026,9 @@ static int btrfs_decompress_bio(struct compressed_bio *cb) int ret; int type = cb->compress_type; - workspace = find_workspace(type); + workspace = get_workspace(type); ret = btrfs_compress_op[type]->decompress_bio(workspace, cb); - free_workspace(type, workspace); + put_workspace(type, workspace); return ret; } @@ -1033,13 +1044,13 @@ int btrfs_decompress(int type, unsigned char *data_in, struct page *dest_page, struct list_head *workspace; int ret; - workspace = find_workspace(type); + workspace = get_workspace(type); ret = btrfs_compress_op[type]->decompress(workspace, data_in, dest_page, start_byte, srclen, destlen); - free_workspace(type, workspace); + put_workspace(type, workspace); return ret; } @@ -1467,7 +1478,7 @@ static void heuristic_collect_sample(struct inode *inode, u64 start, u64 end, */ int btrfs_compress_heuristic(struct inode *inode, u64 start, u64 end) { - struct list_head *ws_list = find_workspace(0); + struct list_head *ws_list = get_workspace(0); struct heuristic_ws *ws; u32 i; u8 byte; @@ -1536,7 +1547,7 @@ int btrfs_compress_heuristic(struct inode *inode, u64 start, u64 end) } out: - free_workspace(0, ws_list); + put_workspace(0, ws_list); return ret; } From patchwork Mon Jan 28 21:24:33 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dennis Zhou X-Patchwork-Id: 10784819 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 3236013B5 for ; Mon, 28 Jan 2019 21:25:31 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 20FC22B279 for ; Mon, 28 Jan 2019 21:25:31 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 1572F2B288; Mon, 28 Jan 2019 21:25:31 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 39B982B279 for ; Mon, 28 Jan 2019 21:25:30 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727408AbfA1VY4 (ORCPT ); Mon, 28 Jan 2019 16:24:56 -0500 Received: from mail-yb1-f195.google.com ([209.85.219.195]:44720 "EHLO mail-yb1-f195.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728368AbfA1VYz (ORCPT ); Mon, 28 Jan 2019 16:24:55 -0500 Received: by mail-yb1-f195.google.com with SMTP id k189so7307212yba.11; Mon, 28 Jan 2019 13:24:55 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=Q0ZvSFCLiR7eTR8xTiZFKMwRMlztmOcs6W4lWC+wmE8=; b=cjVhheisfakN3e9wPIk/t6uubHotDSpqs3XTvOqYpyypK18Ks1BOFZj4Is1WqQaldS BAegOjlvZrsGwsF20YgDVBxCy7lGW/3b+2BDc1OEY4wUMDM9H3KMKlPzh4gMXC3slnF2 lalCWCLHCegQcUIBMlQfxgQJsGypv3BS6Et/5u95pkbU2FGu1ByKfAj412O9ENwSKZkm 1ksNjRc2X4qLRVv5e/G2SM5g67ZnicADAG/C3QfOh04W2GgxQxkirqv5uy5pzncq74Q5 6gDFCNGi9k2ymG/MJItm4Iw1YYGEkSlsLv4cIKgX47K5YXJ10J6/PD5N6gOfBXY+xly2 Kp9w== X-Gm-Message-State: AHQUAubgFJn+/q6hcaKGTzl9rbW95nYGBKcbqgHOti1PSxzuk/2vIHDp nKol17pSYKs54LTxbBrAk+o= X-Google-Smtp-Source: AHgI3Ia36WbIIpamVRjNH4SkkElnXA8w6+SF1a/chvpdveen5rElxe2KQp3/mXPDdUoRY/eSluY9Yw== X-Received: by 2002:a25:6742:: with SMTP id b63mr9724713ybc.115.1548710694505; Mon, 28 Jan 2019 13:24:54 -0800 (PST) Received: from dennisz-mbp.thefacebook.com ([199.201.65.135]) by smtp.gmail.com with ESMTPSA id a7sm24170415ywb.18.2019.01.28.13.24.53 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Mon, 28 Jan 2019 13:24:53 -0800 (PST) From: Dennis Zhou To: David Sterba , Josef Bacik , Chris Mason , Omar Sandoval , Nick Terrell Cc: kernel-team@fb.com, linux-btrfs@vger.kernel.org, linux-kernel@vger.kernel.org, Dennis Zhou Subject: [PATCH 07/11] btrfs: move to fn pointers for get/put workspaces Date: Mon, 28 Jan 2019 16:24:33 -0500 Message-Id: <20190128212437.11597-8-dennis@kernel.org> X-Mailer: git-send-email 2.13.5 In-Reply-To: <20190128212437.11597-1-dennis@kernel.org> References: <20190128212437.11597-1-dennis@kernel.org> Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP The previous patch added generic helpers for get_workspace() and put_workspace(). Now, we can migrate ownership of the workspace_manager to be in the compression type code as the compression code itself doesn't care beyond being able to get a workspace. The init/cleanup and get/put methods are abstracted so each compression algorithm can decide how they want to manage their workspaces. Signed-off-by: Dennis Zhou Reviewed-by: Josef Bacik --- fs/btrfs/compression.c | 101 +++++++++++++++++++++++------------------ fs/btrfs/compression.h | 26 +++++++++++ fs/btrfs/lzo.c | 26 +++++++++++ fs/btrfs/zlib.c | 26 +++++++++++ fs/btrfs/zstd.c | 26 +++++++++++ 5 files changed, 160 insertions(+), 45 deletions(-) diff --git a/fs/btrfs/compression.c b/fs/btrfs/compression.c index 2e748d8785f0..ab694760ffdb 100644 --- a/fs/btrfs/compression.c +++ b/fs/btrfs/compression.c @@ -732,6 +732,28 @@ struct heuristic_ws { struct list_head list; }; +static struct workspace_manager heuristic_wsm; + +static void heuristic_init_workspace_manager(void) +{ + btrfs_init_workspace_manager(&heuristic_wsm, &btrfs_heuristic_compress); +} + +static void heuristic_cleanup_workspace_manager(void) +{ + btrfs_cleanup_workspace_manager(&heuristic_wsm); +} + +static struct list_head *heuristic_get_workspace(void) +{ + return btrfs_get_workspace(&heuristic_wsm); +} + +static void heuristic_put_workspace(struct list_head *ws) +{ + btrfs_put_workspace(&heuristic_wsm, ws); +} + static void free_heuristic_ws(struct list_head *ws) { struct heuristic_ws *workspace; @@ -772,24 +794,14 @@ static struct list_head *alloc_heuristic_ws(void) } const struct btrfs_compress_op btrfs_heuristic_compress = { + .init_workspace_manager = heuristic_init_workspace_manager, + .cleanup_workspace_manager = heuristic_cleanup_workspace_manager, + .get_workspace = heuristic_get_workspace, + .put_workspace = heuristic_put_workspace, .alloc_workspace = alloc_heuristic_ws, .free_workspace = free_heuristic_ws, }; -struct workspace_manager { - const struct btrfs_compress_op *ops; - struct list_head idle_ws; - spinlock_t ws_lock; - /* Number of free workspaces */ - int free_ws; - /* Total number of allocated workspaces */ - atomic_t total_ws; - /* Waiters for a free workspace */ - wait_queue_head_t ws_wait; -}; - -static struct workspace_manager wsm[BTRFS_COMPRESS_TYPES]; - static const struct btrfs_compress_op * const btrfs_compress_op[] = { &btrfs_heuristic_compress, &btrfs_zlib_compress, @@ -797,33 +809,33 @@ static const struct btrfs_compress_op * const btrfs_compress_op[] = { &btrfs_zstd_compress, }; -static void btrfs_init_workspace_manager(int type) +void btrfs_init_workspace_manager(struct workspace_manager *wsm, + const struct btrfs_compress_op *ops) { - struct workspace_manager *wsman = &wsm[type]; struct list_head *workspace; - wsman->ops = btrfs_compress_op[type]; + wsm->ops = ops; - INIT_LIST_HEAD(&wsman->idle_ws); - spin_lock_init(&wsman->ws_lock); - atomic_set(&wsman->total_ws, 0); - init_waitqueue_head(&wsman->ws_wait); + INIT_LIST_HEAD(&wsm->idle_ws); + spin_lock_init(&wsm->ws_lock); + atomic_set(&wsm->total_ws, 0); + init_waitqueue_head(&wsm->ws_wait); /* * Preallocate one workspace for each compression type so * we can guarantee forward progress in the worst case */ - workspace = wsman->ops->alloc_workspace(); + workspace = wsm->ops->alloc_workspace(); if (IS_ERR(workspace)) { pr_warn("BTRFS: cannot preallocate compression workspace, will try later\n"); } else { - atomic_set(&wsman->total_ws, 1); - wsman->free_ws = 1; - list_add(workspace, &wsman->idle_ws); + atomic_set(&wsm->total_ws, 1); + wsm->free_ws = 1; + list_add(workspace, &wsm->idle_ws); } } -static void btrfs_cleanup_workspace_manager(struct workspace_manager *wsman) +void btrfs_cleanup_workspace_manager(struct workspace_manager *wsman) { struct list_head *ws; @@ -841,7 +853,7 @@ static void btrfs_cleanup_workspace_manager(struct workspace_manager *wsman) * Preallocation makes a forward progress guarantees and we do not return * errors. */ -static struct list_head *btrfs_get_workspace(struct workspace_manager *wsman) +struct list_head *btrfs_get_workspace(struct workspace_manager *wsm) { struct list_head *workspace; int cpus = num_online_cpus(); @@ -852,11 +864,11 @@ static struct list_head *btrfs_get_workspace(struct workspace_manager *wsman) wait_queue_head_t *ws_wait; int *free_ws; - idle_ws = &wsman->idle_ws; - ws_lock = &wsman->ws_lock; - total_ws = &wsman->total_ws; - ws_wait = &wsman->ws_wait; - free_ws = &wsman->free_ws; + idle_ws = &wsm->idle_ws; + ws_lock = &wsm->ws_lock; + total_ws = &wsm->total_ws; + ws_wait = &wsm->ws_wait; + free_ws = &wsm->free_ws; again: spin_lock(ws_lock); @@ -887,7 +899,7 @@ static struct list_head *btrfs_get_workspace(struct workspace_manager *wsman) * context of btrfs_compress_bio/btrfs_compress_pages */ nofs_flag = memalloc_nofs_save(); - workspace = wsman->ops->alloc_workspace(); + workspace = wsm->ops->alloc_workspace(); memalloc_nofs_restore(nofs_flag); if (IS_ERR(workspace)) { @@ -920,15 +932,14 @@ static struct list_head *btrfs_get_workspace(struct workspace_manager *wsman) static struct list_head *get_workspace(int type) { - return btrfs_get_workspace(&wsm[type]); + return btrfs_compress_op[type]->get_workspace(); } /* * put a workspace struct back on the list or free it if we have enough * idle ones sitting around */ -static void btrfs_put_workspace(struct workspace_manager *wsman, - struct list_head *ws) +void btrfs_put_workspace(struct workspace_manager *wsm, struct list_head *ws) { struct list_head *idle_ws; spinlock_t *ws_lock; @@ -936,11 +947,11 @@ static void btrfs_put_workspace(struct workspace_manager *wsman, wait_queue_head_t *ws_wait; int *free_ws; - idle_ws = &wsman->idle_ws; - ws_lock = &wsman->ws_lock; - total_ws = &wsman->total_ws; - ws_wait = &wsman->ws_wait; - free_ws = &wsman->free_ws; + idle_ws = &wsm->idle_ws; + ws_lock = &wsm->ws_lock; + total_ws = &wsm->total_ws; + ws_wait = &wsm->ws_wait; + free_ws = &wsm->free_ws; spin_lock(ws_lock); if (*free_ws <= num_online_cpus()) { @@ -951,7 +962,7 @@ static void btrfs_put_workspace(struct workspace_manager *wsman, } spin_unlock(ws_lock); - wsman->ops->free_workspace(ws); + wsm->ops->free_workspace(ws); atomic_dec(total_ws); wake: cond_wake_up(ws_wait); @@ -959,7 +970,7 @@ static void btrfs_put_workspace(struct workspace_manager *wsman, static void put_workspace(int type, struct list_head *ws) { - return btrfs_put_workspace(&wsm[type], ws); + return btrfs_compress_op[type]->put_workspace(ws); } /* @@ -1059,7 +1070,7 @@ void __init btrfs_init_compress(void) int i; for (i = 0; i < BTRFS_COMPRESS_TYPES; i++) - btrfs_init_workspace_manager(i); + btrfs_compress_op[i]->init_workspace_manager(); } void __cold btrfs_exit_compress(void) @@ -1067,7 +1078,7 @@ void __cold btrfs_exit_compress(void) int i; for (i = 0; i < BTRFS_COMPRESS_TYPES; i++) - btrfs_cleanup_workspace_manager(&wsm[i]); + btrfs_compress_op[i]->cleanup_workspace_manager(); } /* diff --git a/fs/btrfs/compression.h b/fs/btrfs/compression.h index 53a8b9e93217..05342ad081d6 100644 --- a/fs/btrfs/compression.h +++ b/fs/btrfs/compression.h @@ -100,7 +100,33 @@ enum btrfs_compression_type { BTRFS_COMPRESS_TYPES = 4, }; +struct workspace_manager { + const struct btrfs_compress_op *ops; + struct list_head idle_ws; + spinlock_t ws_lock; + /* Number of free workspaces */ + int free_ws; + /* Total number of allocated workspaces */ + atomic_t total_ws; + /* Waiters for a free workspace */ + wait_queue_head_t ws_wait; +}; + +void btrfs_init_workspace_manager(struct workspace_manager *wsm, + const struct btrfs_compress_op *ops); +struct list_head *btrfs_get_workspace(struct workspace_manager *wsm); +void btrfs_put_workspace(struct workspace_manager *wsm, struct list_head *ws); +void btrfs_cleanup_workspace_manager(struct workspace_manager *wsm); + struct btrfs_compress_op { + void (*init_workspace_manager)(void); + + void (*cleanup_workspace_manager)(void); + + struct list_head *(*get_workspace)(void); + + void (*put_workspace)(struct list_head *ws); + struct list_head *(*alloc_workspace)(void); void (*free_workspace)(struct list_head *workspace); diff --git a/fs/btrfs/lzo.c b/fs/btrfs/lzo.c index 90639140439f..f0837b2c8e94 100644 --- a/fs/btrfs/lzo.c +++ b/fs/btrfs/lzo.c @@ -61,6 +61,28 @@ struct workspace { struct list_head list; }; +static struct workspace_manager wsm; + +static void lzo_init_workspace_manager(void) +{ + btrfs_init_workspace_manager(&wsm, &btrfs_lzo_compress); +} + +static void lzo_cleanup_workspace_manager(void) +{ + btrfs_cleanup_workspace_manager(&wsm); +} + +static struct list_head *lzo_get_workspace(void) +{ + return btrfs_get_workspace(&wsm); +} + +static void lzo_put_workspace(struct list_head *ws) +{ + btrfs_put_workspace(&wsm, ws); +} + static void lzo_free_workspace(struct list_head *ws) { struct workspace *workspace = list_entry(ws, struct workspace, list); @@ -490,6 +512,10 @@ static void lzo_set_level(struct list_head *ws, unsigned int type) } const struct btrfs_compress_op btrfs_lzo_compress = { + .init_workspace_manager = lzo_init_workspace_manager, + .cleanup_workspace_manager = lzo_cleanup_workspace_manager, + .get_workspace = lzo_get_workspace, + .put_workspace = lzo_put_workspace, .alloc_workspace = lzo_alloc_workspace, .free_workspace = lzo_free_workspace, .compress_pages = lzo_compress_pages, diff --git a/fs/btrfs/zlib.c b/fs/btrfs/zlib.c index 1480b3eee306..04687bf692e3 100644 --- a/fs/btrfs/zlib.c +++ b/fs/btrfs/zlib.c @@ -27,6 +27,28 @@ struct workspace { int level; }; +static struct workspace_manager wsm; + +static void zlib_init_workspace_manager(void) +{ + btrfs_init_workspace_manager(&wsm, &btrfs_zlib_compress); +} + +static void zlib_cleanup_workspace_manager(void) +{ + btrfs_cleanup_workspace_manager(&wsm); +} + +static struct list_head *zlib_get_workspace(void) +{ + return btrfs_get_workspace(&wsm); +} + +static void zlib_put_workspace(struct list_head *ws) +{ + btrfs_put_workspace(&wsm, ws); +} + static void zlib_free_workspace(struct list_head *ws) { struct workspace *workspace = list_entry(ws, struct workspace, list); @@ -402,6 +424,10 @@ static void zlib_set_level(struct list_head *ws, unsigned int type) } const struct btrfs_compress_op btrfs_zlib_compress = { + .init_workspace_manager = zlib_init_workspace_manager, + .cleanup_workspace_manager = zlib_cleanup_workspace_manager, + .get_workspace = zlib_get_workspace, + .put_workspace = zlib_put_workspace, .alloc_workspace = zlib_alloc_workspace, .free_workspace = zlib_free_workspace, .compress_pages = zlib_compress_pages, diff --git a/fs/btrfs/zstd.c b/fs/btrfs/zstd.c index af6ec59972f5..b06eaf171be7 100644 --- a/fs/btrfs/zstd.c +++ b/fs/btrfs/zstd.c @@ -41,6 +41,28 @@ struct workspace { ZSTD_outBuffer out_buf; }; +static struct workspace_manager wsm; + +static void zstd_init_workspace_manager(void) +{ + btrfs_init_workspace_manager(&wsm, &btrfs_zstd_compress); +} + +static void zstd_cleanup_workspace_manager(void) +{ + btrfs_cleanup_workspace_manager(&wsm); +} + +static struct list_head *zstd_get_workspace(void) +{ + return btrfs_get_workspace(&wsm); +} + +static void zstd_put_workspace(struct list_head *ws) +{ + btrfs_put_workspace(&wsm, ws); +} + static void zstd_free_workspace(struct list_head *ws) { struct workspace *workspace = list_entry(ws, struct workspace, list); @@ -424,6 +446,10 @@ static void zstd_set_level(struct list_head *ws, unsigned int type) } const struct btrfs_compress_op btrfs_zstd_compress = { + .init_workspace_manager = zstd_init_workspace_manager, + .cleanup_workspace_manager = zstd_cleanup_workspace_manager, + .get_workspace = zstd_get_workspace, + .put_workspace = zstd_put_workspace, .alloc_workspace = zstd_alloc_workspace, .free_workspace = zstd_free_workspace, .compress_pages = zstd_compress_pages, From patchwork Mon Jan 28 21:24:34 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dennis Zhou X-Patchwork-Id: 10784817 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id B38A613B5 for ; Mon, 28 Jan 2019 21:25:28 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id A15D52B279 for ; Mon, 28 Jan 2019 21:25:28 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 953982B288; Mon, 28 Jan 2019 21:25:28 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id D80C02B279 for ; Mon, 28 Jan 2019 21:25:27 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728486AbfA1VZX (ORCPT ); Mon, 28 Jan 2019 16:25:23 -0500 Received: from mail-yw1-f67.google.com ([209.85.161.67]:37090 "EHLO mail-yw1-f67.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728316AbfA1VY4 (ORCPT ); Mon, 28 Jan 2019 16:24:56 -0500 Received: by mail-yw1-f67.google.com with SMTP id h193so7358275ywc.4; Mon, 28 Jan 2019 13:24:56 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=OpMPEOdbVcWt4t/2TFv/NEECu9N5O2HbNXitx/LNP7k=; b=YDiERm27/Vo14tpG3VYqNdE0FXV3tw0SEehD2b0hzu6ExoNgbgXFcUx8v+3KSaVPJC eUb79wZ7TKOBrHbNj2ZQnjbZD2VojuKIUBLfwjI3ZVRwetRz5hq5mLoxjV8q6/qC+LA9 L2PjFM1p86zz/SYS8PvYzDiMTFuHfNUgu7DNRfXgWHAi3GngZd7thhiGk32+8trRZ2YS f/ZruV4pyYVA7/IafCs2uvSPz4gwuIuuPX2owLx/AGKtaa3B3GD5eJQzQ/Ueute5gB1R DwuZUoHyJ63AG+M8HEYwtLW9Q98OMqgSjfl0vev0Bt3PBETW7ttf3nthuHRd48b0cIND j4IQ== X-Gm-Message-State: AJcUukfm56ZhBFqLriSOasCCNbmRuoMivuDj2ZSYkrrN7UUcX5Xz8UZv qNAqSW5wI4f5e4S15HQ+GHU= X-Google-Smtp-Source: ALg8bN4TlsfktrxYLCaEyhNx98IE9Q0yEcHlTEAGh9XyZj0y1KqBrakMwg1rKEmOTKXc3ZUIW/WZeQ== X-Received: by 2002:a81:378b:: with SMTP id e133mr22220086ywa.79.1548710695992; Mon, 28 Jan 2019 13:24:55 -0800 (PST) Received: from dennisz-mbp.thefacebook.com ([199.201.65.135]) by smtp.gmail.com with ESMTPSA id a7sm24170415ywb.18.2019.01.28.13.24.54 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Mon, 28 Jan 2019 13:24:55 -0800 (PST) From: Dennis Zhou To: David Sterba , Josef Bacik , Chris Mason , Omar Sandoval , Nick Terrell Cc: kernel-team@fb.com, linux-btrfs@vger.kernel.org, linux-kernel@vger.kernel.org, Dennis Zhou Subject: [PATCH 08/11] btrfs: plumb level through the compression interface Date: Mon, 28 Jan 2019 16:24:34 -0500 Message-Id: <20190128212437.11597-9-dennis@kernel.org> X-Mailer: git-send-email 2.13.5 In-Reply-To: <20190128212437.11597-1-dennis@kernel.org> References: <20190128212437.11597-1-dennis@kernel.org> Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Zlib compression supports multiple levels, but doesn't require changing in how a workspace itself is created and managed. Zstd introduces a different memory requirement such that higher levels of compression require more memory. This requires changes in how the alloc()/get() methods work for zstd. This pach plumbs compression level through the interface as a parameter in preparation for zstd compression levels. This gives the compression types opportunity to create/manage based on the compression level. Signed-off-by: Dennis Zhou Reviewed-by: Nikolay Borisov Reviewed-by: Josef Bacik --- fs/btrfs/compression.c | 31 ++++++++++++++++--------------- fs/btrfs/compression.h | 7 ++++--- fs/btrfs/lzo.c | 6 +++--- fs/btrfs/zlib.c | 7 ++++--- fs/btrfs/zstd.c | 6 +++--- 5 files changed, 30 insertions(+), 27 deletions(-) diff --git a/fs/btrfs/compression.c b/fs/btrfs/compression.c index ab694760ffdb..e509071eaa69 100644 --- a/fs/btrfs/compression.c +++ b/fs/btrfs/compression.c @@ -744,9 +744,9 @@ static void heuristic_cleanup_workspace_manager(void) btrfs_cleanup_workspace_manager(&heuristic_wsm); } -static struct list_head *heuristic_get_workspace(void) +static struct list_head *heuristic_get_workspace(unsigned int level) { - return btrfs_get_workspace(&heuristic_wsm); + return btrfs_get_workspace(&heuristic_wsm, level); } static void heuristic_put_workspace(struct list_head *ws) @@ -766,7 +766,7 @@ static void free_heuristic_ws(struct list_head *ws) kfree(workspace); } -static struct list_head *alloc_heuristic_ws(void) +static struct list_head *alloc_heuristic_ws(unsigned int level) { struct heuristic_ws *ws; @@ -825,7 +825,7 @@ void btrfs_init_workspace_manager(struct workspace_manager *wsm, * Preallocate one workspace for each compression type so * we can guarantee forward progress in the worst case */ - workspace = wsm->ops->alloc_workspace(); + workspace = wsm->ops->alloc_workspace(0); if (IS_ERR(workspace)) { pr_warn("BTRFS: cannot preallocate compression workspace, will try later\n"); } else { @@ -853,7 +853,8 @@ void btrfs_cleanup_workspace_manager(struct workspace_manager *wsman) * Preallocation makes a forward progress guarantees and we do not return * errors. */ -struct list_head *btrfs_get_workspace(struct workspace_manager *wsm) +struct list_head *btrfs_get_workspace(struct workspace_manager *wsm, + unsigned int level) { struct list_head *workspace; int cpus = num_online_cpus(); @@ -899,7 +900,7 @@ struct list_head *btrfs_get_workspace(struct workspace_manager *wsm) * context of btrfs_compress_bio/btrfs_compress_pages */ nofs_flag = memalloc_nofs_save(); - workspace = wsm->ops->alloc_workspace(); + workspace = wsm->ops->alloc_workspace(level); memalloc_nofs_restore(nofs_flag); if (IS_ERR(workspace)) { @@ -930,9 +931,9 @@ struct list_head *btrfs_get_workspace(struct workspace_manager *wsm) return workspace; } -static struct list_head *get_workspace(int type) +static struct list_head *get_workspace(int type, int level) { - return btrfs_compress_op[type]->get_workspace(); + return btrfs_compress_op[type]->get_workspace(level); } /* @@ -1003,12 +1004,13 @@ int btrfs_compress_pages(unsigned int type_level, struct address_space *mapping, unsigned long *total_out) { int type = BTRFS_COMPRESS_TYPE(type_level); + int level = BTRFS_COMPRESS_LEVEL(type_level); struct list_head *workspace; int ret; - workspace = get_workspace(type); + workspace = get_workspace(type, level); - btrfs_compress_op[type]->set_level(workspace, type_level); + btrfs_compress_op[type]->set_level(workspace, level); ret = btrfs_compress_op[type]->compress_pages(workspace, mapping, start, pages, out_pages, @@ -1037,7 +1039,7 @@ static int btrfs_decompress_bio(struct compressed_bio *cb) int ret; int type = cb->compress_type; - workspace = get_workspace(type); + workspace = get_workspace(type, 0); ret = btrfs_compress_op[type]->decompress_bio(workspace, cb); put_workspace(type, workspace); @@ -1055,13 +1057,12 @@ int btrfs_decompress(int type, unsigned char *data_in, struct page *dest_page, struct list_head *workspace; int ret; - workspace = get_workspace(type); - + workspace = get_workspace(type, 0); ret = btrfs_compress_op[type]->decompress(workspace, data_in, dest_page, start_byte, srclen, destlen); - put_workspace(type, workspace); + return ret; } @@ -1489,7 +1490,7 @@ static void heuristic_collect_sample(struct inode *inode, u64 start, u64 end, */ int btrfs_compress_heuristic(struct inode *inode, u64 start, u64 end) { - struct list_head *ws_list = get_workspace(0); + struct list_head *ws_list = get_workspace(0, 0); struct heuristic_ws *ws; u32 i; u8 byte; diff --git a/fs/btrfs/compression.h b/fs/btrfs/compression.h index 05342ad081d6..e3627139bc5c 100644 --- a/fs/btrfs/compression.h +++ b/fs/btrfs/compression.h @@ -114,7 +114,8 @@ struct workspace_manager { void btrfs_init_workspace_manager(struct workspace_manager *wsm, const struct btrfs_compress_op *ops); -struct list_head *btrfs_get_workspace(struct workspace_manager *wsm); +struct list_head *btrfs_get_workspace(struct workspace_manager *wsm, + unsigned int level); void btrfs_put_workspace(struct workspace_manager *wsm, struct list_head *ws); void btrfs_cleanup_workspace_manager(struct workspace_manager *wsm); @@ -123,11 +124,11 @@ struct btrfs_compress_op { void (*cleanup_workspace_manager)(void); - struct list_head *(*get_workspace)(void); + struct list_head *(*get_workspace)(unsigned int level); void (*put_workspace)(struct list_head *ws); - struct list_head *(*alloc_workspace)(void); + struct list_head *(*alloc_workspace)(unsigned int level); void (*free_workspace)(struct list_head *workspace); diff --git a/fs/btrfs/lzo.c b/fs/btrfs/lzo.c index f0837b2c8e94..f132af45a924 100644 --- a/fs/btrfs/lzo.c +++ b/fs/btrfs/lzo.c @@ -73,9 +73,9 @@ static void lzo_cleanup_workspace_manager(void) btrfs_cleanup_workspace_manager(&wsm); } -static struct list_head *lzo_get_workspace(void) +static struct list_head *lzo_get_workspace(unsigned int level) { - return btrfs_get_workspace(&wsm); + return btrfs_get_workspace(&wsm, level); } static void lzo_put_workspace(struct list_head *ws) @@ -93,7 +93,7 @@ static void lzo_free_workspace(struct list_head *ws) kfree(workspace); } -static struct list_head *lzo_alloc_workspace(void) +static struct list_head *lzo_alloc_workspace(unsigned int level) { struct workspace *workspace; diff --git a/fs/btrfs/zlib.c b/fs/btrfs/zlib.c index 04687bf692e3..e2173d0c4fd3 100644 --- a/fs/btrfs/zlib.c +++ b/fs/btrfs/zlib.c @@ -39,9 +39,9 @@ static void zlib_cleanup_workspace_manager(void) btrfs_cleanup_workspace_manager(&wsm); } -static struct list_head *zlib_get_workspace(void) +static struct list_head *zlib_get_workspace(unsigned int level) { - return btrfs_get_workspace(&wsm); + return btrfs_get_workspace(&wsm, level); } static void zlib_put_workspace(struct list_head *ws) @@ -58,7 +58,7 @@ static void zlib_free_workspace(struct list_head *ws) kfree(workspace); } -static struct list_head *zlib_alloc_workspace(void) +static struct list_head *zlib_alloc_workspace(unsigned int level) { struct workspace *workspace; int workspacesize; @@ -71,6 +71,7 @@ static struct list_head *zlib_alloc_workspace(void) zlib_inflate_workspacesize()); workspace->strm.workspace = kvmalloc(workspacesize, GFP_KERNEL); workspace->buf = kmalloc(PAGE_SIZE, GFP_KERNEL); + workspace->level = level; if (!workspace->strm.workspace || !workspace->buf) goto fail; diff --git a/fs/btrfs/zstd.c b/fs/btrfs/zstd.c index b06eaf171be7..404101864220 100644 --- a/fs/btrfs/zstd.c +++ b/fs/btrfs/zstd.c @@ -53,9 +53,9 @@ static void zstd_cleanup_workspace_manager(void) btrfs_cleanup_workspace_manager(&wsm); } -static struct list_head *zstd_get_workspace(void) +static struct list_head *zstd_get_workspace(unsigned int level) { - return btrfs_get_workspace(&wsm); + return btrfs_get_workspace(&wsm, level); } static void zstd_put_workspace(struct list_head *ws) @@ -72,7 +72,7 @@ static void zstd_free_workspace(struct list_head *ws) kfree(workspace); } -static struct list_head *zstd_alloc_workspace(void) +static struct list_head *zstd_alloc_workspace(unsigned int level) { ZSTD_parameters params = zstd_get_btrfs_parameters(ZSTD_BTRFS_MAX_INPUT); From patchwork Mon Jan 28 21:24:35 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dennis Zhou X-Patchwork-Id: 10784815 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id EBD0513B4 for ; Mon, 28 Jan 2019 21:25:19 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id DAC462B279 for ; Mon, 28 Jan 2019 21:25:19 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id CF57D2B288; Mon, 28 Jan 2019 21:25:19 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 47B212B27E for ; Mon, 28 Jan 2019 21:25:19 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728368AbfA1VY7 (ORCPT ); Mon, 28 Jan 2019 16:24:59 -0500 Received: from mail-yw1-f66.google.com ([209.85.161.66]:36697 "EHLO mail-yw1-f66.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728394AbfA1VY6 (ORCPT ); Mon, 28 Jan 2019 16:24:58 -0500 Received: by mail-yw1-f66.google.com with SMTP id i73so7356243ywg.3; Mon, 28 Jan 2019 13:24:57 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=wisqPDSfzfuGtfQPMvdQY71lpH/gg72mzhu83259kzY=; b=jZK6tNFIYRpY+yIcfDkagJ3sEtejj63ieLD6SSHnkljUruyYwBWr70Ss6/eYXLKGxi 258QGiyZCIfIpOspjxWnpeqB35A91/1zHYSfckKam3yRxmWyHmQyOk2DCDxy1oPtX8PZ MOFgXd0vBcixyM6/IybK3cTtYathShnG8Ek6JXPPCeY0jOjXvrB+25GUUd6rpOF9dIzw 3RXCqaTc/WVfAe/wVdnqI7idQhZ4QdSoXxYxNvur5yUoX+im8l9igZeDgDQ02Pm+ZWUZ Frs3yipJHmTIB/iClVQ1tkoehLATNwL4S3p9Ejt7Siv7hiztNfK2UgTZoaxx+4JuRan2 IO/g== X-Gm-Message-State: AJcUukd9MVyk0S4KlqUYBNbcqWtwLqN4xgNGnSxuJjAikyJm75dM0r50 d55s3f3TBiaSz+bgi9/0KbI= X-Google-Smtp-Source: ALg8bN5UMRGWhuyjNZXutlYtAafP3bI86CSqqidL++DGdcxPR12bJhZHQ2o08FAbnjiOU8UrKsUueg== X-Received: by 2002:a81:c93:: with SMTP id 141mr22834674ywm.356.1548710697375; Mon, 28 Jan 2019 13:24:57 -0800 (PST) Received: from dennisz-mbp.thefacebook.com ([199.201.65.135]) by smtp.gmail.com with ESMTPSA id a7sm24170415ywb.18.2019.01.28.13.24.56 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Mon, 28 Jan 2019 13:24:56 -0800 (PST) From: Dennis Zhou To: David Sterba , Josef Bacik , Chris Mason , Omar Sandoval , Nick Terrell Cc: kernel-team@fb.com, linux-btrfs@vger.kernel.org, linux-kernel@vger.kernel.org, Dennis Zhou Subject: [PATCH 09/11] btrfs: change set_level() to bound the level passed in Date: Mon, 28 Jan 2019 16:24:35 -0500 Message-Id: <20190128212437.11597-10-dennis@kernel.org> X-Mailer: git-send-email 2.13.5 In-Reply-To: <20190128212437.11597-1-dennis@kernel.org> References: <20190128212437.11597-1-dennis@kernel.org> Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Currently, the only user of set_level() is zlib which sets an internal workspace parameter. As level is now plumbed into get_workspace(), this can be handled there rather than separately. This repurposes set_level() to bound the level passed in so it can be used when setting the mounts compression level and as well as verifying the level before getting a workspace. The other benefit is this divides the meaning of compress(0) and get_workspace(0). The former means we want to use the default compression level of the compression type. The latter means we can use any workspace available. Signed-off-by: Dennis Zhou --- fs/btrfs/compression.c | 23 +++++++++++++++-------- fs/btrfs/compression.h | 4 ++-- fs/btrfs/lzo.c | 3 ++- fs/btrfs/super.c | 4 +++- fs/btrfs/zlib.c | 18 +++++++++++------- fs/btrfs/zstd.c | 3 ++- 6 files changed, 35 insertions(+), 20 deletions(-) diff --git a/fs/btrfs/compression.c b/fs/btrfs/compression.c index e509071eaa69..a552c6f61e6d 100644 --- a/fs/btrfs/compression.c +++ b/fs/btrfs/compression.c @@ -1008,9 +1008,9 @@ int btrfs_compress_pages(unsigned int type_level, struct address_space *mapping, struct list_head *workspace; int ret; - workspace = get_workspace(type, level); + level = btrfs_compress_op[type]->set_level(level); - btrfs_compress_op[type]->set_level(workspace, level); + workspace = get_workspace(type, level); ret = btrfs_compress_op[type]->compress_pages(workspace, mapping, start, pages, out_pages, @@ -1563,14 +1563,21 @@ int btrfs_compress_heuristic(struct inode *inode, u64 start, u64 end) return ret; } -unsigned int btrfs_compress_str2level(const char *str) +unsigned int btrfs_compress_str2level(unsigned int type, const char *str) { - if (strncmp(str, "zlib", 4) != 0) + unsigned int level; + int ret; + + if (!type) return 0; - /* Accepted form: zlib:1 up to zlib:9 and nothing left after the number */ - if (str[4] == ':' && '1' <= str[5] && str[5] <= '9' && str[6] == 0) - return str[5] - '0'; + if (str[0] == ':') { + ret = kstrtouint(str + 1, 10, &level); + if (ret) + level = 0; + } + + level = btrfs_compress_op[type]->set_level(level); - return BTRFS_ZLIB_DEFAULT_LEVEL; + return level; } diff --git a/fs/btrfs/compression.h b/fs/btrfs/compression.h index e3627139bc5c..d607be40aa0e 100644 --- a/fs/btrfs/compression.h +++ b/fs/btrfs/compression.h @@ -90,7 +90,7 @@ blk_status_t btrfs_submit_compressed_write(struct inode *inode, u64 start, blk_status_t btrfs_submit_compressed_read(struct inode *inode, struct bio *bio, int mirror_num, unsigned long bio_flags); -unsigned btrfs_compress_str2level(const char *str); +unsigned int btrfs_compress_str2level(unsigned int type, const char *str); enum btrfs_compression_type { BTRFS_COMPRESS_NONE = 0, @@ -149,7 +149,7 @@ struct btrfs_compress_op { unsigned long start_byte, size_t srclen, size_t destlen); - void (*set_level)(struct list_head *ws, unsigned int type); + unsigned int (*set_level)(unsigned int level); }; extern const struct btrfs_compress_op btrfs_heuristic_compress; diff --git a/fs/btrfs/lzo.c b/fs/btrfs/lzo.c index f132af45a924..579d53ae256f 100644 --- a/fs/btrfs/lzo.c +++ b/fs/btrfs/lzo.c @@ -507,8 +507,9 @@ static int lzo_decompress(struct list_head *ws, unsigned char *data_in, return ret; } -static void lzo_set_level(struct list_head *ws, unsigned int type) +static unsigned int lzo_set_level(unsigned int level) { + return 0; } const struct btrfs_compress_op btrfs_lzo_compress = { diff --git a/fs/btrfs/super.c b/fs/btrfs/super.c index c5586ffd1426..b28dff207383 100644 --- a/fs/btrfs/super.c +++ b/fs/btrfs/super.c @@ -529,7 +529,9 @@ int btrfs_parse_options(struct btrfs_fs_info *info, char *options, if (token != Opt_compress && token != Opt_compress_force) info->compress_level = - btrfs_compress_str2level(args[0].from); + btrfs_compress_str2level( + BTRFS_COMPRESS_ZLIB, + args[0].from + 4); btrfs_set_opt(info->mount_opt, COMPRESS); btrfs_clear_opt(info->mount_opt, NODATACOW); btrfs_clear_opt(info->mount_opt, NODATASUM); diff --git a/fs/btrfs/zlib.c b/fs/btrfs/zlib.c index e2173d0c4fd3..388b1f000fca 100644 --- a/fs/btrfs/zlib.c +++ b/fs/btrfs/zlib.c @@ -41,7 +41,12 @@ static void zlib_cleanup_workspace_manager(void) static struct list_head *zlib_get_workspace(unsigned int level) { - return btrfs_get_workspace(&wsm, level); + struct list_head *ws = btrfs_get_workspace(&wsm, level); + struct workspace *workspace = list_entry(ws, struct workspace, list); + + workspace->level = level; + + return ws; } static void zlib_put_workspace(struct list_head *ws) @@ -413,15 +418,14 @@ static int zlib_decompress(struct list_head *ws, unsigned char *data_in, return ret; } -static void zlib_set_level(struct list_head *ws, unsigned int type) +static unsigned int zlib_set_level(unsigned int level) { - struct workspace *workspace = list_entry(ws, struct workspace, list); - unsigned int level = BTRFS_COMPRESS_LEVEL(type); - - if (level > 9) + if (!level) + level = BTRFS_ZLIB_DEFAULT_LEVEL; + else if (level > 9) level = 9; - workspace->level = level > 0 ? level : 3; + return level; } const struct btrfs_compress_op btrfs_zlib_compress = { diff --git a/fs/btrfs/zstd.c b/fs/btrfs/zstd.c index 404101864220..43f3be755b8c 100644 --- a/fs/btrfs/zstd.c +++ b/fs/btrfs/zstd.c @@ -441,8 +441,9 @@ static int zstd_decompress(struct list_head *ws, unsigned char *data_in, return ret; } -static void zstd_set_level(struct list_head *ws, unsigned int type) +static unsigned int zstd_set_level(unsigned int level) { + return ZSTD_BTRFS_DEFAULT_LEVEL; } const struct btrfs_compress_op btrfs_zstd_compress = { From patchwork Mon Jan 28 21:24:36 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dennis Zhou X-Patchwork-Id: 10784813 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 6CE1413B5 for ; Mon, 28 Jan 2019 21:25:16 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 55D4A2B279 for ; Mon, 28 Jan 2019 21:25:16 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 49DBE2B288; Mon, 28 Jan 2019 21:25:16 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id E7E932B279 for ; Mon, 28 Jan 2019 21:25:15 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728461AbfA1VZK (ORCPT ); Mon, 28 Jan 2019 16:25:10 -0500 Received: from mail-yw1-f67.google.com ([209.85.161.67]:38113 "EHLO mail-yw1-f67.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728416AbfA1VY7 (ORCPT ); Mon, 28 Jan 2019 16:24:59 -0500 Received: by mail-yw1-f67.google.com with SMTP id d190so7356779ywb.5; Mon, 28 Jan 2019 13:24:59 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=i95HDMYBMKJoHPehH+6R+py5+DMch2uym+tajUEb64U=; b=cfhCLJOc45EhyGkR5uZlhot+DKHiqJVXdCjIvua9NQbk3fG8SZff/3SiM0ik4G+29i zkJRhwNyYnelUE1tmLFmtBvKvVDN+WP5effZ2mDXiAcKW9H9xQ8cvr0TStjmjEKaAhUh wzzrGXaeJt4zPqLFdJg770SH9RgcxtYBO9MrlFigo/MUouAXMmM2qXxmMGae0Vma2XU5 IrN0s8ZgJmGGBbwXgWYWAh6qJRoFrraO3rQAPmwddmYDTnSjXL5qbyAi581MC0orhRqI YYJGPiP+fPUwn9DNILJR/kBU/Jkvm9lO7jn4rJ5ZFtxIL0cSRQ7K45ZfGe2ZAthxqO6m bN3A== X-Gm-Message-State: AJcUukcfsrwWfxhf2R9WVVr8UZaI+olTInYMN0OQYW8aGhL2BBTmg1Jg 4227k7IjcMpKOVYDUhBov5o= X-Google-Smtp-Source: ALg8bN4+2RtE73arjUvpCBYxFMWs7AZsXIQ4B0tYbFHffl1Xt59wM08CqjISXIkFVJHZYHIOZfTkjw== X-Received: by 2002:a81:af0c:: with SMTP id n12mr21761482ywh.446.1548710698757; Mon, 28 Jan 2019 13:24:58 -0800 (PST) Received: from dennisz-mbp.thefacebook.com ([199.201.65.135]) by smtp.gmail.com with ESMTPSA id a7sm24170415ywb.18.2019.01.28.13.24.57 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Mon, 28 Jan 2019 13:24:57 -0800 (PST) From: Dennis Zhou To: David Sterba , Josef Bacik , Chris Mason , Omar Sandoval , Nick Terrell Cc: kernel-team@fb.com, linux-btrfs@vger.kernel.org, linux-kernel@vger.kernel.org, Dennis Zhou Subject: [PATCH 10/11] btrfs: zstd use the passed through level instead of default Date: Mon, 28 Jan 2019 16:24:36 -0500 Message-Id: <20190128212437.11597-11-dennis@kernel.org> X-Mailer: git-send-email 2.13.5 In-Reply-To: <20190128212437.11597-1-dennis@kernel.org> References: <20190128212437.11597-1-dennis@kernel.org> Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Zstd currently only supports the default level of compression. This patch switches to using the level passed in for btrfs zstd configuration. Zstd workspaces now keep track of the requested level as this can differ from the size of the workspace. Signed-off-by: Dennis Zhou Reviewed-by: Nikolay Borisov --- fs/btrfs/zstd.c | 19 +++++++++++++------ 1 file changed, 13 insertions(+), 6 deletions(-) diff --git a/fs/btrfs/zstd.c b/fs/btrfs/zstd.c index 43f3be755b8c..a951d4fe77f7 100644 --- a/fs/btrfs/zstd.c +++ b/fs/btrfs/zstd.c @@ -21,10 +21,10 @@ #define ZSTD_BTRFS_MAX_INPUT (1 << ZSTD_BTRFS_MAX_WINDOWLOG) #define ZSTD_BTRFS_DEFAULT_LEVEL 3 -static ZSTD_parameters zstd_get_btrfs_parameters(size_t src_len) +static ZSTD_parameters zstd_get_btrfs_parameters(unsigned int level, + size_t src_len) { - ZSTD_parameters params = ZSTD_getParams(ZSTD_BTRFS_DEFAULT_LEVEL, - src_len, 0); + ZSTD_parameters params = ZSTD_getParams(level, src_len, 0); if (params.cParams.windowLog > ZSTD_BTRFS_MAX_WINDOWLOG) params.cParams.windowLog = ZSTD_BTRFS_MAX_WINDOWLOG; @@ -36,6 +36,7 @@ struct workspace { void *mem; size_t size; char *buf; + unsigned int req_level; struct list_head list; ZSTD_inBuffer in_buf; ZSTD_outBuffer out_buf; @@ -55,7 +56,12 @@ static void zstd_cleanup_workspace_manager(void) static struct list_head *zstd_get_workspace(unsigned int level) { - return btrfs_get_workspace(&wsm, level); + struct list_head *ws = btrfs_get_workspace(&wsm, level); + struct workspace *workspace = list_entry(ws, struct workspace, list); + + workspace->req_level = level; + + return ws; } static void zstd_put_workspace(struct list_head *ws) @@ -75,7 +81,7 @@ static void zstd_free_workspace(struct list_head *ws) static struct list_head *zstd_alloc_workspace(unsigned int level) { ZSTD_parameters params = - zstd_get_btrfs_parameters(ZSTD_BTRFS_MAX_INPUT); + zstd_get_btrfs_parameters(level, ZSTD_BTRFS_MAX_INPUT); struct workspace *workspace; workspace = kzalloc(sizeof(*workspace), GFP_KERNEL); @@ -117,7 +123,8 @@ static int zstd_compress_pages(struct list_head *ws, unsigned long len = *total_out; const unsigned long nr_dest_pages = *out_pages; unsigned long max_out = nr_dest_pages * PAGE_SIZE; - ZSTD_parameters params = zstd_get_btrfs_parameters(len); + ZSTD_parameters params = zstd_get_btrfs_parameters(workspace->req_level, + len); *out_pages = 0; *total_out = 0; From patchwork Mon Jan 28 21:24:37 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dennis Zhou X-Patchwork-Id: 10784811 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 2CF6213B5 for ; Mon, 28 Jan 2019 21:25:10 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 184242B279 for ; Mon, 28 Jan 2019 21:25:10 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 08B3E2B288; Mon, 28 Jan 2019 21:25:10 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id C226E2B279 for ; Mon, 28 Jan 2019 21:25:08 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728444AbfA1VZC (ORCPT ); Mon, 28 Jan 2019 16:25:02 -0500 Received: from mail-yb1-f196.google.com ([209.85.219.196]:38288 "EHLO mail-yb1-f196.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728439AbfA1VZB (ORCPT ); Mon, 28 Jan 2019 16:25:01 -0500 Received: by mail-yb1-f196.google.com with SMTP id x9so7322193ybj.5; Mon, 28 Jan 2019 13:25:00 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=Q6V2t4AO/NQ51NpDSK0iwO1J92PFaU8+yoOOQaviHM4=; b=fp1HrYOEkdaeyIPdOKfVNiIhSvlqd4DunGTvWyL12yh4LNTIPW0THH58EfZahrOgLg 569WirSynzKzpxlkOJmkos4iffWIzmXvvguVeNWL3CT3hywUNWjAaWMSqttCEzv0EmbZ aPEYmFQPviJnEHIelDPI8sLFke/uauPa09KOx/YLEaOVLlaF7k4SwLZy4pVnAugtq5v2 bhITrWoB/bAqm2KJdW2nJ1h9YhLy+nOvQU5od1ypaE9PvDoYq3Z66ISkAtJ/B0OmJ27Y OhgPhI1+O+4PtVsnl+v51ZXXZ9z+u16I4lhIxOnNaSdIA19JyTkWcBK7rfoOQ/YMNVu2 /RwQ== X-Gm-Message-State: AJcUukfu4AZEwGIjeJd4BiShMMsOyZxN1oxiqYXcgs7fcxuvpvMEA9MO XrTZSlwjX2erdqvNzMPETtA= X-Google-Smtp-Source: ALg8bN6QT/dtSJDFmyckWDZyA4N68JITtWiBg7ui6u7SCP7r7+FifrxbCWEuW7QcI+ytqhO0akRtwQ== X-Received: by 2002:a25:a001:: with SMTP id x1mr21495015ybh.169.1548710700195; Mon, 28 Jan 2019 13:25:00 -0800 (PST) Received: from dennisz-mbp.thefacebook.com ([199.201.65.135]) by smtp.gmail.com with ESMTPSA id a7sm24170415ywb.18.2019.01.28.13.24.58 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Mon, 28 Jan 2019 13:24:59 -0800 (PST) From: Dennis Zhou To: David Sterba , Josef Bacik , Chris Mason , Omar Sandoval , Nick Terrell Cc: kernel-team@fb.com, linux-btrfs@vger.kernel.org, linux-kernel@vger.kernel.org, Dennis Zhou , Omar Sandoval Subject: [PATCH 11/11] btrfs: add zstd compression level support Date: Mon, 28 Jan 2019 16:24:37 -0500 Message-Id: <20190128212437.11597-12-dennis@kernel.org> X-Mailer: git-send-email 2.13.5 In-Reply-To: <20190128212437.11597-1-dennis@kernel.org> References: <20190128212437.11597-1-dennis@kernel.org> Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Zstd compression requires different amounts of memory for each level of compression. The prior patches implemented indirection to allow for each compression type to manage their workspaces independently. This patch uses this indirection to implement compression level support for zstd. As mentioned above, a requirement that differs zstd from zlib is that higher levels of compression require more memory. To manage this, each compression level has its own queue of workspaces. A global LRU is used to help with reclaim. To guarantee forward progress, a max level workspace is preallocated and hidden from the LRU. When getting a workspace, it uses a bitmap to identify the levels that are populated and scans up. If it finds a workspace that is greater than it, it uses it, but does not update the last_used time and the corresponding place in the LRU. This provides a mechanism to decrease memory utilization as we only keep around workspaces that are sized appropriately for the in use compression levels. By knowing which compression levels have available workspaces, we can recycle rather than always create new workspaces as well as take advantage of the preallocated max level for forward progress. If we hit memory pressure, we sleep on the max level workspace. We continue to rescan in case we can use a smaller workspace, but eventually should be able to obtain the max level workspace or allocate one again should memory pressure subside. The memory requirement for decompression is the same as level 1, and therefore can use any of available workspace. The number of workspaces is bound by an upper limit of the workqueue's limit which currently is 2 (percpu limit). Second, a reclaim timer is used to free inactive/improperly sized workspaces. The reclaim timer is set to 67s to avoid colliding with transaction commit (every 30s) and attempts to reclaim any unused workspace older than 45s. Repeating the experiment from v2 [1], the Silesia corpus was copied to a btrfs filesystem 10 times and then read back after dropping the caches. The btrfs filesystem was on an SSD. Level Ratio Compression (MB/s) Decompression (MB/s) 1 2.658 438.47 910.51 2 2.744 364.86 886.55 3 2.801 336.33 828.41 4 2.858 286.71 886.55 5 2.916 212.77 556.84 6 2.363 119.82 990.85 7 3.000 154.06 849.30 8 3.011 159.54 875.03 9 3.025 100.51 940.15 10 3.033 118.97 616.26 11 3.036 94.19 802.11 12 3.037 73.45 931.49 13 3.041 55.17 835.26 14 3.087 44.70 716.78 15 3.126 37.30 878.84 [1] https://lore.kernel.org/linux-btrfs/20181031181108.289340-1-terrelln@fb.com/ Signed-off-by: Dennis Zhou Cc: Nick Terrell Cc: Omar Sandoval --- fs/btrfs/super.c | 6 +- fs/btrfs/zstd.c | 229 +++++++++++++++++++++++++++++++++++++++++++++-- 2 files changed, 226 insertions(+), 9 deletions(-) diff --git a/fs/btrfs/super.c b/fs/btrfs/super.c index b28dff207383..0ecc513cb56c 100644 --- a/fs/btrfs/super.c +++ b/fs/btrfs/super.c @@ -544,9 +544,13 @@ int btrfs_parse_options(struct btrfs_fs_info *info, char *options, btrfs_clear_opt(info->mount_opt, NODATASUM); btrfs_set_fs_incompat(info, COMPRESS_LZO); no_compress = 0; - } else if (strcmp(args[0].from, "zstd") == 0) { + } else if (strncmp(args[0].from, "zstd", 4) == 0) { compress_type = "zstd"; info->compress_type = BTRFS_COMPRESS_ZSTD; + info->compress_level = + btrfs_compress_str2level( + BTRFS_COMPRESS_ZSTD, + args[0].from + 4); btrfs_set_opt(info->mount_opt, COMPRESS); btrfs_clear_opt(info->mount_opt, NODATACOW); btrfs_clear_opt(info->mount_opt, NODATASUM); diff --git a/fs/btrfs/zstd.c b/fs/btrfs/zstd.c index a951d4fe77f7..ce9b466c197f 100644 --- a/fs/btrfs/zstd.c +++ b/fs/btrfs/zstd.c @@ -6,20 +6,27 @@ */ #include +#include #include #include #include #include +#include #include #include #include #include #include #include "compression.h" +#include "ctree.h" #define ZSTD_BTRFS_MAX_WINDOWLOG 17 #define ZSTD_BTRFS_MAX_INPUT (1 << ZSTD_BTRFS_MAX_WINDOWLOG) #define ZSTD_BTRFS_DEFAULT_LEVEL 3 +#define ZSTD_BTRFS_MAX_LEVEL 15 +#define ZSTD_BTRFS_RECLAIM_NS (45 * NSEC_PER_SEC) +/* 67s to avoid clashing with transaction commit (every 30s) */ +#define ZSTD_BTRFS_RECLAIM_JIFFIES (67 * HZ) static ZSTD_parameters zstd_get_btrfs_parameters(unsigned int level, size_t src_len) @@ -36,37 +43,234 @@ struct workspace { void *mem; size_t size; char *buf; + unsigned int level; unsigned int req_level; + u64 last_used; struct list_head list; + struct list_head lru_list; ZSTD_inBuffer in_buf; ZSTD_outBuffer out_buf; }; -static struct workspace_manager wsm; +struct zstd_workspace_manager { + const struct btrfs_compress_op *ops; + spinlock_t lock; + struct list_head lru_list; + struct list_head idle_ws[ZSTD_BTRFS_MAX_LEVEL]; + unsigned long active_map; + wait_queue_head_t wait; + struct timer_list timer; +}; + +static struct zstd_workspace_manager wsm; + +static inline struct workspace *list_to_workspace(struct list_head *list) +{ + return container_of(list, struct workspace, list); +} + +/* + * zstd_reclaim_timer_fn - reclaim timer + * @t: timer + * + * This is scheduled every ZSTD_BTRFS_RECLAIM_JIFFIES and checks if a workspace + * has not been used by the corresponding level for ZSTD_BTRFS_RECLAIM_NS. + */ +static void zstd_reclaim_timer_fn(struct timer_list *t) +{ + u64 now = ktime_get_ns(); + struct list_head *pos, *next; + + spin_lock(&wsm.lock); + + if (list_empty(&wsm.lru_list)) { + spin_unlock(&wsm.lock); + return; + } + + list_for_each_prev_safe(pos, next, &wsm.lru_list) { + struct workspace *victim = container_of(pos, struct workspace, + lru_list); + unsigned int level; + + if (now < victim->last_used + ZSTD_BTRFS_RECLAIM_NS) + break; + + /* workspace is in use */ + if (victim->req_level) + continue; + + level = victim->level; + list_del(&victim->lru_list); + list_del(&victim->list); + wsm.ops->free_workspace(&victim->list); + + if (list_empty(&wsm.idle_ws[level - 1])) + clear_bit(level - 1, &wsm.active_map); + + } + + if (!list_empty(&wsm.lru_list)) + mod_timer(&wsm.timer, jiffies + ZSTD_BTRFS_RECLAIM_JIFFIES); + + spin_unlock(&wsm.lock); +} static void zstd_init_workspace_manager(void) { - btrfs_init_workspace_manager(&wsm, &btrfs_zstd_compress); + struct list_head *ws; + int i; + + wsm.ops = &btrfs_zstd_compress; + spin_lock_init(&wsm.lock); + init_waitqueue_head(&wsm.wait); + timer_setup(&wsm.timer, zstd_reclaim_timer_fn, 0); + + INIT_LIST_HEAD(&wsm.lru_list); + for (i = 0; i < ZSTD_BTRFS_MAX_LEVEL; i++) + INIT_LIST_HEAD(&wsm.idle_ws[i]); + + ws = wsm.ops->alloc_workspace(ZSTD_BTRFS_MAX_LEVEL); + if (IS_ERR(ws)) { + pr_warn("BTRFS: cannot preallocate zstd compression workspace\n"); + } else { + set_bit(ZSTD_BTRFS_MAX_LEVEL - 1, &wsm.active_map); + list_add(ws, &wsm.idle_ws[ZSTD_BTRFS_MAX_LEVEL - 1]); + } } static void zstd_cleanup_workspace_manager(void) { - btrfs_cleanup_workspace_manager(&wsm); + struct workspace *workspace; + int i; + + for (i = 0; i < ZSTD_BTRFS_MAX_LEVEL; i++) { + while (!list_empty(&wsm.idle_ws[i])) { + workspace = container_of(wsm.idle_ws[i].next, + struct workspace, list); + list_del(&workspace->list); + list_del(&workspace->lru_list); + wsm.ops->free_workspace(&workspace->list); + } + } +} + +/* + * zstd_find_workspace - find workspace + * @level: compression level + * + * This iterates over the set bits in the active_map beginning at the requested + * compression level. This lets us utilize already allocated workspaces before + * allocating a new one. If the workspace is of a larger size, it is used, but + * the place in the lru_list and last_used times are not updated. This is to + * offer the opportunity to reclaim the workspace in favor of allocating an + * appropriately sized one in the future. + */ +static struct list_head *zstd_find_workspace(unsigned int level) +{ + struct list_head *ws; + struct workspace *workspace; + int i = level - 1; + + spin_lock(&wsm.lock); + for_each_set_bit_from(i, &wsm.active_map, ZSTD_BTRFS_MAX_LEVEL) { + if (!list_empty(&wsm.idle_ws[i])) { + ws = wsm.idle_ws[i].next; + workspace = list_to_workspace(ws); + list_del_init(ws); + /* keep its place if it's a lower level using this */ + workspace->req_level = level; + if (level == workspace->level) + list_del(&workspace->lru_list); + if (list_empty(&wsm.idle_ws[i])) + clear_bit(i, &wsm.active_map); + spin_unlock(&wsm.lock); + return ws; + } + } + spin_unlock(&wsm.lock); + + return NULL; } +/* + * zstd_get_workspace - zstd's get_workspace + * @level: compression level + * + * If @level is 0, then any compression level can be used. Therefore, we begin + * scanning from 1. We first scan through possible workspaces and then after + * attempt to allocate a new workspace. If we fail to allocate one due to + * memory pressure, go to sleep waiting for the max level workspace to free up. + */ static struct list_head *zstd_get_workspace(unsigned int level) { - struct list_head *ws = btrfs_get_workspace(&wsm, level); - struct workspace *workspace = list_entry(ws, struct workspace, list); + struct list_head *ws; + unsigned long nofs_flag; - workspace->req_level = level; + /* level == 0 means we can use any workspace */ + if (!level) + level = 1; + +again: + ws = zstd_find_workspace(level); + if (ws) + return ws; + + nofs_flag = memalloc_nofs_save(); + ws = wsm.ops->alloc_workspace(level); + memalloc_nofs_restore(nofs_flag); + + if (IS_ERR(ws)) { + DEFINE_WAIT(wait); + + prepare_to_wait(&wsm.wait, &wait, TASK_UNINTERRUPTIBLE); + schedule(); + finish_wait(&wsm.wait, &wait); + + goto again; + } return ws; } +/* + * zstd_put_workspace - zstd put_workspace + * @ws: list_head for the workspace + * + * When putting back a workspace, we only need to update the LRU if we are of + * the requested compression level. Here is where we continue to protect the + * max level workspace or update last_used accordingly. If the reclaim timer + * isn't set, it is also set here. Only the max level workspace tries and wakes + * up waiting workspaces. + */ static void zstd_put_workspace(struct list_head *ws) { - btrfs_put_workspace(&wsm, ws); + struct workspace *workspace = list_to_workspace(ws); + + spin_lock(&wsm.lock); + + /* a node is only taken off the lru if we are the corresponding level */ + if (workspace->req_level == workspace->level) { + /* hide a max level workspace from reclaim */ + if (list_empty(&wsm.idle_ws[ZSTD_BTRFS_MAX_LEVEL - 1])) { + INIT_LIST_HEAD(&workspace->lru_list); + } else { + workspace->last_used = ktime_get_ns(); + list_add(&workspace->lru_list, &wsm.lru_list); + if (!timer_pending(&wsm.timer)) + mod_timer(&wsm.timer, + jiffies + ZSTD_BTRFS_RECLAIM_JIFFIES); + } + } + + set_bit(workspace->level - 1, &wsm.active_map); + list_add(&workspace->list, &wsm.idle_ws[workspace->level - 1]); + workspace->req_level = 0; + + spin_unlock(&wsm.lock); + + if (workspace->level == ZSTD_BTRFS_MAX_LEVEL) + cond_wake_up(&wsm.wait); } static void zstd_free_workspace(struct list_head *ws) @@ -93,10 +297,14 @@ static struct list_head *zstd_alloc_workspace(unsigned int level) ZSTD_DStreamWorkspaceBound(ZSTD_BTRFS_MAX_INPUT)); workspace->mem = kvmalloc(workspace->size, GFP_KERNEL); workspace->buf = kmalloc(PAGE_SIZE, GFP_KERNEL); + workspace->level = level; + workspace->req_level = level; + workspace->last_used = ktime_get_ns(); if (!workspace->mem || !workspace->buf) goto fail; INIT_LIST_HEAD(&workspace->list); + INIT_LIST_HEAD(&workspace->lru_list); return &workspace->list; fail: @@ -450,7 +658,12 @@ static int zstd_decompress(struct list_head *ws, unsigned char *data_in, static unsigned int zstd_set_level(unsigned int level) { - return ZSTD_BTRFS_DEFAULT_LEVEL; + if (!level) + level = ZSTD_BTRFS_DEFAULT_LEVEL; + else if (level > ZSTD_BTRFS_MAX_LEVEL) + level = ZSTD_BTRFS_MAX_LEVEL; + + return level; } const struct btrfs_compress_op btrfs_zstd_compress = {