From patchwork Fri Jan 22 20:46:46 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Omar Sandoval X-Patchwork-Id: 12040457 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH,MAILING_LIST_MULTI, MENTIONS_GIT_HOSTING,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5270AC433E6 for ; Fri, 22 Jan 2021 20:50:59 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 1168E23B09 for ; Fri, 22 Jan 2021 20:50:59 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1730028AbhAVUuS (ORCPT ); Fri, 22 Jan 2021 15:50:18 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49014 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1730125AbhAVUsD (ORCPT ); Fri, 22 Jan 2021 15:48:03 -0500 Received: from mail-pg1-x529.google.com (mail-pg1-x529.google.com [IPv6:2607:f8b0:4864:20::529]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B825DC061786 for ; Fri, 22 Jan 2021 12:47:15 -0800 (PST) Received: by mail-pg1-x529.google.com with SMTP id i5so4636069pgo.1 for ; Fri, 22 Jan 2021 12:47:15 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=osandov-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=SbGUA0qvIztEUivipPk+uMPWbkIun+CS9rfwLtftEGA=; b=VNTaJQyzqgMUf2bykgCnP/0Smb9RZ3fUkY1bEEk0/v6X4HX6oU3NemRHThtrG/kbcX X3w5dDAAhHOK5QHszpgqjOM9XEgwqx5MIf92WdRwsZguEkhOLh9s5uTpvJSJ3F9O74cP 0G93EHMthoPDEY/0FZWvccvTreM7iKLv/ykz5mVpaDk4+snLs0gJNLNYiB9TLxbyj5aN dGduN1uYnike/yE+BktozbvjaYaGjENqSNaEo/abdjPidhrWi5goDiSz7SPPfk3KfCz1 VPZYmI1mxOix4FZV5zs0VXMv2i8iJhAMnprc4RxaQEx076n4+22aJ0SCBacBCTq9yJEM iHJw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=SbGUA0qvIztEUivipPk+uMPWbkIun+CS9rfwLtftEGA=; b=sBzD+Y6c+6r2led9QJJV8mH0aGVy0Ng5Po8PvUrTXxmFeLUdG2pGaCOPeEHBK0en79 7KbEwvVFfYaUlzMt4uoKHE97ff+Y3mlavAHxleDCCVBx7ajO8SW0jTPxk6fSzhNix6Z5 3J5dS2iTY4pKa8mO2b3SgnrbqJUrF8QcDEnddyts1pC+5kgQPauO8idpYF2SoODE+ar9 RMpX/Zwu2xxshhfWplnb9EVEtM/bMUZtXEuY3G45KxHFGYdyWIRsg2I0gmuo1G9Zi3Rl xEeVmnOGLfhd1/ZBKbTVArToNCoxgeZr6eMGbYYOfEMZT5kDDNTLc1Z2gleHtZIACaop x+Xg== X-Gm-Message-State: AOAM530Ulc+QjnMNVK9vMrD26bsseuEeT0ACWT8vdmxKN5hwaZkWZaDl CH/tjcA01OG/VgMMTU0iKGKo1A== X-Google-Smtp-Source: ABdhPJy9fUypKEpbuM+Oepmztp98ZRomAQTyUsNmVY9Wx7Vnjv2cdEOp/sBHsDoiGH/+F+d3tCWHMQ== X-Received: by 2002:a62:aa06:0:b029:19d:f4d3:335e with SMTP id e6-20020a62aa060000b029019df4d3335emr1611187pff.60.1611348435252; Fri, 22 Jan 2021 12:47:15 -0800 (PST) Received: from relinquished.tfbnw.net ([2620:10d:c090:400::5:ea88]) by smtp.gmail.com with ESMTPSA id j18sm4092900pfc.99.2021.01.22.12.47.13 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 22 Jan 2021 12:47:14 -0800 (PST) From: Omar Sandoval To: linux-fsdevel@vger.kernel.org, linux-btrfs@vger.kernel.org, Al Viro , Christoph Hellwig Cc: Dave Chinner , Jann Horn , Amir Goldstein , Aleksa Sarai , linux-api@vger.kernel.org, kernel-team@fb.com Subject: [PATCH v7 00/10] fs: interface for directly reading/writing compressed data Date: Fri, 22 Jan 2021 12:46:46 -0800 Message-Id: X-Mailer: git-send-email 2.30.0 MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org From: Omar Sandoval This series adds an API for reading compressed data on a filesystem without decompressing it as well as support for writing compressed data directly to the filesystem. As with the previous submissions, I've included a man page patch describing the API. I have test cases (including fsstress support) and example programs which I'll send up [1]. The main use-case is Btrfs send/receive: currently, when sending data from one compressed filesystem to another, the sending side decompresses the data and the receiving side recompresses it before writing it out. This is wasteful and can be avoided if we can just send and write compressed extents. The patches implementing the send/receive support will be sent shortly. Patches 1-3 add the VFS support and UAPI. Patch 4 is a fix that this series depends on; it can be merged independently. Patches 5-8 are Btrfs prep patches. Patch 9 adds Btrfs encoded read support and patch 10 adds Btrfs encoded write support. These patches are based on Dave Sterba's Btrfs misc-next branch [2], which is in turn currently based on v5.11-rc4. Changes since v6 [3]: - Dropped O_CLOEXEC requirement for O_ALLOW_ENCODED. - Readded lost FMODE_ENCODED_IO check. - Used macros instead of enum for ENCODED_IOV_COMPRESSION_* and ENCODED_IOV_ENCRYPTION_*. - Moved encoded I/O definitions to their own UAPI header file. - Fixed style nits. - Added reviewed-bys. - Addressed man page nits. 1: https://github.com/osandov/xfstests/tree/rwf-encoded 2: https://github.com/kdave/btrfs-devel/tree/misc-next 3: https://lore.kernel.org/linux-btrfs/cover.1605723568.git.osandov@fb.com/ Omar Sandoval (10): iov_iter: add copy_struct_from_iter() fs: add O_ALLOW_ENCODED open flag fs: add RWF_ENCODED for reading/writing compressed data btrfs: fix check_data_csum() error message for direct I/O btrfs: don't advance offset for compressed bios in btrfs_csum_one_bio() btrfs: add ram_bytes and offset to btrfs_ordered_extent btrfs: support different disk extent size for delalloc btrfs: optionally extend i_size in cow_file_range_inline() btrfs: implement RWF_ENCODED reads btrfs: implement RWF_ENCODED writes Documentation/filesystems/encoded_io.rst | 74 ++ Documentation/filesystems/index.rst | 1 + arch/alpha/include/uapi/asm/fcntl.h | 1 + arch/parisc/include/uapi/asm/fcntl.h | 1 + arch/sparc/include/uapi/asm/fcntl.h | 1 + fs/btrfs/compression.c | 12 +- fs/btrfs/compression.h | 6 +- fs/btrfs/ctree.h | 9 +- fs/btrfs/delalloc-space.c | 18 +- fs/btrfs/file-item.c | 35 +- fs/btrfs/file.c | 46 +- fs/btrfs/inode.c | 936 ++++++++++++++++++++--- fs/btrfs/ordered-data.c | 80 +- fs/btrfs/ordered-data.h | 18 +- fs/btrfs/relocation.c | 4 +- fs/fcntl.c | 10 +- fs/namei.c | 4 + fs/read_write.c | 168 +++- include/linux/encoded_io.h | 17 + include/linux/fcntl.h | 2 +- include/linux/fs.h | 13 + include/linux/uio.h | 2 + include/uapi/asm-generic/fcntl.h | 4 + include/uapi/linux/encoded_io.h | 30 + include/uapi/linux/fs.h | 5 +- lib/iov_iter.c | 82 ++ 26 files changed, 1378 insertions(+), 201 deletions(-) create mode 100644 Documentation/filesystems/encoded_io.rst create mode 100644 include/linux/encoded_io.h create mode 100644 include/uapi/linux/encoded_io.h