| Message ID | cover.1701468305.git.josef@toxicpanda.com (mailing list archive) |
|---|---|
| Headers | show |
| Series | btrfs: add fscrypt support | expand |
On Fri, Dec 01, 2023 at 05:10:57PM -0500, Josef Bacik wrote: > Hello, > > v3 can be found here > > https://lore.kernel.org/linux-btrfs/cover.1697480198.git.josef@toxicpanda.com/ Sorry Eric, it's been a long week and I forgot how to use email, didn't cc you or linux-fscrypt on this series. It's on fsdevel and the btrfs list. Thanks, Josef
On Fri, Dec 01, 2023 at 05:10:57PM -0500, Josef Bacik wrote: > Hello, > > v3 can be found here > > https://lore.kernel.org/linux-btrfs/cover.1697480198.git.josef@toxicpanda.com/ > > There's been a longer delay between versions than I'd like, this was mostly due > to Plumbers, Holidays, and then uncovering a bunch of new issues with '-o > test_dummy_encryption'. I'm still working through some of the btrfs specific > failures, but the fscrypt side appears to be stable. I had to add a few changes > to fscrypt since the last time, but nothing earth shattering, just moving the > keyring destruction and adding a helper we need for btrfs send to work properly. > > This is passing a good chunk of the fstests, at this point the majority appear > to be cases where I need to exclude the test when using test_dummy_encryption > because of various limitations of our tools or other infrastructure related > things. > > I likely will have a follow-up series with more fixes, but the bulk of this is > unchanged since the last posting. There were some bug fixes and such but the > overall design remains the same. Thanks, > Well, it wouldn't be Linux kernel development without patchsets that don't mention what they apply to... I managed to apply this for reviewing after spending a while choosing the base that seemed to work best (6d3880a76eedd from kdave/for-next) and resolving conflicts. It would save a lot of time if proper information was included, though. - Eric
On Mon, Dec 04, 2023 at 05:49:17PM -0800, Eric Biggers wrote: > On Fri, Dec 01, 2023 at 05:10:57PM -0500, Josef Bacik wrote: > > Hello, > > > > v3 can be found here > > > > https://lore.kernel.org/linux-btrfs/cover.1697480198.git.josef@toxicpanda.com/ > > > > There's been a longer delay between versions than I'd like, this was mostly due > > to Plumbers, Holidays, and then uncovering a bunch of new issues with '-o > > test_dummy_encryption'. I'm still working through some of the btrfs specific > > failures, but the fscrypt side appears to be stable. I had to add a few changes > > to fscrypt since the last time, but nothing earth shattering, just moving the > > keyring destruction and adding a helper we need for btrfs send to work properly. > > > > This is passing a good chunk of the fstests, at this point the majority appear > > to be cases where I need to exclude the test when using test_dummy_encryption > > because of various limitations of our tools or other infrastructure related > > things. > > > > I likely will have a follow-up series with more fixes, but the bulk of this is > > unchanged since the last posting. There were some bug fixes and such but the > > overall design remains the same. Thanks, > > > > Well, it wouldn't be Linux kernel development without patchsets that don't > mention what they apply to... I managed to apply this for reviewing after > spending a while choosing the base that seemed to work best (6d3880a76eedd from > kdave/for-next) and resolving conflicts. It would save a lot of time if proper > information was included, though. Right, the base branch should have been mentioned, we have the development git tree on github and not on kernel.org. For future reference: https://btrfs.readthedocs.io/en/latest/Source-repositories.html#kernel-module
On Tue, Dec 05, 2023 at 03:16:55PM +0100, David Sterba wrote: > > we have the development git tree on github and not on kernel.org. Then the MAINTAINERS entry for btrfs needs to be updated, as it points to kernel.org. I already had to update it for you guys the last time it changed (https://git.kernel.org/linus/eb91db63a90d8f8e). Not sure why you guys can't keep it up to date. BTW, https://github.com/btrfs/linux exists but it's the wrong one too, which makes it extra confusing. - Eric
Hi Josef and Sweet Tea, On Fri, Dec 01, 2023 at 05:10:57PM -0500, Josef Bacik wrote: > Hello, > > v3 can be found here > > https://lore.kernel.org/linux-btrfs/cover.1697480198.git.josef@toxicpanda.com/ > > There's been a longer delay between versions than I'd like, this was mostly due > to Plumbers, Holidays, and then uncovering a bunch of new issues with '-o > test_dummy_encryption'. I'm still working through some of the btrfs specific > failures, but the fscrypt side appears to be stable. I had to add a few changes > to fscrypt since the last time, but nothing earth shattering, just moving the > keyring destruction and adding a helper we need for btrfs send to work properly. > > This is passing a good chunk of the fstests, at this point the majority appear > to be cases where I need to exclude the test when using test_dummy_encryption > because of various limitations of our tools or other infrastructure related > things. > > I likely will have a follow-up series with more fixes, but the bulk of this is > unchanged since the last posting. There were some bug fixes and such but the > overall design remains the same. Thanks, > Is there a plan for someone to keep working on this? I think it was finally getting somewhere, but the work on it seems to have stopped. Thanks, - Eric
On Tue, Apr 09, 2024 at 07:42:22PM -0400, Eric Biggers wrote: > Hi Josef and Sweet Tea, > > On Fri, Dec 01, 2023 at 05:10:57PM -0500, Josef Bacik wrote: > > Hello, > > > > v3 can be found here > > > > https://lore.kernel.org/linux-btrfs/cover.1697480198.git.josef@toxicpanda.com/ > > > > There's been a longer delay between versions than I'd like, this was mostly due > > to Plumbers, Holidays, and then uncovering a bunch of new issues with '-o > > test_dummy_encryption'. I'm still working through some of the btrfs specific > > failures, but the fscrypt side appears to be stable. I had to add a few changes > > to fscrypt since the last time, but nothing earth shattering, just moving the > > keyring destruction and adding a helper we need for btrfs send to work properly. > > > > This is passing a good chunk of the fstests, at this point the majority appear > > to be cases where I need to exclude the test when using test_dummy_encryption > > because of various limitations of our tools or other infrastructure related > > things. > > > > I likely will have a follow-up series with more fixes, but the bulk of this is > > unchanged since the last posting. There were some bug fixes and such but the > > overall design remains the same. Thanks, > > > > Is there a plan for someone to keep working on this? I think it was finally > getting somewhere, but the work on it seems to have stopped. > I fixed up all your review comments, but yes we don't care about this internally anymore so it's been de-prioritized. I have to rebase onto the new stuff, re-run tests, fix any bugs that may have creeped in, but the current code addressed all of your comments. Once I get time to get back to this you'll have a new version in your inbox, but that may be some time. Thanks, Josef
Hello, v3 can be found here https://lore.kernel.org/linux-btrfs/cover.1697480198.git.josef@toxicpanda.com/ There's been a longer delay between versions than I'd like, this was mostly due to Plumbers, Holidays, and then uncovering a bunch of new issues with '-o test_dummy_encryption'. I'm still working through some of the btrfs specific failures, but the fscrypt side appears to be stable. I had to add a few changes to fscrypt since the last time, but nothing earth shattering, just moving the keyring destruction and adding a helper we need for btrfs send to work properly. This is passing a good chunk of the fstests, at this point the majority appear to be cases where I need to exclude the test when using test_dummy_encryption because of various limitations of our tools or other infrastructure related things. I likely will have a follow-up series with more fixes, but the bulk of this is unchanged since the last posting. There were some bug fixes and such but the overall design remains the same. Thanks, Josef v3->v4: - Added support for '-o test_dummy_encryption' at Eric's suggestion, this uncovered a load of issues. - Preliminary work to support decrypting names for our various name resolution ioctls. I didn't get everything but I got the ones we depend on in fstests. - Preliminary work for send of an encrypted directory with the key loaded. There's probably still bugs in here, but it doesn't crash anymore. - Fixed how we limit the bio size to work with direct and buffered io. - Fixed using the wrong fscrypt extent context for writes into prealloc extents. Josef Bacik (31): fs: move fscrypt keyring destruction to after ->put_super fscrypt: add per-extent encryption support fscrypt: add a fscrypt_inode_open helper fscrypt: conditionally don't wipe mk secret until the last active user is done blk-crypto: add a process bio callback fscrypt: add documentation about extent encryption btrfs: add infrastructure for safe em freeing btrfs: add fscrypt_info and encryption_type to ordered_extent btrfs: plumb through setting the fscrypt_info for ordered extents btrfs: plumb the fscrypt extent context through create_io_em btrfs: populate the ordered_extent with the fscrypt context btrfs: keep track of fscrypt info and orig_start for dio reads btrfs: add an optional encryption context to the end of file extents btrfs: pass through fscrypt_extent_info to the file extent helpers btrfs: pass the fscrypt_info through the replace extent infrastructure btrfs: implement the fscrypt extent encryption hooks btrfs: setup fscrypt_extent_info for new extents btrfs: populate ordered_extent with the orig offset btrfs: set the bio fscrypt context when applicable btrfs: add a bio argument to btrfs_csum_one_bio btrfs: add orig_logical to btrfs_bio btrfs: limit encrypted writes to 256 segments btrfs: implement process_bio cb for fscrypt btrfs: add test_dummy_encryption support btrfs: don't rewrite ret from inode_permission btrfs: move inode_to_path higher in backref.c btrfs: make btrfs_ref_to_path handle encrypted filenames btrfs: don't search back for dir inode item in INO_LOOKUP_USER btrfs: deal with encrypted symlinks in send btrfs: decrypt file names for send btrfs: load the inode context before sending writes Omar Sandoval (7): fscrypt: expose fscrypt_nokey_name btrfs: disable various operations on encrypted inodes btrfs: start using fscrypt hooks btrfs: add inode encryption contexts btrfs: add new FEATURE_INCOMPAT_ENCRYPT flag btrfs: adapt readdir for encrypted and nokey names btrfs: implement fscrypt ioctls Sweet Tea Dorminy (8): btrfs: disable verity on encrypted inodes btrfs: handle nokey names. btrfs: add encryption to CONFIG_BTRFS_DEBUG btrfs: add get_devices hook for fscrypt btrfs: turn on inlinecrypt mount option for encrypt btrfs: set file extent encryption excplicitly btrfs: add fscrypt_info and encryption_type to extent_map btrfs: explicitly track file extent length for replace and drop Documentation/filesystems/fscrypt.rst | 41 ++ block/blk-crypto-fallback.c | 40 ++ block/blk-crypto-internal.h | 8 + block/blk-crypto-profile.c | 2 + block/blk-crypto.c | 6 +- fs/btrfs/Makefile | 1 + fs/btrfs/accessors.h | 50 +++ fs/btrfs/backref.c | 114 ++++-- fs/btrfs/bio.c | 75 +++- fs/btrfs/bio.h | 6 + fs/btrfs/btrfs_inode.h | 3 +- fs/btrfs/compression.c | 6 + fs/btrfs/ctree.h | 4 + fs/btrfs/defrag.c | 10 +- fs/btrfs/delayed-inode.c | 29 +- fs/btrfs/delayed-inode.h | 6 +- fs/btrfs/dir-item.c | 108 +++++- fs/btrfs/dir-item.h | 11 +- fs/btrfs/disk-io.c | 1 + fs/btrfs/extent_io.c | 114 +++++- fs/btrfs/extent_io.h | 3 + fs/btrfs/extent_map.c | 102 ++++- fs/btrfs/extent_map.h | 12 + fs/btrfs/file-item.c | 17 +- fs/btrfs/file-item.h | 7 +- fs/btrfs/file.c | 16 +- fs/btrfs/fs.h | 6 +- fs/btrfs/fscrypt.c | 412 ++++++++++++++++++++ fs/btrfs/fscrypt.h | 112 ++++++ fs/btrfs/inode.c | 518 ++++++++++++++++++++------ fs/btrfs/ioctl.c | 68 ++-- fs/btrfs/ordered-data.c | 36 +- fs/btrfs/ordered-data.h | 21 +- fs/btrfs/reflink.c | 8 + fs/btrfs/root-tree.c | 8 +- fs/btrfs/root-tree.h | 2 +- fs/btrfs/send.c | 133 ++++++- fs/btrfs/super.c | 75 ++++ fs/btrfs/sysfs.c | 6 + fs/btrfs/tree-checker.c | 66 +++- fs/btrfs/tree-log.c | 26 +- fs/btrfs/verity.c | 3 + fs/crypto/crypto.c | 10 +- fs/crypto/fname.c | 39 +- fs/crypto/fscrypt_private.h | 44 +++ fs/crypto/hooks.c | 42 +++ fs/crypto/inline_crypt.c | 87 ++++- fs/crypto/keyring.c | 18 +- fs/crypto/keysetup.c | 155 ++++++++ fs/crypto/policy.c | 59 +++ fs/super.c | 12 +- include/linux/blk-crypto.h | 9 +- include/linux/fscrypt.h | 130 +++++++ include/uapi/linux/btrfs.h | 1 + include/uapi/linux/btrfs_tree.h | 35 +- 55 files changed, 2619 insertions(+), 314 deletions(-) create mode 100644 fs/btrfs/fscrypt.c create mode 100644 fs/btrfs/fscrypt.h