| Message ID | 20240131230827.207552-1-bschubert@ddn.com (mailing list archive) |
|---|---|
| Headers | show |
| Series | fuse: inode IO modes and mmap | expand |
On Thu, Feb 1, 2024 at 1:08 AM Bernd Schubert <bschubert@ddn.com> wrote: > > This series is mostly about mmap, direct-IO and inode IO modes. > (new in this series is FOPEN_CACHE_IO). > It brings back the shared lock for FOPEN_DIRECT_IO when > FUSE_DIRECT_IO_ALLOW_MMAP is set and is also preparation > work for Amirs work on fuse-passthrough and also for For the interested: https://github.com/amir73il/linux/commits/fuse-backing-fd-010224/ Bernd, Can you push this series to your v2 branch so that I can rebase my branch on top of it? > shared lock O_DIRECT and direct-IO code consolidation I have > patches for. > > Patch 1/5 was already posted before > https://patchwork.kernel.org/project/linux-fsdevel/patch/20231213150703.6262-1-bschubert@ddn.com/ > but is included here again, as especially patch 5/5 has a > dependency on it. Amir has also spotted a typo in the commit message > of the initial patch, which is corrected here. > > Patches 2/5 and 3/5 add helper functions, which are needed by the > main patch (5/5) in this series and are be also needed by another > fuse direct-IO series. That series needs the helper functions in > fuse_cache_write_iter, thus, these new helpers are above that > function. > > Patch 4/5 allows to fail fuse_finish_open and is a preparation > to handle conflicting IO modes from the server side and will also be > needed for fuse passthrough. > > Patch 5/5 is the main patch in the series, which adds inode > IO modes, which is needed to re-enable shared DIO writes locks > when FUSE_DIRECT_IO_ALLOW_MMAP is set. Furthermore, these IO modes > are also needed by Amirs WIP fuse passthrough work. > > The conflict of FUSE_DIRECT_IO_ALLOW_MMAP and > FOPEN_PARALLEL_DIRECT_WRITES was detected by xfstest generic/095. > This patch series was tested by running a loop of that test > and also by multiple runs of the complete xfstest suite. > For testing with libfuse a version is needed that includes this > pull request > https://github.com/libfuse/libfuse/pull/870 Heh, this is already merged :) For the record, I understand that you ran this test with passthrough_hp. In which configurations --direct-io? --nocache? only default? Thanks for pushing this through! Amir.
Hi Amir, sorry for a bit late reply (*). On 2/1/24 11:30, Amir Goldstein wrote: > On Thu, Feb 1, 2024 at 1:08 AM Bernd Schubert <bschubert@ddn.com> wrote: >> >> This series is mostly about mmap, direct-IO and inode IO modes. >> (new in this series is FOPEN_CACHE_IO). >> It brings back the shared lock for FOPEN_DIRECT_IO when >> FUSE_DIRECT_IO_ALLOW_MMAP is set and is also preparation >> work for Amirs work on fuse-passthrough and also for > > For the interested: > https://github.com/amir73il/linux/commits/fuse-backing-fd-010224/ > > Bernd, > > Can you push this series to your v2 branch so that I can rebase > my branch on top of it? Do you mind if I push this to a different branch to keep the branch clean? > >> shared lock O_DIRECT and direct-IO code consolidation I have >> patches for. >> >> Patch 1/5 was already posted before >> https://patchwork.kernel.org/project/linux-fsdevel/patch/20231213150703.6262-1-bschubert@ddn.com/ >> but is included here again, as especially patch 5/5 has a >> dependency on it. Amir has also spotted a typo in the commit message >> of the initial patch, which is corrected here. >> >> Patches 2/5 and 3/5 add helper functions, which are needed by the >> main patch (5/5) in this series and are be also needed by another >> fuse direct-IO series. That series needs the helper functions in >> fuse_cache_write_iter, thus, these new helpers are above that >> function. >> >> Patch 4/5 allows to fail fuse_finish_open and is a preparation >> to handle conflicting IO modes from the server side and will also be >> needed for fuse passthrough. >> >> Patch 5/5 is the main patch in the series, which adds inode >> IO modes, which is needed to re-enable shared DIO writes locks >> when FUSE_DIRECT_IO_ALLOW_MMAP is set. Furthermore, these IO modes >> are also needed by Amirs WIP fuse passthrough work. >> >> The conflict of FUSE_DIRECT_IO_ALLOW_MMAP and >> FOPEN_PARALLEL_DIRECT_WRITES was detected by xfstest generic/095. >> This patch series was tested by running a loop of that test >> and also by multiple runs of the complete xfstest suite. >> For testing with libfuse a version is needed that includes this >> pull request >> https://github.com/libfuse/libfuse/pull/870 > > Heh, this is already merged :) Yeah, just not in any libfuse release yet. Btw, for some reasons I need to comment out "_require_aio" in tests/generic/095 to get the test to do something, I need to investigate why. > > For the record, I understand that you ran this test with passthrough_hp. > In which configurations --direct-io? --nocache? only default? I always test with both, which is why testing takes a bit time with all tests. Also both modes for generic/095, but that one with an additional loop (10 iterations), as in loop mode it found some issues in the initial patches. For the --direct-io I actually need a slight patch to get that passed through from local.config to passthrough_hp via /sbin/mount.fuse.passthrough. Will submit that to libfuse later on. > > Thanks for pushing this through! Well, thanks a lot for all of your work on it! Cheers, Bernd PS (*): _Another_ broken bicycle in the middle of way to the office.... Half of the spokes of my rear wheel broke, out of the nowhere.
On Thu, Feb 1, 2024 at 4:30 PM Bernd Schubert <bernd.schubert@fastmail.fm> wrote: > > Hi Amir, > > sorry for a bit late reply (*). > > On 2/1/24 11:30, Amir Goldstein wrote: > > On Thu, Feb 1, 2024 at 1:08 AM Bernd Schubert <bschubert@ddn.com> wrote: > >> > >> This series is mostly about mmap, direct-IO and inode IO modes. > >> (new in this series is FOPEN_CACHE_IO). > >> It brings back the shared lock for FOPEN_DIRECT_IO when > >> FUSE_DIRECT_IO_ALLOW_MMAP is set and is also preparation > >> work for Amirs work on fuse-passthrough and also for > > > > For the interested: > > https://github.com/amir73il/linux/commits/fuse-backing-fd-010224/ > > > > Bernd, > > > > Can you push this series to your v2 branch so that I can rebase > > my branch on top of it? > > Do you mind if I push this to a different branch to keep the branch clean? > I don't mind, I just thought that it would be nice if the branch fuse_io_mode-v2 actually contained the patches that were posted as V2 to the mailing list... Thanks, Amir.
On 2/1/24 16:56, Amir Goldstein wrote: > On Thu, Feb 1, 2024 at 4:30 PM Bernd Schubert > <bernd.schubert@fastmail.fm> wrote: >> >> Hi Amir, >> >> sorry for a bit late reply (*). >> >> On 2/1/24 11:30, Amir Goldstein wrote: >>> On Thu, Feb 1, 2024 at 1:08 AM Bernd Schubert <bschubert@ddn.com> wrote: >>>> >>>> This series is mostly about mmap, direct-IO and inode IO modes. >>>> (new in this series is FOPEN_CACHE_IO). >>>> It brings back the shared lock for FOPEN_DIRECT_IO when >>>> FUSE_DIRECT_IO_ALLOW_MMAP is set and is also preparation >>>> work for Amirs work on fuse-passthrough and also for >>> >>> For the interested: >>> https://github.com/amir73il/linux/commits/fuse-backing-fd-010224/ >>> >>> Bernd, >>> >>> Can you push this series to your v2 branch so that I can rebase >>> my branch on top of it? >> >> Do you mind if I push this to a different branch to keep the branch clean? >> > > I don't mind, I just thought that it would be nice if the branch fuse_io_mode-v2 > actually contained the patches that were posted as V2 to the mailing list... Ah sorry, I misread. I thought you wanted me to pull fuse-backing-fd-010224 onto the fuse_io_mode-v2 branch. I just pushed posted v2 to my branch. Thanks, Bernd
This series is mostly about mmap, direct-IO and inode IO modes. (new in this series is FOPEN_CACHE_IO). It brings back the shared lock for FOPEN_DIRECT_IO when FUSE_DIRECT_IO_ALLOW_MMAP is set and is also preparation work for Amirs work on fuse-passthrough and also for shared lock O_DIRECT and direct-IO code consolidation I have patches for. Patch 1/5 was already posted before https://patchwork.kernel.org/project/linux-fsdevel/patch/20231213150703.6262-1-bschubert@ddn.com/ but is included here again, as especially patch 5/5 has a dependency on it. Amir has also spotted a typo in the commit message of the initial patch, which is corrected here. Patches 2/5 and 3/5 add helper functions, which are needed by the main patch (5/5) in this series and are be also needed by another fuse direct-IO series. That series needs the helper functions in fuse_cache_write_iter, thus, these new helpers are above that function. Patch 4/5 allows to fail fuse_finish_open and is a preparation to handle conflicting IO modes from the server side and will also be needed for fuse passthrough. Patch 5/5 is the main patch in the series, which adds inode IO modes, which is needed to re-enable shared DIO writes locks when FUSE_DIRECT_IO_ALLOW_MMAP is set. Furthermore, these IO modes are also needed by Amirs WIP fuse passthrough work. The conflict of FUSE_DIRECT_IO_ALLOW_MMAP and FOPEN_PARALLEL_DIRECT_WRITES was detected by xfstest generic/095. This patch series was tested by running a loop of that test and also by multiple runs of the complete xfstest suite. For testing with libfuse a version is needed that includes this pull request https://github.com/libfuse/libfuse/pull/870 To: Miklos Szeredi <miklos@szeredi.hu> Cc: linux-fsdevel@vger.kernel.org Cc: Amir Goldstein <amir73il@gmail.com> Cc: Bernd Schubert <bernd.schubert@fastmail.fm> --- Changes in v2: Amir: - Added one more patch (4/5) "fuse: prepare for failing open response" - Updated the 5/5 (previously 4/4) fuse: introduce inode io modes - Fix FOPEN_CACHE_IO release, release could have happened without being actually taken (error cases, directory, dax-inode) - fuse_file_io_open does not implicitly set FOPEN_DIRECT_IO anymore, but waits until concurrent shared-locked direct-io is done or switched an exclusive lock). Amir Goldstein (2): fuse: prepare for failing open response fuse: introduce inode io modes Bernd Schubert (3): fuse: Fix VM_MAYSHARE and direct_io_allow_mmap fuse: Create helper function if DIO write needs exclusive lock fuse: Add fuse_dio_lock/unlock helper functions fs/fuse/dir.c | 8 +- fs/fuse/file.c | 339 ++++++++++++++++++++++++++++++++------ fs/fuse/fuse_i.h | 81 ++++++++- include/uapi/linux/fuse.h | 2 + 4 files changed, 375 insertions(+), 55 deletions(-)