mbox series

[v2,00/16] Evictable fanotify marks

Message ID 20220329074904.2980320-1-amir73il@gmail.com (mailing list archive)
Headers show
Series Evictable fanotify marks | expand

Message

Amir Goldstein March 29, 2022, 7:48 a.m. UTC 
Jan,

Following the discussion on direct reclaim of fsnotify marks [2],
this patch set includes your suggested fixes to core code along with
implementation of fanotify evictable marks (rebrand of volatile marks).

The LTP test I wrote [3] reproduces that deadlock within seconds on my
small test VM if the FSNOTIFY_GROUP_NOFS flag is removed from fanotify.

To be more exact, depending on the value of vfs_cache_pressure set by
the test, either a deadlock or lockdep warning (or both) are reproduced.
I chose a high value of 500, which usually reproduces only the lockdep
warning, but worked better and faster on several systems I tested on.

Thanks,
Amir.

Changes since v1 [1]:
- Fixes for direct reclaim deadlock
- Add ioctl for direct reclaim test
- Rebrand as FAN_MARK_EVICTABLE
- Remove FAN_MARK_CREATE and allow clearing FAN_MARK_EVICTABLE
- Replace connector proxy_iref with HAS_IREF flag
- Take iref in fsnotify_reclac_mark() rather than on add mark to list
- Remove fsnotify_add_mark() allow_dups/flags argument
- Remove pr_debug() prints

[1] https://lore.kernel.org/r/20220307155741.1352405-1-amir73il@gmail.com/
[2] https://lore.kernel.org/r/20220321112310.vpr7oxro2xkz5llh@quack3.lan/
[3] https://github.com/amir73il/ltp/commits/fan_evictable

Amir Goldstein (16):
  inotify: show inotify mask flags in proc fdinfo
  inotify: move control flags from mask to mark flags
  fsnotify: pass flags argument to fsnotify_add_mark() via mark
  fsnotify: remove unneeded refcounts of s_fsnotify_connectors
  fsnotify: fix wrong lockdep annotations
  fsnotify: create helpers for group mark_mutex lock
  inotify: use fsnotify group lock helpers
  audit: use fsnotify group lock helpers
  nfsd: use fsnotify group lock helpers
  dnotify: use fsnotify group lock helpers
  fsnotify: allow adding an inode mark without pinning inode
  fanotify: factor out helper fanotify_mark_update_flags()
  fanotify: implement "evictable" inode marks
  fanotify: add FAN_IOC_SET_MARK_PAGE_ORDER ioctl for testing
  fanotify: use fsnotify group lock helpers
  fanotify: enable "evictable" inode marks

 fs/nfsd/filecache.c                  |  12 +--
 fs/notify/dnotify/dnotify.c          |  13 +--
 fs/notify/fanotify/fanotify.c        |   6 +-
 fs/notify/fanotify/fanotify.h        |  12 +++
 fs/notify/fanotify/fanotify_user.c   | 128 +++++++++++++++++++++------
 fs/notify/fdinfo.c                   |  22 ++---
 fs/notify/fsnotify.c                 |   4 +-
 fs/notify/group.c                    |  11 +++
 fs/notify/inotify/inotify.h          |  19 ++++
 fs/notify/inotify/inotify_fsnotify.c |   2 +-
 fs/notify/inotify/inotify_user.c     |  46 ++++++----
 fs/notify/mark.c                     | 126 +++++++++++++++-----------
 include/linux/fanotify.h             |   1 +
 include/linux/fsnotify_backend.h     |  86 ++++++++++++++----
 include/uapi/linux/fanotify.h        |   5 ++
 kernel/audit_fsnotify.c              |   3 +-
 kernel/audit_tree.c                  |  32 +++----
 17 files changed, 370 insertions(+), 158 deletions(-)

Comments

Amir Goldstein April 9, 2022, 4:12 p.m. UTC | #1 
On Tue, Mar 29, 2022 at 10:49 AM Amir Goldstein <amir73il@gmail.com> wrote:
>
> Jan,
>
> Following the discussion on direct reclaim of fsnotify marks [2],
> this patch set includes your suggested fixes to core code along with
> implementation of fanotify evictable marks (rebrand of volatile marks).
>
> The LTP test I wrote [3] reproduces that deadlock within seconds on my
> small test VM if the FSNOTIFY_GROUP_NOFS flag is removed from fanotify.
>
> To be more exact, depending on the value of vfs_cache_pressure set by
> the test, either a deadlock or lockdep warning (or both) are reproduced.
> I chose a high value of 500, which usually reproduces only the lockdep
> warning, but worked better and faster on several systems I tested on.
>
> Thanks,
> Amir.
>
> Changes since v1 [1]:
> - Fixes for direct reclaim deadlock
> - Add ioctl for direct reclaim test
> - Rebrand as FAN_MARK_EVICTABLE
> - Remove FAN_MARK_CREATE and allow clearing FAN_MARK_EVICTABLE
> - Replace connector proxy_iref with HAS_IREF flag
> - Take iref in fsnotify_reclac_mark() rather than on add mark to list
> - Remove fsnotify_add_mark() allow_dups/flags argument
> - Remove pr_debug() prints
>
> [1] https://lore.kernel.org/r/20220307155741.1352405-1-amir73il@gmail.com/
> [2] https://lore.kernel.org/r/20220321112310.vpr7oxro2xkz5llh@quack3.lan/
> [3] https://github.com/amir73il/ltp/commits/fan_evictable

And here is a first man-page draft:

https://github.com/amir73il/man-pages/commits/fan_evictable

Thanks,
Amir.