[v2,00/16] Fanotify event with name info

Message

Amir Goldstein Feb. 17, 2020, 1:14 p.m. UTC

Jan,

This is v2 of the fanotify name info series.
The user requirement for the name info feature, as well as early UAPI
discussions can be found in this [1] lore thread.
The "prep" part of v1 was posted to the list [2] and includes two
minor bug fixes, but I decided not to split the submission into
two series this posting.

The patches are also available on my github branch fanotify_name [3]
along with LTP tests [4], man page draft [5] and a demo [6].

Patches 1-7 are cleanup and minor re-factoring in prep for the name
info patches.

Patches 8-9 are fixes for minor bug that I found during the work.
The referred LTP branch [4] includes improvements to ltp tests fanotify09
and fanotify15 to cover these bugs.  I did not mark those patches for
stable, because backporting is not trivial and the bugs are really minor.
For the same reason, I did not bother to provide bug fix patches that are
not dependent on the cleanup patches.

Patches 10-13 implement the new event type FAN_DIR_MODIFY per your
suggestion, which includes the directory fid and entry name info.

Patches 14-15 implement the FAN_REPORT_NAME init flag for reporting
name info on path type events.

Patch 16 is a "bonus" patch that implements an unprivileged fanotify
watch.  It is not proposed for merging at this time, but is provided
in order to demonstrate how name info reporting is applicable for an
unprivileged watcher, should we decide to implement the feature.
LTP tests, man page draft for unprivileged fanotify written by Matthew
Bobrowski are available on fanotify_unpriv branches in respective trees.

The inotify demo branch [6] includes a script test_demo.sh whose
output [7] can be seen here below.  The demo generates filesystem
events including file and directory renames, sleeps 2 seconds and
then reads the events from the queue and generates a report on
changes in the filesystem.

At event report time, the watcher uses open_by_handle_at(2) to report
up-to-date paths for parent dirs.  The last name element in the path
is reported as it was recorded at event time, but the watcher uses
fstatat(2) to check whether the reported entry is negative or positive.
Negative entry paths are annotated with "(deleted)" postfix.

The idea is that file change monitors will use this information to
query the content of modified directories and file and update a
secondary data structure or take other actions.

The demo scripts can run as root and non-root user.  When run as
non-root user, if the bonus FAN_UNPRIVILEGED patch is applied, it
demonstrates the unprivileged fanotify recursive watcher and produces
the exact same report information as the privileged filesystem watcher.

Thanks,
Amir.

Changes since v1:
- A few more cleanup patches
- Drop the abstract take_name_snapshot() vfs interface change
- Do not obfuscate event type for path type events
- Deal with the corner cases of event on root and disconnected dentry
- Bonus FAN_UNPRIVILEGED patch

[1] https://lore.kernel.org/linux-fsdevel/[e-mail]/
[2] https://lore.kernel.org/linux-fsdevel/[e-mail]/
[3] https://github.com/amir73il/linux/commits/fanotify_name
[4] https://github.com/amir73il/ltp/commits/fanotify_name
[5] https://github.com/amir73il/man-pages/commits/fanotify_name
[6] https://github.com/amir73il/inotify-tools/commits/fanotify_name
[7] Demo run of inotifywatch race free monitor
==============================================
~# ./test_demo.sh /vdf
+ WD=/vdf
+ cd /vdf
+ rm -rf a
+ mkdir -p a/b/c/d/e/f/g/
+ touch a/b/c/0 a/b/c/1 a/b/c/d/e/f/g/0
+ id -u
+ [ 0 = 0 ]
+ MODE=--global
+ EVENTS=-e dir_modify  -e modify -e attrib  -e close_write
+ sleep 1
+ inotifywatch --global -e dir_modify -e modify -e attrib -e
close_write --timeout -2 /vdf
Establishing filesystem global watch...
Finished establishing watches, now collecting statistics.
Sleeping for 2 seconds...
+
+ t=Create files and dirs...
+ touch a/0 a/1 a/2 a/3
+ mkdir a/dir0 a/dir1 a/dir2
+
+ t=Rename files and dirs...
+ mv a/0 a/3
+ mv a/dir0 a/dir3
+
+ t=Delete files and dirs...
+ rm a/1
+ rmdir a/dir1
+
+ t=Modify files and dirs...
+ chmod +x a/b/c/d
+ echo
+
+ t=Move files and dirs...
+ mv a/b/c/1 a/b/c/d/e/f/g/1
+ mv a/b/c/d/e/f/g a/b/c/d/e/G
+
[fid=fd50.0.2007403;name='0'] /vdf/a/0 (deleted)
[fid=fd50.0.2007403;name='1'] /vdf/a/1 (deleted)
[fid=fd50.0.2007403;name='2'] /vdf/a/2
[fid=fd50.0.2007403;name='3'] /vdf/a/3
[fid=fd50.0.2007403;name='dir0'] /vdf/a/dir0 (deleted)
[fid=fd50.0.2007403;name='dir1'] /vdf/a/dir1 (deleted)
[fid=fd50.0.2007403;name='dir2'] /vdf/a/dir2
[fid=fd50.0.2007403;name='dir3'] /vdf/a/dir3
[fid=fd50.0.86;name='d'] /vdf/a/b/c/d
[fid=fd50.0.86;name='0'] /vdf/a/b/c/0
[fid=fd50.0.86;name='1'] /vdf/a/b/c/1 (deleted)
[fid=fd50.0.87;name='1'] /vdf/a/b/c/d/e/G/1
[fid=fd50.0.3000083;name='g'] /vdf/a/b/c/d/e/f/g (deleted)
[fid=fd50.0.2007404;name='G'] /vdf/a/b/c/d/e/G
total  modify  attrib  close_write  dir_modify  filename
3      0       1       1            2           /vdf/a/0 (deleted)
3      0       1       1            2           /vdf/a/1 (deleted)
3      0       1       1            2           /vdf/a/3
2      0       1       1            1           /vdf/a/2
2      0       0       0            2           /vdf/a/dir0 (deleted)
2      0       0       0            2           /vdf/a/dir1 (deleted)
1      0       0       0            1           /vdf/a/dir2
1      0       0       0            1           /vdf/a/dir3
1      0       1       0            0           /vdf/a/b/c/d
1      1       0       1            0           /vdf/a/b/c/0
1      0       0       0            1           /vdf/a/b/c/1 (deleted)
1      0       0       0            1           /vdf/a/b/c/d/e/G/1
1      0       0       0            1           /vdf/a/b/c/d/e/f/g (deleted)
1      0       0       0            1           /vdf/a/b/c/d/e/G
==============================================

Amir Goldstein (16):
  fsnotify: tidy up FS_ and FAN_ constants
  fsnotify: factor helpers fsnotify_dentry() and fsnotify_file()
  fsnotify: funnel all dirent events through fsnotify_name()
  fsnotify: use helpers to access data by data_type
  fsnotify: simplify arguments passing to fsnotify_parent()
  fsnotify: pass dentry instead of inode for events possible on child
  fsnotify: replace inode pointer with tag
  fanotify: merge duplicate events on parent and child
  fanotify: fix merging marks masks with FAN_ONDIR
  fanotify: send FAN_DIR_MODIFY event flavor with dir inode and name
  fanotify: prepare to encode both parent and child fid's
  fanotify: record name info for FAN_DIR_MODIFY event
  fanotify: report name info for FAN_DIR_MODIFY event
  fanotify: report parent fid + name with FAN_REPORT_NAME
  fanotify: refine rules for when name is reported
  fanotify: support limited functionality for unprivileged users

 fs/notify/fanotify/fanotify.c        | 231 +++++++++++++++++++++------
 fs/notify/fanotify/fanotify.h        | 111 ++++++++++---
 fs/notify/fanotify/fanotify_user.c   | 182 +++++++++++++++++----
 fs/notify/fsnotify.c                 |  22 +--
 fs/notify/inotify/inotify_fsnotify.c |  10 +-
 include/linux/fanotify.h             |  21 ++-
 include/linux/fsnotify.h             | 135 +++++++---------
 include/linux/fsnotify_backend.h     |  87 +++++++---
 include/uapi/linux/fanotify.h        |  11 +-
 kernel/audit_fsnotify.c              |  13 +-
 kernel/audit_watch.c                 |  16 +-
 11 files changed, 584 insertions(+), 255 deletions(-)


base-commit: 11a48a5a18c63fd7621bb050228cebf13566e4d8

Comments

Matthew Bobrowski Feb. 20, 2020, 10:10 p.m. UTC | #1

On Mon, Feb 17, 2020 at 03:14:39PM +0200, Amir Goldstein wrote:
> This is v2 of the fanotify name info series.
>
> The user requirement for the name info feature, as well as early UAPI
> discussions can be found in this [1] lore thread.

Oh, wonderful. I'm keen to have this feature come to fruition.

After my wedding, which is this Saturday (tomorrow), I'll come around
to reviewing this series.

/M