From patchwork Sun Nov 25 13:43:39 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Amir Goldstein X-Patchwork-Id: 10696823 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 43E975A4 for ; Sun, 25 Nov 2018 13:44:15 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 288042916A for ; Sun, 25 Nov 2018 13:44:15 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 19BFA291E8; Sun, 25 Nov 2018 13:44:15 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id ED5A52916A for ; Sun, 25 Nov 2018 13:44:13 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726165AbeKZAfS (ORCPT ); Sun, 25 Nov 2018 19:35:18 -0500 Received: from mail-wm1-f67.google.com ([209.85.128.67]:55425 "EHLO mail-wm1-f67.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726098AbeKZAfS (ORCPT ); Sun, 25 Nov 2018 19:35:18 -0500 Received: by mail-wm1-f67.google.com with SMTP id y139so15549134wmc.5; Sun, 25 Nov 2018 05:44:10 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id; bh=IigtccnG94E0DZy1YKjDp6vu8lxN3YkyVmFbQwsXRcE=; b=iRtO8C6dRus0JowKleTXIH1u68S4X7nM3pwFtHGKnjQ05nM+1LWpygyrpFfG7pH0xa kfJ6n1P7ZljHqCZwGiJnUIIqV8Zu3DAo63qZR/y3sE3ypmvVx/yFl8yCYh66tDFCdVAy xCvjoxhwDQkh1ckCzGzpCf5WfAQ/eF4FhphFELsa4vyLVWJDe/zfY7YHN3CuNECwB6mX BKf7iUNjoydEyS6B6vcHym9FHh7V++ux6UkyZNhkfERDsVcs36QC4W0AoxslUXBwwlL4 gzppaYsi7YDz9SM/zO4zL42c78mReo6saNX2VCfXNEfFCW57K+ttCNtRHm08hHWCD2/n ZG0A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id; bh=IigtccnG94E0DZy1YKjDp6vu8lxN3YkyVmFbQwsXRcE=; b=V2in1y2y4mTiG4ISKdmhD5bMOyeg8kH/DSq+pFXZJm3gBptRUnAmrXdi6u9TSJcms0 tY8ApuHiEYpPLT6A26UEwnWF65OT3h4YrWHFob8cJ1ovnAbCN0o6Oot4fiXq2cPCp0fo B1Acbr9gYUXUuku6T6HSLqv6HZLxe7LKX8xpTrRzpBM8YrCvkEBOgChCI2nHVdWD/orw ZasZS2ArOTz8qi9+nTn6EjWCNnTd1Ehxr2jTNZ24JaORgE8OXm4o38xWFhBDhxMFxiya 2DEm/1eItMr1c1+61L4OV6aDK11kDp6f8yHCz+sepXDGwpaYPpNARJd6B9h3dZClhPnG IrFg== X-Gm-Message-State: AA+aEWZe4M00fTQ+TqN/hWutf9Pe5CKq1LyGJN+EpVMo+FcX6Rq0Q7XE mifVExI8vHiJQdpnh+QkbJ4BTYsP X-Google-Smtp-Source: AJdET5cmbRQa0grepu355dvStohmfEuUR+Ja9UdTMTHsGxFzOLF5SWL/khymaDOPHuWid83JQuxaIw== X-Received: by 2002:a1c:c919:: with SMTP id f25mr21088030wmb.75.1543153449467; Sun, 25 Nov 2018 05:44:09 -0800 (PST) Received: from localhost.localdomain (bzq-166-168-31-246.red.bezeqint.net. [31.168.166.246]) by smtp.gmail.com with ESMTPSA id 125sm17354274wml.35.2018.11.25.05.44.08 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sun, 25 Nov 2018 05:44:08 -0800 (PST) From: Amir Goldstein To: Jan Kara Cc: Matthew Bobrowski , linux-fsdevel@vger.kernel.org, linux-api@vger.kernel.org Subject: [PATCH v3 00/13] fanotify: add support for more event types Date: Sun, 25 Nov 2018 15:43:39 +0200 Message-Id: <20181125134352.21499-1-amir73il@gmail.com> X-Mailer: git-send-email 2.17.1 Sender: linux-fsdevel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Jan, This is the 3rd revision of patch series to add support for filesystem change monitoring to fanotify. It incorporates the changes you requested in review of v2 patches. The complete work is available on fanotify_dirent branch [1] on my tree. The end game is to use: fd = fanotify_init(FAN_CLASS_NOTIF|FAN_REPORT_FID, ...); rc = fanotify_mark(fd, FAN_MARK_FILESYSTEM, FAN_CREATE|FAN_DELETE...); to monitor changes to a large scale namespace. This functionality was not available with inotify API, which does not scale well with recursive directory watches and was not available with fanotify API, which did not support directory modification events. I have tested this work with some preliminary LTP tests [2] and with a prototype of global filesystem monitor based on inotify-tools [3]. Please see below a demo output [4] from filesystem monitor prototype. Note that the "watches" in the prototype are userland entries to map fid to path. The kernel has but one mark per super block. Matthew Bobrowski has agreed to help me with writing more tests and man pages (thanks Matthew!). Thanks, Amir. Changes since v2: - Discard FSNOTIFY_EVENT_DENTRY data type changes - Cache fsid in connector instead of calling vfs_statfs() on every event - Deny setting fid watch on filesystem with no fsid (tmpfs) - Deny setting fid watch on filesystem with non root fsid (btrfs subvol) - Report FAN_ONDIR for all event types with FAN_REPORT_FID [1] https://github.com/amir73il/linux/commits/fanotify_dirent [2] https://github.com/amir73il/ltp/commits/fanotify_dirent [3] https://github.com/amir73il/inotify-tools/commits/fanotify_dirent [4] Demo run of inotifywait monitor on 2 filesystems and 1 subvolume: ================= ... /dev/vdf on /vdf type xfs (rw,attr2,inode64,noquota) /dev/vde on /mnt type btrfs (rw,space_cache,subvolid=257,subvol=/subvol) /dev/vde on /vde type btrfs (rw,space_cache,subvolid=5,subvol=/) root@kvm-xfstests:~# inotifywait -m -g /vde /vdf Setting up global filesystem watches. Watches established. /vde/ OPEN,ISDIR /vdf/ OPEN,ISDIR root@kvm-xfstests:~# mkdir -p /mnt/a/b/c/d/e/ && touch /mnt/a/b/c/d/e/x ...Start watching /vde/subvol (fid=f74e7a26.a635b2c5.100...) /vde/subvol CREATE,ISDIR ...Start watching /vde/subvol/a (fid=f74e7a26.a635b2c5.101...) /vde/subvol/a CLOSE_NOWRITE,OPEN,CREATE,CLOSE,ISDIR /vde/subvol CLOSE_NOWRITE,OPEN,CLOSE,ISDIR ...Start watching /vde/subvol/a/b (fid=f74e7a26.a635b2c5.102...) /vde/subvol/a/b CLOSE_NOWRITE,OPEN,CLOSE,ISDIR /vde/subvol/a CLOSE_NOWRITE,OPEN,CLOSE,ISDIR /vde/subvol/a/b CLOSE_NOWRITE,OPEN,CLOSE,ISDIR /vde/subvol/a/b CREATE,ISDIR ...Start watching /vde/subvol/a/b/c (fid=f74e7a26.a635b2c5.103...) /vde/subvol/a/b/c CLOSE_NOWRITE,OPEN,CLOSE,ISDIR /vde/subvol/a/b/c CLOSE_NOWRITE,OPEN,CLOSE,ISDIR /vde/subvol/a/b/c CREATE,ISDIR ...Start watching /vde/subvol/a/b/c/d (fid=f74e7a26.a635b2c5.104...) /vde/subvol/a/b/c/d CLOSE_NOWRITE,OPEN,CREATE,CLOSE,ISDIR /vde/subvol/a/b/c/d CLOSE_NOWRITE,OPEN,CLOSE,ISDIR ...Start watching /vde/subvol/a/b/c/d/e (fid=f74e7a26.a635b2c5.105...) /vde/subvol/a/b/c/d/e CREATE ...Start watching /vde/subvol/a/b/c/d/e/x (fid=f74e7a26.a635b2c5.106...) /vde/subvol/a/b/c/d/e/x OPEN /vde/subvol/a/b/c/d/e CLOSE_NOWRITE,OPEN,CLOSE,ISDIR /vde/subvol/a/b/c/d/e/x CLOSE_NOWRITE,OPEN,CLOSE /vde/subvol/a/b/c/d/e/x ATTRIB /vde/subvol/a/b/c/d/e/x CLOSE_WRITE,CLOSE root@kvm-xfstests:~# touch /vde/a /vdf/a /mnt/a /vde/ CREATE ...Start watching /vde/a (fid=f74e7a26.a635b2c5.101...) /vde/a OPEN /vde/a ATTRIB /vde/a CLOSE_NOWRITE,OPEN,CLOSE /vde/a CLOSE_WRITE,CLOSE /vdf/ CREATE ...Start watching /vdf/a (fid=fd50.0.105...) /vdf/a OPEN /vdf/a CLOSE_NOWRITE,OPEN,CLOSE /vdf/a ATTRIB,CLOSE_WRITE,CLOSE /vde/subvol/a ATTRIB,ISDIR root@kvm-xfstests:~# rm -rf /vde/a /vdf/a /mnt/a /vde/a ATTRIB,DELETE_SELF /vde/ DELETE /vdf/a ATTRIB,DELETE_SELF /vdf/ DELETE /vde/subvol/a OPEN,ISDIR /vde/subvol/a ACCESS,ISDIR /vde/subvol/a CLOSE_NOWRITE,CLOSE,ISDIR /vde/subvol/a OPEN,ISDIR /vde/subvol/a ACCESS,ISDIR /vde/subvol/a ACCESS,ISDIR /vde/subvol/a/b OPEN,ISDIR /vde/subvol/a/b ACCESS,ISDIR /vde/subvol/a/b CLOSE_NOWRITE,CLOSE,ISDIR /vde/subvol/a/b OPEN,ISDIR /vde/subvol/a/b ACCESS,ISDIR /vde/subvol/a/b ACCESS,ISDIR /vde/subvol/a/b/c ACCESS,OPEN,ISDIR /vde/subvol/a/b/c CLOSE_NOWRITE,CLOSE,ISDIR /vde/subvol/a/b/c OPEN,ISDIR /vde/subvol/a/b/c ACCESS,ISDIR /vde/subvol/a/b/c/d OPEN,ISDIR /vde/subvol/a/b/c/d ACCESS,ISDIR /vde/subvol/a/b/c/d CLOSE_NOWRITE,CLOSE,ISDIR /vde/subvol/a/b/c/d OPEN,ISDIR /vde/subvol/a/b/c/d ACCESS,ISDIR /vde/subvol/a/b/c/d ACCESS,ISDIR /vde/subvol/a/b/c/d/e OPEN,ISDIR /vde/subvol/a/b/c/d/e ACCESS,ISDIR /vde/subvol/a/b/c/d/e CLOSE_NOWRITE,CLOSE,ISDIR /vde/subvol/a/b/c/d/e OPEN,ISDIR /vde/subvol/a/b/c/d/e ACCESS,ISDIR /vde/subvol/a/b/c/d/e ACCESS,ISDIR /vde/subvol/a/b/c/d/e/x ATTRIB,DELETE_SELF /vde/subvol/a/b/c/d/e DELETE /vde/subvol/a/b/c/d/e CLOSE_NOWRITE,CLOSE,ISDIR /vde/subvol/a/b/c/d/e DELETE_SELF /vde/subvol/a/b/c/d DELETE,ISDIR /vde/subvol/a/b/c/d CLOSE_NOWRITE,CLOSE,ISDIR /vde/subvol/a/b/c/d DELETE_SELF /vde/subvol/a/b/c DELETE,ISDIR /vde/subvol/a/b/c CLOSE_NOWRITE,CLOSE,ISDIR /vde/subvol/a/b/c DELETE_SELF /vde/subvol/a/b DELETE,ISDIR /vde/subvol/a/b CLOSE_NOWRITE,CLOSE,ISDIR /vde/subvol/a/b DELETE_SELF /vde/subvol/a DELETE,ISDIR /vde/subvol/a CLOSE_NOWRITE,CLOSE,ISDIR /vde/subvol/a DELETE_SELF /vde/subvol DELETE,ISDIR ===================== Amir Goldstein (13): fsnotify: annotate directory entry modification events fsnotify: send all event types to super block marks fanotify: rename struct fanotify_{,perm_}event_info fanotify: define the structures to report a unique file identifier fanotify: classify events that hold a file identifier fanotify: encode file identifier for FAN_REPORT_FID fanotify: copy event fid info to user fanotify: enable FAN_REPORT_FID init flag fanotify: cache fsid in fsnotify_mark_connector fanotify: check FS_ISDIR flag instead of d_is_dir() fanotify: support events with data type FSNOTIFY_EVENT_INODE fanotify: add support for create/attrib/move/delete events fanotify: report FAN_ONDIR to listener with FAN_REPORT_FID fs/notify/fanotify/fanotify.c | 214 ++++++++++++++++++++++++----- fs/notify/fanotify/fanotify.h | 77 +++++++++-- fs/notify/fanotify/fanotify_user.c | 213 +++++++++++++++++++++++----- fs/notify/fsnotify.c | 15 +- fs/notify/mark.c | 77 +++++++++-- fs/statfs.c | 4 +- include/linux/fanotify.h | 33 ++++- include/linux/fsnotify.h | 46 +++++-- include/linux/fsnotify_backend.h | 60 +++++--- include/linux/statfs.h | 3 + include/uapi/linux/fanotify.h | 46 ++++++- 11 files changed, 642 insertions(+), 146 deletions(-) --- 2.17.1