From patchwork Thu May 11 18:24:24 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Axel Rasmussen X-Patchwork-Id: 13238354 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 97666C7EE22 for ; Thu, 11 May 2023 18:24:37 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id E75176B0072; Thu, 11 May 2023 14:24:36 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id DFE706B0074; Thu, 11 May 2023 14:24:36 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C78386B0075; Thu, 11 May 2023 14:24:36 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id B35FA6B0072 for ; Thu, 11 May 2023 14:24:36 -0400 (EDT) Received: from smtpin16.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 56001AE86E for ; Thu, 11 May 2023 18:24:36 +0000 (UTC) X-FDA: 80778799752.16.DD2DDDB Received: from mail-yb1-f201.google.com (mail-yb1-f201.google.com [209.85.219.201]) by imf29.hostedemail.com (Postfix) with ESMTP id 8489212000E for ; Thu, 11 May 2023 18:24:34 +0000 (UTC) Authentication-Results: imf29.hostedemail.com; dkim=pass header.d=google.com header.s=20221208 header.b=zpyNZKJd; spf=pass (imf29.hostedemail.com: domain of 34TJdZA0KCOQGdKRXGYSaYYKTMUUMRK.IUSROTad-SSQbGIQ.UXM@flex--axelrasmussen.bounces.google.com designates 209.85.219.201 as permitted sender) smtp.mailfrom=34TJdZA0KCOQGdKRXGYSaYYKTMUUMRK.IUSROTad-SSQbGIQ.UXM@flex--axelrasmussen.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1683829474; a=rsa-sha256; cv=none; b=Bbe+k+WJWtTEXU3tgF+L9kyppe87/vM+jffct8U/ZSotubuomSKqDAKbeOxzWo05ZLZWHr vh2BkCoFVocWbaGqCB0fOW5i3MwcwOPJZbotxwE3WCqmvhPJqTarCS29XnWGjV5nwy5HTl aSXMqbSpBS4uS5DdySPLQByNTis4TdQ= ARC-Authentication-Results: i=1; imf29.hostedemail.com; dkim=pass header.d=google.com header.s=20221208 header.b=zpyNZKJd; spf=pass (imf29.hostedemail.com: domain of 34TJdZA0KCOQGdKRXGYSaYYKTMUUMRK.IUSROTad-SSQbGIQ.UXM@flex--axelrasmussen.bounces.google.com designates 209.85.219.201 as permitted sender) smtp.mailfrom=34TJdZA0KCOQGdKRXGYSaYYKTMUUMRK.IUSROTad-SSQbGIQ.UXM@flex--axelrasmussen.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1683829474; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding:in-reply-to: references:dkim-signature; bh=pi6QoZ5SCLGgP2sxC+MkaEgbY1VXhaAz9TEP3dwrDPs=; b=JBfMN1lva1Md5JzD0hjX9sIJe3oWJsX4TOmukC+7dDZ5GO3IsHe4qYuVxFu9Ozp1F4yvzR Sxm9MIzs+PN6tTHgSdrk7BJNHpRpLz3HZV5EiWWARz/crdPaperD6Mb9XhEcbF67aPFjM/ J/8VMx2wdxog3CVHF2fZX7uvOddyh6k= Received: by mail-yb1-f201.google.com with SMTP id 3f1490d57ef6-b9a7e65b34aso16105916276.0 for ; Thu, 11 May 2023 11:24:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1683829473; x=1686421473; h=cc:to:from:subject:message-id:mime-version:date:from:to:cc:subject :date:message-id:reply-to; bh=pi6QoZ5SCLGgP2sxC+MkaEgbY1VXhaAz9TEP3dwrDPs=; b=zpyNZKJdhGrBM4FCCCFdLJNUeHu7VB5x1zBza86yuRYT//8gc05k3M+A1XAjXeHYQU XsEPOjhIWX936abY0RAnUz756Co2yEtSnUBul6v6sCxjIPkPJgNCvPmNFhIN62HuVOE1 irkrkZ4ocjIKKQeR0IfXiH2yEHFBjv0I6mZrVd3b7aEQFXXsYHEuLzZoJ1h6qn/7NkBG Kjw+bEIhpmgecGP8AQgNQNhH64/X0SBEYMkbt0iVzYsNyZ87+y4B+sEecTBCV4ftXZHx CwAYGbE8i4Tg39vPlkbL8N9Ogy15Z6Fr4rHTurnmFTOuV0J1qAkQm721UxKaMxjwzdwc hf5Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1683829473; x=1686421473; h=cc:to:from:subject:message-id:mime-version:date:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=pi6QoZ5SCLGgP2sxC+MkaEgbY1VXhaAz9TEP3dwrDPs=; b=XQFh0eioz7BW7W9gHt0jo+KvkHt0MsphhgnyGOFn0oPQWwwoT5udlmcwENDEHLdTrz NSsB/RkCampfXNK1Qco/K4e2SN7o3Oytp5G5cJW45meOfHsWwvdNM9hLdWv1tZ733gcK HSQ8NimvPNUReNnmv3FXBWHOYlLYpIE8cPB9cqA6ngBQyUN73QKRVVKunMZiX9ufNl0H Z9T43y2uibT9aU55xR0D+YAsN9TdOQqhFZXPK88AptMM+HPsxFY1FukUVQ1pzVNw6i2H q3p7TndqupmgYty6uu5hBUJKYBydSy0jI8bTSbfxeal1VEIb3kW+o01abSgGkt8quSzV 436Q== X-Gm-Message-State: AC+VfDwvpm/1ftp8mHltEVQb1By90U6Z968Vccu7XvV3fmffzBucqc76 0LxRwbnh2NjFwXRFZxbjBAtgPqOHjm37WHwK4DQY X-Google-Smtp-Source: ACHHUZ7u4hIr1abfAjSfObToFMnjaZ5a7i67wz1z8K+AHbBfqmL+8z0DeI0fgjzjTRoAf5zNOgmTizL/c3RRg3dEL+oU X-Received: from axel.svl.corp.google.com ([2620:15c:2d4:203:1119:8675:ddb3:1e7a]) (user=axelrasmussen job=sendgmr) by 2002:a05:6902:114e:b0:b4a:3896:bc17 with SMTP id p14-20020a056902114e00b00b4a3896bc17mr9915594ybu.0.1683829473641; Thu, 11 May 2023 11:24:33 -0700 (PDT) Date: Thu, 11 May 2023 11:24:24 -0700 Mime-Version: 1.0 X-Mailer: git-send-email 2.40.1.606.ga4b1b128d6-goog Message-ID: <20230511182426.1898675-1-axelrasmussen@google.com> Subject: [PATCH 1/3] mm: userfaultfd: add new UFFDIO_SIGBUS ioctl From: Axel Rasmussen To: Alexander Viro , Andrew Morton , Christian Brauner , David Hildenbrand , Hongchen Zhang , Huang Ying , James Houghton , "Liam R. Howlett" , Miaohe Lin , "Mike Rapoport (IBM)" , Nadav Amit , Naoya Horiguchi , Peter Xu , Shuah Khan , ZhangPeng Cc: Axel Rasmussen , linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-kselftest@vger.kernel.org X-Rspam-User: X-Rspamd-Server: rspam02 X-Rspamd-Queue-Id: 8489212000E X-Stat-Signature: 1dathz53sffbur7pdh16tosg5de1d7fu X-HE-Tag: 1683829474-159546 X-HE-Meta: U2FsdGVkX1/Xz8Ep4BqAwDKb7sb4/F3VhPe1RjgTrk8DBtZqQSsIk23cKZJ+5W36ctE+EwJ9RyXikFROHUkg4q/XfwTeEACsHi/6qd8Y+Yf9UH+m7fLhCsO5ls5r+w9yrSP5IxTGp9KkF6JsvLv7t64q+IpZ1sklgl0BfaYGqz7aAHWLBSfJXTN7i4BBmMbPMyFNfHpzq5KUV3ctwAm9u9f7LFaaHacHAPrV9ZNuPBTdS0WfSF070GpMrU3EJFggM3xeRWKUtRE1azMIU6ov24gQHT4JAJa2ijQi6hrpZQaFhAiurk2wDiuX1eKzahKiqu7tteKOWaob1+4+aIY2HyMZNU4bfY4aFj5hXIOtBhOqtlym016zuurefUw8HKL3id940uWVdH5uembzdUyG/yCBm8P6sbHKr3s7EIzgQfwL0ubgPtQf1+sZR4NwhO3vnygjqp98HpU5zCxvFTsQRp3TgD6XX2vo2bSWwZE/7j9lEmasnrNP5VYFY53nMun7rivNz8W0ZTSxfCO4682o2jsGrsSj3J7TSgmNVQI/j/a8IWbIjKhZCRlGFdCOTnjJEXy8zTNpEdJFAltg9X03ugwvq1fB7r8aT3hEWW6kQ0KpDzADUs24ZCg1nHr4SHS2iaDVwSTkcPpWl3/GKiaqJ1Jmiz7KRYv1V7BfxUTsrVB6lS+K9ADhbDJ69GeBh9C0aRGN1owWSOI9ne2PEl+uV8CbXENF10vKKhi1Et4tsZWfkULCqZwbLav5/vQDXduLJ25i2zgF1+lSnDMqruxdY7mJSv/3INhQwDqeLMRGJj48p+NEwEplGbIEmaF5cCNRLGt4QIxmkXKhEYRO5iZF/QwhT0K1ejN4bnlBY86BuUli8gtHioL0T85ZNgTmDP5g1hu0mFYSJAObzZ1VO2BbLl8kqKOyJ1E/l+ayFg8Am73L30oSSrgb3O03Ic6LxGoI8Rxoc94HavfN+nZawqE jt2Rbim1 /QCTnifGTynpTAgogoG/ULi2P3YDwy7xr6f57PmioT4zUg1JSG7mN1DlDiZDn1ilRRhYVzdGNjJeRoeZ4En0NmSpJjWdfCERlaB2cBw5CQikBY3R6w/PECk+zhrnzH7j+Yeqonx6GdLeu6f36brFys5OQI7HMpb/Ne2Uvkt8lLelR/6wfu0ATSwy1nBH66iDPAHiHjoqhPIC2YOvX8EkfaiF7OX7W4PZbOUsOd9EKXrob/Kg2DhkIgp4fCiCddDXZ/1AIm7Oa+tWE6v+BwJof5A1Y+vL/G6p/3Bc+kXuigA/cLnUSoe/gvGoWxBLudH+/4Xzr6RCeW+yn+jNOt6WU/qdxmuPBhVcVLNM0EMhxzdJLO0Q2JGmTYn48KUCsOTF2Eir5SnsdmzYrUSpi8Tdyiq7f1ZqgyvZz1010ElViCOGzoMBa8bWJdcHSikaQS5gU3OaHGcwoQa2WbMDGlIJ0N7WHXPeS8IDYr083G17A5+RCk5gJQr0KyesZysLUOdg0PyNLtsoS1WpXlpdjou7kzgF0oVH9QSeOM/kNwN/daL3X8aKljKbIPr0U+qvozvnCZZYKXzEDAJYE4/JF29C83tc/iOngf+Q2dePpnXMuTPjtav1LoSIZWWL53HxvnyVh/MZatbAjvHyFt/Ht/LRXRsUvIyQppRtHY3VgMpO7lf9LZosnWHpG+ycztiJoyVKQ91OT8VmesJFwiBZ6bo7rhn1d4Q== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: The basic idea here is to "simulate" memory poisoning for VMs. A VM running on some host might encounter a memory error, after which some page(s) are poisoned (i.e., future accesses SIGBUS). They expect that once poisoned, pages can never become "un-poisoned". So, when we live migrate the VM, we need to preserve the poisoned status of these pages. When live migrating, we try to get the guest running on its new host as quickly as possible. So, we start it running before all memory has been copied, and before we're certain which pages should be poisoned or not. So the basic way to use this new feature is: - On the new host, the guest's memory is registered with userfaultfd, in either MISSING or MINOR mode (doesn't really matter for this purpose). - On any first access, we get a userfaultfd event. At this point we can communicate with the old host to find out if the page was poisoned. - If so, we can respond with a UFFDIO_SIGBUS - this places a swap marker so any future accesses will SIGBUS. Because the pte is now "present", future accesses won't generate more userfaultfd events, they'll just SIGBUS directly. UFFDIO_SIGBUS does not handle unmapping previously-present PTEs. This isn't needed, because during live migration we want to intercept all accesses with userfaultfd (not just writes, so WP mode isn't useful for this). So whether minor or missing mode is being used (or both), the PTE won't be present in any case, so handling that case isn't needed. Signed-off-by: Axel Rasmussen --- fs/userfaultfd.c | 63 ++++++++++++++++++++++++++++++++ include/linux/swapops.h | 3 +- include/linux/userfaultfd_k.h | 4 ++ include/uapi/linux/userfaultfd.h | 25 +++++++++++-- mm/memory.c | 4 ++ mm/userfaultfd.c | 62 ++++++++++++++++++++++++++++++- 6 files changed, 156 insertions(+), 5 deletions(-) diff --git a/fs/userfaultfd.c b/fs/userfaultfd.c index 0fd96d6e39ce..edc2928dae2b 100644 --- a/fs/userfaultfd.c +++ b/fs/userfaultfd.c @@ -1966,6 +1966,66 @@ static int userfaultfd_continue(struct userfaultfd_ctx *ctx, unsigned long arg) return ret; } +static inline int userfaultfd_sigbus(struct userfaultfd_ctx *ctx, unsigned long arg) +{ + __s64 ret; + struct uffdio_sigbus uffdio_sigbus; + struct uffdio_sigbus __user *user_uffdio_sigbus; + struct userfaultfd_wake_range range; + + user_uffdio_sigbus = (struct uffdio_sigbus __user *)arg; + + ret = -EAGAIN; + if (atomic_read(&ctx->mmap_changing)) + goto out; + + ret = -EFAULT; + if (copy_from_user(&uffdio_sigbus, user_uffdio_sigbus, + /* don't copy the output fields */ + sizeof(uffdio_sigbus) - (sizeof(__s64)))) + goto out; + + ret = validate_range(ctx->mm, uffdio_sigbus.range.start, + uffdio_sigbus.range.len); + if (ret) + goto out; + + ret = -EINVAL; + /* double check for wraparound just in case. */ + if (uffdio_sigbus.range.start + uffdio_sigbus.range.len <= + uffdio_sigbus.range.start) { + goto out; + } + if (uffdio_sigbus.mode & ~UFFDIO_SIGBUS_MODE_DONTWAKE) + goto out; + + if (mmget_not_zero(ctx->mm)) { + ret = mfill_atomic_sigbus(ctx->mm, uffdio_sigbus.range.start, + uffdio_sigbus.range.len, + &ctx->mmap_changing, 0); + mmput(ctx->mm); + } else { + return -ESRCH; + } + + if (unlikely(put_user(ret, &user_uffdio_sigbus->updated))) + return -EFAULT; + if (ret < 0) + goto out; + + /* len == 0 would wake all */ + BUG_ON(!ret); + range.len = ret; + if (!(uffdio_sigbus.mode & UFFDIO_SIGBUS_MODE_DONTWAKE)) { + range.start = uffdio_sigbus.range.start; + wake_userfault(ctx, &range); + } + ret = range.len == uffdio_sigbus.range.len ? 0 : -EAGAIN; + +out: + return ret; +} + static inline unsigned int uffd_ctx_features(__u64 user_features) { /* @@ -2067,6 +2127,9 @@ static long userfaultfd_ioctl(struct file *file, unsigned cmd, case UFFDIO_CONTINUE: ret = userfaultfd_continue(ctx, arg); break; + case UFFDIO_SIGBUS: + ret = userfaultfd_sigbus(ctx, arg); + break; } return ret; } diff --git a/include/linux/swapops.h b/include/linux/swapops.h index 3a451b7afcb3..fa778a0ae730 100644 --- a/include/linux/swapops.h +++ b/include/linux/swapops.h @@ -405,7 +405,8 @@ typedef unsigned long pte_marker; #define PTE_MARKER_UFFD_WP BIT(0) #define PTE_MARKER_SWAPIN_ERROR BIT(1) -#define PTE_MARKER_MASK (BIT(2) - 1) +#define PTE_MARKER_UFFD_SIGBUS BIT(2) +#define PTE_MARKER_MASK (BIT(3) - 1) static inline swp_entry_t make_pte_marker_entry(pte_marker marker) { diff --git a/include/linux/userfaultfd_k.h b/include/linux/userfaultfd_k.h index d78b01524349..6de1084939c5 100644 --- a/include/linux/userfaultfd_k.h +++ b/include/linux/userfaultfd_k.h @@ -46,6 +46,7 @@ enum mfill_atomic_mode { MFILL_ATOMIC_COPY, MFILL_ATOMIC_ZEROPAGE, MFILL_ATOMIC_CONTINUE, + MFILL_ATOMIC_SIGBUS, NR_MFILL_ATOMIC_MODES, }; @@ -83,6 +84,9 @@ extern ssize_t mfill_atomic_zeropage(struct mm_struct *dst_mm, extern ssize_t mfill_atomic_continue(struct mm_struct *dst_mm, unsigned long dst_start, unsigned long len, atomic_t *mmap_changing, uffd_flags_t flags); +extern ssize_t mfill_atomic_sigbus(struct mm_struct *dst_mm, unsigned long start, + unsigned long len, atomic_t *mmap_changing, + uffd_flags_t flags); extern int mwriteprotect_range(struct mm_struct *dst_mm, unsigned long start, unsigned long len, bool enable_wp, atomic_t *mmap_changing); diff --git a/include/uapi/linux/userfaultfd.h b/include/uapi/linux/userfaultfd.h index 66dd4cd277bd..616e33d3db97 100644 --- a/include/uapi/linux/userfaultfd.h +++ b/include/uapi/linux/userfaultfd.h @@ -39,7 +39,8 @@ UFFD_FEATURE_MINOR_SHMEM | \ UFFD_FEATURE_EXACT_ADDRESS | \ UFFD_FEATURE_WP_HUGETLBFS_SHMEM | \ - UFFD_FEATURE_WP_UNPOPULATED) + UFFD_FEATURE_WP_UNPOPULATED | \ + UFFD_FEATURE_SIGBUS_IOCTL) #define UFFD_API_IOCTLS \ ((__u64)1 << _UFFDIO_REGISTER | \ (__u64)1 << _UFFDIO_UNREGISTER | \ @@ -49,12 +50,14 @@ (__u64)1 << _UFFDIO_COPY | \ (__u64)1 << _UFFDIO_ZEROPAGE | \ (__u64)1 << _UFFDIO_WRITEPROTECT | \ - (__u64)1 << _UFFDIO_CONTINUE) + (__u64)1 << _UFFDIO_CONTINUE | \ + (__u64)1 << _UFFDIO_SIGBUS) #define UFFD_API_RANGE_IOCTLS_BASIC \ ((__u64)1 << _UFFDIO_WAKE | \ (__u64)1 << _UFFDIO_COPY | \ + (__u64)1 << _UFFDIO_WRITEPROTECT | \ (__u64)1 << _UFFDIO_CONTINUE | \ - (__u64)1 << _UFFDIO_WRITEPROTECT) + (__u64)1 << _UFFDIO_SIGBUS) /* * Valid ioctl command number range with this API is from 0x00 to @@ -71,6 +74,7 @@ #define _UFFDIO_ZEROPAGE (0x04) #define _UFFDIO_WRITEPROTECT (0x06) #define _UFFDIO_CONTINUE (0x07) +#define _UFFDIO_SIGBUS (0x08) #define _UFFDIO_API (0x3F) /* userfaultfd ioctl ids */ @@ -91,6 +95,8 @@ struct uffdio_writeprotect) #define UFFDIO_CONTINUE _IOWR(UFFDIO, _UFFDIO_CONTINUE, \ struct uffdio_continue) +#define UFFDIO_SIGBUS _IOWR(UFFDIO, _UFFDIO_SIGBUS, \ + struct uffdio_sigbus) /* read() structure */ struct uffd_msg { @@ -225,6 +231,7 @@ struct uffdio_api { #define UFFD_FEATURE_EXACT_ADDRESS (1<<11) #define UFFD_FEATURE_WP_HUGETLBFS_SHMEM (1<<12) #define UFFD_FEATURE_WP_UNPOPULATED (1<<13) +#define UFFD_FEATURE_SIGBUS_IOCTL (1<<14) __u64 features; __u64 ioctls; @@ -321,6 +328,18 @@ struct uffdio_continue { __s64 mapped; }; +struct uffdio_sigbus { + struct uffdio_range range; +#define UFFDIO_SIGBUS_MODE_DONTWAKE ((__u64)1<<0) + __u64 mode; + + /* + * Fields below here are written by the ioctl and must be at the end: + * the copy_from_user will not read past here. + */ + __s64 updated; +}; + /* * Flags for the userfaultfd(2) system call itself. */ diff --git a/mm/memory.c b/mm/memory.c index f69fbc251198..e4b4207c2590 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -3675,6 +3675,10 @@ static vm_fault_t handle_pte_marker(struct vm_fault *vmf) if (WARN_ON_ONCE(!marker)) return VM_FAULT_SIGBUS; + /* SIGBUS explicitly requested for this PTE. */ + if (marker & PTE_MARKER_UFFD_SIGBUS) + return VM_FAULT_SIGBUS; + /* Higher priority than uffd-wp when data corrupted */ if (marker & PTE_MARKER_SWAPIN_ERROR) return VM_FAULT_SIGBUS; diff --git a/mm/userfaultfd.c b/mm/userfaultfd.c index e97a0b4889fc..933587eebd5d 100644 --- a/mm/userfaultfd.c +++ b/mm/userfaultfd.c @@ -278,6 +278,51 @@ static int mfill_atomic_pte_continue(pmd_t *dst_pmd, goto out; } +/* Handles UFFDIO_SIGBUS for all non-hugetlb VMAs. */ +static int mfill_atomic_pte_sigbus(pmd_t *dst_pmd, + struct vm_area_struct *dst_vma, + unsigned long dst_addr, + uffd_flags_t flags) +{ + int ret; + struct mm_struct *dst_mm = dst_vma->vm_mm; + pte_t _dst_pte, *dst_pte; + spinlock_t *ptl; + + _dst_pte = make_pte_marker(PTE_MARKER_UFFD_SIGBUS); + dst_pte = pte_offset_map_lock(dst_mm, dst_pmd, dst_addr, &ptl); + + if (vma_is_shmem(dst_vma)) { + struct inode *inode; + pgoff_t offset, max_off; + + /* serialize against truncate with the page table lock */ + inode = dst_vma->vm_file->f_inode; + offset = linear_page_index(dst_vma, dst_addr); + max_off = DIV_ROUND_UP(i_size_read(inode), PAGE_SIZE); + ret = -EFAULT; + if (unlikely(offset >= max_off)) + goto out_unlock; + } + + ret = -EEXIST; + /* + * For now, we don't handle unmapping pages, so only support filling in + * none PTEs, or replacing PTE markers. + */ + if (!pte_none_mostly(*dst_pte)) + goto out_unlock; + + set_pte_at(dst_mm, dst_addr, dst_pte, _dst_pte); + + /* No need to invalidate - it was non-present before */ + update_mmu_cache(dst_vma, dst_addr, dst_pte); + ret = 0; +out_unlock: + pte_unmap_unlock(dst_pte, ptl); + return ret; +} + static pmd_t *mm_alloc_pmd(struct mm_struct *mm, unsigned long address) { pgd_t *pgd; @@ -328,8 +373,12 @@ static __always_inline ssize_t mfill_atomic_hugetlb( * supported by hugetlb. A PMD_SIZE huge pages may exist as used * by THP. Since we can not reliably insert a zero page, this * feature is not supported. + * + * PTE marker handling for hugetlb is a bit special, so for now + * UFFDIO_SIGBUS is not supported. */ - if (uffd_flags_mode_is(flags, MFILL_ATOMIC_ZEROPAGE)) { + if (uffd_flags_mode_is(flags, MFILL_ATOMIC_ZEROPAGE) || + uffd_flags_mode_is(flags, MFILL_ATOMIC_SIGBUS)) { mmap_read_unlock(dst_mm); return -EINVAL; } @@ -473,6 +522,9 @@ static __always_inline ssize_t mfill_atomic_pte(pmd_t *dst_pmd, if (uffd_flags_mode_is(flags, MFILL_ATOMIC_CONTINUE)) { return mfill_atomic_pte_continue(dst_pmd, dst_vma, dst_addr, flags); + } else if (uffd_flags_mode_is(flags, MFILL_ATOMIC_SIGBUS)) { + return mfill_atomic_pte_sigbus(dst_pmd, dst_vma, + dst_addr, flags); } /* @@ -694,6 +746,14 @@ ssize_t mfill_atomic_continue(struct mm_struct *dst_mm, unsigned long start, uffd_flags_set_mode(flags, MFILL_ATOMIC_CONTINUE)); } +ssize_t mfill_atomic_sigbus(struct mm_struct *dst_mm, unsigned long start, + unsigned long len, atomic_t *mmap_changing, + uffd_flags_t flags) +{ + return mfill_atomic(dst_mm, start, 0, len, mmap_changing, + uffd_flags_set_mode(flags, MFILL_ATOMIC_SIGBUS)); +} + long uffd_wp_range(struct vm_area_struct *dst_vma, unsigned long start, unsigned long len, bool enable_wp) { From patchwork Thu May 11 18:24:25 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Axel Rasmussen X-Patchwork-Id: 13238355 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6529CC77B7F for ; Thu, 11 May 2023 18:24:39 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id C52A36B0074; Thu, 11 May 2023 14:24:38 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id BDC2C6B0075; Thu, 11 May 2023 14:24:38 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 9E04E6B0078; Thu, 11 May 2023 14:24:38 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 8E1706B0074 for ; Thu, 11 May 2023 14:24:38 -0400 (EDT) Received: from smtpin02.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 51E3C1C7DBF for ; Thu, 11 May 2023 18:24:38 +0000 (UTC) X-FDA: 80778799836.02.C2A9449 Received: from mail-yb1-f202.google.com (mail-yb1-f202.google.com [209.85.219.202]) by imf13.hostedemail.com (Postfix) with ESMTP id 52AC420006 for ; Thu, 11 May 2023 18:24:36 +0000 (UTC) Authentication-Results: imf13.hostedemail.com; dkim=pass header.d=google.com header.s=20221208 header.b=55rn+NwT; spf=pass (imf13.hostedemail.com: domain of 34zJdZA0KCOYIfMTZIaUcaaMVOWWOTM.KWUTQVcf-UUSdIKS.WZO@flex--axelrasmussen.bounces.google.com designates 209.85.219.202 as permitted sender) smtp.mailfrom=34zJdZA0KCOYIfMTZIaUcaaMVOWWOTM.KWUTQVcf-UUSdIKS.WZO@flex--axelrasmussen.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1683829476; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=LjUrEJJ1UhaYRGAtGgSCY4PNxG4jgAwHbuTYYPklZWY=; b=74WItFJrP5GX7cSvhz5bHBwPM5hao7Q0SFREAFdeUZISmBnfo3uSyS77GFF5w7WEGUhCQs eUxBgBVWzGkCBTSoZZfXG4D/ZyVaqCKSKalS/MwtMz8AzvTYmBStiePMm2uLTqGjzVJtZk zZu7TDM4cYL4JoOx9T78YRg7YJzEo5Q= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1683829476; a=rsa-sha256; cv=none; b=SBTm87AT78FW7s5YiW0gTsl34u5uewwU/HXw8iofErBg3UDHALpE7/T8Nksu/4hw2OW1D8 rnTrS0YS9ub0CSeLAVjuxKO163oEoxqrFwyJVBb+RyXmkSmWhwR1+N3I7p3UdwA35nmzYT OeLJbEqxX1hVrDh6+vEc0NAUGSTUbhI= ARC-Authentication-Results: i=1; imf13.hostedemail.com; dkim=pass header.d=google.com header.s=20221208 header.b=55rn+NwT; spf=pass (imf13.hostedemail.com: domain of 34zJdZA0KCOYIfMTZIaUcaaMVOWWOTM.KWUTQVcf-UUSdIKS.WZO@flex--axelrasmussen.bounces.google.com designates 209.85.219.202 as permitted sender) smtp.mailfrom=34zJdZA0KCOYIfMTZIaUcaaMVOWWOTM.KWUTQVcf-UUSdIKS.WZO@flex--axelrasmussen.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com Received: by mail-yb1-f202.google.com with SMTP id 3f1490d57ef6-ba2b9ecfadaso10144911276.2 for ; Thu, 11 May 2023 11:24:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1683829475; x=1686421475; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=LjUrEJJ1UhaYRGAtGgSCY4PNxG4jgAwHbuTYYPklZWY=; b=55rn+NwTS9qkN6o3EHQH9tg5nvgfFAoVfIi3Ps/Vx2UwN/K9+YFrkS/lHydkVA1cxR iowaGUww07beE3ntNfoSK93Fb1ceSg9jZe+0OTW9gkOExIsspD5MajhZSLmPMFNhTmMt gYuZl1WAugwIYddYiwJQaO7nq+GtN5jEQblGC1v623/h5p5sdc5Fgzt9DS2LFIrWogJ1 uW449MkphLCiIDXFZtwUGTOaENHChuPdfQqfcJ0QFEkS+4bIncJToGHvvADNkuPgQdkG R3TQXk6+FSc/Ziwfvc5dNOcdLIdSb14/ToPdMXENIjkpjZO8fNcFVpbUtLbpkbE1w/zq 6kkQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1683829475; x=1686421475; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=LjUrEJJ1UhaYRGAtGgSCY4PNxG4jgAwHbuTYYPklZWY=; b=HHPeeEK/KEDqT60l4lUs79MrZ95RJ5DHVfugHkLXxcavXMeUYi8QsGKStjY73n9h3X tsrp9RL9/8J1FgyrHlj0hVLM9X3GhoF3B5hc/dHd+9W/WzAVP7+B+bE+jET7bs8tuZiR STKj7/yGT6zOIw7sHxPEhVIiShJG0lNQ1ZkFwHhnrVw9ffz4LQMO0BGBwTzzKyljkVvL c5C2KVIAapQNKp2o+REy6vgp00IBXddGnCgt4/OUKywVSpic87FDyhoczXZXhTk5HAug RJ8QLVMrtF6fnbLTA1VBp4iJS1yJmWiShflM4qUoOzWMke56gXT7cwxA5IjkZ3FN16Kc LOYA== X-Gm-Message-State: AC+VfDw1R3mWXfytvloh4z4py48vB3LAxe8M+etUu4eBdYalq/ePa7mK wrJ4SpWc/xYZzv3yNIm38+YWJCgz0U+lma/QceXz X-Google-Smtp-Source: ACHHUZ5tH1DV1GALQ1Ob1t3RTbPs1BmNd7Keg/peKTDa9epscPLL3nUgdVKOqLkDNSlWMLhG0C8f2l/D/e6LqknAyPsd X-Received: from axel.svl.corp.google.com ([2620:15c:2d4:203:1119:8675:ddb3:1e7a]) (user=axelrasmussen job=sendgmr) by 2002:a05:6902:154f:b0:b78:8bd8:6e77 with SMTP id r15-20020a056902154f00b00b788bd86e77mr14316791ybu.8.1683829475459; Thu, 11 May 2023 11:24:35 -0700 (PDT) Date: Thu, 11 May 2023 11:24:25 -0700 In-Reply-To: <20230511182426.1898675-1-axelrasmussen@google.com> Mime-Version: 1.0 References: <20230511182426.1898675-1-axelrasmussen@google.com> X-Mailer: git-send-email 2.40.1.606.ga4b1b128d6-goog Message-ID: <20230511182426.1898675-2-axelrasmussen@google.com> Subject: [PATCH 2/3] selftests/mm: refactor uffd_poll_thread to allow custom fault handlers From: Axel Rasmussen To: Alexander Viro , Andrew Morton , Christian Brauner , David Hildenbrand , Hongchen Zhang , Huang Ying , James Houghton , "Liam R. Howlett" , Miaohe Lin , "Mike Rapoport (IBM)" , Nadav Amit , Naoya Horiguchi , Peter Xu , Shuah Khan , ZhangPeng Cc: Axel Rasmussen , linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-kselftest@vger.kernel.org X-Stat-Signature: 8d8hm1kepoheff1pmh9cfbhw44ez56uk X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 52AC420006 X-Rspam-User: X-HE-Tag: 1683829476-773733 X-HE-Meta: U2FsdGVkX19QtW4aVhcUN6G8APKZIdSI101q1i7x4qbb5D/hV2NEIVDkjk/MnW7Vf08nhAn/C1KRGGtn7ABH82oPtVutklxeLVJ0Fezl1LzzNdHQoJGrrQ6e9Q63+xjAvZk4VGAcSmZr6WuTwwwqS7NC2jtdp8zin8jS7i3xAYatW4D2H3/Bp3zH87nmF8lFUCND71r/HWFZoH2NteabUDJUFELEOTCGOchwdDrnuc13l0mpYU1ztNWICSfMMeam+QGbMVfaFYYbh6y+IYbPXaz8HOrhSmlZ7eORgNE7uaWrIxcTuzG9bOfNHFXHjcDbZw+L5+clCmUn4tZft1zZobNezrwYQy+Qv4i2EC8l54BS+Q4wVEouWwc1T1mDjTc20nN5ZdqZsfuN4vaxwtA3tFxneAdJd9x3jyEfbwYekHkD1In9iKDNqLnFztdh7gsvVsjqTS09f16FoG2p2FHc0AEBnci9HPiWmupsjkGiFNo/oU42jGgmJaon7Ns2qIRPLQnn7n8kilhhKCTlH77smmwhXheQL2uJ2IP25E+Rq+v90p1kU15yNBfRb+KGj0GffdBc92zlqW2l3HzYN+1gGcdien5lAAqruFxEVehRj8wy1NzZsxJc92zmPetcO68fdgeSfqiP15DsKqhnrP+9R3cg5c14CLaxx6Ct7S9i/UX4cz4HT1+ZL+4/W4OBvygCqFoXp/VFSooLr57SUqDj8/NI0DWoATDg93zHpRym2mN8+5Al8+dS9Rx4BDzU7QXFu5EPjbLjP7sP8be5gft8FksCULgBi5j0xeaqinlKEE/ukrdqfMATSPvApwvHuaEmHcc80LyjkGJ1UdmhrCY+jQqnxa/LDiCXAXfzvJvmseMnOYvamrDTjoCmAoMsYpyIlfouemRVF02q/lEPCO9sCzVxJgWpte+BwRVpKtZk3Fi2W1s9iyopaUTibZi15bHCzQj3Qa1RUlP5D+a3iw/ lyDRL0eO +mm81LegZh4ZkFvkmKrjZcCQS41dmNgs6ze/+5aZ9sitV8QXqy6rr+rfqYS9L9zOwjVVCa8vqcYYM3Ypb9jux4Sr7plsMgILEVmsMJrH/9UzdmBlWjnotAtX0BHIy8Eij9r2A/pEduRXDLMBsuHwvPcCKUivs93AkG7TyjKGqJCm/vDDavBTcjULImVAJglz81oHubd1JUXb3cu4vDmGd934cTajfn3Pr+jGUiW7qtlSs163FAD/iZuHZCgUBKWpLpvnoAFn7p9q4vKNsRgJ/pNj/+thsDy82YVDZ1K0agE3WW2fnEOwIBmb+bScwk82YLrezmcK52PO3qrtQOVJiSp04yNHKK+vDHlMhvlmELPfJ0m/V7qeZpJYh7ohrEh4MF1zJeqi80MuWaNgOYt3A7pK6omHyOzbRwN/0BgQd9HvKA4W+uiNzxLUwhiKnZEA0i4cO8qawM9DijtHc03PPbFFCWqF34lsaRrk3v6+L6yEEd3tQFrVOu+eTFBONyUFEvSYU4KCUS+snv7G20VIXnknveMc6IdwShi2Dtftgyu0R6zIUtVZKdj/EU0fE2GreNtBsTW2Wh5hIUonzt6fqpy39q2sfSYlMYFiVayo2K+hKlGUqH9QgWuEUC3A7+6sr5YZ+qDiiCTNhgOTJLJy9Kcmaavl7EUUwvaGe+FPRLtAwK4itUU3eZd6XVBIRRxwSnXc4uEfIkUAqoKs= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Previously, we had "one fault handler to rule them all", which used several branches to deal with all of the scenarios required by all of the various tests. In upcoming patches, I plan to add a new test, which has its own slightly different fault handling logic. Instead of continuing to add cruft to the existing fault handler, let's allow tests to define custom ones, separate from other tests. Signed-off-by: Axel Rasmussen --- tools/testing/selftests/mm/uffd-common.c | 5 ++++- tools/testing/selftests/mm/uffd-common.h | 3 +++ tools/testing/selftests/mm/uffd-stress.c | 6 ++---- 3 files changed, 9 insertions(+), 5 deletions(-) diff --git a/tools/testing/selftests/mm/uffd-common.c b/tools/testing/selftests/mm/uffd-common.c index 61c6250adf93..c9756dbffe7d 100644 --- a/tools/testing/selftests/mm/uffd-common.c +++ b/tools/testing/selftests/mm/uffd-common.c @@ -499,6 +499,9 @@ void *uffd_poll_thread(void *arg) int ret; char tmp_chr; + if (!args->handle_fault) + args->handle_fault = uffd_handle_page_fault; + pollfd[0].fd = uffd; pollfd[0].events = POLLIN; pollfd[1].fd = pipefd[cpu*2]; @@ -527,7 +530,7 @@ void *uffd_poll_thread(void *arg) err("unexpected msg event %u\n", msg.event); break; case UFFD_EVENT_PAGEFAULT: - uffd_handle_page_fault(&msg, args); + args->handle_fault(&msg, args); break; case UFFD_EVENT_FORK: close(uffd); diff --git a/tools/testing/selftests/mm/uffd-common.h b/tools/testing/selftests/mm/uffd-common.h index 6068f2346b86..b28d88b9937e 100644 --- a/tools/testing/selftests/mm/uffd-common.h +++ b/tools/testing/selftests/mm/uffd-common.h @@ -77,6 +77,9 @@ struct uffd_args { unsigned long missing_faults; unsigned long wp_faults; unsigned long minor_faults; + + /* A custom fault handler; defaults to uffd_handle_page_fault. */ + void (*handle_fault)(struct uffd_msg *msg, struct uffd_args *args); }; struct uffd_test_ops { diff --git a/tools/testing/selftests/mm/uffd-stress.c b/tools/testing/selftests/mm/uffd-stress.c index f1ad9eef1c3a..47e1464935a8 100644 --- a/tools/testing/selftests/mm/uffd-stress.c +++ b/tools/testing/selftests/mm/uffd-stress.c @@ -199,10 +199,8 @@ static int stress(struct uffd_args *args) locking_thread, (void *)cpu)) return 1; if (bounces & BOUNCE_POLL) { - if (pthread_create(&uffd_threads[cpu], &attr, - uffd_poll_thread, - (void *)&args[cpu])) - return 1; + if (pthread_create(&uffd_threads[cpu], &attr, uffd_poll_thread, &args[cpu])) + err("uffd_poll_thread create"); } else { if (pthread_create(&uffd_threads[cpu], &attr, uffd_read_thread, From patchwork Thu May 11 18:24:26 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Axel Rasmussen X-Patchwork-Id: 13238356 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2DFB7C7EE2D for ; Thu, 11 May 2023 18:24:41 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 85C216B0075; Thu, 11 May 2023 14:24:40 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 76F486B0078; Thu, 11 May 2023 14:24:40 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 59CB56B007B; Thu, 11 May 2023 14:24:40 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 431B76B0075 for ; Thu, 11 May 2023 14:24:40 -0400 (EDT) Received: from smtpin15.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id DAF271A0E24 for ; Thu, 11 May 2023 18:24:39 +0000 (UTC) X-FDA: 80778799878.15.92B0576 Received: from mail-yb1-f201.google.com (mail-yb1-f201.google.com [209.85.219.201]) by imf21.hostedemail.com (Postfix) with ESMTP id 1F3561C0006 for ; Thu, 11 May 2023 18:24:37 +0000 (UTC) Authentication-Results: imf21.hostedemail.com; dkim=pass header.d=google.com header.s=20221208 header.b=uljCWBeU; spf=pass (imf21.hostedemail.com: domain of 35TJdZA0KCOgKhOVbKcWeccOXQYYQVO.MYWVSXeh-WWUfKMU.YbQ@flex--axelrasmussen.bounces.google.com designates 209.85.219.201 as permitted sender) smtp.mailfrom=35TJdZA0KCOgKhOVbKcWeccOXQYYQVO.MYWVSXeh-WWUfKMU.YbQ@flex--axelrasmussen.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1683829478; a=rsa-sha256; cv=none; b=iSrwEBE8rNwcSBihUmp2cCDU+JAupwlITLQbaozF3ts4KC8lQ39TgqR1yIXzzuu2zV3Cy8 NANDacPkgWKsKYapNzUHz01x+/jivYnPD7YHZhCJ3h85S8BI/uHtftC02ig7lwXAwakSGZ +jbeJK8vFHb4v8zMLWRMS3Zb3oVyXnY= ARC-Authentication-Results: i=1; imf21.hostedemail.com; dkim=pass header.d=google.com header.s=20221208 header.b=uljCWBeU; spf=pass (imf21.hostedemail.com: domain of 35TJdZA0KCOgKhOVbKcWeccOXQYYQVO.MYWVSXeh-WWUfKMU.YbQ@flex--axelrasmussen.bounces.google.com designates 209.85.219.201 as permitted sender) smtp.mailfrom=35TJdZA0KCOgKhOVbKcWeccOXQYYQVO.MYWVSXeh-WWUfKMU.YbQ@flex--axelrasmussen.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1683829478; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=pXR2gVK9OA+wwjaEDZ+OSGVT3sLd4z3OrzMBakd4aKc=; b=J5Ig3pMHupBdQzUJDQ16Zo0IbxwT1K9wzhBedl9Jm5WS+72vV4tp1keU7Q7ix9orLnjc0s svEZDUBs/66oTVXmAw7DIrU505KW35fYp5nZULpXf5JAT5c8tlEy79eBA2Q5kxadgarxA9 gx5dFDfcjLlje1Sv1tqW+HMTQTdTmCc= Received: by mail-yb1-f201.google.com with SMTP id 3f1490d57ef6-ba6f530c9c7so186617276.3 for ; Thu, 11 May 2023 11:24:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1683829477; x=1686421477; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=pXR2gVK9OA+wwjaEDZ+OSGVT3sLd4z3OrzMBakd4aKc=; b=uljCWBeUfmKO+5Of2CDGdnripKh0c8Rzx0Uw+3mUJQMB0vkRyYLVzF+n17pnlhk/Gx 5bJy3V1jOgdbwhD+IzTsmSPyo+npOvf5g9Bg01qvgjSMD6eTNJAt18pXpgDdOP/1R0pG f7W9dTJmQptgBrJNb6rZO42eQOqXUPeL3PmdMoDvsc0jIKFazxlaZQy6iFcTAKrcyb2g Vd4Xof1tpuhpdbe6fUAhZyOPuYJwlVtphMWQs8q/tSYp7h7umSFg6RazMw4joPD08IQq U+PE0b8W/rQAZMz7pnNUhXYfuSp6KyH2zv92pJQFwOvAqZixv97KJ+4OltRNsi03yCWr atQw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1683829477; x=1686421477; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=pXR2gVK9OA+wwjaEDZ+OSGVT3sLd4z3OrzMBakd4aKc=; b=g6lyBdCe4qapuMv+Mmy+1J8LY3jc1hSA3HavHJDdALAWWvBXXptzkGD3lG2SKZfAbw Zfr40TZEiA4YRWhKwsAuZUcp6sh2VMobIJGu9f90N4+hQWoRF3aYM0UQ6w0eFPaSuY6J 5lj6Ihf2Kha2FxV8yCFdooknZmZH0zZQXRdDW7at88igR6IdGFTKMLNEjFtYMm84JQ+/ Nzv9LIunCa1U5Ph1YDq/H9RPL3uluYOxL2m9Zhrf3R3Ka0zkciOyQwB7lAQjQMAYf5hy OwqrUx4oq9HFw7owcN1+Fj6dJvDKWJGcAaKm4eTO/UTcVxkr709ZlLFNnNr8DFJjApSw q8qQ== X-Gm-Message-State: AC+VfDyVj5LxeEQd7FBzm/MI9WmEHk0MknNVodwcudHXA3HNucs6QtSa ZtafHVyRviWEPCvy8Sw39hdIDoasEtlVduFsjKtR X-Google-Smtp-Source: ACHHUZ6/k8WRFVr95FNBEjmhvpwsN42OkGFLi7clqHCCS0gOIKkyVoZZe1vIA3bGUQMYYdRD+oFyrl83UA1YRSgN8vJE X-Received: from axel.svl.corp.google.com ([2620:15c:2d4:203:1119:8675:ddb3:1e7a]) (user=axelrasmussen job=sendgmr) by 2002:a05:6902:114e:b0:b4a:3896:bc17 with SMTP id p14-20020a056902114e00b00b4a3896bc17mr9915667ybu.0.1683829477159; Thu, 11 May 2023 11:24:37 -0700 (PDT) Date: Thu, 11 May 2023 11:24:26 -0700 In-Reply-To: <20230511182426.1898675-1-axelrasmussen@google.com> Mime-Version: 1.0 References: <20230511182426.1898675-1-axelrasmussen@google.com> X-Mailer: git-send-email 2.40.1.606.ga4b1b128d6-goog Message-ID: <20230511182426.1898675-3-axelrasmussen@google.com> Subject: [PATCH 3/3] selftests/mm: add uffd unit test for UFFDIO_SIGBUS From: Axel Rasmussen To: Alexander Viro , Andrew Morton , Christian Brauner , David Hildenbrand , Hongchen Zhang , Huang Ying , James Houghton , "Liam R. Howlett" , Miaohe Lin , "Mike Rapoport (IBM)" , Nadav Amit , Naoya Horiguchi , Peter Xu , Shuah Khan , ZhangPeng Cc: Axel Rasmussen , linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-kselftest@vger.kernel.org X-Rspam-User: X-Rspamd-Server: rspam02 X-Rspamd-Queue-Id: 1F3561C0006 X-Stat-Signature: n48o6h4ccii7r6dz88qe8z4nhr7qkhh7 X-HE-Tag: 1683829477-909629 X-HE-Meta: U2FsdGVkX1/XkxXgfuwXX5ShMS1jWur2ET5fci8wZs9bdblY+K9n9+7F/f7GzouOEdGc+lIGM3Mh632l/dhqR1uHDHOTAB11RzJwkXL/IO05BJ740mRbQRKu4+JxeQ8DD/YHKMkI3FR2ymCSZSv9SQSzj4tIgNTCgA12XJZf49vWWdM63tU0ifbAv/tsbhi0zWhPBEYLnUTGvTXNGlQf+HhJOKS+7nENdDbsyPC89TTvVe/LyWGmYHjyGIWqW6R21T9ivGrIDtFlrbc3SMPuoTiYTlINFBlRykRTtdRMBm/QCyMWLcXvfO2Bw6UWceJNudvN7uyFlqldhZpz7O/GKMwViiwYqe4yFPZ8t4agwiKDvo/UZxyS0ME1ng/EBdgbUmntTOkrVWSTfKkSyXmTYfgGodjcz8Mecch0bM1gJJh4BFARABaPEheVuC2kmpQLNswU73NNwLM866SVSRAFsvM4QYixRuEDIuwr7vE2vP3zin07qhXsAvD4u18vrC4Fu+7v5gZJ6geRTN3ILXPTaN8i0H0jCzZP31gYlFT+XGBeVr0FufskUvs0NfJYcUZ9hmbGMkMY2rzMQ6bwJiUX96kZNYgXHwf6VtKTDFTHlCtCa/c5n2jHJxdUvmxP30UIv+GOgfWji5YF9Pp+8cSNhZeonAyw+5tKN+OmdW6WUKv8f9n1UeX3DpGs4Hz3p4B35cncUByirzlwJDseUXBCX5A9/2hClDSKQwUSLitxiXA/5LauSxCNgI786jy2Wr1BeIEpowE5EH7xGNVagjADFsdB+cHYDudeNg0wN5BtA2tb0/HEYVpGwIE4KBRV1EFxRxrcHSCPTcpiPTlNkW4e2h0VvJ4ZD4ax4vOUx4wNJGRKou6Mn5Kgln/r0HNitkHB7K0cVhlO4ifYHS7oo3MIaj+WFsoR3A5LjHmr74WN48AwkhgU7xtTUrWBfOsrKypEw292IQtLHgnLz5sHZzQ +1GtVHWv YWBRZPct6SUwJm5P7laHc9tV+XKAAQHk67H4GmgQP23JRvjrjznfZKhErEr4r/eG+ncA6Dl2vs/i6USdLcrj4G/Hu0x+wqd00rM1ZoUIgDxFbhcGNUXsL/XOGuYUfuQvCvRsMBQbP8guvMfFiCxOOD3ufTqPlqrweQ+BN0In7ioDsm1FTBbPiCG6Y5o9+BIGDw4R7Gd6Ls1BOGAGOWc2DGZgXCeGt1eohgrOofrHDqngyrp3QTqd7G6jAEJaQPjx6UVwA6KNl+OxThZy/VwLajODClXwYmZjf7e0IFtMCJMIzrEydlKejFaei0x6zltFFCeiEuT/CklUzsKivXdQ63vS3dvQ81BcPKwB8Cu+n9scSjzYvcmQWD9CU3tQOPNiLju1VY8JTLk7F3tdZjwwvYDClp41137hCqUfyM/+rGTAqGN0Inarpq4R8eSCdk7Z656KaDHSBnaKax17okbc7FP6KkTf5KrgFOTXPsJvOEjVtPwUqWBvkIlYExxl6suU5TI4AcmDrzMIwM8OWI6ofErwTrnGLRjurdzAVluh7zHsIOu4Y/Ak+X52F0lyG3zE3Xt81Q2U1LxX+9gyIkFN70SG/qowg5BrMhSufgVkmpOmFI1XFyY7UCnlzHBRGBLlaudWtRQ8P4+9ZVs69iqQBvcV3nbKKcQyp5Jxnx0ZMqnE7ezHwlIxlmlO/EO70LGuXND4lw+pyi1Vis40= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: The test is pretty basic, and exercises UFFDIO_SIGBUS straightforwardly. We register a region with userfaultfd, in missing fault mode. For each fault, we either issue UFFDIO_ZEROPAGE (odd pages) or UFFDIO_SIGBUS (even pages). We read each page in the region, and assert that the odd pages are zeroed as expected, and the even pages yield a SIGBUS as expected. Signed-off-by: Axel Rasmussen --- tools/testing/selftests/mm/uffd-unit-tests.c | 114 ++++++++++++++++++- 1 file changed, 110 insertions(+), 4 deletions(-) diff --git a/tools/testing/selftests/mm/uffd-unit-tests.c b/tools/testing/selftests/mm/uffd-unit-tests.c index 269c86768a02..3eb5a6f9b51f 100644 --- a/tools/testing/selftests/mm/uffd-unit-tests.c +++ b/tools/testing/selftests/mm/uffd-unit-tests.c @@ -881,13 +881,13 @@ static void retry_uffdio_zeropage(int ufd, } } -static bool do_uffdio_zeropage(int ufd, bool has_zeropage) +static bool do_uffdio_zeropage(int ufd, bool has_zeropage, bool test_retry, unsigned long offset) { struct uffdio_zeropage uffdio_zeropage = { 0 }; int ret; __s64 res; - uffdio_zeropage.range.start = (unsigned long) area_dst; + uffdio_zeropage.range.start = (unsigned long) area_dst + offset; uffdio_zeropage.range.len = page_size; uffdio_zeropage.mode = 0; ret = ioctl(ufd, UFFDIO_ZEROPAGE, &uffdio_zeropage); @@ -901,7 +901,7 @@ static bool do_uffdio_zeropage(int ufd, bool has_zeropage) } else if (has_zeropage) { if (res != page_size) err("UFFDIO_ZEROPAGE unexpected size"); - else + else if (test_retry) retry_uffdio_zeropage(ufd, &uffdio_zeropage); return true; } else @@ -938,7 +938,7 @@ static void uffd_zeropage_test(uffd_test_args_t *args) /* Ignore the retval; we already have it */ uffd_register_detect_zeropage(uffd, area_dst_alias, page_size); - if (do_uffdio_zeropage(uffd, has_zeropage)) + if (do_uffdio_zeropage(uffd, has_zeropage, true, 0)) for (i = 0; i < page_size; i++) if (area_dst[i] != 0) err("data non-zero at offset %d\n", i); @@ -952,6 +952,106 @@ static void uffd_zeropage_test(uffd_test_args_t *args) uffd_test_pass(); } +static void do_uffdio_sigbus(int uffd, unsigned long offset) +{ + struct uffdio_sigbus uffdio_sigbus = { 0 }; + int ret; + __s64 res; + + uffdio_sigbus.range.start = (unsigned long) area_dst + offset; + uffdio_sigbus.range.len = page_size; + uffdio_sigbus.mode = 0; + ret = ioctl(uffd, UFFDIO_SIGBUS, &uffdio_sigbus); + res = uffdio_sigbus.updated; + + if (ret) + err("UFFDIO_SIGBUS error: %"PRId64, (int64_t)res); + else if (res != page_size) + err("UFFDIO_SIGBUS unexpected size: %"PRId64, (int64_t)res); +} + +static void uffd_sigbus_ioctl_handle_fault( + struct uffd_msg *msg, struct uffd_args *args) +{ + unsigned long offset; + + if (msg->event != UFFD_EVENT_PAGEFAULT) + err("unexpected msg event %u", msg->event); + + if (msg->arg.pagefault.flags & + (UFFD_PAGEFAULT_FLAG_WP | UFFD_PAGEFAULT_FLAG_MINOR)) + err("unexpected fault type %llu", msg->arg.pagefault.flags); + + offset = (char *)(unsigned long)msg->arg.pagefault.address - area_dst; + offset &= ~(page_size-1); + + /* Odd pages -> zeropage; even pages -> sigbus. */ + if (offset & page_size) { + if (!do_uffdio_zeropage(uffd, true, false, offset)) + err("UFFDIO_ZEROPAGE failed"); + } else { + do_uffdio_sigbus(uffd, offset); + } +} + +static void uffd_sigbus_ioctl_test(uffd_test_args_t *targs) +{ + pthread_t uffd_mon; + char c; + struct uffd_args args = { 0 }; + struct sigaction act = { 0 }; + unsigned long nr_sigbus = 0; + unsigned long nr; + + fcntl(uffd, F_SETFL, uffd_flags | O_NONBLOCK); + + if (!uffd_register_detect_zeropage(uffd, area_dst, nr_pages * page_size)) + err("register failed: no zeropage support"); + + args.handle_fault = uffd_sigbus_ioctl_handle_fault; + if (pthread_create(&uffd_mon, NULL, uffd_poll_thread, &args)) + err("uffd_poll_thread create"); + + sigbuf = &jbuf; + act.sa_sigaction = sighndl; + act.sa_flags = SA_SIGINFO; + if (sigaction(SIGBUS, &act, 0)) + err("sigaction"); + + for (nr = 0; nr < nr_pages; ++nr) { + unsigned long offset = nr * page_size; + const char *bytes = (const char *) area_dst + offset; + const char *i; + + if (sigsetjmp(*sigbuf, 1)) { + /* + * Access below triggered a SIGBUS, which was caught by + * sighndl, which then jumped here. Count this SIGBUS, + * and move on to next page. + */ + ++nr_sigbus; + continue; + } + + for (i = bytes; i < bytes + page_size; ++i) { + if (*i) + err("nonzero byte in area_dst (%p) at %p: %u", + area_dst, i, *i); + } + } + + if (write(pipefd[1], &c, sizeof(c)) != sizeof(c)) + err("pipe write"); + if (pthread_join(uffd_mon, NULL)) + err("pthread_join()"); + + if (nr_sigbus != nr_pages / 2) + err("expected to receive %lu SIGBUS, actually received %lu", + nr_pages / 2, nr_sigbus); + + uffd_test_pass(); +} + /* * Test the returned uffdio_register.ioctls with different register modes. * Note that _UFFDIO_ZEROPAGE is tested separately in the zeropage test. @@ -1127,6 +1227,12 @@ uffd_test_case_t uffd_tests[] = { UFFD_FEATURE_PAGEFAULT_FLAG_WP | UFFD_FEATURE_WP_HUGETLBFS_SHMEM, }, + { + .name = "sigbus-ioctl", + .uffd_fn = uffd_sigbus_ioctl_test, + .mem_targets = MEM_ALL & ~(MEM_HUGETLB | MEM_HUGETLB_PRIVATE), + .uffd_feature_required = UFFD_FEATURE_SIGBUS_IOCTL, + }, }; static void usage(const char *prog)