From patchwork Tue Jan 25 03:37:00 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Peter Xu X-Patchwork-Id: 12723253 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id E1DB0C4332F for ; Tue, 25 Jan 2022 03:37:19 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 551FC6B0083; Mon, 24 Jan 2022 22:37:19 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 500E76B0085; Mon, 24 Jan 2022 22:37:19 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 3A1D36B0087; Mon, 24 Jan 2022 22:37:19 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0040.hostedemail.com [216.40.44.40]) by kanga.kvack.org (Postfix) with ESMTP id 2E0AF6B0083 for ; Mon, 24 Jan 2022 22:37:19 -0500 (EST) Received: from smtpin27.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id ED39892746 for ; Tue, 25 Jan 2022 03:37:18 +0000 (UTC) X-FDA: 79067398956.27.36AB808 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by imf07.hostedemail.com (Postfix) with ESMTP id 5D4BF4001A for ; Tue, 25 Jan 2022 03:37:18 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1643081837; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=ajFuIcB4jTNx/UM4V4dBkhnn/5IgHZMqOqwixvXnOT0=; b=LP/iWZci7ZYc+pn0+L7Ta9IteIXL0hO5EPy41mhftQ9cA/Y2U84kNpEvKiKErqSaurmPUw hi6RYMPLfOiEtR/4ko+SsZAsgy6PToBKjGIQ9Ok3BmP8FOtSdPtYJZQLgqj0JjkP0bP6wJ atLRpIqOj+3GDmzGLZZxN9GEEIz3dKw= Received: from mail-pj1-f69.google.com (mail-pj1-f69.google.com [209.85.216.69]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-633-DhCN0ZqbNwaUVLvM6r-Upw-1; Mon, 24 Jan 2022 22:37:14 -0500 X-MC-Unique: DhCN0ZqbNwaUVLvM6r-Upw-1 Received: by mail-pj1-f69.google.com with SMTP id 19-20020a17090a001300b001b480b09680so11612297pja.2 for ; Mon, 24 Jan 2022 19:37:13 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=ajFuIcB4jTNx/UM4V4dBkhnn/5IgHZMqOqwixvXnOT0=; b=fbnXLxCXRvdUne+0+TLp5OsLhjYzwUHTPihE5E+OB147+o0VFlbWNBk73iTcL8uGhj oo45sDe8LtCmgE3SQMiJaYzXn0Gx3lipZH6dOBNpxpivGvZdpjwzXMS7JwMa1CwoZL/K AEVKUH94MP6z0nTkm6Xm0VBakOa9T2dyKIzpT+B6L80YcgOfO9nN8aoLr6yvURp6eN8Z Z3zkiMx3y1cpWT2GZmqUANhbCA6Hg9dlBH407qhRIeMMsPEMSb8Uhyc8iDqoLln/c/an CZPV8Hf3nsxk4/ahZo2ycOwjuanF7jX69Erkh910gmEZskm3P+KSzip+pAececs6TfhQ 1Nrw== X-Gm-Message-State: AOAM533xfcxm4UvlETbwkY1RSrEzJBy/kEt6IaUM8YQjbxiDk8YIAEPc 1nWtNt8IrG9g02S22mxaIbudee3O3hb0uC0M6qK6CAvm5l8nm8HeiuqL/XnIwyxcWbkHrgXS/Xp BFzDiXtjTSgTVSmkRZv0myk7vYZ+YyTQAqF8FTi+yoSfodsK6luKD2kydmm1j X-Received: by 2002:a17:90b:4a85:: with SMTP id lp5mr856690pjb.25.1643081829871; Mon, 24 Jan 2022 19:37:09 -0800 (PST) X-Google-Smtp-Source: ABdhPJyYgG+4F8lTQQQJjbb6KtDmoIF/2sWEyisHmjmUUYY7rAedb9yb1SPGbOPrnSIEEfMK7MNtgA== X-Received: by 2002:a17:90b:4a85:: with SMTP id lp5mr856662pjb.25.1643081829493; Mon, 24 Jan 2022 19:37:09 -0800 (PST) Received: from localhost.localdomain ([94.177.118.73]) by smtp.gmail.com with ESMTPSA id oj7sm718783pjb.43.2022.01.24.19.37.05 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Mon, 24 Jan 2022 19:37:09 -0800 (PST) From: Peter Xu To: linux-mm@kvack.org, linux-kernel@vger.kernel.org Cc: peterx@redhat.com, Andrew Morton , Andrea Arcangeli , Jan Kara , =?utf-8?b?SsOpcsO0bWUgR2xpc3Nl?= , John Hubbard , "Kirill A . Shutemov" , Alex Williamson Subject: [PATCH] mm: Fix invalid page pointer returned with FOLL_PIN gups Date: Tue, 25 Jan 2022 11:37:00 +0800 Message-Id: <20220125033700.69705-1-peterx@redhat.com> X-Mailer: git-send-email 2.32.0 MIME-Version: 1.0 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-Rspamd-Server: rspam09 X-Rspamd-Queue-Id: 5D4BF4001A X-Stat-Signature: uqerpdugj45q3xiqc98g7forf9ujz8kc X-Rspam-User: nil Authentication-Results: imf07.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b="LP/iWZci"; spf=none (imf07.hostedemail.com: domain of peterx@redhat.com has no SPF policy when checking 170.10.129.124) smtp.mailfrom=peterx@redhat.com; dmarc=pass (policy=none) header.from=redhat.com X-HE-Tag: 1643081838-484071 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Alex reported invalid page pointer returned with pin_user_pages_remote() from vfio after upstream commit 4b6c33b32296 ("vfio/type1: Prepare for batched pinning with struct vfio_batch"). This problem breaks NVIDIA vfio mdev. It turns out that it's not the fault of the vfio commit; however after vfio switches to a full page buffer to store the page pointers it starts to expose the problem easier. The problem is for VM_PFNMAP vmas we should normally fail with an -EFAULT then vfio will carry on to handle the MMIO regions. However when the bug triggered, follow_page_mask() returned -EEXIST for such a page, which will jump over the current page, leaving that entry in **pages untouched. However the caller is not aware of it, hence the caller will reference the page as usual even if the pointer data can be anything. We had that -EEXIST logic since commit 1027e4436b6a ("mm: make GUP handle pfn mapping unless FOLL_GET is requested") which seems very reasonable. It could be that when we reworked GUP with FOLL_PIN we could have overlooked that special path in commit 3faa52c03f44 ("mm/gup: track FOLL_PIN pages"), even if that commit rightfully touched up follow_devmap_pud() on checking FOLL_PIN when it needs to return an -EEXIST. Since at it, add another WARN_ON_ONCE() at the -EEXIST handling to make sure we mustn't have **pages set when reaching there, because otherwise it means the caller will try to read a garbage right after __get_user_pages() returns. Attaching the Fixes to the FOLL_PIN rework commit, as it happened later than 1027e4436b6a. Cc: Andrea Arcangeli Cc: Andrew Morton Cc: Jan Kara Cc: Jérôme Glisse Cc: John Hubbard Cc: Kirill A. Shutemov Fixes: 3faa52c03f44 ("mm/gup: track FOLL_PIN pages") Reported-by: Alex Williamson Debugged-by: Alex Williamson Signed-off-by: Peter Xu --- mm/gup.c | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/mm/gup.c b/mm/gup.c index f0af462ac1e2..8ebc04058e97 100644 --- a/mm/gup.c +++ b/mm/gup.c @@ -440,7 +440,7 @@ static int follow_pfn_pte(struct vm_area_struct *vma, unsigned long address, pte_t *pte, unsigned int flags) { /* No page to get reference */ - if (flags & FOLL_GET) + if (flags & (FOLL_GET | FOLL_PIN)) return -EFAULT; if (flags & FOLL_TOUCH) { @@ -1181,7 +1181,13 @@ static long __get_user_pages(struct mm_struct *mm, /* * Proper page table entry exists, but no corresponding * struct page. + * + * Warn if we jumped over even with a valid **pages. + * It shouldn't trigger in practise, but when there's + * buggy returns on -EEXIST we'll warn before returning + * an invalid page pointer in the array. */ + WARN_ON_ONCE(pages); goto next_page; } else if (IS_ERR(page)) { ret = PTR_ERR(page);