From patchwork Sat Jan 12 00:36:26 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Blake Caldwell X-Patchwork-Id: 10760815 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 1D24A6C5 for ; Sat, 12 Jan 2019 00:36:57 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 0756329F70 for ; Sat, 12 Jan 2019 00:36:57 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id EBADB29F75; Sat, 12 Jan 2019 00:36:56 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 370F129F70 for ; Sat, 12 Jan 2019 00:36:56 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id C602E8E0003; Fri, 11 Jan 2019 19:36:53 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id B4A8A8E0001; Fri, 11 Jan 2019 19:36:53 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 99FDE8E0003; Fri, 11 Jan 2019 19:36:53 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-io1-f70.google.com (mail-io1-f70.google.com [209.85.166.70]) by kanga.kvack.org (Postfix) with ESMTP id 6A8C78E0001 for ; Fri, 11 Jan 2019 19:36:53 -0500 (EST) Received: by mail-io1-f70.google.com with SMTP id q207so14271461iod.18 for ; Fri, 11 Jan 2019 16:36:53 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references:in-reply-to:references; bh=DQywlzhu2L0dB3ib+YzwL3C0Y3sRu2MbV3QclYfYcJI=; b=ieKqfpzCSw1wxsUVRUJviLXl0iTnVWGix8ptm4jC4waRVRbr8c3Tf1b3moeVLpNj+9 j0v/ZkcfasSuUWaXPc1xY4TXo7C2NumOx5rGe8RKbny4C7iE8tTYGjc1Az+jy7DAbA3L m8dOTfmA5+H6Rl3gDS87wDdFMbAnMgiy9oQK+8TY+eehMC/9g8ljvB2+pLou7D+rI4tr Xqafr4UXfZIL6j8/AeUm3VInXxmzwLSWya72boziq6oC66ocu0a2/0nkMMBAgbtzUjTA J5ukNu0I1USAEdnrKnGjjfiADg3nfatU30mBi6ceatT1NPnUq2Nc+g0mqhYWxQ6NC2Jo ApJg== X-Gm-Message-State: AJcUukfLt4B2kulc+OhBcDcymKJr4jcbEYBGxLk+CQk5fI52HorFeVjR sBZqpY89uWzBlmxdtIDzngpf4kklmEGJNW4w107E7IrZ+EwepX1uJCOANdfc0oed++lO18j2T28 i1KXxL3E+KijLaGn/28CRDEWublgjaoGiOouzz4+oESDgyCDaYnBC1ooqws0j05lK4UTDNfAzHI gnf+2gCs1dlh8EGU8OwmE1dD3ym8vkg+VUueqqNb8TjBj9NGoLNrSKk9EuAQh/m0zdC54HL0Gmi haH7LJBshdf6EW+r/kslLOP50jl1gW4ZtAAuh6ddJGjEPJd5YsFD5Hjlh/zJdL0R9RVrXiF7ebk SmOLO+V8rxH6Lo8mQ9L39Aprciqs5cmOWR0Zuc4JRI6P34LrwYC7JFsyAJcKxnTcAUwx6UBt1mt f X-Received: by 2002:a24:2208:: with SMTP id o8mr2480941ito.23.1547253413153; Fri, 11 Jan 2019 16:36:53 -0800 (PST) X-Received: by 2002:a24:2208:: with SMTP id o8mr2480923ito.23.1547253412453; Fri, 11 Jan 2019 16:36:52 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1547253412; cv=none; d=google.com; s=arc-20160816; b=sb9222+gzID1PlqRzxeh5N7CLiQFi/OctNXujmfJc8SDA8sk+xevzpMyg+gPDATwDj rR/S8CbqoCkdxAka9HZXC+JGAjbN6VIgjCJNOrtLDBgJhKkFE0XfKCKQirQfEsbd1w/x i8T++ntCX7rj9q36KYml6hcKKXosFB3UZGNewtRehh05GIGEP7oOrQhZ3y6ildOAOT5G 8zHoUo6tZ05h1hlFYhzqptT1bVYW6hK2wScUqhPbvDff31VQBLDTVA974iPpl5Mwko8w 4N5B6hdrTniulvGhP5RbkifN3Y/HvFQfojHT+eSRAghFTx0uhsACu1URz6ViaK29Atea au+w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:references:in-reply-to:message-id:date :subject:cc:to:from:dkim-signature; bh=DQywlzhu2L0dB3ib+YzwL3C0Y3sRu2MbV3QclYfYcJI=; b=JiNcGWLXJFVbw7d140p3aiHLWgAx5RfRli7FEyh15NG+mPejRBBQMnOBypkchlDGCh D4iPMzDC0Gpg0tXM9hq6AWn/QdgYO5vwvXnImeAG6AG2nKVmfM4bpXWozGLuc8mOkOct OhO1MxurEt8vBEvL9JenlTgFI6CZBY787jllwnSPAjF3bXLsR2OXteTB5saV5aGIDTSS AKdnotDcsPx1qofaQD8/9+RYPpFu7dBcLVb1TQbeZPuMXrXuLrTuui0+tYXtwmBx9XAF pHqfqjDDEdxPCuyKy9jXScARvUS/HOxvPizfVLWZ9Xa4qSGosmwNfgKMIbruvV3H3uga 7alg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@colorado.edu header.s=google header.b="E/zZvYre"; spf=pass (google.com: domain of blake.caldwell@colorado.edu designates 209.85.220.65 as permitted sender) smtp.mailfrom=blake.caldwell@colorado.edu; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=colorado.edu Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id 71sor4872782itw.15.2019.01.11.16.36.52 for (Google Transport Security); Fri, 11 Jan 2019 16:36:52 -0800 (PST) Received-SPF: pass (google.com: domain of blake.caldwell@colorado.edu designates 209.85.220.65 as permitted sender) client-ip=209.85.220.65; Authentication-Results: mx.google.com; dkim=pass header.i=@colorado.edu header.s=google header.b="E/zZvYre"; spf=pass (google.com: domain of blake.caldwell@colorado.edu designates 209.85.220.65 as permitted sender) smtp.mailfrom=blake.caldwell@colorado.edu; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=colorado.edu DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=colorado.edu; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :in-reply-to:references; bh=DQywlzhu2L0dB3ib+YzwL3C0Y3sRu2MbV3QclYfYcJI=; b=E/zZvYre96sSAU9ZRNeglc6uPBklYq4zgUNnF6bmIPQ9/yF19Eq++NLwZXE0LK+xym Kw2l8KwqH9t8R0rl8YaLHeb0peIq8teGPWmWgIq72csix/0aai1cpNRzad66r6WCO5LQ bEl1+9GeKcHn4XCepYCg9ZvdVMbprjF0OAFb76iX1d0YbS3KPEuJkmz5Txx4itWImYo0 r73Ow4twfnCJ3V49/fIciL5adgsLYPltyAFKXZZqjSA+lwQhBjz5qLM8fW+mY1qs3NIH +cHPFsB0SUcnaRnt16PonVQfAQ1kh9ANfHA4gik0EpTL5KPDFLnjB/LW+GoF++zI5EYg Hviw== X-Google-Smtp-Source: ALg8bN7WrOzsoEVaAtr0gLutrY5CE0soTRtEkDJHopWqIs3yi0cH4zbBbnoiiw5v8lc7EXwC4h58Vw== X-Received: by 2002:a24:cfc1:: with SMTP id y184mr2422212itf.72.1547253412052; Fri, 11 Jan 2019 16:36:52 -0800 (PST) Received: from sos1.localdomain (ucb-np1-75.colorado.edu. [128.138.64.75]) by smtp.googlemail.com with ESMTPSA id o63sm1605971ita.8.2019.01.11.16.36.50 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 11 Jan 2019 16:36:50 -0800 (PST) From: Blake Caldwell To: blake.caldwell@colorado.edu Cc: rppt@linux.vnet.ibm.com, xemul@virtuozzo.com, akpm@linux-foundation.org, mike.kravetz@oracle.com, kirill.shutemov@linux.intel.com, Andrea Arcangeli , linux-mm@kvack.org Subject: [PATCH 1/4] userfaultfd: UFFDIO_REMAP: rmap preparation Date: Sat, 12 Jan 2019 00:36:26 +0000 Message-Id: <97a56d8c0d61846bdfa9fa0f8449238781bd5178.1547251023.git.blake.caldwell@colorado.edu> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: References: In-Reply-To: References: X-Bogosity: Ham, tests=bogofilter, spamicity=0.000008, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP From: Andrea Arcangeli As far as the rmap code is concerned, UFFDIO_REMAP only alters the page->mapping and page->index. It does it while holding the page lock. However page_referenced() is doing rmap walks without taking the page lock first, so page_lock_anon_vma_read must be updated to re-check that the page->mapping didn't change after we obtained the anon_vma read lock. UFFDIO_REMAP takes the anon_vma lock for writing before altering the page->mapping, so if the page->mapping is still the same after obtaining the anon_vma read lock (without the page lock), the rmap walks can go ahead safely (and UFFDIO_REMAP will wait the rmap walk to complete before proceeding). UFFDIO_REMAP serializes against itself with the page lock. All other places taking the anon_vma lock while holding the mmap_sem for writing, don't need to check if the page->mapping has changed after taking the anon_vma lock, regardless of the page lock, because UFFDIO_REMAP holds the mmap_sem for reading. There's one constraint enforced to allow this simplification: the source pages passed to UFFDIO_REMAP must be mapped only in one vma, but this constraint is an acceptable tradeoff for UFFDIO_REMAP users. The source addresses passed to UFFDIO_REMAP should be set as VM_DONTCOPY with MADV_DONTFORK to avoid any risk of the mapcount of the pages increasing if some thread of the process forks() before UFFDIO_REMAP run. Acked-by: Pavel Emelyanov Signed-off-by: Andrea Arcangeli --- mm/rmap.c | 13 +++++++++++++ 1 file changed, 13 insertions(+) diff --git a/mm/rmap.c b/mm/rmap.c index 0454ecc2..d8f228d 100644 --- a/mm/rmap.c +++ b/mm/rmap.c @@ -510,6 +510,7 @@ struct anon_vma *page_lock_anon_vma_read(struct page *page) struct anon_vma *root_anon_vma; unsigned long anon_mapping; +repeat: rcu_read_lock(); anon_mapping = (unsigned long)READ_ONCE(page->mapping); if ((anon_mapping & PAGE_MAPPING_FLAGS) != PAGE_MAPPING_ANON) @@ -548,6 +549,18 @@ struct anon_vma *page_lock_anon_vma_read(struct page *page) rcu_read_unlock(); anon_vma_lock_read(anon_vma); + /* + * Check if UFFDIO_REMAP changed the anon_vma. This is needed + * because we don't assume the page was locked. + */ + if (unlikely((unsigned long) READ_ONCE(page->mapping) != + anon_mapping)) { + anon_vma_unlock_read(anon_vma); + put_anon_vma(anon_vma); + anon_vma = NULL; + goto repeat; + } + if (atomic_dec_and_test(&anon_vma->refcount)) { /* * Oops, we held the last refcount, release the lock