From patchwork Sat Oct 13 00:24:28 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrea Arcangeli X-Patchwork-Id: 10639761 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 486AE14BD for ; Sat, 13 Oct 2018 00:24:40 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 39E4E2BAFF for ; Sat, 13 Oct 2018 00:24:40 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 2E78A2BB04; Sat, 13 Oct 2018 00:24:40 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id A55042BAFF for ; Sat, 13 Oct 2018 00:24:39 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id C7F786B0298; Fri, 12 Oct 2018 20:24:33 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id A1D1B6B0296; Fri, 12 Oct 2018 20:24:33 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8C42F6B029A; Fri, 12 Oct 2018 20:24:33 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-qk1-f198.google.com (mail-qk1-f198.google.com [209.85.222.198]) by kanga.kvack.org (Postfix) with ESMTP id 55A676B0296 for ; Fri, 12 Oct 2018 20:24:33 -0400 (EDT) Received: by mail-qk1-f198.google.com with SMTP id l75-v6so13270994qke.23 for ; Fri, 12 Oct 2018 17:24:33 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references; bh=SAn+1otRylh5SKzrbcCwYL1Iufz364Vef4rIrxRO5ig=; b=VOw/2hJkIye0k5dOsXksI0jiCCbcbqUYc7HJ6Y0SEz6zu1bgwcI3RW3wh9oFfyqIBF hT4iX9g/2YRk5Z1kX3tJ7Ld1DZwr0FZzaPkiGcBEv+BC4dsSuFfKwJ0DoMKo6TFXbxD1 0nqE0WkULlUmj+kgYpnksqh99YXKfde7D5ej3T7TFp5oy8ndBLKIhm8uG4Sc34hcTyfz HAqxwse+GmrRSzo0HI9LbuOMUKTTYP8US5pUpBFkSsOtgeWGiiMZ/bCSr418t717Ct4b wr0NBkeXcYB3aq43+u/rbZjYtyIH1A8k6MjaVngKc7QgZJujgFoJ+3QVuoqZeuHDQXcy CtNg== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of aarcange@redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=aarcange@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com X-Gm-Message-State: ABuFfoi8mVE/yOImZeBGFQXDgXvR/tg3cAu49fXIjEr1gQT3uw15uqTj 1FJ0c7G8oJkCezfCNbZAyg1UHwpK5jH8QjrS5DD9zNJk8iEYyacj206+5YO8+aVvH8EmU5e5UDm Sp3QZzj5KaY1VzSK5t7tsu8N+NmoeHR+VBC/V4rPitWsNejcixE8EEhLcfotP2vbjCg== X-Received: by 2002:aed:22e1:: with SMTP id q30-v6mr7933313qtc.145.1539390273118; Fri, 12 Oct 2018 17:24:33 -0700 (PDT) X-Google-Smtp-Source: ACcGV621nv2PzWRHjnoqwpIkggmaaRbOIKCP+isXa2sYSngVmIWIMoGOv4vjflIR7ZrVzqbAOBDm X-Received: by 2002:aed:22e1:: with SMTP id q30-v6mr7933288qtc.145.1539390272483; Fri, 12 Oct 2018 17:24:32 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1539390272; cv=none; d=google.com; s=arc-20160816; b=KfYNQSNcFzWUWhT74Qeunm3GiY2z46TmpH2S+/3WqwDsxiMCseEd+fx6Zuw2j3crHF Sh9GgVLGt7cDmBXgKX3QzB7MjklM/JBh/h1FcqA3vH37CbYoOm72kJrKmBqCVCS130V8 HJmqySWPolymxSPem1hXx6r34OpYJUwmk3V1bHyuf9SqS2BQGXugDeBmqd1hpxjluqX4 egxq9k6Gla5ERsg45kRd64uXBoS+vKnTutXLb+VJpN/XrJCgBCecyEIzPkXm9mfzo+OU 7AtYUeTF/lngvgDv8yvTyKPFBixYMMfG2HkMXTIocJRFyvpM+lcZT3UsLIm/QGEYRZoG CwEA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:cc:to:from; bh=SAn+1otRylh5SKzrbcCwYL1Iufz364Vef4rIrxRO5ig=; b=lOY5oMeMjuUpaA2QtrwvUYyYigSPHVirVOBy9Mx8XkkIl34ptrzaq9VS8ybS3LvXay botbfe+vyLhpGPrVDoRGmQ7AjxHSxm5gOmScEkHRF0xerZXQYobEt3Fm54AWaiz7pGNf 0z/4Y5eGwGcIMOiFOJiY4SmaMZW4SAWga5dJFqlFRH9vxwQzHKaDDxeX1VSDueroCb1a 5iQmAe1gaSIBwZRwcYzEw4NQJ6DoFWJvSQLwUMiuEA+RrkvyyZkIahDKbgYXH3hN5vH/ 6aeSHKwL53LzkMkV7y5CNkKSqmna/6PQOsLuudsmnJDjydlMBaSk0ntEsoSz1cV7PtpB sD+A== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of aarcange@redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=aarcange@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from mx1.redhat.com (mx1.redhat.com. [209.132.183.28]) by mx.google.com with ESMTPS id a21-v6si955864qtp.98.2018.10.12.17.24.32 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 12 Oct 2018 17:24:32 -0700 (PDT) Received-SPF: pass (google.com: domain of aarcange@redhat.com designates 209.132.183.28 as permitted sender) client-ip=209.132.183.28; Authentication-Results: mx.google.com; spf=pass (google.com: domain of aarcange@redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=aarcange@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.phx2.redhat.com [10.5.11.22]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 9D465750CB; Sat, 13 Oct 2018 00:24:31 +0000 (UTC) Received: from sky.random (ovpn-120-22.rdu2.redhat.com [10.10.120.22]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 4CB901001949; Sat, 13 Oct 2018 00:24:31 +0000 (UTC) From: Andrea Arcangeli To: linux-mm@kvack.org Cc: Aaron Tomlin , Mel Gorman , Jerome Glisse , "Kirill A. Shutemov" , Andrew Morton Subject: [PATCH 1/3] mm: thp: fix MADV_DONTNEED vs migrate_misplaced_transhuge_page race condition Date: Fri, 12 Oct 2018 20:24:28 -0400 Message-Id: <20181013002430.698-2-aarcange@redhat.com> In-Reply-To: <20181013002430.698-1-aarcange@redhat.com> References: <20181013002430.698-1-aarcange@redhat.com> X-Scanned-By: MIMEDefang 2.84 on 10.5.11.22 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.39]); Sat, 13 Oct 2018 00:24:31 +0000 (UTC) X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP This is a corollary of ced108037c2aa542b3ed8b7afd1576064ad1362a, 58ceeb6bec86d9140f9d91d71a710e963523d063, 5b7abeae3af8c08c577e599dd0578b9e3ee6687b. When the above three fixes where posted Dave asked https://lkml.kernel.org/r/929b3844-aec2-0111-fef7-8002f9d4e2b9@intel.com but apparently this was missed. The pmdp_clear_flush* in migrate_misplaced_transhuge_page was introduced in commit a54a407fbf7735fd8f7841375574f5d9b0375f93. The important part of such commit is only the part where the page lock is not released until the first do_huge_pmd_numa_page() finished disarming the pagenuma/protnone. The addition of pmdp_clear_flush() wasn't beneficial to such commit and there's no commentary about such an addition either. I guess the pmdp_clear_flush() in such commit was added just in case for safety, but it ended up introducing the MADV_DONTNEED race condition found by Aaron. At that point in time nobody thought of such kind of MADV_DONTNEED race conditions yet (they were fixed later) so the code may have looked more robust by adding the pmdp_clear_flush(). This specific race condition won't destabilize the kernel, but it can confuse userland because after MADV_DONTNEED the memory won't be zeroed out. This also optimizes the code and removes a superflous TLB flush. Reported-by: Aaron Tomlin Signed-off-by: Andrea Arcangeli Acked-by: Mel Gorman Acked-by: Kirill A. Shutemov --- mm/migrate.c | 26 +++++++++++++++++++------- 1 file changed, 19 insertions(+), 7 deletions(-) diff --git a/mm/migrate.c b/mm/migrate.c index d6a2e89b086a..180e3d0ed16d 100644 --- a/mm/migrate.c +++ b/mm/migrate.c @@ -2082,15 +2082,27 @@ int migrate_misplaced_transhuge_page(struct mm_struct *mm, entry = maybe_pmd_mkwrite(pmd_mkdirty(entry), vma); /* - * Clear the old entry under pagetable lock and establish the new PTE. - * Any parallel GUP will either observe the old page blocking on the - * page lock, block on the page table lock or observe the new page. - * The SetPageUptodate on the new page and page_add_new_anon_rmap - * guarantee the copy is visible before the pagetable update. + * Overwrite the old entry under pagetable lock and establish + * the new PTE. Any parallel GUP will either observe the old + * page blocking on the page lock, block on the page table + * lock or observe the new page. The SetPageUptodate on the + * new page and page_add_new_anon_rmap guarantee the copy is + * visible before the pagetable update. */ flush_cache_range(vma, mmun_start, mmun_end); page_add_anon_rmap(new_page, vma, mmun_start, true); - pmdp_huge_clear_flush_notify(vma, mmun_start, pmd); + /* + * At this point the pmd is numa/protnone (i.e. non present) + * and the TLB has already been flushed globally. So no TLB + * can be currently caching this non present pmd mapping. + * There's no need of clearing the pmd before doing + * set_pmd_at(), nor to flush the TLB after + * set_pmd_at(). Clearing the pmd here would introduce a race + * condition against MADV_DONTNEED, beacuse MADV_DONTNEED only + * holds the mmap_sem for reading. If the pmd is set to NULL + * at any given time, MADV_DONTNEED won't wait on the pmd lock + * and it'll skip clearing this pmd. + */ set_pmd_at(mm, mmun_start, pmd, entry); update_mmu_cache_pmd(vma, address, &entry); @@ -2104,7 +2116,7 @@ int migrate_misplaced_transhuge_page(struct mm_struct *mm, * No need to double call mmu_notifier->invalidate_range() callback as * the above pmdp_huge_clear_flush_notify() did already call it. */ - mmu_notifier_invalidate_range_only_end(mm, mmun_start, mmun_end); + mmu_notifier_invalidate_range_end(mm, mmun_start, mmun_end); /* Take an "isolate" reference and put new page on the LRU. */ get_page(new_page);