From patchwork Thu Nov 22 19:52:37 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sasha Levin X-Patchwork-Id: 10694761 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id A27B11709 for ; Thu, 22 Nov 2018 19:54:43 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 8F4C429710 for ; Thu, 22 Nov 2018 19:54:43 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 7E6312AF94; Thu, 22 Nov 2018 19:54:43 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 7BD2429710 for ; Thu, 22 Nov 2018 19:54:41 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 64D636B2CEA; Thu, 22 Nov 2018 14:54:40 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 5F96B6B2CEB; Thu, 22 Nov 2018 14:54:40 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 4EAB06B2CEC; Thu, 22 Nov 2018 14:54:40 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-pg1-f200.google.com (mail-pg1-f200.google.com [209.85.215.200]) by kanga.kvack.org (Postfix) with ESMTP id 0959A6B2CEA for ; Thu, 22 Nov 2018 14:54:40 -0500 (EST) Received: by mail-pg1-f200.google.com with SMTP id h9so3062470pgm.1 for ; Thu, 22 Nov 2018 11:54:40 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references; bh=poLDuUGujotyoCjXCCgq11V1dwqmo3SolxbNlFciDWY=; b=foyQNIAd5GH+zsHPyJ8HfJjvPFwstRoG20Pgnvdr3rzDZdkFIuWYiQS9rDB1qO+gVG VF805lWVAtzaVIY2vGCN7HXcFY8c+BNC4rGbaHy2kp/RxdbACS8sjpJBEzRdbk9YVVCX 7ueLz/0vubc9peQaaP94NoC9J8ctuWk7MULREYl3Vz+8otUFSxB3Yq9OXR8CVhFBBkzh vhsGrblxzUnjeMF5bc9S/yD631GwczwnRCbl+xepwglyjquUPwrwsXvYYJHiBuX6fXAD 5v9w6MS9uqSrSsGcgnQF5/eYS0d2kFLHi6qXN0W8x8++iOOdurzFtbOxzfhy9MwYDxeM 2WBg== X-Gm-Message-State: AA+aEWbh6XSnS7bGB+3Tc8IJjQf4zLzZVH9PeNTXbzOFpg5qDdgV+J3X 0bvH2s8Ru5KbIC6sXcfkGHbt6sRpecD19IsXNzd94+FA42mUFDRFX3P/JNwg0joWcYCYKv8VvNe sByqlxI5+MvYSVrG8QDRzAy8SGvpTNctF9B1g3SQgkaTI1kcCt+meqRc0au5gww7Nsg== X-Received: by 2002:a17:902:7587:: with SMTP id j7mr12610786pll.191.1542916479517; Thu, 22 Nov 2018 11:54:39 -0800 (PST) X-Google-Smtp-Source: AFSGD/XZPwgj8lHtVLwxs7cnfhdn3/QoU8zW1tPc6k/ONBdMPaWCJnT60iHAZYflPJJdb680xCjG X-Received: by 2002:a17:902:7587:: with SMTP id j7mr12610752pll.191.1542916478628; Thu, 22 Nov 2018 11:54:38 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1542916478; cv=none; d=google.com; s=arc-20160816; b=PHBCTXwSYeK830rrQcVr4pM+E/IRNkIedQpDtPqOdh6A+Atu6qsGt/GCGBiSusAb6b QXYwUBXw7osDc44wpn4s+15ZtuHwcZnHAs448mQkvf9emIRfg9FClnxEX+Hy0GMdztqp BGUniANFYYJgJcIvVPpk/1U+9xoJb7OoMyWaAHENFaBXslq+8u5JGBgIoJSWf7du1Id8 Uo02cXn0X5uIr57UZDnHAqIsa4oMnFFnQ0QaoG1Od9Iq1S7Fx0Amj5+3KKXGG6ETot/o 1w9pyWOiq64x3Clwm6qjZSz3zyhQ6I+XDWpN59rV+7UAfCe9Cxd1TPFjyAv68Hodpfod wC6g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=poLDuUGujotyoCjXCCgq11V1dwqmo3SolxbNlFciDWY=; b=pNzBreHC8Cx4QfLsDsZx1whlWbE+Amnq+Ue9zbrhi+mLSWTtbWAcTKvFu6TcboKEEC B7AxaaCBhQ+duP7QFgJ0kNkfpe9snAswbqkT8w7Nb0CFjW9QrUVtwydUY6zNeoNBf7uv BatFjpFwhBwq8oRfEJBHedraf0JeZFJd1ywBQ7us1ArdnC9PO1rBkLL1Ix7Dvl/KibQ3 HBqSF5rzyYNZWD0MUt20HLLT/VfE3bWKqb7o8qrYwb7MaO52oWwj75m7pmx4mCdDMSDZ rStr82qva1+xgFQpq4ZMJegt6SRcDVw68i0awCliABVIx5PJCtqenQwdquhMcr+k5vfQ F9xw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@kernel.org header.s=default header.b="qj+/UAGH"; spf=pass (google.com: domain of sashal@kernel.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=sashal@kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from mail.kernel.org (mail.kernel.org. [198.145.29.99]) by mx.google.com with ESMTPS id h191si21267462pgc.302.2018.11.22.11.54.38 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 22 Nov 2018 11:54:38 -0800 (PST) Received-SPF: pass (google.com: domain of sashal@kernel.org designates 198.145.29.99 as permitted sender) client-ip=198.145.29.99; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=default header.b="qj+/UAGH"; spf=pass (google.com: domain of sashal@kernel.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=sashal@kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from sasha-vm.mshome.net (unknown [37.142.5.207]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id C7C4D20645; Thu, 22 Nov 2018 19:54:35 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1542916478; bh=OIvwv7IS2hqDONDnSt/xuB732ZCGsEvkdES5wVdcztc=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=qj+/UAGHFES19oQT61TWSezhS3+61IVd+s1hLpag4KU2KFCuZ5mcQ6t4m2Ysa3VZk JrGGS5sBtmWiNYTfFDR0+8jojwV79U0lLg8sVzSIyrufeVHTRmwRhUbv3glBTkxPZG ++TI8uRWuM5jcC+0njuOJ5NGeRRzwf3/d2roITSY= From: Sasha Levin To: stable@vger.kernel.org, linux-kernel@vger.kernel.org Cc: Vitaly Wool , Vitaly Wool , Jongseok Kim , Andrew Morton , Linus Torvalds , Sasha Levin , linux-mm@kvack.org Subject: [PATCH AUTOSEL 4.19 33/36] z3fold: fix possible reclaim races Date: Thu, 22 Nov 2018 14:52:37 -0500 Message-Id: <20181122195240.13123-33-sashal@kernel.org> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20181122195240.13123-1-sashal@kernel.org> References: <20181122195240.13123-1-sashal@kernel.org> X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP From: Vitaly Wool [ Upstream commit ca0246bb97c23da9d267c2107c07fb77e38205c9 ] Reclaim and free can race on an object which is basically fine but in order for reclaim to be able to map "freed" object we need to encode object length in the handle. handle_to_chunks() is then introduced to extract object length from a handle and use it during mapping. Moreover, to avoid racing on a z3fold "headless" page release, we should not try to free that page in z3fold_free() if the reclaim bit is set. Also, in the unlikely case of trying to reclaim a page being freed, we should not proceed with that page. While at it, fix the page accounting in reclaim function. This patch supersedes "[PATCH] z3fold: fix reclaim lock-ups". Link: http://lkml.kernel.org/r/20181105162225.74e8837d03583a9b707cf559@gmail.com Signed-off-by: Vitaly Wool Signed-off-by: Jongseok Kim Reported-by-by: Jongseok Kim Reviewed-by: Snild Dolkow Signed-off-by: Andrew Morton Signed-off-by: Linus Torvalds Signed-off-by: Sasha Levin --- mm/z3fold.c | 101 ++++++++++++++++++++++++++++++++-------------------- 1 file changed, 62 insertions(+), 39 deletions(-) diff --git a/mm/z3fold.c b/mm/z3fold.c index 4b366d181f35..aee9b0b8d907 100644 --- a/mm/z3fold.c +++ b/mm/z3fold.c @@ -99,6 +99,7 @@ struct z3fold_header { #define NCHUNKS ((PAGE_SIZE - ZHDR_SIZE_ALIGNED) >> CHUNK_SHIFT) #define BUDDY_MASK (0x3) +#define BUDDY_SHIFT 2 /** * struct z3fold_pool - stores metadata for each z3fold pool @@ -145,7 +146,7 @@ enum z3fold_page_flags { MIDDLE_CHUNK_MAPPED, NEEDS_COMPACTING, PAGE_STALE, - UNDER_RECLAIM + PAGE_CLAIMED, /* by either reclaim or free */ }; /***************** @@ -174,7 +175,7 @@ static struct z3fold_header *init_z3fold_page(struct page *page, clear_bit(MIDDLE_CHUNK_MAPPED, &page->private); clear_bit(NEEDS_COMPACTING, &page->private); clear_bit(PAGE_STALE, &page->private); - clear_bit(UNDER_RECLAIM, &page->private); + clear_bit(PAGE_CLAIMED, &page->private); spin_lock_init(&zhdr->page_lock); kref_init(&zhdr->refcount); @@ -223,8 +224,11 @@ static unsigned long encode_handle(struct z3fold_header *zhdr, enum buddy bud) unsigned long handle; handle = (unsigned long)zhdr; - if (bud != HEADLESS) - handle += (bud + zhdr->first_num) & BUDDY_MASK; + if (bud != HEADLESS) { + handle |= (bud + zhdr->first_num) & BUDDY_MASK; + if (bud == LAST) + handle |= (zhdr->last_chunks << BUDDY_SHIFT); + } return handle; } @@ -234,6 +238,12 @@ static struct z3fold_header *handle_to_z3fold_header(unsigned long handle) return (struct z3fold_header *)(handle & PAGE_MASK); } +/* only for LAST bud, returns zero otherwise */ +static unsigned short handle_to_chunks(unsigned long handle) +{ + return (handle & ~PAGE_MASK) >> BUDDY_SHIFT; +} + /* * (handle & BUDDY_MASK) < zhdr->first_num is possible in encode_handle * but that doesn't matter. because the masking will result in the @@ -720,37 +730,39 @@ static void z3fold_free(struct z3fold_pool *pool, unsigned long handle) page = virt_to_page(zhdr); if (test_bit(PAGE_HEADLESS, &page->private)) { - /* HEADLESS page stored */ - bud = HEADLESS; - } else { - z3fold_page_lock(zhdr); - bud = handle_to_buddy(handle); - - switch (bud) { - case FIRST: - zhdr->first_chunks = 0; - break; - case MIDDLE: - zhdr->middle_chunks = 0; - zhdr->start_middle = 0; - break; - case LAST: - zhdr->last_chunks = 0; - break; - default: - pr_err("%s: unknown bud %d\n", __func__, bud); - WARN_ON(1); - z3fold_page_unlock(zhdr); - return; + /* if a headless page is under reclaim, just leave. + * NB: we use test_and_set_bit for a reason: if the bit + * has not been set before, we release this page + * immediately so we don't care about its value any more. + */ + if (!test_and_set_bit(PAGE_CLAIMED, &page->private)) { + spin_lock(&pool->lock); + list_del(&page->lru); + spin_unlock(&pool->lock); + free_z3fold_page(page); + atomic64_dec(&pool->pages_nr); } + return; } - if (bud == HEADLESS) { - spin_lock(&pool->lock); - list_del(&page->lru); - spin_unlock(&pool->lock); - free_z3fold_page(page); - atomic64_dec(&pool->pages_nr); + /* Non-headless case */ + z3fold_page_lock(zhdr); + bud = handle_to_buddy(handle); + + switch (bud) { + case FIRST: + zhdr->first_chunks = 0; + break; + case MIDDLE: + zhdr->middle_chunks = 0; + break; + case LAST: + zhdr->last_chunks = 0; + break; + default: + pr_err("%s: unknown bud %d\n", __func__, bud); + WARN_ON(1); + z3fold_page_unlock(zhdr); return; } @@ -758,7 +770,7 @@ static void z3fold_free(struct z3fold_pool *pool, unsigned long handle) atomic64_dec(&pool->pages_nr); return; } - if (test_bit(UNDER_RECLAIM, &page->private)) { + if (test_bit(PAGE_CLAIMED, &page->private)) { z3fold_page_unlock(zhdr); return; } @@ -836,20 +848,30 @@ static int z3fold_reclaim_page(struct z3fold_pool *pool, unsigned int retries) } list_for_each_prev(pos, &pool->lru) { page = list_entry(pos, struct page, lru); + + /* this bit could have been set by free, in which case + * we pass over to the next page in the pool. + */ + if (test_and_set_bit(PAGE_CLAIMED, &page->private)) + continue; + + zhdr = page_address(page); if (test_bit(PAGE_HEADLESS, &page->private)) - /* candidate found */ break; - zhdr = page_address(page); - if (!z3fold_page_trylock(zhdr)) + if (!z3fold_page_trylock(zhdr)) { + zhdr = NULL; continue; /* can't evict at this point */ + } kref_get(&zhdr->refcount); list_del_init(&zhdr->buddy); zhdr->cpu = -1; - set_bit(UNDER_RECLAIM, &page->private); break; } + if (!zhdr) + break; + list_del_init(&page->lru); spin_unlock(&pool->lock); @@ -898,6 +920,7 @@ static int z3fold_reclaim_page(struct z3fold_pool *pool, unsigned int retries) if (test_bit(PAGE_HEADLESS, &page->private)) { if (ret == 0) { free_z3fold_page(page); + atomic64_dec(&pool->pages_nr); return 0; } spin_lock(&pool->lock); @@ -905,7 +928,7 @@ static int z3fold_reclaim_page(struct z3fold_pool *pool, unsigned int retries) spin_unlock(&pool->lock); } else { z3fold_page_lock(zhdr); - clear_bit(UNDER_RECLAIM, &page->private); + clear_bit(PAGE_CLAIMED, &page->private); if (kref_put(&zhdr->refcount, release_z3fold_page_locked)) { atomic64_dec(&pool->pages_nr); @@ -964,7 +987,7 @@ static void *z3fold_map(struct z3fold_pool *pool, unsigned long handle) set_bit(MIDDLE_CHUNK_MAPPED, &page->private); break; case LAST: - addr += PAGE_SIZE - (zhdr->last_chunks << CHUNK_SHIFT); + addr += PAGE_SIZE - (handle_to_chunks(handle) << CHUNK_SHIFT); break; default: pr_err("unknown buddy id %d\n", buddy); From patchwork Thu Nov 22 19:52:38 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sasha Levin X-Patchwork-Id: 10694765 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 8189114BD for ; Thu, 22 Nov 2018 19:54:46 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 73BDF29710 for ; Thu, 22 Nov 2018 19:54:46 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 67D482AF94; Thu, 22 Nov 2018 19:54:46 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 4D22029710 for ; Thu, 22 Nov 2018 19:54:45 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 5AD7F6B2CEB; Thu, 22 Nov 2018 14:54:44 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 55CEE6B2CEC; Thu, 22 Nov 2018 14:54:44 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 4738D6B2CED; Thu, 22 Nov 2018 14:54:44 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-pg1-f198.google.com (mail-pg1-f198.google.com [209.85.215.198]) by kanga.kvack.org (Postfix) with ESMTP id 098646B2CEB for ; Thu, 22 Nov 2018 14:54:44 -0500 (EST) Received: by mail-pg1-f198.google.com with SMTP id k125so3048264pga.5 for ; Thu, 22 Nov 2018 11:54:44 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references; bh=Ill0//kn93Bkdu5f9E4sZ5BD2rwFwXfjebJZmHkxkYE=; b=ryroBfqIlMihtF74au+e9cfaGzGr4s3FXILBxlfHFMDLjychMjnBr4h1Hxx3X7DTaj vnwP9tUbMjDkB+jNxBPDiZn75a9NuB/WpMqw/hhg2WuO81/EeBQ+9kne8+hDuS1bx2ev zYsOIhvcSRnhYB6H7OSReId51Qg1y58EINO3LEBFbCn/q5TsHqxqivYhMdPrs7mdcVKW mlkCfntOyLRZFgBCHihX1+e5a+/PbyQV5+7w9g2gNcKzteBXYSG+N5rZRriXcnsC1NwP H51bo+DiS782kUd/craPVb4SSksUsffPNlYtJUW1H4pDimXfLM+2eZnyU107ZkCWCkJR J/1A== X-Gm-Message-State: AA+aEWY8ZJGv3mm5IQN1cA7uiVu+UlJuAGMkgfpcXDsQ1vNs4iOacbmd AEjYaa5Az9c3ZLK2/RNvztGkrdsshJ/i6RLh1tdRVOKZlZVcNBzMIe1CkTEe1T+9fbhAaDLPp2/ Cr2/6D20e2EdxflKH9ZqWwhHg3Esg+dwXhVY7zheInAlEqorHl8o/tWi0Og12alIVOA== X-Received: by 2002:a63:6bc1:: with SMTP id g184mr11445011pgc.25.1542916483696; Thu, 22 Nov 2018 11:54:43 -0800 (PST) X-Google-Smtp-Source: AFSGD/VJCQRx0zDN/ble+xJFUb6gkzC+SnOcNrKeYrrV3xAmKAyvX+M1za/sWBEDxwqZeRJNId/I X-Received: by 2002:a63:6bc1:: with SMTP id g184mr11444976pgc.25.1542916483031; Thu, 22 Nov 2018 11:54:43 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1542916483; cv=none; d=google.com; s=arc-20160816; b=VUQ2SWdH12ZQ+q+nevQDekjsyn4EMDpQZeXU6O7OQLXx3nWgwMujP85Fmw0fhWS3R6 U+R4Pu+wwIjUfyNH0Lj53ItjD+XmXY1YcZoL9dhs++aEeCzPuZZb/qVDmnHuocNb2XSq RC69CxNBtKcyp3ZbXgaLxaXkO+8ZoZaxbDR02yeGNVSF4+VabolQznymFD3kJXFttwTQ TkAA+TwmefEcMSdhaAcOnlrrCpknGWY2Cl1E4C+Ixv6oZgYawlQYuWC2qzQKt+38ks4p t+pCuTy+OjMuidq4P/kH6Bgayw0XUlqtK+z41KhhZnR1eu6gdEvASMhb6Zv3FGTnd2ue fj0A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=Ill0//kn93Bkdu5f9E4sZ5BD2rwFwXfjebJZmHkxkYE=; b=i/VUSlxqqCmvumOuZegWFMHsX8bjeIArPpfDpQ7r08molJG1ARMhKP5eQBsMrDQKYf MA2XQgwvkNyCFrFYNMhgrgWa0kO6nnqGsBQm6ZGkeDY55+IS4M1rRzintABVZYucTc+A L37NJqx03q3f+bltfHxpjzerCyCXE7a8tMUMLKtX4i2ES62RMUkiiwk8DDMimDAqGNx6 A+FTEQskbWEEsv2cLFoidKBTnabNXEIP2kcbLD9F5Tb9Vb4JG6/FciN+3gTaJywC70c9 6WknymT7oQO7zGNCfxbB1DAdKiywsdecBJaKaEuycj5/V/XszDqfjsVaymxFDHSRCEef zRHg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@kernel.org header.s=default header.b=qlgxrVOt; spf=pass (google.com: domain of sashal@kernel.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=sashal@kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from mail.kernel.org (mail.kernel.org. [198.145.29.99]) by mx.google.com with ESMTPS id s123si3095407pgs.93.2018.11.22.11.54.42 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 22 Nov 2018 11:54:43 -0800 (PST) Received-SPF: pass (google.com: domain of sashal@kernel.org designates 198.145.29.99 as permitted sender) client-ip=198.145.29.99; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=default header.b=qlgxrVOt; spf=pass (google.com: domain of sashal@kernel.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=sashal@kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from sasha-vm.mshome.net (unknown [37.142.5.207]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id A626A20672; Thu, 22 Nov 2018 19:54:40 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1542916482; bh=cEI4xZXD/nCA45zDyQn1uqDbcZn+7YfiZfdn9YA3Bbo=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=qlgxrVOt4xGUrFU6Xg8g3hN+owDpAy+xZpsLE/xG4RF5EFOpgbR23ILlrHGU/nlMe tyomIChSXHesT5yQntDQ9AHNbwq/wYZHRXgqplxqzQSodXRfXnB3asyj4jQIMpO/CE UptynUgatfUX/fqromdhpstajHZuHbBk2mIBY0aE= From: Sasha Levin To: stable@vger.kernel.org, linux-kernel@vger.kernel.org Cc: Michal Hocko , Andrew Morton , Linus Torvalds , Sasha Levin , linux-mm@kvack.org Subject: [PATCH AUTOSEL 4.19 34/36] mm, memory_hotplug: check zone_movable in has_unmovable_pages Date: Thu, 22 Nov 2018 14:52:38 -0500 Message-Id: <20181122195240.13123-34-sashal@kernel.org> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20181122195240.13123-1-sashal@kernel.org> References: <20181122195240.13123-1-sashal@kernel.org> X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP From: Michal Hocko [ Upstream commit 9d7899999c62c1a81129b76d2a6ecbc4655e1597 ] Page state checks are racy. Under a heavy memory workload (e.g. stress -m 200 -t 2h) it is quite easy to hit a race window when the page is allocated but its state is not fully populated yet. A debugging patch to dump the struct page state shows has_unmovable_pages: pfn:0x10dfec00, found:0x1, count:0x0 page:ffffea0437fb0000 count:1 mapcount:1 mapping:ffff880e05239841 index:0x7f26e5000 compound_mapcount: 1 flags: 0x5fffffc0090034(uptodate|lru|active|head|swapbacked) Note that the state has been checked for both PageLRU and PageSwapBacked already. Closing this race completely would require some sort of retry logic. This can be tricky and error prone (think of potential endless or long taking loops). Workaround this problem for movable zones at least. Such a zone should only contain movable pages. Commit 15c30bc09085 ("mm, memory_hotplug: make has_unmovable_pages more robust") has told us that this is not strictly true though. Bootmem pages should be marked reserved though so we can move the original check after the PageReserved check. Pages from other zones are still prone to races but we even do not pretend that memory hotremove works for those so pre-mature failure doesn't hurt that much. Link: http://lkml.kernel.org/r/20181106095524.14629-1-mhocko@kernel.org Fixes: 15c30bc09085 ("mm, memory_hotplug: make has_unmovable_pages more robust") Signed-off-by: Michal Hocko Reported-by: Baoquan He Tested-by: Baoquan He Acked-by: Baoquan He Reviewed-by: Oscar Salvador Acked-by: Balbir Singh Signed-off-by: Andrew Morton Signed-off-by: Linus Torvalds Signed-off-by: Sasha Levin --- mm/page_alloc.c | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/mm/page_alloc.c b/mm/page_alloc.c index e2ef1c17942f..3a4065312938 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -7690,6 +7690,14 @@ bool has_unmovable_pages(struct zone *zone, struct page *page, int count, if (PageReserved(page)) goto unmovable; + /* + * If the zone is movable and we have ruled out all reserved + * pages then it should be reasonably safe to assume the rest + * is movable. + */ + if (zone_idx(zone) == ZONE_MOVABLE) + continue; + /* * Hugepages are not in LRU lists, but they're movable. * We need not scan over tail pages bacause we don't From patchwork Thu Nov 22 19:52:39 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sasha Levin X-Patchwork-Id: 10694767 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 505BF14BD for ; Thu, 22 Nov 2018 19:54:49 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 407A129710 for ; Thu, 22 Nov 2018 19:54:49 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 345192AF94; Thu, 22 Nov 2018 19:54:49 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id BE70B29710 for ; Thu, 22 Nov 2018 19:54:48 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 845B06B2CED; Thu, 22 Nov 2018 14:54:47 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 7F4CD6B2CEE; Thu, 22 Nov 2018 14:54:47 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 6BD536B2CEF; Thu, 22 Nov 2018 14:54:47 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-pl1-f197.google.com (mail-pl1-f197.google.com [209.85.214.197]) by kanga.kvack.org (Postfix) with ESMTP id 2C9AD6B2CED for ; Thu, 22 Nov 2018 14:54:47 -0500 (EST) Received: by mail-pl1-f197.google.com with SMTP id ay11so13963977plb.20 for ; Thu, 22 Nov 2018 11:54:47 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references; bh=QhK0vT8PeYSCCLbTom9RASBxXAca0THq6IpC5RhYar8=; b=Prrpooj8ypg1O7BxWJdOF4By5hmcu1ksKojgJIYIl+OP5LVvlxntuK762gIirYjlPQ +ulaTlbbx9qZI+mVnZmnv1Au22fqRBq1kOmRVwBc5c31Hn2hTwf2tNEotWrSbo46Auwy ypIjffxmQqG2OUhWW00nzCcRUYdwEBmmYST+71SFXEGU0ELCxk8hvleu13PRIh0gTddf AFo1DmLCJInrucfPquuNLUvm+xUG8bDmB0534C+XACuvUQ2kuheaNQk3PTCI92bMrPnl 6du0L/xbbPDsPht2TdqD3fzFiHiHw35DzYPb44V32Yc0m87cAASJPk0t0JX7BxLggLxT /D8A== X-Gm-Message-State: AA+aEWZVWWkwQiY3cQrOfmesGPL1Ap0wCSWBJD0c0srpBblGV+HAlphj mN+IywHG8dyjneeVkdpECbDkO6QdsArWhCQjiTc7OqDXK1+y7CTyCW9VMo7ahAkDuprCdQHV0Ow PZTUXYZGpI/XFfA0M5Vsj0Zkrn04sgt1GBYfwCFSIquIP0e3k1A0Xx3lwpYH9OSBhdw== X-Received: by 2002:a62:5486:: with SMTP id i128mr4714408pfb.215.1542916486840; Thu, 22 Nov 2018 11:54:46 -0800 (PST) X-Google-Smtp-Source: AFSGD/VdKYQ6YAvYpKg8qwS6UwhFry9rKm742IA1Rt5tZR06fTVYJhcPpxAKN0t6OTL1pS3GQvwP X-Received: by 2002:a62:5486:: with SMTP id i128mr4714389pfb.215.1542916486302; Thu, 22 Nov 2018 11:54:46 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1542916486; cv=none; d=google.com; s=arc-20160816; b=BE1vHTbO1gA+YX45PVFvvzIpBv3def5Jx9YqvPXPRQ21RSn2Jrdz9x89K/MNBk7Gis LiCFQQIBkt3C+vGEecfiOcnxrfSNbsoyZllybpwqYYvDxK50+zFPgsvL1CVA/hImAzXl thepEH5IAC3T2kDJyAt3dfIoqTwA2Aji6+0ofowuTNwqBTZoCqc7XerQbJu+Qxpf1Wpz Fv1fRs900/HIt2duyxggRQpW+tZ3zBLzZm23UeUtWIKOPpWRBZ0jVrvjaaiYzpVpbKc1 bAC6WHchVYDpWBlstn7Ys8q/D8wnpr1MiB3sNYles4pg/GBpuAh0QkjVE2Hh4aZFlA+k CDZA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=QhK0vT8PeYSCCLbTom9RASBxXAca0THq6IpC5RhYar8=; b=Q3h26jHOz8n3X5IpqxJJuKzlgzRzzx7QOH385G32pgntD6CvECvWnAlk6PVhWMauab EUDUrdJ0FZuYPINmXHg73NJnDlAPRbW/GlvX2smh42syr7vFhqXJbRA3RXgHj7/h/T2+ mvBtGfgHvePEcx8pJDSUm8vZGJ3TQ/R2DWugUAh9XP0w+yA1nJkF0BjwSvQ57hg/yXpk +Ktnd3qSv+EOLpR5buy9EFT/aIIBUYZke5ciqFrSkqsz1ktbN2WE5/eQ7aCqpG7Q+Q/Z g76/vCXYT2Bm+5+QA5AkR+nv5i76DbzmCN+KTEfuaYL2KaPYqLNDv2vJtg8Ce6Dw0Ojs Vpbw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@kernel.org header.s=default header.b=A1bdREAS; spf=pass (google.com: domain of sashal@kernel.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=sashal@kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from mail.kernel.org (mail.kernel.org. [198.145.29.99]) by mx.google.com with ESMTPS id j28si49463654pgm.160.2018.11.22.11.54.46 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 22 Nov 2018 11:54:46 -0800 (PST) Received-SPF: pass (google.com: domain of sashal@kernel.org designates 198.145.29.99 as permitted sender) client-ip=198.145.29.99; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=default header.b=A1bdREAS; spf=pass (google.com: domain of sashal@kernel.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=sashal@kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from sasha-vm.mshome.net (unknown [37.142.5.207]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 5E47120645; Thu, 22 Nov 2018 19:54:43 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1542916486; bh=X7gUEgCCkxL0A2Ds0/+fdp3ZX3Sjdi5FsJEcYoVFCzs=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=A1bdREAScheuDHtrlQIHD/LKoksCL1uZYs+Pe+Zb6CYFH9Me8n53wlAwNeImDd5By /K6cuqmPEDUP7IMaHc6i5Vjo24Qh4ZPuFLuI4PT3Lb8GTGcMIq0Z/2Mzibb3R38v93 5FBZrtdgcxCgn4Z9MSyIf42pgtxIiZnbl++bE7Pk= From: Sasha Levin To: stable@vger.kernel.org, linux-kernel@vger.kernel.org Cc: Yufen Yu , Al Viro , Hugh Dickins , William Kucharski , Andrew Morton , Linus Torvalds , Sasha Levin , linux-mm@kvack.org Subject: [PATCH AUTOSEL 4.19 35/36] tmpfs: make lseek(SEEK_DATA/SEK_HOLE) return ENXIO with a negative offset Date: Thu, 22 Nov 2018 14:52:39 -0500 Message-Id: <20181122195240.13123-35-sashal@kernel.org> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20181122195240.13123-1-sashal@kernel.org> References: <20181122195240.13123-1-sashal@kernel.org> X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP From: Yufen Yu [ Upstream commit 1a413646931cb14442065cfc17561e50f5b5bb44 ] Other filesystems such as ext4, f2fs and ubifs all return ENXIO when lseek (SEEK_DATA or SEEK_HOLE) requests a negative offset. man 2 lseek says : EINVAL whence is not valid. Or: the resulting file offset would be : negative, or beyond the end of a seekable device. : : ENXIO whence is SEEK_DATA or SEEK_HOLE, and the file offset is beyond : the end of the file. Make tmpfs return ENXIO under these circumstances as well. After this, tmpfs also passes xfstests's generic/448. [akpm@linux-foundation.org: rewrite changelog] Link: http://lkml.kernel.org/r/1540434176-14349-1-git-send-email-yuyufen@huawei.com Signed-off-by: Yufen Yu Reviewed-by: Andrew Morton Cc: Al Viro Cc: Hugh Dickins Cc: William Kucharski Signed-off-by: Andrew Morton Signed-off-by: Linus Torvalds Signed-off-by: Sasha Levin --- mm/shmem.c | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/mm/shmem.c b/mm/shmem.c index 446942677cd4..38d228a30fdc 100644 --- a/mm/shmem.c +++ b/mm/shmem.c @@ -2610,9 +2610,7 @@ static loff_t shmem_file_llseek(struct file *file, loff_t offset, int whence) inode_lock(inode); /* We're holding i_mutex so we can access i_size directly */ - if (offset < 0) - offset = -EINVAL; - else if (offset >= inode->i_size) + if (offset < 0 || offset >= inode->i_size) offset = -ENXIO; else { start = offset >> PAGE_SHIFT; From patchwork Thu Nov 22 19:52:40 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sasha Levin X-Patchwork-Id: 10694769 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id CA1B41709 for ; Thu, 22 Nov 2018 19:54:56 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id BB0A929710 for ; Thu, 22 Nov 2018 19:54:56 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id AED5E2AF94; Thu, 22 Nov 2018 19:54:56 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 0D07129710 for ; Thu, 22 Nov 2018 19:54:55 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 140FB6B2CEE; Thu, 22 Nov 2018 14:54:54 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 0F0536B2CEF; Thu, 22 Nov 2018 14:54:54 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id F21506B2CF0; Thu, 22 Nov 2018 14:54:53 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-pf1-f198.google.com (mail-pf1-f198.google.com [209.85.210.198]) by kanga.kvack.org (Postfix) with ESMTP id B2BB06B2CEE for ; Thu, 22 Nov 2018 14:54:53 -0500 (EST) Received: by mail-pf1-f198.google.com with SMTP id v79so3686960pfd.20 for ; Thu, 22 Nov 2018 11:54:53 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references; bh=2yY5eBgeLQ5vyKycEM63SfoZEzyeAKo8tsMENNgCu+A=; b=TsU4VGPw3QZBZINsl5mh/mbgApd2Jlw7uPBY7WZ3WTxbLSLqAM8EiBrUFKTsKitthT CfyJTl3Hh6c6Xd6YhZVeSnLIFyPKsI10u0tBsVtgdwXQOYXkdzcj4QnsCrHkqob8GCkD pQzfjIjrl+eWVM365+6f39YiHto+7npANR6vx5oDkcVtVDvbib99aJ/FN93Nx/vmBwtW qqWrMys31aWkhdL9x2nPokVxqyjtV55vu5CrwQnKcT3cvIZl1EpNZ+CG6rgC6ND6QxVo w4pcqEtzPTnV1K1cg1lY1vTWiSyGWoBld8PIBFTrcrqZRCkEmrHYH0xoXzP4B9/eTxqi 1LgA== X-Gm-Message-State: AA+aEWY7daZqG/r2fodf5cjcNtq4hcnbqdI9u0an6l+cI264MyvWM4wq 2ZY3Bp/uGv5vgocZ/V4Z1TEKA2XyMXxR2e26wsCO3FItNknsalt3uNXf42Vr2lu8jTVSSyKJO0J pQXINLGzSvnriXoI2jr0a9bXC1BpbUiXddv7tyYVkkIk7qcdbwNwR2QFZ3zzY2duRLg== X-Received: by 2002:a63:7418:: with SMTP id p24mr11327564pgc.196.1542916493237; Thu, 22 Nov 2018 11:54:53 -0800 (PST) X-Google-Smtp-Source: AFSGD/WEUJMuspibYXwL8umxLLQtb9rtJvPPHSgLoP/u7dMnkikO1PCvvP2EWhqVvAVS55/kPScG X-Received: by 2002:a63:7418:: with SMTP id p24mr11327534pgc.196.1542916492450; Thu, 22 Nov 2018 11:54:52 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1542916492; cv=none; d=google.com; s=arc-20160816; b=u/19n+u7xZQhquYffsLT87dirR3oKGKULvJh+Jv265ux8mEs7XCP9rPblXpz/zHNhB d28qQ1G/CVljd3Te8yw/mn3V0CfTxNhFEZXjwGOpveagtLC2yxXNlh6Mv6yHaakyf8Hw 67trHq1NP0XYGkmx1HafrwhbB42BcNjxtrJPVAm5R3mFtplYA27/2tUeeaOkRNhKi+Wi zsrt2YQ70zU1AaybjuergUfZW1oXDEFqr9IvkIV4Rw9oFHba0t7kfN6QVSjnJl8KgdDQ jNcF3/CFUPUykv30Iu+XcI8FYP4izy0H9Iz/XL3xR14chpULMBA/DEqUCg5xOxi6Yv5q XjRQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=2yY5eBgeLQ5vyKycEM63SfoZEzyeAKo8tsMENNgCu+A=; b=MebZcC1qQbYLnJ+/voiSVhdrrl4ubLx84tEQ/6aX8FopQ2s1AHk0cKRVCM+vmc6ZZK J/bvh8rbNGLipTNd4aKRA+tZ3oggPAP/qiDdogGt9O9Qzw9mjlJFSEggsAX9Tjy7PxxT o1vLV8ngXq2cme5OdRtxzJZ429hN6JeRlBND0pLlNTsSr+EfxZHugBkb7l/VIbQ/GXnW NipL80sXqP+R82aGkkh/oMFiNZp+WIRa6fSt5W/0dK5LM86SmgNuISEFxnMp/ujmI4Ov WQVutI+LgGHVo5OK2zbOXyXbXTTmWEKnUCv0MWh+FLC7TXY7O8FL7khTGsWUm4RaeBJZ 9AOQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@kernel.org header.s=default header.b=Nus7n6ZQ; spf=pass (google.com: domain of sashal@kernel.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=sashal@kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from mail.kernel.org (mail.kernel.org. [198.145.29.99]) by mx.google.com with ESMTPS id o22si41201070pgb.584.2018.11.22.11.54.52 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 22 Nov 2018 11:54:52 -0800 (PST) Received-SPF: pass (google.com: domain of sashal@kernel.org designates 198.145.29.99 as permitted sender) client-ip=198.145.29.99; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=default header.b=Nus7n6ZQ; spf=pass (google.com: domain of sashal@kernel.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=sashal@kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from sasha-vm.mshome.net (unknown [37.142.5.207]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 169E820672; Thu, 22 Nov 2018 19:54:47 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1542916492; bh=y9AxcDCFbvc982k1P00ZUECBchNE/Mwbycwse3c4ZVY=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Nus7n6ZQurDFgxJm3Jzz8mFVVUlCsR9thWhTDeBT16jAiHcv5lgZrKeG7PzUDrLff jOgkqvNnriLj5bIRBsqndlZNZpi8A0R9wKZpP1I1t5IeGnOR8Agdsvfn2pzwtdUY3u Njk3VTNO5GK6noKdLFvS25X+aXDYldIIS9aZWamY= From: Sasha Levin To: stable@vger.kernel.org, linux-kernel@vger.kernel.org Cc: Michal Hocko , Balbir Singh , Mel Gorman , Pavel Tatashin , Oscar Salvador , Mike Rapoport , Aaron Lu , Joonsoo Kim , Byoungyoung Lee , "Dae R. Jeong" , Andrew Morton , Linus Torvalds , Sasha Levin , linux-mm@kvack.org Subject: [PATCH AUTOSEL 4.19 36/36] mm, page_alloc: check for max order in hot path Date: Thu, 22 Nov 2018 14:52:40 -0500 Message-Id: <20181122195240.13123-36-sashal@kernel.org> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20181122195240.13123-1-sashal@kernel.org> References: <20181122195240.13123-1-sashal@kernel.org> X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP From: Michal Hocko [ Upstream commit c63ae43ba53bc432b414fd73dd5f4b01fcb1ab43 ] Konstantin has noticed that kvmalloc might trigger the following warning: WARNING: CPU: 0 PID: 6676 at mm/vmstat.c:986 __fragmentation_index+0x54/0x60 [...] Call Trace: fragmentation_index+0x76/0x90 compaction_suitable+0x4f/0xf0 shrink_node+0x295/0x310 node_reclaim+0x205/0x250 get_page_from_freelist+0x649/0xad0 __alloc_pages_nodemask+0x12a/0x2a0 kmalloc_large_node+0x47/0x90 __kmalloc_node+0x22b/0x2e0 kvmalloc_node+0x3e/0x70 xt_alloc_table_info+0x3a/0x80 [x_tables] do_ip6t_set_ctl+0xcd/0x1c0 [ip6_tables] nf_setsockopt+0x44/0x60 SyS_setsockopt+0x6f/0xc0 do_syscall_64+0x67/0x120 entry_SYSCALL_64_after_hwframe+0x3d/0xa2 the problem is that we only check for an out of bound order in the slow path and the node reclaim might happen from the fast path already. This is fixable by making sure that kvmalloc doesn't ever use kmalloc for requests that are larger than KMALLOC_MAX_SIZE but this also shows that the code is rather fragile. A recent UBSAN report just underlines that by the following report UBSAN: Undefined behaviour in mm/page_alloc.c:3117:19 shift exponent 51 is too large for 32-bit type 'int' CPU: 0 PID: 6520 Comm: syz-executor1 Not tainted 4.19.0-rc2 #1 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Bochs 01/01/2011 Call Trace: __dump_stack lib/dump_stack.c:77 [inline] dump_stack+0xd2/0x148 lib/dump_stack.c:113 ubsan_epilogue+0x12/0x94 lib/ubsan.c:159 __ubsan_handle_shift_out_of_bounds+0x2b6/0x30b lib/ubsan.c:425 __zone_watermark_ok+0x2c7/0x400 mm/page_alloc.c:3117 zone_watermark_fast mm/page_alloc.c:3216 [inline] get_page_from_freelist+0xc49/0x44c0 mm/page_alloc.c:3300 __alloc_pages_nodemask+0x21e/0x640 mm/page_alloc.c:4370 alloc_pages_current+0xcc/0x210 mm/mempolicy.c:2093 alloc_pages include/linux/gfp.h:509 [inline] __get_free_pages+0x12/0x60 mm/page_alloc.c:4414 dma_mem_alloc+0x36/0x50 arch/x86/include/asm/floppy.h:156 raw_cmd_copyin drivers/block/floppy.c:3159 [inline] raw_cmd_ioctl drivers/block/floppy.c:3206 [inline] fd_locked_ioctl+0xa00/0x2c10 drivers/block/floppy.c:3544 fd_ioctl+0x40/0x60 drivers/block/floppy.c:3571 __blkdev_driver_ioctl block/ioctl.c:303 [inline] blkdev_ioctl+0xb3c/0x1a30 block/ioctl.c:601 block_ioctl+0x105/0x150 fs/block_dev.c:1883 vfs_ioctl fs/ioctl.c:46 [inline] do_vfs_ioctl+0x1c0/0x1150 fs/ioctl.c:687 ksys_ioctl+0x9e/0xb0 fs/ioctl.c:702 __do_sys_ioctl fs/ioctl.c:709 [inline] __se_sys_ioctl fs/ioctl.c:707 [inline] __x64_sys_ioctl+0x7e/0xc0 fs/ioctl.c:707 do_syscall_64+0xc4/0x510 arch/x86/entry/common.c:290 entry_SYSCALL_64_after_hwframe+0x49/0xbe Note that this is not a kvmalloc path. It is just that the fast path really depends on having sanitzed order as well. Therefore move the order check to the fast path. Link: http://lkml.kernel.org/r/20181113094305.GM15120@dhcp22.suse.cz Signed-off-by: Michal Hocko Reported-by: Konstantin Khlebnikov Reported-by: Kyungtae Kim Acked-by: Vlastimil Babka Cc: Balbir Singh Cc: Mel Gorman Cc: Pavel Tatashin Cc: Oscar Salvador Cc: Mike Rapoport Cc: Aaron Lu Cc: Joonsoo Kim Cc: Byoungyoung Lee Cc: "Dae R. Jeong" Signed-off-by: Andrew Morton Signed-off-by: Linus Torvalds Signed-off-by: Sasha Levin --- mm/page_alloc.c | 20 +++++++++----------- 1 file changed, 9 insertions(+), 11 deletions(-) diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 3a4065312938..b721631d78ab 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -4055,17 +4055,6 @@ __alloc_pages_slowpath(gfp_t gfp_mask, unsigned int order, unsigned int cpuset_mems_cookie; int reserve_flags; - /* - * In the slowpath, we sanity check order to avoid ever trying to - * reclaim >= MAX_ORDER areas which will never succeed. Callers may - * be using allocators in order of preference for an area that is - * too large. - */ - if (order >= MAX_ORDER) { - WARN_ON_ONCE(!(gfp_mask & __GFP_NOWARN)); - return NULL; - } - /* * We also sanity check to catch abuse of atomic reserves being used by * callers that are not in atomic context. @@ -4359,6 +4348,15 @@ __alloc_pages_nodemask(gfp_t gfp_mask, unsigned int order, int preferred_nid, gfp_t alloc_mask; /* The gfp_t that was actually used for allocation */ struct alloc_context ac = { }; + /* + * There are several places where we assume that the order value is sane + * so bail out early if the request is out of bound. + */ + if (unlikely(order >= MAX_ORDER)) { + WARN_ON_ONCE(!(gfp_mask & __GFP_NOWARN)); + return NULL; + } + gfp_mask &= gfp_allowed_mask; alloc_mask = gfp_mask; if (!prepare_alloc_pages(gfp_mask, order, preferred_nid, nodemask, &ac, &alloc_mask, &alloc_flags))