From patchwork Mon Jun 24 22:29:46 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Song Liu X-Patchwork-Id: 11014269 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 9E15C1398 for ; Mon, 24 Jun 2019 22:30:07 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 8B44828AAE for ; Mon, 24 Jun 2019 22:30:07 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 7F12D28AC0; Mon, 24 Jun 2019 22:30:07 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id D717628AAE for ; Mon, 24 Jun 2019 22:30:05 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 0CAFD6B0006; Mon, 24 Jun 2019 18:30:02 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id F1C748E0003; Mon, 24 Jun 2019 18:30:01 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D48838E0002; Mon, 24 Jun 2019 18:30:01 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-pg1-f198.google.com (mail-pg1-f198.google.com [209.85.215.198]) by kanga.kvack.org (Postfix) with ESMTP id 8472B6B0006 for ; Mon, 24 Jun 2019 18:30:01 -0400 (EDT) Received: by mail-pg1-f198.google.com with SMTP id k136so3505859pgc.10 for ; Mon, 24 Jun 2019 15:30:01 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:smtp-origin-hostprefix:from :smtp-origin-hostname:to:cc:smtp-origin-cluster:subject:date :message-id:in-reply-to:references:mime-version; bh=EqtD9pgjSn3RpeCA8R1A7DD1Wom8eSWCEiXrtl71yV4=; b=Z4QUr2JWt+ZPJEZ0dxC4WKWDOnKU7fGjD+mT1S2kvztPZ+eBxwC8caH5t6VqYlQij3 w+MOMaSJTUe9lZI/Rh4dUxOoslCq37MX8HCJI4QZh81icbgQNyEuu3Q6Yrdh8UuuLdLG SrZdkeb5yaGSjG6DXm0m3r05PfkkEoDw6t70eaG5RQOt4rUuNZBSCwcrtayHx/otcr5Y pv6cKZSPh8gN/YLgrYfW53Rhn0ZzNZKEU/sqB81dtB/huwe490wsXcBhyVR0yB7ktstH O1hdlo367CKZ4orC13XqDQKB3TWeOME+pxVeztWdBF80NPNXtV6LSdaTRb0eVOdtBAvm Y6Ww== X-Gm-Message-State: APjAAAVe9HJ6Nf7np/9A/QhpPMmKixkjLYWr3l5q+q4uQjojhJC9O4SX hcUCaaAclFbm3g0MRD4ujgTd+Ue/dX9QtcZFImz7sCb09lxylVPl3zB/+aQ7SWqgh79eiDlz7MB VwPgQb+uHFs4byEzxoJt51o8Cq/P6X/B45IVtcxeYle2U/Gg1E82Qr4KGBRRLNzabtw== X-Received: by 2002:a63:8ac3:: with SMTP id y186mr28724964pgd.198.1561415400953; Mon, 24 Jun 2019 15:30:00 -0700 (PDT) X-Google-Smtp-Source: APXvYqx+3ESpxNM5pK04K/v4exAT8IeQE5l/q5zuSyVPU4vhaErHGPvt0/cfJQsjC9ax8RW75PdX X-Received: by 2002:a63:8ac3:: with SMTP id y186mr28724897pgd.198.1561415399994; Mon, 24 Jun 2019 15:29:59 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1561415399; cv=none; d=google.com; s=arc-20160816; b=DFrDdPkR00LOIbxZnWxBtqBSS60H02lFcg8qxWCb0ko3LMSXo/S4lsp8CVvRbgqn55 A8mYxY9dMPD0EBH9RccJRdW08Jjc/tKeLbWU/NEQ1FInKe0ufmmsJNZColVMIvPkbsXb 7uEicYv6qpSLD8CQTdQF++Lo5MFp0WVpRT69C1IhxEGfY6aoRJnhLp23VNHJyJHsdSM9 MOT7P3et6qk+NfYtjeY0YG8Q6rUgrUlNDs/nr7ug6dx7FRz7FvrvTCmjOuvTnXyBW6l1 QefjCvgBjLAEEUzd4SDSoHL5EC7sPnz0HySAA468d8/asb3IpifYDAhBhlfeSbbLx2PI gNIg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=mime-version:references:in-reply-to:message-id:date:subject :smtp-origin-cluster:cc:to:smtp-origin-hostname:from :smtp-origin-hostprefix:dkim-signature; bh=EqtD9pgjSn3RpeCA8R1A7DD1Wom8eSWCEiXrtl71yV4=; b=kEUffcwddJaElI70kQL5cdqvlYZXGMCATw52kUgi+AzRBTO1kb+FuqAngxnF7OvvdI r8LWXWBDHCcUUhe+1HstNRtjQ5IrRgMRIqqVIFDovID52Fgd93UiNYPsaR51bp+PaWw/ jJ2zuDOfjEI/YvHzqhQgpODP7bv2ycqy3ai4Kc+9P0P6o+Yxzq3lIPJrYLsjxbxz29AZ qAsDkJlWMzMsG3a6FLR9a/HuYNgt/Lt6ajU/nYcO6QmwqfSMiqy/rOh26+kl8QvNu6fm wZaGZSvloKen+jrRHCcPIYJX5CuoEZ6gpfd24TpmX7VZV28XyoKnXfJkBya4P3ChkacC Izzg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@fb.com header.s=facebook header.b=N5uvdv1C; spf=pass (google.com: domain of prvs=1078cbd532=songliubraving@fb.com designates 67.231.145.42 as permitted sender) smtp.mailfrom="prvs=1078cbd532=songliubraving@fb.com"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=fb.com Received: from mx0a-00082601.pphosted.com (mx0a-00082601.pphosted.com. [67.231.145.42]) by mx.google.com with ESMTPS id t190si11760752pgd.191.2019.06.24.15.29.59 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 24 Jun 2019 15:29:59 -0700 (PDT) Received-SPF: pass (google.com: domain of prvs=1078cbd532=songliubraving@fb.com designates 67.231.145.42 as permitted sender) client-ip=67.231.145.42; Authentication-Results: mx.google.com; dkim=pass header.i=@fb.com header.s=facebook header.b=N5uvdv1C; spf=pass (google.com: domain of prvs=1078cbd532=songliubraving@fb.com designates 67.231.145.42 as permitted sender) smtp.mailfrom="prvs=1078cbd532=songliubraving@fb.com"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=fb.com Received: from pps.filterd (m0044008.ppops.net [127.0.0.1]) by mx0a-00082601.pphosted.com (8.16.0.27/8.16.0.27) with SMTP id x5OMJPxF026762 for ; Mon, 24 Jun 2019 15:29:59 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fb.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type; s=facebook; bh=EqtD9pgjSn3RpeCA8R1A7DD1Wom8eSWCEiXrtl71yV4=; b=N5uvdv1CGI9JHMSPHd8HFKa/zD9qNyZx+dmMTZHdCM8D2i+k27CqCYDz9eGVOIfss23x GNEiOBo6wKidr9D9bdiPSmhcjUPZKm+Rm+Gd38rFkf9Yw0S7HJyVokcVDVe+6OyU09yI mW4gqSXIq2/BoxckrA/n76U5Y7/cFYmnwLA= Received: from maileast.thefacebook.com ([163.114.130.16]) by mx0a-00082601.pphosted.com with ESMTP id 2tb3v00vqu-4 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NOT) for ; Mon, 24 Jun 2019 15:29:59 -0700 Received: from mx-out.facebook.com (2620:10d:c0a8:1b::d) by mail.thefacebook.com (2620:10d:c0a8:82::c) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.1713.5; Mon, 24 Jun 2019 15:29:58 -0700 Received: by devbig006.ftw2.facebook.com (Postfix, from userid 4523) id 0A56462E206E; Mon, 24 Jun 2019 15:29:58 -0700 (PDT) Smtp-Origin-Hostprefix: devbig From: Song Liu Smtp-Origin-Hostname: devbig006.ftw2.facebook.com To: , , CC: , , , , , , Song Liu Smtp-Origin-Cluster: ftw2c04 Subject: [PATCH v8 1/6] filemap: check compound_head(page)->mapping in filemap_fault() Date: Mon, 24 Jun 2019 15:29:46 -0700 Message-ID: <20190624222951.37076-2-songliubraving@fb.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20190624222951.37076-1-songliubraving@fb.com> References: <20190624222951.37076-1-songliubraving@fb.com> X-FB-Internal: Safe MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:,, definitions=2019-06-24_15:,, signatures=0 X-Proofpoint-Spam-Details: rule=fb_default_notspam policy=fb_default score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=926 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1906240176 X-FB-Internal: deliver X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP Currently, filemap_fault() avoids trace condition with truncate by checking page->mapping == mapping. This does not work for compound pages. This patch let it check compound_head(page)->mapping instead. Acked-by: Rik van Riel Signed-off-by: Song Liu --- mm/filemap.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/mm/filemap.c b/mm/filemap.c index df2006ba0cfa..f5b79a43946d 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -2517,7 +2517,7 @@ vm_fault_t filemap_fault(struct vm_fault *vmf) goto out_retry; /* Did it get truncated? */ - if (unlikely(page->mapping != mapping)) { + if (unlikely(compound_head(page)->mapping != mapping)) { unlock_page(page); put_page(page); goto retry_find; From patchwork Mon Jun 24 22:29:47 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Song Liu X-Patchwork-Id: 11014271 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id D34CD924 for ; Mon, 24 Jun 2019 22:30:08 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id C38E228AAE for ; Mon, 24 Jun 2019 22:30:08 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id B71E528AC0; Mon, 24 Jun 2019 22:30:08 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 4925728AAE for ; Mon, 24 Jun 2019 22:30:08 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 362956B0007; Mon, 24 Jun 2019 18:30:04 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 273538E0003; Mon, 24 Jun 2019 18:30:04 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 0C8BC8E0002; Mon, 24 Jun 2019 18:30:03 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-yb1-f198.google.com (mail-yb1-f198.google.com [209.85.219.198]) by kanga.kvack.org (Postfix) with ESMTP id D496A6B0007 for ; Mon, 24 Jun 2019 18:30:03 -0400 (EDT) Received: by mail-yb1-f198.google.com with SMTP id v83so17698346ybv.17 for ; Mon, 24 Jun 2019 15:30:03 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:smtp-origin-hostprefix:from :smtp-origin-hostname:to:cc:smtp-origin-cluster:subject:date :message-id:in-reply-to:references:mime-version; bh=z7orUW1w2AFawDH57ZTLvYALJSDsMI3HPUuppHQ+jCM=; b=kFezXImE62IAA1aivsjfuJdxRV49RqFoU1DT0iqVnNozYY0AGLZvzvSOzpdp4FORH0 s8O/0GDKnCYqgx/ZRPS+w0IkrLG7huPlytdj/fpNIX1kY/YGVnHTiNPwVYC840wxdk/B IzrmP9HV4tR9lmSRdaR3bSeQTD8FV5GF4KHFh3LAGG8nd2wZLv+h/Q/iECaKwKD0esip OjvLT2hRI9pTSfy8lHSiDAVqVsSY+It1lne57gyoo4JpEulTREIvZ6/LVRd2isNorkI+ kLEHhHUYFma3mByPcAFgr9rrzozPl8avyPrP6JS9Vnlg5tFFpGoIBFspkKqj1SI9c46m Pksg== X-Gm-Message-State: APjAAAWiZpam0ImZRmZ3r1X+g2jOX8816DxVALwtmbURGqZjJzCF+SRN +X5OQKlJreAjZVg9QQpcr2bFY3vuT7FrMaVnU1QxHDOt4wK/AXcATugrB32CLsg+3HdTALohPp7 Q56Eb6+d9tCPdgi+IfGzZPZIvFemK7jk6kNemnPtC+SvzJvMJnCs++CK4fFHRii9g4A== X-Received: by 2002:a25:d47:: with SMTP id 68mr14080832ybn.75.1561415403599; Mon, 24 Jun 2019 15:30:03 -0700 (PDT) X-Google-Smtp-Source: APXvYqwY1cpaI8DD/9pcoBCNhQGJVj9ZMMco0B17qRWx1zefRVk6rD4fQfrS6aPAffsiM8vY9/27 X-Received: by 2002:a25:d47:: with SMTP id 68mr14080813ybn.75.1561415403098; Mon, 24 Jun 2019 15:30:03 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1561415403; cv=none; d=google.com; s=arc-20160816; b=Q5TEdVaWSgKZAm1gVU0MJ/U3GVGAfOnb5+bBqKns6r1D/sFcHCUxL8H8bvPwIfQNph 8tp6uinSNlbgQwof3zZUrOuKtb0dx8BC+TNz87JwfBqQHu4tFSXEQkyg5RbNtKd4vS7R CIlN6nSbH/X7fBaEV5WHEhX7Jc1QrzFxuMR37yWNB+30+qf+8nbvgPKnQsy/8X1acIgE 5gHZQB2XgHdqjxHfJgrMVRa1VCv5je/DBYDQEggpum5fMR9eaAuZYcSKFkJTrqiafsPH I+NM7RmDQheQiN3JJdNeSAv9uLgtPZl2dkydyEFkcaeYZKMPXoTv0oKb6bM3OZZ3ysoP Z8eQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=mime-version:references:in-reply-to:message-id:date:subject :smtp-origin-cluster:cc:to:smtp-origin-hostname:from :smtp-origin-hostprefix:dkim-signature; bh=z7orUW1w2AFawDH57ZTLvYALJSDsMI3HPUuppHQ+jCM=; b=OeC2OHH+2vFqM2Dw+4SZoUZPjoW8Rhy8/3vmvOViDJPV+LB+3LCLZbBwdepTPYsMVy fFvBBPMJcyPNfnBsZkFqhw5Iddkg+Jvsqb2wPPBToGQFbiiTy4FcNZZRrXuFtIx6wabb B7ppd7eMxJF49j1KLojJvg6SeaBiEbLoW4kB5zLQmP2xZHdkktXazTHfnwztKEkDX2Vw uvZijfbEsnnekxh20mElJR3njBQxvVJ7aynOT+wzkTJCtjF1LWuUf5odR/RfpJlGFjge JZt6aWzylvtd1TAONHgF1VE4MGPXTm288oivckcoKT6Ly5GAPlsEAlE8wGLS9bQkDByK J9cw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@fb.com header.s=facebook header.b=UMhjMbAp; spf=pass (google.com: domain of prvs=1078cbd532=songliubraving@fb.com designates 67.231.153.30 as permitted sender) smtp.mailfrom="prvs=1078cbd532=songliubraving@fb.com"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=fb.com Received: from mx0b-00082601.pphosted.com (mx0b-00082601.pphosted.com. [67.231.153.30]) by mx.google.com with ESMTPS id 82si4104819ybs.371.2019.06.24.15.30.03 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 24 Jun 2019 15:30:03 -0700 (PDT) Received-SPF: pass (google.com: domain of prvs=1078cbd532=songliubraving@fb.com designates 67.231.153.30 as permitted sender) client-ip=67.231.153.30; Authentication-Results: mx.google.com; dkim=pass header.i=@fb.com header.s=facebook header.b=UMhjMbAp; spf=pass (google.com: domain of prvs=1078cbd532=songliubraving@fb.com designates 67.231.153.30 as permitted sender) smtp.mailfrom="prvs=1078cbd532=songliubraving@fb.com"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=fb.com Received: from pps.filterd (m0109332.ppops.net [127.0.0.1]) by mx0a-00082601.pphosted.com (8.16.0.27/8.16.0.27) with SMTP id x5OMIYN6021032 for ; Mon, 24 Jun 2019 15:30:02 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fb.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type; s=facebook; bh=z7orUW1w2AFawDH57ZTLvYALJSDsMI3HPUuppHQ+jCM=; b=UMhjMbApjLP862682QVi0t9i6FsW/AFuvXqUNekJR8oZ7bQomxw8jtbXKo8GTlXYQNCZ QiyGfJtQ+xyIcxdVKn0/+n0VsGO95URd2QkSCVmLhn++McKMeCjccY6/XsSMZ7tYK3t3 x3WKz7PGioAgn60vF8exXfTK6VIxBRFSyd8= Received: from mail.thefacebook.com (mailout.thefacebook.com [199.201.64.23]) by mx0a-00082601.pphosted.com with ESMTP id 2taxvpj5yd-2 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Mon, 24 Jun 2019 15:30:02 -0700 Received: from mx-out.facebook.com (2620:10d:c081:10::13) by mail.thefacebook.com (2620:10d:c081:35::129) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA) id 15.1.1713.5; Mon, 24 Jun 2019 15:30:00 -0700 Received: by devbig006.ftw2.facebook.com (Postfix, from userid 4523) id 897BB62E206E; Mon, 24 Jun 2019 15:30:00 -0700 (PDT) Smtp-Origin-Hostprefix: devbig From: Song Liu Smtp-Origin-Hostname: devbig006.ftw2.facebook.com To: , , CC: , , , , , , Song Liu Smtp-Origin-Cluster: ftw2c04 Subject: [PATCH v8 2/6] filemap: update offset check in filemap_fault() Date: Mon, 24 Jun 2019 15:29:47 -0700 Message-ID: <20190624222951.37076-3-songliubraving@fb.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20190624222951.37076-1-songliubraving@fb.com> References: <20190624222951.37076-1-songliubraving@fb.com> X-FB-Internal: Safe MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:,, definitions=2019-06-24_15:,, signatures=0 X-Proofpoint-Spam-Details: rule=fb_default_notspam policy=fb_default score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=797 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1906240176 X-FB-Internal: deliver X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP With THP, current check of offset: VM_BUG_ON_PAGE(page->index != offset, page); is no longer accurate. Update it to: VM_BUG_ON_PAGE(page_to_pgoff(page) != offset, page); Acked-by: Rik van Riel Signed-off-by: Song Liu --- mm/filemap.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/mm/filemap.c b/mm/filemap.c index f5b79a43946d..5f072a113535 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -2522,7 +2522,7 @@ vm_fault_t filemap_fault(struct vm_fault *vmf) put_page(page); goto retry_find; } - VM_BUG_ON_PAGE(page->index != offset, page); + VM_BUG_ON_PAGE(page_to_pgoff(page) != offset, page); /* * We have a locked page in the page cache, now we need to check From patchwork Mon Jun 24 22:29:48 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Song Liu X-Patchwork-Id: 11014281 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 7D46B1398 for ; Mon, 24 Jun 2019 22:30:21 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 6DE3728AAE for ; Mon, 24 Jun 2019 22:30:21 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 6268328AC0; Mon, 24 Jun 2019 22:30:21 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id C82E428AAE for ; Mon, 24 Jun 2019 22:30:20 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 1B33A6B000C; Mon, 24 Jun 2019 18:30:15 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 0C68C8E0003; Mon, 24 Jun 2019 18:30:15 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E59C18E0002; Mon, 24 Jun 2019 18:30:14 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-pl1-f198.google.com (mail-pl1-f198.google.com [209.85.214.198]) by kanga.kvack.org (Postfix) with ESMTP id 9DC146B000C for ; Mon, 24 Jun 2019 18:30:14 -0400 (EDT) Received: by mail-pl1-f198.google.com with SMTP id 91so8022131pla.7 for ; Mon, 24 Jun 2019 15:30:14 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:smtp-origin-hostprefix:from :smtp-origin-hostname:to:cc:smtp-origin-cluster:subject:date :message-id:in-reply-to:references:mime-version; bh=TFx9H37w6mZg+4Tg54qFCnWnzVBpXQ/B3y/2N1lbRw0=; b=QkYua390P/lSwcEnUQ5HmygnLO8GqvFh7kQiygDcsZzzjrksQLTQ33QUt9e34vOb64 8s7vW6Bqadwn0ewtYYD1vgQqH6FcZY4zR9i7pxG4ApZQqdOUw9tqo05Y2xCJ+pZRnceO JRz87hr08tRKyc+60Vi5Vj3pheWi5GMNT6IwJ7VumKBph5+PfYRadoEhtHyHFTDvpYzy rsfuAX5JauOu6DaImqj8RjrF57Q3CEZhDcVlEC2ADXuZQqdTGFtL5s1XN7s+2AUSrg01 lrKvXysMGwxznxDPu95a97MF/q9wuLeVA2I/MdsJ6rZcYn4CzaaL20oiHo79SteR6qc9 WiNQ== X-Gm-Message-State: APjAAAXEA1K82jU1n3vME1rWzQakd1H2GHSV4v9q++oLjZomMkCVB43H 0cLQm9MClrBYYH+mHa2MxzSrPr+jM2tj93AOHpwhDL0O9hI960a4wy/yhAbu/B0GXPKVL+nZC5J xX6xqz1C8XMKPyGJGzNkxy5gKTwztqgUPvUFh+mZBKHdOF41YB6lTyD1PEqEKhzhbZA== X-Received: by 2002:a63:1d2:: with SMTP id 201mr29717306pgb.232.1561415414119; Mon, 24 Jun 2019 15:30:14 -0700 (PDT) X-Google-Smtp-Source: APXvYqzzVvNJLRjUntwvwaj3Vo+MenCGzSP4fa9rhPFWFDOhAjMCYC3RLEIjRKVNFuBEsHCDKLic X-Received: by 2002:a63:1d2:: with SMTP id 201mr29717229pgb.232.1561415413260; Mon, 24 Jun 2019 15:30:13 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1561415413; cv=none; d=google.com; s=arc-20160816; b=owuZjBE5YcgjbWvMQbD7drVTkf0pwT+NaaoMt1SzvxDf+adXb8ykft2j/8vG9tuA+O 9dWUkhvTrM7faLIWU9+ivUhbpK/Jm4S5aecm3H+RgDOeF8XHQOFKdRh3VAHGHXSuSQeS CzAsS5zC6U8BgFEHM90LvAaJnEnQ+6E0lqvqySIphmSNyu7ZOsYBcOZgwCAlkJcrU4Zf CCFMLsIAkvtNfD2d5G5Mcc3Y+9U25R9IsrtLr7Rst5gv5/QaFTEPwsjgZ9kjL17QCaNP O7vSpk3FqTEGbnsPvCvK8vw+xsB5S3cbdyT5fxy1VViUXo9NVsERXxgP7oaBegVnBX6Z 4jvw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=mime-version:references:in-reply-to:message-id:date:subject :smtp-origin-cluster:cc:to:smtp-origin-hostname:from :smtp-origin-hostprefix:dkim-signature; bh=TFx9H37w6mZg+4Tg54qFCnWnzVBpXQ/B3y/2N1lbRw0=; b=kmd4tZ+zjIRvpopwiFFQ83fXr7zf+mpXHDkULsITJsq/SI8GGsGKH/G/Vi9iRy5LHC rdxnXKapMt8WqSPkOZunTaPS78MJ/BWP6L+stYrj1A245xRW3PiYppRGESmjufdy5acw 9KOaZkW74ksKQ0ReSAPIWG2Y2PUC09Au3gaxB8plzf4YzvwNsmSNfFU+4WFBrP5NnV/R pj6/PckUwdgg7FCk+SOuSDl1QhpOIEFy8nbr2+XyE6jIknF7RKQcKV146Xj8dgQu4aXP L7texVyP6xCqjT/X/pe6kA9AnQO6sDNRv5brdsdf6NZ+6DAHeyX6h35iMP7JOOn3GFUJ 0kiA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@fb.com header.s=facebook header.b=nGoZgA4b; spf=pass (google.com: domain of prvs=1078cbd532=songliubraving@fb.com designates 67.231.145.42 as permitted sender) smtp.mailfrom="prvs=1078cbd532=songliubraving@fb.com"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=fb.com Received: from mx0a-00082601.pphosted.com (mx0a-00082601.pphosted.com. [67.231.145.42]) by mx.google.com with ESMTPS id m11si691790pjk.22.2019.06.24.15.30.13 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 24 Jun 2019 15:30:13 -0700 (PDT) Received-SPF: pass (google.com: domain of prvs=1078cbd532=songliubraving@fb.com designates 67.231.145.42 as permitted sender) client-ip=67.231.145.42; Authentication-Results: mx.google.com; dkim=pass header.i=@fb.com header.s=facebook header.b=nGoZgA4b; spf=pass (google.com: domain of prvs=1078cbd532=songliubraving@fb.com designates 67.231.145.42 as permitted sender) smtp.mailfrom="prvs=1078cbd532=songliubraving@fb.com"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=fb.com Received: from pps.filterd (m0109333.ppops.net [127.0.0.1]) by mx0a-00082601.pphosted.com (8.16.0.27/8.16.0.27) with SMTP id x5OMFUoQ002405 for ; Mon, 24 Jun 2019 15:30:12 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fb.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type; s=facebook; bh=TFx9H37w6mZg+4Tg54qFCnWnzVBpXQ/B3y/2N1lbRw0=; b=nGoZgA4bFIxTxSJpW45LP2xb0Gl2w2/yr7YJrL5L7/39wHrN1xyutk3JKAalrkR+AjIW lHPkovThuQR53Zax67lXLt8JyUl90HvrSf3ZJPHPxfVemD4e5F1MW+D91DdJmCFCAMUv LPPap67YCLStiu+1yAGQWgtgjAAUM+zaW9I= Received: from mail.thefacebook.com (mailout.thefacebook.com [199.201.64.23]) by mx0a-00082601.pphosted.com with ESMTP id 2tb2vc94s3-6 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Mon, 24 Jun 2019 15:30:12 -0700 Received: from mx-out.facebook.com (2620:10d:c081:10::13) by mail.thefacebook.com (2620:10d:c081:35::127) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA) id 15.1.1713.5; Mon, 24 Jun 2019 15:30:09 -0700 Received: by devbig006.ftw2.facebook.com (Postfix, from userid 4523) id 4DC0162E206E; Mon, 24 Jun 2019 15:30:03 -0700 (PDT) Smtp-Origin-Hostprefix: devbig From: Song Liu Smtp-Origin-Hostname: devbig006.ftw2.facebook.com To: , , CC: , , , , , , Song Liu Smtp-Origin-Cluster: ftw2c04 Subject: [PATCH v8 3/6] mm,thp: stats for file backed THP Date: Mon, 24 Jun 2019 15:29:48 -0700 Message-ID: <20190624222951.37076-4-songliubraving@fb.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20190624222951.37076-1-songliubraving@fb.com> References: <20190624222951.37076-1-songliubraving@fb.com> X-FB-Internal: Safe MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:,, definitions=2019-06-24_15:,, signatures=0 X-Proofpoint-Spam-Details: rule=fb_default_notspam policy=fb_default score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1906240176 X-FB-Internal: deliver X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP In preparation for non-shmem THP, this patch adds a few stats and exposes them in /proc/meminfo, /sys/bus/node/devices//meminfo, and /proc//task//smaps. This patch is mostly a rewrite of Kirill A. Shutemov's earlier version: https://lkml.kernel.org/r/20170126115819.58875-5-kirill.shutemov@linux.intel.com/ Acked-by: Rik van Riel Signed-off-by: Song Liu --- drivers/base/node.c | 6 ++++++ fs/proc/meminfo.c | 4 ++++ fs/proc/task_mmu.c | 4 +++- include/linux/mmzone.h | 2 ++ mm/vmstat.c | 2 ++ 5 files changed, 17 insertions(+), 1 deletion(-) diff --git a/drivers/base/node.c b/drivers/base/node.c index 8598fcbd2a17..71ae2dc93489 100644 --- a/drivers/base/node.c +++ b/drivers/base/node.c @@ -426,6 +426,8 @@ static ssize_t node_read_meminfo(struct device *dev, "Node %d AnonHugePages: %8lu kB\n" "Node %d ShmemHugePages: %8lu kB\n" "Node %d ShmemPmdMapped: %8lu kB\n" + "Node %d FileHugePages: %8lu kB\n" + "Node %d FilePmdMapped: %8lu kB\n" #endif , nid, K(node_page_state(pgdat, NR_FILE_DIRTY)), @@ -451,6 +453,10 @@ static ssize_t node_read_meminfo(struct device *dev, nid, K(node_page_state(pgdat, NR_SHMEM_THPS) * HPAGE_PMD_NR), nid, K(node_page_state(pgdat, NR_SHMEM_PMDMAPPED) * + HPAGE_PMD_NR), + nid, K(node_page_state(pgdat, NR_FILE_THPS) * + HPAGE_PMD_NR), + nid, K(node_page_state(pgdat, NR_FILE_PMDMAPPED) * HPAGE_PMD_NR) #endif ); diff --git a/fs/proc/meminfo.c b/fs/proc/meminfo.c index 568d90e17c17..bac395fc11f9 100644 --- a/fs/proc/meminfo.c +++ b/fs/proc/meminfo.c @@ -136,6 +136,10 @@ static int meminfo_proc_show(struct seq_file *m, void *v) global_node_page_state(NR_SHMEM_THPS) * HPAGE_PMD_NR); show_val_kb(m, "ShmemPmdMapped: ", global_node_page_state(NR_SHMEM_PMDMAPPED) * HPAGE_PMD_NR); + show_val_kb(m, "FileHugePages: ", + global_node_page_state(NR_FILE_THPS) * HPAGE_PMD_NR); + show_val_kb(m, "FilePmdMapped: ", + global_node_page_state(NR_FILE_PMDMAPPED) * HPAGE_PMD_NR); #endif #ifdef CONFIG_CMA diff --git a/fs/proc/task_mmu.c b/fs/proc/task_mmu.c index 01d4eb0e6bd1..0360e3b2ba89 100644 --- a/fs/proc/task_mmu.c +++ b/fs/proc/task_mmu.c @@ -413,6 +413,7 @@ struct mem_size_stats { unsigned long lazyfree; unsigned long anonymous_thp; unsigned long shmem_thp; + unsigned long file_thp; unsigned long swap; unsigned long shared_hugetlb; unsigned long private_hugetlb; @@ -563,7 +564,7 @@ static void smaps_pmd_entry(pmd_t *pmd, unsigned long addr, else if (is_zone_device_page(page)) /* pass */; else - VM_BUG_ON_PAGE(1, page); + mss->file_thp += HPAGE_PMD_SIZE; smaps_account(mss, page, true, pmd_young(*pmd), pmd_dirty(*pmd), locked); } #else @@ -767,6 +768,7 @@ static void __show_smap(struct seq_file *m, const struct mem_size_stats *mss) SEQ_PUT_DEC(" kB\nLazyFree: ", mss->lazyfree); SEQ_PUT_DEC(" kB\nAnonHugePages: ", mss->anonymous_thp); SEQ_PUT_DEC(" kB\nShmemPmdMapped: ", mss->shmem_thp); + SEQ_PUT_DEC(" kB\nFilePmdMapped: ", mss->file_thp); SEQ_PUT_DEC(" kB\nShared_Hugetlb: ", mss->shared_hugetlb); seq_put_decimal_ull_width(m, " kB\nPrivate_Hugetlb: ", mss->private_hugetlb >> 10, 7); diff --git a/include/linux/mmzone.h b/include/linux/mmzone.h index 70394cabaf4e..827f9b777938 100644 --- a/include/linux/mmzone.h +++ b/include/linux/mmzone.h @@ -234,6 +234,8 @@ enum node_stat_item { NR_SHMEM, /* shmem pages (included tmpfs/GEM pages) */ NR_SHMEM_THPS, NR_SHMEM_PMDMAPPED, + NR_FILE_THPS, + NR_FILE_PMDMAPPED, NR_ANON_THPS, NR_UNSTABLE_NFS, /* NFS unstable pages */ NR_VMSCAN_WRITE, diff --git a/mm/vmstat.c b/mm/vmstat.c index fd7e16ca6996..6afc892a148a 100644 --- a/mm/vmstat.c +++ b/mm/vmstat.c @@ -1158,6 +1158,8 @@ const char * const vmstat_text[] = { "nr_shmem", "nr_shmem_hugepages", "nr_shmem_pmdmapped", + "nr_file_hugepages", + "nr_file_pmdmapped", "nr_anon_transparent_hugepages", "nr_unstable", "nr_vmscan_write", From patchwork Mon Jun 24 22:29:49 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Song Liu X-Patchwork-Id: 11014277 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id EB695924 for ; Mon, 24 Jun 2019 22:30:14 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id DA56B28AAE for ; Mon, 24 Jun 2019 22:30:14 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id CEC6B28AC0; Mon, 24 Jun 2019 22:30:14 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 5373F28AAE for ; Mon, 24 Jun 2019 22:30:14 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id E84186B0008; Mon, 24 Jun 2019 18:30:12 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id E0A828E0003; Mon, 24 Jun 2019 18:30:12 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C5E6E8E0002; Mon, 24 Jun 2019 18:30:12 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-pf1-f199.google.com (mail-pf1-f199.google.com [209.85.210.199]) by kanga.kvack.org (Postfix) with ESMTP id 8F8EF6B0008 for ; Mon, 24 Jun 2019 18:30:12 -0400 (EDT) Received: by mail-pf1-f199.google.com with SMTP id 6so10449503pfi.6 for ; Mon, 24 Jun 2019 15:30:12 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:smtp-origin-hostprefix:from :smtp-origin-hostname:to:cc:smtp-origin-cluster:subject:date :message-id:in-reply-to:references:mime-version; bh=Kr37aYRy2KacRNeWnEOl8WIgUYHSnWgZ3fV+borBCvY=; b=OqcT52xjeEHgMnee7nJgl11HMaz+aVFKU8hsmYV6GBrQS4bk7PQlgw2Fppp3eAPAAU uBDn544Cj0Ua7DCL9t97Nbt50wX0NCY+cVCFszd8hpqT1UrY0xPcJ5zhdPGhA/ey77F3 Unyi9q+66Y8TMKPfnRqDtk+jnOJiBpYfuddQwSh2JcuHSSlHnbz6qTtkub347vNGdYgM I/sBN4b6xuU1UuB+pnOSy6C67yf7i07h3KF9hbR/XHZUuJIL720hIG2OyKvv/uoAqnyF /Vq3pYuUkfkiMekv619iBSME79tSMSHGpN6OanAV8mgHB7RFcf8EfJsAY32G1LRlxJhz jiMQ== X-Gm-Message-State: APjAAAVKjtXWIlIkvknshUvJWyPeyXaWQ2LJzkEeG2rSaEDts2D9D8x6 mMo1b4TQg5+uCY1ygeOSVtyeDr1zR6CyfDdjKOYOcRWlpqepbQBhi2+D4JHjuf+m4XUdZkl4Wka MpjNb/mzlwJPtTzyFfYql4s9Bboiw/TUidhKCTpAHprnOq+9IBJjfTICo9nwB20STWg== X-Received: by 2002:a65:404a:: with SMTP id h10mr36007076pgp.262.1561415412178; Mon, 24 Jun 2019 15:30:12 -0700 (PDT) X-Google-Smtp-Source: APXvYqx8YLJQAF44qS6wFIh/3G+Kuh/GKsDCbrpZoQYKHO7KzxXCCqDwjFMxLMoyPV623x0eFj22 X-Received: by 2002:a65:404a:: with SMTP id h10mr36007015pgp.262.1561415411428; Mon, 24 Jun 2019 15:30:11 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1561415411; cv=none; d=google.com; s=arc-20160816; b=QxjIQjOvsSx9QffWhzcEUj6JTC9A7XSkg2UPKFS0P40mM07KdN7proZ+c65mpBmjBT qlPVBu/NjF3IRTGv2ZoLlH0LLdbY6+6CC/SLhoyqSmEbZzTyspyWAbOaGZLhxy0pI2NW 5b0+BzkW3rOLp30fI+ALl4/Hv0hTe7dWS7I63jgunbpbSbqKp82BaWFGssih+Uc/Wyos 2EKJl9TllGUn67lMQTGroHkwPglsJ99Ep27GagdDvhNfR3bi6D8B+RiyLswQcHDt1JUO 3pon6Qug013NCZjhOH1dxhmpZYYk1yLzdqcJDa1XyZzjQsU/j3e7x6F60UG2mMcY6s7o hxhA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=mime-version:references:in-reply-to:message-id:date:subject :smtp-origin-cluster:cc:to:smtp-origin-hostname:from :smtp-origin-hostprefix:dkim-signature; bh=Kr37aYRy2KacRNeWnEOl8WIgUYHSnWgZ3fV+borBCvY=; b=haaOOCeyRcfVQFrqwh3U2rTmJsotF4ddBWFEdsWxX/NZhpeWzHwnmGXfG6w8S5RdtG IpwNIGrwKImB3U+qn4jG3QJxfOP6A8/isX6BbGKn+RmQZEhR4JBv7qfmE13vY+eg4/Uj jF3JkGB2o3l+9ZiT23sNKwSUuGGsUN6B6ZuTfwzskdr864riyJl/upqCN05ZiFzpDt/c j9JTdWEoXdblDjRf1nAuWaSJWyOzdhkJ4aem6voZlWb4twc3jEFGngGN4FD4B9BHaiVu Ydsi2XP0pN2JqSMqRZGk6QU6M+t8hCZ9q1JwWE3WhCzvHnIhaE+Ze2nKldroVKtS3XEy hRDA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@fb.com header.s=facebook header.b=oHI2WN65; spf=pass (google.com: domain of prvs=1078cbd532=songliubraving@fb.com designates 67.231.145.42 as permitted sender) smtp.mailfrom="prvs=1078cbd532=songliubraving@fb.com"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=fb.com Received: from mx0a-00082601.pphosted.com (mx0a-00082601.pphosted.com. [67.231.145.42]) by mx.google.com with ESMTPS id a22si11138593pgw.60.2019.06.24.15.30.11 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 24 Jun 2019 15:30:11 -0700 (PDT) Received-SPF: pass (google.com: domain of prvs=1078cbd532=songliubraving@fb.com designates 67.231.145.42 as permitted sender) client-ip=67.231.145.42; Authentication-Results: mx.google.com; dkim=pass header.i=@fb.com header.s=facebook header.b=oHI2WN65; spf=pass (google.com: domain of prvs=1078cbd532=songliubraving@fb.com designates 67.231.145.42 as permitted sender) smtp.mailfrom="prvs=1078cbd532=songliubraving@fb.com"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=fb.com Received: from pps.filterd (m0044010.ppops.net [127.0.0.1]) by mx0a-00082601.pphosted.com (8.16.0.27/8.16.0.27) with SMTP id x5OMFof7015249 for ; Mon, 24 Jun 2019 15:30:11 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fb.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type; s=facebook; bh=Kr37aYRy2KacRNeWnEOl8WIgUYHSnWgZ3fV+borBCvY=; b=oHI2WN65t3TCnQDE//aJ/s9nxwaKSLmp8pfBq6Bq5WgQBS+03mEnzm2xT8XTwrotvIAT RErBC9VDCiAyM6aWLLXosrEdEea3qR5kI6PjmEadNZTbc7efeog0HLnU5pFdQng14eEa Mdl0JrumO1UlAS3iHYGFewHgzlbWTxxw/b0= Received: from maileast.thefacebook.com ([163.114.130.16]) by mx0a-00082601.pphosted.com with ESMTP id 2tb3cm9169-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NOT) for ; Mon, 24 Jun 2019 15:30:10 -0700 Received: from mx-out.facebook.com (2620:10d:c0a8:1b::d) by mail.thefacebook.com (2620:10d:c0a8:82::d) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.1713.5; Mon, 24 Jun 2019 15:30:09 -0700 Received: by devbig006.ftw2.facebook.com (Postfix, from userid 4523) id DDC9162E206E; Mon, 24 Jun 2019 15:30:05 -0700 (PDT) Smtp-Origin-Hostprefix: devbig From: Song Liu Smtp-Origin-Hostname: devbig006.ftw2.facebook.com To: , , CC: , , , , , , Song Liu Smtp-Origin-Cluster: ftw2c04 Subject: [PATCH v8 4/6] khugepaged: rename collapse_shmem() and khugepaged_scan_shmem() Date: Mon, 24 Jun 2019 15:29:49 -0700 Message-ID: <20190624222951.37076-5-songliubraving@fb.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20190624222951.37076-1-songliubraving@fb.com> References: <20190624222951.37076-1-songliubraving@fb.com> X-FB-Internal: Safe MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:,, definitions=2019-06-24_15:,, signatures=0 X-Proofpoint-Spam-Details: rule=fb_default_notspam policy=fb_default score=0 priorityscore=1501 malwarescore=0 suspectscore=2 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=627 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1906240176 X-FB-Internal: deliver X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP Next patch will add khugepaged support of non-shmem files. This patch renames these two functions to reflect the new functionality: collapse_shmem() => collapse_file() khugepaged_scan_shmem() => khugepaged_scan_file() Signed-off-by: Song Liu --- mm/khugepaged.c | 23 +++++++++++------------ 1 file changed, 11 insertions(+), 12 deletions(-) diff --git a/mm/khugepaged.c b/mm/khugepaged.c index 0f7419938008..158cad542627 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -1287,7 +1287,7 @@ static void retract_page_tables(struct address_space *mapping, pgoff_t pgoff) } /** - * collapse_shmem - collapse small tmpfs/shmem pages into huge one. + * collapse_file - collapse small tmpfs/shmem pages into huge one. * * Basic scheme is simple, details are more complex: * - allocate and lock a new huge page; @@ -1304,10 +1304,11 @@ static void retract_page_tables(struct address_space *mapping, pgoff_t pgoff) * + restore gaps in the page cache; * + unlock and free huge page; */ -static void collapse_shmem(struct mm_struct *mm, - struct address_space *mapping, pgoff_t start, +static void collapse_file(struct mm_struct *mm, + struct file *file, pgoff_t start, struct page **hpage, int node) { + struct address_space *mapping = file->f_mapping; gfp_t gfp; struct page *new_page; struct mem_cgroup *memcg; @@ -1563,11 +1564,11 @@ static void collapse_shmem(struct mm_struct *mm, /* TODO: tracepoints */ } -static void khugepaged_scan_shmem(struct mm_struct *mm, - struct address_space *mapping, - pgoff_t start, struct page **hpage) +static void khugepaged_scan_file(struct mm_struct *mm, + struct file *file, pgoff_t start, struct page **hpage) { struct page *page = NULL; + struct address_space *mapping = file->f_mapping; XA_STATE(xas, &mapping->i_pages, start); int present, swap; int node = NUMA_NO_NODE; @@ -1631,16 +1632,15 @@ static void khugepaged_scan_shmem(struct mm_struct *mm, result = SCAN_EXCEED_NONE_PTE; } else { node = khugepaged_find_target_node(); - collapse_shmem(mm, mapping, start, hpage, node); + collapse_file(mm, file, start, hpage, node); } } /* TODO: tracepoints */ } #else -static void khugepaged_scan_shmem(struct mm_struct *mm, - struct address_space *mapping, - pgoff_t start, struct page **hpage) +static void khugepaged_scan_file(struct mm_struct *mm, + struct file *file, pgoff_t start, struct page **hpage) { BUILD_BUG(); } @@ -1722,8 +1722,7 @@ static unsigned int khugepaged_scan_mm_slot(unsigned int pages, file = get_file(vma->vm_file); up_read(&mm->mmap_sem); ret = 1; - khugepaged_scan_shmem(mm, file->f_mapping, - pgoff, hpage); + khugepaged_scan_file(mm, file, pgoff, hpage); fput(file); } else { ret = khugepaged_scan_pmd(mm, vma, From patchwork Mon Jun 24 22:29:50 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Song Liu X-Patchwork-Id: 11014279 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 949E8924 for ; Mon, 24 Jun 2019 22:30:18 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 839B228AAE for ; Mon, 24 Jun 2019 22:30:18 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 76FDF28AC0; Mon, 24 Jun 2019 22:30:18 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 8DAD128AAE for ; Mon, 24 Jun 2019 22:30:17 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 55A266B000A; Mon, 24 Jun 2019 18:30:13 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 4E3C18E0003; Mon, 24 Jun 2019 18:30:13 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 335568E0002; Mon, 24 Jun 2019 18:30:13 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-yb1-f200.google.com (mail-yb1-f200.google.com [209.85.219.200]) by kanga.kvack.org (Postfix) with ESMTP id 06E726B000A for ; Mon, 24 Jun 2019 18:30:13 -0400 (EDT) Received: by mail-yb1-f200.google.com with SMTP id o17so7189485yba.13 for ; Mon, 24 Jun 2019 15:30:13 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:smtp-origin-hostprefix:from :smtp-origin-hostname:to:cc:smtp-origin-cluster:subject:date :message-id:in-reply-to:references:mime-version; bh=Rw3bF3LLOPMJXUJryh7BDppGG/fMUSMU6vd9No970FE=; b=JQlUdD/tJ9bqCAE6xSOEowC/nHFO58dJ0GMcFviWnW12v6xOjZoC5UGzp04U6+h7Lo 4qrw2+x8U44+nQlUpDWwxk0mYUSDtcF8ghNyGSQdHaKLZ3yr2Sg3hy31kddZrxevtszg jkUyw9LRSWH58TFOj3uRofMGX0jzP8Gxgvq0qjBGIDSGuahyvPHau579wB2h0PeWDGvO Hv9yoX2NzjuKFatJ3fPskuE7p5yN4yLNLIzNC6i6Pc4ZZCPt9l2wy3JHQsW/nloaVUnw XvQ4/DFLHEFLROe4u7SyiFntL8kAgZo8AY7Io1cvZ+HqEVOqe0YkgNVEtKoZbaMZ7VK5 qr5g== X-Gm-Message-State: APjAAAVnZM0pc7awxaErzpGpaKq4GtKoHEbIpHikclJ3XzpEyi3JgHdj cMUjrnEz8NnNUZRtFqTG/suuB1fw6d7eJ3clTcb534io0+kTs1oZ2bVOF0kj4v89Sp/rzLBaWgf /C2B8p7NHNjH0TBX8kgqv88VVymG62LvcDFemxGeqZOo+Phdvjbd1yeEdeZPv0ZsJ2Q== X-Received: by 2002:a25:8711:: with SMTP id a17mr78160410ybl.456.1561415412744; Mon, 24 Jun 2019 15:30:12 -0700 (PDT) X-Google-Smtp-Source: APXvYqxJfK2Hr3Fy4g5kqbSgn2f8pgNSSp75vomN8APfcLSGY0kLjLUvlSmkGoD96MwrcxK0EpRE X-Received: by 2002:a25:8711:: with SMTP id a17mr78160366ybl.456.1561415411891; Mon, 24 Jun 2019 15:30:11 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1561415411; cv=none; d=google.com; s=arc-20160816; b=xV+ucMOrurOiqChc1RZLiS0Guy0bnXidxYWxMSlSYRfNWU5F1CZrA9ARUw0tnEItHs xsNOTdgE7+bjxyMj3/Ohx17iJuJE+S0DR2KhpZmECJhOz50ifCfCgxLCT/RIPVayf0Xp M4RA5urt2hyKfPjjOfpFKpF1BF2yHAGhjRbj8sxSuSMC2Awvmm1+yoe0ilb5UWUBCzur LhZ/lYlk9ba6dO8FVCzpjxF1Vfl14U4gvuzilMxzdva19Xr0Zau/sowpYhDwViY/VAwD AibdfcMtQTad+7dngrfP/nd0AtY7aErKzKtsQtFfj4bS93vKApeTdEqyqs6jFm5VhpV1 yq8A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=mime-version:references:in-reply-to:message-id:date:subject :smtp-origin-cluster:cc:to:smtp-origin-hostname:from :smtp-origin-hostprefix:dkim-signature; bh=Rw3bF3LLOPMJXUJryh7BDppGG/fMUSMU6vd9No970FE=; b=b7THIOi8z0MTvH2EaveJE7lRWotbRW8SvQBK6zkJhKaUX2qCYfS2g6FtoFdbJBP3J5 JLMuHjt75OUj1fR9xeS+Xv2/TKgTLUSx7JW1WWlbJOGA8pVJhNMzWedAuxMpUzhmAVyT rV2xVt5kSD6owsiU84C315POwjigZM+JOUx4B0frQNqktsnu7oyOq+CHt8RtlKSV+q68 K8L6VvtKWLJGibfl7Z0w0Y0Sepny8HaFKYn/GwPLclRzqsIiZlJwKNKjKJpNItU9DmYM 79eHenGef9V40RO9hmaiFRUesMAMq2fTPXK93mYuKTLLq371vxqfWj1XhL+Y1FkO6v/q wwKQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@fb.com header.s=facebook header.b=Jwy8GSjC; spf=pass (google.com: domain of prvs=1078cbd532=songliubraving@fb.com designates 67.231.153.30 as permitted sender) smtp.mailfrom="prvs=1078cbd532=songliubraving@fb.com"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=fb.com Received: from mx0a-00082601.pphosted.com (mx0b-00082601.pphosted.com. [67.231.153.30]) by mx.google.com with ESMTPS id d3si4735882ybf.451.2019.06.24.15.30.11 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 24 Jun 2019 15:30:11 -0700 (PDT) Received-SPF: pass (google.com: domain of prvs=1078cbd532=songliubraving@fb.com designates 67.231.153.30 as permitted sender) client-ip=67.231.153.30; Authentication-Results: mx.google.com; dkim=pass header.i=@fb.com header.s=facebook header.b=Jwy8GSjC; spf=pass (google.com: domain of prvs=1078cbd532=songliubraving@fb.com designates 67.231.153.30 as permitted sender) smtp.mailfrom="prvs=1078cbd532=songliubraving@fb.com"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=fb.com Received: from pps.filterd (m0089730.ppops.net [127.0.0.1]) by m0089730.ppops.net (8.16.0.27/8.16.0.27) with SMTP id x5OMItOx008134 for ; Mon, 24 Jun 2019 15:30:11 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fb.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type; s=facebook; bh=Rw3bF3LLOPMJXUJryh7BDppGG/fMUSMU6vd9No970FE=; b=Jwy8GSjCz6tJtVLMP/bG6sFn79YOWcADOe0SRp8LclM0hhi3eG6FIfrSe8iemnvpARZy 7SLA5+O8EE+P8fhmk3TdESYxAa2FiASuNEvp3wTBEZ0EG3bz/yqV185Gqh9mnLxpnaC9 wTanwUphnm4zgE8p15zr82gdyuFFazGrWF0= Received: from maileast.thefacebook.com ([163.114.130.16]) by m0089730.ppops.net with ESMTP id 2tb3gw8y46-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NOT) for ; Mon, 24 Jun 2019 15:30:11 -0700 Received: from mx-out.facebook.com (2620:10d:c0a8:1b::d) by mail.thefacebook.com (2620:10d:c0a8:83::4) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.1713.5; Mon, 24 Jun 2019 15:30:10 -0700 Received: by devbig006.ftw2.facebook.com (Postfix, from userid 4523) id 2EFAF62E206E; Mon, 24 Jun 2019 15:30:09 -0700 (PDT) Smtp-Origin-Hostprefix: devbig From: Song Liu Smtp-Origin-Hostname: devbig006.ftw2.facebook.com To: , , CC: , , , , , , Song Liu Smtp-Origin-Cluster: ftw2c04 Subject: [PATCH v8 5/6] mm,thp: add read-only THP support for (non-shmem) FS Date: Mon, 24 Jun 2019 15:29:50 -0700 Message-ID: <20190624222951.37076-6-songliubraving@fb.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20190624222951.37076-1-songliubraving@fb.com> References: <20190624222951.37076-1-songliubraving@fb.com> X-FB-Internal: Safe MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:,, definitions=2019-06-24_15:,, signatures=0 X-Proofpoint-Spam-Details: rule=fb_default_notspam policy=fb_default score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1906240176 X-FB-Internal: deliver X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP This patch is (hopefully) the first step to enable THP for non-shmem filesystems. This patch enables an application to put part of its text sections to THP via madvise, for example: madvise((void *)0x600000, 0x200000, MADV_HUGEPAGE); We tried to reuse the logic for THP on tmpfs. Currently, write is not supported for non-shmem THP. khugepaged will only process vma with VM_DENYWRITE. sys_mmap() ignores VM_DENYWRITE requests (see ksys_mmap_pgoff). The only way to create vma with VM_DENYWRITE is execve(). This requirement limits non-shmem THP to text sections. The next patch will handle writes, which would only happen when the all the vmas with VM_DENYWRITE are unmapped. An EXPERIMENTAL config, READ_ONLY_THP_FOR_FS, is added to gate this feature. Acked-by: Rik van Riel Signed-off-by: Song Liu --- mm/Kconfig | 11 ++++++ mm/filemap.c | 4 +-- mm/khugepaged.c | 94 +++++++++++++++++++++++++++++++++++++++++-------- mm/rmap.c | 12 ++++--- 4 files changed, 100 insertions(+), 21 deletions(-) diff --git a/mm/Kconfig b/mm/Kconfig index f0c76ba47695..0a8fd589406d 100644 --- a/mm/Kconfig +++ b/mm/Kconfig @@ -762,6 +762,17 @@ config GUP_BENCHMARK See tools/testing/selftests/vm/gup_benchmark.c +config READ_ONLY_THP_FOR_FS + bool "Read-only THP for filesystems (EXPERIMENTAL)" + depends on TRANSPARENT_HUGE_PAGECACHE && SHMEM + + help + Allow khugepaged to put read-only file-backed pages in THP. + + This is marked experimental because it is a new feature. Write + support of file THPs will be developed in the next few release + cycles. + config ARCH_HAS_PTE_SPECIAL bool diff --git a/mm/filemap.c b/mm/filemap.c index 5f072a113535..e79ceccdc6df 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -203,8 +203,8 @@ static void unaccount_page_cache_page(struct address_space *mapping, __mod_node_page_state(page_pgdat(page), NR_SHMEM, -nr); if (PageTransHuge(page)) __dec_node_page_state(page, NR_SHMEM_THPS); - } else { - VM_BUG_ON_PAGE(PageTransHuge(page), page); + } else if (PageTransHuge(page)) { + __dec_node_page_state(page, NR_FILE_THPS); } /* diff --git a/mm/khugepaged.c b/mm/khugepaged.c index 158cad542627..acbbbeaa083c 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -48,6 +48,7 @@ enum scan_result { SCAN_CGROUP_CHARGE_FAIL, SCAN_EXCEED_SWAP_PTE, SCAN_TRUNCATED, + SCAN_PAGE_HAS_PRIVATE, }; #define CREATE_TRACE_POINTS @@ -404,7 +405,11 @@ static bool hugepage_vma_check(struct vm_area_struct *vma, (vm_flags & VM_NOHUGEPAGE) || test_bit(MMF_DISABLE_THP, &vma->vm_mm->flags)) return false; - if (shmem_file(vma->vm_file)) { + + if (shmem_file(vma->vm_file) || + (IS_ENABLED(CONFIG_READ_ONLY_THP_FOR_FS) && + vma->vm_file && + (vm_flags & VM_DENYWRITE))) { if (!IS_ENABLED(CONFIG_TRANSPARENT_HUGE_PAGECACHE)) return false; return IS_ALIGNED((vma->vm_start >> PAGE_SHIFT) - vma->vm_pgoff, @@ -456,8 +461,9 @@ int khugepaged_enter_vma_merge(struct vm_area_struct *vma, unsigned long hstart, hend; /* - * khugepaged does not yet work on non-shmem files or special - * mappings. And file-private shmem THP is not supported. + * khugepaged only supports read-only files for non-shmem files. + * khugepaged does not yet work on special mappings. And + * file-private shmem THP is not supported. */ if (!hugepage_vma_check(vma, vm_flags)) return 0; @@ -1287,12 +1293,12 @@ static void retract_page_tables(struct address_space *mapping, pgoff_t pgoff) } /** - * collapse_file - collapse small tmpfs/shmem pages into huge one. + * collapse_file - collapse filemap/tmpfs/shmem pages into huge one. * * Basic scheme is simple, details are more complex: * - allocate and lock a new huge page; * - scan page cache replacing old pages with the new one - * + swap in pages if necessary; + * + swap/gup in pages if necessary; * + fill in gaps; * + keep old pages around in case rollback is required; * - if replacing succeeds: @@ -1316,7 +1322,9 @@ static void collapse_file(struct mm_struct *mm, LIST_HEAD(pagelist); XA_STATE_ORDER(xas, &mapping->i_pages, start, HPAGE_PMD_ORDER); int nr_none = 0, result = SCAN_SUCCEED; + bool is_shmem = shmem_file(file); + VM_BUG_ON(!IS_ENABLED(CONFIG_READ_ONLY_THP_FOR_FS) && !is_shmem); VM_BUG_ON(start & (HPAGE_PMD_NR - 1)); /* Only allocate from the target node */ @@ -1348,7 +1356,8 @@ static void collapse_file(struct mm_struct *mm, } while (1); __SetPageLocked(new_page); - __SetPageSwapBacked(new_page); + if (is_shmem) + __SetPageSwapBacked(new_page); new_page->index = start; new_page->mapping = mapping; @@ -1363,7 +1372,7 @@ static void collapse_file(struct mm_struct *mm, struct page *page = xas_next(&xas); VM_BUG_ON(index != xas.xa_index); - if (!page) { + if (is_shmem && !page) { /* * Stop if extent has been truncated or hole-punched, * and is now completely empty. @@ -1384,7 +1393,7 @@ static void collapse_file(struct mm_struct *mm, continue; } - if (xa_is_value(page) || !PageUptodate(page)) { + if (is_shmem && (xa_is_value(page) || !PageUptodate(page))) { xas_unlock_irq(&xas); /* swap in or instantiate fallocated page */ if (shmem_getpage(mapping->host, index, &page, @@ -1392,6 +1401,29 @@ static void collapse_file(struct mm_struct *mm, result = SCAN_FAIL; goto xa_unlocked; } + } else if (!page || xa_is_value(page)) { + xas_unlock_irq(&xas); + page_cache_sync_readahead(mapping, &file->f_ra, file, + index, PAGE_SIZE); + /* drain pagevecs to help isolate_lru_page() */ + lru_add_drain(); + page = find_lock_page(mapping, index); + if (unlikely(page == NULL)) { + result = SCAN_FAIL; + goto xa_unlocked; + } + } else if (!PageUptodate(page)) { + VM_BUG_ON(is_shmem); + xas_unlock_irq(&xas); + wait_on_page_locked(page); + if (!trylock_page(page)) { + result = SCAN_PAGE_LOCK; + goto xa_unlocked; + } + get_page(page); + } else if (!is_shmem && PageDirty(page)) { + result = SCAN_FAIL; + goto xa_locked; } else if (trylock_page(page)) { get_page(page); xas_unlock_irq(&xas); @@ -1426,6 +1458,12 @@ static void collapse_file(struct mm_struct *mm, goto out_unlock; } + if (page_has_private(page) && + !try_to_release_page(page, GFP_KERNEL)) { + result = SCAN_PAGE_HAS_PRIVATE; + break; + } + if (page_mapped(page)) unmap_mapping_pages(mapping, index, 1, false); @@ -1463,12 +1501,18 @@ static void collapse_file(struct mm_struct *mm, goto xa_unlocked; } - __inc_node_page_state(new_page, NR_SHMEM_THPS); + if (is_shmem) + __inc_node_page_state(new_page, NR_SHMEM_THPS); + else + __inc_node_page_state(new_page, NR_FILE_THPS); + if (nr_none) { struct zone *zone = page_zone(new_page); __mod_node_page_state(zone->zone_pgdat, NR_FILE_PAGES, nr_none); - __mod_node_page_state(zone->zone_pgdat, NR_SHMEM, nr_none); + if (is_shmem) + __mod_node_page_state(zone->zone_pgdat, + NR_SHMEM, nr_none); } xa_locked: @@ -1506,10 +1550,15 @@ static void collapse_file(struct mm_struct *mm, SetPageUptodate(new_page); page_ref_add(new_page, HPAGE_PMD_NR - 1); - set_page_dirty(new_page); mem_cgroup_commit_charge(new_page, memcg, false, true); + + if (is_shmem) { + set_page_dirty(new_page); + lru_cache_add_anon(new_page); + } else { + lru_cache_add_file(new_page); + } count_memcg_events(memcg, THP_COLLAPSE_ALLOC, 1); - lru_cache_add_anon(new_page); /* * Remove pte page tables, so we can re-fault the page as huge. @@ -1524,7 +1573,9 @@ static void collapse_file(struct mm_struct *mm, /* Something went wrong: roll back page cache changes */ xas_lock_irq(&xas); mapping->nrpages -= nr_none; - shmem_uncharge(mapping->host, nr_none); + + if (is_shmem) + shmem_uncharge(mapping->host, nr_none); xas_set(&xas, start); xas_for_each(&xas, page, end - 1) { @@ -1607,6 +1658,17 @@ static void khugepaged_scan_file(struct mm_struct *mm, break; } + if (page_has_private(page) && trylock_page(page)) { + int ret; + + ret = try_to_release_page(page, GFP_KERNEL); + unlock_page(page); + if (!ret) { + result = SCAN_PAGE_HAS_PRIVATE; + break; + } + } + if (page_count(page) != 1 + page_mapcount(page)) { result = SCAN_PAGE_COUNT; break; @@ -1713,11 +1775,13 @@ static unsigned int khugepaged_scan_mm_slot(unsigned int pages, VM_BUG_ON(khugepaged_scan.address < hstart || khugepaged_scan.address + HPAGE_PMD_SIZE > hend); - if (shmem_file(vma->vm_file)) { + if (vma->vm_file) { struct file *file; pgoff_t pgoff = linear_page_index(vma, khugepaged_scan.address); - if (!shmem_huge_enabled(vma)) + + if (shmem_file(vma->vm_file) + && !shmem_huge_enabled(vma)) goto skip; file = get_file(vma->vm_file); up_read(&mm->mmap_sem); diff --git a/mm/rmap.c b/mm/rmap.c index e5dfe2ae6b0d..87cfa2c19eda 100644 --- a/mm/rmap.c +++ b/mm/rmap.c @@ -1192,8 +1192,10 @@ void page_add_file_rmap(struct page *page, bool compound) } if (!atomic_inc_and_test(compound_mapcount_ptr(page))) goto out; - VM_BUG_ON_PAGE(!PageSwapBacked(page), page); - __inc_node_page_state(page, NR_SHMEM_PMDMAPPED); + if (PageSwapBacked(page)) + __inc_node_page_state(page, NR_SHMEM_PMDMAPPED); + else + __inc_node_page_state(page, NR_FILE_PMDMAPPED); } else { if (PageTransCompound(page) && page_mapping(page)) { VM_WARN_ON_ONCE(!PageLocked(page)); @@ -1232,8 +1234,10 @@ static void page_remove_file_rmap(struct page *page, bool compound) } if (!atomic_add_negative(-1, compound_mapcount_ptr(page))) goto out; - VM_BUG_ON_PAGE(!PageSwapBacked(page), page); - __dec_node_page_state(page, NR_SHMEM_PMDMAPPED); + if (PageSwapBacked(page)) + __dec_node_page_state(page, NR_SHMEM_PMDMAPPED); + else + __dec_node_page_state(page, NR_FILE_PMDMAPPED); } else { if (!atomic_add_negative(-1, &page->_mapcount)) goto out; From patchwork Mon Jun 24 22:29:51 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Song Liu X-Patchwork-Id: 11014283 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id E06921398 for ; Mon, 24 Jun 2019 22:30:24 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id D043528AAE for ; Mon, 24 Jun 2019 22:30:24 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id C44A428AC0; Mon, 24 Jun 2019 22:30:24 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 1D50E28AAE for ; Mon, 24 Jun 2019 22:30:24 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D240C6B000D; Mon, 24 Jun 2019 18:30:18 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id C5EC18E0003; Mon, 24 Jun 2019 18:30:18 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A3C928E0002; Mon, 24 Jun 2019 18:30:18 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-yb1-f198.google.com (mail-yb1-f198.google.com [209.85.219.198]) by kanga.kvack.org (Postfix) with ESMTP id 6E4176B000D for ; Mon, 24 Jun 2019 18:30:18 -0400 (EDT) Received: by mail-yb1-f198.google.com with SMTP id z124so17765612ybz.15 for ; Mon, 24 Jun 2019 15:30:18 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:smtp-origin-hostprefix:from :smtp-origin-hostname:to:cc:smtp-origin-cluster:subject:date :message-id:in-reply-to:references:mime-version; bh=1YmphlJ3H8jOCDJ7k4G2S7xvfsjoBsLcepmZ6MADQ8Y=; b=B+2zDIjnGBXObk478UxlNziqm/Mh4PdWMVUCHUSoMkOwuWZbPKKbD0mwA+SUUwSkmL 40ufQRGzbGMuGCg+/gNIAiFLQFZnGU7BEr3h5OUqexRTXANmltsEKWTfM4oslY+rdrur deq7fLBwEavxdMzl8nV68DhCmn+A9mc83ilE5kCLFghHMc2XfQa+aLorJu3JJI2E9eMB lWln4yCVyiMOGIBU3HzhC5yi9KwYk6bUjvxLJmQM8JpRvcHtyhwGHEgLBYW1DtALSVdV FSn/h1pfV0yLkMpsrPaWYmfb/Z48SnG2BquU4AdVq5QTERn3BJ4eu8Eerir6QK/0B/Os lB9g== X-Gm-Message-State: APjAAAVKWvvtyiRA2Jw0aIFQEYVNnnW9nieFKmpM9lQdRvCxu7hJy5cj faQhHTorFwFEHYSqcdEH+8iD5zF3M3OyyL/hBFPouHCVs69hx5XPmmvYxlmcD2RDpp8YdqtB3e+ NPbkDSjaOwWzYqmAYJHw31xCdZyYVjb2GbxKly+uLAc8ghYBhL67an/y2o2AVbKFvMg== X-Received: by 2002:a25:6649:: with SMTP id z9mr35841467ybm.42.1561415418184; Mon, 24 Jun 2019 15:30:18 -0700 (PDT) X-Google-Smtp-Source: APXvYqw3zurfRyyB7Ov/0mIbkHbqBX9PR5JQBlPKCEgd8oYoM6WP0nHUFBvFZMxTApiYLp6C/wcY X-Received: by 2002:a25:6649:: with SMTP id z9mr35841426ybm.42.1561415417255; Mon, 24 Jun 2019 15:30:17 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1561415417; cv=none; d=google.com; s=arc-20160816; b=nIa3InT3Ayh5gY+uNYrct9Br1br1l20qWHPFFQUd314a3G8CXNfCLtja/K/WyqG86o OyYNK2YZLr/zNf9oqGkIYJpp3pBP+yCP6Xcf26Rz7fBKpgwmngDTw35o9hC+SKBBDIoL jWVUjyXUJTl8SmAOSxsrJVNhkQkzctaSScxTTcyT4VwuolV45xtxowMfT9jkh9QnRRwl r3hTDfXOhjHF6hLBhA6c0L2OuTD/zAar9w+QMTRrap6OvSZ4Fblxw3qp+Zeg38dBA26p 0DhTQXLkQw0wEggSSNjcBqZGaoGjc/SghMsX/LVR/UlHKdXFxyAGJLULtWO/q18V0dl6 toGQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=mime-version:references:in-reply-to:message-id:date:subject :smtp-origin-cluster:cc:to:smtp-origin-hostname:from :smtp-origin-hostprefix:dkim-signature; bh=1YmphlJ3H8jOCDJ7k4G2S7xvfsjoBsLcepmZ6MADQ8Y=; b=LoEpzUobzeF1DftVmKDr9xArKQbvpcm9xE0iSw4JVdzUSCpxQ7EMhzYKhI+R+t1rOl hed7ihxGTD8oxJcbjG+1JGeI7eNxpqM7Q1CxpBDQo9RE4nTwWofsykdUyK0UivT/pWVC yrqJDUofTjk4z36yODlG8KTTJnSNFvxO8yvnv92XO/0gsLqTuqsy307sM3wmpfPaubZa a398iGK0+3S1StI1eLpbix9fsFDFUAItB/06fa2rW9cLtlieY/gOXAYpA5CBRIUu8ONI gelL0MZ67nuA11h+UQ99DGwAoFHWEyzQ9IwA69BZm54GJeaLrJvjD+0qC4wgI/5gRHsT KV/A== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@fb.com header.s=facebook header.b=bDtECx7J; spf=pass (google.com: domain of prvs=1078cbd532=songliubraving@fb.com designates 67.231.153.30 as permitted sender) smtp.mailfrom="prvs=1078cbd532=songliubraving@fb.com"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=fb.com Received: from mx0b-00082601.pphosted.com (mx0b-00082601.pphosted.com. [67.231.153.30]) by mx.google.com with ESMTPS id h20si4259774ywa.373.2019.06.24.15.30.17 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 24 Jun 2019 15:30:17 -0700 (PDT) Received-SPF: pass (google.com: domain of prvs=1078cbd532=songliubraving@fb.com designates 67.231.153.30 as permitted sender) client-ip=67.231.153.30; Authentication-Results: mx.google.com; dkim=pass header.i=@fb.com header.s=facebook header.b=bDtECx7J; spf=pass (google.com: domain of prvs=1078cbd532=songliubraving@fb.com designates 67.231.153.30 as permitted sender) smtp.mailfrom="prvs=1078cbd532=songliubraving@fb.com"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=fb.com Received: from pps.filterd (m0109331.ppops.net [127.0.0.1]) by mx0a-00082601.pphosted.com (8.16.0.27/8.16.0.27) with SMTP id x5OMIE4t000694 for ; Mon, 24 Jun 2019 15:30:17 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fb.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type; s=facebook; bh=1YmphlJ3H8jOCDJ7k4G2S7xvfsjoBsLcepmZ6MADQ8Y=; b=bDtECx7JjGTraAQgcG+LiSc0bC+DOB+jM2yn4IYkK2W7vzJreSCkFUik5nANicKPctGH GEVwBJchObVFymR3+HxGxfiY0ESoyufbZdV3sAYjbm4gpcIuIgq1f+GyrXDCCZutciMc 81z1y+YcZ+IWajUeJ1ItDgGTxIa7xRIEgMg= Received: from maileast.thefacebook.com ([163.114.130.16]) by mx0a-00082601.pphosted.com with ESMTP id 2tb6j2g6a8-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NOT) for ; Mon, 24 Jun 2019 15:30:16 -0700 Received: from mx-out.facebook.com (2620:10d:c0a8:1b::d) by mail.thefacebook.com (2620:10d:c0a8:83::4) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.1713.5; Mon, 24 Jun 2019 15:30:16 -0700 Received: by devbig006.ftw2.facebook.com (Postfix, from userid 4523) id E384862E206E; Mon, 24 Jun 2019 15:30:11 -0700 (PDT) Smtp-Origin-Hostprefix: devbig From: Song Liu Smtp-Origin-Hostname: devbig006.ftw2.facebook.com To: , , CC: , , , , , , Song Liu Smtp-Origin-Cluster: ftw2c04 Subject: [PATCH v8 6/6] mm,thp: avoid writes to file with THP in pagecache Date: Mon, 24 Jun 2019 15:29:51 -0700 Message-ID: <20190624222951.37076-7-songliubraving@fb.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20190624222951.37076-1-songliubraving@fb.com> References: <20190624222951.37076-1-songliubraving@fb.com> X-FB-Internal: Safe MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:,, definitions=2019-06-24_15:,, signatures=0 X-Proofpoint-Spam-Details: rule=fb_default_notspam policy=fb_default score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=801 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1906240176 X-FB-Internal: deliver X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP In previous patch, an application could put part of its text section in THP via madvise(). These THPs will be protected from writes when the application is still running (TXTBSY). However, after the application exits, the file is available for writes. This patch avoids writes to file THP by dropping page cache for the file when the file is open for write. A new counter nr_thps is added to struct address_space. In do_last(), if the file is open for write and nr_thps is non-zero, we drop page cache for the whole file. Reported-by: kbuild test robot Signed-off-by: Song Liu --- fs/inode.c | 2 ++ fs/namei.c | 23 ++++++++++++++++++++++- include/linux/fs.h | 28 ++++++++++++++++++++++++++++ mm/filemap.c | 1 + mm/khugepaged.c | 4 +++- 5 files changed, 56 insertions(+), 2 deletions(-) diff --git a/fs/inode.c b/fs/inode.c index df6542ec3b88..7f27a5fd147b 100644 --- a/fs/inode.c +++ b/fs/inode.c @@ -181,6 +181,8 @@ int inode_init_always(struct super_block *sb, struct inode *inode) mapping->flags = 0; mapping->wb_err = 0; atomic_set(&mapping->i_mmap_writable, 0); + if (IS_ENABLED(CONFIG_READ_ONLY_THP_FOR_FS)) + atomic_set(&mapping->nr_thps, 0); mapping_set_gfp_mask(mapping, GFP_HIGHUSER_MOVABLE); mapping->private_data = NULL; mapping->writeback_index = 0; diff --git a/fs/namei.c b/fs/namei.c index 20831c2fbb34..3d95e94029cc 100644 --- a/fs/namei.c +++ b/fs/namei.c @@ -3249,6 +3249,23 @@ static int lookup_open(struct nameidata *nd, struct path *path, return error; } +/* + * The file is open for write, so it is not mmapped with VM_DENYWRITE. If + * it still has THP in page cache, drop the whole file from pagecache + * before processing writes. This helps us avoid handling write back of + * THP for now. + */ +static inline void release_file_thp(struct file *file) +{ + if (IS_ENABLED(CONFIG_READ_ONLY_THP_FOR_FS)) { + struct inode *inode = file_inode(file); + + if (inode_is_open_for_write(inode) && + filemap_nr_thps(inode->i_mapping)) + truncate_pagecache(inode, 0); + } +} + /* * Handle the last step of open() */ @@ -3418,7 +3435,11 @@ static int do_last(struct nameidata *nd, goto out; opened: error = ima_file_check(file, op->acc_mode); - if (!error && will_truncate) + if (error) + goto out; + + release_file_thp(file); + if (will_truncate) error = handle_truncate(file); out: if (unlikely(error > 0)) { diff --git a/include/linux/fs.h b/include/linux/fs.h index f7fdfe93e25d..20443d63692e 100644 --- a/include/linux/fs.h +++ b/include/linux/fs.h @@ -427,6 +427,7 @@ int pagecache_write_end(struct file *, struct address_space *mapping, * @i_pages: Cached pages. * @gfp_mask: Memory allocation flags to use for allocating pages. * @i_mmap_writable: Number of VM_SHARED mappings. + * @nr_thps: Number of THPs in the pagecache (non-shmem only). * @i_mmap: Tree of private and shared mappings. * @i_mmap_rwsem: Protects @i_mmap and @i_mmap_writable. * @nrpages: Number of page entries, protected by the i_pages lock. @@ -444,6 +445,10 @@ struct address_space { struct xarray i_pages; gfp_t gfp_mask; atomic_t i_mmap_writable; +#ifdef CONFIG_READ_ONLY_THP_FOR_FS + /* number of thp, only for non-shmem files */ + atomic_t nr_thps; +#endif struct rb_root_cached i_mmap; struct rw_semaphore i_mmap_rwsem; unsigned long nrpages; @@ -2790,6 +2795,29 @@ static inline errseq_t filemap_sample_wb_err(struct address_space *mapping) return errseq_sample(&mapping->wb_err); } +static inline int filemap_nr_thps(struct address_space *mapping) +{ + if (IS_ENABLED(CONFIG_READ_ONLY_THP_FOR_FS)) + return atomic_read(&mapping->nr_thps); + return 0; +} + +static inline void filemap_nr_thps_inc(struct address_space *mapping) +{ + if (IS_ENABLED(CONFIG_READ_ONLY_THP_FOR_FS)) + atomic_inc(&mapping->nr_thps); + else + WARN_ON_ONCE(1); +} + +static inline void filemap_nr_thps_dec(struct address_space *mapping) +{ + if (IS_ENABLED(CONFIG_READ_ONLY_THP_FOR_FS)) + atomic_dec(&mapping->nr_thps); + else + WARN_ON_ONCE(1); +} + extern int vfs_fsync_range(struct file *file, loff_t start, loff_t end, int datasync); extern int vfs_fsync(struct file *file, int datasync); diff --git a/mm/filemap.c b/mm/filemap.c index e79ceccdc6df..a8e86c136381 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -205,6 +205,7 @@ static void unaccount_page_cache_page(struct address_space *mapping, __dec_node_page_state(page, NR_SHMEM_THPS); } else if (PageTransHuge(page)) { __dec_node_page_state(page, NR_FILE_THPS); + filemap_nr_thps_dec(mapping); } /* diff --git a/mm/khugepaged.c b/mm/khugepaged.c index acbbbeaa083c..0bbc6be51197 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -1503,8 +1503,10 @@ static void collapse_file(struct mm_struct *mm, if (is_shmem) __inc_node_page_state(new_page, NR_SHMEM_THPS); - else + else { __inc_node_page_state(new_page, NR_FILE_THPS); + filemap_nr_thps_inc(mapping); + } if (nr_none) { struct zone *zone = page_zone(new_page);