From patchwork Tue Jun 25 00:12:41 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Song Liu X-Patchwork-Id: 11014415 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 6D0D114E5 for ; Tue, 25 Jun 2019 00:13:03 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 5F00228A91 for ; Tue, 25 Jun 2019 00:13:03 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 5294228AD2; Tue, 25 Jun 2019 00:13:03 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id E367B28A91 for ; Tue, 25 Jun 2019 00:13:02 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id E0D026B0007; Mon, 24 Jun 2019 20:12:59 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id DC0C58E0003; Mon, 24 Jun 2019 20:12:59 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id BEB6F8E0002; Mon, 24 Jun 2019 20:12:59 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-pf1-f200.google.com (mail-pf1-f200.google.com [209.85.210.200]) by kanga.kvack.org (Postfix) with ESMTP id 88E276B0007 for ; Mon, 24 Jun 2019 20:12:59 -0400 (EDT) Received: by mail-pf1-f200.google.com with SMTP id y5so10580312pfb.20 for ; Mon, 24 Jun 2019 17:12:59 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:smtp-origin-hostprefix:from :smtp-origin-hostname:to:cc:smtp-origin-cluster:subject:date :message-id:in-reply-to:references:mime-version; bh=EqtD9pgjSn3RpeCA8R1A7DD1Wom8eSWCEiXrtl71yV4=; b=iZ5OLpaex2zBFIsKaoTlQnbM4DyYsD7gp2i9fg48VSVr3p6+6r/Nz4UJuQ2Srmq6gk zN3+06/oklo67Xb5cD+2e8dvS9js0wxCTEWZ12Un0jBrTByxreYJFIf6E/K0hpWXoyEG mUwa3nhOLrR/SxfuS9NmeUN+DUNHk2xlnEQl6al0I7x76Zi/jEr652xXeELZ/fJqeIRL gZ+0eXdCjxoFM2UbAj/LG8BIDrj76XIwcxZ0fKtD1WMnTvfpFTRW5AdkBKYFQtIc25Ce 3g8TFd6VT+o8TxA/DExz56tX7ehPuKBKvB+9z8ikz3q7P5GfXxgNJ96adaoj8Oa0ShDJ AuqA== X-Gm-Message-State: APjAAAXlr6qB+k7ZOkOQnK84FhEdnJhsaHOGhB3g30DzMfjMNsi8zMMZ Vw5dABPF3NYuPvXficscjt3znxcmDmxnONuYyrXTUmLOliDiKtbeJSlvMD8Ny+1ExySSwZ5dcm+ kURsF+2sWjScAABeCsgfG+9hGmvmyiajXTzY2EV7CY6CsA3MbLtY+pG/wfHlzg15bVQ== X-Received: by 2002:a17:90a:ab0b:: with SMTP id m11mr28876803pjq.73.1561421579067; Mon, 24 Jun 2019 17:12:59 -0700 (PDT) X-Google-Smtp-Source: APXvYqxVZevcd22eOt5JA764j5aEbq1dUxXzWd/YVYL+XpF+8N1wR97eXMVVg2rZsN4RoyIhToLE X-Received: by 2002:a17:90a:ab0b:: with SMTP id m11mr28876748pjq.73.1561421578414; Mon, 24 Jun 2019 17:12:58 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1561421578; cv=none; d=google.com; s=arc-20160816; b=PG1lZhhgcasl9Alzn4Bsy/4Oc38SwrMk8dDOhgf3xRNpT9bhAwOHa5lZcEhuz844nx tYf57O4LN3gQRSLKccteVdf21TMWyo0fKkhb7SZMs73pcJ+PswWARJZglMNQ4c9wEZ2L /ZLnhMNKRKRvfGN+V8lewIl7nPPKVHc+6UfDVz2GZCAGuPN7ci3HzQVC1xhBrxpWiX2X XloclHTjKrHekO5gbJH6Y4kOnwYY6XodPwRgMigOooKQ36B2T8HF77M0BGyoYQP02IPL ZfPljVNVEDRvgy2+2jfESu6Hq4b5bFhvdnpPq6GCsQUeBrrVur77+t0nNwcqxeF/ptZ9 wFqw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=mime-version:references:in-reply-to:message-id:date:subject :smtp-origin-cluster:cc:to:smtp-origin-hostname:from :smtp-origin-hostprefix:dkim-signature; bh=EqtD9pgjSn3RpeCA8R1A7DD1Wom8eSWCEiXrtl71yV4=; b=f2or+E+Xq0BRc0iUyvRyQgmM9yeHJZrehq57EmPtpKkVjfjxNebtBkkCRsjMrIGxru Wgf86ym5a9/WJM1LenpAhQGUfsqtxi1vjOa3V0IZhk7NXm1EmFrpSA87fO6ZFDFysMuA 8VKL4yTUKmr+2PHZgBLiZ8TvwW0lyHDOxpGVNsJWbQ6BNTxReNlzC3bWmR6ixq4ygVDU jN4S8SFjhgdGsJzl9DiHjHtcdbB9rpSQ02nFCkh5C8bZmFF8JOSK89arPcwXJ4O7/V0i QMexgZ+EYCW6K8eaVtJC3inLQ4oyVmWcsBQab29nLyTn2EYovXhkapa6KMstWNAirDHb +qzw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@fb.com header.s=facebook header.b=Xn8tQ4Ka; spf=pass (google.com: domain of prvs=1079b839a8=songliubraving@fb.com designates 67.231.145.42 as permitted sender) smtp.mailfrom="prvs=1079b839a8=songliubraving@fb.com"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=fb.com Received: from mx0a-00082601.pphosted.com (mx0a-00082601.pphosted.com. [67.231.145.42]) by mx.google.com with ESMTPS id f17si943435pjq.18.2019.06.24.17.12.58 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 24 Jun 2019 17:12:58 -0700 (PDT) Received-SPF: pass (google.com: domain of prvs=1079b839a8=songliubraving@fb.com designates 67.231.145.42 as permitted sender) client-ip=67.231.145.42; Authentication-Results: mx.google.com; dkim=pass header.i=@fb.com header.s=facebook header.b=Xn8tQ4Ka; spf=pass (google.com: domain of prvs=1079b839a8=songliubraving@fb.com designates 67.231.145.42 as permitted sender) smtp.mailfrom="prvs=1079b839a8=songliubraving@fb.com"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=fb.com Received: from pps.filterd (m0109334.ppops.net [127.0.0.1]) by mx0a-00082601.pphosted.com (8.16.0.27/8.16.0.27) with SMTP id x5P08SIt029551 for ; Mon, 24 Jun 2019 17:12:57 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fb.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type; s=facebook; bh=EqtD9pgjSn3RpeCA8R1A7DD1Wom8eSWCEiXrtl71yV4=; b=Xn8tQ4KaSTAaKIrmEDK3g/qzHWbvSllORMtOv+5HabJUKfKEJ0HB8+eoHS3xJ4uFOPSZ y2S8Ruf83O1MB6fgymW4amomT+HuhZMUrVvHB5llYgz5qJE2UPUrG5ToGfp8bRoB9rrj DnUANIts4NU0oCOdc8XJK+Deh959QBDK+EA= Received: from mail.thefacebook.com (mailout.thefacebook.com [199.201.64.23]) by mx0a-00082601.pphosted.com with ESMTP id 2tb56ugu40-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Mon, 24 Jun 2019 17:12:57 -0700 Received: from mx-out.facebook.com (2620:10d:c081:10::13) by mail.thefacebook.com (2620:10d:c081:35::126) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA) id 15.1.1713.5; Mon, 24 Jun 2019 17:12:56 -0700 Received: by devbig006.ftw2.facebook.com (Postfix, from userid 4523) id 924A462E206E; Mon, 24 Jun 2019 17:12:52 -0700 (PDT) Smtp-Origin-Hostprefix: devbig From: Song Liu Smtp-Origin-Hostname: devbig006.ftw2.facebook.com To: , , CC: , , , , , , Song Liu Smtp-Origin-Cluster: ftw2c04 Subject: [PATCH v9 1/6] filemap: check compound_head(page)->mapping in filemap_fault() Date: Mon, 24 Jun 2019 17:12:41 -0700 Message-ID: <20190625001246.685563-2-songliubraving@fb.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20190625001246.685563-1-songliubraving@fb.com> References: <20190625001246.685563-1-songliubraving@fb.com> X-FB-Internal: Safe MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:,, definitions=2019-06-24_16:,, signatures=0 X-Proofpoint-Spam-Details: rule=fb_default_notspam policy=fb_default score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=921 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1906250000 X-FB-Internal: deliver X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP Currently, filemap_fault() avoids trace condition with truncate by checking page->mapping == mapping. This does not work for compound pages. This patch let it check compound_head(page)->mapping instead. Acked-by: Rik van Riel Signed-off-by: Song Liu Acked-by: Johannes Weiner --- mm/filemap.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/mm/filemap.c b/mm/filemap.c index df2006ba0cfa..f5b79a43946d 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -2517,7 +2517,7 @@ vm_fault_t filemap_fault(struct vm_fault *vmf) goto out_retry; /* Did it get truncated? */ - if (unlikely(page->mapping != mapping)) { + if (unlikely(compound_head(page)->mapping != mapping)) { unlock_page(page); put_page(page); goto retry_find; From patchwork Tue Jun 25 00:12:42 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Song Liu X-Patchwork-Id: 11014413 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 6AE2E14E5 for ; Tue, 25 Jun 2019 00:13:01 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 5C13128A91 for ; Tue, 25 Jun 2019 00:13:01 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 4FDFC28AD2; Tue, 25 Jun 2019 00:13:01 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 06D6028A91 for ; Tue, 25 Jun 2019 00:13:00 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 8B5386B0006; Mon, 24 Jun 2019 20:12:58 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 868C68E0003; Mon, 24 Jun 2019 20:12:58 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 755F08E0002; Mon, 24 Jun 2019 20:12:58 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-yb1-f197.google.com (mail-yb1-f197.google.com [209.85.219.197]) by kanga.kvack.org (Postfix) with ESMTP id 5476B6B0006 for ; Mon, 24 Jun 2019 20:12:58 -0400 (EDT) Received: by mail-yb1-f197.google.com with SMTP id g23so13885681ybj.20 for ; Mon, 24 Jun 2019 17:12:58 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:smtp-origin-hostprefix:from :smtp-origin-hostname:to:cc:smtp-origin-cluster:subject:date :message-id:in-reply-to:references:mime-version; bh=z7orUW1w2AFawDH57ZTLvYALJSDsMI3HPUuppHQ+jCM=; b=DjrJM26pHIYAL/LGrkQvra+C3vD+M/19/OoIRlnYaDtcys352txOoSVjAe9+9hr9RC riM779k/1sqz4RA7T0lllVw1DQkGm4TX+vHD2pQzSwHtQiRyRSxx8gXGYtjUnVaMIaDE H0e4j9GSxpkI67mtKhDJqHKZ1Por3msTbp/5tEcUECO2idgt2QSurnFpcseYuzvD6AF9 /eJXKVxk6po3fMcvSAx2Az9y95ItD+43RN/RFoxNU1h+mquj8qRknaQe7nthzMBLyYET lcu21+w4Pa3VoxJo9sYci1BI+0JrJs65d/CfsTJ0iubvRSK/ukor+RHAXNfBcq7Lx9Al CjFw== X-Gm-Message-State: APjAAAXiW0djvvLWCg3+HlQ7eIIUZUoNLa1NSv5mCNhlWyhmwX4EYx1X ZuyAHdRQBvgL8Q91SCC5UM66EPBFfNuL4cf490zCOh2FjSbC5MzAPqYx4CiG59jQRE0ze5OXzm/ mijYqTlJU4H7TeRrFE6wPyElgxcgbAn4xIzS3Y5TMNn7abUwuhhpMV4iqTN22g0XRmg== X-Received: by 2002:a0d:d9c8:: with SMTP id b191mr3279782ywe.186.1561421577970; Mon, 24 Jun 2019 17:12:57 -0700 (PDT) X-Google-Smtp-Source: APXvYqxi/K8cEnKT2bxi7UB+C2PD4NG/gjQYQtoLNl6IEaIyeZ/qj7FlvPQrhqoDo3l8kuAwIT+V X-Received: by 2002:a0d:d9c8:: with SMTP id b191mr3279771ywe.186.1561421577530; Mon, 24 Jun 2019 17:12:57 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1561421577; cv=none; d=google.com; s=arc-20160816; b=MjiSu9vj+eX9wvEM5OS3ihU/Q5cC/Z3FbbTb1hYhDg1DFVi/Qt4BsBZZesuD2liBYT NygECtYvHoTcO6hpQCriZ35SEkoVKMwLZ6VlZOCnSVDXJo7uj2jFGMEIwiYdxGNDL8Ud TtaPfT4o3j1blcWFNDD3Dzkf6j/50OXN+K8NQZJRa4OiVRvZ1IOjfX++rxUY7jxvT4zU LmpoNErP+F/DqqLQx5UhfDGTkg9+2+L8/9UulGvBKPx6L3UNqMhtKI05wtMFltC1ySPQ NS5GPnCBEq2NxRaa92ZGuQL3ySOsPIcKMgKqKTSGrxxkMfCgZDQEeCd+eEzbBPgDQpZ7 ZyyA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=mime-version:references:in-reply-to:message-id:date:subject :smtp-origin-cluster:cc:to:smtp-origin-hostname:from :smtp-origin-hostprefix:dkim-signature; bh=z7orUW1w2AFawDH57ZTLvYALJSDsMI3HPUuppHQ+jCM=; b=s6+lFgHjaN4WAQm7hyBX7S+sX4X8F9H8V4ceDwAGy+ZGFzEWu/0BDjUTQY9uVOr5aN eP7wea9bVsuqD72atDv/ludZaFqGKLxq1EGWd8tWHBhzqM2W590z9SGmyIl2SHYnU/dH 9byGQ6JZdqmYyhiyIGF+rIfbPB7nwkQn7xsrB5cDJEmHkxSM4dSZrbtWjmoTrTu1gzZN DwjApxabz/NUmt+SZ6ApBC+OLW78/WC/RaKPvX8SmKinugpjoLpPVTrA/1u7HRhdmU89 TbDZZ0rZuszgVdJVquZfoC23TCf5oPNzi8hDPM8yfO1K7yYNr5suASTzyZ8GEt1Qaduh N3Lg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@fb.com header.s=facebook header.b=Bjb3xuQs; spf=pass (google.com: domain of prvs=1079b839a8=songliubraving@fb.com designates 67.231.153.30 as permitted sender) smtp.mailfrom="prvs=1079b839a8=songliubraving@fb.com"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=fb.com Received: from mx0a-00082601.pphosted.com (mx0b-00082601.pphosted.com. [67.231.153.30]) by mx.google.com with ESMTPS id 66si4359971ybm.237.2019.06.24.17.12.57 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 24 Jun 2019 17:12:57 -0700 (PDT) Received-SPF: pass (google.com: domain of prvs=1079b839a8=songliubraving@fb.com designates 67.231.153.30 as permitted sender) client-ip=67.231.153.30; Authentication-Results: mx.google.com; dkim=pass header.i=@fb.com header.s=facebook header.b=Bjb3xuQs; spf=pass (google.com: domain of prvs=1079b839a8=songliubraving@fb.com designates 67.231.153.30 as permitted sender) smtp.mailfrom="prvs=1079b839a8=songliubraving@fb.com"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=fb.com Received: from pps.filterd (m0089730.ppops.net [127.0.0.1]) by m0089730.ppops.net (8.16.0.27/8.16.0.27) with SMTP id x5P08sK9031652 for ; Mon, 24 Jun 2019 17:12:57 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fb.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type; s=facebook; bh=z7orUW1w2AFawDH57ZTLvYALJSDsMI3HPUuppHQ+jCM=; b=Bjb3xuQs6YXfMYORrs2euWeYoJm3krK9tGMMWXiCR+iwQp37+xprNLGkO4fpNz36Nb1d /G0eoHEN0j01YC6AaXkViVvCwmNcYyAAUiHNaAyE/Rmi0WPXOtFl3Mlgj4PN1DvfVJMd PMHCwZSAqf7XdgtPSf2dw/C8Fnj7doJLkDY= Received: from mail.thefacebook.com (mailout.thefacebook.com [199.201.64.23]) by m0089730.ppops.net with ESMTP id 2tb3gw98wn-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Mon, 24 Jun 2019 17:12:57 -0700 Received: from mx-out.facebook.com (2620:10d:c081:10::13) by mail.thefacebook.com (2620:10d:c081:35::129) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA) id 15.1.1713.5; Mon, 24 Jun 2019 17:12:56 -0700 Received: by devbig006.ftw2.facebook.com (Postfix, from userid 4523) id EB6A462E206E; Mon, 24 Jun 2019 17:12:55 -0700 (PDT) Smtp-Origin-Hostprefix: devbig From: Song Liu Smtp-Origin-Hostname: devbig006.ftw2.facebook.com To: , , CC: , , , , , , Song Liu Smtp-Origin-Cluster: ftw2c04 Subject: [PATCH v9 2/6] filemap: update offset check in filemap_fault() Date: Mon, 24 Jun 2019 17:12:42 -0700 Message-ID: <20190625001246.685563-3-songliubraving@fb.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20190625001246.685563-1-songliubraving@fb.com> References: <20190625001246.685563-1-songliubraving@fb.com> X-FB-Internal: Safe MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:,, definitions=2019-06-24_16:,, signatures=0 X-Proofpoint-Spam-Details: rule=fb_default_notspam policy=fb_default score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=797 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1906250000 X-FB-Internal: deliver X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP With THP, current check of offset: VM_BUG_ON_PAGE(page->index != offset, page); is no longer accurate. Update it to: VM_BUG_ON_PAGE(page_to_pgoff(page) != offset, page); Acked-by: Rik van Riel Signed-off-by: Song Liu Acked-by: Johannes Weiner --- mm/filemap.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/mm/filemap.c b/mm/filemap.c index f5b79a43946d..5f072a113535 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -2522,7 +2522,7 @@ vm_fault_t filemap_fault(struct vm_fault *vmf) put_page(page); goto retry_find; } - VM_BUG_ON_PAGE(page->index != offset, page); + VM_BUG_ON_PAGE(page_to_pgoff(page) != offset, page); /* * We have a locked page in the page cache, now we need to check From patchwork Tue Jun 25 00:12:43 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Song Liu X-Patchwork-Id: 11014417 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id DA70E6C5 for ; Tue, 25 Jun 2019 00:13:07 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id CBE0328A91 for ; Tue, 25 Jun 2019 00:13:07 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id BFB8228AD2; Tue, 25 Jun 2019 00:13:07 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 90E9D28A91 for ; Tue, 25 Jun 2019 00:13:06 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id CFED06B0008; Mon, 24 Jun 2019 20:13:04 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id CACDD8E0003; Mon, 24 Jun 2019 20:13:04 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B77A78E0002; Mon, 24 Jun 2019 20:13:04 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-yb1-f199.google.com (mail-yb1-f199.google.com [209.85.219.199]) by kanga.kvack.org (Postfix) with ESMTP id 90C2A6B0008 for ; Mon, 24 Jun 2019 20:13:04 -0400 (EDT) Received: by mail-yb1-f199.google.com with SMTP id t14so5300933ybt.5 for ; Mon, 24 Jun 2019 17:13:04 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:smtp-origin-hostprefix:from :smtp-origin-hostname:to:cc:smtp-origin-cluster:subject:date :message-id:in-reply-to:references:mime-version; bh=TFx9H37w6mZg+4Tg54qFCnWnzVBpXQ/B3y/2N1lbRw0=; b=aU0iI5Z9B28g8Novsd3xn/CbHpY6bUN5TAO4MoTpFh+1GQUeFvRQyK6B5Fl0qDTAwu POnYzEl88BMibFB/C6YlhdusAPpbg1aN1lDf3c9U9+xkg+rEWR0OSVMKfzZwfS13sOao 5FOEEgIlwY2LRgib3te2ia7cV6xMkT0BjIkW2W0KDTu+OnWM/5lRx3AOFwxqs6kLPsrS XxNixuVAshJW6ctAdwOI6eixeA7b4I0cWUcD37cW7UIdObLNLOLMfJlqEg4haGCk3mp4 95wKnYgVIqwPFNRKBPhC+h8fMelXFs2y9dfRYJtH9M1R2ebxZ+t05CB1gOkWmo6yThg1 nPzg== X-Gm-Message-State: APjAAAWZhvCeG3daLwlT5MToVpsHXYr21nhCoCh30twEpSdSJetIughC hEkcVKY/eWBajqxejpSdw7DmJfCckq+phjQ4iw/o410pH2k/Kybca9s3w5YG0uOkUnrdVXXBHp4 XPMjH6noEF6Qb3ygWyY9vwt5FzFS2XTdlSxhLGUxcRG9Cw0Q7ZtqdFKZ/CYR/f6roMQ== X-Received: by 2002:a25:3a44:: with SMTP id h65mr80389989yba.449.1561421584345; Mon, 24 Jun 2019 17:13:04 -0700 (PDT) X-Google-Smtp-Source: APXvYqy9B4XCfcxiej0J8kvY0i++ydzdoDMGmt6D1w0eyHX6cbzYhSGJ2yqTRhqhsmxfpu5Jw1CU X-Received: by 2002:a25:3a44:: with SMTP id h65mr80389974yba.449.1561421583796; Mon, 24 Jun 2019 17:13:03 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1561421583; cv=none; d=google.com; s=arc-20160816; b=q6e/KxGMZcGvar4lZaqW7S3eMMNKAJy6HgPG6O+T8CeQwUveGLYs3VFZV88wpikcJu LlXeHA1nB6EEbTGZQ+M78KuVQn2qVucit/4B6d/+BhWtvfHpb9uJCi1UnfmB+QcsGs3O DZP33sAkprN3BCULjZLv4iGmKBG/AeVcVvr5CMhvttLjPYNAj048l2ip1UB6eVFPUslf I9RdTH3bUQrjbGx2Lt2nuuRKw+LJIgpfOiGBz7+sojEksUIW8KkkPQehRRsrEIDqjHnN qdn0yCQpjTFv9OQvpw6PBrhdD0qeSa+RfZJfoJ8ltPvx1MRDNws8QT8vQEoavndbs+QR jspA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=mime-version:references:in-reply-to:message-id:date:subject :smtp-origin-cluster:cc:to:smtp-origin-hostname:from :smtp-origin-hostprefix:dkim-signature; bh=TFx9H37w6mZg+4Tg54qFCnWnzVBpXQ/B3y/2N1lbRw0=; b=0pMInnnox7in2fO13kJ9kKCcbshxFdmDFkd7xXxtqDqYXJdJ0byu8OMpRBoYvB9llY NCb3jjDljOrBtsb7y/3EHVufmF69UP8NUlUGMpR04WZhKLOL8IwKcWJhmFMH+Nb5IazP ZF4ydAPosmjjlJ+vAkb7jsgmmQC3Z0iZHTzd4gqOP1EpHlEY+CtAoUgol2b2ll9IL5yz Mxi0fGSXk9Pw0PEM3KbAh0DZsqYxFFDPI1M5ekvczFDAgaISUDtIx4C3XCF67KI2mLhb y21OSgSu3TaYpPoP8Rj7zNjTtMgyxXLGzc4+hhOecA75+gbDBDP6PQbCbvJfk0A+s3I8 WGtg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@fb.com header.s=facebook header.b=YOqjNJ1Z; spf=pass (google.com: domain of prvs=1079b839a8=songliubraving@fb.com designates 67.231.153.30 as permitted sender) smtp.mailfrom="prvs=1079b839a8=songliubraving@fb.com"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=fb.com Received: from mx0a-00082601.pphosted.com (mx0b-00082601.pphosted.com. [67.231.153.30]) by mx.google.com with ESMTPS id r6si4430973ybm.450.2019.06.24.17.13.03 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 24 Jun 2019 17:13:03 -0700 (PDT) Received-SPF: pass (google.com: domain of prvs=1079b839a8=songliubraving@fb.com designates 67.231.153.30 as permitted sender) client-ip=67.231.153.30; Authentication-Results: mx.google.com; dkim=pass header.i=@fb.com header.s=facebook header.b=YOqjNJ1Z; spf=pass (google.com: domain of prvs=1079b839a8=songliubraving@fb.com designates 67.231.153.30 as permitted sender) smtp.mailfrom="prvs=1079b839a8=songliubraving@fb.com"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=fb.com Received: from pps.filterd (m0089730.ppops.net [127.0.0.1]) by m0089730.ppops.net (8.16.0.27/8.16.0.27) with SMTP id x5P09FcS031911 for ; Mon, 24 Jun 2019 17:13:03 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fb.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type; s=facebook; bh=TFx9H37w6mZg+4Tg54qFCnWnzVBpXQ/B3y/2N1lbRw0=; b=YOqjNJ1ZSRhFiiRzZ6i6hCn4Q8zUNgxKC5HotrmxTrVVkvMXwz4ifQ84LDmXq0nOGIXA iU8/tEBk5/3yfhnUGPCqnKQsc1AQgJJpf9dHc0UoVfnoRvbOEtz/UBV4WH8VNZlhcGad mnvsemCNpShLGPDpOc2anHuA/kyrnoBT8oM= Received: from mail.thefacebook.com (mailout.thefacebook.com [199.201.64.23]) by m0089730.ppops.net with ESMTP id 2tb3gw98x1-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Mon, 24 Jun 2019 17:13:03 -0700 Received: from mx-out.facebook.com (2620:10d:c081:10::13) by mail.thefacebook.com (2620:10d:c081:35::126) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA) id 15.1.1713.5; Mon, 24 Jun 2019 17:13:02 -0700 Received: by devbig006.ftw2.facebook.com (Postfix, from userid 4523) id 3AA9762E206E; Mon, 24 Jun 2019 17:12:59 -0700 (PDT) Smtp-Origin-Hostprefix: devbig From: Song Liu Smtp-Origin-Hostname: devbig006.ftw2.facebook.com To: , , CC: , , , , , , Song Liu Smtp-Origin-Cluster: ftw2c04 Subject: [PATCH v9 3/6] mm,thp: stats for file backed THP Date: Mon, 24 Jun 2019 17:12:43 -0700 Message-ID: <20190625001246.685563-4-songliubraving@fb.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20190625001246.685563-1-songliubraving@fb.com> References: <20190625001246.685563-1-songliubraving@fb.com> X-FB-Internal: Safe MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:,, definitions=2019-06-24_16:,, signatures=0 X-Proofpoint-Spam-Details: rule=fb_default_notspam policy=fb_default score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1906250000 X-FB-Internal: deliver X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP In preparation for non-shmem THP, this patch adds a few stats and exposes them in /proc/meminfo, /sys/bus/node/devices//meminfo, and /proc//task//smaps. This patch is mostly a rewrite of Kirill A. Shutemov's earlier version: https://lkml.kernel.org/r/20170126115819.58875-5-kirill.shutemov@linux.intel.com/ Acked-by: Rik van Riel Signed-off-by: Song Liu Acked-by: Johannes Weiner --- drivers/base/node.c | 6 ++++++ fs/proc/meminfo.c | 4 ++++ fs/proc/task_mmu.c | 4 +++- include/linux/mmzone.h | 2 ++ mm/vmstat.c | 2 ++ 5 files changed, 17 insertions(+), 1 deletion(-) diff --git a/drivers/base/node.c b/drivers/base/node.c index 8598fcbd2a17..71ae2dc93489 100644 --- a/drivers/base/node.c +++ b/drivers/base/node.c @@ -426,6 +426,8 @@ static ssize_t node_read_meminfo(struct device *dev, "Node %d AnonHugePages: %8lu kB\n" "Node %d ShmemHugePages: %8lu kB\n" "Node %d ShmemPmdMapped: %8lu kB\n" + "Node %d FileHugePages: %8lu kB\n" + "Node %d FilePmdMapped: %8lu kB\n" #endif , nid, K(node_page_state(pgdat, NR_FILE_DIRTY)), @@ -451,6 +453,10 @@ static ssize_t node_read_meminfo(struct device *dev, nid, K(node_page_state(pgdat, NR_SHMEM_THPS) * HPAGE_PMD_NR), nid, K(node_page_state(pgdat, NR_SHMEM_PMDMAPPED) * + HPAGE_PMD_NR), + nid, K(node_page_state(pgdat, NR_FILE_THPS) * + HPAGE_PMD_NR), + nid, K(node_page_state(pgdat, NR_FILE_PMDMAPPED) * HPAGE_PMD_NR) #endif ); diff --git a/fs/proc/meminfo.c b/fs/proc/meminfo.c index 568d90e17c17..bac395fc11f9 100644 --- a/fs/proc/meminfo.c +++ b/fs/proc/meminfo.c @@ -136,6 +136,10 @@ static int meminfo_proc_show(struct seq_file *m, void *v) global_node_page_state(NR_SHMEM_THPS) * HPAGE_PMD_NR); show_val_kb(m, "ShmemPmdMapped: ", global_node_page_state(NR_SHMEM_PMDMAPPED) * HPAGE_PMD_NR); + show_val_kb(m, "FileHugePages: ", + global_node_page_state(NR_FILE_THPS) * HPAGE_PMD_NR); + show_val_kb(m, "FilePmdMapped: ", + global_node_page_state(NR_FILE_PMDMAPPED) * HPAGE_PMD_NR); #endif #ifdef CONFIG_CMA diff --git a/fs/proc/task_mmu.c b/fs/proc/task_mmu.c index 01d4eb0e6bd1..0360e3b2ba89 100644 --- a/fs/proc/task_mmu.c +++ b/fs/proc/task_mmu.c @@ -413,6 +413,7 @@ struct mem_size_stats { unsigned long lazyfree; unsigned long anonymous_thp; unsigned long shmem_thp; + unsigned long file_thp; unsigned long swap; unsigned long shared_hugetlb; unsigned long private_hugetlb; @@ -563,7 +564,7 @@ static void smaps_pmd_entry(pmd_t *pmd, unsigned long addr, else if (is_zone_device_page(page)) /* pass */; else - VM_BUG_ON_PAGE(1, page); + mss->file_thp += HPAGE_PMD_SIZE; smaps_account(mss, page, true, pmd_young(*pmd), pmd_dirty(*pmd), locked); } #else @@ -767,6 +768,7 @@ static void __show_smap(struct seq_file *m, const struct mem_size_stats *mss) SEQ_PUT_DEC(" kB\nLazyFree: ", mss->lazyfree); SEQ_PUT_DEC(" kB\nAnonHugePages: ", mss->anonymous_thp); SEQ_PUT_DEC(" kB\nShmemPmdMapped: ", mss->shmem_thp); + SEQ_PUT_DEC(" kB\nFilePmdMapped: ", mss->file_thp); SEQ_PUT_DEC(" kB\nShared_Hugetlb: ", mss->shared_hugetlb); seq_put_decimal_ull_width(m, " kB\nPrivate_Hugetlb: ", mss->private_hugetlb >> 10, 7); diff --git a/include/linux/mmzone.h b/include/linux/mmzone.h index 70394cabaf4e..827f9b777938 100644 --- a/include/linux/mmzone.h +++ b/include/linux/mmzone.h @@ -234,6 +234,8 @@ enum node_stat_item { NR_SHMEM, /* shmem pages (included tmpfs/GEM pages) */ NR_SHMEM_THPS, NR_SHMEM_PMDMAPPED, + NR_FILE_THPS, + NR_FILE_PMDMAPPED, NR_ANON_THPS, NR_UNSTABLE_NFS, /* NFS unstable pages */ NR_VMSCAN_WRITE, diff --git a/mm/vmstat.c b/mm/vmstat.c index fd7e16ca6996..6afc892a148a 100644 --- a/mm/vmstat.c +++ b/mm/vmstat.c @@ -1158,6 +1158,8 @@ const char * const vmstat_text[] = { "nr_shmem", "nr_shmem_hugepages", "nr_shmem_pmdmapped", + "nr_file_hugepages", + "nr_file_pmdmapped", "nr_anon_transparent_hugepages", "nr_unstable", "nr_vmscan_write", From patchwork Tue Jun 25 00:12:44 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Song Liu X-Patchwork-Id: 11014419 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id AAF2114E5 for ; Tue, 25 Jun 2019 00:13:10 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 9D06628A91 for ; Tue, 25 Jun 2019 00:13:10 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 9126028AD2; Tue, 25 Jun 2019 00:13:10 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 0D87A28A91 for ; Tue, 25 Jun 2019 00:13:10 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id C5F456B000A; Mon, 24 Jun 2019 20:13:06 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id C11128E0003; Mon, 24 Jun 2019 20:13:06 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B015F8E0002; Mon, 24 Jun 2019 20:13:06 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-yb1-f200.google.com (mail-yb1-f200.google.com [209.85.219.200]) by kanga.kvack.org (Postfix) with ESMTP id 8ED0F6B000A for ; Mon, 24 Jun 2019 20:13:06 -0400 (EDT) Received: by mail-yb1-f200.google.com with SMTP id z4so18345476ybo.4 for ; Mon, 24 Jun 2019 17:13:06 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:smtp-origin-hostprefix:from :smtp-origin-hostname:to:cc:smtp-origin-cluster:subject:date :message-id:in-reply-to:references:mime-version; bh=Kr37aYRy2KacRNeWnEOl8WIgUYHSnWgZ3fV+borBCvY=; b=KZsZqdKqTjqrjptSjoGVX+foCovty4yAzbTfZCueXazl9AJXW2dySfhWDPiYJYzhpb iG8DeMGXmeI2eXS5ffUQyyKHxqXxvIG0H0I82CuxGQdT1wuBj9i3Bg6gTVY6QbD45SlY MBnU0ngSZzDncl+qh3Xdl4Sio9edpeGoonO3tGuLtk7n+Go4H0F7kGhTZ51/aDaeXC5V xWKRX3ev9Wri4GwXc/2dSOrs+7cURmlntbk0an93I9yMYbykQEZPXEeaeYcsN0x9tpCH 2F7IQIO0FmYGbwsOmliDMk+yD5bEFatLwBrLLvG8RZC9usYxZkbSgBB9AX/MRBwFzh57 ZoDQ== X-Gm-Message-State: APjAAAVhD43NkuzEBM3UmFajyFGOqg+0JHe8R+zIPxWs6y7TATqcqxGE oo/LLePDy3k4sewDBuu2RF4uON1TxWsBP13ZHAU0g1Eyc+V8OPRjKn689c52suuaDVYvg6NbqIF QKxrf2CjTl5tPKM+gu8n0qo2nA88e5zpSET5efcXarqggM6lWUli+hJLVs6mSUOt6XQ== X-Received: by 2002:a25:1454:: with SMTP id 81mr37546020ybu.96.1561421586335; Mon, 24 Jun 2019 17:13:06 -0700 (PDT) X-Google-Smtp-Source: APXvYqxYiY+pVbgQI1koNmpUJ4EytuTL9sb62mGFS8hVhF0fCr6ixuaCUyvwWrsScuaJceUQbVfs X-Received: by 2002:a25:1454:: with SMTP id 81mr37546011ybu.96.1561421585811; Mon, 24 Jun 2019 17:13:05 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1561421585; cv=none; d=google.com; s=arc-20160816; b=CqpB866Bpqd43HCc9zGm8VGcp7gJl1NMwZ1r9qj9GN/WkNVByiJKQuVIlPKxeCVLif JeUt+8DQ4gG5POoTqSgTVz68D3BR6MEmYm4qqpamnE5iZ/33gINNd8DCHfgfQXEWFwbX 4lfqxELt5AFFvmsmvq+s5Efkfdv9BPWsIQRH5yM5hVzfyQImykropvclIJ0ZJFRXpDUY 5eZkS6YaLmZrn07IhPGKtgZEjD5F2B0DuJCGluyQ+wkyWNRai5MRCNGtoW1Lh623go35 fL/Fundf9Ye3rcIoM0B6LWYP6ygXwJWqqe3MG/ggohuPr60M4VVIIXuTYSVi3bv5X9Uq ylow== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=mime-version:references:in-reply-to:message-id:date:subject :smtp-origin-cluster:cc:to:smtp-origin-hostname:from :smtp-origin-hostprefix:dkim-signature; bh=Kr37aYRy2KacRNeWnEOl8WIgUYHSnWgZ3fV+borBCvY=; b=T2aTh3Vccta/WCSle1k9WdYSLSIfwEf4TSpH2j3jgygZPoM0DrkeO9a1bPs3j/hKwY qZAaw+qWlmgxBtKiqqiddG3pPN3EVnzTzs4P2g4T5WtEcpZBsnO0y0wwaz+UBo4vDwM4 bSeO1eO7QuZeYxOVfLiP0G0trp6D/85F+u7rmct+NzxuWEC675QUKV4Sk7c1hHulYxlY M+aMiI5W6WX6olMi15GTHH/VpQIlAWcJjZMKPn/e/XnX5QkAH+v9qHrD8e4dhoF2+WgX S01D/hQUdqT2dI3c9ElQJwq/o8So8XS+2VRszgYsl3+E4vQWa9K6kveKsm61y9EyTEjz 6CWA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@fb.com header.s=facebook header.b=PbgtNCDz; spf=pass (google.com: domain of prvs=1079b839a8=songliubraving@fb.com designates 67.231.153.30 as permitted sender) smtp.mailfrom="prvs=1079b839a8=songliubraving@fb.com"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=fb.com Received: from mx0a-00082601.pphosted.com (mx0b-00082601.pphosted.com. [67.231.153.30]) by mx.google.com with ESMTPS id v70si2800560ybe.488.2019.06.24.17.13.05 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 24 Jun 2019 17:13:05 -0700 (PDT) Received-SPF: pass (google.com: domain of prvs=1079b839a8=songliubraving@fb.com designates 67.231.153.30 as permitted sender) client-ip=67.231.153.30; Authentication-Results: mx.google.com; dkim=pass header.i=@fb.com header.s=facebook header.b=PbgtNCDz; spf=pass (google.com: domain of prvs=1079b839a8=songliubraving@fb.com designates 67.231.153.30 as permitted sender) smtp.mailfrom="prvs=1079b839a8=songliubraving@fb.com"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=fb.com Received: from pps.filterd (m0001255.ppops.net [127.0.0.1]) by mx0b-00082601.pphosted.com (8.16.0.27/8.16.0.27) with SMTP id x5P0C2wW002816 for ; Mon, 24 Jun 2019 17:13:05 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fb.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type; s=facebook; bh=Kr37aYRy2KacRNeWnEOl8WIgUYHSnWgZ3fV+borBCvY=; b=PbgtNCDzYwbcMDtTXSXCHRmWRfbMZn8ph3GVIZcgyrJ4iXSMnpMCUEuIprfzqQxTugLt CTJUOFz3gJfM5B+/JWlIoxw4Re2hAbQm75Gf3rhH0irVDpUbGb/Ka4rWYoQQ9EUtANFL +hnoUF91iNrCNMSBInBfRt1BvO0049fa7j0= Received: from maileast.thefacebook.com ([163.114.130.16]) by mx0b-00082601.pphosted.com with ESMTP id 2tb22xspgt-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NOT) for ; Mon, 24 Jun 2019 17:13:05 -0700 Received: from mx-out.facebook.com (2620:10d:c0a8:1b::d) by mail.thefacebook.com (2620:10d:c0a8:82::e) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.1713.5; Mon, 24 Jun 2019 17:13:04 -0700 Received: by devbig006.ftw2.facebook.com (Postfix, from userid 4523) id ED70962E093E; Mon, 24 Jun 2019 17:13:02 -0700 (PDT) Smtp-Origin-Hostprefix: devbig From: Song Liu Smtp-Origin-Hostname: devbig006.ftw2.facebook.com To: , , CC: , , , , , , Song Liu Smtp-Origin-Cluster: ftw2c04 Subject: [PATCH v9 4/6] khugepaged: rename collapse_shmem() and khugepaged_scan_shmem() Date: Mon, 24 Jun 2019 17:12:44 -0700 Message-ID: <20190625001246.685563-5-songliubraving@fb.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20190625001246.685563-1-songliubraving@fb.com> References: <20190625001246.685563-1-songliubraving@fb.com> X-FB-Internal: Safe MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:,, definitions=2019-06-24_16:,, signatures=0 X-Proofpoint-Spam-Details: rule=fb_default_notspam policy=fb_default score=0 priorityscore=1501 malwarescore=0 suspectscore=2 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=627 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1906250000 X-FB-Internal: deliver X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP Next patch will add khugepaged support of non-shmem files. This patch renames these two functions to reflect the new functionality: collapse_shmem() => collapse_file() khugepaged_scan_shmem() => khugepaged_scan_file() Signed-off-by: Song Liu Acked-by: Rik van Riel Acked-by: Johannes Weiner --- mm/khugepaged.c | 23 +++++++++++------------ 1 file changed, 11 insertions(+), 12 deletions(-) diff --git a/mm/khugepaged.c b/mm/khugepaged.c index 0f7419938008..158cad542627 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -1287,7 +1287,7 @@ static void retract_page_tables(struct address_space *mapping, pgoff_t pgoff) } /** - * collapse_shmem - collapse small tmpfs/shmem pages into huge one. + * collapse_file - collapse small tmpfs/shmem pages into huge one. * * Basic scheme is simple, details are more complex: * - allocate and lock a new huge page; @@ -1304,10 +1304,11 @@ static void retract_page_tables(struct address_space *mapping, pgoff_t pgoff) * + restore gaps in the page cache; * + unlock and free huge page; */ -static void collapse_shmem(struct mm_struct *mm, - struct address_space *mapping, pgoff_t start, +static void collapse_file(struct mm_struct *mm, + struct file *file, pgoff_t start, struct page **hpage, int node) { + struct address_space *mapping = file->f_mapping; gfp_t gfp; struct page *new_page; struct mem_cgroup *memcg; @@ -1563,11 +1564,11 @@ static void collapse_shmem(struct mm_struct *mm, /* TODO: tracepoints */ } -static void khugepaged_scan_shmem(struct mm_struct *mm, - struct address_space *mapping, - pgoff_t start, struct page **hpage) +static void khugepaged_scan_file(struct mm_struct *mm, + struct file *file, pgoff_t start, struct page **hpage) { struct page *page = NULL; + struct address_space *mapping = file->f_mapping; XA_STATE(xas, &mapping->i_pages, start); int present, swap; int node = NUMA_NO_NODE; @@ -1631,16 +1632,15 @@ static void khugepaged_scan_shmem(struct mm_struct *mm, result = SCAN_EXCEED_NONE_PTE; } else { node = khugepaged_find_target_node(); - collapse_shmem(mm, mapping, start, hpage, node); + collapse_file(mm, file, start, hpage, node); } } /* TODO: tracepoints */ } #else -static void khugepaged_scan_shmem(struct mm_struct *mm, - struct address_space *mapping, - pgoff_t start, struct page **hpage) +static void khugepaged_scan_file(struct mm_struct *mm, + struct file *file, pgoff_t start, struct page **hpage) { BUILD_BUG(); } @@ -1722,8 +1722,7 @@ static unsigned int khugepaged_scan_mm_slot(unsigned int pages, file = get_file(vma->vm_file); up_read(&mm->mmap_sem); ret = 1; - khugepaged_scan_shmem(mm, file->f_mapping, - pgoff, hpage); + khugepaged_scan_file(mm, file, pgoff, hpage); fput(file); } else { ret = khugepaged_scan_pmd(mm, vma, From patchwork Tue Jun 25 00:12:45 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Song Liu X-Patchwork-Id: 11014423 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id A44AF6C5 for ; Tue, 25 Jun 2019 00:13:18 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 947B228A91 for ; Tue, 25 Jun 2019 00:13:18 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 8775F28AD2; Tue, 25 Jun 2019 00:13:18 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 4139528A91 for ; Tue, 25 Jun 2019 00:13:17 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 3123E6B000C; Mon, 24 Jun 2019 20:13:11 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 29E598E0002; Mon, 24 Jun 2019 20:13:11 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 001C06B000C; Mon, 24 Jun 2019 20:13:10 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-yb1-f198.google.com (mail-yb1-f198.google.com [209.85.219.198]) by kanga.kvack.org (Postfix) with ESMTP id CBA0A8E0002 for ; Mon, 24 Jun 2019 20:13:10 -0400 (EDT) Received: by mail-yb1-f198.google.com with SMTP id d6so18145712ybj.16 for ; Mon, 24 Jun 2019 17:13:10 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:smtp-origin-hostprefix:from :smtp-origin-hostname:to:cc:smtp-origin-cluster:subject:date :message-id:in-reply-to:references:mime-version; bh=Rw3bF3LLOPMJXUJryh7BDppGG/fMUSMU6vd9No970FE=; b=VfB25HWjKGex2cGDMJBB32Sdhs9YL6sELimP20LZQBpL4eW+CJGy8KO3GOAYmCq2T+ tmeXwgPasz7w9cl96BfeVVC3hw5WLDOOki5cD69YHC/jILNKe1x3hlyyu9a+QLiUf0Ux HiMt3abPx8coxpaA2LnyzglT/EBN3gkptmQZiTjhj4KehMn58p1LEHlr8Mu972MA3wEY YtO8mnxKaQRyiBBoEFb8/iI2EsKsTNe2DgbwUuPS8amf5QN5H2g80INqwo3NK13RdmeD vfWQwZViXtCXqSqdwH16JQietx8GUFdTb/QjjwS5IwrYrj8xThfDXznShxF/+nysFwFg zx4A== X-Gm-Message-State: APjAAAU2b/4uXmSwSV/c5lNqBqyTrs/kxHIYurceQnUNq8O9Y/g/5t3i jXZ0i+HuvxCxYKg3WL4tZVCquvrLptbW0i0DraKWHK/qXs30dESGL5PVJbG9qFwgtFHqeAtmaDS U3Cwb/ARhZvUijtxU0Bxf3zUYHTURwh84G3hhmmrFq1nRXjYu3j4x58X+voPBdp3VZA== X-Received: by 2002:a25:3506:: with SMTP id c6mr67822631yba.325.1561421590552; Mon, 24 Jun 2019 17:13:10 -0700 (PDT) X-Google-Smtp-Source: APXvYqweIRr9a6MRREe2RCKQ4MlskOz3w7ZQmMmAzaDtjz4Tyt80aGm5KuNQ8CEScTxC8EMJCGum X-Received: by 2002:a25:3506:: with SMTP id c6mr67822613yba.325.1561421589708; Mon, 24 Jun 2019 17:13:09 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1561421589; cv=none; d=google.com; s=arc-20160816; b=QRyjmB7mtPdVZvwmWzG3/c8qXl3hz6bCK25wo378K980q9ASXoVJPoOdcrYVyfiGHX LUvo8w3cALXEqhSmt8KIIKV8W0M8z+tcL7gzyJVsllo9ju9Oetsfop1frfp3Js+xj2i4 BPCWu5IpBbkmL4FSJBP9uqj+eV1Mk9cyQopRfVz4raG86bn+mY3ByOm4HSJd6KhFYFr4 J2TjMDdQXnwX2m3e9+6l+J3KOUOhaXjkPKdfgHqrua4WBN6kuNizUHY7I1iyL9DSCJXW 8o5CSs11/IIYPdeJIT3bguZwG9u4VDVmq4qfiutu4evYaJIvI1jdsGWVeoWDf0h+jzlD hg/g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=mime-version:references:in-reply-to:message-id:date:subject :smtp-origin-cluster:cc:to:smtp-origin-hostname:from :smtp-origin-hostprefix:dkim-signature; bh=Rw3bF3LLOPMJXUJryh7BDppGG/fMUSMU6vd9No970FE=; b=rsCBLy3U/zqPPE5vfiMW95Wq0IUlXR46QTqO/8F3KwcqJ33jfo/P751Rr4o1k9nmaz 5eCfSE/dBqeMcZ9QnLVlmlXjPYTIXCrk8LaPzroIHGHj58Fo23UJT2yBbfy2QxTfPksf wUeSXAIONyezHzcdSgmnvncRkaea9xOO1BkMcC82D67KNiyS/7dEoBjZTfiglpKqm9A4 Ry+dnw4aMfo3UBcijBBxWnNmgioLR4Qn2xXQDgdQvRUxUYOVdrmbkvfiWg84Vcu6B35w WgmdAO2sTMAFtaujZYmrB1BL8Wp4Nb7jFwtWm5NNLC0UWxGS+GkhS8SW2GTj6z7Y4SpS AJUg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@fb.com header.s=facebook header.b=mO3hm+2L; spf=pass (google.com: domain of prvs=1079b839a8=songliubraving@fb.com designates 67.231.153.30 as permitted sender) smtp.mailfrom="prvs=1079b839a8=songliubraving@fb.com"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=fb.com Received: from mx0b-00082601.pphosted.com (mx0b-00082601.pphosted.com. [67.231.153.30]) by mx.google.com with ESMTPS id h124si4283041ywe.429.2019.06.24.17.13.09 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 24 Jun 2019 17:13:09 -0700 (PDT) Received-SPF: pass (google.com: domain of prvs=1079b839a8=songliubraving@fb.com designates 67.231.153.30 as permitted sender) client-ip=67.231.153.30; Authentication-Results: mx.google.com; dkim=pass header.i=@fb.com header.s=facebook header.b=mO3hm+2L; spf=pass (google.com: domain of prvs=1079b839a8=songliubraving@fb.com designates 67.231.153.30 as permitted sender) smtp.mailfrom="prvs=1079b839a8=songliubraving@fb.com"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=fb.com Received: from pps.filterd (m0148460.ppops.net [127.0.0.1]) by mx0a-00082601.pphosted.com (8.16.0.27/8.16.0.27) with SMTP id x5P08x8k018099 for ; Mon, 24 Jun 2019 17:13:09 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fb.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type; s=facebook; bh=Rw3bF3LLOPMJXUJryh7BDppGG/fMUSMU6vd9No970FE=; b=mO3hm+2LH2rxdCsKSlHAreXhi4s/QrX6zYlIKEWOdii/Z8Qtje+3L2WT/pfT19C7XR7P meQyvDqrCqtIO62oiuWgWRKFbNp71Y3WJmfmngOkO6DhPmJFDC9hq6JqbX7caVu9iKlj dwT5+vhjugrOSrQte2PFkW+iLn4/QKJo2ls= Received: from mail.thefacebook.com (mailout.thefacebook.com [199.201.64.23]) by mx0a-00082601.pphosted.com with ESMTP id 2tawbtarad-2 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Mon, 24 Jun 2019 17:13:09 -0700 Received: from mx-out.facebook.com (2620:10d:c081:10::13) by mail.thefacebook.com (2620:10d:c081:35::127) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA) id 15.1.1713.5; Mon, 24 Jun 2019 17:13:07 -0700 Received: by devbig006.ftw2.facebook.com (Postfix, from userid 4523) id 362D562E206E; Mon, 24 Jun 2019 17:13:05 -0700 (PDT) Smtp-Origin-Hostprefix: devbig From: Song Liu Smtp-Origin-Hostname: devbig006.ftw2.facebook.com To: , , CC: , , , , , , Song Liu Smtp-Origin-Cluster: ftw2c04 Subject: [PATCH v9 5/6] mm,thp: add read-only THP support for (non-shmem) FS Date: Mon, 24 Jun 2019 17:12:45 -0700 Message-ID: <20190625001246.685563-6-songliubraving@fb.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20190625001246.685563-1-songliubraving@fb.com> References: <20190625001246.685563-1-songliubraving@fb.com> X-FB-Internal: Safe MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:,, definitions=2019-06-24_16:,, signatures=0 X-Proofpoint-Spam-Details: rule=fb_default_notspam policy=fb_default score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1906250000 X-FB-Internal: deliver X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP This patch is (hopefully) the first step to enable THP for non-shmem filesystems. This patch enables an application to put part of its text sections to THP via madvise, for example: madvise((void *)0x600000, 0x200000, MADV_HUGEPAGE); We tried to reuse the logic for THP on tmpfs. Currently, write is not supported for non-shmem THP. khugepaged will only process vma with VM_DENYWRITE. sys_mmap() ignores VM_DENYWRITE requests (see ksys_mmap_pgoff). The only way to create vma with VM_DENYWRITE is execve(). This requirement limits non-shmem THP to text sections. The next patch will handle writes, which would only happen when the all the vmas with VM_DENYWRITE are unmapped. An EXPERIMENTAL config, READ_ONLY_THP_FOR_FS, is added to gate this feature. Acked-by: Rik van Riel Signed-off-by: Song Liu --- mm/Kconfig | 11 ++++++ mm/filemap.c | 4 +-- mm/khugepaged.c | 94 +++++++++++++++++++++++++++++++++++++++++-------- mm/rmap.c | 12 ++++--- 4 files changed, 100 insertions(+), 21 deletions(-) diff --git a/mm/Kconfig b/mm/Kconfig index f0c76ba47695..0a8fd589406d 100644 --- a/mm/Kconfig +++ b/mm/Kconfig @@ -762,6 +762,17 @@ config GUP_BENCHMARK See tools/testing/selftests/vm/gup_benchmark.c +config READ_ONLY_THP_FOR_FS + bool "Read-only THP for filesystems (EXPERIMENTAL)" + depends on TRANSPARENT_HUGE_PAGECACHE && SHMEM + + help + Allow khugepaged to put read-only file-backed pages in THP. + + This is marked experimental because it is a new feature. Write + support of file THPs will be developed in the next few release + cycles. + config ARCH_HAS_PTE_SPECIAL bool diff --git a/mm/filemap.c b/mm/filemap.c index 5f072a113535..e79ceccdc6df 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -203,8 +203,8 @@ static void unaccount_page_cache_page(struct address_space *mapping, __mod_node_page_state(page_pgdat(page), NR_SHMEM, -nr); if (PageTransHuge(page)) __dec_node_page_state(page, NR_SHMEM_THPS); - } else { - VM_BUG_ON_PAGE(PageTransHuge(page), page); + } else if (PageTransHuge(page)) { + __dec_node_page_state(page, NR_FILE_THPS); } /* diff --git a/mm/khugepaged.c b/mm/khugepaged.c index 158cad542627..acbbbeaa083c 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -48,6 +48,7 @@ enum scan_result { SCAN_CGROUP_CHARGE_FAIL, SCAN_EXCEED_SWAP_PTE, SCAN_TRUNCATED, + SCAN_PAGE_HAS_PRIVATE, }; #define CREATE_TRACE_POINTS @@ -404,7 +405,11 @@ static bool hugepage_vma_check(struct vm_area_struct *vma, (vm_flags & VM_NOHUGEPAGE) || test_bit(MMF_DISABLE_THP, &vma->vm_mm->flags)) return false; - if (shmem_file(vma->vm_file)) { + + if (shmem_file(vma->vm_file) || + (IS_ENABLED(CONFIG_READ_ONLY_THP_FOR_FS) && + vma->vm_file && + (vm_flags & VM_DENYWRITE))) { if (!IS_ENABLED(CONFIG_TRANSPARENT_HUGE_PAGECACHE)) return false; return IS_ALIGNED((vma->vm_start >> PAGE_SHIFT) - vma->vm_pgoff, @@ -456,8 +461,9 @@ int khugepaged_enter_vma_merge(struct vm_area_struct *vma, unsigned long hstart, hend; /* - * khugepaged does not yet work on non-shmem files or special - * mappings. And file-private shmem THP is not supported. + * khugepaged only supports read-only files for non-shmem files. + * khugepaged does not yet work on special mappings. And + * file-private shmem THP is not supported. */ if (!hugepage_vma_check(vma, vm_flags)) return 0; @@ -1287,12 +1293,12 @@ static void retract_page_tables(struct address_space *mapping, pgoff_t pgoff) } /** - * collapse_file - collapse small tmpfs/shmem pages into huge one. + * collapse_file - collapse filemap/tmpfs/shmem pages into huge one. * * Basic scheme is simple, details are more complex: * - allocate and lock a new huge page; * - scan page cache replacing old pages with the new one - * + swap in pages if necessary; + * + swap/gup in pages if necessary; * + fill in gaps; * + keep old pages around in case rollback is required; * - if replacing succeeds: @@ -1316,7 +1322,9 @@ static void collapse_file(struct mm_struct *mm, LIST_HEAD(pagelist); XA_STATE_ORDER(xas, &mapping->i_pages, start, HPAGE_PMD_ORDER); int nr_none = 0, result = SCAN_SUCCEED; + bool is_shmem = shmem_file(file); + VM_BUG_ON(!IS_ENABLED(CONFIG_READ_ONLY_THP_FOR_FS) && !is_shmem); VM_BUG_ON(start & (HPAGE_PMD_NR - 1)); /* Only allocate from the target node */ @@ -1348,7 +1356,8 @@ static void collapse_file(struct mm_struct *mm, } while (1); __SetPageLocked(new_page); - __SetPageSwapBacked(new_page); + if (is_shmem) + __SetPageSwapBacked(new_page); new_page->index = start; new_page->mapping = mapping; @@ -1363,7 +1372,7 @@ static void collapse_file(struct mm_struct *mm, struct page *page = xas_next(&xas); VM_BUG_ON(index != xas.xa_index); - if (!page) { + if (is_shmem && !page) { /* * Stop if extent has been truncated or hole-punched, * and is now completely empty. @@ -1384,7 +1393,7 @@ static void collapse_file(struct mm_struct *mm, continue; } - if (xa_is_value(page) || !PageUptodate(page)) { + if (is_shmem && (xa_is_value(page) || !PageUptodate(page))) { xas_unlock_irq(&xas); /* swap in or instantiate fallocated page */ if (shmem_getpage(mapping->host, index, &page, @@ -1392,6 +1401,29 @@ static void collapse_file(struct mm_struct *mm, result = SCAN_FAIL; goto xa_unlocked; } + } else if (!page || xa_is_value(page)) { + xas_unlock_irq(&xas); + page_cache_sync_readahead(mapping, &file->f_ra, file, + index, PAGE_SIZE); + /* drain pagevecs to help isolate_lru_page() */ + lru_add_drain(); + page = find_lock_page(mapping, index); + if (unlikely(page == NULL)) { + result = SCAN_FAIL; + goto xa_unlocked; + } + } else if (!PageUptodate(page)) { + VM_BUG_ON(is_shmem); + xas_unlock_irq(&xas); + wait_on_page_locked(page); + if (!trylock_page(page)) { + result = SCAN_PAGE_LOCK; + goto xa_unlocked; + } + get_page(page); + } else if (!is_shmem && PageDirty(page)) { + result = SCAN_FAIL; + goto xa_locked; } else if (trylock_page(page)) { get_page(page); xas_unlock_irq(&xas); @@ -1426,6 +1458,12 @@ static void collapse_file(struct mm_struct *mm, goto out_unlock; } + if (page_has_private(page) && + !try_to_release_page(page, GFP_KERNEL)) { + result = SCAN_PAGE_HAS_PRIVATE; + break; + } + if (page_mapped(page)) unmap_mapping_pages(mapping, index, 1, false); @@ -1463,12 +1501,18 @@ static void collapse_file(struct mm_struct *mm, goto xa_unlocked; } - __inc_node_page_state(new_page, NR_SHMEM_THPS); + if (is_shmem) + __inc_node_page_state(new_page, NR_SHMEM_THPS); + else + __inc_node_page_state(new_page, NR_FILE_THPS); + if (nr_none) { struct zone *zone = page_zone(new_page); __mod_node_page_state(zone->zone_pgdat, NR_FILE_PAGES, nr_none); - __mod_node_page_state(zone->zone_pgdat, NR_SHMEM, nr_none); + if (is_shmem) + __mod_node_page_state(zone->zone_pgdat, + NR_SHMEM, nr_none); } xa_locked: @@ -1506,10 +1550,15 @@ static void collapse_file(struct mm_struct *mm, SetPageUptodate(new_page); page_ref_add(new_page, HPAGE_PMD_NR - 1); - set_page_dirty(new_page); mem_cgroup_commit_charge(new_page, memcg, false, true); + + if (is_shmem) { + set_page_dirty(new_page); + lru_cache_add_anon(new_page); + } else { + lru_cache_add_file(new_page); + } count_memcg_events(memcg, THP_COLLAPSE_ALLOC, 1); - lru_cache_add_anon(new_page); /* * Remove pte page tables, so we can re-fault the page as huge. @@ -1524,7 +1573,9 @@ static void collapse_file(struct mm_struct *mm, /* Something went wrong: roll back page cache changes */ xas_lock_irq(&xas); mapping->nrpages -= nr_none; - shmem_uncharge(mapping->host, nr_none); + + if (is_shmem) + shmem_uncharge(mapping->host, nr_none); xas_set(&xas, start); xas_for_each(&xas, page, end - 1) { @@ -1607,6 +1658,17 @@ static void khugepaged_scan_file(struct mm_struct *mm, break; } + if (page_has_private(page) && trylock_page(page)) { + int ret; + + ret = try_to_release_page(page, GFP_KERNEL); + unlock_page(page); + if (!ret) { + result = SCAN_PAGE_HAS_PRIVATE; + break; + } + } + if (page_count(page) != 1 + page_mapcount(page)) { result = SCAN_PAGE_COUNT; break; @@ -1713,11 +1775,13 @@ static unsigned int khugepaged_scan_mm_slot(unsigned int pages, VM_BUG_ON(khugepaged_scan.address < hstart || khugepaged_scan.address + HPAGE_PMD_SIZE > hend); - if (shmem_file(vma->vm_file)) { + if (vma->vm_file) { struct file *file; pgoff_t pgoff = linear_page_index(vma, khugepaged_scan.address); - if (!shmem_huge_enabled(vma)) + + if (shmem_file(vma->vm_file) + && !shmem_huge_enabled(vma)) goto skip; file = get_file(vma->vm_file); up_read(&mm->mmap_sem); diff --git a/mm/rmap.c b/mm/rmap.c index e5dfe2ae6b0d..87cfa2c19eda 100644 --- a/mm/rmap.c +++ b/mm/rmap.c @@ -1192,8 +1192,10 @@ void page_add_file_rmap(struct page *page, bool compound) } if (!atomic_inc_and_test(compound_mapcount_ptr(page))) goto out; - VM_BUG_ON_PAGE(!PageSwapBacked(page), page); - __inc_node_page_state(page, NR_SHMEM_PMDMAPPED); + if (PageSwapBacked(page)) + __inc_node_page_state(page, NR_SHMEM_PMDMAPPED); + else + __inc_node_page_state(page, NR_FILE_PMDMAPPED); } else { if (PageTransCompound(page) && page_mapping(page)) { VM_WARN_ON_ONCE(!PageLocked(page)); @@ -1232,8 +1234,10 @@ static void page_remove_file_rmap(struct page *page, bool compound) } if (!atomic_add_negative(-1, compound_mapcount_ptr(page))) goto out; - VM_BUG_ON_PAGE(!PageSwapBacked(page), page); - __dec_node_page_state(page, NR_SHMEM_PMDMAPPED); + if (PageSwapBacked(page)) + __dec_node_page_state(page, NR_SHMEM_PMDMAPPED); + else + __dec_node_page_state(page, NR_FILE_PMDMAPPED); } else { if (!atomic_add_negative(-1, &page->_mapcount)) goto out; From patchwork Tue Jun 25 00:12:46 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Song Liu X-Patchwork-Id: 11014421 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 99E1914E5 for ; Tue, 25 Jun 2019 00:13:14 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 8949228A91 for ; Tue, 25 Jun 2019 00:13:14 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 7A8C828AD2; Tue, 25 Jun 2019 00:13:14 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id A96D528A91 for ; Tue, 25 Jun 2019 00:13:13 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 0067C6B000D; Mon, 24 Jun 2019 20:13:11 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id EAB588E0005; Mon, 24 Jun 2019 20:13:10 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D2E068E0003; Mon, 24 Jun 2019 20:13:10 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-yw1-f71.google.com (mail-yw1-f71.google.com [209.85.161.71]) by kanga.kvack.org (Postfix) with ESMTP id AB6D06B000C for ; Mon, 24 Jun 2019 20:13:10 -0400 (EDT) Received: by mail-yw1-f71.google.com with SMTP id y205so18827851ywy.19 for ; Mon, 24 Jun 2019 17:13:10 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:smtp-origin-hostprefix:from :smtp-origin-hostname:to:cc:smtp-origin-cluster:subject:date :message-id:in-reply-to:references:mime-version; bh=3XZFERUpQR9pfQReUygYqP0Ugwxnn7ZlmK3Hj2ioXA8=; b=qJaK3vviaTsax6edArKr7mOHB6CgwIySLnDL6QpPbhrzpeENvC6QCdq6i8UXva3IPB T3AKxCYbpu7/mSGLAUS8IdcjTtyq10be1tnQgWBF+Gto24nLYfBqGPEIVTGRbSqHIBCW WDlA9grX7q1pvndlh3pRfz33Vk4c6f2OWptrzpRZDptJ8GqEmqjoOYhUNEXd4j2r+kR8 66LpjH11VN9+2LZvXC6KFV+acfgPEV2hpbxCRIPpAGMYsKSW047qyt7E/T/7TOtClJZ/ /q/IZfkyoUTg8oDBO2r0kVzwHQB2jtnBBB+CN3EeBdOTCYPiRtW4OiqCcYGX/Fd/aCP5 XRAg== X-Gm-Message-State: APjAAAUmZmYY5ETNxVZwK1lgNbnVM0qfCYYP4P+i/oCnscs7C756DV7v vK3tTtDmPFIwJOjRDeufjFpXQoiVG0qkeS8I6RomRBORjxEPML7RWDzPMbHbvFhsj0vGC8EejKB 2JTRkOAgiY1bBSGtbJr1HIjTb6H5XWekIi7s5tx4oSpcxjZ2yTrmJkIFmMGlbXZezPA== X-Received: by 2002:a81:bc41:: with SMTP id b1mr70500882ywl.404.1561421590465; Mon, 24 Jun 2019 17:13:10 -0700 (PDT) X-Google-Smtp-Source: APXvYqwVgj6D/WWWVTVRu6oGsyzwH8uvFpskpwus6QvfQw80colQmGhDuetHNsn+gDVOWTTpZwgI X-Received: by 2002:a81:bc41:: with SMTP id b1mr70500857ywl.404.1561421589766; Mon, 24 Jun 2019 17:13:09 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1561421589; cv=none; d=google.com; s=arc-20160816; b=s0cDgHTgYpBnjZRusIXAkHFmBxm9XBKA5SDQEL4YUuEIxyXQ+2gy6cB12uFD/ryCPw mMBFfDNrP5D5nkFSBKMPdULK3gnbbPIAaUFFwSuKMlyiDKhWYFie4MENJBO3Zf3kp61b ALcq09wZRllgCeOP8BYBSd1bWld1kAV9hKyHQvIsMMJuVwbDH3ev31m8JPCHwsu7E1pF zxu7U90t2w7bEIewh1DhK6pZLmNeYmhnZcDB8olRJimSoJJZ/81b1jUoYuZ4XQaC30R8 pXH54QCWUZryNzYhX4gP8kQ3loZ1deCoCngLLJAPHOYa0KMNIDfAPtwIRac+uL6b4Tbx Jc9g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=mime-version:references:in-reply-to:message-id:date:subject :smtp-origin-cluster:cc:to:smtp-origin-hostname:from :smtp-origin-hostprefix:dkim-signature; bh=3XZFERUpQR9pfQReUygYqP0Ugwxnn7ZlmK3Hj2ioXA8=; b=kFJDVR+7iYUaokG374mN4E/Y52P8FWppybtYZYHHesJIFDOITZcwU7fzjDnpV3E0MY BGDvB5H2086y0aTitRPlQOzJ0PEhS96gh0F696Er6QI831X6oNiM69yeh2v2jl/YCyyq B3gf/EG7pb2wVjMxiUZLAVDvctjHcMEklNOnB5AMNQD9sl0qXgNa3jKr1Dpnu8V/Oz9f 0chzsLNewpejd/diPMNrSWrHKKgVlBdZDVCAxQ/2PT125Rs04TXeAzRFrB1ubyh0X2Xf EoKQwV7NO7ywbRRCSjMkv6Sn8JSXVPG94IR2j3CwvqjydNcKMSt1PwyW+Eaup2ozY2bh XIlg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@fb.com header.s=facebook header.b=gP9uYvk9; spf=pass (google.com: domain of prvs=1079b839a8=songliubraving@fb.com designates 67.231.153.30 as permitted sender) smtp.mailfrom="prvs=1079b839a8=songliubraving@fb.com"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=fb.com Received: from mx0b-00082601.pphosted.com (mx0b-00082601.pphosted.com. [67.231.153.30]) by mx.google.com with ESMTPS id t198si1026131ywc.273.2019.06.24.17.13.09 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 24 Jun 2019 17:13:09 -0700 (PDT) Received-SPF: pass (google.com: domain of prvs=1079b839a8=songliubraving@fb.com designates 67.231.153.30 as permitted sender) client-ip=67.231.153.30; Authentication-Results: mx.google.com; dkim=pass header.i=@fb.com header.s=facebook header.b=gP9uYvk9; spf=pass (google.com: domain of prvs=1079b839a8=songliubraving@fb.com designates 67.231.153.30 as permitted sender) smtp.mailfrom="prvs=1079b839a8=songliubraving@fb.com"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=fb.com Received: from pps.filterd (m0148460.ppops.net [127.0.0.1]) by mx0a-00082601.pphosted.com (8.16.0.27/8.16.0.27) with SMTP id x5P09dlU018475 for ; Mon, 24 Jun 2019 17:13:09 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fb.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type; s=facebook; bh=3XZFERUpQR9pfQReUygYqP0Ugwxnn7ZlmK3Hj2ioXA8=; b=gP9uYvk9m8yAdtzGQs0XBLX8EnTnPpfzTgbRHqWiehHM03h4T22lEs/DkZ5H4o6gkQ7K KXlIyL0QYQ9hjbl6u4GESqmPMbO+19kdVAeiH2mWYhOqCTCGyyK8b9luL+QmCW/ac1m2 4Jbmts9ZD8cF41JQBoBYvFBTYtSUv33Go8o= Received: from maileast.thefacebook.com ([163.114.130.16]) by mx0a-00082601.pphosted.com with ESMTP id 2tawbtarag-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NOT) for ; Mon, 24 Jun 2019 17:13:09 -0700 Received: from mx-out.facebook.com (2620:10d:c0a8:1b::d) by mail.thefacebook.com (2620:10d:c0a8:82::f) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.1713.5; Mon, 24 Jun 2019 17:13:08 -0700 Received: by devbig006.ftw2.facebook.com (Postfix, from userid 4523) id C93A962E206E; Mon, 24 Jun 2019 17:13:07 -0700 (PDT) Smtp-Origin-Hostprefix: devbig From: Song Liu Smtp-Origin-Hostname: devbig006.ftw2.facebook.com To: , , CC: , , , , , , Song Liu Smtp-Origin-Cluster: ftw2c04 Subject: [PATCH v9 6/6] mm,thp: avoid writes to file with THP in pagecache Date: Mon, 24 Jun 2019 17:12:46 -0700 Message-ID: <20190625001246.685563-7-songliubraving@fb.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20190625001246.685563-1-songliubraving@fb.com> References: <20190625001246.685563-1-songliubraving@fb.com> X-FB-Internal: Safe MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:,, definitions=2019-06-24_16:,, signatures=0 X-Proofpoint-Spam-Details: rule=fb_default_notspam policy=fb_default score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=801 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1906250000 X-FB-Internal: deliver X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP In previous patch, an application could put part of its text section in THP via madvise(). These THPs will be protected from writes when the application is still running (TXTBSY). However, after the application exits, the file is available for writes. This patch avoids writes to file THP by dropping page cache for the file when the file is open for write. A new counter nr_thps is added to struct address_space. In do_last(), if the file is open for write and nr_thps is non-zero, we drop page cache for the whole file. Reported-by: kbuild test robot Signed-off-by: Song Liu Acked-by: Rik van Riel --- fs/inode.c | 3 +++ fs/namei.c | 23 ++++++++++++++++++++++- include/linux/fs.h | 32 ++++++++++++++++++++++++++++++++ mm/filemap.c | 1 + mm/khugepaged.c | 4 +++- 5 files changed, 61 insertions(+), 2 deletions(-) diff --git a/fs/inode.c b/fs/inode.c index df6542ec3b88..518113a4e219 100644 --- a/fs/inode.c +++ b/fs/inode.c @@ -181,6 +181,9 @@ int inode_init_always(struct super_block *sb, struct inode *inode) mapping->flags = 0; mapping->wb_err = 0; atomic_set(&mapping->i_mmap_writable, 0); +#ifdef CONFIG_READ_ONLY_THP_FOR_FS + atomic_set(&mapping->nr_thps, 0); +#endif mapping_set_gfp_mask(mapping, GFP_HIGHUSER_MOVABLE); mapping->private_data = NULL; mapping->writeback_index = 0; diff --git a/fs/namei.c b/fs/namei.c index 20831c2fbb34..3d95e94029cc 100644 --- a/fs/namei.c +++ b/fs/namei.c @@ -3249,6 +3249,23 @@ static int lookup_open(struct nameidata *nd, struct path *path, return error; } +/* + * The file is open for write, so it is not mmapped with VM_DENYWRITE. If + * it still has THP in page cache, drop the whole file from pagecache + * before processing writes. This helps us avoid handling write back of + * THP for now. + */ +static inline void release_file_thp(struct file *file) +{ + if (IS_ENABLED(CONFIG_READ_ONLY_THP_FOR_FS)) { + struct inode *inode = file_inode(file); + + if (inode_is_open_for_write(inode) && + filemap_nr_thps(inode->i_mapping)) + truncate_pagecache(inode, 0); + } +} + /* * Handle the last step of open() */ @@ -3418,7 +3435,11 @@ static int do_last(struct nameidata *nd, goto out; opened: error = ima_file_check(file, op->acc_mode); - if (!error && will_truncate) + if (error) + goto out; + + release_file_thp(file); + if (will_truncate) error = handle_truncate(file); out: if (unlikely(error > 0)) { diff --git a/include/linux/fs.h b/include/linux/fs.h index f7fdfe93e25d..082fc581c7fc 100644 --- a/include/linux/fs.h +++ b/include/linux/fs.h @@ -427,6 +427,7 @@ int pagecache_write_end(struct file *, struct address_space *mapping, * @i_pages: Cached pages. * @gfp_mask: Memory allocation flags to use for allocating pages. * @i_mmap_writable: Number of VM_SHARED mappings. + * @nr_thps: Number of THPs in the pagecache (non-shmem only). * @i_mmap: Tree of private and shared mappings. * @i_mmap_rwsem: Protects @i_mmap and @i_mmap_writable. * @nrpages: Number of page entries, protected by the i_pages lock. @@ -444,6 +445,10 @@ struct address_space { struct xarray i_pages; gfp_t gfp_mask; atomic_t i_mmap_writable; +#ifdef CONFIG_READ_ONLY_THP_FOR_FS + /* number of thp, only for non-shmem files */ + atomic_t nr_thps; +#endif struct rb_root_cached i_mmap; struct rw_semaphore i_mmap_rwsem; unsigned long nrpages; @@ -2790,6 +2795,33 @@ static inline errseq_t filemap_sample_wb_err(struct address_space *mapping) return errseq_sample(&mapping->wb_err); } +static inline int filemap_nr_thps(struct address_space *mapping) +{ +#ifdef CONFIG_READ_ONLY_THP_FOR_FS + return atomic_read(&mapping->nr_thps); +#else + return 0; +#endif +} + +static inline void filemap_nr_thps_inc(struct address_space *mapping) +{ +#ifdef CONFIG_READ_ONLY_THP_FOR_FS + atomic_inc(&mapping->nr_thps); +#else + WARN_ON_ONCE(1); +#endif +} + +static inline void filemap_nr_thps_dec(struct address_space *mapping) +{ +#ifdef CONFIG_READ_ONLY_THP_FOR_FS + atomic_dec(&mapping->nr_thps); +#else + WARN_ON_ONCE(1); +#endif +} + extern int vfs_fsync_range(struct file *file, loff_t start, loff_t end, int datasync); extern int vfs_fsync(struct file *file, int datasync); diff --git a/mm/filemap.c b/mm/filemap.c index e79ceccdc6df..a8e86c136381 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -205,6 +205,7 @@ static void unaccount_page_cache_page(struct address_space *mapping, __dec_node_page_state(page, NR_SHMEM_THPS); } else if (PageTransHuge(page)) { __dec_node_page_state(page, NR_FILE_THPS); + filemap_nr_thps_dec(mapping); } /* diff --git a/mm/khugepaged.c b/mm/khugepaged.c index acbbbeaa083c..0bbc6be51197 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -1503,8 +1503,10 @@ static void collapse_file(struct mm_struct *mm, if (is_shmem) __inc_node_page_state(new_page, NR_SHMEM_THPS); - else + else { __inc_node_page_state(new_page, NR_FILE_THPS); + filemap_nr_thps_inc(mapping); + } if (nr_none) { struct zone *zone = page_zone(new_page);