From patchwork Thu Jun 20 16:19:03 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Takaya Saeki X-Patchwork-Id: 13705812 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 372E0C27C79 for ; Thu, 20 Jun 2024 16:19:15 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id ABBAC8D00BF; Thu, 20 Jun 2024 12:19:14 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id A6BE68D00AF; Thu, 20 Jun 2024 12:19:14 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 90CEB8D00BF; Thu, 20 Jun 2024 12:19:14 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 736888D00AF for ; Thu, 20 Jun 2024 12:19:14 -0400 (EDT) Received: from smtpin30.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 1ED53A00B7 for ; Thu, 20 Jun 2024 16:19:14 +0000 (UTC) X-FDA: 82251776628.30.639C51F Received: from mail-pl1-f178.google.com (mail-pl1-f178.google.com [209.85.214.178]) by imf22.hostedemail.com (Postfix) with ESMTP id 4D815C001B for ; Thu, 20 Jun 2024 16:19:12 +0000 (UTC) Authentication-Results: imf22.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=m346NW2G; spf=pass (imf22.hostedemail.com: domain of takayas@chromium.org designates 209.85.214.178 as permitted sender) smtp.mailfrom=takayas@chromium.org; dmarc=pass (policy=none) header.from=chromium.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1718900346; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:references:dkim-signature; bh=gDTwzMRp7r7igVV+ZTOkGMM8uRoM9/iMzbfcbOs3NYs=; b=QNl1ijB9s7YbIrMvo1f4U6SY/8n5Vch+FXXzk58+ucyYnNofjX8wp+ZFDHZwTQAML3Rt20 fomoSiRhWr0dnFUxeu8C0ISTxcyguXB9xbVndWXfiGOhDJTy/AhrRheE+P7tlRmE+VJMjp lHb5rPEhszAIICfHKJjf5l65zNyweUU= ARC-Authentication-Results: i=1; imf22.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=m346NW2G; spf=pass (imf22.hostedemail.com: domain of takayas@chromium.org designates 209.85.214.178 as permitted sender) smtp.mailfrom=takayas@chromium.org; dmarc=pass (policy=none) header.from=chromium.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1718900347; a=rsa-sha256; cv=none; b=Dg+kKBoDHwnefUknCqQTMmAL33Eks6fIOaRTHhto5mSnse15MPQB4LhLmqE/G/aW4lmk3T DiS7Nq+A3BcHwW2h3QOwZPhMwcZTepRWDiNWyukV2oNhSFr0KrdQnfdtuqkP/HSVNip6Qd laOuE+8Jb1HkEqvmeyyE7acuqJTP8Cs= Received: by mail-pl1-f178.google.com with SMTP id d9443c01a7336-1f9cd92b146so7374355ad.3 for ; Thu, 20 Jun 2024 09:19:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1718900351; x=1719505151; darn=kvack.org; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:from:to:cc:subject:date:message-id:reply-to; bh=gDTwzMRp7r7igVV+ZTOkGMM8uRoM9/iMzbfcbOs3NYs=; b=m346NW2G6HCAXOCZzAlpM4Ri4tleh+3S2YKCRV72wJnXRDje01TilC3s+K8EuZDKf0 +9yACvbcOhR+S1GCsDHIorlyhROPUiZL9YJMZFd0UvA+2XHLF70LvXq29LouEPgpZD7j itqFRC0cv+yHghJF+nmNymMQYMDIm0HK7SPyo= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1718900351; x=1719505151; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=gDTwzMRp7r7igVV+ZTOkGMM8uRoM9/iMzbfcbOs3NYs=; b=ISckXQ5vUpES21CKT2tEoQ5EYTsxhXiNyMX385VTBzj5mYAkL3/oIIcicagetXbhDr pqWiK6ReiSdfWHFU8v7eEc3i2GNXxkOqEgT9NBC/Uze1s9N1n1ovUvCwwIWlH30GSLT8 70+QgE06du7Ti/+3IkH4ZYGAkBKvWheFN7NJ9vzSP/sniKqw8KKpOOEKd6xjdm7vkVH9 ClRgXLChwLuZOi0baY0TPx++5oJWSdb06beSF+mZysA5vxa/xvPl7ehrNSr84Dl5cLWl AvvrDmTnp2OU8va6oam3LsaOJFcfrmBDAQDVUe4bqA+yHQ52AQLgQGvTuLEHeNTcyw+n ertg== X-Forwarded-Encrypted: i=1; AJvYcCX2j5dIatFxyfFVcZuzU5gkeu9EQjKsdWc+4FRSGSuF+Nngj3vWyuiBpkL37h8ZYnBRvTHEV4/M7v12nF7IHnglcqc= X-Gm-Message-State: AOJu0YxTem2Y7wgLB6HTgx8JQ/iIEno4pSMVAAKO/7215MtvsXWwBTLy xVdSZnrDN7RAcLmCIHloIbWYsX5TtYwCtB4C/fMd/P18TvsheB28tSCuD56w X-Google-Smtp-Source: AGHT+IFSW1DtPmzfkyXjIzCznsvC4863obnU5C/WxGZH3fmU++B5UFx8192lW0ggQO9pfsgefF865g== X-Received: by 2002:a17:903:11cc:b0:1f6:ea71:34ee with SMTP id d9443c01a7336-1f9aa3ce954mr69656465ad.16.1718900351103; Thu, 20 Jun 2024 09:19:11 -0700 (PDT) Received: from localhost (148.175.199.104.bc.googleusercontent.com. [104.199.175.148]) by smtp.gmail.com with UTF8SMTPSA id d9443c01a7336-1f9c72085f9sm18696855ad.79.2024.06.20.09.19.08 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Thu, 20 Jun 2024 09:19:10 -0700 (PDT) From: Takaya Saeki To: Matthew Wilcox , Andrew Morton Cc: Steven Rostedt , Masami Hiramatsu , Mathieu Desnoyers , Junichi Uekawa , linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, Takaya Saeki Subject: [PATCH v2] filemap: add trace events for get_pages, map_pages, and fault Date: Thu, 20 Jun 2024 16:19:03 +0000 Message-ID: <20240620161903.3176859-1-takayas@chromium.org> X-Mailer: git-send-email 2.45.2.627.g7a2c4fd464-goog MIME-Version: 1.0 X-Rspamd-Server: rspam11 X-Rspamd-Queue-Id: 4D815C001B X-Stat-Signature: pt463sweb4mdo37wumb1hoh584d96roc X-Rspam-User: X-HE-Tag: 1718900352-400169 X-HE-Meta: U2FsdGVkX1/ebnv4+qBatDms1iyiPO6FMpXoUn9qZbVSMj0uMGoaPzd9LujlnlMG4gRO0EVaurt8puEt72P8kfGUa/9/vpyFWzfeGfbNguutA4VwCUJsAO+KhmebYN52tjTseXpaqJxbvHFWTFOQPub9x0misTD3U8S68B4PiKhUNqsHrGFX9EkyyjenpJfRlMC2FIQ4Xp01wj1Xf4g1SAZJd+QKdHLR5XcOfinyFnYg9Nziq3FQ7z49K7iKGmsrbLD49KJak55rzh2uxPhFO/tLtOjf4UX9TjRVr5RTJgva5TGfLXMMwfxqqqWBL4eF0sowZV84Dh5uSyuQWHxhEezYbiBTXNuVDvI/qyXj2BRmMigqWDwcJDKpSZghtVP0NsaNItJVgM3PScL/wpgOwHHLypOXpfPbLQVyXn7dJI5U1NWaLSTWY2QS+n5KRJ7/f6cqqZZlbSNTLnUbyMCVgZa05cKcjr2ZiAS1tnqPJbvqGNa4u9zVRPU2NQ9ZIsDx50PZpU2J3r1e7uelv1n3IQ4+D4U5Mfok5gLnn/84B6pWiuHFMfgeQmLACw/aPk6pl5kDC+g7QnRa0uKt+Apz4KKvYvVb/12gl/U3DI2+aj05Sy8UUd9wcxTz/Fa8JHF3uGfhCOPyJ5xkUcVnHkxbWYxdqBYIrE7nKIjjSOJvrmh57d4Y9g5BGeUHWHAjQyfvbWYe6/w2L4lH1iODTlD25cxpFml5VrnO4WfuIaNkZ8Vqeh3K7dic4ViWzES1oQ8fjBEqyO8otSfrlofwvbZ6+VSBf+OdUIYujNn5N0pzns2yrHmJFwGA8fL2w+UC/LTPi5Edt1N9UHuZ+Z3AY+Odh582eK3PXWXBSZWGcXRCe8+TiD8OhU5y/VGPp6HQFJkoQ8zXMAmgAzQ3uFAmKTX1A3hXGo7+6yqbipTvi3L21zon9ulTZCEjCQDUIx5URiGi01Z0VVA06IJg+BmiX5z cACoT0OD 2c+j7h5BQVbZIoEsMLBMeCOd0qla9pzuMI7r3+HLVygDx5dpLIdy5zC98Ez3wzFkHh5Fe9r8fFxcBTicEzu8VbSBy0XRB7sLTltCapx2OEmSWXOcbwTU9lhFmr8IJaOaP6eGmJ+ZiOwN5In8E1BfndAg7ltrdy3me6zfxSvJSG0kcfkJU0ogUMHT8ERB28QPDS7ZGy4webyICQ4orNUmUB5ZJAPtG7b9JMJAl6W64+isyRXlKOH1YbfV0P1pAM/rBFFGx2bfm4bKtYfMJiSA4aBjW2Q2PcyH1jhSNCtc+IrbCEQx0roZOTeRgXvsdnMf/c8XVGJBHD3GOydHKIrfS71E9nfsabzc+GSDxiD2pBb3Izf34AKQa4JlpAdUMYXRkLKk2 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: To allow precise tracking of page caches accessed, add new tracepoints that trigger when a process actually accesses them. The ureadahead program used by ChromeOS traces the disk access of programs as they start up at boot up. It uses mincore(2) or the 'mm_filemap_add_to_page_cache' trace event to accomplish this. It stores this information in a "pack" file and on subsequent boots, it will read the pack file and call readahead(2) on the information so that disk storage can be loaded into RAM before the applications actually need it. A problem we see is that due to the kernel's readahead algorithm that can aggressively pull in more data than needed (to try and accomplish the same goal) and this data is also recorded. The end result is that the pack file contains a lot of pages on disk that are never actually used. Calling readahead(2) on these unused pages can slow down the system boot up times. To solve this, add 3 new trace events, get_pages, map_pages, and fault. These will be used to trace the pages are not only pulled in from disk, but are actually used by the application. Only those pages will be stored in the pack file, and this helps out the performance of boot up. With the combination of these 3 new trace events and mm_filemap_add_to_page_cache, we observed a reduction in the pack file by 7.3% - 20% on ChromeOS varying by device. Signed-off-by: Takaya Saeki Reviewed-by: Masami Hiramatsu (Google) Reviewed-by: Steven Rostedt (Google) --- Changelog between v2 and v1 - Fix a file offset type usage by casting pgoff_t to loff_t - Fixed format string of dev and inode include/trace/events/filemap.h | 84 ++++++++++++++++++++++++++++++++++ mm/filemap.c | 4 ++ 2 files changed, 88 insertions(+) V1:https://lore.kernel.org/all/20240618093656.1944210-1-takayas@chromium.org/ diff --git a/include/trace/events/filemap.h b/include/trace/events/filemap.h index 46c89c1e460c..3a94bd633bf0 100644 --- a/include/trace/events/filemap.h +++ b/include/trace/events/filemap.h @@ -56,6 +56,90 @@ DEFINE_EVENT(mm_filemap_op_page_cache, mm_filemap_add_to_page_cache, TP_ARGS(folio) ); +DECLARE_EVENT_CLASS(mm_filemap_op_page_cache_range, + + TP_PROTO( + struct address_space *mapping, + pgoff_t index, + pgoff_t last_index + ), + + TP_ARGS(mapping, index, last_index), + + TP_STRUCT__entry( + __field(unsigned long, i_ino) + __field(dev_t, s_dev) + __field(unsigned long, index) + __field(unsigned long, last_index) + ), + + TP_fast_assign( + __entry->i_ino = mapping->host->i_ino; + if (mapping->host->i_sb) + __entry->s_dev = + mapping->host->i_sb->s_dev; + else + __entry->s_dev = mapping->host->i_rdev; + __entry->index = index; + __entry->last_index = last_index; + ), + + TP_printk( + "dev=%d:%d ino=%lx ofs=%lld max_ofs=%lld", + MAJOR(__entry->s_dev), + MINOR(__entry->s_dev), __entry->i_ino, + ((loff_t)__entry->index) << PAGE_SHIFT, + ((loff_t)__entry->last_index) << PAGE_SHIFT + ) +); + +DEFINE_EVENT(mm_filemap_op_page_cache_range, mm_filemap_get_pages, + TP_PROTO( + struct address_space *mapping, + pgoff_t index, + pgoff_t last_index + ), + TP_ARGS(mapping, index, last_index) +); + +DEFINE_EVENT(mm_filemap_op_page_cache_range, mm_filemap_map_pages, + TP_PROTO( + struct address_space *mapping, + pgoff_t index, + pgoff_t last_index + ), + TP_ARGS(mapping, index, last_index) +); + +TRACE_EVENT(mm_filemap_fault, + TP_PROTO(struct address_space *mapping, pgoff_t index), + + TP_ARGS(mapping, index), + + TP_STRUCT__entry( + __field(unsigned long, i_ino) + __field(dev_t, s_dev) + __field(unsigned long, index) + ), + + TP_fast_assign( + __entry->i_ino = mapping->host->i_ino; + if (mapping->host->i_sb) + __entry->s_dev = + mapping->host->i_sb->s_dev; + else + __entry->s_dev = mapping->host->i_rdev; + __entry->index = index; + ), + + TP_printk( + "dev=%d:%d ino=%lx ofs=%lld", + MAJOR(__entry->s_dev), + MINOR(__entry->s_dev), __entry->i_ino, + ((loff_t)__entry->index) << PAGE_SHIFT + ) +); + TRACE_EVENT(filemap_set_wb_err, TP_PROTO(struct address_space *mapping, errseq_t eseq), diff --git a/mm/filemap.c b/mm/filemap.c index 876cc64aadd7..39f9d7fb3d2c 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -2556,6 +2556,7 @@ static int filemap_get_pages(struct kiocb *iocb, size_t count, goto err; } + trace_mm_filemap_get_pages(mapping, index, last_index); return 0; err: if (err < 0) @@ -3286,6 +3287,8 @@ vm_fault_t filemap_fault(struct vm_fault *vmf) if (unlikely(index >= max_idx)) return VM_FAULT_SIGBUS; + trace_mm_filemap_fault(mapping, index); + /* * Do we have something in the page cache already? */ @@ -3652,6 +3655,7 @@ vm_fault_t filemap_map_pages(struct vm_fault *vmf, } while ((folio = next_uptodate_folio(&xas, mapping, end_pgoff)) != NULL); add_mm_counter(vma->vm_mm, folio_type, rss); pte_unmap_unlock(vmf->pte, vmf->ptl); + trace_mm_filemap_map_pages(mapping, start_pgoff, end_pgoff); out: rcu_read_unlock();