From patchwork Tue Aug 13 10:03:12 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Takaya Saeki X-Patchwork-Id: 13761691 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id E2C4DC52D7C for ; Tue, 13 Aug 2024 10:03:24 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 6621E6B00A4; Tue, 13 Aug 2024 06:03:24 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 5E87E6B00A9; Tue, 13 Aug 2024 06:03:24 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 460CE6B00AA; Tue, 13 Aug 2024 06:03:24 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 238266B00A4 for ; Tue, 13 Aug 2024 06:03:24 -0400 (EDT) Received: from smtpin19.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id D3B231C3361 for ; Tue, 13 Aug 2024 10:03:23 +0000 (UTC) X-FDA: 82446784686.19.6BBDAEC Received: from mail-pg1-f170.google.com (mail-pg1-f170.google.com [209.85.215.170]) by imf20.hostedemail.com (Postfix) with ESMTP id 05DF41C002C for ; Tue, 13 Aug 2024 10:03:20 +0000 (UTC) Authentication-Results: imf20.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=ceSLNaYZ; spf=pass (imf20.hostedemail.com: domain of takayas@chromium.org designates 209.85.215.170 as permitted sender) smtp.mailfrom=takayas@chromium.org; dmarc=pass (policy=none) header.from=chromium.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1723543330; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:references:dkim-signature; bh=NoxN63bBPaM/wCd8ZvlEoxvS8xfn7jCuCDDPzEXx9AA=; b=hHhB9VVv02wXok2As8VcU51npXiTHfBV6YHv8P20ub7zrHzZGfacZvaf5/Upg2QWS9ozIS pUjsM+k1Ki8wk2v2JdbSq8IqxF+n/eiMNPZgkqrhqod0AI8AOD3jFF1FPaJ3FHycIVTgFq iNy8IgnXR9ytbxSsLhPuILDmbKfyuPw= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1723543330; a=rsa-sha256; cv=none; b=H/Osczu2oiKsFkWFLZXNQhD45MtQdfR5aQlPkz9k7J98e5HVuQ78fc8HkQP/H1hKe7bSeU kxnGhwlzKAl0vRW+z1OzaJkQGxXvI3e8MIY9UtDMbUN3zFoH7PIPeAq/uTYC2UZe8jHGZU omyXqUT2WiKtHCK8epLnwAa6vBo8w9s= ARC-Authentication-Results: i=1; imf20.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=ceSLNaYZ; spf=pass (imf20.hostedemail.com: domain of takayas@chromium.org designates 209.85.215.170 as permitted sender) smtp.mailfrom=takayas@chromium.org; dmarc=pass (policy=none) header.from=chromium.org Received: by mail-pg1-f170.google.com with SMTP id 41be03b00d2f7-7a10b293432so3740407a12.0 for ; Tue, 13 Aug 2024 03:03:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1723543399; x=1724148199; darn=kvack.org; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:from:to:cc:subject:date:message-id:reply-to; bh=NoxN63bBPaM/wCd8ZvlEoxvS8xfn7jCuCDDPzEXx9AA=; b=ceSLNaYZMN3mTRrhHbR5FkoZqbkA+fhCaQBtOyI6ESJ2xdnO8bGuDCQXsW/VG5+Xk9 yREt70NXd+tH6WsqAj20a3BOxUCADG1t9WPDyprgmrL31bhLVxpZZdc3fuEPjn6TsuiC NELc+uPg60nrWhyh0AieYWSdoC4M+TRmAoUrY= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1723543399; x=1724148199; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=NoxN63bBPaM/wCd8ZvlEoxvS8xfn7jCuCDDPzEXx9AA=; b=YTUU7zu3VqvWf1a321BNDQl9GZ/hpIzktaW+dVphxQH+siWvt4jLpbsRLieHqxRxLT l8UZTnYid4PFiF+vgmy74c/udi/pNRVevg4WGgBHowUDBVoIiMDALHgGDYVdSBr2r5g2 u8TLl3HXZ41LnZxzCYj6j2Sxp7ejVEj4Joj/yUZ4JiIRJJR0IHuZZ1PiPw+4zGBi+kp1 yz0ykAkC/E2bhqd7Gjk9EtpeHQM5tQek0Rw1jiSTtKM/0IJQluq5N6oh2mHhh2Z/GSD7 kOMmkF8xRGFg8aWQZKchMQxok690QfBRIAIaSU6UwD29759rRlINtiDZ9haChyLJk8F3 KqtA== X-Forwarded-Encrypted: i=1; AJvYcCWr2QFdjMRSH7h+/YNf4jXJ/e6w+pWGKR4g2dm2o4+DEKsQUuXKAKF4WxCFF5LlM0WABO8X1sxwFBvso4WMs4C0w30= X-Gm-Message-State: AOJu0YwUqC7L0hgC80pBwgt+TDxm5rRpyvSfmbnr90u1qDx04VlYck5S crV/uTyQIpJX/QipDbB6iVxy7QAbAaRyaAfiGA7E7CL9cPxL6PlkO34u8Kwq X-Google-Smtp-Source: AGHT+IFosWR7zkd7jgDT2HviKT6vEjybZ5RhsLylZWQSu4p1a07i1ADAXGsNcDrKZxIxqrj/LlkKPw== X-Received: by 2002:a05:6a20:96c8:b0:1c8:d7b7:dcbd with SMTP id adf61e73a8af0-1c8d7b7dcf2mr2811688637.1.1723543399534; Tue, 13 Aug 2024 03:03:19 -0700 (PDT) Received: from localhost (0.223.81.34.bc.googleusercontent.com. [34.81.223.0]) by smtp.gmail.com with UTF8SMTPSA id 41be03b00d2f7-7c697a549d8sm944650a12.67.2024.08.13.03.03.16 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Tue, 13 Aug 2024 03:03:18 -0700 (PDT) From: Takaya Saeki To: Matthew Wilcox , Andrew Morton Cc: Steven Rostedt , Masami Hiramatsu , Mathieu Desnoyers , Junichi Uekawa , linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, Takaya Saeki Subject: [PATCH v3] filemap: add trace events for get_pages, map_pages, and fault Date: Tue, 13 Aug 2024 10:03:12 +0000 Message-ID: <20240813100312.3930505-1-takayas@chromium.org> X-Mailer: git-send-email 2.46.0.76.ge559c4bf1a-goog MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 05DF41C002C X-Stat-Signature: ekki8cq9yxp1oc3pgis3fqt31gwtgye5 X-HE-Tag: 1723543400-572032 X-HE-Meta: U2FsdGVkX1+2+B/z9wOaQF+5jdVJn+w3dMprNujawiiNdtCQq9a4tDBcvBfBk9g6/vN+KjSXgi6JwxMFl3FKAri0qRMqDZAvfpC8lY38KUia+XwitsisldcDHHIlJqE5jfF/VC1le3RlpGjIgS1XlvnLxrT01ZSPW4HkLSPMXQafPoFkzcZvoXf7HeaTZSfAJfTtn726YVmEvTyTBAjaFTa/5IO50116XLclLaObrX9BFiiAZCMVhpd7cFL625IbaFZCuW3JpvjC2vW8X8EDCzq6lk52bEDKuyucIDK5EtnKDDLX10iIU1NvMu8W2BxsCQlccOQL+zwErVSCLqA6kb8Y78QBzl153j6OGWTC0YlUEjjYQMlsv/a/VH1sgZ7KP75uObDB+rpqMGmLf8RSCak9iI3sPIHITPRGu9ev9OYUiKIOc2xGTVAPhPODEWYKLiFCyveMycHkPVcNC9sJAf1QHSEIE6mJKRVpF73Q0+U43uBpZnk3BZK1L3Hqi5lZp35oKOCUgP67+jbsdp7ILwZ1dS+gjvLFdI++W11i9FYTDUbJnIjXwO6PPXRMhEXVxOhWUdiEQgxfW8bEr8Qu3pyqgtfj0hplj90tqkB2poLqZjU8e4PqA7pmnNYvVuAwurqdxFuuNNCjHumP2Ci400d5UD78TA29vbrJsnxtI1uD4k6djAdCsTX39wH0X6nno8HPoZFI5OHeJRl4NsO4p95B+lQ7964L/A5lssfgidisY4GU5CgJiIQHIxHM9B+K5ia2hKkr30brWsFUbSF+7NQttwEIbiGnkvQ4qkkQj8kS+wwFKpHoKyJYZ0Q/eXsVosyx/U3x7OnLKGpf3gN4rTfwuQPoKB0mYkAlWef+k5XwOUQFQGKJYcFExBpprCYnuXf3IfLEQS6a9cZjtcwzL+sC2ZZ9b117ObLHc76YIaMxPjWUcx1y47YSePo/6oFq0cgwmfSnGGCBh2PegIO +WVPZ27c TKVCTrE1zShQA6QKEa/jnRTKc5TPAcptVsDdt/lZMtj+zvdJ20Mb7SI/aE6Q0pJ7FQ16d5zblHHwcmfhg+8tsPh3I3zHGYqc89Bi5gaheFa2269UfLJsfPWaFeHDqKmhv/atbkCSzgBwW75PuJOcBvG8++6QUzOUt4pqXAeALTZSfrWemAmuRZXn8G7c/U4pg8+FQIXc5PNvWZN4N7vioJw7dzzo3tTld5q+1jPgW6C5RY4H9otvGCMeExC+eeztAaV8ejQNUFsfpCAASqkUDlUeXRLAUSVw1i7hQ/fkPBvb3e31a/GVhEcR/yUpbUscO6ZmMokzLLr1GlnqaSK7n12W+3PnB4oQEdwgzOMb1cuTFu6NUVZb+gpiopHhucg+2wqeR49ldjl/wPQseEQJljCxlo0IlFIw2M5baceDCEjF+PimoMx5UTKLt4w== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: To allow precise tracking of page caches accessed, add new tracepoints that trigger when a process actually accesses them. The ureadahead program used by ChromeOS traces the disk access of programs as they start up at boot up. It uses mincore(2) or the 'mm_filemap_add_to_page_cache' trace event to accomplish this. It stores this information in a "pack" file and on subsequent boots, it will read the pack file and call readahead(2) on the information so that disk storage can be loaded into RAM before the applications actually need it. A problem we see is that due to the kernel's readahead algorithm that can aggressively pull in more data than needed (to try and accomplish the same goal) and this data is also recorded. The end result is that the pack file contains a lot of pages on disk that are never actually used. Calling readahead(2) on these unused pages can slow down the system boot up times. To solve this, add 3 new trace events, get_pages, map_pages, and fault. These will be used to trace the pages are not only pulled in from disk, but are actually used by the application. Only those pages will be stored in the pack file, and this helps out the performance of boot up. With the combination of these 3 new trace events and mm_filemap_add_to_page_cache, we observed a reduction in the pack file by 7.3% - 20% on ChromeOS varying by device. Signed-off-by: Takaya Saeki Reviewed-by: Masami Hiramatsu (Google) Reviewed-by: Steven Rostedt (Google) --- Changelog between v3 and v2 - Use a range notation in the printf format Changelog between v2 and v1 - Fix a file offset type usage by casting pgoff_t to loff_t - Fix format string of dev and inode include/trace/events/filemap.h | 84 ++++++++++++++++++++++++++++++++++ mm/filemap.c | 4 ++ 2 files changed, 88 insertions(+) V2:https://lore.kernel.org/all/20240620161903.3176859-1-takayas@chromium.org/ V1:https://lore.kernel.org/all/20240618093656.1944210-1-takayas@chromium.org/ diff --git a/include/trace/events/filemap.h b/include/trace/events/filemap.h index 46c89c1e460c..f48fe637bfd2 100644 --- a/include/trace/events/filemap.h +++ b/include/trace/events/filemap.h @@ -56,6 +56,90 @@ DEFINE_EVENT(mm_filemap_op_page_cache, mm_filemap_add_to_page_cache, TP_ARGS(folio) ); +DECLARE_EVENT_CLASS(mm_filemap_op_page_cache_range, + + TP_PROTO( + struct address_space *mapping, + pgoff_t index, + pgoff_t last_index + ), + + TP_ARGS(mapping, index, last_index), + + TP_STRUCT__entry( + __field(unsigned long, i_ino) + __field(dev_t, s_dev) + __field(unsigned long, index) + __field(unsigned long, last_index) + ), + + TP_fast_assign( + __entry->i_ino = mapping->host->i_ino; + if (mapping->host->i_sb) + __entry->s_dev = + mapping->host->i_sb->s_dev; + else + __entry->s_dev = mapping->host->i_rdev; + __entry->index = index; + __entry->last_index = last_index; + ), + + TP_printk( + "dev=%d:%d ino=%lx ofs=%lld-%lld", + MAJOR(__entry->s_dev), + MINOR(__entry->s_dev), __entry->i_ino, + ((loff_t)__entry->index) << PAGE_SHIFT, + ((((loff_t)__entry->last_index + 1) << PAGE_SHIFT) - 1) + ) +); + +DEFINE_EVENT(mm_filemap_op_page_cache_range, mm_filemap_get_pages, + TP_PROTO( + struct address_space *mapping, + pgoff_t index, + pgoff_t last_index + ), + TP_ARGS(mapping, index, last_index) +); + +DEFINE_EVENT(mm_filemap_op_page_cache_range, mm_filemap_map_pages, + TP_PROTO( + struct address_space *mapping, + pgoff_t index, + pgoff_t last_index + ), + TP_ARGS(mapping, index, last_index) +); + +TRACE_EVENT(mm_filemap_fault, + TP_PROTO(struct address_space *mapping, pgoff_t index), + + TP_ARGS(mapping, index), + + TP_STRUCT__entry( + __field(unsigned long, i_ino) + __field(dev_t, s_dev) + __field(unsigned long, index) + ), + + TP_fast_assign( + __entry->i_ino = mapping->host->i_ino; + if (mapping->host->i_sb) + __entry->s_dev = + mapping->host->i_sb->s_dev; + else + __entry->s_dev = mapping->host->i_rdev; + __entry->index = index; + ), + + TP_printk( + "dev=%d:%d ino=%lx ofs=%lld", + MAJOR(__entry->s_dev), + MINOR(__entry->s_dev), __entry->i_ino, + ((loff_t)__entry->index) << PAGE_SHIFT + ) +); + TRACE_EVENT(filemap_set_wb_err, TP_PROTO(struct address_space *mapping, errseq_t eseq), diff --git a/mm/filemap.c b/mm/filemap.c index d62150418b91..925eef5e16f0 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -2556,6 +2556,7 @@ static int filemap_get_pages(struct kiocb *iocb, size_t count, goto err; } + trace_mm_filemap_get_pages(mapping, index, last_index); return 0; err: if (err < 0) @@ -3287,6 +3288,8 @@ vm_fault_t filemap_fault(struct vm_fault *vmf) if (unlikely(index >= max_idx)) return VM_FAULT_SIGBUS; + trace_mm_filemap_fault(mapping, index); + /* * Do we have something in the page cache already? */ @@ -3653,6 +3656,7 @@ vm_fault_t filemap_map_pages(struct vm_fault *vmf, } while ((folio = next_uptodate_folio(&xas, mapping, end_pgoff)) != NULL); add_mm_counter(vma->vm_mm, folio_type, rss); pte_unmap_unlock(vmf->pte, vmf->ptl); + trace_mm_filemap_map_pages(mapping, start_pgoff, end_pgoff); out: rcu_read_unlock();