From patchwork Thu Jan 10 22:07:13 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Suren Baghdasaryan X-Patchwork-Id: 10756989 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 1C84991E for ; Thu, 10 Jan 2019 22:07:59 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 01C1729EF0 for ; Thu, 10 Jan 2019 22:07:59 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id E5B8C29EF5; Thu, 10 Jan 2019 22:07:58 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE, USER_IN_DEF_DKIM_WL autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 47CAE29EF0 for ; Thu, 10 Jan 2019 22:07:58 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id E1FEC8E0002; Thu, 10 Jan 2019 17:07:56 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id DCF158E0001; Thu, 10 Jan 2019 17:07:56 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id CE45F8E0002; Thu, 10 Jan 2019 17:07:56 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-pf1-f198.google.com (mail-pf1-f198.google.com [209.85.210.198]) by kanga.kvack.org (Postfix) with ESMTP id 8EB8F8E0001 for ; Thu, 10 Jan 2019 17:07:56 -0500 (EST) Received: by mail-pf1-f198.google.com with SMTP id e89so8720567pfb.17 for ; Thu, 10 Jan 2019 14:07:56 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:mime-version:content-transfer-encoding; bh=ig5qBTqAOOl2ALQFgSUOe0w9ro7QA1aOFe2gQIbsj8Y=; b=k8sSNIvrW7OXNgcWsTtIkk1gvVJR595KvsN3TF4u3TLpkMvPnJvDigUSJnafoT7SDN joqowb4NNLy2FtsGvDLFQmO3EebacGyGuxqUK5hEMln0ZRCcls9R2RxXegzVfR8W6ni5 81Qlv3tEy7bbb6HTach18KBzZWa5ELpK2KlUFOo1jJTCxHKSNGPfgblctyQUjU66jtb3 xaynve799//PMFyqIE7cElrC5UP5v1J/JjYg3AZXu7RaD+jUGl6FD3BnY8SWFyiDctyM kACahYpv+GLYSmiDjNm8RPV2ZX22XufLmAbu8/OQUfsn6Sx04Cp/SCurc/fdx+ra5T8u Zs1Q== X-Gm-Message-State: AJcUukcb/ZLhvJW3f0111EXfNjT0Y7EOwGiZctwMwBjl0Xh2ubt5POxq /UGNNBIwOyw8bE7ZS+wX/PKdxXdn9LjeawnloSmTNcB9WU+XvpBXYTb2znQUMm7+QDnTvxCX4M2 sSwXfM0oh7UxQD/frhy+ArQt0jnTOxYptU0VSI19mAIZVH8WtrHEoWgE5OXGOCvb/iGcvkuMOR8 6gnzspTuto4lcwi16g6y17j36FGwLV0XXfCvcgIezpIuqpSP6WAYm5gRYIZJLBMu2uw+ORazE77 B9hfnHJY0halPGomb0ZnDJskax41s+Q52GpvSvuVbw9iPtgULC47g7BMG/bkySYmi65Ur3bs8+Y BLllBuucrDsrzYB3O0E05va5m/ITCNCd7KBBmRdlJzLZl7NWDCYOrhJP1Gn4dZxVeo+C6GbggHc W X-Received: by 2002:a63:5964:: with SMTP id j36mr10977035pgm.210.1547158076050; Thu, 10 Jan 2019 14:07:56 -0800 (PST) X-Received: by 2002:a63:5964:: with SMTP id j36mr10976991pgm.210.1547158075171; Thu, 10 Jan 2019 14:07:55 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1547158075; cv=none; d=google.com; s=arc-20160816; b=RBMuf+0NRWOOPheRILllX2RlRSaVn7163x45h3QkkTq9DBObwKg+LjUlHOITpO9Tmj z3Uo/o8ouMYZda3QWQYSCOGG8WSLzzh7bFWvMvCWP4jOjW55WaU/zk4cKdDySRAK4lR0 r0kYq9viiNhx6nqcw4qQFJqlP5j+XybL5eMTh8jH8vlWdfRjVSZuv+EovZD/TkSVDqJp a7Lz6GZ6VSLzfTapUJ13cXjpMD91CEDEkQ1Qu6YX3T6XxJtfqFxjVPtffVrYy7nHWg20 2s0B+kKhlaSfxXLCFUZKC1MDhxfbt8doFFIc9yVHZ91XDfV544Eqq8Z5yR2Y1+GnbXjY BXVQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:dkim-signature; bh=ig5qBTqAOOl2ALQFgSUOe0w9ro7QA1aOFe2gQIbsj8Y=; b=tCrcLR2tSxZzEK+fdGM9JzfYK++TTVKoCGOKxynyCtloUwqsZ/JuCs2pzlW/nTqDdc MRWEhACWybey0RnAWi04ud9obG+QOSk3vz//i7DJrBNlbou4/3Sh8Lm05eZhe1yIKUtU yRtwPfjpUu0x3k3SK+NQ36AI3CmCvo3G0LnB0hbHId+is8UfRuDheyCRTnTUiHQ00HPm Ea3PmDmaKjeVGMDg0457RWwjNSwHTmvqpol+s33t3hrPRsNrskuPbOwL+M0hdIsB1kpy jQu1OR7TYLAsWLJh1IAHCnzYfVjM37vZWHD/q6L1Zu07R0HkcxOX/c0S1V07t3rl5Vxn INPw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=crAkPTvP; spf=pass (google.com: domain of surenb@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=surenb@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id i10sor502617plt.38.2019.01.10.14.07.55 for (Google Transport Security); Thu, 10 Jan 2019 14:07:55 -0800 (PST) Received-SPF: pass (google.com: domain of surenb@google.com designates 209.85.220.65 as permitted sender) client-ip=209.85.220.65; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=crAkPTvP; spf=pass (google.com: domain of surenb@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=surenb@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=ig5qBTqAOOl2ALQFgSUOe0w9ro7QA1aOFe2gQIbsj8Y=; b=crAkPTvPLfl2A5lFXysX3vaxjF/2y1f3sxNNf1vPvrnJgrf1xy0zKoKE0zzMIP8sS4 0S4/4dPxl8/cpbT2+JIamABRXt4zrS5tHeKi87QxVixpM8yUAq1DkjhfZCKFm3gr/FEf r83CtOyVbwKYyJrYqv5cQI+2NexV0ZWdKBlYrAizrbZ1vMfqm8xh1WQK+W4l2XUDA73F EaY+oWW1RgTZl8J/azSIUL/Kstz71dOR7Hd7iLpZyF7k2F4Boy6FL/vx9o8H56KDhfPC dVpshDaNnUqN8JlZA++d4n+h8jU7ydNXmt4QX2GlX88chDJ+D4pRC1IsvTCI2aE7T49c EAMw== X-Google-Smtp-Source: ALg8bN73ksNs0MN12T2hkHxqycU/FXXGxYcKJ+wlnrOwmW+OoBldscti3YwBq5ZkSW4B9iPNx51+KA== X-Received: by 2002:a17:902:6b87:: with SMTP id p7mr12264757plk.282.1547158074549; Thu, 10 Jan 2019 14:07:54 -0800 (PST) Received: from surenb0.mtv.corp.google.com ([2620:0:1000:1612:3320:4357:47df:276b]) by smtp.googlemail.com with ESMTPSA id p7sm152958019pfj.72.2019.01.10.14.07.53 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 10 Jan 2019 14:07:53 -0800 (PST) From: Suren Baghdasaryan To: gregkh@linuxfoundation.org Cc: tj@kernel.org, lizefan@huawei.com, hannes@cmpxchg.org, axboe@kernel.dk, dennis@kernel.org, dennisszhou@gmail.com, mingo@redhat.com, peterz@infradead.org, akpm@linux-foundation.org, corbet@lwn.net, cgroups@vger.kernel.org, linux-mm@kvack.org, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, kernel-team@android.com, Suren Baghdasaryan Subject: [PATCH v2 0/5] psi: pressure stall monitors v2 Date: Thu, 10 Jan 2019 14:07:13 -0800 Message-Id: <20190110220718.261134-1-surenb@google.com> X-Mailer: git-send-email 2.20.1.97.g81188d93c3-goog MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP This is respin of: https://lwn.net/ml/linux-kernel/20181214171508.7791-1-surenb@google.com/ Android is adopting psi to detect and remedy memory pressure that results in stuttering and decreased responsiveness on mobile devices. Psi gives us the stall information, but because we're dealing with latencies in the millisecond range, periodically reading the pressure files to detect stalls in a timely fashion is not feasible. Psi also doesn't aggregate its averages at a high-enough frequency right now. This patch series extends the psi interface such that users can configure sensitive latency thresholds and use poll() and friends to be notified when these are breached. As high-frequency aggregation is costly, it implements an aggregation method that is optimized for fast, short-interval averaging, and makes the aggregation frequency adaptive, such that high-frequency updates only happen while monitored stall events are actively occurring. With these patches applied, Android can monitor for, and ward off, mounting memory shortages before they cause problems for the user. For example, using memory stall monitors in userspace low memory killer daemon (lmkd) we can detect mounting pressure and kill less important processes before device becomes visibly sluggish. In our memory stress testing psi memory monitors produce roughly 10x less false positives compared to vmpressure signals. Having ability to specify multiple triggers for the same psi metric allows other parts of Android framework to monitor memory state of the device and act accordingly. The new interface is straight-forward. The user opens one of the pressure files for writing and writes a trigger description into the file descriptor that defines the stall state - some or full, and the maximum stall time over a given window of time. E.g.: /* Signal when stall time exceeds 100ms of a 1s window */ char trigger[] = "full 100000 1000000" fd = open("/proc/pressure/memory") write(fd, trigger, sizeof(trigger)) while (poll() >= 0) { ... }; close(fd); When the monitored stall state is entered, psi adapts its aggregation frequency according to what the configured time window requires in order to emit event signals in a timely fashion. Once the stalling subsides, aggregation reverts back to normal. The trigger is associated with the open file descriptor. To stop monitoring, the user only needs to close the file descriptor and the trigger is discarded. Patches 1-4 prepare the psi code for polling support. Patch 5 implements the adaptive polling logic, the pressure growth detection optimized for short intervals, and hooks up write() and poll() on the pressure files. The patches were developed in collaboration with Johannes Weiner. The patches are based on 4.20-rc7. Johannes Weiner (2): fs: kernfs: add poll file operation kernel: cgroup: add poll file operation Suren Baghdasaryan (3): psi: introduce state_mask to represent stalled psi states psi: rename psi fields in preparation for psi trigger addition psi: introduce psi monitor Documentation/accounting/psi.txt | 104 ++++++ fs/kernfs/file.c | 31 +- include/linux/cgroup-defs.h | 4 + include/linux/kernfs.h | 6 + include/linux/psi.h | 10 + include/linux/psi_types.h | 83 ++++- kernel/cgroup/cgroup.c | 119 +++++- kernel/sched/psi.c | 597 ++++++++++++++++++++++++++++--- 8 files changed, 885 insertions(+), 69 deletions(-) Changes in v2: - Preserved psi idle mode, as per Peter - Changed input parser to use sscanf, as per Peter - Removed percentage threshold support, as per Johannes and Joel - Added explicit numbers corresponding to NR_PSI_TASK_COUNTS, NR_PSI_RESOURCES and NR_PSI_STATES, as per Peter - Added a note about psi_group_cpu struct new size in the changelog of the patch introducing new member (0003-...), as per Peter - Fixed 64-bit division using div_u64, as per kbuild test robot