From patchwork Fri May 4 06:57:49 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mahendran Ganesh X-Patchwork-Id: 10380053 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 9DB0B6038F for ; Fri, 4 May 2018 06:58:38 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 8D95029356 for ; Fri, 4 May 2018 06:58:38 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 807D42935A; Fri, 4 May 2018 06:58:38 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, FREEMAIL_FROM, MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id EFC7B29356 for ; Fri, 4 May 2018 06:58:37 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 1C7696B0010; Fri, 4 May 2018 02:58:37 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 14ED16B0011; Fri, 4 May 2018 02:58:37 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id F32776B0012; Fri, 4 May 2018 02:58:36 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-pf0-f198.google.com (mail-pf0-f198.google.com [209.85.192.198]) by kanga.kvack.org (Postfix) with ESMTP id B37D86B0010 for ; Fri, 4 May 2018 02:58:36 -0400 (EDT) Received: by mail-pf0-f198.google.com with SMTP id w7so13748107pfd.9 for ; Thu, 03 May 2018 23:58:36 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references; bh=jEdbOTaI9m7PNHd1fJWH+JX73cpNYLhM0ko0o7VzxCA=; b=Nf4QXHw1PeTgPZDivp0Irlcaw0iNyf8cRc43lnqBsWbKA5vvYlLW3nKqS4wk6WspAe fnQXInxHrHdQUtQkvVXzUBQ0e3b2vJ1srsbb/g1OxEp9L8M/JgRm8iRSu9ORhYl2F9vR M/KRL4YP9z112KhmhNc8Y4FEjJcK1XcnlqSgXeVWbsSVHON4PZDMgQJGs9uEhqGMVPqP pOINFpE/DCqzyWK0/+v35Nc3jnjXWK4jFgt8elnV8kjs5fcosL6RuDgQrVMl3i/6q622 ikMaHlN5gR8TRG3yBrk6pyu9OB6Z4Vw7IK1EplDZ5mT5ORbSTrIbGPVQOhX+I0qwEcPW eyuQ== X-Gm-Message-State: ALQs6tAroXLn/ACSEpO/pWBWmm/VCH9haNDR05nPTr9cS31odpMvpEB+ 29NDHQazvpbB5dvaGSzPGxeNGlvpsPrDaJb3O2/sorxA1Fl5nA6pwukTHAUhbkKN/XHa5HYv72m sivs/THYimfXxHOjxls+032UpOL4PCDPnk37lzexqIajr4TNBFA8wGLqnHbf05HL51/Q5JccIQz EgDwUCXcoEHOSMsrzRmee6LU12MY8Qi8AnlZGnEuGrtYQ5Amrxsy1fFAIXDUjKgV8SqdlaKLWfB iRpVo2G/uqnLAjP5A/v9xxclwRnMhJmwZmxL5GcRMd3EM/pzNHsc+spmBDVLu9uGY7mo7eHpO4z laDIONZoQo9nZ2hBsqVGtxQMjyI2KLkSWICWVEyGZk9rs4yw97pFH9Im7KgeSbSc47UtEtYRW3e N X-Received: by 2002:a65:4944:: with SMTP id q4-v6mr21911794pgs.424.1525417116401; Thu, 03 May 2018 23:58:36 -0700 (PDT) X-Received: by 2002:a65:4944:: with SMTP id q4-v6mr21911774pgs.424.1525417115672; Thu, 03 May 2018 23:58:35 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1525417115; cv=none; d=google.com; s=arc-20160816; b=zAQhGTTM1XIK/OAwwBOw7Z5Agz2EyMuhlhlD2uwIs2gfuNHc5XfzoT7OYZrnHKJqso IsHxC+oj38fCRc4Ao9hb3r1zEC6FYhj/GWqkT8LoXwjteOKciOVU7SXsfReiHOf1q2mA rBJb1sg5Y1kLTRkfsDIqzIZtQsiXVOTrdOcKkzNP23zOOr4Kj3AQ/FRYILooCnjB1aOa 6m4pGWwwm/zbsM1EzfoaEvyNOzSxZbpItMDNO3UGLn/uE0tn0p4umsW+JNEWs760LndE XgWrd44bHhAKn6jw451xf+iRO7Nw50BMusuLOeQ30IYzDlWGVUG4f/5kRtUw8xj8t3Cv priw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature:arc-authentication-results; bh=jEdbOTaI9m7PNHd1fJWH+JX73cpNYLhM0ko0o7VzxCA=; b=MkBVS92imVa7eBGPghwaVFyBRmRBdABlG6jIc4NjL4KYEY37cmQERSn3OHyY04kEgI jJcwMwjVSYZLkF9NxSFcLKiTFOjxAT0Q1te05PP8t5NBx3vOPBKBdI1WlXnPNAJx6v65 5qh2VsWp87B8avyxu6AmbTM940sOs70GcryzDks8F1PyjGPD0KWMWlNTJTui1CHZKnGp 34dTuyDs4eOosvSUga/FFRES1UyiLf/YS2ESfYFYI0vkBCIuSQxp711+gmg0wZ0ND3WV 1xitrdqG3TF2OP1YQ7vrwPAkoVIOa9qtgxw2UkwjDrs1SJ8d6AoCXdA02mZJkSYxv3fN cl6g== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gmail.com header.s=20161025 header.b=ZWnQ+dHG; spf=pass (google.com: domain of opensource.ganesh@gmail.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=opensource.ganesh@gmail.com; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id b15-v6sor111530plk.105.2018.05.03.23.58.35 for (Google Transport Security); Thu, 03 May 2018 23:58:35 -0700 (PDT) Received-SPF: pass (google.com: domain of opensource.ganesh@gmail.com designates 209.85.220.65 as permitted sender) client-ip=209.85.220.65; Authentication-Results: mx.google.com; dkim=pass header.i=@gmail.com header.s=20161025 header.b=ZWnQ+dHG; spf=pass (google.com: domain of opensource.ganesh@gmail.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=opensource.ganesh@gmail.com; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=jEdbOTaI9m7PNHd1fJWH+JX73cpNYLhM0ko0o7VzxCA=; b=ZWnQ+dHGsJv5Lzil3i8Y324N7h03qLiLKSobkozWNJ4BTemwYeQcA9fkPMsij2F0/e KaR3CaBUZsPjYq5asdogXvKfSOSKYfSRTD/eJEvXG4ZrRC10Ohw6kIiRYDARAcDGn3O9 hYVdc5imnoiXb7rE0SctNyzjRAEIdi6U4NGCU+Mph5Jhcy0F0bFiUs6ZiECjx3luPYcp 5s9oII+oa1zBVbEikio7Uyj2MXkMs53RtKoDdpx9bQ8RONd4JHeNNw01AdO8nhwRSqLP iBmgwqIGHRTGb5xPLNCFWWgNPdGuWe1/Pg64qQFP4XCNoABFDE6TW5sQ8DZrzpCR0wIg rblA== X-Google-Smtp-Source: AB8JxZrok5zeWukneTSus8l4SJMtD6+A8hRvPfiopcBrg2gpPzQ+8VA+tgFWk/tzGYLViSi3Ke/7KA== X-Received: by 2002:a17:902:b184:: with SMTP id s4-v6mr19048295plr.359.1525417115403; Thu, 03 May 2018 23:58:35 -0700 (PDT) Received: from localhost (outboundhk.mxmail.xiaomi.com. [207.226.244.124]) by smtp.gmail.com with ESMTPSA id p24sm20520473pfk.58.2018.05.03.23.58.34 (version=TLS1_2 cipher=AES128-SHA bits=128/128); Thu, 03 May 2018 23:58:34 -0700 (PDT) From: Ganesh Mahendran To: ldufour@linux.vnet.ibm.com, catalin.marinas@arm.com, will.deacon@arm.com, mark.rutland@arm.com, cpandya@codeaurora.org, punit.agrawal@arm.com Cc: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, Ganesh Mahendran Subject: [PATCH v2 2/2] arm64/mm: add speculative page fault Date: Fri, 4 May 2018 14:57:49 +0800 Message-Id: <1525417069-27401-2-git-send-email-opensource.ganesh@gmail.com> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1525417069-27401-1-git-send-email-opensource.ganesh@gmail.com> References: <1525417069-27401-1-git-send-email-opensource.ganesh@gmail.com> X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP This patch enables the speculative page fault on the arm64 architecture. I completed spf porting in 4.9. From the test result, we can see app launching time improved by about 10% in average. For the apps which have more than 50 threads, 15% or even more improvement can be got. Signed-off-by: Ganesh Mahendran --- v2: move find_vma() to do_page_fault() remove IS_ENABLED() remove fault != VM_FAULT_SIGSEGV check initilize vma = NULL --- arch/arm64/mm/fault.c | 29 +++++++++++++++++++++++++---- 1 file changed, 25 insertions(+), 4 deletions(-) diff --git a/arch/arm64/mm/fault.c b/arch/arm64/mm/fault.c index 4165485..efd5956 100644 --- a/arch/arm64/mm/fault.c +++ b/arch/arm64/mm/fault.c @@ -320,14 +320,12 @@ static void do_bad_area(unsigned long addr, unsigned int esr, struct pt_regs *re #define VM_FAULT_BADMAP 0x010000 #define VM_FAULT_BADACCESS 0x020000 -static int __do_page_fault(struct mm_struct *mm, unsigned long addr, +static int __do_page_fault(struct vm_area_struct *vma, unsigned long addr, unsigned int mm_flags, unsigned long vm_flags, struct task_struct *tsk) { - struct vm_area_struct *vma; int fault; - vma = find_vma(mm, addr); fault = VM_FAULT_BADMAP; if (unlikely(!vma)) goto out; @@ -371,6 +369,7 @@ static int __kprobes do_page_fault(unsigned long addr, unsigned int esr, int fault, major = 0; unsigned long vm_flags = VM_READ | VM_WRITE; unsigned int mm_flags = FAULT_FLAG_ALLOW_RETRY | FAULT_FLAG_KILLABLE; + struct vm_area_struct *vma = NULL; if (notify_page_fault(regs, esr)) return 0; @@ -410,6 +409,16 @@ static int __kprobes do_page_fault(unsigned long addr, unsigned int esr, perf_sw_event(PERF_COUNT_SW_PAGE_FAULTS, 1, regs, addr); /* + * let's try a speculative page fault without grabbing the + * mmap_sem. + */ + fault = handle_speculative_fault(mm, addr, mm_flags, &vma); + if (fault != VM_FAULT_RETRY) { + perf_sw_event(PERF_COUNT_SW_SPF, 1, regs, addr); + goto done; + } + + /* * As per x86, we may deadlock here. However, since the kernel only * validly references user space from well defined areas of the code, * we can bug out early if this is from code which shouldn't. @@ -431,7 +440,10 @@ static int __kprobes do_page_fault(unsigned long addr, unsigned int esr, #endif } - fault = __do_page_fault(mm, addr, mm_flags, vm_flags, tsk); + if (!vma || !can_reuse_spf_vma(vma, addr)) + vma = find_vma(mm, addr); + + fault = __do_page_fault(vma, addr, mm_flags, vm_flags, tsk); major |= fault & VM_FAULT_MAJOR; if (fault & VM_FAULT_RETRY) { @@ -454,11 +466,20 @@ static int __kprobes do_page_fault(unsigned long addr, unsigned int esr, if (mm_flags & FAULT_FLAG_ALLOW_RETRY) { mm_flags &= ~FAULT_FLAG_ALLOW_RETRY; mm_flags |= FAULT_FLAG_TRIED; + + /* + * Do not try to reuse this vma and fetch it + * again since we will release the mmap_sem. + */ + vma = NULL; + goto retry; } } up_read(&mm->mmap_sem); +done: + /* * Handle the "normal" (no error) case first. */