From patchwork Tue Apr 21 14:07:42 2015 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: William Cohen X-Patchwork-Id: 6249021 Return-Path: X-Original-To: patchwork-linux-arm@patchwork.kernel.org Delivered-To: patchwork-parsemail@patchwork1.web.kernel.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.136]) by patchwork1.web.kernel.org (Postfix) with ESMTP id C09039F313 for ; Tue, 21 Apr 2015 14:10:41 +0000 (UTC) Received: from mail.kernel.org (localhost [127.0.0.1]) by mail.kernel.org (Postfix) with ESMTP id B9C5D20303 for ; Tue, 21 Apr 2015 14:10:40 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.9]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id CE424202F0 for ; Tue, 21 Apr 2015 14:10:39 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.80.1 #2 (Red Hat Linux)) id 1YkYqt-0001yP-KR; Tue, 21 Apr 2015 14:08:39 +0000 Received: from mx1.redhat.com ([209.132.183.28]) by bombadil.infradead.org with esmtps (Exim 4.80.1 #2 (Red Hat Linux)) id 1YkYql-0001pz-VV for linux-arm-kernel@lists.infradead.org; Tue, 21 Apr 2015 14:08:32 +0000 Received: from int-mx09.intmail.prod.int.phx2.redhat.com (int-mx09.intmail.prod.int.phx2.redhat.com [10.5.11.22]) by mx1.redhat.com (8.14.4/8.14.4) with ESMTP id t3LE7iDL020722 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=FAIL); Tue, 21 Apr 2015 10:07:44 -0400 Received: from [10.13.129.102] (dhcp129-102.rdu.redhat.com [10.13.129.102]) by int-mx09.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id t3LE7gqw002861; Tue, 21 Apr 2015 10:07:43 -0400 Message-ID: <553659AE.4090809@redhat.com> Date: Tue, 21 Apr 2015 10:07:42 -0400 From: William Cohen User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:31.0) Gecko/20100101 Thunderbird/31.6.0 MIME-Version: 1.0 To: Masami Hiramatsu , David Long Subject: Re: [PATCH v6 0/6] arm64: Add kernel probes (kprobes) support References: <1429561187-3661-1-git-send-email-dave.long@linaro.org> <55363791.4070706@hitachi.com> In-Reply-To: <55363791.4070706@hitachi.com> X-Scanned-By: MIMEDefang 2.68 on 10.5.11.22 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20150421_070832_077767_E8BDCCB6 X-CRM114-Status: GOOD ( 20.32 ) X-Spam-Score: -5.0 (-----) Cc: "Jon Medhurst \(Tixy\)" , Steve Capper , Ananth N Mavinakayanahalli , Catalin Marinas , Will Deacon , linux-kernel@vger.kernel.org, Anil S Keshavamurthy , sandeepa.s.prabhu@gmail.com, Russell King , davem@davemloft.net, linux-arm-kernel@lists.infradead.org X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org X-Spam-Status: No, score=-4.2 required=5.0 tests=BAYES_00, RCVD_IN_DNSWL_MED, T_RP_MATCHES_RCVD, UNPARSEABLE_RELAY autolearn=unavailable version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on mail.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP On 04/21/2015 07:42 AM, Masami Hiramatsu wrote: > (2015/04/21 5:19), David Long wrote: >> From: "David A. Long" >> >> This patchset is heavily based on Sandeepa Prabhu's ARM v8 kprobes patches, >> first seen in October 2013. This version attempts to address concerns raised by >> reviewers and also fixes problems discovered during testing. >> >> This patchset adds support for kernel probes(kprobes), jump probes(jprobes) >> and return probes(kretprobes) support for ARM64. >> >> The kprobes mechanism makes use of software breakpoint and single stepping >> support available in the ARM v8 kernel. >> > [...] >> Changes since v5 include: >> >> 1) Replaced installation of breakpoint hook with direct call from the >> handlers in debug-monitors.c, as requested. >> 2) Reject probing of instructions that read the interrupt mask, in >> addition to instructions that set it. >> 3) Cleaned up comments describing usage of Debug Mask. >> 4) Added KPROBE_REENTER case in reenter_kprobe. >> 5) Corrected the ifdef'd definitions for notify_page_fault() to be >> consistent when KPROBES is not configed. >> 6) Changed "cpsr" to "pstate" for HAVE_REGS_AND_STACK_ACCESS_API feature. >> 7) Added back in missing new files in previous patch. >> 8) Changed two instances of pr_warning() to pr_warn(). > > Looks OK to me:) > BTW, have you tried to build and test this with CONFIG_KPROBE_EVENT? > If so, you can also test it by tools/testing/selftests/ftrace/ftracetest. > >> Note that there seems to be at least a potential issue with kprobes >> on multiple (possibly all) platforms having to do with use of kfree >> inside of the kretprobes trampoline handler. This has manifested >> occasionally in systemtap testing on arm64. There does not appear to >> be an simple solution to the problem. > > No, trampoline handler must call recycle_rp_inst() instead of kfree > to return kretprobe instance to the pool. Hmm, I should look into it. > > Thank you, > Hi, I have noticed when running the systemtap testsuite even with this newest revision of the arm64 kprobe patches the system will start spewing the following message: Unexpected kernel single-step exception at EL1 That is triggered by the functioncallcount.stp test in the systemtap examples. The test is instrumenting many function calls and returns in the memory management code. It appears that the problem is triggered during the kfree call/return towards the end of trampoline_probe_handler. The single_step_handler function in debug-monitors.c calls kprobe_single_step_handler which indicates that the breakpoint is not a kprobe breakpoint. Thus, single_step_handler starts flagging the breakpoint with the above message. It would be good if the warning message included and address to be a bit more informative. I put in some addition code (the attached patch) to print out register information to diagnose what was going on. Other architecture do use kfree in the kretprobe trampoline handlers but do not seem to encounter this problem. -Will diff --git a/arch/arm64/kernel/debug-monitors.c b/arch/arm64/kernel/debug-monitors.c index dae7bb4..ec5a1b2 100644 --- a/arch/arm64/kernel/debug-monitors.c +++ b/arch/arm64/kernel/debug-monitors.c @@ -262,6 +262,19 @@ static int single_step_handler(unsigned long addr, unsigned int esr, if (!handler_found) { pr_warning("Unexpected kernel single-step exception at EL1\n"); + { + struct kprobe_ctlblk *kcb = get_kprobe_ctlblk(); + pr_warning("kcb->ss_ctx.ss_status = %ld\n", + kcb->ss_ctx.ss_status); + printk("kcb->ss_ctx.match_addr = %lx ", + kcb->ss_ctx.match_addr); + print_symbol("%s\n", kcb->ss_ctx.match_addr); + printk("instruction_pointer(regs) = %lx ", + instruction_pointer(regs)); + print_symbol("%s\n", instruction_pointer(regs)); + show_regs(regs); + BUG(); + } /* * Re-enable stepping since we know that we will be * returning to regs.