From patchwork Sun Sep 29 14:42:30 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oleg Nesterov X-Patchwork-Id: 13815136 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A462717ADF8 for ; Sun, 29 Sep 2024 14:42:51 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1727620973; cv=none; b=HGHC8yTSTuDWAnGPcmt9OQussKmqS19EU1hKkH8kGEefRmauS5X5Qk0oSm6ttgq/WXhq13WnnQO2brq3yEykjuAHrioKLwm8SSq/IIyfmxxORppGQ3K42/V57I3t0vSZ68s528ZuhAd2knYvxxmlUjZ3ouNc1tochMmfIYoFPag= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1727620973; c=relaxed/simple; bh=6WTZso5WGhvtb2/OxjllcDfJ4f6hMAM6NKTHQqMFnIQ=; h=Date:From:To:Cc:Subject:Message-ID:MIME-Version:Content-Type: Content-Disposition:In-Reply-To; b=Ky97EZV5PMWtiyJjczjJmRBpZWTBjfvyQ1y1q6t+H8QoGvNpelwss41adq+mHIR4l/5VE8cEWz8vHDmb7hyOs7o9RHX26Rts9A6GkKz+Slqr8HAcPusz6YH9I3FYqae73XC3New2exo+xUqeAAVbWxy8+/JIDEsotlt2Wbp5VfI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=IH8kdwcO; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="IH8kdwcO" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1727620970; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to; bh=UjMkCez+I6XaIxZPG+o4NZ0Bsl2LfAkraC0d9Mrst+o=; b=IH8kdwcOpkZWw84ebQ19dbXehgBPuh/BUGMgFSC2Ijuj+eZIy877bMg+Cr6QEuhj8OulWh VM0fvV7v+oePjXsqzhpgRi2hVukSIqZbQXK29dt7xk+KjHdKU+gJ0amWsfiuzsp3ECbGWC Scm1qHzw4m1+HDCskVikMPKyfzSvdAo= Received: from mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-158-doMx__8qNg-wZ2uUUCItBQ-1; Sun, 29 Sep 2024 10:42:48 -0400 X-MC-Unique: doMx__8qNg-wZ2uUUCItBQ-1 Received: from mx-prod-int-04.mail-002.prod.us-west-2.aws.redhat.com (unknown [10.30.177.40]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 82CE018E6A6C; Sun, 29 Sep 2024 14:42:47 +0000 (UTC) Received: from dhcp-27-174.brq.redhat.com (unknown [10.45.224.44]) by mx-prod-int-04.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with SMTP id B8DC01944B22; Sun, 29 Sep 2024 14:42:44 +0000 (UTC) Received: by dhcp-27-174.brq.redhat.com (nbSMTP-1.00) for uid 1000 oleg@redhat.com; Sun, 29 Sep 2024 16:42:34 +0200 (CEST) Date: Sun, 29 Sep 2024 16:42:30 +0200 From: Oleg Nesterov To: Andrii Nakryiko , Jiri Olsa , Masami Hiramatsu , Peter Zijlstra Cc: Liao Chang , linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org Subject: [PATCH 1/7] uprobes: don't abuse get_utask() in pre_ssout() and prepare_uretprobe() Message-ID: <20240929144230.GA9468@redhat.com> Precedence: bulk X-Mailing-List: linux-trace-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <20240929144201.GA9429@redhat.com> User-Agent: Mutt/1.5.24 (2015-08-30) X-Scanned-By: MIMEDefang 3.0 on 10.30.177.40 handle_swbp() calls get_utask() before prepare_uretprobe() or pre_ssout() can be called, they can simply use current->utask which can't be NULL. Signed-off-by: Oleg Nesterov --- kernel/events/uprobes.c | 14 +++----------- 1 file changed, 3 insertions(+), 11 deletions(-) diff --git a/kernel/events/uprobes.c b/kernel/events/uprobes.c index 4b52cb2ae6d6..2a9cdd5c82d7 100644 --- a/kernel/events/uprobes.c +++ b/kernel/events/uprobes.c @@ -1908,18 +1908,14 @@ static void cleanup_return_instances(struct uprobe_task *utask, bool chained, static void prepare_uretprobe(struct uprobe *uprobe, struct pt_regs *regs) { - struct return_instance *ri; - struct uprobe_task *utask; + struct uprobe_task *utask = current->utask; unsigned long orig_ret_vaddr, trampoline_vaddr; + struct return_instance *ri; bool chained; if (!get_xol_area()) return; - utask = get_utask(); - if (!utask) - return; - if (utask->depth >= MAX_URETPROBE_DEPTH) { printk_ratelimited(KERN_INFO "uprobe: omit uretprobe due to" " nestedness limit pid/tgid=%d/%d\n", @@ -1980,14 +1976,10 @@ static void prepare_uretprobe(struct uprobe *uprobe, struct pt_regs *regs) static int pre_ssout(struct uprobe *uprobe, struct pt_regs *regs, unsigned long bp_vaddr) { - struct uprobe_task *utask; + struct uprobe_task *utask = current->utask; unsigned long xol_vaddr; int err; - utask = get_utask(); - if (!utask) - return -ENOMEM; - if (!try_get_uprobe(uprobe)) return -EINVAL; From patchwork Sun Sep 29 14:42:35 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oleg Nesterov X-Patchwork-Id: 13815137 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id AA4FF152790 for ; Sun, 29 Sep 2024 14:42:58 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1727620980; cv=none; b=cd/6Y/jknYPcQgu3KwGlzj/RsR5k84+i29RXTe1/IjYyQLuGJH50bZgq8GtyPL2fXNJZ0/jNfAkY+XCQKKh9YwPZb48/AbkoYf08Pg/rtdMRf5A1ZX4Z6+CqYsTknFsTPTqVPwakR5FBE24HTiIRN97tpjowvIKYZjGDhD7wIhU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1727620980; c=relaxed/simple; bh=SW0a/2y9x5XuxLHYhEeUJoJubePyKUh+UH4ExDXUbI8=; h=Date:From:To:Cc:Subject:Message-ID:MIME-Version:Content-Type: Content-Disposition:In-Reply-To; b=sNHiRhNPVKcAeoFwuTMP30VofADessurhTzXeS01isgJegfB8BJzRXbO35S17dBqYnybYvCAC53icqNKOEUrdb7oF2EVa3ZBOSZuHy3G4hhsUk16qesYIejCXdZy0oo89MxJpBq9FPHQ/+MywPCm/0CrIxkcxvOIGM1I0/IfoPI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=av+c0kuL; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="av+c0kuL" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1727620977; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to; bh=+dC6z3bA6xQ8q0W/xhGSjQmv+2uYs7hECJrc8EdB/mM=; b=av+c0kuLkNz+ZWl6WpCCip9j58lMRsD6wdOrfyk2rBnThvKe9xd+//et6HmrLgI+waiSiP lFss/K4SdtHSa0mlXTM/TD4o0qwM9CcRtd4xnknseHZuQAMJBYyTeJEFKB1i9Ia03dkNrX Ur3hBmSHL0lcTs/U4YsibqzVg8oc+nk= Received: from mx-prod-mc-02.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-219-mQEtUGoCOaCeMjArfJysMQ-1; Sun, 29 Sep 2024 10:42:54 -0400 X-MC-Unique: mQEtUGoCOaCeMjArfJysMQ-1 Received: from mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com (unknown [10.30.177.15]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-02.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 206071919156; Sun, 29 Sep 2024 14:42:52 +0000 (UTC) Received: from dhcp-27-174.brq.redhat.com (unknown [10.45.224.44]) by mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with SMTP id 6E3E41979060; Sun, 29 Sep 2024 14:42:49 +0000 (UTC) Received: by dhcp-27-174.brq.redhat.com (nbSMTP-1.00) for uid 1000 oleg@redhat.com; Sun, 29 Sep 2024 16:42:38 +0200 (CEST) Date: Sun, 29 Sep 2024 16:42:35 +0200 From: Oleg Nesterov To: Andrii Nakryiko , Jiri Olsa , Masami Hiramatsu , Peter Zijlstra Cc: Liao Chang , linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org Subject: [PATCH 2/7] uprobes: sanitiize xol_free_insn_slot() Message-ID: <20240929144235.GA9471@redhat.com> Precedence: bulk X-Mailing-List: linux-trace-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <20240929144201.GA9429@redhat.com> User-Agent: Mutt/1.5.24 (2015-08-30) X-Scanned-By: MIMEDefang 3.0 on 10.30.177.15 1. Clear utask->xol_vaddr unconditionally, even if this addr is not valid, xol_free_insn_slot() should never return with utask->xol_vaddr != NULL. 2. Add a comment to explain why do we need to validate slot_addr. 3. Simplify the validation above. We can simply check offset < PAGE_SIZE, unsigned underflows are fine, it should work if slot_addr < area->vaddr. 4. Kill the unnecessary "slot_nr >= UINSNS_PER_PAGE" check, slot_nr must be valid if offset < PAGE_SIZE. The next patches will cleanup this function even more. Signed-off-by: Oleg Nesterov --- kernel/events/uprobes.c | 21 +++++++++------------ 1 file changed, 9 insertions(+), 12 deletions(-) diff --git a/kernel/events/uprobes.c b/kernel/events/uprobes.c index 2a9cdd5c82d7..3023714b83f2 100644 --- a/kernel/events/uprobes.c +++ b/kernel/events/uprobes.c @@ -1683,8 +1683,8 @@ static unsigned long xol_get_insn_slot(struct uprobe *uprobe) static void xol_free_insn_slot(struct task_struct *tsk) { struct xol_area *area; - unsigned long vma_end; unsigned long slot_addr; + unsigned long offset; if (!tsk->mm || !tsk->mm->uprobes_state.xol_area || !tsk->utask) return; @@ -1693,24 +1693,21 @@ static void xol_free_insn_slot(struct task_struct *tsk) if (unlikely(!slot_addr)) return; + tsk->utask->xol_vaddr = 0; area = tsk->mm->uprobes_state.xol_area; - vma_end = area->vaddr + PAGE_SIZE; - if (area->vaddr <= slot_addr && slot_addr < vma_end) { - unsigned long offset; - int slot_nr; - - offset = slot_addr - area->vaddr; - slot_nr = offset / UPROBE_XOL_SLOT_BYTES; - if (slot_nr >= UINSNS_PER_PAGE) - return; + offset = slot_addr - area->vaddr; + /* + * slot_addr must fit into [area->vaddr, area->vaddr + PAGE_SIZE). + * This check can only fail if the "[uprobes]" vma was mremap'ed. + */ + if (offset < PAGE_SIZE) { + int slot_nr = offset / UPROBE_XOL_SLOT_BYTES; clear_bit(slot_nr, area->bitmap); atomic_dec(&area->slot_count); smp_mb__after_atomic(); /* pairs with prepare_to_wait() */ if (waitqueue_active(&area->wq)) wake_up(&area->wq); - - tsk->utask->xol_vaddr = 0; } } From patchwork Sun Sep 29 14:42:39 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oleg Nesterov X-Patchwork-Id: 13815138 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 55F6917D346 for ; Sun, 29 Sep 2024 14:43:02 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1727620983; cv=none; b=MeD/+xpQR18IKbJH1WO2u3lbkesx17NIzl9tOfiBbrHEs6sm6b0Cx8q4nrdvZ2cI8gTuWV5OXQHQ47ovaGALEJ+Kzmeiozd4vFUKT15Fk6563hcRbTh7MTi6MNiPxB91M5tRFWPfu5UHY1Oju7qALK2Ydl+RMbgilx9g867ObrA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1727620983; c=relaxed/simple; bh=r9g2mDKo1VyZh/uQk94rwCQODx7VEt13cnsjwkriLJY=; h=Date:From:To:Cc:Subject:Message-ID:MIME-Version:Content-Type: Content-Disposition:In-Reply-To; b=b2jFb8E6uWeHdgsTWC14Fok/RPtGHP9rm70qpiApYO9q9XdKVpRAdXRo4y53g4xEAB0pKKBB1GP4ntHD5eHzKRCDCFc2QwdGNjeslvE7r4P3nfCW7MO5lFywMExPs/gKu2ctPEIokvMDpo56lHhYsmTBz/OEcMDv2FBfUpKgyQk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=AMgAssKR; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="AMgAssKR" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1727620981; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to; bh=mEl9ds8cbxbLGs6JnmKeAQsfkw4QJ2bnOpSp9pzGta8=; b=AMgAssKRdhMznu4P+I1AFafYG15ofOaz41szQaD6HV4QqUgAqoLOWNRk841qvicIM1baBf wLBT4DDUhgY+gje97BY41zJYt1R2WFKQ+Yba8LdHscyk04jDe7iDxn3p6wakzYpjDOz3+A FeygGk7KLkvug/4DdKuKd6CpUYwWGSI= Received: from mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-647-Yhzs64IxNfqpMlvjWoTwfw-1; Sun, 29 Sep 2024 10:42:58 -0400 X-MC-Unique: Yhzs64IxNfqpMlvjWoTwfw-1 Received: from mx-prod-int-04.mail-002.prod.us-west-2.aws.redhat.com (unknown [10.30.177.40]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id B460119B9ABA; Sun, 29 Sep 2024 14:42:56 +0000 (UTC) Received: from dhcp-27-174.brq.redhat.com (unknown [10.45.224.44]) by mx-prod-int-04.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with SMTP id 0ED1B1944B22; Sun, 29 Sep 2024 14:42:53 +0000 (UTC) Received: by dhcp-27-174.brq.redhat.com (nbSMTP-1.00) for uid 1000 oleg@redhat.com; Sun, 29 Sep 2024 16:42:43 +0200 (CEST) Date: Sun, 29 Sep 2024 16:42:39 +0200 From: Oleg Nesterov To: Andrii Nakryiko , Jiri Olsa , Masami Hiramatsu , Peter Zijlstra Cc: Liao Chang , linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org Subject: [PATCH 3/7] uprobes: kill the unnecessary put_uprobe/xol_free_insn_slot in uprobe_free_utask() Message-ID: <20240929144239.GA9475@redhat.com> Precedence: bulk X-Mailing-List: linux-trace-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <20240929144201.GA9429@redhat.com> User-Agent: Mutt/1.5.24 (2015-08-30) X-Scanned-By: MIMEDefang 3.0 on 10.30.177.40 If pre_ssout() succeeds and sets utask->active_uprobe and utask->xol_vaddr the task must not exit until it calls handle_singlestep() which does the necessary put_uprobe() and xol_free_insn_slot(). Remove put_uprobe() and xol_free_insn_slot() from uprobe_free_utask(). With this change xol_free_insn_slot() can't hit xol_area/utask/xol_vaddr == NULL, we can kill the unnecessary checks checks and simplify this function more. Signed-off-by: Oleg Nesterov --- kernel/events/uprobes.c | 24 +++++------------------- 1 file changed, 5 insertions(+), 19 deletions(-) diff --git a/kernel/events/uprobes.c b/kernel/events/uprobes.c index 3023714b83f2..4619de10772e 100644 --- a/kernel/events/uprobes.c +++ b/kernel/events/uprobes.c @@ -1676,28 +1676,16 @@ static unsigned long xol_get_insn_slot(struct uprobe *uprobe) } /* - * xol_free_insn_slot - If slot was earlier allocated by - * @xol_get_insn_slot(), make the slot available for - * subsequent requests. + * xol_free_insn_slot - free the slot allocated by xol_get_insn_slot() */ static void xol_free_insn_slot(struct task_struct *tsk) { - struct xol_area *area; - unsigned long slot_addr; - unsigned long offset; - - if (!tsk->mm || !tsk->mm->uprobes_state.xol_area || !tsk->utask) - return; - - slot_addr = tsk->utask->xol_vaddr; - if (unlikely(!slot_addr)) - return; + struct xol_area *area = tsk->mm->uprobes_state.xol_area; + unsigned long offset = tsk->utask->xol_vaddr - area->vaddr; tsk->utask->xol_vaddr = 0; - area = tsk->mm->uprobes_state.xol_area; - offset = slot_addr - area->vaddr; /* - * slot_addr must fit into [area->vaddr, area->vaddr + PAGE_SIZE). + * xol_vaddr must fit into [area->vaddr, area->vaddr + PAGE_SIZE). * This check can only fail if the "[uprobes]" vma was mremap'ed. */ if (offset < PAGE_SIZE) { @@ -1767,14 +1755,12 @@ void uprobe_free_utask(struct task_struct *t) if (!utask) return; - if (utask->active_uprobe) - put_uprobe(utask->active_uprobe); + WARN_ON_ONCE(utask->active_uprobe || utask->xol_vaddr); ri = utask->return_instances; while (ri) ri = free_ret_instance(ri); - xol_free_insn_slot(t); kfree(utask); t->utask = NULL; } From patchwork Sun Sep 29 14:42:44 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oleg Nesterov X-Patchwork-Id: 13815139 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 30D1017E003 for ; Sun, 29 Sep 2024 14:43:06 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1727620988; cv=none; b=GEoo7yyuHSOMMSYVpX0MMHn8c3wn0RNT1iGsFtgdV8zeml1nv9saC5XjNdsqtjItxLHO6dL5GRTkXlSkzrNsLVuCms+BI2xt7zWDyQBOHUi5SHenjfIhTnuDW85FBs+NKid4DrFDyVgGolFZ9vXAEEqW9qg8NrLrX9/P2iahbhc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1727620988; c=relaxed/simple; bh=Qx3crvMAj4oJepAs7rbIw8d48jGe7BvkfQ64tpGkwjk=; h=Date:From:To:Cc:Subject:Message-ID:MIME-Version:Content-Type: Content-Disposition:In-Reply-To; b=unUm3Rts3U/FIBm16NeLyCqFoTi17GiSqgkWh+4vmJvMpFKCnPx8tEzFbUZ5J2huv1g7iHzOiYlzZunT/uG66UpSpQMmwb9MPL4Pr6fvJcFpr7QCKkM63ndnxPq9iNQ9VyP7KfqNDJj/wSv7lcbiiLF8Kpgc/hILXGlkdhJ0nyg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=HJAwTOzy; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="HJAwTOzy" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1727620986; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to; bh=qbjSc/cnZvendjL1YV2TjPPnNKKr87Pa0C3JPTQomg0=; b=HJAwTOzySelOaGMtDTCU1oII6s1ovRpS6DqnbAuYEndbiwwhPRg2j+uiWAen1nLnnEr+Zx H/ccxQXqBvAbZcE864HIzFxqc3yUWve1PQXcaTRL3wa3eyRKPgzLiU08ej5awcBkU/bup8 Nsd4BBRiJqjT43Gn91+9nCwybwEETUU= Received: from mx-prod-mc-04.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-556-J86Hbp8iMtqg-Jva_QujWQ-1; Sun, 29 Sep 2024 10:43:02 -0400 X-MC-Unique: J86Hbp8iMtqg-Jva_QujWQ-1 Received: from mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com (unknown [10.30.177.15]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-04.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 3FC73190ECE8; Sun, 29 Sep 2024 14:43:01 +0000 (UTC) Received: from dhcp-27-174.brq.redhat.com (unknown [10.45.224.44]) by mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with SMTP id 8A1D01979060; Sun, 29 Sep 2024 14:42:58 +0000 (UTC) Received: by dhcp-27-174.brq.redhat.com (nbSMTP-1.00) for uid 1000 oleg@redhat.com; Sun, 29 Sep 2024 16:42:47 +0200 (CEST) Date: Sun, 29 Sep 2024 16:42:44 +0200 From: Oleg Nesterov To: Andrii Nakryiko , Jiri Olsa , Masami Hiramatsu , Peter Zijlstra Cc: Liao Chang , linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org Subject: [PATCH 4/7] uprobes: simplify xol_take_insn_slot() and its caller Message-ID: <20240929144244.GA9480@redhat.com> Precedence: bulk X-Mailing-List: linux-trace-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <20240929144201.GA9429@redhat.com> User-Agent: Mutt/1.5.24 (2015-08-30) X-Scanned-By: MIMEDefang 3.0 on 10.30.177.15 The do / while (slot_nr >= UINSNS_PER_PAGE) loop in xol_take_insn_slot() makes no sense, the checked condition is always true. Change this code to use the "for (;;)" loop, this way we do not need to change slot_nr if test_and_set_bit() fails. Also, kill the unnecessary xol_vaddr != NULL check in xol_get_insn_slot(), xol_take_insn_slot() never returns NULL. Signed-off-by: Oleg Nesterov --- kernel/events/uprobes.c | 17 ++++------------- 1 file changed, 4 insertions(+), 13 deletions(-) diff --git a/kernel/events/uprobes.c b/kernel/events/uprobes.c index 4619de10772e..bfe106ecad38 100644 --- a/kernel/events/uprobes.c +++ b/kernel/events/uprobes.c @@ -1631,25 +1631,20 @@ void uprobe_dup_mmap(struct mm_struct *oldmm, struct mm_struct *newmm) */ static unsigned long xol_take_insn_slot(struct xol_area *area) { - unsigned long slot_addr; - int slot_nr; + unsigned int slot_nr; - do { + for (;;) { slot_nr = find_first_zero_bit(area->bitmap, UINSNS_PER_PAGE); if (slot_nr < UINSNS_PER_PAGE) { if (!test_and_set_bit(slot_nr, area->bitmap)) break; - - slot_nr = UINSNS_PER_PAGE; continue; } wait_event(area->wq, (atomic_read(&area->slot_count) < UINSNS_PER_PAGE)); - } while (slot_nr >= UINSNS_PER_PAGE); + } - slot_addr = area->vaddr + (slot_nr * UPROBE_XOL_SLOT_BYTES); atomic_inc(&area->slot_count); - - return slot_addr; + return area->vaddr + slot_nr * UPROBE_XOL_SLOT_BYTES; } /* @@ -1666,12 +1661,8 @@ static unsigned long xol_get_insn_slot(struct uprobe *uprobe) return 0; xol_vaddr = xol_take_insn_slot(area); - if (unlikely(!xol_vaddr)) - return 0; - arch_uprobe_copy_ixol(area->page, xol_vaddr, &uprobe->arch.ixol, sizeof(uprobe->arch.ixol)); - return xol_vaddr; } From patchwork Sun Sep 29 14:42:48 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oleg Nesterov X-Patchwork-Id: 13815140 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 246FF17C9B9 for ; Sun, 29 Sep 2024 14:43:11 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1727620993; cv=none; b=uE7KtVDhM1av9HoPsG4PHOMtL/17VygqFcw3DsEgWQC8cuBY4WcdzoNhmIMANwo5/qZfUsChX6Z9J+i5DpYShGbCRxfLRqha9VwwTWz9VbDOFiM8e2yWgc71Kbw/ztXJCI+4hR15QBcpFozR3/iKZlx51Go0HaL3cn3msKLq0fg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1727620993; c=relaxed/simple; bh=4yqWNnyhvMBa++hA/O2fKRiM/JnaiH2w9lyssaDVduU=; h=Date:From:To:Cc:Subject:Message-ID:MIME-Version:Content-Type: Content-Disposition:In-Reply-To; b=H5D7xZUlT7IldEV7tlzlN5wc5DQYe8uhRaNm2Z2Du6XK5bVjsO/VnnSBLlD7VRXs/PtXmyZR4dk0fJ7bHVglk1XAnovsH9Cz9pAB/egUR6MKobSw8boQoHPrXErNW8D1r6aNLRmIdNqpWzywpLgGZacA6Yo0FoVyceWerYDvS+o= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=AwtUS/Fb; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="AwtUS/Fb" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1727620991; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to; bh=1HZRMRvEFHJMDMiUg5OPpXavNPgBWInozzwyVSKhvk4=; b=AwtUS/FbZU6if3usuQd80S6ZFEVd5WPCl4vxXllJ2+24TdK031ACI5s0F5eMBMaUIgNZ4b /zmK15Q8wqdvMZmv4i7tMbVOuekrvxMycMOqnTEb1mdURw0UjBau7WMiPIpruhkth1aXKc FKe7Ek2ZNVwzVt6n4KisIxaf/BsB15A= Received: from mx-prod-mc-04.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-640-b_c1T7tlOgqfbMOZEy7KrQ-1; Sun, 29 Sep 2024 10:43:07 -0400 X-MC-Unique: b_c1T7tlOgqfbMOZEy7KrQ-1 Received: from mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com (unknown [10.30.177.12]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-04.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id D218E19030A7; Sun, 29 Sep 2024 14:43:05 +0000 (UTC) Received: from dhcp-27-174.brq.redhat.com (unknown [10.45.224.44]) by mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with SMTP id 2ACA519560AE; Sun, 29 Sep 2024 14:43:02 +0000 (UTC) Received: by dhcp-27-174.brq.redhat.com (nbSMTP-1.00) for uid 1000 oleg@redhat.com; Sun, 29 Sep 2024 16:42:52 +0200 (CEST) Date: Sun, 29 Sep 2024 16:42:48 +0200 From: Oleg Nesterov To: Andrii Nakryiko , Jiri Olsa , Masami Hiramatsu , Peter Zijlstra Cc: Liao Chang , linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org Subject: [PATCH 5/7] uprobes: move the initialization of utask->xol_vaddr from pre_ssout() to xol_get_insn_slot() Message-ID: <20240929144248.GA9483@redhat.com> Precedence: bulk X-Mailing-List: linux-trace-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <20240929144201.GA9429@redhat.com> User-Agent: Mutt/1.5.24 (2015-08-30) X-Scanned-By: MIMEDefang 3.0 on 10.30.177.12 This simplifies the code and makes xol_get_insn_slot() symmetric with xol_free_insn_slot() which clears utask->xol_vaddr. Signed-off-by: Oleg Nesterov --- kernel/events/uprobes.c | 22 ++++++++-------------- 1 file changed, 8 insertions(+), 14 deletions(-) diff --git a/kernel/events/uprobes.c b/kernel/events/uprobes.c index bfe106ecad38..a7223be5ac2e 100644 --- a/kernel/events/uprobes.c +++ b/kernel/events/uprobes.c @@ -1649,21 +1649,19 @@ static unsigned long xol_take_insn_slot(struct xol_area *area) /* * xol_get_insn_slot - allocate a slot for xol. - * Returns the allocated slot address or 0. */ -static unsigned long xol_get_insn_slot(struct uprobe *uprobe) +static bool xol_get_insn_slot(struct uprobe *uprobe) { - struct xol_area *area; - unsigned long xol_vaddr; + struct uprobe_task *utask = current->utask; + struct xol_area *area = get_xol_area(); - area = get_xol_area(); if (!area) - return 0; + return false; - xol_vaddr = xol_take_insn_slot(area); - arch_uprobe_copy_ixol(area->page, xol_vaddr, + utask->xol_vaddr = xol_take_insn_slot(area); + arch_uprobe_copy_ixol(area->page, utask->xol_vaddr, &uprobe->arch.ixol, sizeof(uprobe->arch.ixol)); - return xol_vaddr; + return true; } /* @@ -1951,21 +1949,17 @@ static int pre_ssout(struct uprobe *uprobe, struct pt_regs *regs, unsigned long bp_vaddr) { struct uprobe_task *utask = current->utask; - unsigned long xol_vaddr; int err; if (!try_get_uprobe(uprobe)) return -EINVAL; - xol_vaddr = xol_get_insn_slot(uprobe); - if (!xol_vaddr) { + if (!xol_get_insn_slot(uprobe)) { err = -ENOMEM; goto err_out; } - utask->xol_vaddr = xol_vaddr; utask->vaddr = bp_vaddr; - err = arch_uprobe_pre_xol(&uprobe->arch, regs); if (unlikely(err)) { xol_free_insn_slot(current); From patchwork Sun Sep 29 14:42:53 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oleg Nesterov X-Patchwork-Id: 13815141 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7AFB9183088 for ; Sun, 29 Sep 2024 14:43:16 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1727620998; cv=none; b=nW8ijxR7NebKBOI4RwGqKhXmYwlK/vOkxUXfDZoPdTel9bppyOT12fFlrJVw3K+49pOxUYUCNjkzVP/CJ+xoVBa728my3iDvizpOe+ptNyyGVNJlWHmx5KF1lo+l8dcks4xxvQBSptB92MONcUu3ke4mG27juMVj8fis1/WwaWc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1727620998; c=relaxed/simple; bh=mnE4trvkE9/VgExdbnRnWJG5l/jEO8ThaYtZ1F8rCuk=; h=Date:From:To:Cc:Subject:Message-ID:MIME-Version:Content-Type: Content-Disposition:In-Reply-To; b=jE+tTaK0vIVhE+5p5+FOmliXrYAReNzE+Dc1MLZm0cjopq4JdIFyGlHV3i5/Rvs9fn/Q3KAftCokEet+3FcqDAlg44o3uSZbpmH7QR7aA/o88vapRcOXD0kEdE0psw8pGwKF5f7SZ6cK6tL2If28kMBww52h0eaJmmlHlkRtWhU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=C7Fdbxnb; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="C7Fdbxnb" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1727620995; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to; bh=sQ1CTBKnRWK4uC6arvzLMaDJzn3AO5JvwnwPCBgfSjU=; b=C7FdbxnbFRa9aY3+3spPSqdilvMynsnTm1cml9NvEhK841q1ydf2pdJZ4+VeVsga3e6DYz +uOBQetSSImndxhRWyD8y+RT015lE92HnXddJj+vYf1FkHkLqezr1MwWmD4jerSjpHnzuH yGExQ0TUKhPSZzuGQPeyh9doCBqoDvg= Received: from mx-prod-mc-04.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-284-aVe4cFt4PJiC0Hz1rE7SrA-1; Sun, 29 Sep 2024 10:43:12 -0400 X-MC-Unique: aVe4cFt4PJiC0Hz1rE7SrA-1 Received: from mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (unknown [10.30.177.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-04.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id A353219030A7; Sun, 29 Sep 2024 14:43:10 +0000 (UTC) Received: from dhcp-27-174.brq.redhat.com (unknown [10.45.224.44]) by mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with SMTP id D883B3003E40; Sun, 29 Sep 2024 14:43:07 +0000 (UTC) Received: by dhcp-27-174.brq.redhat.com (nbSMTP-1.00) for uid 1000 oleg@redhat.com; Sun, 29 Sep 2024 16:42:57 +0200 (CEST) Date: Sun, 29 Sep 2024 16:42:53 +0200 From: Oleg Nesterov To: Andrii Nakryiko , Jiri Olsa , Masami Hiramatsu , Peter Zijlstra Cc: Liao Chang , linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org Subject: [PATCH 6/7] uprobes: pass utask to xol_get_insn_slot() and xol_free_insn_slot() Message-ID: <20240929144253.GA9487@redhat.com> Precedence: bulk X-Mailing-List: linux-trace-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <20240929144201.GA9429@redhat.com> User-Agent: Mutt/1.5.24 (2015-08-30) X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.4 Add the "struct uprobe_task *utask" argument to xol_get_insn_slot() and xol_free_insn_slot(), their callers already have it so we can avoid the unnecessary dereference and simplify the code. Kill the "tsk" argument of xol_free_insn_slot(), it is always current. Signed-off-by: Oleg Nesterov --- kernel/events/uprobes.c | 17 ++++++++--------- 1 file changed, 8 insertions(+), 9 deletions(-) diff --git a/kernel/events/uprobes.c b/kernel/events/uprobes.c index a7223be5ac2e..da45d0e5bcf4 100644 --- a/kernel/events/uprobes.c +++ b/kernel/events/uprobes.c @@ -1650,9 +1650,8 @@ static unsigned long xol_take_insn_slot(struct xol_area *area) /* * xol_get_insn_slot - allocate a slot for xol. */ -static bool xol_get_insn_slot(struct uprobe *uprobe) +static bool xol_get_insn_slot(struct uprobe *uprobe, struct uprobe_task *utask) { - struct uprobe_task *utask = current->utask; struct xol_area *area = get_xol_area(); if (!area) @@ -1667,12 +1666,12 @@ static bool xol_get_insn_slot(struct uprobe *uprobe) /* * xol_free_insn_slot - free the slot allocated by xol_get_insn_slot() */ -static void xol_free_insn_slot(struct task_struct *tsk) +static void xol_free_insn_slot(struct uprobe_task *utask) { - struct xol_area *area = tsk->mm->uprobes_state.xol_area; - unsigned long offset = tsk->utask->xol_vaddr - area->vaddr; + struct xol_area *area = current->mm->uprobes_state.xol_area; + unsigned long offset = utask->xol_vaddr - area->vaddr; - tsk->utask->xol_vaddr = 0; + utask->xol_vaddr = 0; /* * xol_vaddr must fit into [area->vaddr, area->vaddr + PAGE_SIZE). * This check can only fail if the "[uprobes]" vma was mremap'ed. @@ -1954,7 +1953,7 @@ pre_ssout(struct uprobe *uprobe, struct pt_regs *regs, unsigned long bp_vaddr) if (!try_get_uprobe(uprobe)) return -EINVAL; - if (!xol_get_insn_slot(uprobe)) { + if (!xol_get_insn_slot(uprobe, utask)) { err = -ENOMEM; goto err_out; } @@ -1962,7 +1961,7 @@ pre_ssout(struct uprobe *uprobe, struct pt_regs *regs, unsigned long bp_vaddr) utask->vaddr = bp_vaddr; err = arch_uprobe_pre_xol(&uprobe->arch, regs); if (unlikely(err)) { - xol_free_insn_slot(current); + xol_free_insn_slot(utask); goto err_out; } @@ -2313,7 +2312,7 @@ static void handle_singlestep(struct uprobe_task *utask, struct pt_regs *regs) put_uprobe(uprobe); utask->active_uprobe = NULL; utask->state = UTASK_RUNNING; - xol_free_insn_slot(current); + xol_free_insn_slot(utask); spin_lock_irq(¤t->sighand->siglock); recalc_sigpending(); /* see uprobe_deny_signal() */ From patchwork Sun Sep 29 14:42:58 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oleg Nesterov X-Patchwork-Id: 13815142 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5CDD1183CC9 for ; Sun, 29 Sep 2024 14:43:21 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1727621002; cv=none; b=o7u913SI74dM0CK9AHAqS6pdbiONIsn62gn9Bop0MYwEQfb1/veNarNivFBG3fHWFnnVKf6D3QkbrWXdQFKxJkTZxp2dB8QHBcJRIlj3qRy2QDr2DPvOHJHjbLzmTiiKOOjFg12fiIq0dWNQeBSVTSgQ7U+sRzo39FA5qNnGNQ8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1727621002; c=relaxed/simple; bh=FfaZhrQq34BZlheLdSwojN4sapArypT72+Y+nLTOLDA=; h=Date:From:To:Cc:Subject:Message-ID:MIME-Version:Content-Type: Content-Disposition:In-Reply-To; b=HfBnbHLGkdmbH4HZ9zr6ATiqY5e158mmKy8m0m7hCYUuNM1XpeELRbMX2oARkYbDV97uLyMr+gwfuuZjjSlEmJu+OPB3TioiRk+SMDTIVdMrqD9Y3EDWc92L9QuQWe638r7/+fq6FZTS6rbOtWpQyTFj/Y0/7yt7Y6LUmXDf1CA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=fcOlL1X9; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="fcOlL1X9" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1727621000; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to; bh=PFEiL3F5MSjU++Jxe01AvOazddBzZZRjIDhG9rgyzoQ=; b=fcOlL1X9TFa4n8u/GCm8dWpVUOpDmdhiRws/8o/KJTDva0nQyHtKi7KGxGe6lE7ac6hsli 2V3CV4FlleOhUfOy9o/0TLn1BX6sm5N0o6J39/T81V+DJgBi9hH+oQCLEKT0unIZei5kLy NzbyHbJfnuoqjHyCTBZrKuAAnbGMU/k= Received: from mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-530-2aiClEGVNq6dSkKRgCxqKg-1; Sun, 29 Sep 2024 10:43:16 -0400 X-MC-Unique: 2aiClEGVNq6dSkKRgCxqKg-1 Received: from mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com (unknown [10.30.177.15]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 42BB0195FE21; Sun, 29 Sep 2024 14:43:15 +0000 (UTC) Received: from dhcp-27-174.brq.redhat.com (unknown [10.45.224.44]) by mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with SMTP id 90E271979060; Sun, 29 Sep 2024 14:43:12 +0000 (UTC) Received: by dhcp-27-174.brq.redhat.com (nbSMTP-1.00) for uid 1000 oleg@redhat.com; Sun, 29 Sep 2024 16:43:01 +0200 (CEST) Date: Sun, 29 Sep 2024 16:42:58 +0200 From: Oleg Nesterov To: Andrii Nakryiko , Jiri Olsa , Masami Hiramatsu , Peter Zijlstra Cc: Liao Chang , linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org Subject: [PATCH 7/7] uprobes: deny mremap(xol_vma) Message-ID: <20240929144258.GA9492@redhat.com> Precedence: bulk X-Mailing-List: linux-trace-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <20240929144201.GA9429@redhat.com> User-Agent: Mutt/1.5.24 (2015-08-30) X-Scanned-By: MIMEDefang 3.0 on 10.30.177.15 kernel/events/uprobes.c assumes that xol_area->vaddr is always correct but a malicious application can remap its "[uprobes]" vma to another adress to confuse the kernel. Introduce xol_mremap() to make this impossible. With this change utask->xol_vaddr in xol_free_insn_slot() can't be invalid, we can turn the offset check into WARN_ON_ONCE(offset >= PAGE_SIZE). Signed-off-by: Oleg Nesterov --- kernel/events/uprobes.c | 30 +++++++++++++++++------------- 1 file changed, 17 insertions(+), 13 deletions(-) diff --git a/kernel/events/uprobes.c b/kernel/events/uprobes.c index da45d0e5bcf4..20c58b6ee1ad 100644 --- a/kernel/events/uprobes.c +++ b/kernel/events/uprobes.c @@ -1475,9 +1475,15 @@ static vm_fault_t xol_fault(const struct vm_special_mapping *sm, return 0; } +static int xol_mremap(const struct vm_special_mapping *sm, struct vm_area_struct *new_vma) +{ + return -EPERM; +} + static const struct vm_special_mapping xol_mapping = { .name = "[uprobes]", .fault = xol_fault, + .mremap = xol_mremap, }; /* Slot allocation for XOL */ @@ -1670,21 +1676,19 @@ static void xol_free_insn_slot(struct uprobe_task *utask) { struct xol_area *area = current->mm->uprobes_state.xol_area; unsigned long offset = utask->xol_vaddr - area->vaddr; + unsigned int slot_nr; utask->xol_vaddr = 0; - /* - * xol_vaddr must fit into [area->vaddr, area->vaddr + PAGE_SIZE). - * This check can only fail if the "[uprobes]" vma was mremap'ed. - */ - if (offset < PAGE_SIZE) { - int slot_nr = offset / UPROBE_XOL_SLOT_BYTES; - - clear_bit(slot_nr, area->bitmap); - atomic_dec(&area->slot_count); - smp_mb__after_atomic(); /* pairs with prepare_to_wait() */ - if (waitqueue_active(&area->wq)) - wake_up(&area->wq); - } + /* xol_vaddr must fit into [area->vaddr, area->vaddr + PAGE_SIZE) */ + if (WARN_ON_ONCE(offset >= PAGE_SIZE)) + return; + + slot_nr = offset / UPROBE_XOL_SLOT_BYTES; + clear_bit(slot_nr, area->bitmap); + atomic_dec(&area->slot_count); + smp_mb__after_atomic(); /* pairs with prepare_to_wait() */ + if (waitqueue_active(&area->wq)) + wake_up(&area->wq); } void __weak arch_uprobe_copy_ixol(struct page *page, unsigned long vaddr,