diff mbox series

[v1] trace: Fix race in trace_open and buffer resize call

Message ID 1600955705-27382-1-git-send-email-gkohli@codeaurora.org (mailing list archive)
State Superseded
Headers show
Series [v1] trace: Fix race in trace_open and buffer resize call | expand

Commit Message

Gaurav Kohli Sept. 24, 2020, 1:55 p.m. UTC
Below race can come, if trace_open and resize of
cpu buffer is running parallely on different cpus
CPUX                                CPUY
				    ring_buffer_resize
				    atomic_read(&buffer->resize_disabled)
tracing_open
tracing_reset_online_cpus
ring_buffer_reset_cpu
rb_reset_cpu
				    rb_update_pages
				    remove/insert pages
resetting pointer
This race can cause data abort or some times infinte loop in
rb_remove_pages and rb_insert_pages while checking pages
for sanity.

Signed-off-by: Gaurav Kohli <gkohli@codeaurora.org>
Cc: stable@vger.kernel.org
---
Changes since v0:
  -Addressed Steven's review comments.

Comments

Gaurav Kohli Oct. 5, 2020, 4:39 a.m. UTC | #1
Hi Steven,

please let us know, if below looks good to you or need modifications.

Thanks
Gaurav

On 9/24/2020 7:25 PM, Gaurav Kohli wrote:
> Below race can come, if trace_open and resize of
> cpu buffer is running parallely on different cpus
> CPUX                                CPUY
> 				    ring_buffer_resize
> 				    atomic_read(&buffer->resize_disabled)
> tracing_open
> tracing_reset_online_cpus
> ring_buffer_reset_cpu
> rb_reset_cpu
> 				    rb_update_pages
> 				    remove/insert pages
> resetting pointer
> This race can cause data abort or some times infinte loop in
> rb_remove_pages and rb_insert_pages while checking pages
> for sanity.
> 
> Signed-off-by: Gaurav Kohli <gkohli@codeaurora.org>
> Cc: stable@vger.kernel.org
> ---
> Changes since v0:
>    -Addressed Steven's review comments.
> 
> diff --git a/kernel/trace/ring_buffer.c b/kernel/trace/ring_buffer.c
> index 93ef0ab..15bf28b 100644
> --- a/kernel/trace/ring_buffer.c
> +++ b/kernel/trace/ring_buffer.c
> @@ -4866,6 +4866,9 @@ void ring_buffer_reset_cpu(struct trace_buffer *buffer, int cpu)
>   	if (!cpumask_test_cpu(cpu, buffer->cpumask))
>   		return;
>   
> +	/* prevent another thread from changing buffer sizes */
> +	mutex_lock(&buffer->mutex);
> +
>   	atomic_inc(&cpu_buffer->resize_disabled);
>   	atomic_inc(&cpu_buffer->record_disabled);
>   
> @@ -4876,6 +4879,8 @@ void ring_buffer_reset_cpu(struct trace_buffer *buffer, int cpu)
>   
>   	atomic_dec(&cpu_buffer->record_disabled);
>   	atomic_dec(&cpu_buffer->resize_disabled);
> +
> +	mutex_unlock(&buffer->mutex);
>   }
>   EXPORT_SYMBOL_GPL(ring_buffer_reset_cpu);
>   
> @@ -4889,6 +4894,9 @@ void ring_buffer_reset_online_cpus(struct trace_buffer *buffer)
>   	struct ring_buffer_per_cpu *cpu_buffer;
>   	int cpu;
>   
> +	/* prevent another thread from changing buffer sizes */
> +	mutex_lock(&buffer->mutex);
> +
>   	for_each_online_buffer_cpu(buffer, cpu) {
>   		cpu_buffer = buffer->buffers[cpu];
>   
> @@ -4907,6 +4915,8 @@ void ring_buffer_reset_online_cpus(struct trace_buffer *buffer)
>   		atomic_dec(&cpu_buffer->record_disabled);
>   		atomic_dec(&cpu_buffer->resize_disabled);
>   	}
> +
> +	mutex_unlock(&buffer->mutex);
>   }
>   
>   /**
>
Steven Rostedt Oct. 5, 2020, 2:25 p.m. UTC | #2
On Mon, 5 Oct 2020 10:09:34 +0530
Gaurav Kohli <gkohli@codeaurora.org> wrote:

> Hi Steven,
> 
> please let us know, if below looks good to you or need modifications.

Strange, I don't have your original email in my inbox. I do have it in my
LKML folder, but that's way too big for me to read. I checked my server
logs. I found where I received this from LKML, but there's no log that I
received this directly.

How are you sending out your patches? If it doesn't land in my inbox, I'll
never see it.

-- Steve
Steven Rostedt Oct. 5, 2020, 2:27 p.m. UTC | #3
On Mon, 5 Oct 2020 10:25:15 -0400
Steven Rostedt <rostedt@goodmis.org> wrote:

> On Mon, 5 Oct 2020 10:09:34 +0530
> Gaurav Kohli <gkohli@codeaurora.org> wrote:
> 
> > Hi Steven,
> > 
> > please let us know, if below looks good to you or need modifications.  
> 
> Strange, I don't have your original email in my inbox. I do have it in my
> LKML folder, but that's way too big for me to read. I checked my server
> logs. I found where I received this from LKML, but there's no log that I
> received this directly.
> 
> How are you sending out your patches? If it doesn't land in my inbox, I'll
> never see it.
>

BTW, this is the second time this has happened to me. The first time I
assumed it may have been me accidentally deleting it. Now I'm thinking it's
on your end.

I have yet to received a patch from you directly. Last time I had to pull
it from my LKML folder after you replied to me (letting me know you sent
it).

-- Steve
Gaurav Kohli Oct. 5, 2020, 4:29 p.m. UTC | #4
On 10/5/2020 7:57 PM, Steven Rostedt wrote:
> On Mon, 5 Oct 2020 10:25:15 -0400
> Steven Rostedt <rostedt@goodmis.org> wrote:
> 
>> On Mon, 5 Oct 2020 10:09:34 +0530
>> Gaurav Kohli <gkohli@codeaurora.org> wrote:
>>
>>> Hi Steven,
>>>
>>> please let us know, if below looks good to you or need modifications.
>>
>> Strange, I don't have your original email in my inbox. I do have it in my
>> LKML folder, but that's way too big for me to read. I checked my server
>> logs. I found where I received this from LKML, but there's no log that I
>> received this directly.
>>
>> How are you sending out your patches? If it doesn't land in my inbox, I'll
>> never see it.
>>
> 
> BTW, this is the second time this has happened to me. The first time I
> assumed it may have been me accidentally deleting it. Now I'm thinking it's
> on your end.
> 
> I have yet to received a patch from you directly. Last time I had to pull
> it from my LKML folder after you replied to me (letting me know you sent
> it).
> 

> -- Steve
> 

Hi Steven,

I am using normal git send-email(never saw problem with this), Not sure 
what is wrong. In my older mail i have kept you in to and rest in cc.

Let me try to resent it.
Steven Rostedt Oct. 5, 2020, 4:32 p.m. UTC | #5
On Mon, 5 Oct 2020 21:59:02 +0530
Gaurav Kohli <gkohli@codeaurora.org> wrote:

> Hi Steven,
> 
> I am using normal git send-email(never saw problem with this), Not sure 
> what is wrong. In my older mail i have kept you in to and rest in cc.
> 
> Let me try to resent it.

The Cc is working (I got it in my LKML box), but I don't see any connection
in my server. Note, the rostedt@goodmis.org is a server at my home.

-- Steve
Gaurav Kohli Oct. 5, 2020, 5:38 p.m. UTC | #6
On 10/5/2020 10:02 PM, Steven Rostedt wrote:
> On Mon, 5 Oct 2020 21:59:02 +0530
> Gaurav Kohli <gkohli@codeaurora.org> wrote:
> 
>> Hi Steven,
>>
>> I am using normal git send-email(never saw problem with this), Not sure
>> what is wrong. In my older mail i have kept you in to and rest in cc.
>>
>> Let me try to resent it.
> 
> The Cc is working (I got it in my LKML box), but I don't see any connection
> in my server. Note, the rostedt@goodmis.org is a server at my home.
> 
> -- Steve
> 

Thanks for update, i will verify my account settings and will send my 
patch tomorrow.
diff mbox series

Patch

diff --git a/kernel/trace/ring_buffer.c b/kernel/trace/ring_buffer.c
index 93ef0ab..15bf28b 100644
--- a/kernel/trace/ring_buffer.c
+++ b/kernel/trace/ring_buffer.c
@@ -4866,6 +4866,9 @@  void ring_buffer_reset_cpu(struct trace_buffer *buffer, int cpu)
 	if (!cpumask_test_cpu(cpu, buffer->cpumask))
 		return;
 
+	/* prevent another thread from changing buffer sizes */
+	mutex_lock(&buffer->mutex);
+
 	atomic_inc(&cpu_buffer->resize_disabled);
 	atomic_inc(&cpu_buffer->record_disabled);
 
@@ -4876,6 +4879,8 @@  void ring_buffer_reset_cpu(struct trace_buffer *buffer, int cpu)
 
 	atomic_dec(&cpu_buffer->record_disabled);
 	atomic_dec(&cpu_buffer->resize_disabled);
+
+	mutex_unlock(&buffer->mutex);
 }
 EXPORT_SYMBOL_GPL(ring_buffer_reset_cpu);
 
@@ -4889,6 +4894,9 @@  void ring_buffer_reset_online_cpus(struct trace_buffer *buffer)
 	struct ring_buffer_per_cpu *cpu_buffer;
 	int cpu;
 
+	/* prevent another thread from changing buffer sizes */
+	mutex_lock(&buffer->mutex);
+
 	for_each_online_buffer_cpu(buffer, cpu) {
 		cpu_buffer = buffer->buffers[cpu];
 
@@ -4907,6 +4915,8 @@  void ring_buffer_reset_online_cpus(struct trace_buffer *buffer)
 		atomic_dec(&cpu_buffer->record_disabled);
 		atomic_dec(&cpu_buffer->resize_disabled);
 	}
+
+	mutex_unlock(&buffer->mutex);
 }
 
 /**