diff mbox series

[2/2] riscv: tlb: avoid tlb flushing if fullmm == 1

Message ID 20231228084642.1765-3-jszhang@kernel.org (mailing list archive)
State New, archived
Headers show
Series riscv: tlb: avoid tlb flushing on exit & execve | expand

Commit Message

Jisheng Zhang Dec. 28, 2023, 8:46 a.m. UTC
The mmu_gather code sets fullmm=1 when tearing down the entire address
space for an mm_struct on exit or execve. So if the underlying platform
supports ASID, the tlb flushing can be avoided because the ASID
allocator will never re-allocate a dirty ASID.

Use the performance of Process creation in unixbench on T-HEAD TH1520
platform is improved by about 4%.

Signed-off-by: Jisheng Zhang <jszhang@kernel.org>
---
 arch/riscv/include/asm/tlb.h | 9 +++++++++
 1 file changed, 9 insertions(+)

Comments

Alexandre Ghiti Dec. 30, 2023, 6:26 p.m. UTC | #1
Hi Jisheng,

On 28/12/2023 09:46, Jisheng Zhang wrote:
> The mmu_gather code sets fullmm=1 when tearing down the entire address
> space for an mm_struct on exit or execve. So if the underlying platform
> supports ASID, the tlb flushing can be avoided because the ASID
> allocator will never re-allocate a dirty ASID.
>
> Use the performance of Process creation in unixbench on T-HEAD TH1520
> platform is improved by about 4%.
>
> Signed-off-by: Jisheng Zhang <jszhang@kernel.org>
> ---
>   arch/riscv/include/asm/tlb.h | 9 +++++++++
>   1 file changed, 9 insertions(+)
>
> diff --git a/arch/riscv/include/asm/tlb.h b/arch/riscv/include/asm/tlb.h
> index 1eb5682b2af6..35f3c214332e 100644
> --- a/arch/riscv/include/asm/tlb.h
> +++ b/arch/riscv/include/asm/tlb.h
> @@ -12,10 +12,19 @@ static void tlb_flush(struct mmu_gather *tlb);
>   
>   #define tlb_flush tlb_flush
>   #include <asm-generic/tlb.h>
> +#include <asm/mmu_context.h>
>   
>   static inline void tlb_flush(struct mmu_gather *tlb)
>   {
>   #ifdef CONFIG_MMU
> +	/*
> +	 * If ASID is supported, the ASID allocator will either invalidate the
> +	 * ASID or mark it as used. So we can avoid TLB invalidation when
> +	 * pulling down a full mm.
> +	 */


Given the number of bits are limited for the ASID, at some point we'll 
reuse previously allocated ASID so the ASID allocator must make sure to 
invalidate the entries when reusing an ASID: can you point where this is 
done?

Thanks,

Alex


> +	if (static_branch_likely(&use_asid_allocator) && tlb->fullmm)
> +		return;
> +
>   	if (tlb->fullmm || tlb->need_flush_all)
>   		flush_tlb_mm(tlb->mm);
>   	else
Jisheng Zhang Jan. 2, 2024, 3:12 a.m. UTC | #2
On Sat, Dec 30, 2023 at 07:26:11PM +0100, Alexandre Ghiti wrote:
> Hi Jisheng,

Hi Alex,

> 
> On 28/12/2023 09:46, Jisheng Zhang wrote:
> > The mmu_gather code sets fullmm=1 when tearing down the entire address
> > space for an mm_struct on exit or execve. So if the underlying platform
> > supports ASID, the tlb flushing can be avoided because the ASID
> > allocator will never re-allocate a dirty ASID.
> > 
> > Use the performance of Process creation in unixbench on T-HEAD TH1520
> > platform is improved by about 4%.
> > 
> > Signed-off-by: Jisheng Zhang <jszhang@kernel.org>
> > ---
> >   arch/riscv/include/asm/tlb.h | 9 +++++++++
> >   1 file changed, 9 insertions(+)
> > 
> > diff --git a/arch/riscv/include/asm/tlb.h b/arch/riscv/include/asm/tlb.h
> > index 1eb5682b2af6..35f3c214332e 100644
> > --- a/arch/riscv/include/asm/tlb.h
> > +++ b/arch/riscv/include/asm/tlb.h
> > @@ -12,10 +12,19 @@ static void tlb_flush(struct mmu_gather *tlb);
> >   #define tlb_flush tlb_flush
> >   #include <asm-generic/tlb.h>
> > +#include <asm/mmu_context.h>
> >   static inline void tlb_flush(struct mmu_gather *tlb)
> >   {
> >   #ifdef CONFIG_MMU
> > +	/*
> > +	 * If ASID is supported, the ASID allocator will either invalidate the
> > +	 * ASID or mark it as used. So we can avoid TLB invalidation when
> > +	 * pulling down a full mm.
> > +	 */
> 
> 
> Given the number of bits are limited for the ASID, at some point we'll reuse
> previously allocated ASID so the ASID allocator must make sure to invalidate
> the entries when reusing an ASID: can you point where this is done?

Per my understanding of the code, the path would be
set_mm_asid()
  __new_context()
    __flush_context()  // set context_tlb_flush_pending
if (need_flush_tlb)
  local_flush_tlb_all()

Thanks
 
> 
> > +	if (static_branch_likely(&use_asid_allocator) && tlb->fullmm)
> > +		return;
> > +
> >   	if (tlb->fullmm || tlb->need_flush_all)
> >   		flush_tlb_mm(tlb->mm);
> >   	else
Alexandre Ghiti Jan. 4, 2024, 1 p.m. UTC | #3
On 02/01/2024 04:12, Jisheng Zhang wrote:
> On Sat, Dec 30, 2023 at 07:26:11PM +0100, Alexandre Ghiti wrote:
>> Hi Jisheng,
> Hi Alex,
>
>> On 28/12/2023 09:46, Jisheng Zhang wrote:
>>> The mmu_gather code sets fullmm=1 when tearing down the entire address
>>> space for an mm_struct on exit or execve. So if the underlying platform
>>> supports ASID, the tlb flushing can be avoided because the ASID
>>> allocator will never re-allocate a dirty ASID.
>>>
>>> Use the performance of Process creation in unixbench on T-HEAD TH1520
>>> platform is improved by about 4%.
>>>
>>> Signed-off-by: Jisheng Zhang <jszhang@kernel.org>
>>> ---
>>>    arch/riscv/include/asm/tlb.h | 9 +++++++++
>>>    1 file changed, 9 insertions(+)
>>>
>>> diff --git a/arch/riscv/include/asm/tlb.h b/arch/riscv/include/asm/tlb.h
>>> index 1eb5682b2af6..35f3c214332e 100644
>>> --- a/arch/riscv/include/asm/tlb.h
>>> +++ b/arch/riscv/include/asm/tlb.h
>>> @@ -12,10 +12,19 @@ static void tlb_flush(struct mmu_gather *tlb);
>>>    #define tlb_flush tlb_flush
>>>    #include <asm-generic/tlb.h>
>>> +#include <asm/mmu_context.h>
>>>    static inline void tlb_flush(struct mmu_gather *tlb)
>>>    {
>>>    #ifdef CONFIG_MMU
>>> +	/*
>>> +	 * If ASID is supported, the ASID allocator will either invalidate the
>>> +	 * ASID or mark it as used. So we can avoid TLB invalidation when
>>> +	 * pulling down a full mm.
>>> +	 */
>>
>> Given the number of bits are limited for the ASID, at some point we'll reuse
>> previously allocated ASID so the ASID allocator must make sure to invalidate
>> the entries when reusing an ASID: can you point where this is done?
> Per my understanding of the code, the path would be
> set_mm_asid()
>    __new_context()
>      __flush_context()  // set context_tlb_flush_pending
> if (need_flush_tlb)
>    local_flush_tlb_all()


Ok thanks, so feel free to add:

Reviewed-by: Alexandre Ghiti <alexghiti@rivosinc.com>

Thanks!

Alex


>
> Thanks
>   
>>> +	if (static_branch_likely(&use_asid_allocator) && tlb->fullmm)
>>> +		return;
>>> +
>>>    	if (tlb->fullmm || tlb->need_flush_all)
>>>    		flush_tlb_mm(tlb->mm);
>>>    	else
> _______________________________________________
> linux-riscv mailing list
> linux-riscv@lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-riscv
diff mbox series

Patch

diff --git a/arch/riscv/include/asm/tlb.h b/arch/riscv/include/asm/tlb.h
index 1eb5682b2af6..35f3c214332e 100644
--- a/arch/riscv/include/asm/tlb.h
+++ b/arch/riscv/include/asm/tlb.h
@@ -12,10 +12,19 @@  static void tlb_flush(struct mmu_gather *tlb);
 
 #define tlb_flush tlb_flush
 #include <asm-generic/tlb.h>
+#include <asm/mmu_context.h>
 
 static inline void tlb_flush(struct mmu_gather *tlb)
 {
 #ifdef CONFIG_MMU
+	/*
+	 * If ASID is supported, the ASID allocator will either invalidate the
+	 * ASID or mark it as used. So we can avoid TLB invalidation when
+	 * pulling down a full mm.
+	 */
+	if (static_branch_likely(&use_asid_allocator) && tlb->fullmm)
+		return;
+
 	if (tlb->fullmm || tlb->need_flush_all)
 		flush_tlb_mm(tlb->mm);
 	else