| Message ID | 20220302084624.33340-1-songmuchun@bytedance.com (mailing list archive) |
|---|---|
| State | New |
| Headers | show |
| Series | [v2,RESEND,1/2] arm64: avoid flushing icache multiple times on contiguous HugeTLB | expand |
On Wed, Mar 02, 2022 at 04:46:23PM +0800, Muchun Song wrote: > When a contiguous HugeTLB page is mapped, set_pte_at() will be called > CONT_PTES/CONT_PMDS times. Therefore, __sync_icache_dcache() will > flush cache multiple times if the page is executable (to ensure > the I-D cache coherency). However, the first flushing cache already > covers subsequent cache flush operations. So only flusing cache > for the head page if it is a HugeTLB page to avoid redundant cache > flushing. In the next patch, it is also depends on this change > since the tail vmemmap pages of HugeTLB is mapped with read-only > meanning only head page struct can be modified. > > Signed-off-by: Muchun Song <songmuchun@bytedance.com> Reviewed-by: Catalin Marinas <catalin.marinas@arm.com> (for this patch only, I have yet to figure out whether Anshuman's and Mark's comments have been addressed in patch 2)
On Wed, 2 Mar 2022 16:46:23 +0800, Muchun Song wrote: > When a contiguous HugeTLB page is mapped, set_pte_at() will be called > CONT_PTES/CONT_PMDS times. Therefore, __sync_icache_dcache() will > flush cache multiple times if the page is executable (to ensure > the I-D cache coherency). However, the first flushing cache already > covers subsequent cache flush operations. So only flusing cache > for the head page if it is a HugeTLB page to avoid redundant cache > flushing. In the next patch, it is also depends on this change > since the tail vmemmap pages of HugeTLB is mapped with read-only > meanning only head page struct can be modified. > > [...] Applied first patch only to arm64 (for-next/mm), thanks! [1/2] arm64: avoid flushing icache multiple times on contiguous HugeTLB https://git.kernel.org/arm64/c/cf5a501d985b Cheers,
diff --git a/arch/arm64/mm/flush.c b/arch/arm64/mm/flush.c index 2aaf950b906c..a06c6ac770d4 100644 --- a/arch/arm64/mm/flush.c +++ b/arch/arm64/mm/flush.c @@ -52,6 +52,13 @@ void __sync_icache_dcache(pte_t pte) { struct page *page = pte_page(pte); + /* + * HugeTLB pages are always fully mapped, so only setting head page's + * PG_dcache_clean flag is enough. + */ + if (PageHuge(page)) + page = compound_head(page); + if (!test_bit(PG_dcache_clean, &page->flags)) { sync_icache_aliases((unsigned long)page_address(page), (unsigned long)page_address(page) +
When a contiguous HugeTLB page is mapped, set_pte_at() will be called CONT_PTES/CONT_PMDS times. Therefore, __sync_icache_dcache() will flush cache multiple times if the page is executable (to ensure the I-D cache coherency). However, the first flushing cache already covers subsequent cache flush operations. So only flusing cache for the head page if it is a HugeTLB page to avoid redundant cache flushing. In the next patch, it is also depends on this change since the tail vmemmap pages of HugeTLB is mapped with read-only meanning only head page struct can be modified. Signed-off-by: Muchun Song <songmuchun@bytedance.com> --- arch/arm64/mm/flush.c | 7 +++++++ 1 file changed, 7 insertions(+)