diff mbox series

[5/9] mm/page_alloc: inline __rmqueue_pcplist

Message ID 20210325114228.27719-6-mgorman@techsingularity.net (mailing list archive)
State New
Headers show
Series Introduce a bulk order-0 page allocator with two in-tree users | expand

Commit Message

Mel Gorman March 25, 2021, 11:42 a.m. UTC
From: Jesper Dangaard Brouer <brouer@redhat.com>

When __alloc_pages_bulk() got introduced two callers of __rmqueue_pcplist
exist and the compiler chooses to not inline this function.

 ./scripts/bloat-o-meter vmlinux-before vmlinux-inline__rmqueue_pcplist
add/remove: 0/1 grow/shrink: 2/0 up/down: 164/-125 (39)
Function                                     old     new   delta
rmqueue                                     2197    2296     +99
__alloc_pages_bulk                          1921    1986     +65
__rmqueue_pcplist                            125       -    -125
Total: Before=19374127, After=19374166, chg +0.00%

modprobe page_bench04_bulk loops=$((10**7))

Type:time_bulk_page_alloc_free_array
 -  Per elem: 106 cycles(tsc) 29.595 ns (step:64)
 - (measurement period time:0.295955434 sec time_interval:295955434)
 - (invoke count:10000000 tsc_interval:1065447105)

Before:
 - Per elem: 110 cycles(tsc) 30.633 ns (step:64)

Signed-off-by: Jesper Dangaard Brouer <brouer@redhat.com>
Signed-off-by: Mel Gorman <mgorman@techsingularity.net>
---
 mm/page_alloc.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

Comments

Vlastimil Babka April 12, 2021, 10:59 a.m. UTC | #1
On 3/25/21 12:42 PM, Mel Gorman wrote:
> From: Jesper Dangaard Brouer <brouer@redhat.com>
> 
> When __alloc_pages_bulk() got introduced two callers of __rmqueue_pcplist
> exist and the compiler chooses to not inline this function.
> 
>  ./scripts/bloat-o-meter vmlinux-before vmlinux-inline__rmqueue_pcplist
> add/remove: 0/1 grow/shrink: 2/0 up/down: 164/-125 (39)
> Function                                     old     new   delta
> rmqueue                                     2197    2296     +99
> __alloc_pages_bulk                          1921    1986     +65
> __rmqueue_pcplist                            125       -    -125
> Total: Before=19374127, After=19374166, chg +0.00%
> 
> modprobe page_bench04_bulk loops=$((10**7))
> 
> Type:time_bulk_page_alloc_free_array
>  -  Per elem: 106 cycles(tsc) 29.595 ns (step:64)
>  - (measurement period time:0.295955434 sec time_interval:295955434)
>  - (invoke count:10000000 tsc_interval:1065447105)
> 
> Before:
>  - Per elem: 110 cycles(tsc) 30.633 ns (step:64)
> 
> Signed-off-by: Jesper Dangaard Brouer <brouer@redhat.com>
> Signed-off-by: Mel Gorman <mgorman@techsingularity.net>

Acked-by: Vlastimil Babka <vbabka@suse.cz>

> ---
>  mm/page_alloc.c | 3 ++-
>  1 file changed, 2 insertions(+), 1 deletion(-)
> 
> diff --git a/mm/page_alloc.c b/mm/page_alloc.c
> index 1ec18121268b..d900e92884b2 100644
> --- a/mm/page_alloc.c
> +++ b/mm/page_alloc.c
> @@ -3415,7 +3415,8 @@ static inline void zone_statistics(struct zone *preferred_zone, struct zone *z)
>  }
>  
>  /* Remove page from the per-cpu list, caller must protect the list */
> -static struct page *__rmqueue_pcplist(struct zone *zone, int migratetype,
> +static inline
> +struct page *__rmqueue_pcplist(struct zone *zone, int migratetype,
>  			unsigned int alloc_flags,
>  			struct per_cpu_pages *pcp,
>  			struct list_head *list)
>
diff mbox series

Patch

diff --git a/mm/page_alloc.c b/mm/page_alloc.c
index 1ec18121268b..d900e92884b2 100644
--- a/mm/page_alloc.c
+++ b/mm/page_alloc.c
@@ -3415,7 +3415,8 @@  static inline void zone_statistics(struct zone *preferred_zone, struct zone *z)
 }
 
 /* Remove page from the per-cpu list, caller must protect the list */
-static struct page *__rmqueue_pcplist(struct zone *zone, int migratetype,
+static inline
+struct page *__rmqueue_pcplist(struct zone *zone, int migratetype,
 			unsigned int alloc_flags,
 			struct per_cpu_pages *pcp,
 			struct list_head *list)