[v3,0/5] proc: improve root readdir latency with many threads

Message

Brian Foster Dec. 2, 2022, 5:16 p.m. UTC

Hi all,

Here's v3 of the /proc readdir optimization patches. See v1 for the full
introductary cover letter.

Most of the feedback received to this point has been around switching
the pid code over to use the xarray api instead of the idr. Matt Wilcox
posted most of the code to do that. I cleaned it up a bit and posted a
standalone series for that here [1], but didn't receive any feedback.
Patches 1-3 of this series are essentially a repost of [1].

Patches 4-5 are otherwise mostly the same as v2 outside of switching
over to use the xarray bits instead of the idr/radix-tree.

Thoughts, reviews, flames appreciated.

Brian

[1] https://lore.kernel.org/linux-mm/[e-mail]/

v3:
- Drop radix-tree fixups.
- Convert pid idr usage to xarray.
- Replace tgid radix-tree tag set/lookup to use xarray mark.
v2: https://lore.kernel.org/linux-fsdevel/[e-mail]/
- Clean up idr helpers to be more generic.
- Use ->idr_base properly.
- Lift tgid iteration helper into pid.c to abstract tag logic from
  users.
v1: https://lore.kernel.org/linux-fsdevel/[e-mail]/

Brian Foster (5):
  pid: replace pidmap_lock with xarray lock
  pid: split cyclic id allocation cursor from idr
  pid: switch pid_namespace from idr to xarray
  pid: mark pids associated with group leader tasks
  procfs: use efficient tgid pid search on root readdir

 arch/powerpc/platforms/cell/spufs/sched.c |   2 +-
 fs/proc/base.c                            |  17 +--
 fs/proc/loadavg.c                         |   2 +-
 include/linux/pid.h                       |   3 +-
 include/linux/pid_namespace.h             |   9 +-
 include/linux/threads.h                   |   2 +-
 init/main.c                               |   3 +-
 kernel/fork.c                             |   2 +-
 kernel/pid.c                              | 177 +++++++++++++---------
 kernel/pid_namespace.c                    |  23 ++-
 10 files changed, 132 insertions(+), 108 deletions(-)

Comments

Ian Kent Dec. 12, 2022, 2:05 a.m. UTC | #1

On 3/12/22 01:16, Brian Foster wrote:
> Hi all,
>
> Here's v3 of the /proc readdir optimization patches. See v1 for the full
> introductary cover letter.
>
> Most of the feedback received to this point has been around switching
> the pid code over to use the xarray api instead of the idr. Matt Wilcox
> posted most of the code to do that. I cleaned it up a bit and posted a
> standalone series for that here [1], but didn't receive any feedback.
> Patches 1-3 of this series are essentially a repost of [1].
>
> Patches 4-5 are otherwise mostly the same as v2 outside of switching
> over to use the xarray bits instead of the idr/radix-tree.
>
> Thoughts, reviews, flames appreciated.


It looks like there's not much happens with this change so far.


Mathew, could we at least include this in linux-next, to see if

there is anything obvious to worry about since we are fiddling

with the pid numbering ... is there anything we need to do

differently for these to be included in next?


Ian

>
> Brian
>
> [1] https://lore.kernel.org/linux-mm/[e-mail]/
>
> v3:
> - Drop radix-tree fixups.
> - Convert pid idr usage to xarray.
> - Replace tgid radix-tree tag set/lookup to use xarray mark.
> v2: https://lore.kernel.org/linux-fsdevel/[e-mail]/
> - Clean up idr helpers to be more generic.
> - Use ->idr_base properly.
> - Lift tgid iteration helper into pid.c to abstract tag logic from
>    users.
> v1: https://lore.kernel.org/linux-fsdevel/[e-mail]/
>
> Brian Foster (5):
>    pid: replace pidmap_lock with xarray lock
>    pid: split cyclic id allocation cursor from idr
>    pid: switch pid_namespace from idr to xarray
>    pid: mark pids associated with group leader tasks
>    procfs: use efficient tgid pid search on root readdir
>
>   arch/powerpc/platforms/cell/spufs/sched.c |   2 +-
>   fs/proc/base.c                            |  17 +--
>   fs/proc/loadavg.c                         |   2 +-
>   include/linux/pid.h                       |   3 +-
>   include/linux/pid_namespace.h             |   9 +-
>   include/linux/threads.h                   |   2 +-
>   init/main.c                               |   3 +-
>   kernel/fork.c                             |   2 +-
>   kernel/pid.c                              | 177 +++++++++++++---------
>   kernel/pid_namespace.c                    |  23 ++-
>   10 files changed, 132 insertions(+), 108 deletions(-)
>