| Message ID | 20210718214134.2619099-2-surenb@google.com (mailing list archive) |
|---|---|
| State | New |
| Headers | show |
| Series | [v2,1/3] mm, oom: move task_will_free_mem up in the file to be used in process_mrelease | expand |
On 18.07.21 23:41, Suren Baghdasaryan wrote: > In modern systems it's not unusual to have a system component monitoring > memory conditions of the system and tasked with keeping system memory > pressure under control. One way to accomplish that is to kill > non-essential processes to free up memory for more important ones. > Examples of this are Facebook's OOM killer daemon called oomd and > Android's low memory killer daemon called lmkd. > For such system component it's important to be able to free memory > quickly and efficiently. Unfortunately the time process takes to free > up its memory after receiving a SIGKILL might vary based on the state > of the process (uninterruptible sleep), size and OPP level of the core > the process is running. A mechanism to free resources of the target > process in a more predictable way would improve system's ability to > control its memory pressure. > Introduce process_mrelease system call that releases memory of a dying > process from the context of the caller. This way the memory is freed in > a more controllable way with CPU affinity and priority of the caller. > The workload of freeing the memory will also be charged to the caller. > The operation is allowed only on a dying process. > > Previously I proposed a number of alternatives to accomplish this: > - https://lore.kernel.org/patchwork/patch/1060407 extending > pidfd_send_signal to allow memory reaping using oom_reaper thread; > - https://lore.kernel.org/patchwork/patch/1338196 extending > pidfd_send_signal to reap memory of the target process synchronously from > the context of the caller; > - https://lore.kernel.org/patchwork/patch/1344419/ to add MADV_DONTNEED > support for process_madvise implementing synchronous memory reaping. To me, this looks a lot cleaner. Although I do wonder why we need two separate mechanisms to achieve the end goal 1. send sigkill 2. process_mrelease As 2. doesn't make sense without 1. it somehow feels like it would be optimal to achieve both steps in a single syscall. But I remember there were discussions around that. > > The end of the last discussion culminated with suggestion to introduce a > dedicated system call (https://lore.kernel.org/patchwork/patch/1344418/#1553875) > The reasoning was that the new variant of process_madvise > a) does not work on an address range > b) is destructive > c) doesn't share much code at all with the rest of process_madvise > From the userspace point of view it was awkward and inconvenient to provide > memory range for this operation that operates on the entire address space. > Using special flags or address values to specify the entire address space > was too hacky. > > The API is as follows, > > int process_mrelease(int pidfd, unsigned int flags); > > DESCRIPTION > The process_mrelease() system call is used to free the memory of > a process which was sent a SIGKILL signal. > > The pidfd selects the process referred to by the PID file > descriptor. > (See pidofd_open(2) for further information) > > The flags argument is reserved for future use; currently, this > argument must be specified as 0. > > RETURN VALUE > On success, process_mrelease() returns 0. On error, -1 is > returned and errno is set to indicate the error. > > ERRORS > EBADF pidfd is not a valid PID file descriptor. > > EAGAIN Failed to release part of the address space. > > EINVAL flags is not 0. > > EINVAL The task does not have a pending SIGKILL or its memory is > shared with another process with no pending SIGKILL. > > ENOSYS This system call is not supported by kernels built with no > MMU support (CONFIG_MMU=n). > > ESRCH The target process does not exist (i.e., it has terminated > and been waited on). > > Signed-off-by: Suren Baghdasaryan <surenb@google.com> > --- > mm/oom_kill.c | 55 +++++++++++++++++++++++++++++++++++++++++++++++++++ > 1 file changed, 55 insertions(+) > > diff --git a/mm/oom_kill.c b/mm/oom_kill.c > index d04a13dc9fde..7fbfa70d4e97 100644 > --- a/mm/oom_kill.c > +++ b/mm/oom_kill.c > @@ -28,6 +28,7 @@ > #include <linux/sched/task.h> > #include <linux/sched/debug.h> > #include <linux/swap.h> > +#include <linux/syscalls.h> > #include <linux/timex.h> > #include <linux/jiffies.h> > #include <linux/cpuset.h> > @@ -755,10 +756,64 @@ static int __init oom_init(void) > return 0; > } > subsys_initcall(oom_init) > + > +SYSCALL_DEFINE2(process_mrelease, int, pidfd, unsigned int, flags) > +{ > + struct pid *pid; > + struct task_struct *task; > + struct mm_struct *mm = NULL; > + unsigned int f_flags; > + long ret = 0; Nit: reverse Christmas tree. > + > + if (flags != 0) > + return -EINVAL; > + > + pid = pidfd_get_pid(pidfd, &f_flags); > + if (IS_ERR(pid)) > + return PTR_ERR(pid); > + > + task = get_pid_task(pid, PIDTYPE_PID); > + if (!task) { > + ret = -ESRCH; > + goto put_pid; > + } > + > + /* > + * If the task is dying and in the process of releasing its memory > + * then get its mm. > + */ > + task_lock(task); > + if (task_will_free_mem(task) && (task->flags & PF_KTHREAD) == 0) { > + mm = task->mm; > + mmget(mm); > + } AFAIU, while holding the task_lock, task->mm won't change and we cannot see a concurrent exit_mm()->mmput(). So the mm structure and the VMAs won't go away while holding the task_lock(). I do wonder if we need the mmget() at all here. Also, I wonder if it would be worth dropping the task_lock() while reaping - to unblock anybody else wanting to lock the task. Getting a hold of the mm and locking the mmap_lock would be sufficient I guess. In general, looks quite good to me.
On Wed, Jul 21, 2021 at 1:02 AM David Hildenbrand <david@redhat.com> wrote: > > On 18.07.21 23:41, Suren Baghdasaryan wrote: > > In modern systems it's not unusual to have a system component monitoring > > memory conditions of the system and tasked with keeping system memory > > pressure under control. One way to accomplish that is to kill > > non-essential processes to free up memory for more important ones. > > Examples of this are Facebook's OOM killer daemon called oomd and > > Android's low memory killer daemon called lmkd. > > For such system component it's important to be able to free memory > > quickly and efficiently. Unfortunately the time process takes to free > > up its memory after receiving a SIGKILL might vary based on the state > > of the process (uninterruptible sleep), size and OPP level of the core > > the process is running. A mechanism to free resources of the target > > process in a more predictable way would improve system's ability to > > control its memory pressure. > > Introduce process_mrelease system call that releases memory of a dying > > process from the context of the caller. This way the memory is freed in > > a more controllable way with CPU affinity and priority of the caller. > > The workload of freeing the memory will also be charged to the caller. > > The operation is allowed only on a dying process. > > > > Previously I proposed a number of alternatives to accomplish this: > > - https://lore.kernel.org/patchwork/patch/1060407 extending > > pidfd_send_signal to allow memory reaping using oom_reaper thread; > > - https://lore.kernel.org/patchwork/patch/1338196 extending > > pidfd_send_signal to reap memory of the target process synchronously from > > the context of the caller; > > - https://lore.kernel.org/patchwork/patch/1344419/ to add MADV_DONTNEED > > support for process_madvise implementing synchronous memory reaping. > > To me, this looks a lot cleaner. Although I do wonder why we need two > separate mechanisms to achieve the end goal > > 1. send sigkill > 2. process_mrelease > > As 2. doesn't make sense without 1. it somehow feels like it would be > optimal to achieve both steps in a single syscall. But I remember there > were discussions around that. Yep, we recently discussed the approach in this thread: https://lore.kernel.org/patchwork/patch/1450952/#1652452 > > > > > The end of the last discussion culminated with suggestion to introduce a > > dedicated system call (https://lore.kernel.org/patchwork/patch/1344418/#1553875) > > The reasoning was that the new variant of process_madvise > > a) does not work on an address range > > b) is destructive > > c) doesn't share much code at all with the rest of process_madvise > > From the userspace point of view it was awkward and inconvenient to provide > > memory range for this operation that operates on the entire address space. > > Using special flags or address values to specify the entire address space > > was too hacky. > > > > The API is as follows, > > > > int process_mrelease(int pidfd, unsigned int flags); > > > > DESCRIPTION > > The process_mrelease() system call is used to free the memory of > > a process which was sent a SIGKILL signal. > > > > The pidfd selects the process referred to by the PID file > > descriptor. > > (See pidofd_open(2) for further information) > > > > The flags argument is reserved for future use; currently, this > > argument must be specified as 0. > > > > RETURN VALUE > > On success, process_mrelease() returns 0. On error, -1 is > > returned and errno is set to indicate the error. > > > > ERRORS > > EBADF pidfd is not a valid PID file descriptor. > > > > EAGAIN Failed to release part of the address space. > > > > EINVAL flags is not 0. > > > > EINVAL The task does not have a pending SIGKILL or its memory is > > shared with another process with no pending SIGKILL. > > > > ENOSYS This system call is not supported by kernels built with no > > MMU support (CONFIG_MMU=n). > > > > ESRCH The target process does not exist (i.e., it has terminated > > and been waited on). > > > > Signed-off-by: Suren Baghdasaryan <surenb@google.com> > > --- > > mm/oom_kill.c | 55 +++++++++++++++++++++++++++++++++++++++++++++++++++ > > 1 file changed, 55 insertions(+) > > > > diff --git a/mm/oom_kill.c b/mm/oom_kill.c > > index d04a13dc9fde..7fbfa70d4e97 100644 > > --- a/mm/oom_kill.c > > +++ b/mm/oom_kill.c > > @@ -28,6 +28,7 @@ > > #include <linux/sched/task.h> > > #include <linux/sched/debug.h> > > #include <linux/swap.h> > > +#include <linux/syscalls.h> > > #include <linux/timex.h> > > #include <linux/jiffies.h> > > #include <linux/cpuset.h> > > @@ -755,10 +756,64 @@ static int __init oom_init(void) > > return 0; > > } > > subsys_initcall(oom_init) > > + > > +SYSCALL_DEFINE2(process_mrelease, int, pidfd, unsigned int, flags) > > +{ > > + struct pid *pid; > > + struct task_struct *task; > > + struct mm_struct *mm = NULL; > > + unsigned int f_flags; > > + long ret = 0; > > Nit: reverse Christmas tree. Ack. Will reorder like this: struct mm_struct *mm = NULL; struct task_struct *task; unsigned int f_flags; struct pid *pid; long ret = 0; > > > + > > + if (flags != 0) > > + return -EINVAL; > > + > > + pid = pidfd_get_pid(pidfd, &f_flags); > > + if (IS_ERR(pid)) > > + return PTR_ERR(pid); > > + > > + task = get_pid_task(pid, PIDTYPE_PID); > > + if (!task) { > > + ret = -ESRCH; > > + goto put_pid; > > + } > > + > > + /* > > + * If the task is dying and in the process of releasing its memory > > + * then get its mm. > > + */ > > + task_lock(task); > > + if (task_will_free_mem(task) && (task->flags & PF_KTHREAD) == 0) { > > + mm = task->mm; > > + mmget(mm); > > + } > > AFAIU, while holding the task_lock, task->mm won't change and we cannot > see a concurrent exit_mm()->mmput(). So the mm structure and the VMAs > won't go away while holding the task_lock(). I do wonder if we need the > mmget() at all here. > > Also, I wonder if it would be worth dropping the task_lock() while > reaping - to unblock anybody else wanting to lock the task. Getting a > hold of the mm and locking the mmap_lock would be sufficient I guess. Let me take a closer look at the locking sequence here and will follow up afterwards. Thanks for the review! > > > In general, looks quite good to me. > > -- > Thanks, > > David / dhildenb >
On Wed, Jul 21, 2021 at 8:43 AM Suren Baghdasaryan <surenb@google.com> wrote: > > On Wed, Jul 21, 2021 at 1:02 AM David Hildenbrand <david@redhat.com> wrote: > > > > On 18.07.21 23:41, Suren Baghdasaryan wrote: > > > In modern systems it's not unusual to have a system component monitoring > > > memory conditions of the system and tasked with keeping system memory > > > pressure under control. One way to accomplish that is to kill > > > non-essential processes to free up memory for more important ones. > > > Examples of this are Facebook's OOM killer daemon called oomd and > > > Android's low memory killer daemon called lmkd. > > > For such system component it's important to be able to free memory > > > quickly and efficiently. Unfortunately the time process takes to free > > > up its memory after receiving a SIGKILL might vary based on the state > > > of the process (uninterruptible sleep), size and OPP level of the core > > > the process is running. A mechanism to free resources of the target > > > process in a more predictable way would improve system's ability to > > > control its memory pressure. > > > Introduce process_mrelease system call that releases memory of a dying > > > process from the context of the caller. This way the memory is freed in > > > a more controllable way with CPU affinity and priority of the caller. > > > The workload of freeing the memory will also be charged to the caller. > > > The operation is allowed only on a dying process. > > > > > > Previously I proposed a number of alternatives to accomplish this: > > > - https://lore.kernel.org/patchwork/patch/1060407 extending > > > pidfd_send_signal to allow memory reaping using oom_reaper thread; > > > - https://lore.kernel.org/patchwork/patch/1338196 extending > > > pidfd_send_signal to reap memory of the target process synchronously from > > > the context of the caller; > > > - https://lore.kernel.org/patchwork/patch/1344419/ to add MADV_DONTNEED > > > support for process_madvise implementing synchronous memory reaping. > > > > To me, this looks a lot cleaner. Although I do wonder why we need two > > separate mechanisms to achieve the end goal > > > > 1. send sigkill > > 2. process_mrelease > > > > As 2. doesn't make sense without 1. it somehow feels like it would be > > optimal to achieve both steps in a single syscall. But I remember there > > were discussions around that. > > Yep, we recently discussed the approach in this thread: > https://lore.kernel.org/patchwork/patch/1450952/#1652452 > > > > > > > > > The end of the last discussion culminated with suggestion to introduce a > > > dedicated system call (https://lore.kernel.org/patchwork/patch/1344418/#1553875) > > > The reasoning was that the new variant of process_madvise > > > a) does not work on an address range > > > b) is destructive > > > c) doesn't share much code at all with the rest of process_madvise > > > From the userspace point of view it was awkward and inconvenient to provide > > > memory range for this operation that operates on the entire address space. > > > Using special flags or address values to specify the entire address space > > > was too hacky. > > > > > > The API is as follows, > > > > > > int process_mrelease(int pidfd, unsigned int flags); > > > > > > DESCRIPTION > > > The process_mrelease() system call is used to free the memory of > > > a process which was sent a SIGKILL signal. > > > > > > The pidfd selects the process referred to by the PID file > > > descriptor. > > > (See pidofd_open(2) for further information) > > > > > > The flags argument is reserved for future use; currently, this > > > argument must be specified as 0. > > > > > > RETURN VALUE > > > On success, process_mrelease() returns 0. On error, -1 is > > > returned and errno is set to indicate the error. > > > > > > ERRORS > > > EBADF pidfd is not a valid PID file descriptor. > > > > > > EAGAIN Failed to release part of the address space. > > > > > > EINVAL flags is not 0. > > > > > > EINVAL The task does not have a pending SIGKILL or its memory is > > > shared with another process with no pending SIGKILL. > > > > > > ENOSYS This system call is not supported by kernels built with no > > > MMU support (CONFIG_MMU=n). > > > > > > ESRCH The target process does not exist (i.e., it has terminated > > > and been waited on). > > > > > > Signed-off-by: Suren Baghdasaryan <surenb@google.com> > > > --- > > > mm/oom_kill.c | 55 +++++++++++++++++++++++++++++++++++++++++++++++++++ > > > 1 file changed, 55 insertions(+) > > > > > > diff --git a/mm/oom_kill.c b/mm/oom_kill.c > > > index d04a13dc9fde..7fbfa70d4e97 100644 > > > --- a/mm/oom_kill.c > > > +++ b/mm/oom_kill.c > > > @@ -28,6 +28,7 @@ > > > #include <linux/sched/task.h> > > > #include <linux/sched/debug.h> > > > #include <linux/swap.h> > > > +#include <linux/syscalls.h> > > > #include <linux/timex.h> > > > #include <linux/jiffies.h> > > > #include <linux/cpuset.h> > > > @@ -755,10 +756,64 @@ static int __init oom_init(void) > > > return 0; > > > } > > > subsys_initcall(oom_init) > > > + > > > +SYSCALL_DEFINE2(process_mrelease, int, pidfd, unsigned int, flags) > > > +{ > > > + struct pid *pid; > > > + struct task_struct *task; > > > + struct mm_struct *mm = NULL; > > > + unsigned int f_flags; > > > + long ret = 0; > > > > Nit: reverse Christmas tree. > > Ack. Will reorder like this: > > struct mm_struct *mm = NULL; > struct task_struct *task; > unsigned int f_flags; > struct pid *pid; > long ret = 0; > > > > > > + > > > + if (flags != 0) > > > + return -EINVAL; > > > + > > > + pid = pidfd_get_pid(pidfd, &f_flags); > > > + if (IS_ERR(pid)) > > > + return PTR_ERR(pid); > > > + > > > + task = get_pid_task(pid, PIDTYPE_PID); > > > + if (!task) { > > > + ret = -ESRCH; > > > + goto put_pid; > > > + } > > > + > > > + /* > > > + * If the task is dying and in the process of releasing its memory > > > + * then get its mm. > > > + */ > > > + task_lock(task); > > > + if (task_will_free_mem(task) && (task->flags & PF_KTHREAD) == 0) { > > > + mm = task->mm; > > > + mmget(mm); > > > + } > > > > AFAIU, while holding the task_lock, task->mm won't change and we cannot > > see a concurrent exit_mm()->mmput(). So the mm structure and the VMAs > > won't go away while holding the task_lock(). I do wonder if we need the > > mmget() at all here. We do mmget() here to ensure mm is stable when it is passed later to __oom_reap_task_mm(mm)/mmap_read_lock(mm)/mmap_read_unlock(mm). Note that during those calls we do not hold task_lock anymore. > > > > Also, I wonder if it would be worth dropping the task_lock() while > > reaping - to unblock anybody else wanting to lock the task. As I mentioned above, we do not hold task_lock during reaping. We release it right after we call task_will_free_mem(), which checks that the task is exiting. task_lock is held during the call to task_will_free_mem() to satisfy the requirement listed in that function's comment: "Caller has to make sure that task->mm is stable (hold task_lock or it operates on the current)". > > Getting a hold of the mm and locking the mmap_lock would be sufficient I guess. That's exactly what I do here. The simplified sequence is: task_lock if (task_will_free_mem()) mm=mmget() task_unlock if (!mm) return; mmap_read_lock(mm) __oom_reap_task_mm(mm) mmap_read_unlock(mm) mmput(mm) Or did I misunderstand your comments? > > Let me take a closer look at the locking sequence here and will follow > up afterwards. > Thanks for the review! > > > > > > > In general, looks quite good to me. > > > > -- > > Thanks, > > > > David / dhildenb > >
>>> Getting a hold of the mm and locking the mmap_lock would be sufficient I guess. > > That's exactly what I do here. The simplified sequence is: > > task_lock > if (task_will_free_mem()) > mm=mmget() > task_unlock > if (!mm) return; > > mmap_read_lock(mm) > __oom_reap_task_mm(mm) > mmap_read_unlock(mm) > mmput(mm) > > Or did I misunderstand your comments? Oh, sorry, my tired eyes confused "put_task_struct()" with "task_unlock()" and even "mmget()" with "mmgrab()" ... So this is essentially get_task_mm() with an additional task_will_free_mem() check. LGHTM! :)
diff --git a/mm/oom_kill.c b/mm/oom_kill.c index d04a13dc9fde..7fbfa70d4e97 100644 --- a/mm/oom_kill.c +++ b/mm/oom_kill.c @@ -28,6 +28,7 @@ #include <linux/sched/task.h> #include <linux/sched/debug.h> #include <linux/swap.h> +#include <linux/syscalls.h> #include <linux/timex.h> #include <linux/jiffies.h> #include <linux/cpuset.h> @@ -755,10 +756,64 @@ static int __init oom_init(void) return 0; } subsys_initcall(oom_init) + +SYSCALL_DEFINE2(process_mrelease, int, pidfd, unsigned int, flags) +{ + struct pid *pid; + struct task_struct *task; + struct mm_struct *mm = NULL; + unsigned int f_flags; + long ret = 0; + + if (flags != 0) + return -EINVAL; + + pid = pidfd_get_pid(pidfd, &f_flags); + if (IS_ERR(pid)) + return PTR_ERR(pid); + + task = get_pid_task(pid, PIDTYPE_PID); + if (!task) { + ret = -ESRCH; + goto put_pid; + } + + /* + * If the task is dying and in the process of releasing its memory + * then get its mm. + */ + task_lock(task); + if (task_will_free_mem(task) && (task->flags & PF_KTHREAD) == 0) { + mm = task->mm; + mmget(mm); + } + task_unlock(task); + if (!mm) { + ret = -EINVAL; + goto put_task; + } + + mmap_read_lock(mm); + if (!__oom_reap_task_mm(mm)) + ret = -EAGAIN; + mmap_read_unlock(mm); + + mmput(mm); +put_task: + put_task_struct(task); +put_pid: + put_pid(pid); + return ret; +} #else static inline void wake_oom_reaper(struct task_struct *tsk) { } + +SYSCALL_DEFINE2(process_mrelease, int, pidfd, unsigned int, flags) +{ + return -ENOSYS; +} #endif /* CONFIG_MMU */ /**
In modern systems it's not unusual to have a system component monitoring memory conditions of the system and tasked with keeping system memory pressure under control. One way to accomplish that is to kill non-essential processes to free up memory for more important ones. Examples of this are Facebook's OOM killer daemon called oomd and Android's low memory killer daemon called lmkd. For such system component it's important to be able to free memory quickly and efficiently. Unfortunately the time process takes to free up its memory after receiving a SIGKILL might vary based on the state of the process (uninterruptible sleep), size and OPP level of the core the process is running. A mechanism to free resources of the target process in a more predictable way would improve system's ability to control its memory pressure. Introduce process_mrelease system call that releases memory of a dying process from the context of the caller. This way the memory is freed in a more controllable way with CPU affinity and priority of the caller. The workload of freeing the memory will also be charged to the caller. The operation is allowed only on a dying process. Previously I proposed a number of alternatives to accomplish this: - https://lore.kernel.org/patchwork/patch/1060407 extending pidfd_send_signal to allow memory reaping using oom_reaper thread; - https://lore.kernel.org/patchwork/patch/1338196 extending pidfd_send_signal to reap memory of the target process synchronously from the context of the caller; - https://lore.kernel.org/patchwork/patch/1344419/ to add MADV_DONTNEED support for process_madvise implementing synchronous memory reaping. The end of the last discussion culminated with suggestion to introduce a dedicated system call (https://lore.kernel.org/patchwork/patch/1344418/#1553875) The reasoning was that the new variant of process_madvise a) does not work on an address range b) is destructive c) doesn't share much code at all with the rest of process_madvise From the userspace point of view it was awkward and inconvenient to provide memory range for this operation that operates on the entire address space. Using special flags or address values to specify the entire address space was too hacky. The API is as follows, int process_mrelease(int pidfd, unsigned int flags); DESCRIPTION The process_mrelease() system call is used to free the memory of a process which was sent a SIGKILL signal. The pidfd selects the process referred to by the PID file descriptor. (See pidofd_open(2) for further information) The flags argument is reserved for future use; currently, this argument must be specified as 0. RETURN VALUE On success, process_mrelease() returns 0. On error, -1 is returned and errno is set to indicate the error. ERRORS EBADF pidfd is not a valid PID file descriptor. EAGAIN Failed to release part of the address space. EINVAL flags is not 0. EINVAL The task does not have a pending SIGKILL or its memory is shared with another process with no pending SIGKILL. ENOSYS This system call is not supported by kernels built with no MMU support (CONFIG_MMU=n). ESRCH The target process does not exist (i.e., it has terminated and been waited on). Signed-off-by: Suren Baghdasaryan <surenb@google.com> --- mm/oom_kill.c | 55 +++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 55 insertions(+)