| Message ID | 20210805170859.2389276-1-surenb@google.com (mailing list archive) |
|---|---|
| State | New |
| Headers | show |
| Series | [v7,1/2] mm: introduce process_mrelease system call | expand |
On 05.08.21 19:08, Suren Baghdasaryan wrote: > In modern systems it's not unusual to have a system component monitoring > memory conditions of the system and tasked with keeping system memory > pressure under control. One way to accomplish that is to kill > non-essential processes to free up memory for more important ones. > Examples of this are Facebook's OOM killer daemon called oomd and > Android's low memory killer daemon called lmkd. > For such system component it's important to be able to free memory > quickly and efficiently. Unfortunately the time process takes to free > up its memory after receiving a SIGKILL might vary based on the state > of the process (uninterruptible sleep), size and OPP level of the core > the process is running. A mechanism to free resources of the target > process in a more predictable way would improve system's ability to > control its memory pressure. > Introduce process_mrelease system call that releases memory of a dying > process from the context of the caller. This way the memory is freed in > a more controllable way with CPU affinity and priority of the caller. > The workload of freeing the memory will also be charged to the caller. > The operation is allowed only on a dying process. > > After previous discussions [1, 2, 3] the decision was made [4] to introduce > a dedicated system call to cover this use case. > > The API is as follows, > > int process_mrelease(int pidfd, unsigned int flags); > > DESCRIPTION > The process_mrelease() system call is used to free the memory of > an exiting process. > > The pidfd selects the process referred to by the PID file > descriptor. > (See pidfd_open(2) for further information) > > The flags argument is reserved for future use; currently, this > argument must be specified as 0. > > RETURN VALUE > On success, process_mrelease() returns 0. On error, -1 is > returned and errno is set to indicate the error. > > ERRORS > EBADF pidfd is not a valid PID file descriptor. > > EAGAIN Failed to release part of the address space. > > EINTR The call was interrupted by a signal; see signal(7). > > EINVAL flags is not 0. > > EINVAL The memory of the task cannot be released because the > process is not exiting, the address space is shared > with another live process or there is a core dump in > progress. > > ENOSYS This system call is not supported, for example, without > MMU support built into Linux. > > ESRCH The target process does not exist (i.e., it has terminated > and been waited on). > > [1] https://lore.kernel.org/lkml/20190411014353.113252-3-surenb@google.com/ > [2] https://lore.kernel.org/linux-api/20201113173448.1863419-1-surenb@google.com/ > [3] https://lore.kernel.org/linux-api/20201124053943.1684874-3-surenb@google.com/ > [4] https://lore.kernel.org/linux-api/20201223075712.GA4719@lst.de/ > > Signed-off-by: Suren Baghdasaryan <surenb@google.com> > --- > changes in v7: > - Fixed pidfd_open misspelling, per Andrew Morton > - Fixed wrong task pinning after find_lock_task_mm() issue, per Michal Hocko > - Moved MMF_OOM_SKIP check before task_will_free_mem(), per Michal Hocko > > mm/oom_kill.c | 73 +++++++++++++++++++++++++++++++++++++++++++++++++++ > 1 file changed, 73 insertions(+) > > diff --git a/mm/oom_kill.c b/mm/oom_kill.c > index c729a4c4a1ac..a4d917b43c73 100644 > --- a/mm/oom_kill.c > +++ b/mm/oom_kill.c > @@ -28,6 +28,7 @@ > #include <linux/sched/task.h> > #include <linux/sched/debug.h> > #include <linux/swap.h> > +#include <linux/syscalls.h> > #include <linux/timex.h> > #include <linux/jiffies.h> > #include <linux/cpuset.h> > @@ -1141,3 +1142,75 @@ void pagefault_out_of_memory(void) > out_of_memory(&oc); > mutex_unlock(&oom_lock); > } > + > +SYSCALL_DEFINE2(process_mrelease, int, pidfd, unsigned int, flags) > +{ > +#ifdef CONFIG_MMU > + struct mm_struct *mm = NULL; > + struct task_struct *task; > + struct task_struct *p; > + unsigned int f_flags; > + struct pid *pid; > + long ret = 0; > + > + if (flags) > + return -EINVAL; > + > + pid = pidfd_get_pid(pidfd, &f_flags); > + if (IS_ERR(pid)) > + return PTR_ERR(pid); > + > + task = get_pid_task(pid, PIDTYPE_PID); > + if (!task) { > + ret = -ESRCH; > + goto put_pid; > + } > + > + /* > + * If the task is dying and in the process of releasing its memory > + * then get its mm. > + */ > + p = find_lock_task_mm(task); > + if (!p) { > + ret = -ESRCH; > + goto put_pid; > + } > + if (task != p) { > + get_task_struct(p); Wouldn't we want to obtain the mm from p ? I thought that was the whole exercise of going via find_lock_task_mm().
On Thu, Aug 5, 2021 at 10:29 AM David Hildenbrand <david@redhat.com> wrote: > > On 05.08.21 19:08, Suren Baghdasaryan wrote: > > In modern systems it's not unusual to have a system component monitoring > > memory conditions of the system and tasked with keeping system memory > > pressure under control. One way to accomplish that is to kill > > non-essential processes to free up memory for more important ones. > > Examples of this are Facebook's OOM killer daemon called oomd and > > Android's low memory killer daemon called lmkd. > > For such system component it's important to be able to free memory > > quickly and efficiently. Unfortunately the time process takes to free > > up its memory after receiving a SIGKILL might vary based on the state > > of the process (uninterruptible sleep), size and OPP level of the core > > the process is running. A mechanism to free resources of the target > > process in a more predictable way would improve system's ability to > > control its memory pressure. > > Introduce process_mrelease system call that releases memory of a dying > > process from the context of the caller. This way the memory is freed in > > a more controllable way with CPU affinity and priority of the caller. > > The workload of freeing the memory will also be charged to the caller. > > The operation is allowed only on a dying process. > > > > After previous discussions [1, 2, 3] the decision was made [4] to introduce > > a dedicated system call to cover this use case. > > > > The API is as follows, > > > > int process_mrelease(int pidfd, unsigned int flags); > > > > DESCRIPTION > > The process_mrelease() system call is used to free the memory of > > an exiting process. > > > > The pidfd selects the process referred to by the PID file > > descriptor. > > (See pidfd_open(2) for further information) > > > > The flags argument is reserved for future use; currently, this > > argument must be specified as 0. > > > > RETURN VALUE > > On success, process_mrelease() returns 0. On error, -1 is > > returned and errno is set to indicate the error. > > > > ERRORS > > EBADF pidfd is not a valid PID file descriptor. > > > > EAGAIN Failed to release part of the address space. > > > > EINTR The call was interrupted by a signal; see signal(7). > > > > EINVAL flags is not 0. > > > > EINVAL The memory of the task cannot be released because the > > process is not exiting, the address space is shared > > with another live process or there is a core dump in > > progress. > > > > ENOSYS This system call is not supported, for example, without > > MMU support built into Linux. > > > > ESRCH The target process does not exist (i.e., it has terminated > > and been waited on). > > > > [1] https://lore.kernel.org/lkml/20190411014353.113252-3-surenb@google.com/ > > [2] https://lore.kernel.org/linux-api/20201113173448.1863419-1-surenb@google.com/ > > [3] https://lore.kernel.org/linux-api/20201124053943.1684874-3-surenb@google.com/ > > [4] https://lore.kernel.org/linux-api/20201223075712.GA4719@lst.de/ > > > > Signed-off-by: Suren Baghdasaryan <surenb@google.com> > > --- > > changes in v7: > > - Fixed pidfd_open misspelling, per Andrew Morton > > - Fixed wrong task pinning after find_lock_task_mm() issue, per Michal Hocko > > - Moved MMF_OOM_SKIP check before task_will_free_mem(), per Michal Hocko > > > > mm/oom_kill.c | 73 +++++++++++++++++++++++++++++++++++++++++++++++++++ > > 1 file changed, 73 insertions(+) > > > > diff --git a/mm/oom_kill.c b/mm/oom_kill.c > > index c729a4c4a1ac..a4d917b43c73 100644 > > --- a/mm/oom_kill.c > > +++ b/mm/oom_kill.c > > @@ -28,6 +28,7 @@ > > #include <linux/sched/task.h> > > #include <linux/sched/debug.h> > > #include <linux/swap.h> > > +#include <linux/syscalls.h> > > #include <linux/timex.h> > > #include <linux/jiffies.h> > > #include <linux/cpuset.h> > > @@ -1141,3 +1142,75 @@ void pagefault_out_of_memory(void) > > out_of_memory(&oc); > > mutex_unlock(&oom_lock); > > } > > + > > +SYSCALL_DEFINE2(process_mrelease, int, pidfd, unsigned int, flags) > > +{ > > +#ifdef CONFIG_MMU > > + struct mm_struct *mm = NULL; > > + struct task_struct *task; > > + struct task_struct *p; > > + unsigned int f_flags; > > + struct pid *pid; > > + long ret = 0; > > + > > + if (flags) > > + return -EINVAL; > > + > > + pid = pidfd_get_pid(pidfd, &f_flags); > > + if (IS_ERR(pid)) > > + return PTR_ERR(pid); > > + > > + task = get_pid_task(pid, PIDTYPE_PID); > > + if (!task) { > > + ret = -ESRCH; > > + goto put_pid; > > + } > > + > > + /* > > + * If the task is dying and in the process of releasing its memory > > + * then get its mm. > > + */ > > + p = find_lock_task_mm(task); > > + if (!p) { > > + ret = -ESRCH; > > + goto put_pid; > > + } > > + if (task != p) { > > + get_task_struct(p); > > > Wouldn't we want to obtain the mm from p ? I thought that was the whole > exercise of going via find_lock_task_mm(). Yes, that's what we do after checking task_will_free_mem(). task_will_free_mem() requires us to hold task_lock and find_lock_task_mm() achieves that ensuring that mm is still valid, but it might return a task other than the original one. That's why we do this dance with pinning the new task and unpinning the original one. The same dance is performed in __oom_kill_process(). I was contemplating adding a parameter to find_lock_task_mm() to request this unpin/pin be done within that function but then decided to keep it simple for now. Did I address your question or did I misunderstand it? > > -- > Thanks, > > David / dhildenb > > -- > To unsubscribe from this group and stop receiving emails from it, send an email to kernel-team+unsubscribe@android.com. >
On 05.08.21 19:49, Suren Baghdasaryan wrote: > On Thu, Aug 5, 2021 at 10:29 AM David Hildenbrand <david@redhat.com> wrote: >> >> On 05.08.21 19:08, Suren Baghdasaryan wrote: >>> In modern systems it's not unusual to have a system component monitoring >>> memory conditions of the system and tasked with keeping system memory >>> pressure under control. One way to accomplish that is to kill >>> non-essential processes to free up memory for more important ones. >>> Examples of this are Facebook's OOM killer daemon called oomd and >>> Android's low memory killer daemon called lmkd. >>> For such system component it's important to be able to free memory >>> quickly and efficiently. Unfortunately the time process takes to free >>> up its memory after receiving a SIGKILL might vary based on the state >>> of the process (uninterruptible sleep), size and OPP level of the core >>> the process is running. A mechanism to free resources of the target >>> process in a more predictable way would improve system's ability to >>> control its memory pressure. >>> Introduce process_mrelease system call that releases memory of a dying >>> process from the context of the caller. This way the memory is freed in >>> a more controllable way with CPU affinity and priority of the caller. >>> The workload of freeing the memory will also be charged to the caller. >>> The operation is allowed only on a dying process. >>> >>> After previous discussions [1, 2, 3] the decision was made [4] to introduce >>> a dedicated system call to cover this use case. >>> >>> The API is as follows, >>> >>> int process_mrelease(int pidfd, unsigned int flags); >>> >>> DESCRIPTION >>> The process_mrelease() system call is used to free the memory of >>> an exiting process. >>> >>> The pidfd selects the process referred to by the PID file >>> descriptor. >>> (See pidfd_open(2) for further information) >>> >>> The flags argument is reserved for future use; currently, this >>> argument must be specified as 0. >>> >>> RETURN VALUE >>> On success, process_mrelease() returns 0. On error, -1 is >>> returned and errno is set to indicate the error. >>> >>> ERRORS >>> EBADF pidfd is not a valid PID file descriptor. >>> >>> EAGAIN Failed to release part of the address space. >>> >>> EINTR The call was interrupted by a signal; see signal(7). >>> >>> EINVAL flags is not 0. >>> >>> EINVAL The memory of the task cannot be released because the >>> process is not exiting, the address space is shared >>> with another live process or there is a core dump in >>> progress. >>> >>> ENOSYS This system call is not supported, for example, without >>> MMU support built into Linux. >>> >>> ESRCH The target process does not exist (i.e., it has terminated >>> and been waited on). >>> >>> [1] https://lore.kernel.org/lkml/20190411014353.113252-3-surenb@google.com/ >>> [2] https://lore.kernel.org/linux-api/20201113173448.1863419-1-surenb@google.com/ >>> [3] https://lore.kernel.org/linux-api/20201124053943.1684874-3-surenb@google.com/ >>> [4] https://lore.kernel.org/linux-api/20201223075712.GA4719@lst.de/ >>> >>> Signed-off-by: Suren Baghdasaryan <surenb@google.com> >>> --- >>> changes in v7: >>> - Fixed pidfd_open misspelling, per Andrew Morton >>> - Fixed wrong task pinning after find_lock_task_mm() issue, per Michal Hocko >>> - Moved MMF_OOM_SKIP check before task_will_free_mem(), per Michal Hocko >>> >>> mm/oom_kill.c | 73 +++++++++++++++++++++++++++++++++++++++++++++++++++ >>> 1 file changed, 73 insertions(+) >>> >>> diff --git a/mm/oom_kill.c b/mm/oom_kill.c >>> index c729a4c4a1ac..a4d917b43c73 100644 >>> --- a/mm/oom_kill.c >>> +++ b/mm/oom_kill.c >>> @@ -28,6 +28,7 @@ >>> #include <linux/sched/task.h> >>> #include <linux/sched/debug.h> >>> #include <linux/swap.h> >>> +#include <linux/syscalls.h> >>> #include <linux/timex.h> >>> #include <linux/jiffies.h> >>> #include <linux/cpuset.h> >>> @@ -1141,3 +1142,75 @@ void pagefault_out_of_memory(void) >>> out_of_memory(&oc); >>> mutex_unlock(&oom_lock); >>> } >>> + >>> +SYSCALL_DEFINE2(process_mrelease, int, pidfd, unsigned int, flags) >>> +{ >>> +#ifdef CONFIG_MMU >>> + struct mm_struct *mm = NULL; >>> + struct task_struct *task; >>> + struct task_struct *p; >>> + unsigned int f_flags; >>> + struct pid *pid; >>> + long ret = 0; >>> + >>> + if (flags) >>> + return -EINVAL; >>> + >>> + pid = pidfd_get_pid(pidfd, &f_flags); >>> + if (IS_ERR(pid)) >>> + return PTR_ERR(pid); >>> + >>> + task = get_pid_task(pid, PIDTYPE_PID); >>> + if (!task) { >>> + ret = -ESRCH; >>> + goto put_pid; >>> + } >>> + >>> + /* >>> + * If the task is dying and in the process of releasing its memory >>> + * then get its mm. >>> + */ >>> + p = find_lock_task_mm(task); >>> + if (!p) { >>> + ret = -ESRCH; >>> + goto put_pid; >>> + } >>> + if (task != p) { >>> + get_task_struct(p); >> >> >> Wouldn't we want to obtain the mm from p ? I thought that was the whole >> exercise of going via find_lock_task_mm(). > > Yes, that's what we do after checking task_will_free_mem(). > task_will_free_mem() requires us to hold task_lock and > find_lock_task_mm() achieves that ensuring that mm is still valid, but > it might return a task other than the original one. That's why we do > this dance with pinning the new task and unpinning the original one. > The same dance is performed in __oom_kill_process(). I was > contemplating adding a parameter to find_lock_task_mm() to request > this unpin/pin be done within that function but then decided to keep > it simple for now. > Did I address your question or did I misunderstand it? Excuse my tired eyes, I missed the "task = p;" Feel free to carry my ack along, even if there are minor changes.
On Thu, Aug 5, 2021 at 10:50 AM Suren Baghdasaryan <surenb@google.com> wrote: > > On Thu, Aug 5, 2021 at 10:29 AM David Hildenbrand <david@redhat.com> wrote: > > > > On 05.08.21 19:08, Suren Baghdasaryan wrote: > > > In modern systems it's not unusual to have a system component monitoring > > > memory conditions of the system and tasked with keeping system memory > > > pressure under control. One way to accomplish that is to kill > > > non-essential processes to free up memory for more important ones. > > > Examples of this are Facebook's OOM killer daemon called oomd and > > > Android's low memory killer daemon called lmkd. > > > For such system component it's important to be able to free memory > > > quickly and efficiently. Unfortunately the time process takes to free > > > up its memory after receiving a SIGKILL might vary based on the state > > > of the process (uninterruptible sleep), size and OPP level of the core > > > the process is running. A mechanism to free resources of the target > > > process in a more predictable way would improve system's ability to > > > control its memory pressure. > > > Introduce process_mrelease system call that releases memory of a dying > > > process from the context of the caller. This way the memory is freed in > > > a more controllable way with CPU affinity and priority of the caller. > > > The workload of freeing the memory will also be charged to the caller. > > > The operation is allowed only on a dying process. > > > > > > After previous discussions [1, 2, 3] the decision was made [4] to introduce > > > a dedicated system call to cover this use case. > > > > > > The API is as follows, > > > > > > int process_mrelease(int pidfd, unsigned int flags); > > > > > > DESCRIPTION > > > The process_mrelease() system call is used to free the memory of > > > an exiting process. > > > > > > The pidfd selects the process referred to by the PID file > > > descriptor. > > > (See pidfd_open(2) for further information) > > > > > > The flags argument is reserved for future use; currently, this > > > argument must be specified as 0. > > > > > > RETURN VALUE > > > On success, process_mrelease() returns 0. On error, -1 is > > > returned and errno is set to indicate the error. > > > > > > ERRORS > > > EBADF pidfd is not a valid PID file descriptor. > > > > > > EAGAIN Failed to release part of the address space. > > > > > > EINTR The call was interrupted by a signal; see signal(7). > > > > > > EINVAL flags is not 0. > > > > > > EINVAL The memory of the task cannot be released because the > > > process is not exiting, the address space is shared > > > with another live process or there is a core dump in > > > progress. > > > > > > ENOSYS This system call is not supported, for example, without > > > MMU support built into Linux. > > > > > > ESRCH The target process does not exist (i.e., it has terminated > > > and been waited on). > > > > > > [1] https://lore.kernel.org/lkml/20190411014353.113252-3-surenb@google.com/ > > > [2] https://lore.kernel.org/linux-api/20201113173448.1863419-1-surenb@google.com/ > > > [3] https://lore.kernel.org/linux-api/20201124053943.1684874-3-surenb@google.com/ > > > [4] https://lore.kernel.org/linux-api/20201223075712.GA4719@lst.de/ > > > > > > Signed-off-by: Suren Baghdasaryan <surenb@google.com> > > > --- > > > changes in v7: > > > - Fixed pidfd_open misspelling, per Andrew Morton > > > - Fixed wrong task pinning after find_lock_task_mm() issue, per Michal Hocko > > > - Moved MMF_OOM_SKIP check before task_will_free_mem(), per Michal Hocko > > > > > > mm/oom_kill.c | 73 +++++++++++++++++++++++++++++++++++++++++++++++++++ > > > 1 file changed, 73 insertions(+) > > > > > > diff --git a/mm/oom_kill.c b/mm/oom_kill.c > > > index c729a4c4a1ac..a4d917b43c73 100644 > > > --- a/mm/oom_kill.c > > > +++ b/mm/oom_kill.c > > > @@ -28,6 +28,7 @@ > > > #include <linux/sched/task.h> > > > #include <linux/sched/debug.h> > > > #include <linux/swap.h> > > > +#include <linux/syscalls.h> > > > #include <linux/timex.h> > > > #include <linux/jiffies.h> > > > #include <linux/cpuset.h> > > > @@ -1141,3 +1142,75 @@ void pagefault_out_of_memory(void) > > > out_of_memory(&oc); > > > mutex_unlock(&oom_lock); > > > } > > > + > > > +SYSCALL_DEFINE2(process_mrelease, int, pidfd, unsigned int, flags) > > > +{ > > > +#ifdef CONFIG_MMU > > > + struct mm_struct *mm = NULL; > > > + struct task_struct *task; > > > + struct task_struct *p; > > > + unsigned int f_flags; > > > + struct pid *pid; > > > + long ret = 0; > > > + > > > + if (flags) > > > + return -EINVAL; > > > + > > > + pid = pidfd_get_pid(pidfd, &f_flags); > > > + if (IS_ERR(pid)) > > > + return PTR_ERR(pid); > > > + > > > + task = get_pid_task(pid, PIDTYPE_PID); > > > + if (!task) { > > > + ret = -ESRCH; > > > + goto put_pid; > > > + } > > > + > > > + /* > > > + * If the task is dying and in the process of releasing its memory > > > + * then get its mm. > > > + */ > > > + p = find_lock_task_mm(task); > > > + if (!p) { > > > + ret = -ESRCH; > > > + goto put_pid; > > > + } > > > + if (task != p) { > > > + get_task_struct(p); > > > > > > Wouldn't we want to obtain the mm from p ? I thought that was the whole > > exercise of going via find_lock_task_mm(). > > Yes, that's what we do after checking task_will_free_mem(). > task_will_free_mem() requires us to hold task_lock and > find_lock_task_mm() achieves that ensuring that mm is still valid, but > it might return a task other than the original one. That's why we do > this dance with pinning the new task and unpinning the original one. > The same dance is performed in __oom_kill_process(). I was > contemplating adding a parameter to find_lock_task_mm() to request > this unpin/pin be done within that function but then decided to keep > it simple for now. > Did I address your question or did I misunderstand it? > One question I have is why mmget() and not mmgrab()? I see mmgrab() in oom_kill.c.
On Thu, Aug 5, 2021 at 10:56 AM Shakeel Butt <shakeelb@google.com> wrote: > > On Thu, Aug 5, 2021 at 10:50 AM Suren Baghdasaryan <surenb@google.com> wrote: > > > > On Thu, Aug 5, 2021 at 10:29 AM David Hildenbrand <david@redhat.com> wrote: > > > > > > On 05.08.21 19:08, Suren Baghdasaryan wrote: > > > > In modern systems it's not unusual to have a system component monitoring > > > > memory conditions of the system and tasked with keeping system memory > > > > pressure under control. One way to accomplish that is to kill > > > > non-essential processes to free up memory for more important ones. > > > > Examples of this are Facebook's OOM killer daemon called oomd and > > > > Android's low memory killer daemon called lmkd. > > > > For such system component it's important to be able to free memory > > > > quickly and efficiently. Unfortunately the time process takes to free > > > > up its memory after receiving a SIGKILL might vary based on the state > > > > of the process (uninterruptible sleep), size and OPP level of the core > > > > the process is running. A mechanism to free resources of the target > > > > process in a more predictable way would improve system's ability to > > > > control its memory pressure. > > > > Introduce process_mrelease system call that releases memory of a dying > > > > process from the context of the caller. This way the memory is freed in > > > > a more controllable way with CPU affinity and priority of the caller. > > > > The workload of freeing the memory will also be charged to the caller. > > > > The operation is allowed only on a dying process. > > > > > > > > After previous discussions [1, 2, 3] the decision was made [4] to introduce > > > > a dedicated system call to cover this use case. > > > > > > > > The API is as follows, > > > > > > > > int process_mrelease(int pidfd, unsigned int flags); > > > > > > > > DESCRIPTION > > > > The process_mrelease() system call is used to free the memory of > > > > an exiting process. > > > > > > > > The pidfd selects the process referred to by the PID file > > > > descriptor. > > > > (See pidfd_open(2) for further information) > > > > > > > > The flags argument is reserved for future use; currently, this > > > > argument must be specified as 0. > > > > > > > > RETURN VALUE > > > > On success, process_mrelease() returns 0. On error, -1 is > > > > returned and errno is set to indicate the error. > > > > > > > > ERRORS > > > > EBADF pidfd is not a valid PID file descriptor. > > > > > > > > EAGAIN Failed to release part of the address space. > > > > > > > > EINTR The call was interrupted by a signal; see signal(7). > > > > > > > > EINVAL flags is not 0. > > > > > > > > EINVAL The memory of the task cannot be released because the > > > > process is not exiting, the address space is shared > > > > with another live process or there is a core dump in > > > > progress. > > > > > > > > ENOSYS This system call is not supported, for example, without > > > > MMU support built into Linux. > > > > > > > > ESRCH The target process does not exist (i.e., it has terminated > > > > and been waited on). > > > > > > > > [1] https://lore.kernel.org/lkml/20190411014353.113252-3-surenb@google.com/ > > > > [2] https://lore.kernel.org/linux-api/20201113173448.1863419-1-surenb@google.com/ > > > > [3] https://lore.kernel.org/linux-api/20201124053943.1684874-3-surenb@google.com/ > > > > [4] https://lore.kernel.org/linux-api/20201223075712.GA4719@lst.de/ > > > > > > > > Signed-off-by: Suren Baghdasaryan <surenb@google.com> > > > > --- > > > > changes in v7: > > > > - Fixed pidfd_open misspelling, per Andrew Morton > > > > - Fixed wrong task pinning after find_lock_task_mm() issue, per Michal Hocko > > > > - Moved MMF_OOM_SKIP check before task_will_free_mem(), per Michal Hocko > > > > > > > > mm/oom_kill.c | 73 +++++++++++++++++++++++++++++++++++++++++++++++++++ > > > > 1 file changed, 73 insertions(+) > > > > > > > > diff --git a/mm/oom_kill.c b/mm/oom_kill.c > > > > index c729a4c4a1ac..a4d917b43c73 100644 > > > > --- a/mm/oom_kill.c > > > > +++ b/mm/oom_kill.c > > > > @@ -28,6 +28,7 @@ > > > > #include <linux/sched/task.h> > > > > #include <linux/sched/debug.h> > > > > #include <linux/swap.h> > > > > +#include <linux/syscalls.h> > > > > #include <linux/timex.h> > > > > #include <linux/jiffies.h> > > > > #include <linux/cpuset.h> > > > > @@ -1141,3 +1142,75 @@ void pagefault_out_of_memory(void) > > > > out_of_memory(&oc); > > > > mutex_unlock(&oom_lock); > > > > } > > > > + > > > > +SYSCALL_DEFINE2(process_mrelease, int, pidfd, unsigned int, flags) > > > > +{ > > > > +#ifdef CONFIG_MMU > > > > + struct mm_struct *mm = NULL; > > > > + struct task_struct *task; > > > > + struct task_struct *p; > > > > + unsigned int f_flags; > > > > + struct pid *pid; > > > > + long ret = 0; > > > > + > > > > + if (flags) > > > > + return -EINVAL; > > > > + > > > > + pid = pidfd_get_pid(pidfd, &f_flags); > > > > + if (IS_ERR(pid)) > > > > + return PTR_ERR(pid); > > > > + > > > > + task = get_pid_task(pid, PIDTYPE_PID); > > > > + if (!task) { > > > > + ret = -ESRCH; > > > > + goto put_pid; > > > > + } > > > > + > > > > + /* > > > > + * If the task is dying and in the process of releasing its memory > > > > + * then get its mm. > > > > + */ > > > > + p = find_lock_task_mm(task); > > > > + if (!p) { > > > > + ret = -ESRCH; > > > > + goto put_pid; > > > > + } > > > > + if (task != p) { > > > > + get_task_struct(p); > > > > > > > > > Wouldn't we want to obtain the mm from p ? I thought that was the whole > > > exercise of going via find_lock_task_mm(). > > > > Yes, that's what we do after checking task_will_free_mem(). > > task_will_free_mem() requires us to hold task_lock and > > find_lock_task_mm() achieves that ensuring that mm is still valid, but > > it might return a task other than the original one. That's why we do > > this dance with pinning the new task and unpinning the original one. > > The same dance is performed in __oom_kill_process(). I was > > contemplating adding a parameter to find_lock_task_mm() to request > > this unpin/pin be done within that function but then decided to keep > > it simple for now. > > Did I address your question or did I misunderstand it? > > > > One question I have is why mmget() and not mmgrab()? I see mmgrab() in > oom_kill.c. You are likely right here. The caller's context probably can't be considered a "real user" when reaping the mm. However, we take an mmap_lock shortly after, so not sure if in practice there is much difference. Michal, WDYT?
On Thu 05-08-21 10:08:58, Suren Baghdasaryan wrote: [...] > + /* > + * If the task is dying and in the process of releasing its memory > + * then get its mm. > + */ > + p = find_lock_task_mm(task); > + if (!p) { > + ret = -ESRCH; > + goto put_pid; > + } > + if (task != p) { > + get_task_struct(p); > + put_task_struct(task); > + task = p; > + } Why do you need to take a reference to the p here? You are under task_lock so this will not go away and you only need p to get your mm. > + > + /* If the work has been done already, just exit with success */ > + if (test_bit(MMF_OOM_SKIP, &task->mm->flags)) > + goto put_task; You want to release the task_lock > + > + if (task_will_free_mem(task) && (task->flags & PF_KTHREAD) == 0) { you want task_will_free_mem(p) and what is the point of the PF_KTHREAD check? > + mm = task->mm; > + mmget(mm); All you need is to make sure mm will not get released under your feet once task_lock is released so mmgrab is the right thing to do here. The address space can be torn down in parallel and that is OK and desirable. I think you really want something like this: if (flags) return -EINVAL; pid = pidfd_get_pid(fd, &f_flags); if (IS_ERR(pid)) return PTR_ERR(pid); task = get_pid_task(pid, PIDTYPE_PID); if (!task) { ret = -ESRCH; goto put_pid; } /* * Make sure to chose a thread which still has a reference to mm * during the group exit */ p = find_lock_task_mm(task); if (!p) { ret = -ESRCH; goto put_task; } mm = task->mm; mmgrab(mm); reap = true; /* If the work has been done already, just exit with success */ if (test_bit(MMF_OOM_SKIP, &mm->flags)) { reap = false; } else if (!task_will_free_mem(p)) { reap = false; ret = -EINVAL; } task_unlock(p); if (!reap) goto dropmm;; /* Do the work*/ dropmm: mmdrop(mm); put_task: put_task(task); put_pid: put_pid(pid); return ret;
On Thu 05-08-21 11:37:06, Suren Baghdasaryan wrote: [...] > > One question I have is why mmget() and not mmgrab()? I see mmgrab() in > > oom_kill.c. > > You are likely right here. The caller's context probably can't be > considered a "real user" when reaping the mm. However, we take an > mmap_lock shortly after, so not sure if in practice there is much > difference. > Michal, WDYT? As explained in other response. mmget is to pin address space to not go away. You do not need that for this purpose. All you need is to pin mm to not go away. Address space can be unmapped concurrently.
On Thu, Aug 5, 2021 at 11:40 PM Michal Hocko <mhocko@suse.com> wrote: > [...] > I think you really want something like this: > > if (flags) > return -EINVAL; > > pid = pidfd_get_pid(fd, &f_flags); > if (IS_ERR(pid)) > return PTR_ERR(pid); > task = get_pid_task(pid, PIDTYPE_PID); > if (!task) { > ret = -ESRCH; > goto put_pid; > } > > /* > * Make sure to chose a thread which still has a reference to mm > * during the group exit > */ > p = find_lock_task_mm(task); > if (!p) { > ret = -ESRCH; > goto put_task; > } > > mm = task->mm; mm = p->mm; > mmgrab(mm); > reap = true; > /* If the work has been done already, just exit with success */ > if (test_bit(MMF_OOM_SKIP, &mm->flags)) { > reap = false; > } else if (!task_will_free_mem(p)) { > reap = false; > ret = -EINVAL; > } > task_unlock(p); > > if (!reap) > goto dropmm;; > > /* Do the work*/ > > > dropmm: > mmdrop(mm); > put_task: > put_task(task); > put_pid: > put_pid(pid); > > return ret; > > -- > Michal Hocko > SUSE Labs
On Fri 06-08-21 02:23:17, Shakeel Butt wrote: > On Thu, Aug 5, 2021 at 11:40 PM Michal Hocko <mhocko@suse.com> wrote: > > > [...] > > I think you really want something like this: > > > > if (flags) > > return -EINVAL; > > > > pid = pidfd_get_pid(fd, &f_flags); > > if (IS_ERR(pid)) > > return PTR_ERR(pid); > > task = get_pid_task(pid, PIDTYPE_PID); > > if (!task) { > > ret = -ESRCH; > > goto put_pid; > > } > > > > /* > > * Make sure to chose a thread which still has a reference to mm > > * during the group exit > > */ > > p = find_lock_task_mm(task); > > if (!p) { > > ret = -ESRCH; > > goto put_task; > > } > > > > mm = task->mm; > > mm = p->mm; right. Thanks!
On Thu, Aug 5, 2021 at 11:40 PM Michal Hocko <mhocko@suse.com> wrote: > > On Thu 05-08-21 10:08:58, Suren Baghdasaryan wrote: > [...] > > + /* > > + * If the task is dying and in the process of releasing its memory > > + * then get its mm. > > + */ > > + p = find_lock_task_mm(task); > > + if (!p) { > > + ret = -ESRCH; > > + goto put_pid; > > + } > > + if (task != p) { > > + get_task_struct(p); > > + put_task_struct(task); > > + task = p; > > + } > > Why do you need to take a reference to the p here? You are under > task_lock so this will not go away and you only need p to get your mm. True. > > > + > > + /* If the work has been done already, just exit with success */ > > + if (test_bit(MMF_OOM_SKIP, &task->mm->flags)) > > + goto put_task; > > You want to release the task_lock Missed it again :( > > > + > > + if (task_will_free_mem(task) && (task->flags & PF_KTHREAD) == 0) { > > you want task_will_free_mem(p) and what is the point of the PF_KTHREAD > check? Yeah, looks like task_will_free_mem() covers that case already. > > > + mm = task->mm; > > + mmget(mm); > > All you need is to make sure mm will not get released under your feet > once task_lock is released so mmgrab is the right thing to do here. The > address space can be torn down in parallel and that is OK and desirable. > > I think you really want something like this: > > if (flags) > return -EINVAL; > > pid = pidfd_get_pid(fd, &f_flags); > if (IS_ERR(pid)) > return PTR_ERR(pid); > task = get_pid_task(pid, PIDTYPE_PID); > if (!task) { > ret = -ESRCH; > goto put_pid; > } > > /* > * Make sure to chose a thread which still has a reference to mm > * during the group exit > */ > p = find_lock_task_mm(task); > if (!p) { > ret = -ESRCH; > goto put_task; > } > > mm = task->mm; > mmgrab(mm); > reap = true; > /* If the work has been done already, just exit with success */ > if (test_bit(MMF_OOM_SKIP, &mm->flags)) { > reap = false; > } else if (!task_will_free_mem(p)) { > reap = false; > ret = -EINVAL; > } > task_unlock(p); > > if (!reap) > goto dropmm;; > > /* Do the work*/ > > > dropmm: > mmdrop(mm); > put_task: > put_task(task); > put_pid: > put_pid(pid); > > return ret; > This is indeed simpler to follow. I'll adopt your version. Thanks! > -- > Michal Hocko > SUSE Labs
On Fri, Aug 6, 2021 at 9:07 AM Suren Baghdasaryan <surenb@google.com> wrote: > > On Thu, Aug 5, 2021 at 11:40 PM Michal Hocko <mhocko@suse.com> wrote: > > > > On Thu 05-08-21 10:08:58, Suren Baghdasaryan wrote: > > [...] > > > + /* > > > + * If the task is dying and in the process of releasing its memory > > > + * then get its mm. > > > + */ > > > + p = find_lock_task_mm(task); > > > + if (!p) { > > > + ret = -ESRCH; > > > + goto put_pid; > > > + } > > > + if (task != p) { > > > + get_task_struct(p); > > > + put_task_struct(task); > > > + task = p; > > > + } > > > > Why do you need to take a reference to the p here? You are under > > task_lock so this will not go away and you only need p to get your mm. > > True. > > > > > > + > > > + /* If the work has been done already, just exit with success */ > > > + if (test_bit(MMF_OOM_SKIP, &task->mm->flags)) > > > + goto put_task; > > > > You want to release the task_lock > > Missed it again :( > > > > > > + > > > + if (task_will_free_mem(task) && (task->flags & PF_KTHREAD) == 0) { > > > > you want task_will_free_mem(p) and what is the point of the PF_KTHREAD > > check? > > Yeah, looks like task_will_free_mem() covers that case already. > > > > > > + mm = task->mm; > > > + mmget(mm); > > > > All you need is to make sure mm will not get released under your feet > > once task_lock is released so mmgrab is the right thing to do here. The > > address space can be torn down in parallel and that is OK and desirable. > > > > I think you really want something like this: > > > > if (flags) > > return -EINVAL; > > > > pid = pidfd_get_pid(fd, &f_flags); > > if (IS_ERR(pid)) > > return PTR_ERR(pid); > > task = get_pid_task(pid, PIDTYPE_PID); > > if (!task) { > > ret = -ESRCH; > > goto put_pid; > > } > > > > /* > > * Make sure to chose a thread which still has a reference to mm > > * during the group exit > > */ > > p = find_lock_task_mm(task); > > if (!p) { > > ret = -ESRCH; > > goto put_task; > > } > > > > mm = task->mm; > > mmgrab(mm); > > reap = true; > > /* If the work has been done already, just exit with success */ > > if (test_bit(MMF_OOM_SKIP, &mm->flags)) { > > reap = false; > > } else if (!task_will_free_mem(p)) { > > reap = false; > > ret = -EINVAL; > > } > > task_unlock(p); > > > > if (!reap) > > goto dropmm;; > > > > /* Do the work*/ > > > > > > dropmm: > > mmdrop(mm); > > put_task: > > put_task(task); > > put_pid: > > put_pid(pid); > > > > return ret; > > > > This is indeed simpler to follow. I'll adopt your version. Thanks! v8 is posted at https://lore.kernel.org/patchwork/patch/1473697/ Testing shows performance improvement from replacing mmget with mmgrab. > > > -- > > Michal Hocko > > SUSE Labs
diff --git a/mm/oom_kill.c b/mm/oom_kill.c index c729a4c4a1ac..a4d917b43c73 100644 --- a/mm/oom_kill.c +++ b/mm/oom_kill.c @@ -28,6 +28,7 @@ #include <linux/sched/task.h> #include <linux/sched/debug.h> #include <linux/swap.h> +#include <linux/syscalls.h> #include <linux/timex.h> #include <linux/jiffies.h> #include <linux/cpuset.h> @@ -1141,3 +1142,75 @@ void pagefault_out_of_memory(void) out_of_memory(&oc); mutex_unlock(&oom_lock); } + +SYSCALL_DEFINE2(process_mrelease, int, pidfd, unsigned int, flags) +{ +#ifdef CONFIG_MMU + struct mm_struct *mm = NULL; + struct task_struct *task; + struct task_struct *p; + unsigned int f_flags; + struct pid *pid; + long ret = 0; + + if (flags) + return -EINVAL; + + pid = pidfd_get_pid(pidfd, &f_flags); + if (IS_ERR(pid)) + return PTR_ERR(pid); + + task = get_pid_task(pid, PIDTYPE_PID); + if (!task) { + ret = -ESRCH; + goto put_pid; + } + + /* + * If the task is dying and in the process of releasing its memory + * then get its mm. + */ + p = find_lock_task_mm(task); + if (!p) { + ret = -ESRCH; + goto put_pid; + } + if (task != p) { + get_task_struct(p); + put_task_struct(task); + task = p; + } + + /* If the work has been done already, just exit with success */ + if (test_bit(MMF_OOM_SKIP, &task->mm->flags)) + goto put_task; + + if (task_will_free_mem(task) && (task->flags & PF_KTHREAD) == 0) { + mm = task->mm; + mmget(mm); + } + task_unlock(task); + if (!mm) { + ret = -EINVAL; + goto put_task; + } + + if (mmap_read_lock_killable(mm)) { + ret = -EINTR; + goto put_mm; + } + if (!__oom_reap_task_mm(mm)) + ret = -EAGAIN; + mmap_read_unlock(mm); + +put_mm: + mmput(mm); +put_task: + put_task_struct(task); +put_pid: + put_pid(pid); + return ret; +#else + return -ENOSYS; +#endif /* CONFIG_MMU */ +}
In modern systems it's not unusual to have a system component monitoring memory conditions of the system and tasked with keeping system memory pressure under control. One way to accomplish that is to kill non-essential processes to free up memory for more important ones. Examples of this are Facebook's OOM killer daemon called oomd and Android's low memory killer daemon called lmkd. For such system component it's important to be able to free memory quickly and efficiently. Unfortunately the time process takes to free up its memory after receiving a SIGKILL might vary based on the state of the process (uninterruptible sleep), size and OPP level of the core the process is running. A mechanism to free resources of the target process in a more predictable way would improve system's ability to control its memory pressure. Introduce process_mrelease system call that releases memory of a dying process from the context of the caller. This way the memory is freed in a more controllable way with CPU affinity and priority of the caller. The workload of freeing the memory will also be charged to the caller. The operation is allowed only on a dying process. After previous discussions [1, 2, 3] the decision was made [4] to introduce a dedicated system call to cover this use case. The API is as follows, int process_mrelease(int pidfd, unsigned int flags); DESCRIPTION The process_mrelease() system call is used to free the memory of an exiting process. The pidfd selects the process referred to by the PID file descriptor. (See pidfd_open(2) for further information) The flags argument is reserved for future use; currently, this argument must be specified as 0. RETURN VALUE On success, process_mrelease() returns 0. On error, -1 is returned and errno is set to indicate the error. ERRORS EBADF pidfd is not a valid PID file descriptor. EAGAIN Failed to release part of the address space. EINTR The call was interrupted by a signal; see signal(7). EINVAL flags is not 0. EINVAL The memory of the task cannot be released because the process is not exiting, the address space is shared with another live process or there is a core dump in progress. ENOSYS This system call is not supported, for example, without MMU support built into Linux. ESRCH The target process does not exist (i.e., it has terminated and been waited on). [1] https://lore.kernel.org/lkml/20190411014353.113252-3-surenb@google.com/ [2] https://lore.kernel.org/linux-api/20201113173448.1863419-1-surenb@google.com/ [3] https://lore.kernel.org/linux-api/20201124053943.1684874-3-surenb@google.com/ [4] https://lore.kernel.org/linux-api/20201223075712.GA4719@lst.de/ Signed-off-by: Suren Baghdasaryan <surenb@google.com> --- changes in v7: - Fixed pidfd_open misspelling, per Andrew Morton - Fixed wrong task pinning after find_lock_task_mm() issue, per Michal Hocko - Moved MMF_OOM_SKIP check before task_will_free_mem(), per Michal Hocko mm/oom_kill.c | 73 +++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 73 insertions(+)