[1/3] kthread: add a mechanism to store cgroup info

Commit Message

Shaohua Li Sept. 7, 2017, 2 a.m. UTC

From: Shaohua Li <[e-mail]>

kthread usually runs jobs on behalf of other threads. The jobs should be
charged to cgroup of original threads. But the jobs run in a kthread,
where we lose the cgroup context of original threads. The patch adds a
machanism to record cgroup info of original threads in kthread context.
Later we can retrieve the cgroup info and attach the cgroup info to jobs.

Since this mechanism is only required by kthread, we store the cgroup
info in kthread data instead of generic task_struct.

Signed-off-by: Shaohua Li <[e-mail]>
---
 include/linux/kthread.h | 13 +++++++++++
 include/linux/sched.h   |  1 +
 kernel/kthread.c        | 57 ++++++++++++++++++++++++++++++++++++++++++++++++-
 3 files changed, 70 insertions(+), 1 deletion(-)

Comments

Tejun Heo Sept. 8, 2017, 2:35 p.m. UTC | #1

Hello,

On Wed, Sep 06, 2017 at 07:00:51PM -0700, Shaohua Li wrote:
> +#ifdef CONFIG_CGROUPS
> +void kthread_set_orig_css(struct cgroup_subsys_state *css);
> +struct cgroup_subsys_state *kthread_get_orig_css(void);
> +void kthread_reset_orig_css(void);

* It's a bit weird to associate a kthread with a css without being
  specific.  If what's needed is a generic association (this kthread
  is temporarily servicing this cgroup), it should be associated with
  the cgroup.  But, I think it'd be better to make it specific instead
  - ie. kthread_set_io_css().

* Do we need the reset function to be separate?  Can't we just clear
  it when the set function is called with NULL css?

* For the accessor, can we do sth like kthread_orig_css() (or
  kthread_io_css())?  "get" is overloaded between set/get and get/put,
  so it can get confusing.

> diff --git a/include/linux/sched.h b/include/linux/sched.h
> index c05ac5f..ab2295d 100644
> --- a/include/linux/sched.h
> +++ b/include/linux/sched.h
> @@ -1276,6 +1276,7 @@ extern struct pid *cad_pid;
>  #define PF_SWAPWRITE		0x00800000	/* Allowed to write to swap */
>  #define PF_NO_SETAFFINITY	0x04000000	/* Userland is not allowed to meddle with cpus_allowed */
>  #define PF_MCE_EARLY		0x08000000      /* Early kill for mce process policy */
> +#define PF_KTHREAD_HAS_CSS	0x10000000	/* kthread has css info attached */

Do we need a separate flag for this?  kthreads already have PF_KTHREAD
set.  I'm not sure why we'd need another flag.  Once we know it's a
kthread, we can just access its css pointer field, right?

Thanks.

Tejun Heo Sept. 8, 2017, 2:38 p.m. UTC | #2

On Fri, Sep 08, 2017 at 07:35:37AM -0700, Tejun Heo wrote:
> * It's a bit weird to associate a kthread with a css without being
>   specific.  If what's needed is a generic association (this kthread
>   is temporarily servicing this cgroup), it should be associated with
>   the cgroup.  But, I think it'd be better to make it specific instead
>   - ie. kthread_set_io_css().

kthread[_set]_blkcg_css() probably is a more consistent name.

Thanks.

Tejun Heo Sept. 8, 2017, 2:40 p.m. UTC | #3

On Fri, Sep 08, 2017 at 07:38:46AM -0700, Tejun Heo wrote:
> On Fri, Sep 08, 2017 at 07:35:37AM -0700, Tejun Heo wrote:
> > * It's a bit weird to associate a kthread with a css without being
> >   specific.  If what's needed is a generic association (this kthread
> >   is temporarily servicing this cgroup), it should be associated with
> >   the cgroup.  But, I think it'd be better to make it specific instead
> >   - ie. kthread_set_io_css().
> 
> kthread[_set]_blkcg_css() probably is a more consistent name.

Sorry, I meant kthread[_associate]_blkcg() so that it's consistent
with bio[_associate]_blkcg().

Thanks.

Shaohua Li Sept. 8, 2017, 4:51 p.m. UTC | #4

On Fri, Sep 08, 2017 at 07:35:37AM -0700, Tejun Heo wrote:
> Hello,
> 
> On Wed, Sep 06, 2017 at 07:00:51PM -0700, Shaohua Li wrote:
> > +#ifdef CONFIG_CGROUPS
> > +void kthread_set_orig_css(struct cgroup_subsys_state *css);
> > +struct cgroup_subsys_state *kthread_get_orig_css(void);
> > +void kthread_reset_orig_css(void);
> 
> * It's a bit weird to associate a kthread with a css without being
>   specific.  If what's needed is a generic association (this kthread
>   is temporarily servicing this cgroup), it should be associated with
>   the cgroup.  But, I think it'd be better to make it specific instead
>   - ie. kthread_set_io_css().
> 
> * Do we need the reset function to be separate?  Can't we just clear
>   it when the set function is called with NULL css?
> 
> * For the accessor, can we do sth like kthread_orig_css() (or
>   kthread_io_css())?  "get" is overloaded between set/get and get/put,
>   so it can get confusing.
> 
> > diff --git a/include/linux/sched.h b/include/linux/sched.h
> > index c05ac5f..ab2295d 100644
> > --- a/include/linux/sched.h
> > +++ b/include/linux/sched.h
> > @@ -1276,6 +1276,7 @@ extern struct pid *cad_pid;
> >  #define PF_SWAPWRITE		0x00800000	/* Allowed to write to swap */
> >  #define PF_NO_SETAFFINITY	0x04000000	/* Userland is not allowed to meddle with cpus_allowed */
> >  #define PF_MCE_EARLY		0x08000000      /* Early kill for mce process policy */
> > +#define PF_KTHREAD_HAS_CSS	0x10000000	/* kthread has css info attached */
> 
> Do we need a separate flag for this?  kthreads already have PF_KTHREAD
> set.  I'm not sure why we'd need another flag.  Once we know it's a
> kthread, we can just access its css pointer field, right?

Ok, all suggestions are good, will update.

Thanks,
Shaohua

Patch

diff --git a/include/linux/kthread.h b/include/linux/kthread.h
index 82e197e..3eb24d1 100644
--- a/include/linux/kthread.h
+++ b/include/linux/kthread.h
@@ -3,6 +3,7 @@ 
 /* Simple interface for creating and stopping kernel threads without mess. */
 #include <linux/err.h>
 #include <linux/sched.h>
+#include <linux/cgroup.h>
 
 __printf(4, 5)
 struct task_struct *kthread_create_on_node(int (*threadfn)(void *data),
@@ -198,4 +199,16 @@  bool kthread_cancel_delayed_work_sync(struct kthread_delayed_work *work);
 
 void kthread_destroy_worker(struct kthread_worker *worker);
 
+#ifdef CONFIG_CGROUPS
+void kthread_set_orig_css(struct cgroup_subsys_state *css);
+struct cgroup_subsys_state *kthread_get_orig_css(void);
+void kthread_reset_orig_css(void);
+#else
+static inline void kthread_set_orig_css(struct cgroup_subsys_state *css) { }
+static inline struct cgroup_subsys_state *kthread_get_orig_css(void)
+{
+	return NULL;
+}
+static inline void kthread_reset_orig_css(void) { }
+#endif
 #endif /* _LINUX_KTHREAD_H */
diff --git a/include/linux/sched.h b/include/linux/sched.h
index c05ac5f..ab2295d 100644
--- a/include/linux/sched.h
+++ b/include/linux/sched.h
@@ -1276,6 +1276,7 @@  extern struct pid *cad_pid;
 #define PF_SWAPWRITE		0x00800000	/* Allowed to write to swap */
 #define PF_NO_SETAFFINITY	0x04000000	/* Userland is not allowed to meddle with cpus_allowed */
 #define PF_MCE_EARLY		0x08000000      /* Early kill for mce process policy */
+#define PF_KTHREAD_HAS_CSS	0x10000000	/* kthread has css info attached */
 #define PF_MUTEX_TESTER		0x20000000	/* Thread belongs to the rt mutex tester */
 #define PF_FREEZER_SKIP		0x40000000	/* Freezer should not count it as freezable */
 #define PF_SUSPEND_TASK		0x80000000      /* This thread called freeze_processes() and should not be frozen */
diff --git a/kernel/kthread.c b/kernel/kthread.c
index 26db528..d084ef3 100644
--- a/kernel/kthread.c
+++ b/kernel/kthread.c
@@ -20,7 +20,6 @@ 
 #include <linux/freezer.h>
 #include <linux/ptrace.h>
 #include <linux/uaccess.h>
-#include <linux/cgroup.h>
 #include <trace/events/sched.h>
 
 static DEFINE_SPINLOCK(kthread_create_lock);
@@ -47,6 +46,7 @@  struct kthread {
 	void *data;
 	struct completion parked;
 	struct completion exited;
+	struct cgroup_subsys_state *orig_css;
 };
 
 enum KTHREAD_BITS {
@@ -1153,3 +1153,58 @@  void kthread_destroy_worker(struct kthread_worker *worker)
 	kfree(worker);
 }
 EXPORT_SYMBOL(kthread_destroy_worker);
+
+#ifdef CONFIG_CGROUPS
+/**
+ * kthread_set_orig_css - set the original css for current thread
+ * @css: the cgroup info
+ *
+ * Current thread must be a kthread. The thread is running jobs on behalf of
+ * other threads. In some cases, we expect the jobs attach cgroup info of
+ * original threads instead of that of current thread. This function stores
+ * original thread's cgroup info in current kthread context for later
+ * retrieval.
+ */
+void kthread_set_orig_css(struct cgroup_subsys_state *css)
+{
+	struct kthread *kthread = to_kthread(current);
+
+	if (css) {
+		css_get(css);
+		kthread->orig_css = css;
+		current->flags |= PF_KTHREAD_HAS_CSS;
+	}
+}
+EXPORT_SYMBOL(kthread_set_orig_css);
+
+/**
+ * kthread_get_orig_css - get the stored original cgroup info
+ *
+ * Current thread must be a kthread.
+ */
+struct cgroup_subsys_state *kthread_get_orig_css(void)
+{
+	if (current->flags & PF_KTHREAD_HAS_CSS)
+		return to_kthread(current)->orig_css;
+	return NULL;
+}
+EXPORT_SYMBOL(kthread_get_orig_css);
+
+/**
+ * kthread_reset_orig_css - clear stored cgroup info
+ *
+ * Current thread must be a kthread.
+ */
+void kthread_reset_orig_css(void)
+{
+	struct kthread *kthread = to_kthread(current);
+	struct cgroup_subsys_state *css;
+
+	css = kthread->orig_css;
+	if (css)
+		css_put(css);
+	kthread->orig_css = NULL;
+	current->flags &= ~PF_KTHREAD_HAS_CSS;
+}
+EXPORT_SYMBOL(kthread_reset_orig_css);
+#endif