xen-blkback: prevent premature module unload
diff mbox series

Message ID 20191210145305.6605-1-pdurrant@amazon.com
State New
Headers show
Series
  • xen-blkback: prevent premature module unload
Related show

Commit Message

Paul Durrant Dec. 10, 2019, 2:53 p.m. UTC
Objects allocated by xen_blkif_alloc come from the 'blkif_cache' kmem
cache. This cache is destoyed when xen-blkif is unloaded so it is
necessary to wait for the deferred free routine used for such objects to
complete. This necessity was missed in commit 14855954f636 "xen-blkback:
allow module to be cleanly unloaded". This patch fixes the problem by
taking/releasing extra module references in xen_blkif_alloc/free()
respectively.

Signed-off-by: Paul Durrant <pdurrant@amazon.com>
---
Cc: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
Cc: "Roger Pau Monné" <roger.pau@citrix.com>
Cc: Jens Axboe <axboe@kernel.dk>
---
 drivers/block/xen-blkback/xenbus.c | 10 ++++++++++
 1 file changed, 10 insertions(+)

Comments

Roger Pau Monne Dec. 11, 2019, 11:28 a.m. UTC | #1
On Tue, Dec 10, 2019 at 02:53:05PM +0000, Paul Durrant wrote:
> Objects allocated by xen_blkif_alloc come from the 'blkif_cache' kmem
> cache. This cache is destoyed when xen-blkif is unloaded so it is
> necessary to wait for the deferred free routine used for such objects to
> complete. This necessity was missed in commit 14855954f636 "xen-blkback:
> allow module to be cleanly unloaded". This patch fixes the problem by
> taking/releasing extra module references in xen_blkif_alloc/free()
> respectively.
> 
> Signed-off-by: Paul Durrant <pdurrant@amazon.com>

Reviewed-by: Roger Pau Monné <roger.pau@citrix.com>

One nit below.

> ---
> Cc: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
> Cc: "Roger Pau Monné" <roger.pau@citrix.com>
> Cc: Jens Axboe <axboe@kernel.dk>
> ---
>  drivers/block/xen-blkback/xenbus.c | 10 ++++++++++
>  1 file changed, 10 insertions(+)
> 
> diff --git a/drivers/block/xen-blkback/xenbus.c b/drivers/block/xen-blkback/xenbus.c
> index e8c5c54e1d26..59d576d27ca7 100644
> --- a/drivers/block/xen-blkback/xenbus.c
> +++ b/drivers/block/xen-blkback/xenbus.c
> @@ -171,6 +171,15 @@ static struct xen_blkif *xen_blkif_alloc(domid_t domid)
>  	blkif->domid = domid;
>  	atomic_set(&blkif->refcnt, 1);
>  	init_completion(&blkif->drain_complete);
> +
> +	/*
> +	 * Because freeing back to the cache may be deferred, it is not
> +	 * safe to unload the module (and hence destroy the cache) until
> +	 * this has completed. To prevent premature unloading, take an
> +	 * extra module reference here and release only when the object
> +	 * has been free back to the cache.
                    ^ freed
> +	 */
> +	__module_get(THIS_MODULE);
>  	INIT_WORK(&blkif->free_work, xen_blkif_deferred_free);
>  
>  	return blkif;
> @@ -320,6 +329,7 @@ static void xen_blkif_free(struct xen_blkif *blkif)
>  
>  	/* Make sure everything is drained before shutting down */
>  	kmem_cache_free(xen_blkif_cachep, blkif);
> +	module_put(THIS_MODULE);
>  }
>  
>  int __init xen_blkif_interface_init(void)
> -- 
> 2.20.1
>
Paul Durrant Dec. 11, 2019, 1:27 p.m. UTC | #2
> -----Original Message-----
> From: Roger Pau Monné <roger.pau@citrix.com>
> Sent: 11 December 2019 11:29
> To: Durrant, Paul <pdurrant@amazon.com>
> Cc: xen-devel@lists.xenproject.org; linux-block@vger.kernel.org; linux-
> kernel@vger.kernel.org; Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>;
> Jens Axboe <axboe@kernel.dk>
> Subject: Re: [PATCH] xen-blkback: prevent premature module unload
> 
> On Tue, Dec 10, 2019 at 02:53:05PM +0000, Paul Durrant wrote:
> > Objects allocated by xen_blkif_alloc come from the 'blkif_cache' kmem
> > cache. This cache is destoyed when xen-blkif is unloaded so it is
> > necessary to wait for the deferred free routine used for such objects to
> > complete. This necessity was missed in commit 14855954f636 "xen-blkback:
> > allow module to be cleanly unloaded". This patch fixes the problem by
> > taking/releasing extra module references in xen_blkif_alloc/free()
> > respectively.
> >
> > Signed-off-by: Paul Durrant <pdurrant@amazon.com>
> 
> Reviewed-by: Roger Pau Monné <roger.pau@citrix.com>
> 
> One nit below.
> 
> > ---
> > Cc: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
> > Cc: "Roger Pau Monné" <roger.pau@citrix.com>
> > Cc: Jens Axboe <axboe@kernel.dk>
> > ---
> >  drivers/block/xen-blkback/xenbus.c | 10 ++++++++++
> >  1 file changed, 10 insertions(+)
> >
> > diff --git a/drivers/block/xen-blkback/xenbus.c b/drivers/block/xen-
> blkback/xenbus.c
> > index e8c5c54e1d26..59d576d27ca7 100644
> > --- a/drivers/block/xen-blkback/xenbus.c
> > +++ b/drivers/block/xen-blkback/xenbus.c
> > @@ -171,6 +171,15 @@ static struct xen_blkif *xen_blkif_alloc(domid_t
> domid)
> >  	blkif->domid = domid;
> >  	atomic_set(&blkif->refcnt, 1);
> >  	init_completion(&blkif->drain_complete);
> > +
> > +	/*
> > +	 * Because freeing back to the cache may be deferred, it is not
> > +	 * safe to unload the module (and hence destroy the cache) until
> > +	 * this has completed. To prevent premature unloading, take an
> > +	 * extra module reference here and release only when the object
> > +	 * has been free back to the cache.
>                     ^ freed

Oh yes. Can this be done on commit, or would you like me to send a v2?

  Paul

> > +	 */
> > +	__module_get(THIS_MODULE);
> >  	INIT_WORK(&blkif->free_work, xen_blkif_deferred_free);
> >
> >  	return blkif;
> > @@ -320,6 +329,7 @@ static void xen_blkif_free(struct xen_blkif *blkif)
> >
> >  	/* Make sure everything is drained before shutting down */
> >  	kmem_cache_free(xen_blkif_cachep, blkif);
> > +	module_put(THIS_MODULE);
> >  }
> >
> >  int __init xen_blkif_interface_init(void)
> > --
> > 2.20.1
> >
Roger Pau Monne Dec. 11, 2019, 1:55 p.m. UTC | #3
On Wed, Dec 11, 2019 at 01:27:42PM +0000, Durrant, Paul wrote:
> > -----Original Message-----
> > From: Roger Pau Monné <roger.pau@citrix.com>
> > Sent: 11 December 2019 11:29
> > To: Durrant, Paul <pdurrant@amazon.com>
> > Cc: xen-devel@lists.xenproject.org; linux-block@vger.kernel.org; linux-
> > kernel@vger.kernel.org; Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>;
> > Jens Axboe <axboe@kernel.dk>
> > Subject: Re: [PATCH] xen-blkback: prevent premature module unload
> > 
> > On Tue, Dec 10, 2019 at 02:53:05PM +0000, Paul Durrant wrote:
> > > Objects allocated by xen_blkif_alloc come from the 'blkif_cache' kmem
> > > cache. This cache is destoyed when xen-blkif is unloaded so it is
> > > necessary to wait for the deferred free routine used for such objects to
> > > complete. This necessity was missed in commit 14855954f636 "xen-blkback:
> > > allow module to be cleanly unloaded". This patch fixes the problem by
> > > taking/releasing extra module references in xen_blkif_alloc/free()
> > > respectively.
> > >
> > > Signed-off-by: Paul Durrant <pdurrant@amazon.com>
> > 
> > Reviewed-by: Roger Pau Monné <roger.pau@citrix.com>
> > 
> > One nit below.
> > 
> > > ---
> > > Cc: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
> > > Cc: "Roger Pau Monné" <roger.pau@citrix.com>
> > > Cc: Jens Axboe <axboe@kernel.dk>
> > > ---
> > >  drivers/block/xen-blkback/xenbus.c | 10 ++++++++++
> > >  1 file changed, 10 insertions(+)
> > >
> > > diff --git a/drivers/block/xen-blkback/xenbus.c b/drivers/block/xen-
> > blkback/xenbus.c
> > > index e8c5c54e1d26..59d576d27ca7 100644
> > > --- a/drivers/block/xen-blkback/xenbus.c
> > > +++ b/drivers/block/xen-blkback/xenbus.c
> > > @@ -171,6 +171,15 @@ static struct xen_blkif *xen_blkif_alloc(domid_t
> > domid)
> > >  	blkif->domid = domid;
> > >  	atomic_set(&blkif->refcnt, 1);
> > >  	init_completion(&blkif->drain_complete);
> > > +
> > > +	/*
> > > +	 * Because freeing back to the cache may be deferred, it is not
> > > +	 * safe to unload the module (and hence destroy the cache) until
> > > +	 * this has completed. To prevent premature unloading, take an
> > > +	 * extra module reference here and release only when the object
> > > +	 * has been free back to the cache.
> >                     ^ freed
> 
> Oh yes. Can this be done on commit, or would you like me to send a v2?

Adjusting on commit would be fine for me, but it's up to Juergen since
he is the one that will pick this up. IIRC the module unload patches
didn't go through the block subsystem.

Thanks, Roger.
Jürgen Groß Dec. 11, 2019, 1:56 p.m. UTC | #4
On 11.12.19 14:55, Roger Pau Monné wrote:
> On Wed, Dec 11, 2019 at 01:27:42PM +0000, Durrant, Paul wrote:
>>> -----Original Message-----
>>> From: Roger Pau Monné <roger.pau@citrix.com>
>>> Sent: 11 December 2019 11:29
>>> To: Durrant, Paul <pdurrant@amazon.com>
>>> Cc: xen-devel@lists.xenproject.org; linux-block@vger.kernel.org; linux-
>>> kernel@vger.kernel.org; Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>;
>>> Jens Axboe <axboe@kernel.dk>
>>> Subject: Re: [PATCH] xen-blkback: prevent premature module unload
>>>
>>> On Tue, Dec 10, 2019 at 02:53:05PM +0000, Paul Durrant wrote:
>>>> Objects allocated by xen_blkif_alloc come from the 'blkif_cache' kmem
>>>> cache. This cache is destoyed when xen-blkif is unloaded so it is
>>>> necessary to wait for the deferred free routine used for such objects to
>>>> complete. This necessity was missed in commit 14855954f636 "xen-blkback:
>>>> allow module to be cleanly unloaded". This patch fixes the problem by
>>>> taking/releasing extra module references in xen_blkif_alloc/free()
>>>> respectively.
>>>>
>>>> Signed-off-by: Paul Durrant <pdurrant@amazon.com>
>>>
>>> Reviewed-by: Roger Pau Monné <roger.pau@citrix.com>
>>>
>>> One nit below.
>>>
>>>> ---
>>>> Cc: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
>>>> Cc: "Roger Pau Monné" <roger.pau@citrix.com>
>>>> Cc: Jens Axboe <axboe@kernel.dk>
>>>> ---
>>>>   drivers/block/xen-blkback/xenbus.c | 10 ++++++++++
>>>>   1 file changed, 10 insertions(+)
>>>>
>>>> diff --git a/drivers/block/xen-blkback/xenbus.c b/drivers/block/xen-
>>> blkback/xenbus.c
>>>> index e8c5c54e1d26..59d576d27ca7 100644
>>>> --- a/drivers/block/xen-blkback/xenbus.c
>>>> +++ b/drivers/block/xen-blkback/xenbus.c
>>>> @@ -171,6 +171,15 @@ static struct xen_blkif *xen_blkif_alloc(domid_t
>>> domid)
>>>>   	blkif->domid = domid;
>>>>   	atomic_set(&blkif->refcnt, 1);
>>>>   	init_completion(&blkif->drain_complete);
>>>> +
>>>> +	/*
>>>> +	 * Because freeing back to the cache may be deferred, it is not
>>>> +	 * safe to unload the module (and hence destroy the cache) until
>>>> +	 * this has completed. To prevent premature unloading, take an
>>>> +	 * extra module reference here and release only when the object
>>>> +	 * has been free back to the cache.
>>>                      ^ freed
>>
>> Oh yes. Can this be done on commit, or would you like me to send a v2?
> 
> Adjusting on commit would be fine for me, but it's up to Juergen since
> he is the one that will pick this up. IIRC the module unload patches
> didn't go through the block subsystem.

Oh, right. Yes, will fix this when committing.


Juergen
Paul Durrant Dec. 11, 2019, 1:57 p.m. UTC | #5
> -----Original Message-----
> From: Roger Pau Monné <roger.pau@citrix.com>
> Sent: 11 December 2019 13:55
> To: Durrant, Paul <pdurrant@amazon.com>; Juergen Gross <jgross@suse.com>
> Cc: xen-devel@lists.xenproject.org; linux-block@vger.kernel.org; linux-
> kernel@vger.kernel.org; Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>;
> Jens Axboe <axboe@kernel.dk>
> Subject: Re: [PATCH] xen-blkback: prevent premature module unload
> 
> On Wed, Dec 11, 2019 at 01:27:42PM +0000, Durrant, Paul wrote:
> > > -----Original Message-----
> > > From: Roger Pau Monné <roger.pau@citrix.com>
> > > Sent: 11 December 2019 11:29
> > > To: Durrant, Paul <pdurrant@amazon.com>
> > > Cc: xen-devel@lists.xenproject.org; linux-block@vger.kernel.org;
> linux-
> > > kernel@vger.kernel.org; Konrad Rzeszutek Wilk
> <konrad.wilk@oracle.com>;
> > > Jens Axboe <axboe@kernel.dk>
> > > Subject: Re: [PATCH] xen-blkback: prevent premature module unload
> > >
> > > On Tue, Dec 10, 2019 at 02:53:05PM +0000, Paul Durrant wrote:
> > > > Objects allocated by xen_blkif_alloc come from the 'blkif_cache'
> kmem
> > > > cache. This cache is destoyed when xen-blkif is unloaded so it is
> > > > necessary to wait for the deferred free routine used for such
> objects to
> > > > complete. This necessity was missed in commit 14855954f636 "xen-
> blkback:
> > > > allow module to be cleanly unloaded". This patch fixes the problem
> by
> > > > taking/releasing extra module references in xen_blkif_alloc/free()
> > > > respectively.
> > > >
> > > > Signed-off-by: Paul Durrant <pdurrant@amazon.com>
> > >
> > > Reviewed-by: Roger Pau Monné <roger.pau@citrix.com>
> > >
> > > One nit below.
> > >
> > > > ---
> > > > Cc: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
> > > > Cc: "Roger Pau Monné" <roger.pau@citrix.com>
> > > > Cc: Jens Axboe <axboe@kernel.dk>
> > > > ---
> > > >  drivers/block/xen-blkback/xenbus.c | 10 ++++++++++
> > > >  1 file changed, 10 insertions(+)
> > > >
> > > > diff --git a/drivers/block/xen-blkback/xenbus.c b/drivers/block/xen-
> > > blkback/xenbus.c
> > > > index e8c5c54e1d26..59d576d27ca7 100644
> > > > --- a/drivers/block/xen-blkback/xenbus.c
> > > > +++ b/drivers/block/xen-blkback/xenbus.c
> > > > @@ -171,6 +171,15 @@ static struct xen_blkif
> *xen_blkif_alloc(domid_t
> > > domid)
> > > >  	blkif->domid = domid;
> > > >  	atomic_set(&blkif->refcnt, 1);
> > > >  	init_completion(&blkif->drain_complete);
> > > > +
> > > > +	/*
> > > > +	 * Because freeing back to the cache may be deferred, it is
> not
> > > > +	 * safe to unload the module (and hence destroy the cache)
> until
> > > > +	 * this has completed. To prevent premature unloading, take an
> > > > +	 * extra module reference here and release only when the
> object
> > > > +	 * has been free back to the cache.
> > >                     ^ freed
> >
> > Oh yes. Can this be done on commit, or would you like me to send a v2?
> 
> Adjusting on commit would be fine for me, but it's up to Juergen since
> he is the one that will pick this up. IIRC the module unload patches
> didn't go through the block subsystem.

True. I forgot manually add Juergen cc list.

  Paul

> 
> Thanks, Roger.
Jürgen Groß Dec. 13, 2019, 9 a.m. UTC | #6
On 10.12.19 15:53, Paul Durrant wrote:
> Objects allocated by xen_blkif_alloc come from the 'blkif_cache' kmem
> cache. This cache is destoyed when xen-blkif is unloaded so it is
> necessary to wait for the deferred free routine used for such objects to
> complete. This necessity was missed in commit 14855954f636 "xen-blkback:
> allow module to be cleanly unloaded". This patch fixes the problem by
> taking/releasing extra module references in xen_blkif_alloc/free()
> respectively.
> 
> Signed-off-by: Paul Durrant <pdurrant@amazon.com>

Pushed to xen/tip.git for-linus-5.5b


Juergen

Patch
diff mbox series

diff --git a/drivers/block/xen-blkback/xenbus.c b/drivers/block/xen-blkback/xenbus.c
index e8c5c54e1d26..59d576d27ca7 100644
--- a/drivers/block/xen-blkback/xenbus.c
+++ b/drivers/block/xen-blkback/xenbus.c
@@ -171,6 +171,15 @@  static struct xen_blkif *xen_blkif_alloc(domid_t domid)
 	blkif->domid = domid;
 	atomic_set(&blkif->refcnt, 1);
 	init_completion(&blkif->drain_complete);
+
+	/*
+	 * Because freeing back to the cache may be deferred, it is not
+	 * safe to unload the module (and hence destroy the cache) until
+	 * this has completed. To prevent premature unloading, take an
+	 * extra module reference here and release only when the object
+	 * has been free back to the cache.
+	 */
+	__module_get(THIS_MODULE);
 	INIT_WORK(&blkif->free_work, xen_blkif_deferred_free);
 
 	return blkif;
@@ -320,6 +329,7 @@  static void xen_blkif_free(struct xen_blkif *blkif)
 
 	/* Make sure everything is drained before shutting down */
 	kmem_cache_free(xen_blkif_cachep, blkif);
+	module_put(THIS_MODULE);
 }
 
 int __init xen_blkif_interface_init(void)