| Message ID | 20210121050328.7891-1-dongli.zhang@oracle.com (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
| Series | [1/1] vhost scsi: allocate vhost_scsi with GFP_NOWAIT to avoid delay | expand |
On 2021/1/21 13:03, Dongli Zhang wrote: > The size of 'struct vhost_scsi' is order-10 (~2.3MB). It may take long time > delay by kzalloc() to compact memory pages when there is a lack of > high-order pages. As a result, there is latency to create a VM (with > vhost-scsi) or to hotadd vhost-scsi-based storage. > > The prior commit 595cb754983d ("vhost/scsi: use vmalloc for order-10 > allocation") prefers to fallback only when really needed, while this patch > changes allocation to GFP_NOWAIT in order to avoid the delay caused by > memory page compact. > > Cc: Aruna Ramakrishna <aruna.ramakrishna@oracle.com> > Cc: Joe Jin <joe.jin@oracle.com> > Signed-off-by: Dongli Zhang <dongli.zhang@oracle.com> > --- > Another option is to rework by reducing the size of 'struct vhost_scsi', > e.g., by replacing inline vhost_scsi.vqs with just memory pointers while > each vhost_scsi.vqs[i] should be allocated separately. Please let me > know if that option is better. > > drivers/vhost/scsi.c | 2 +- > 1 file changed, 1 insertion(+), 1 deletion(-) > > diff --git a/drivers/vhost/scsi.c b/drivers/vhost/scsi.c > index 4ce9f00ae10e..85eaa4e883f4 100644 > --- a/drivers/vhost/scsi.c > +++ b/drivers/vhost/scsi.c > @@ -1814,7 +1814,7 @@ static int vhost_scsi_open(struct inode *inode, struct file *f) > struct vhost_virtqueue **vqs; > int r = -ENOMEM, i; > > - vs = kzalloc(sizeof(*vs), GFP_KERNEL | __GFP_NOWARN | __GFP_RETRY_MAYFAIL); > + vs = kzalloc(sizeof(*vs), GFP_NOWAIT | __GFP_NOWARN); > if (!vs) { > vs = vzalloc(sizeof(*vs)); > if (!vs) Can we use kvzalloc? Thanks
On 1/21/21 1:00 AM, Jason Wang wrote: > > On 2021/1/21 13:03, Dongli Zhang wrote: >> The size of 'struct vhost_scsi' is order-10 (~2.3MB). It may take long time >> delay by kzalloc() to compact memory pages when there is a lack of >> high-order pages. As a result, there is latency to create a VM (with >> vhost-scsi) or to hotadd vhost-scsi-based storage. >> >> The prior commit 595cb754983d ("vhost/scsi: use vmalloc for order-10 >> allocation") prefers to fallback only when really needed, while this patch >> changes allocation to GFP_NOWAIT in order to avoid the delay caused by >> memory page compact. >> >> Cc: Aruna Ramakrishna <aruna.ramakrishna@oracle.com> >> Cc: Joe Jin <joe.jin@oracle.com> >> Signed-off-by: Dongli Zhang <dongli.zhang@oracle.com> >> --- >> Another option is to rework by reducing the size of 'struct vhost_scsi', >> e.g., by replacing inline vhost_scsi.vqs with just memory pointers while >> each vhost_scsi.vqs[i] should be allocated separately. Please let me >> know if that option is better. >> >> drivers/vhost/scsi.c | 2 +- >> 1 file changed, 1 insertion(+), 1 deletion(-) >> >> diff --git a/drivers/vhost/scsi.c b/drivers/vhost/scsi.c >> index 4ce9f00ae10e..85eaa4e883f4 100644 >> --- a/drivers/vhost/scsi.c >> +++ b/drivers/vhost/scsi.c >> @@ -1814,7 +1814,7 @@ static int vhost_scsi_open(struct inode *inode, struct >> file *f) >> struct vhost_virtqueue **vqs; >> int r = -ENOMEM, i; >> - vs = kzalloc(sizeof(*vs), GFP_KERNEL | __GFP_NOWARN | >> __GFP_RETRY_MAYFAIL); >> + vs = kzalloc(sizeof(*vs), GFP_NOWAIT | __GFP_NOWARN); >> if (!vs) { >> vs = vzalloc(sizeof(*vs)); >> if (!vs) > > > Can we use kvzalloc? > Thank you very much for the suggestion. To use 'GFP_NOWAIT' will avoid any direct compact in __alloc_pages_slowpath(), while to use kvzalloc() will just avoid retrying direct compact for multiple times. Although the latter will still do direct compact (without retry), I think it is better than the former using GFP_NOWAIT. I will send v2 with kvzalloc(). Thank you very much! Dongli Zhang
diff --git a/drivers/vhost/scsi.c b/drivers/vhost/scsi.c index 4ce9f00ae10e..85eaa4e883f4 100644 --- a/drivers/vhost/scsi.c +++ b/drivers/vhost/scsi.c @@ -1814,7 +1814,7 @@ static int vhost_scsi_open(struct inode *inode, struct file *f) struct vhost_virtqueue **vqs; int r = -ENOMEM, i; - vs = kzalloc(sizeof(*vs), GFP_KERNEL | __GFP_NOWARN | __GFP_RETRY_MAYFAIL); + vs = kzalloc(sizeof(*vs), GFP_NOWAIT | __GFP_NOWARN); if (!vs) { vs = vzalloc(sizeof(*vs)); if (!vs)
The size of 'struct vhost_scsi' is order-10 (~2.3MB). It may take long time delay by kzalloc() to compact memory pages when there is a lack of high-order pages. As a result, there is latency to create a VM (with vhost-scsi) or to hotadd vhost-scsi-based storage. The prior commit 595cb754983d ("vhost/scsi: use vmalloc for order-10 allocation") prefers to fallback only when really needed, while this patch changes allocation to GFP_NOWAIT in order to avoid the delay caused by memory page compact. Cc: Aruna Ramakrishna <aruna.ramakrishna@oracle.com> Cc: Joe Jin <joe.jin@oracle.com> Signed-off-by: Dongli Zhang <dongli.zhang@oracle.com> --- Another option is to rework by reducing the size of 'struct vhost_scsi', e.g., by replacing inline vhost_scsi.vqs with just memory pointers while each vhost_scsi.vqs[i] should be allocated separately. Please let me know if that option is better. drivers/vhost/scsi.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)