| Message ID | 1491226221-24621-1-git-send-email-kyle.fortin@oracle.com (mailing list archive) |
|---|---|
| State | Changes Requested, archived |
| Headers | show |
On Mon, Apr 03, 2017 at 06:30:21AM -0700, Kyle Fortin wrote: > iscsiadm session login can fail with the following error: > > iscsiadm: Could not login to [iface: default, target: iqn.1986-03.com... > iscsiadm: initiator reported error (9 - internal error) > > When /etc/iscsi/iscsid.conf sets node.session.cmds_max = 4096, it results > in 64K-sized kmallocs per session. A system under fragmented slab > pressure may not have any 64K objects available and fail iscsiadm session > login. Even though memory objects of a smaller size are available, the > large order allocation ends up failing. > > The kernel will print a warning and dump_stack, like below: There is a series of patches in Andrew's mmotm tree, which introduces a kvmalloc() function, that does exactly what you're looking for. Maybe you want to base your patch on top of it.
On Apr 3, 2017, at 9:41 AM, Johannes Thumshirn <jthumshirn@suse.de> wrote: > > On Mon, Apr 03, 2017 at 06:30:21AM -0700, Kyle Fortin wrote: >> iscsiadm session login can fail with the following error: >> >> iscsiadm: Could not login to [iface: default, target: iqn.1986-03.com... >> iscsiadm: initiator reported error (9 - internal error) >> >> When /etc/iscsi/iscsid.conf sets node.session.cmds_max = 4096, it results >> in 64K-sized kmallocs per session. A system under fragmented slab >> pressure may not have any 64K objects available and fail iscsiadm session >> login. Even though memory objects of a smaller size are available, the >> large order allocation ends up failing. >> >> The kernel will print a warning and dump_stack, like below: > > There is a series of patches in Andrew's mmotm tree, which introduces > a kvmalloc() function, that does exactly what you're looking for. > > Maybe you want to base your patch on top of it. > > -- > Johannes Thumshirn Storage > jthumshirn@suse.de +49 911 74053 689 > SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nürnberg > GF: Felix Imendörffer, Jane Smithard, Graham Norton > HRB 21284 (AG Nürnberg) > Key fingerprint = EC38 9CAB C2C4 F25D 8600 D0D0 0393 969D 2D76 0850 Thanks Johannes. I’ll take a look. -- Kyle Fortin - Oracle Linux Engineering
On Mon, Apr 3, 2017 at 6:30 AM, Kyle Fortin <kyle.fortin@oracle.com> wrote: > > for (i = 0; i < q->max; i++) > kfree(q->pool[i]); > - kfree(q->pool); > + if (q->is_pool_vmalloc) you could do something like: if (is_vmalloc_addr(q->pool)) vfree(...); else kfree(..); And then remove the bool. Chetan
On Apr 3, 2017, at 3:46 PM, Chet L <chetanloke@gmail.com> wrote: > > On Mon, Apr 3, 2017 at 6:30 AM, Kyle Fortin <kyle.fortin@oracle.com> wrote: > >> >> for (i = 0; i < q->max; i++) >> kfree(q->pool[i]); >> - kfree(q->pool); >> + if (q->is_pool_vmalloc) > > you could do something like: > > if (is_vmalloc_addr(q->pool)) > vfree(...); > else > kfree(..); > > And then remove the bool. > > Chetan Using linux-mmots.git which includes the new kvmalloc api, this patch is greatly simplified to a 2 character change (‘v’ x 2) of using kvmalloc / kvfree for the iscsi_pool allocation. When kvmalloc is accepted into mainline and makes it into scsi.git, then I’ll post the v2 patch using that. -- Kyle Fortin - Oracle Linux Engineering
diff --git a/drivers/scsi/libiscsi.c b/drivers/scsi/libiscsi.c index 3fca34a675af..5a622ba2f10d 100644 --- a/drivers/scsi/libiscsi.c +++ b/drivers/scsi/libiscsi.c @@ -27,6 +27,7 @@ #include <linux/log2.h> #include <linux/slab.h> #include <linux/module.h> +#include <linux/vmalloc.h> #include <asm/unaligned.h> #include <net/tcp.h> #include <scsi/scsi_cmnd.h> @@ -2546,6 +2547,7 @@ int iscsi_eh_recover_target(struct scsi_cmnd *sc) iscsi_pool_init(struct iscsi_pool *q, int max, void ***items, int item_size) { int i, num_arrays = 1; + int alloc_size; memset(q, 0, sizeof(*q)); @@ -2555,7 +2557,13 @@ int iscsi_eh_recover_target(struct scsi_cmnd *sc) * the array. */ if (items) num_arrays++; - q->pool = kzalloc(num_arrays * max * sizeof(void*), GFP_KERNEL); + + alloc_size = num_arrays * max * sizeof(void *); + if (alloc_size > PAGE_SIZE) { + q->pool = vzalloc(alloc_size); + q->is_pool_vmalloc = true; + } else + q->pool = kzalloc(alloc_size, GFP_KERNEL); if (q->pool == NULL) return -ENOMEM; @@ -2589,7 +2597,10 @@ void iscsi_pool_free(struct iscsi_pool *q) for (i = 0; i < q->max; i++) kfree(q->pool[i]); - kfree(q->pool); + if (q->is_pool_vmalloc) + vfree(q->pool); + else + kfree(q->pool); } EXPORT_SYMBOL_GPL(iscsi_pool_free); diff --git a/include/scsi/libiscsi.h b/include/scsi/libiscsi.h index 583875ea136a..e3421e527559 100644 --- a/include/scsi/libiscsi.h +++ b/include/scsi/libiscsi.h @@ -258,6 +258,7 @@ struct iscsi_pool { struct kfifo queue; /* FIFO Queue */ void **pool; /* Pool of elements */ int max; /* Max number of elements */ + bool is_pool_vmalloc; }; /* Session's states */