[1/1] libceph: use memalloc flags for net IO

Commit Message

Mike Christie Oct. 16, 2014, 6:50 a.m. UTC

From: Mike Christie <[e-mail]>

This patch has ceph's lib code use the memalloc flags.

If the VM layer needs to write data out to free up memory to handle new
allocation requests, the block layer must be able to make forward progress.
To handle that requirement we use structs like mempools to reserve memory for
objects like bios and requests.

The problem is when we send/receive block layer requests over the network
layer, net skb allocations can fail and the system can lock up.
To solve this, the memalloc related flags were added. NBD, iSCSI
and NFS uses these flags to tell the network/vm layer that it should
use memory reserves to fullfill allcation requests for structs like
skbs.

I am running ceph in a bunch of VMs in my laptop, so this patch was
not tested very harshly. The patch was made over Linus's tree. I tried
to make it over the ceph-client tree but noticed it was very old (3.3
kernel or something).

Signed-off-by: Mike Christie <[e-mail]>
---
 net/ceph/messenger.c |    7 ++++++-
 1 files changed, 6 insertions(+), 1 deletions(-)

Comments

Ilya Dryomov Oct. 16, 2014, 10:10 a.m. UTC | #1

On Thu, Oct 16, 2014 at 10:50 AM,  <[e-mail]> wrote:
> From: Mike Christie <[e-mail]>
>
> This patch has ceph's lib code use the memalloc flags.
>
> If the VM layer needs to write data out to free up memory to handle new
> allocation requests, the block layer must be able to make forward progress.
> To handle that requirement we use structs like mempools to reserve memory for
> objects like bios and requests.
>
> The problem is when we send/receive block layer requests over the network
> layer, net skb allocations can fail and the system can lock up.
> To solve this, the memalloc related flags were added. NBD, iSCSI
> and NFS uses these flags to tell the network/vm layer that it should
> use memory reserves to fullfill allcation requests for structs like
> skbs.

Applied with my Reviewed-by.

>
> I am running ceph in a bunch of VMs in my laptop, so this patch was
> not tested very harshly. The patch was made over Linus's tree. I tried
> to make it over the ceph-client tree but noticed it was very old (3.3
> kernel or something).

I'm guessing you tried Sage's k.org tree.  We do everything on github
(testing branch), k.org tree is used for pull requests only so master
is not updated.

https://github.com/ceph/ceph-client

Thanks,

                Ilya
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to [e-mail]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Patch

diff --git a/net/ceph/messenger.c b/net/ceph/messenger.c
index 559c9f6..a2d9c97 100644
--- a/net/ceph/messenger.c
+++ b/net/ceph/messenger.c
@@ -484,7 +484,7 @@  static int ceph_tcp_connect(struct ceph_connection *con)
 			       IPPROTO_TCP, &sock);
 	if (ret)
 		return ret;
-	sock->sk->sk_allocation = GFP_NOFS;
+	sock->sk->sk_allocation = GFP_NOFS | __GFP_MEMALLOC;
 
 #ifdef CONFIG_LOCKDEP
 	lockdep_set_class(&sock->sk->sk_lock, &socket_class);
@@ -510,6 +510,7 @@  static int ceph_tcp_connect(struct ceph_connection *con)
 		return ret;
 	}
 	con->sock = sock;
+	sk_set_memalloc(sock->sk);
 	return 0;
 }
 
@@ -2769,8 +2770,11 @@  static void con_work(struct work_struct *work)
 {
 	struct ceph_connection *con = container_of(work, struct ceph_connection,
 						   work.work);
+	unsigned long pflags = current->flags;
 	bool fault;
 
+	current->flags |= PF_MEMALLOC;
+
 	mutex_lock(&con->mutex);
 	while (true) {
 		int ret;
@@ -2824,6 +2828,7 @@  static void con_work(struct work_struct *work)
 		con_fault_finish(con);
 
 	con->ops->put(con);
+	tsk_restore_flags(current, pflags, PF_MEMALLOC);
 }
 
 /*