| Message ID | 20151027181235.GA27038@phlsvsds.ph.intel.com (mailing list archive) |
|---|---|
| State | Rejected |
| Headers | show |
On Tue, Oct 27, 2015 at 02:12:36PM -0400, ira.weiny wrote: > On Tue, Oct 27, 2015 at 09:17:40PM +0530, Saurabh Sengar wrote: > > replace GFP_KERNEL with GFP_ATOMIC, as code while holding a spinlock > > should be atomic > > GFP_KERNEL may sleep and can cause deadlock, where as GFP_ATOMIC may > > fail but certainly avoids deadlock > > Great catch. Thanks! > > However, gfp_t is passed to send_mad and we should pass that down and use it. > spin_lock_irqsave(&ib_nl_request_lock, flags); > - ret = ib_nl_send_msg(query); > + ret = ib_nl_send_msg(query, gfp_mask); A spin lock is guarenteed held around ib_nl_send_msg, so it's allocations have to be atomic, can't use gfp_mask here.. I do wonder if it is a good idea to call ib_nl_send_msg with a spinlock held though.. Would be nice to see that go away. Jason -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Tue, Oct 27, 2015 at 12:16:52PM -0600, Jason Gunthorpe wrote: > On Tue, Oct 27, 2015 at 02:12:36PM -0400, ira.weiny wrote: > > On Tue, Oct 27, 2015 at 09:17:40PM +0530, Saurabh Sengar wrote: > > > replace GFP_KERNEL with GFP_ATOMIC, as code while holding a spinlock > > > should be atomic > > > GFP_KERNEL may sleep and can cause deadlock, where as GFP_ATOMIC may > > > fail but certainly avoids deadlock > > > > Great catch. Thanks! > > > > However, gfp_t is passed to send_mad and we should pass that down and use it. > > > spin_lock_irqsave(&ib_nl_request_lock, flags); > > - ret = ib_nl_send_msg(query); > > + ret = ib_nl_send_msg(query, gfp_mask); > > A spin lock is guarenteed held around ib_nl_send_msg, so it's > allocations have to be atomic, can't use gfp_mask here.. > > I do wonder if it is a good idea to call ib_nl_send_msg with a spinlock > held though.. Would be nice to see that go away. Ah, yea my bad. Ira > > Jason -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
> -----Original Message----- > From: Jason Gunthorpe [mailto:jgunthorpe@obsidianresearch.com] > Sent: Tuesday, October 27, 2015 2:17 PM > To: Weiny, Ira > Cc: Saurabh Sengar; dledford@redhat.com; Hefty, Sean; > hal.rosenstock@gmail.com; yun.wang@profitbricks.com; Wan, Kaike; linux- > rdma@vger.kernel.org; linux-kernel@vger.kernel.org > Subject: Re: [PATCH] IB/sa: replace GFP_KERNEL with GFP_ATOMIC > > On Tue, Oct 27, 2015 at 02:12:36PM -0400, ira.weiny wrote: > > On Tue, Oct 27, 2015 at 09:17:40PM +0530, Saurabh Sengar wrote: > > > replace GFP_KERNEL with GFP_ATOMIC, as code while holding a spinlock > > > should be atomic GFP_KERNEL may sleep and can cause deadlock, where > > > as GFP_ATOMIC may fail but certainly avoids deadlock > > > > Great catch. Thanks! > > > > However, gfp_t is passed to send_mad and we should pass that down and > use it. > > > spin_lock_irqsave(&ib_nl_request_lock, flags); > > - ret = ib_nl_send_msg(query); > > + ret = ib_nl_send_msg(query, gfp_mask); > > A spin lock is guarenteed held around ib_nl_send_msg, so it's allocations > have to be atomic, can't use gfp_mask here.. > > I do wonder if it is a good idea to call ib_nl_send_msg with a spinlock held > though.. Would be nice to see that go away. We have to hold the lock to protect against a race condition that a quick response will try to free the request from the ib_nl_request_list before we even put it on the list. > > Jason -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Tue, Oct 27, 2015 at 06:56:50PM +0000, Wan, Kaike wrote: > > I do wonder if it is a good idea to call ib_nl_send_msg with a spinlock held > > though.. Would be nice to see that go away. > > We have to hold the lock to protect against a race condition that a > quick response will try to free the request from the > ib_nl_request_list before we even put it on the list. Put is on the list first? Use a kref? Doesn't look like a big deal to clean this up. Jason -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
> > On Tue, Oct 27, 2015 at 06:56:50PM +0000, Wan, Kaike wrote: > > > > I do wonder if it is a good idea to call ib_nl_send_msg with a > > > spinlock held though.. Would be nice to see that go away. > > > > We have to hold the lock to protect against a race condition that a > > quick response will try to free the request from the > > ib_nl_request_list before we even put it on the list. > > Put is on the list first? Use a kref? Doesn't look like a big deal to clean this up. > > Jason Not sure what "Put is on the list first?" means. I think it is valid to build the request, if success, add to the list, then send it. That would solve the problem you mention above. Was that what you hand in mind, Jason? I don't have time to work on this right now, not sure about Kaike. Until we can remove the spinlock the current proposed patch should be applied in the interim. Sorry for the noise before. Reviewed-By: Ira Weiny <ira.weiny@intel.com> -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Wed, 28 Oct 2015 at 04:30:10 +0000, Weiny, Ira wrote: > Until we can remove the spinlock the current proposed patch should be applied > in the interim. Sorry for the noise before. > Reviewed-By: Ira Weiny <ira.weiny@intel.com> Hi, is this patch expected to be applied ? -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
I don't think so. The following patch has rendered this patch unnecessary: https://patchwork.kernel.org/patch/7526811/ Kaike > -----Original Message----- > From: linux-rdma-owner@vger.kernel.org [mailto:linux-rdma- > owner@vger.kernel.org] On Behalf Of Saurabh Sengar > Sent: Friday, November 13, 2015 2:42 AM > To: Weiny, Ira; Wan, Kaike > Cc: Hefty, Sean; hal.rosenstock@gmail.com; dledford@redhat.com; linux- > rdma@vger.kernel.org; linux-kernel@vger.kernel.org > Subject: Re: [PATCH] IB/sa: replace GFP_KERNEL with GFP_ATOMIC > > On Wed, 28 Oct 2015 at 04:30:10 +0000, Weiny, Ira wrote: > > Until we can remove the spinlock the current proposed patch should be > > applied in the interim. Sorry for the noise before. > > > Reviewed-By: Ira Weiny <ira.weiny@intel.com> > > Hi, > > is this patch expected to be applied ? > -- > To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the > body of a message to majordomo@vger.kernel.org More majordomo info at > http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Fri, 13 Nov 2015 at 10:47:52 +0000, Wan, Kaike wrote: > I don't think so. > The following patch has rendered this patch unnecessary: > https://patchwork.kernel.org/patch/7526811/ > Kaike OK, could you please use "Reported-by" tag of my name in your patch Regards, Saurabh -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
diff --git a/drivers/infiniband/core/sa_query.c b/drivers/infiniband/core/sa_query.c index 8c014b33d8e0..54d454042b28 100644 --- a/drivers/infiniband/core/sa_query.c +++ b/drivers/infiniband/core/sa_query.c @@ -512,7 +512,7 @@ static int ib_nl_get_path_rec_attrs_len(ib_sa_comp_mask comp_mask) return len; } -static int ib_nl_send_msg(struct ib_sa_query *query) +static int ib_nl_send_msg(struct ib_sa_query *query, gfp_t gfp_mask) { struct sk_buff *skb = NULL; struct nlmsghdr *nlh; @@ -526,7 +526,7 @@ static int ib_nl_send_msg(struct ib_sa_query *query) if (len <= 0) return -EMSGSIZE; - skb = nlmsg_new(len, GFP_KERNEL); + skb = nlmsg_new(len, gfp_mask); if (!skb) return -ENOMEM; @@ -544,7 +544,7 @@ static int ib_nl_send_msg(struct ib_sa_query *query) /* Repair the nlmsg header length */ nlmsg_end(skb, nlh); - ret = ibnl_multicast(skb, nlh, RDMA_NL_GROUP_LS, GFP_KERNEL); + ret = ibnl_multicast(skb, nlh, RDMA_NL_GROUP_LS, gfp_mask); if (!ret) ret = len; else @@ -553,7 +553,7 @@ static int ib_nl_send_msg(struct ib_sa_query *query) return ret; } -static int ib_nl_make_request(struct ib_sa_query *query) +static int ib_nl_make_request(struct ib_sa_query *query, gfp_t gfp_mask) { unsigned long flags; unsigned long delay; @@ -563,7 +563,7 @@ static int ib_nl_make_request(struct ib_sa_query *query) query->seq = (u32)atomic_inc_return(&ib_nl_sa_request_seq); spin_lock_irqsave(&ib_nl_request_lock, flags); - ret = ib_nl_send_msg(query); + ret = ib_nl_send_msg(query, gfp_mask); if (ret <= 0) { ret = -EIO; goto request_out; @@ -1105,7 +1105,7 @@ static int send_mad(struct ib_sa_query *query, int timeout_ms, gfp_t gfp_mask) if (query->flags & IB_SA_ENABLE_LOCAL_SERVICE) { if (!ibnl_chk_listeners(RDMA_NL_GROUP_LS)) { - if (!ib_nl_make_request(query)) + if (!ib_nl_make_request(query, gfp_mask)) return id; } ib_sa_disable_local_svc(query);