diff mbox

ceph: optimizing cap allocation

Message ID 1519468529-67956-1-git-send-email-cgxu519@icloud.com (mailing list archive)
State New, archived
Headers show

Commit Message

Chengguang Xu Feb. 24, 2018, 10:35 a.m. UTC
When setting high volume of caps_min_count or having many
unreserved caps, unused caps may always keep in the ->caps_list
even can't get new cap from kmem_cache_alloc because lack of
maximum limitation of caps_avail_count. Hence reuse caps in
->caps_list if available, it's maybe better than setting max
limitation of caps_avail_count and releasing unused caps when
reaching the limit.

Signed-off-by: Chengguang Xu <cgxu519@icloud.com>
---
Only compile tested.

 fs/ceph/caps.c | 17 +++++++++++++++++
 1 file changed, 17 insertions(+)

Comments

Yan, Zheng March 5, 2018, 2:52 a.m. UTC | #1
> On 24 Feb 2018, at 18:35, Chengguang Xu <cgxu519@icloud.com> wrote:
> 
> When setting high volume of caps_min_count or having many
> unreserved caps, unused caps may always keep in the ->caps_list
> even can't get new cap from kmem_cache_alloc because lack of
> maximum limitation of caps_avail_count. Hence reuse caps in
> ->caps_list if available, it's maybe better than setting max
> limitation of caps_avail_count and releasing unused caps when
> reaching the limit.
> 
> Signed-off-by: Chengguang Xu <cgxu519@icloud.com>
> ---
> Only compile tested.
> 
> fs/ceph/caps.c | 17 +++++++++++++++++
> 1 file changed, 17 insertions(+)
> 
> diff --git a/fs/ceph/caps.c b/fs/ceph/caps.c
> index 6582c45..b598a24 100644
> --- a/fs/ceph/caps.c
> +++ b/fs/ceph/caps.c
> @@ -280,6 +280,23 @@ struct ceph_cap *ceph_get_cap(struct ceph_mds_client *mdsc,
> 	/* temporary, until we do something about cap import/export */
> 	if (!ctx) {
> 		cap = kmem_cache_alloc(ceph_cap_cachep, GFP_NOFS);
> +		if (!cap) {
> +			spin_lock(&mdsc->caps_list_lock);
> +			if (mdsc->caps_avail_count) {
> +				BUG_ON(list_empty(&mdsc->caps_list));
> +
> +				mdsc->caps_avail_count--;
> +				mdsc->caps_use_count++;
> +				cap = list_first_entry(&mdsc->caps_list,
> +						struct ceph_cap, caps_item);
> +				list_del(&cap->caps_item);
> +
> +				BUG_ON(mdsc->caps_total_count != mdsc->caps_use_count +
> +				       mdsc->caps_reserve_count + mdsc->caps_avail_count);
> +			}
> +			spin_unlock(&mdsc->caps_list_lock);
> +		}
> +

should skip executing following code. I fixed it locally

> 		if (cap) {
> 			spin_lock(&mdsc->caps_list_lock);
> 			mdsc->caps_use_count++;
> -- 
> 1.8.3.1
> 

--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Chengguang Xu March 5, 2018, 4:16 a.m. UTC | #2
> 在 2018年3月5日,上午10:52,Yan, Zheng <zyan@redhat.com> 写道:
> 
> 
> 
>> On 24 Feb 2018, at 18:35, Chengguang Xu <cgxu519@icloud.com> wrote:
>> 
>> When setting high volume of caps_min_count or having many
>> unreserved caps, unused caps may always keep in the ->caps_list
>> even can't get new cap from kmem_cache_alloc because lack of
>> maximum limitation of caps_avail_count. Hence reuse caps in
>> ->caps_list if available, it's maybe better than setting max
>> limitation of caps_avail_count and releasing unused caps when
>> reaching the limit.
>> 
>> Signed-off-by: Chengguang Xu <cgxu519@icloud.com>
>> ---
>> Only compile tested.
>> 
>> fs/ceph/caps.c | 17 +++++++++++++++++
>> 1 file changed, 17 insertions(+)
>> 
>> diff --git a/fs/ceph/caps.c b/fs/ceph/caps.c
>> index 6582c45..b598a24 100644
>> --- a/fs/ceph/caps.c
>> +++ b/fs/ceph/caps.c
>> @@ -280,6 +280,23 @@ struct ceph_cap *ceph_get_cap(struct ceph_mds_client *mdsc,
>> 	/* temporary, until we do something about cap import/export */
>> 	if (!ctx) {
>> 		cap = kmem_cache_alloc(ceph_cap_cachep, GFP_NOFS);
>> +		if (!cap) {
>> +			spin_lock(&mdsc->caps_list_lock);
>> +			if (mdsc->caps_avail_count) {
>> +				BUG_ON(list_empty(&mdsc->caps_list));
>> +
>> +				mdsc->caps_avail_count--;
>> +				mdsc->caps_use_count++;
>> +				cap = list_first_entry(&mdsc->caps_list,
>> +						struct ceph_cap, caps_item);
>> +				list_del(&cap->caps_item);
>> +
>> +				BUG_ON(mdsc->caps_total_count != mdsc->caps_use_count +
>> +				       mdsc->caps_reserve_count + mdsc->caps_avail_count);
>> +			}
>> +			spin_unlock(&mdsc->caps_list_lock);
>> +		}
>> +
> 
> should skip executing following code. I fixed it locally

Yes, that should be in else part. Thanks for your fix.

> 
>> 		if (cap) {
>> 			spin_lock(&mdsc->caps_list_lock);
>> 			mdsc->caps_use_count++;
>> -- 
>> 1.8.3.1

--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
diff mbox

Patch

diff --git a/fs/ceph/caps.c b/fs/ceph/caps.c
index 6582c45..b598a24 100644
--- a/fs/ceph/caps.c
+++ b/fs/ceph/caps.c
@@ -280,6 +280,23 @@  struct ceph_cap *ceph_get_cap(struct ceph_mds_client *mdsc,
 	/* temporary, until we do something about cap import/export */
 	if (!ctx) {
 		cap = kmem_cache_alloc(ceph_cap_cachep, GFP_NOFS);
+		if (!cap) {
+			spin_lock(&mdsc->caps_list_lock);
+			if (mdsc->caps_avail_count) {
+				BUG_ON(list_empty(&mdsc->caps_list));
+
+				mdsc->caps_avail_count--;
+				mdsc->caps_use_count++;
+				cap = list_first_entry(&mdsc->caps_list,
+						struct ceph_cap, caps_item);
+				list_del(&cap->caps_item);
+
+				BUG_ON(mdsc->caps_total_count != mdsc->caps_use_count +
+				       mdsc->caps_reserve_count + mdsc->caps_avail_count);
+			}
+			spin_unlock(&mdsc->caps_list_lock);
+		}
+
 		if (cap) {
 			spin_lock(&mdsc->caps_list_lock);
 			mdsc->caps_use_count++;