From patchwork Mon Dec 17 15:15:16 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Shamir Rabinovitch X-Patchwork-Id: 10733651 X-Patchwork-Delegate: jgg@ziepe.ca Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 8408C14DE for ; Mon, 17 Dec 2018 15:22:37 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 747952A0C7 for ; Mon, 17 Dec 2018 15:22:37 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 6856E2A0CC; Mon, 17 Dec 2018 15:22:37 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id A741F2A0C7 for ; Mon, 17 Dec 2018 15:22:36 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726887AbeLQPWg (ORCPT ); Mon, 17 Dec 2018 10:22:36 -0500 Received: from userp2130.oracle.com ([156.151.31.86]:36288 "EHLO userp2130.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726637AbeLQPWg (ORCPT ); Mon, 17 Dec 2018 10:22:36 -0500 Received: from pps.filterd (userp2130.oracle.com [127.0.0.1]) by userp2130.oracle.com (8.16.0.22/8.16.0.22) with SMTP id wBHFEPYt140856; Mon, 17 Dec 2018 15:22:03 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references; s=corp-2018-07-02; bh=Cyf0CqVVqFzU9WwM3dmfOGa0nM1tnaNyB6cISYKFsn4=; b=PQzPzAbtAqKQwEcnHEIvJLoxyc7Xr2a8g2xenpnP055qxASdXNwPQjVfTdX/GrZczKZZ QFn8xAbZMZabW4MVEBqCyC0PtW4e0OmqY0ktbTyCV2efpZSuFEhowQy4dFuH0vWxo3RG HlieilcGPGrICdcXnbqE667k37BKPVW5irIZMcQSQNenWenROtB4s2Wu7MlzTkGdFmJX SHfeWTcPR1Evrb0WZexiLc2t0qSopogN45WTDMqoKkwQg5yDZfYrFrKWFmsnCrDO2KpR FiYue8Afm8oj40YN/8qmgT3Ww60+C1Q0CYyyUNF7lge9WZInQka5NeWEVAtFQyQiYU01 pw== Received: from userv0022.oracle.com (userv0022.oracle.com [156.151.31.74]) by userp2130.oracle.com with ESMTP id 2pcs1te7qa-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 17 Dec 2018 15:22:03 +0000 Received: from aserv0122.oracle.com (aserv0122.oracle.com [141.146.126.236]) by userv0022.oracle.com (8.14.4/8.14.4) with ESMTP id wBHFM1kB018320 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 17 Dec 2018 15:22:02 GMT Received: from abhmp0015.oracle.com (abhmp0015.oracle.com [141.146.116.21]) by aserv0122.oracle.com (8.14.4/8.14.4) with ESMTP id wBHFLxPQ018489; Mon, 17 Dec 2018 15:21:59 GMT Received: from localhost.localdomain (/77.124.81.87) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Mon, 17 Dec 2018 07:21:58 -0800 From: Shamir Rabinovitch To: linux-rdma@vger.kernel.org Cc: dledford@redhat.com, jgg@ziepe.ca, Shamir Rabinovitch , Leon Romanovsky , Parav Pandit , Daniel Jurgens , Steve Wise , Mark Bloch , Randy Dunlap , Sagi Grimberg , Max Gurtovoy , Jack Morgenstein , Yossi Itigin , Yishai Hadas , Matan Barak , Ariel Levkovich , "Michael J. Ruhl" , Denis Drozdov , Hans Westgaard Ry Subject: [PATCH v4 1/4] RDMA/restrack: resource-tracker should not use uobject pointers Date: Mon, 17 Dec 2018 17:15:16 +0200 Message-Id: <20181217151528.30743-2-shamir.rabinovitch@oracle.com> X-Mailer: git-send-email 2.17.2 In-Reply-To: <20181217151528.30743-1-shamir.rabinovitch@oracle.com> References: <20181217151528.30743-1-shamir.rabinovitch@oracle.com> X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=9110 signatures=668679 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=1 malwarescore=0 phishscore=0 bulkscore=0 spamscore=0 mlxscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1812170136 Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP having uobject pointer embedded in ib core objects is not aligned with future shared ib_x model. Future patches will change this so resource tracker must not use this pointer as well Signed-off-by: Shamir Rabinovitch --- drivers/infiniband/core/cma.c | 2 +- drivers/infiniband/core/core_priv.h | 5 ++- drivers/infiniband/core/cq.c | 2 +- drivers/infiniband/core/restrack.c | 42 ++++++++----------- drivers/infiniband/core/uverbs_cmd.c | 8 ++-- drivers/infiniband/core/uverbs_std_types_cq.c | 2 +- drivers/infiniband/core/verbs.c | 6 +-- include/rdma/restrack.h | 16 +++++-- 8 files changed, 44 insertions(+), 39 deletions(-) diff --git a/drivers/infiniband/core/cma.c b/drivers/infiniband/core/cma.c index 15d5bb7bf6bb..63a7cc00bae0 100644 --- a/drivers/infiniband/core/cma.c +++ b/drivers/infiniband/core/cma.c @@ -494,7 +494,7 @@ static void _cma_attach_to_dev(struct rdma_id_private *id_priv, id_priv->id.route.addr.dev_addr.transport = rdma_node_get_transport(cma_dev->device->node_type); list_add_tail(&id_priv->list, &cma_dev->id_list); - rdma_restrack_add(&id_priv->res); + rdma_restrack_kadd(&id_priv->res); } static void cma_attach_to_dev(struct rdma_id_private *id_priv, diff --git a/drivers/infiniband/core/core_priv.h b/drivers/infiniband/core/core_priv.h index cc7535c5e192..f1cfd994c588 100644 --- a/drivers/infiniband/core/core_priv.h +++ b/drivers/infiniband/core/core_priv.h @@ -297,7 +297,10 @@ static inline struct ib_qp *_ib_create_qp(struct ib_device *dev, */ if (attr->qp_type < IB_QPT_XRC_INI) { qp->res.type = RDMA_RESTRACK_QP; - rdma_restrack_add(&qp->res); + if (uobj) + rdma_restrack_uadd(&qp->res); + else + rdma_restrack_kadd(&qp->res); } else qp->res.valid = false; diff --git a/drivers/infiniband/core/cq.c b/drivers/infiniband/core/cq.c index b1e5365ddafa..e961652d77cb 100644 --- a/drivers/infiniband/core/cq.c +++ b/drivers/infiniband/core/cq.c @@ -162,7 +162,7 @@ struct ib_cq *__ib_alloc_cq(struct ib_device *dev, void *private, cq->res.type = RDMA_RESTRACK_CQ; rdma_restrack_set_task(&cq->res, caller); - rdma_restrack_add(&cq->res); + rdma_restrack_kadd(&cq->res); switch (cq->poll_ctx) { case IB_POLL_DIRECT: diff --git a/drivers/infiniband/core/restrack.c b/drivers/infiniband/core/restrack.c index 3dd316159f5f..d743c68d7c30 100644 --- a/drivers/infiniband/core/restrack.c +++ b/drivers/infiniband/core/restrack.c @@ -139,27 +139,6 @@ static struct ib_device *res_to_dev(struct rdma_restrack_entry *res) } } -static bool res_is_user(struct rdma_restrack_entry *res) -{ - switch (res->type) { - case RDMA_RESTRACK_PD: - return container_of(res, struct ib_pd, res)->uobject; - case RDMA_RESTRACK_CQ: - return container_of(res, struct ib_cq, res)->uobject; - case RDMA_RESTRACK_QP: - return container_of(res, struct ib_qp, res)->uobject; - case RDMA_RESTRACK_CM_ID: - return !res->kern_name; - case RDMA_RESTRACK_MR: - return container_of(res, struct ib_mr, res)->pd->uobject; - case RDMA_RESTRACK_CTX: - return true; - default: - WARN_ONCE(true, "Wrong resource tracking type %u\n", res->type); - return false; - } -} - void rdma_restrack_set_task(struct rdma_restrack_entry *res, const char *caller) { @@ -175,17 +154,17 @@ void rdma_restrack_set_task(struct rdma_restrack_entry *res, } EXPORT_SYMBOL(rdma_restrack_set_task); -void rdma_restrack_add(struct rdma_restrack_entry *res) +static void rdma_restrack_add(struct rdma_restrack_entry *res) { struct ib_device *dev = res_to_dev(res); if (!dev) return; - if (res->type != RDMA_RESTRACK_CM_ID || !res_is_user(res)) + if (res->type != RDMA_RESTRACK_CM_ID || rdma_is_kernel_res(res)) res->task = NULL; - if (res_is_user(res)) { + if (!rdma_is_kernel_res(res)) { if (!res->task) rdma_restrack_set_task(res, NULL); res->kern_name = NULL; @@ -201,7 +180,20 @@ void rdma_restrack_add(struct rdma_restrack_entry *res) hash_add(dev->res.hash, &res->node, res->type); up_write(&dev->res.rwsem); } -EXPORT_SYMBOL(rdma_restrack_add); + +void rdma_restrack_kadd(struct rdma_restrack_entry *res) +{ + res->user = false; + rdma_restrack_add(res); +} +EXPORT_SYMBOL(rdma_restrack_kadd); + +void rdma_restrack_uadd(struct rdma_restrack_entry *res) +{ + res->user = true; + rdma_restrack_add(res); +} +EXPORT_SYMBOL(rdma_restrack_uadd); int __must_check rdma_restrack_get(struct rdma_restrack_entry *res) { diff --git a/drivers/infiniband/core/uverbs_cmd.c b/drivers/infiniband/core/uverbs_cmd.c index b70749542471..f68d234b09f2 100644 --- a/drivers/infiniband/core/uverbs_cmd.c +++ b/drivers/infiniband/core/uverbs_cmd.c @@ -260,7 +260,7 @@ static int ib_uverbs_get_context(struct uverbs_attr_bundle *attrs) fd_install(resp.async_fd, filp); ucontext->res.type = RDMA_RESTRACK_CTX; - rdma_restrack_add(&ucontext->res); + rdma_restrack_uadd(&ucontext->res); /* * Make sure that ib_uverbs_get_ucontext() sees the pointer update @@ -470,7 +470,7 @@ static int ib_uverbs_alloc_pd(struct uverbs_attr_bundle *attrs) memset(&resp, 0, sizeof resp); resp.pd_handle = uobj->id; pd->res.type = RDMA_RESTRACK_PD; - rdma_restrack_add(&pd->res); + rdma_restrack_uadd(&pd->res); ret = uverbs_response(attrs, &resp, sizeof(resp)); if (ret) @@ -785,7 +785,7 @@ static int ib_uverbs_reg_mr(struct uverbs_attr_bundle *attrs) mr->uobject = uobj; atomic_inc(&pd->usecnt); mr->res.type = RDMA_RESTRACK_MR; - rdma_restrack_add(&mr->res); + rdma_restrack_uadd(&mr->res); uobj->object = mr; @@ -1062,7 +1062,7 @@ static struct ib_ucq_object *create_cq(struct uverbs_attr_bundle *attrs, resp.response_length = uverbs_response_length(attrs, sizeof(resp)); cq->res.type = RDMA_RESTRACK_CQ; - rdma_restrack_add(&cq->res); + rdma_restrack_uadd(&cq->res); ret = uverbs_response(attrs, &resp, sizeof(resp)); if (ret) diff --git a/drivers/infiniband/core/uverbs_std_types_cq.c b/drivers/infiniband/core/uverbs_std_types_cq.c index 859518eab583..26fe4ce91e0e 100644 --- a/drivers/infiniband/core/uverbs_std_types_cq.c +++ b/drivers/infiniband/core/uverbs_std_types_cq.c @@ -126,7 +126,7 @@ static int UVERBS_HANDLER(UVERBS_METHOD_CQ_CREATE)( obj->uobject.user_handle = user_handle; atomic_set(&cq->usecnt, 0); cq->res.type = RDMA_RESTRACK_CQ; - rdma_restrack_add(&cq->res); + rdma_restrack_uadd(&cq->res); ret = uverbs_copy_to(attrs, UVERBS_ATTR_CREATE_CQ_RESP_CQE, &cq->cqe, sizeof(cq->cqe)); diff --git a/drivers/infiniband/core/verbs.c b/drivers/infiniband/core/verbs.c index 178899e3ce73..65d2f58cbfb5 100644 --- a/drivers/infiniband/core/verbs.c +++ b/drivers/infiniband/core/verbs.c @@ -265,7 +265,7 @@ struct ib_pd *__ib_alloc_pd(struct ib_device *device, unsigned int flags, pd->res.type = RDMA_RESTRACK_PD; rdma_restrack_set_task(&pd->res, caller); - rdma_restrack_add(&pd->res); + rdma_restrack_kadd(&pd->res); if (mr_access_flags) { struct ib_mr *mr; @@ -1890,7 +1890,7 @@ struct ib_cq *__ib_create_cq(struct ib_device *device, atomic_set(&cq->usecnt, 0); cq->res.type = RDMA_RESTRACK_CQ; rdma_restrack_set_task(&cq->res, caller); - rdma_restrack_add(&cq->res); + rdma_restrack_kadd(&cq->res); } return cq; @@ -1971,7 +1971,7 @@ struct ib_mr *ib_alloc_mr(struct ib_pd *pd, atomic_inc(&pd->usecnt); mr->need_inval = false; mr->res.type = RDMA_RESTRACK_MR; - rdma_restrack_add(&mr->res); + rdma_restrack_kadd(&mr->res); } return mr; diff --git a/include/rdma/restrack.h b/include/rdma/restrack.h index f34aa96e4518..bf5f32c38072 100644 --- a/include/rdma/restrack.h +++ b/include/rdma/restrack.h @@ -116,6 +116,10 @@ struct rdma_restrack_entry { * @type: various objects in restrack database */ enum rdma_restrack_type type; + /** + * @user: user resource + */ + bool user; }; /** @@ -141,10 +145,16 @@ int rdma_restrack_count(struct rdma_restrack_root *res, struct pid_namespace *ns); /** - * rdma_restrack_add() - add object to the reource tracking database + * rdma_restrack_kadd() - add kernel object to the reource tracking database + * @res: resource entry + */ +void rdma_restrack_kadd(struct rdma_restrack_entry *res); + +/** + * rdma_restrack_uadd() - add user object to the reource tracking database * @res: resource entry */ -void rdma_restrack_add(struct rdma_restrack_entry *res); +void rdma_restrack_uadd(struct rdma_restrack_entry *res); /** * rdma_restrack_del() - delete object from the reource tracking database @@ -159,7 +169,7 @@ void rdma_restrack_del(struct rdma_restrack_entry *res); */ static inline bool rdma_is_kernel_res(struct rdma_restrack_entry *res) { - return !res->task; + return !res->user; } /**