From patchwork Mon Jan 11 17:56:43 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ira Weiny X-Patchwork-Id: 8007671 Return-Path: X-Original-To: patchwork-linux-rdma@patchwork.kernel.org Delivered-To: patchwork-parsemail@patchwork1.web.kernel.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.136]) by patchwork1.web.kernel.org (Postfix) with ESMTP id 27E569F88A for ; Mon, 11 Jan 2016 17:58:01 +0000 (UTC) Received: from mail.kernel.org (localhost [127.0.0.1]) by mail.kernel.org (Postfix) with ESMTP id 49EC520263 for ; Mon, 11 Jan 2016 17:58:00 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 412F720256 for ; Mon, 11 Jan 2016 17:57:59 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1759609AbcAKR54 (ORCPT ); Mon, 11 Jan 2016 12:57:56 -0500 Received: from mga14.intel.com ([192.55.52.115]:59348 "EHLO mga14.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1759923AbcAKR54 (ORCPT ); Mon, 11 Jan 2016 12:57:56 -0500 Received: from fmsmga004.fm.intel.com ([10.253.24.48]) by fmsmga103.fm.intel.com with ESMTP; 11 Jan 2016 09:57:34 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.20,554,1444719600"; d="scan'208";a="27474497" Received: from phlsvsds.ph.intel.com ([10.228.195.38]) by fmsmga004.fm.intel.com with ESMTP; 11 Jan 2016 09:57:33 -0800 Received: from phlsvsds.ph.intel.com (localhost.localdomain [127.0.0.1]) by phlsvsds.ph.intel.com (8.13.8/8.13.8) with ESMTP id u0BHvWml015139; Mon, 11 Jan 2016 12:57:32 -0500 Received: (from iweiny@localhost) by phlsvsds.ph.intel.com (8.13.8/8.13.8/Submit) id u0BHvWbx015136; Mon, 11 Jan 2016 12:57:32 -0500 X-Authentication-Warning: phlsvsds.ph.intel.com: iweiny set sender to ira.weiny@intel.com using -f From: ira.weiny@intel.com To: dledford@redhat.com, linux-rdma@vger.kernel.org Cc: gregkh@linuxfoundation.org, devel@driverdev.osuosl.org, Mitko Haralanov Subject: [RESUBMIT PATCH v2 07/14] staging/rdma/hfi1: Add definitions and support functions for TID groups Date: Mon, 11 Jan 2016 12:56:43 -0500 Message-Id: <1452535010-14087-8-git-send-email-ira.weiny@intel.com> X-Mailer: git-send-email 1.7.1 In-Reply-To: <1452535010-14087-1-git-send-email-ira.weiny@intel.com> References: <1452535010-14087-1-git-send-email-ira.weiny@intel.com> Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org X-Spam-Status: No, score=-6.9 required=5.0 tests=BAYES_00, RCVD_IN_DNSWL_HI, RP_MATCHES_RCVD, UNPARSEABLE_RELAY autolearn=unavailable version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on mail.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Mitko Haralanov Definitions and functions use to manage sets of TID/RcvArray groups. These will be used by the TID cacheline functionality coming with later patches. TID groups (or RcvArray groups) are groups of TID/RcvArray entries organized in sets of 8 and aligned on cacheline boundaries. The TID/RcvArray entries are managed in this way to make taking advantage of write-combining easier - each group is a entire cacheline. rcv_array_wc_fill() is provided to allow of generating writes to TIDs which are not currently being used in order to cause the flush of the write-combining buffer. Signed-off-by: Mitko Haralanov Reviewed-by: Ira Weiny --- drivers/staging/rdma/hfi1/user_exp_rcv.c | 64 ++++++++++++++++++++++++++++++++ 1 file changed, 64 insertions(+) diff --git a/drivers/staging/rdma/hfi1/user_exp_rcv.c b/drivers/staging/rdma/hfi1/user_exp_rcv.c index bafeddf67c8f..7f15024daab9 100644 --- a/drivers/staging/rdma/hfi1/user_exp_rcv.c +++ b/drivers/staging/rdma/hfi1/user_exp_rcv.c @@ -52,6 +52,14 @@ #include "user_exp_rcv.h" #include "trace.h" +struct tid_group { + struct list_head list; + unsigned base; + u8 size; + u8 used; + u8 map; +}; + struct mmu_rb_node { struct rb_node rbnode; unsigned long virt; @@ -75,6 +83,8 @@ static const char * const mmu_types[] = { "RANGE" }; +#define EXP_TID_SET_EMPTY(set) (set.count == 0 && list_empty(&set.list)) + static inline int mmu_addr_cmp(struct mmu_rb_node *, unsigned long, unsigned long); static struct mmu_rb_node *mmu_rb_search_by_addr(struct rb_root *, @@ -94,6 +104,43 @@ static inline void mmu_notifier_range_start(struct mmu_notifier *, struct mm_struct *, unsigned long, unsigned long); +static inline void exp_tid_group_init(struct exp_tid_set *set) +{ + INIT_LIST_HEAD(&set->list); + set->count = 0; +} + +static inline void tid_group_remove(struct tid_group *grp, + struct exp_tid_set *set) +{ + list_del_init(&grp->list); + set->count--; +} + +static inline void tid_group_add_tail(struct tid_group *grp, + struct exp_tid_set *set) +{ + list_add_tail(&grp->list, &set->list); + set->count++; +} + +static inline struct tid_group *tid_group_pop(struct exp_tid_set *set) +{ + struct tid_group *grp = + list_first_entry(&set->list, struct tid_group, list); + list_del_init(&grp->list); + set->count--; + return grp; +} + +static inline void tid_group_move(struct tid_group *group, + struct exp_tid_set *s1, + struct exp_tid_set *s2) +{ + tid_group_remove(group, s1); + tid_group_add_tail(group, s2); +} + static struct mmu_notifier_ops __maybe_unused mn_opts = { .invalidate_page = mmu_notifier_page, .invalidate_range_start = mmu_notifier_range_start, @@ -114,6 +161,23 @@ int hfi1_user_exp_rcv_free(struct hfi1_filedata *fd) return -EINVAL; } +/* + * Write an "empty" RcvArray entry. + * This function exists so the TID registaration code can use it + * to write to unused/unneeded entries and still take advantage + * of the WC performance improvements. The HFI will ignore this + * write to the RcvArray entry. + */ +static inline void rcv_array_wc_fill(struct hfi1_devdata *dd, u32 index) +{ + /* + * Doing the WC fill writes only makes sense if the device is + * present and the RcvArray has been mapped as WC memory. + */ + if ((dd->flags & HFI1_PRESENT) && dd->rcvarray_wc) + writeq(0, dd->rcvarray_wc + (index * 8)); +} + int hfi1_user_exp_rcv_setup(struct file *fp, struct hfi1_tid_info *tinfo) { return -EINVAL;