From patchwork Wed Apr 4 09:15:53 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alex Rosenbaum X-Patchwork-Id: 10322301 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 013EF60390 for ; Wed, 4 Apr 2018 09:15:59 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id E93CE28D5B for ; Wed, 4 Apr 2018 09:15:58 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id DDDE428E00; Wed, 4 Apr 2018 09:15:58 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.8 required=2.0 tests=BAYES_00, DKIM_ADSP_CUSTOM_MED, DKIM_SIGNED, FREEMAIL_FROM, RCVD_IN_DNSWL_HI, T_DKIM_INVALID autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id E190728D5B for ; Wed, 4 Apr 2018 09:15:57 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1750916AbeDDJP4 (ORCPT ); Wed, 4 Apr 2018 05:15:56 -0400 Received: from mail-it0-f66.google.com ([209.85.214.66]:40619 "EHLO mail-it0-f66.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1750826AbeDDJPz (ORCPT ); Wed, 4 Apr 2018 05:15:55 -0400 Received: by mail-it0-f66.google.com with SMTP id u62-v6so15791801ita.5 for ; Wed, 04 Apr 2018 02:15:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:from:date:message-id:subject:to:cc; bh=52gCJgqsZWB35m7MpQM00Rm7UPzActbhW8m+Ih7AcGk=; b=ccnNOBBjQmUdqVXHUUyHdiRDJqrUlcuENvgzBtUzhQLCNYrJElpOH+hSoeAPIQPn5k dvDTpdWbnVKcAm52dWJRGnhvb6jm+PuYeh8Je5vWANTdSFHnjyWU6Px0gXX3kOC2dSeD 1z+EGs1m7UuavtkF3eQUIB+O83NG6MTHaw2GcmsXl9rzlLcOww1p+fJRFYeO3VpaFNjL Z4LWdIXEoklxJ/MF4RVj/QgArbvIwICI7wI/1ZDlT27NTo+9UjqzmIBYQcxzFpKJH2wD UNyPVLqOpNqEhXjVFYMJej+lyU7NNBY+mXrS2fbeDZ9th50CfVxaXsVNK9u5fibrZiRh 8QeA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to:cc; bh=52gCJgqsZWB35m7MpQM00Rm7UPzActbhW8m+Ih7AcGk=; b=g0t/VmoPl63K0AG8ASDZUEbmtTc0nfIjuwQ/V/ngALj+ASPbSvz19m0SAsaAwvfOqh 8ytdn/6Otiopea/DVJYcfFFmksFTa2OqXgEtBLvoNlYCyfTum2IERHTmlFkMBkT0JezL w+HfB4cvo1QZKdb0KsdXn7xhKId1Yx37kKqvhxupTNptwXhWU5tQCWO7T2QfXvRtcpxu O/wF65bCAzZ0KSJQt9hg9IG03JYaARQLEhBRcxIMsTzYXkHgK/FHFRI2RiWa5LdARGqG 1Nl9zXmdPx0WvLth0dMSkdePf2MLzxq5MxjQHo7rkCEhJgffgvBQG4TzZYQg2Ghv6f9+ 1QAQ== X-Gm-Message-State: ALQs6tCPQnqtHHmVw+5S6gtmSA54TsJoxBSUHwY7IDtMLMpUb0+RtD0d 4nb8V0xFHK6kvmwb3CY3A9igOyBxEaPU/vVdsA2vnmYV X-Google-Smtp-Source: AIpwx4+YGi7JMR3qM2wI9epn+WOWZo2m9LImSXAAAt2f8csjmUte0RedMAr9qkmSidh9IiR1sKSSPviQCx/d9Kvl1Og= X-Received: by 2002:a24:2d0d:: with SMTP id x13-v6mr8262321itx.54.1522833354137; Wed, 04 Apr 2018 02:15:54 -0700 (PDT) MIME-Version: 1.0 Received: by 10.2.93.195 with HTTP; Wed, 4 Apr 2018 02:15:53 -0700 (PDT) From: Alex Rosenbaum Date: Wed, 4 Apr 2018 12:15:53 +0300 Message-ID: Subject: [PATCH RFC] New verbs API for flow action: ENCAP To: linux-rdma@vger.kernel.org Cc: Yishai Hadas , Leon Romanovsky , "Alex @ Mellanox" , Mark Bloch Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP This new verbs flow action allows the user to create an encapsulation operation to be done by the NIC and attach it to a steering entry as an action. ibv_create_flow_action_encap() is introduced which accepts the encapsulation protocol and a buffer which contains the encapsulating headers. We expose one variant at this time: IBV_FLOW_ACTION_ENCAP_RAW: Allows user to provide a custom/unspecified encapsulation protocol buffer. See example code which demo's usage of of IBV_FLOW_ACTION_ENCAP_RAW, to define an IPv4 VXLAN encapsulation header, in which case the user provides the following headers in the buffer: Ethernet, IPv4, UDP and VXLAN. Setting up an ENCAP flow action for an egress flow steering rule can fully offload software based route lookup of an overlay network with the addition of mapping an additional sge's when sending traffic on a RAW QP. Signed-off-by: Mark Bloch Signed-off-by: Alex Rosenbaum --- libibverbs/examples/encap_flow.c | 152 +++++++++++++++++++++++ libibverbs/man/ibv_create_flow_action_encap.3.md | 82 ++++++++++++ libibverbs/verbs.h | 24 ++++ 3 files changed, 258 insertions(+) create mode 100644 libibverbs/examples/encap_flow.c create mode 100644 libibverbs/man/ibv_create_flow_action_encap.3.md uint64_t dm_offset, size_t length, unsigned int access); @@ -1963,6 +1972,21 @@ ibv_modify_flow_action_esp(struct ibv_flow_action *action, return vctx->modify_flow_action_esp(action, esp); } +static inline struct ibv_flow_action * +ibv_create_flow_action_encap(struct ibv_context *ctx, + struct ibv_flow_action_encap_attr *encap) +{ + struct verbs_context *vctx = verbs_get_ctx_op(ctx, + create_flow_action_encap); + + if (!vctx) { + errno = ENOSYS; + return NULL; + } + + return vctx->create_flow_action_encap(ctx, encap); +} + static inline int ibv_destroy_flow_action(struct ibv_flow_action *action) { struct verbs_context *vctx = verbs_get_ctx_op(action->context, diff --git a/libibverbs/examples/encap_flow.c b/libibverbs/examples/encap_flow.c new file mode 100644 index 0000000..1e8e547 --- /dev/null +++ b/libibverbs/examples/encap_flow.c @@ -0,0 +1,152 @@ +/* + * Copyright (c) 2018 Mellanox Technologies, Ltd. All rights reserved. + * + * This software is available to you under a choice of one of two + * licenses. You may choose to be licensed under the terms of the GNU + * General Public License (GPL) Version 2, available from the file + * COPYING in the main directory of this source tree, or the + * OpenIB.org BSD license below: + * + * Redistribution and use in source and binary forms, with or + * without modification, are permitted provided that the following + * conditions are met: + * + * - Redistributions of source code must retain the above + * copyright notice, this list of conditions and the following + * disclaimer. + * + * - Redistributions in binary form must reproduce the above + * copyright notice, this list of conditions and the following + * disclaimer in the documentation and/or other materials + * provided with the distribution. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, + * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF + * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND + * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS + * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN + * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN + * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE + * SOFTWARE. + */ + +#include + +#include +#include + +#include //Provides declarations for ethernet header +#include //Provides declarations for udp header +#include //Provides declarations for ip header + +#include + +struct vxlanhdr { + __be32 flags; + __be32 vni; +}; + +static uint8_t buf[sizeof(struct ethhdr) + + sizeof(struct iphdr) + + sizeof(struct udphdr) + + sizeof(struct vxlanhdr)] = {}; + +struct ethhdr eth = {.h_source = {0x52, 0x54, 0x00, 0xf7, 0x56, 0xe2}, + .h_dest = {0x52, 0x54, 0x00, 0xf7, 0x56, 0xe2}, + .h_proto = 0x0800}; + +struct iphdr ip = {.version = 4, + .ihl = 5, + .tos = 0, + .tot_len = 0, + .id = 0x73a7, + .frag_off = 0x02, + .ttl = 64, + .protocol = 17, + .check = 0, + .saddr = (0x1d << 24) | (0x00 << 16) | (0x0a << 8) | 0x64, + .daddr = (0x1a << 24) | (0x00 << 16) | (0x0a << 8) | 0x78}; + +struct udphdr udp = {.source = 0, + .dest = 4789, + .len = 0, + .check = 0}; + +struct vxlanhdr vxlan = {.flags = 0x0800, + .vni = 0x0028}; + +static int create_encap_hdr(struct ibv_device *ib_dev) +{ + struct ibv_flow_action_encap_attr attr; + struct ibv_flow_action *action; + struct ethhdr *tmp_eth = (struct ethhdr *)buf; + struct iphdr *tmp_ip = (struct iphdr *)(tmp_eth + 1); + struct udphdr *tmp_udp = (struct udphdr *)(tmp_ip + 1); + struct vxlanhdr *tmp_vxlan = (struct vxlanhdr *)(tmp_udp + 1); + struct ibv_context *ctx; + int ret = 0; + + ctx = ibv_open_device(ib_dev); + if (!ctx) { + fprintf(stderr, "Failed to open device\n"); + return ret; + } + + memcpy(tmp_eth, ð, sizeof(*tmp_eth)); + memcpy(tmp_ip, &ip, sizeof(*tmp_ip)); + memcpy(tmp_udp, &udp, sizeof(*tmp_udp)); + memcpy(tmp_vxlan, &vxlan, sizeof(*tmp_vxlan)); + + attr.hdr_ptr = buf; + attr.hdr_len = sizeof(buf); + attr.hdr_proto = IBV_FLOW_ACTION_ENCAP_RAW; + + action = ibv_create_flow_action_encap(ctx, &attr); + + printf("%s- got action %p\n", __func__, action); + if (action) + ibv_destroy_flow_action(action); + + ibv_close_device(ctx); + + return ret; +} + +int main(int argc, char *argv[]) +{ + struct ibv_device **dev_list; + struct ibv_device *ib_dev; + char *ib_devname = NULL; + int ret = 1; + int i; + + dev_list = ibv_get_device_list(NULL); + + if (!dev_list) { + perror("Failed to get IB devices list"); + return 1; + } + + if (argc != 2) { + perror("Didn't provide IB device name"); + goto free_dev_list; + } + + ib_devname = argv[1]; + for (i = 0; dev_list[i]; ++i) + if (!strcmp(ibv_get_device_name(dev_list[i]), ib_devname)) + break; + + ib_dev = dev_list[i]; + if (!ib_dev) { + fprintf(stderr, "IB device %s not found\n", ib_devname); + goto free_dev_list; + } + + ret = create_encap_hdr(ib_dev); + +free_dev_list: + ibv_free_device_list(dev_list); + + return ret; +} diff --git a/libibverbs/man/ibv_create_flow_action_encap.3.md b/libibverbs/man/ibv_create_flow_action_encap.3.md new file mode 100644 index 0000000..5ff18d9 --- /dev/null +++ b/libibverbs/man/ibv_create_flow_action_encap.3.md @@ -0,0 +1,82 @@ +--- +date: 2018-03-27 +footer: libibverbs +header: "Libibverbs Programmer's Manual" +layout: page +license: 'Licensed under the OpenIB.org BSD license (FreeBSD Variant) - See COPYING.md' +section: 3 +title: ibv_flow_action_encap(3) +tagline: Verbs +--- + +# NAME + +ibv_flow_action_encap(3) -- Flow action encap for verbs(3) + +# SYNOPSIS + +```c +#include + +struct ibv_flow_action * +ibv_create_flow_action_encap(struct ibv_context *ctx, + struct ibv_flow_action_encap_attr *attr); + +int ibv_destroy_flow_action(struct ibv_flow_action *action); +``` + +# DESCRIPTION + +An ENCAP flow steering action allows a flow steering rule to encapsulate +a packet after matching. + +After the action is created, it should be associated with a *struct +ibv_flow_attr* using *struct ibv_flow_spec_action_handle* flow specification. +An action can be associated with multiple flows. + +# ARGUMENTS + +*ctx* +: RDMA device context to create the action on. + +*attr* +: ENCAP type and header to be used. + +*action* +: Existing action to be destroyed. + +## *action* Argument + +```c +struct ibv_flow_action_encap_attr { + enum ibv_flow_action_encap hdr_proto; + uint16_t hdr_len; + void *hdr_ptr; +}; +``` + +*hdr_proto* +: Encapsulation protocol to be used. + +*hdr_len* +: Length in bytes of the provided encap headers. + +*hdr_ptr* +: Buffing containing the encap headers. + +## Encapsulate protocols (*ibv_flow_action_encap*) + +A user can create flow actions that implement different encapsulation +protocols. + +*IBV_FLOW_ACTION_ENCAP_RAW* provides the ability to provide unspecified +encapsulation protocol, this can be expressed by passing it in *hdr_proto*. + +# RETURN VALUE + +Upon success *ibv_create_flow_action_encap* will return a new *struct +ibv_flow_action* object, on error NULL will be returned and errno will be set. + +# SEE ALSO + +*ibv_create_flow(3)*, *ibv_destroy_flow_action(3)* diff --git a/libibverbs/verbs.h b/libibverbs/verbs.h index eb57824..2f5fc8f 100644 --- a/libibverbs/verbs.h +++ b/libibverbs/verbs.h @@ -44,6 +44,7 @@ #include #include #include +#include #ifdef __cplusplus #include @@ -1583,6 +1584,12 @@ struct ibv_flow_action_esp_attr { uint32_t esn; }; +struct ibv_flow_action_encap_attr { + enum ibv_flow_action_encap hdr_proto; + uint16_t hdr_len; + void *hdr_ptr; +}; + struct ibv_device; struct ibv_context; @@ -1737,6 +1737,8 @@ struct ibv_values_ex { struct verbs_context { /* "grows up" - new fields go here */ + struct ibv_flow_action *(*create_flow_action_encap)(struct ibv_context *context, + struct ibv_flow_action_encap_attr *attr); struct ibv_mr *(*reg_dm_mr)(struct ibv_pd *pd, struct ibv_dm *dm,