From patchwork Tue May 23 09:37:47 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Donald Hunter X-Patchwork-Id: 13251993 X-Patchwork-Delegate: kuba@kernel.org Received: from lindbergh.monkeyblade.net (lindbergh.monkeyblade.net [23.128.96.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1DBF51E500 for ; Tue, 23 May 2023 09:38:10 +0000 (UTC) Received: from mail-qk1-x736.google.com (mail-qk1-x736.google.com [IPv6:2607:f8b0:4864:20::736]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 565CA119 for ; Tue, 23 May 2023 02:38:08 -0700 (PDT) Received: by mail-qk1-x736.google.com with SMTP id af79cd13be357-74e4f839ae4so365434385a.0 for ; Tue, 23 May 2023 02:38:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1684834687; x=1687426687; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=59RNToYcEgAWQQb6QpKGjchAYAYXRoRp8Ed2xVUT/rI=; b=bgTyCsTvxJiBEJdCgXXsd4xb2WtUYUZjtA6Of2ttq8CzFgEkNBJrtciooym4gxA594 bcuwCIvWVtMmMJumRSBYa0PdJcitTIuVv1WTyILTiCCTitvtRfVlefj2dcYgS2oMGPg7 1WvPuX29tBmN7McsftW/WQLZCRCdJC58xrTrWdBeQeezTylXo6CwAoj84Jv721miaAPZ uCAh3FNb3I8BcuQFdoLAkHOii/ckB8VaYvXg6EUPHmchotH2R7teis1FT7WIiBZwnmBf v5Nc96I1402vKlrE5P8CJqUlFsY8oZsi8/ys0Hsfgmi/xZtuDnhERxGh8QQYNrAAYQXy MkkQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684834687; x=1687426687; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=59RNToYcEgAWQQb6QpKGjchAYAYXRoRp8Ed2xVUT/rI=; b=GyzC1v0qg2Z/oI4CU9wKmjzAxzkkweq6gLoYpwhDF1CWLwrZHgNWTQZh5Iq265Ooxw owq7DJelWxpI8LHKVkBLW9oSuIfDWP51DietzE3T1SrFJ/U+WIk0HFAJE5veUmw3Psqr sb3+gsSVEFdbxo+RQe0PzUkROAiA5Snzw2214+y/JoItAeebX1Y+qBqGp0nJqaL6G/in rL6L3NgBFYmzZUAWg16lKpIVct3gPqcI5DS8Z/LFL0qK6/RtjXRr5+lB8tV5sPem4c+B MgHgtPWEm/uZ85iO5JcIRwKI3fvg4CV5iteGs65WRjGeGExM6umH105LukiZOOPeQp3c YjFg== X-Gm-Message-State: AC+VfDzOBO2oHxd/OSCW/oPCmIFizsibU6ihzUFUVTXLx+Jh03vHnZKv VOO5YwHEYWUWOYOxxGRzbukpPTgOEal2RQUl X-Google-Smtp-Source: ACHHUZ7DjrsokfqjW6xYtv4kPz489h8lGe94NY/dk7Zbpx4KHcbDIq3/LtBlFbgei5adH9R6BG8cuQ== X-Received: by 2002:a05:6214:492:b0:623:42c5:6125 with SMTP id pt18-20020a056214049200b0062342c56125mr29395413qvb.7.1684834686353; Tue, 23 May 2023 02:38:06 -0700 (PDT) Received: from imac.redhat.com ([88.97.103.74]) by smtp.gmail.com with ESMTPSA id x7-20020a05622a000700b003bf9f9f1844sm2758128qtw.71.2023.05.23.02.38.04 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 23 May 2023 02:38:05 -0700 (PDT) From: Donald Hunter To: netdev@vger.kernel.org, Jakub Kicinski , "David S. Miller" , Eric Dumazet , Paolo Abeni Cc: donald.hunter@redhat.com, Donald Hunter Subject: [PATCH net-next v2 1/2] tools: ynl: Use dict of predefined Structs to decode scalar types Date: Tue, 23 May 2023 10:37:47 +0100 Message-Id: <20230523093748.61518-2-donald.hunter@gmail.com> X-Mailer: git-send-email 2.39.0 In-Reply-To: <20230523093748.61518-1-donald.hunter@gmail.com> References: <20230523093748.61518-1-donald.hunter@gmail.com> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FREEMAIL_FROM, RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE, URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net X-Patchwork-Delegate: kuba@kernel.org Use a dict of predefined Struct() objects to decode scalar types in native, big or little endian format. This removes the repetitive code for the scalar variants and ensures all the signed variants are supported. Signed-off-by: Donald Hunter --- tools/net/ynl/lib/ynl.py | 101 +++++++++++++++++---------------------- 1 file changed, 44 insertions(+), 57 deletions(-) diff --git a/tools/net/ynl/lib/ynl.py b/tools/net/ynl/lib/ynl.py index aa77bcae4807..6185ba27f2e7 100644 --- a/tools/net/ynl/lib/ynl.py +++ b/tools/net/ynl/lib/ynl.py @@ -1,10 +1,12 @@ # SPDX-License-Identifier: GPL-2.0 OR BSD-3-Clause +from collections import namedtuple import functools import os import random import socket import struct +from struct import Struct import yaml from .nlspec import SpecFamily @@ -76,10 +78,17 @@ class NlError(Exception): class NlAttr: - type_formats = { 'u8' : ('B', 1), 's8' : ('b', 1), - 'u16': ('H', 2), 's16': ('h', 2), - 'u32': ('I', 4), 's32': ('i', 4), - 'u64': ('Q', 8), 's64': ('q', 8) } + ScalarFormat = namedtuple('ScalarFormat', ['native', 'big', 'little']) + type_formats = { + 'u8' : ScalarFormat(Struct('B'), Struct("B"), Struct("B")), + 's8' : ScalarFormat(Struct('b'), Struct("b"), Struct("b")), + 'u16': ScalarFormat(Struct('H'), Struct(">H"), Struct("h"), Struct("I"), Struct("i"), Struct("Q"), Struct("q"), Struct("" if byte_order == "big-endian" else "<" - return "" + return format.big if byte_order == "big-endian" \ + else format.little + return format.native - def as_u8(self): - return struct.unpack("B", self.raw)[0] - - def as_u16(self, byte_order=None): - endian = NlAttr.format_byte_order(byte_order) - return struct.unpack(f"{endian}H", self.raw)[0] - - def as_u32(self, byte_order=None): - endian = NlAttr.format_byte_order(byte_order) - return struct.unpack(f"{endian}I", self.raw)[0] - - def as_u64(self, byte_order=None): - endian = NlAttr.format_byte_order(byte_order) - return struct.unpack(f"{endian}Q", self.raw)[0] + def as_scalar(self, attr_type, byte_order=None): + format = self.get_format(attr_type, byte_order) + return format.unpack(self.raw)[0] def as_strz(self): return self.raw.decode('ascii')[:-1] @@ -115,17 +116,17 @@ class NlAttr: return self.raw def as_c_array(self, type): - format, _ = self.type_formats[type] - return list({ x[0] for x in struct.iter_unpack(format, self.raw) }) + format = self.get_format(type) + return [ x[0] for x in format.iter_unpack(self.raw) ] def as_struct(self, members): value = dict() offset = 0 for m in members: # TODO: handle non-scalar members - format, size = self.type_formats[m.type] - decoded = struct.unpack_from(format, self.raw, offset) - offset += size + format = self.get_format(m.type) + decoded = format.unpack_from(self.raw, offset) + offset += format.size value[m.name] = decoded[0] return value @@ -184,11 +185,11 @@ class NlMsg: if extack.type == Netlink.NLMSGERR_ATTR_MSG: self.extack['msg'] = extack.as_strz() elif extack.type == Netlink.NLMSGERR_ATTR_MISS_TYPE: - self.extack['miss-type'] = extack.as_u32() + self.extack['miss-type'] = extack.as_scalar('u32') elif extack.type == Netlink.NLMSGERR_ATTR_MISS_NEST: - self.extack['miss-nest'] = extack.as_u32() + self.extack['miss-nest'] = extack.as_scalar('u32') elif extack.type == Netlink.NLMSGERR_ATTR_OFFS: - self.extack['bad-attr-offs'] = extack.as_u32() + self.extack['bad-attr-offs'] = extack.as_scalar('u32') else: if 'unknown' not in self.extack: self.extack['unknown'] = [] @@ -272,11 +273,11 @@ def _genl_load_families(): fam = dict() for attr in gm.raw_attrs: if attr.type == Netlink.CTRL_ATTR_FAMILY_ID: - fam['id'] = attr.as_u16() + fam['id'] = attr.as_scalar('u16') elif attr.type == Netlink.CTRL_ATTR_FAMILY_NAME: fam['name'] = attr.as_strz() elif attr.type == Netlink.CTRL_ATTR_MAXATTR: - fam['maxattr'] = attr.as_u32() + fam['maxattr'] = attr.as_scalar('u32') elif attr.type == Netlink.CTRL_ATTR_MCAST_GROUPS: fam['mcast'] = dict() for entry in NlAttrs(attr.raw): @@ -286,7 +287,7 @@ def _genl_load_families(): if entry_attr.type == Netlink.CTRL_ATTR_MCAST_GRP_NAME: mcast_name = entry_attr.as_strz() elif entry_attr.type == Netlink.CTRL_ATTR_MCAST_GRP_ID: - mcast_id = entry_attr.as_u32() + mcast_id = entry_attr.as_scalar('u32') if mcast_name and mcast_id is not None: fam['mcast'][mcast_name] = mcast_id if 'name' in fam and 'id' in fam: @@ -304,9 +305,9 @@ class GenlMsg: self.fixed_header_attrs = dict() for m in fixed_header_members: - format, size = NlAttr.type_formats[m.type] - decoded = struct.unpack_from(format, nl_msg.raw, offset) - offset += size + format = NlAttr.get_format(m.type) + decoded = format.unpack_from(nl_msg.raw, offset) + offset += format.size self.fixed_header_attrs[m.name] = decoded[0] self.raw = nl_msg.raw[offset:] @@ -381,21 +382,13 @@ class YnlFamily(SpecFamily): attr_payload += self._add_attr(attr['nested-attributes'], subname, subvalue) elif attr["type"] == 'flag': attr_payload = b'' - elif attr["type"] == 'u8': - attr_payload = struct.pack("B", int(value)) - elif attr["type"] == 'u16': - endian = NlAttr.format_byte_order(attr.byte_order) - attr_payload = struct.pack(f"{endian}H", int(value)) - elif attr["type"] == 'u32': - endian = NlAttr.format_byte_order(attr.byte_order) - attr_payload = struct.pack(f"{endian}I", int(value)) - elif attr["type"] == 'u64': - endian = NlAttr.format_byte_order(attr.byte_order) - attr_payload = struct.pack(f"{endian}Q", int(value)) elif attr["type"] == 'string': attr_payload = str(value).encode('ascii') + b'\x00' elif attr["type"] == 'binary': attr_payload = value + elif attr['type'] in NlAttr.type_formats: + format = NlAttr.get_format(attr['type'], attr.byte_order) + attr_payload = format.pack(int(value)) else: raise Exception(f'Unknown type at {space} {name} {value} {attr["type"]}') @@ -434,22 +427,16 @@ class YnlFamily(SpecFamily): if attr_spec["type"] == 'nest': subdict = self._decode(NlAttrs(attr.raw), attr_spec['nested-attributes']) decoded = subdict - elif attr_spec['type'] == 'u8': - decoded = attr.as_u8() - elif attr_spec['type'] == 'u16': - decoded = attr.as_u16(attr_spec.byte_order) - elif attr_spec['type'] == 'u32': - decoded = attr.as_u32(attr_spec.byte_order) - elif attr_spec['type'] == 'u64': - decoded = attr.as_u64(attr_spec.byte_order) elif attr_spec["type"] == 'string': decoded = attr.as_strz() elif attr_spec["type"] == 'binary': decoded = self._decode_binary(attr, attr_spec) elif attr_spec["type"] == 'flag': decoded = True + elif attr_spec["type"] in NlAttr.type_formats: + decoded = attr.as_scalar(attr_spec['type'], attr_spec.byte_order) else: - raise Exception(f'Unknown {attr.type} {attr_spec["name"]} {attr_spec["type"]}') + raise Exception(f'Unknown {attr_spec["type"]} with name {attr_spec["name"]}') if not attr_spec.is_multi: rsp[attr_spec['name']] = decoded @@ -555,8 +542,8 @@ class YnlFamily(SpecFamily): fixed_header_members = self.consts[op.fixed_header].members for m in fixed_header_members: value = vals.pop(m.name) - format, _ = NlAttr.type_formats[m.type] - msg += struct.pack(format, value) + format = NlAttr.get_format(m.type) + msg += format.pack(value) for name, value in vals.items(): msg += self._add_attr(op.attr_set.name, name, value) msg = _genl_msg_finalize(msg) From patchwork Tue May 23 09:37:48 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Donald Hunter X-Patchwork-Id: 13251994 X-Patchwork-Delegate: kuba@kernel.org Received: from lindbergh.monkeyblade.net (lindbergh.monkeyblade.net [23.128.96.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 560421E500 for ; Tue, 23 May 2023 09:38:11 +0000 (UTC) Received: from mail-qt1-x836.google.com (mail-qt1-x836.google.com [IPv6:2607:f8b0:4864:20::836]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E0D47120 for ; Tue, 23 May 2023 02:38:09 -0700 (PDT) Received: by mail-qt1-x836.google.com with SMTP id d75a77b69052e-3f6a3a76665so12784131cf.1 for ; Tue, 23 May 2023 02:38:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1684834688; x=1687426688; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=JGqlSrjcDP9z0RjV/6OUxF9TUChHYrSZ7d1FAzFKzFU=; b=QKeSCGEExy6U0F48myNihsAVHYvUd10367+93vIvK4FKzaLd40YQzK9yOIS9iL8355 pZcG0MWpwhwAq8o3CmtiYbotNwj08gejIXrA/iCwNSbf3iL1b6CQTgWOTUq1Hgt66kZW obUvGnXxJtR08i68d7pZg4g92eWeRybt8FUs9QqBTR2rtjs50uYiFasDxqu+jBU+teD6 WhT4MgTZN7g3pJx3ztNsmpy4avENKRb3lXrXxmQH80nxnd2Ip4PWMiDBllS9Nq6WrjY1 uIh89BC8+2n/psCx87OghYhyyw4D2W8DoPy/lT4VNgszG38Ph+HEgUStGcjMm1LY3225 bb/Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684834688; x=1687426688; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=JGqlSrjcDP9z0RjV/6OUxF9TUChHYrSZ7d1FAzFKzFU=; b=j0PCuNl213Gt5D94r6rNY30r3xP4By8Xj4ujb42RNKcU3W/e1eK5o1j/6aZtK8oEjd nnlnY26DBI73pbeUSRCoA4FLuvUi6bJBHRzzv+eLyrmuQ7P2R05HSjweZwzFjyk1xnhj 0ScwvgcypYowof9zsMK9INlYU0WTYoMwYoa/0oLoKZtMK0k0cwNaKNuPs6giPH+ykWOD 2H70r9/tiAfwxPeXSm+ye4UgpqbbK7I54IsqBVlwu72lT5I15NkEn6UXZVVdGDoaOOYK 4bjyWpOjp+aclta01AhtbTAPonnSWJ894fHGvfnaP0gkWMX7g8LFlhdyV2PdLz/u84KH R0qg== X-Gm-Message-State: AC+VfDwEOIHS2Dk1BNpvMYscX8zAVbMQ6W82BDGfzKva+7XtNk0mTZU/ a82pD7usk91KtVqvyrlixdgMlDyFll0IemzJ X-Google-Smtp-Source: ACHHUZ4oBbQh7WLfzcdilZ9SbuYxGa1yNYV7IrUFpeg6cU1EbjvV1Ncs6alsS5P1k2tNro9TsvFhdg== X-Received: by 2002:a05:622a:138d:b0:3f6:adda:afd6 with SMTP id o13-20020a05622a138d00b003f6addaafd6mr9271079qtk.10.1684834688583; Tue, 23 May 2023 02:38:08 -0700 (PDT) Received: from imac.redhat.com ([88.97.103.74]) by smtp.gmail.com with ESMTPSA id x7-20020a05622a000700b003bf9f9f1844sm2758128qtw.71.2023.05.23.02.38.07 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 23 May 2023 02:38:08 -0700 (PDT) From: Donald Hunter To: netdev@vger.kernel.org, Jakub Kicinski , "David S. Miller" , Eric Dumazet , Paolo Abeni Cc: donald.hunter@redhat.com, Donald Hunter Subject: [PATCH net-next v2 2/2] tools: ynl: Handle byte-order in struct members Date: Tue, 23 May 2023 10:37:48 +0100 Message-Id: <20230523093748.61518-3-donald.hunter@gmail.com> X-Mailer: git-send-email 2.39.0 In-Reply-To: <20230523093748.61518-1-donald.hunter@gmail.com> References: <20230523093748.61518-1-donald.hunter@gmail.com> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FREEMAIL_FROM, RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE, URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net X-Patchwork-Delegate: kuba@kernel.org Add support for byte-order in struct members in the genetlink-legacy spec. Signed-off-by: Donald Hunter Acked-by: Jakub Kicinski --- Documentation/netlink/genetlink-legacy.yaml | 2 ++ tools/net/ynl/lib/nlspec.py | 4 +++- tools/net/ynl/lib/ynl.py | 6 +++--- 3 files changed, 8 insertions(+), 4 deletions(-) diff --git a/Documentation/netlink/genetlink-legacy.yaml b/Documentation/netlink/genetlink-legacy.yaml index b33541a51d6b..b5319cde9e17 100644 --- a/Documentation/netlink/genetlink-legacy.yaml +++ b/Documentation/netlink/genetlink-legacy.yaml @@ -122,6 +122,8 @@ properties: enum: [ u8, u16, u32, u64, s8, s16, s32, s64, string ] len: $ref: '#/$defs/len-or-define' + byte-order: + enum: [ little-endian, big-endian ] # End genetlink-legacy attribute-sets: diff --git a/tools/net/ynl/lib/nlspec.py b/tools/net/ynl/lib/nlspec.py index a0241add3839..c624cdfde223 100644 --- a/tools/net/ynl/lib/nlspec.py +++ b/tools/net/ynl/lib/nlspec.py @@ -226,11 +226,13 @@ class SpecStructMember(SpecElement): Represents a single struct member attribute. Attributes: - type string, type of the member attribute + type string, type of the member attribute + byte_order string or None for native byte order """ def __init__(self, family, yaml): super().__init__(family, yaml) self.type = yaml['type'] + self.byte_order = yaml.get('byte-order') class SpecStruct(SpecElement): diff --git a/tools/net/ynl/lib/ynl.py b/tools/net/ynl/lib/ynl.py index 6185ba27f2e7..39a2296c0003 100644 --- a/tools/net/ynl/lib/ynl.py +++ b/tools/net/ynl/lib/ynl.py @@ -124,7 +124,7 @@ class NlAttr: offset = 0 for m in members: # TODO: handle non-scalar members - format = self.get_format(m.type) + format = self.get_format(m.type, m.byte_order) decoded = format.unpack_from(self.raw, offset) offset += format.size value[m.name] = decoded[0] @@ -305,7 +305,7 @@ class GenlMsg: self.fixed_header_attrs = dict() for m in fixed_header_members: - format = NlAttr.get_format(m.type) + format = NlAttr.get_format(m.type, m.byte_order) decoded = format.unpack_from(nl_msg.raw, offset) offset += format.size self.fixed_header_attrs[m.name] = decoded[0] @@ -542,7 +542,7 @@ class YnlFamily(SpecFamily): fixed_header_members = self.consts[op.fixed_header].members for m in fixed_header_members: value = vals.pop(m.name) - format = NlAttr.get_format(m.type) + format = NlAttr.get_format(m.type, m.byte_order) msg += format.pack(value) for name, value in vals.items(): msg += self._add_attr(op.attr_set.name, name, value)