[RFC,v6,2/6] dpll: Add DPLL framework base functions

Commit Message

Vadim Fedorenko March 12, 2023, 2:28 a.m. UTC

DPLL framework is used to represent and configure DPLL devices
in systems. Each device that has DPLL and can configure sources
and outputs can use this framework. Netlink interface is used to
provide configuration data and to receive notification messages
about changes in the configuration or status of DPLL device.
Inputs and outputs of the DPLL device are represented as special
objects which could be dynamically added to and removed from DPLL
device.

Changes:
dpll: adapt changes after introduction of dpll yaml spec
dpll: redesign after review comments, fix minor issues
dpll: add get pin command
dpll: _get/_put approach for creating and realesing pin or dpll objects
dpll: lock access to dplls with global lock
dpll: lock access to pins with global lock

dpll: replace cookie with clock id
dpll: add clock class

Provide userspace with clock class value of DPLL with dpll device dump
netlink request. Clock class is assigned by driver allocating a dpll
device. Clock class values are defined as specified in:
ITU-T G.8273.2/Y.1368.2 recommendation.

dpll: follow one naming schema in dpll subsys

dpll: fix dpll device naming scheme

Fix dpll device naming scheme by use of new pattern.
"dpll_%s_%d_%d", where:
- %s - dev_name(parent) of parent device,
- %d (1) - enum value of dpll type,
- %d (2) - device index provided by parent device.

dpll: remove description length parameter

dpll: fix muxed/shared pin registration

Let the kernel module to register a shared or muxed pin without finding
it or its parent. Instead use a parent/shared pin description to find
correct pin internally in dpll_core, simplifing a dpll API.

dpll: move function comments to dpll_core.c, fix exports
dpll: remove single-use helper functions
dpll: merge device register with alloc
dpll: lock and unlock mutex on dpll device release
dpll: move dpll_type to uapi header
dpll: rename DPLLA_DUMP_FILTER to DPLLA_FILTER
dpll: rename dpll_pin_state to dpll_pin_mode
dpll: rename DPLL_MODE_FORCED to DPLL_MODE_MANUAL
dpll: remove DPLL_CHANGE_PIN_TYPE enum value

Co-developed-by: Milena Olech <[e-mail]>
Signed-off-by: Milena Olech <[e-mail]>
Co-developed-by: Michal Michalik <[e-mail]>
Signed-off-by: Michal Michalik <[e-mail]>
Co-developed-by: Arkadiusz Kubalewski <[e-mail]>
Signed-off-by: Arkadiusz Kubalewski <[e-mail]>
Signed-off-by: Vadim Fedorenko <[e-mail]>
---
 MAINTAINERS                 |    9 +
 drivers/Kconfig             |    2 +
 drivers/Makefile            |    1 +
 drivers/dpll/Kconfig        |    7 +
 drivers/dpll/Makefile       |   10 +
 drivers/dpll/dpll_core.c    |  835 +++++++++++++++++++++++++++
 drivers/dpll/dpll_core.h    |   99 ++++
 drivers/dpll/dpll_netlink.c | 1065 +++++++++++++++++++++++++++++++++++
 drivers/dpll/dpll_netlink.h |   30 +
 include/linux/dpll.h        |  284 ++++++++++
 10 files changed, 2342 insertions(+)
 create mode 100644 drivers/dpll/Kconfig
 create mode 100644 drivers/dpll/Makefile
 create mode 100644 drivers/dpll/dpll_core.c
 create mode 100644 drivers/dpll/dpll_core.h
 create mode 100644 drivers/dpll/dpll_netlink.c
 create mode 100644 drivers/dpll/dpll_netlink.h
 create mode 100644 include/linux/dpll.h

Comments

Jiri Pirko March 13, 2023, 4:21 p.m. UTC | #1

Sun, Mar 12, 2023 at 03:28:03AM CET, [e-mail] wrote:
>DPLL framework is used to represent and configure DPLL devices
>in systems. Each device that has DPLL and can configure sources
>and outputs can use this framework. Netlink interface is used to
>provide configuration data and to receive notification messages
>about changes in the configuration or status of DPLL device.
>Inputs and outputs of the DPLL device are represented as special
>objects which could be dynamically added to and removed from DPLL
>device.
>
>Changes:
>dpll: adapt changes after introduction of dpll yaml spec
>dpll: redesign after review comments, fix minor issues
>dpll: add get pin command
>dpll: _get/_put approach for creating and realesing pin or dpll objects
>dpll: lock access to dplls with global lock
>dpll: lock access to pins with global lock
>
>dpll: replace cookie with clock id
>dpll: add clock class

Looks like some leftover in patch description.


>
>Provide userspace with clock class value of DPLL with dpll device dump
>netlink request. Clock class is assigned by driver allocating a dpll
>device. Clock class values are defined as specified in:
>ITU-T G.8273.2/Y.1368.2 recommendation.
>
>dpll: follow one naming schema in dpll subsys
>
>dpll: fix dpll device naming scheme
>
>Fix dpll device naming scheme by use of new pattern.
>"dpll_%s_%d_%d", where:
>- %s - dev_name(parent) of parent device,
>- %d (1) - enum value of dpll type,
>- %d (2) - device index provided by parent device.
>
>dpll: remove description length parameter
>
>dpll: fix muxed/shared pin registration
>
>Let the kernel module to register a shared or muxed pin without finding
>it or its parent. Instead use a parent/shared pin description to find
>correct pin internally in dpll_core, simplifing a dpll API.
>
>dpll: move function comments to dpll_core.c, fix exports
>dpll: remove single-use helper functions
>dpll: merge device register with alloc
>dpll: lock and unlock mutex on dpll device release
>dpll: move dpll_type to uapi header
>dpll: rename DPLLA_DUMP_FILTER to DPLLA_FILTER
>dpll: rename dpll_pin_state to dpll_pin_mode
>dpll: rename DPLL_MODE_FORCED to DPLL_MODE_MANUAL
>dpll: remove DPLL_CHANGE_PIN_TYPE enum value

I'm confused, some messed-up squash?


>
>Co-developed-by: Milena Olech <[e-mail]>
>Signed-off-by: Milena Olech <[e-mail]>
>Co-developed-by: Michal Michalik <[e-mail]>
>Signed-off-by: Michal Michalik <[e-mail]>
>Co-developed-by: Arkadiusz Kubalewski <[e-mail]>
>Signed-off-by: Arkadiusz Kubalewski <[e-mail]>
>Signed-off-by: Vadim Fedorenko <[e-mail]>
>---
> MAINTAINERS                 |    9 +
> drivers/Kconfig             |    2 +
> drivers/Makefile            |    1 +
> drivers/dpll/Kconfig        |    7 +
> drivers/dpll/Makefile       |   10 +
> drivers/dpll/dpll_core.c    |  835 +++++++++++++++++++++++++++
> drivers/dpll/dpll_core.h    |   99 ++++
> drivers/dpll/dpll_netlink.c | 1065 +++++++++++++++++++++++++++++++++++
> drivers/dpll/dpll_netlink.h |   30 +
> include/linux/dpll.h        |  284 ++++++++++
> 10 files changed, 2342 insertions(+)
> create mode 100644 drivers/dpll/Kconfig
> create mode 100644 drivers/dpll/Makefile
> create mode 100644 drivers/dpll/dpll_core.c
> create mode 100644 drivers/dpll/dpll_core.h
> create mode 100644 drivers/dpll/dpll_netlink.c
> create mode 100644 drivers/dpll/dpll_netlink.h
> create mode 100644 include/linux/dpll.h
>
>diff --git a/MAINTAINERS b/MAINTAINERS
>index edd3d562beee..0222b19af545 100644
>--- a/MAINTAINERS
>+++ b/MAINTAINERS
>@@ -6289,6 +6289,15 @@ F:	Documentation/networking/device_drivers/ethernet/freescale/dpaa2/switch-drive
> F:	drivers/net/ethernet/freescale/dpaa2/dpaa2-switch*
> F:	drivers/net/ethernet/freescale/dpaa2/dpsw*
> 
>+DPLL CLOCK SUBSYSTEM
>+M:	Vadim Fedorenko <[e-mail]>
>+M:	Arkadiusz Kubalewski <[e-mail]>
>+L:	[e-mail]
>+S:	Maintained
>+F:	drivers/dpll/*
>+F:	include/net/dpll.h
>+F:	include/uapi/linux/dpll.h
>+
> DRBD DRIVER
> M:	Philipp Reisner <[e-mail]>
> M:	Lars Ellenberg <[e-mail]>
>diff --git a/drivers/Kconfig b/drivers/Kconfig
>index 968bd0a6fd78..453df9e1210d 100644
>--- a/drivers/Kconfig
>+++ b/drivers/Kconfig
>@@ -241,4 +241,6 @@ source "drivers/peci/Kconfig"
> 
> source "drivers/hte/Kconfig"
> 
>+source "drivers/dpll/Kconfig"
>+
> endmenu
>diff --git a/drivers/Makefile b/drivers/Makefile
>index 20b118dca999..9ffb554507ef 100644
>--- a/drivers/Makefile
>+++ b/drivers/Makefile
>@@ -194,3 +194,4 @@ obj-$(CONFIG_MOST)		+= most/
> obj-$(CONFIG_PECI)		+= peci/
> obj-$(CONFIG_HTE)		+= hte/
> obj-$(CONFIG_DRM_ACCEL)		+= accel/
>+obj-$(CONFIG_DPLL)		+= dpll/
>diff --git a/drivers/dpll/Kconfig b/drivers/dpll/Kconfig
>new file mode 100644
>index 000000000000..a4cae73f20d3
>--- /dev/null
>+++ b/drivers/dpll/Kconfig
>@@ -0,0 +1,7 @@
>+# SPDX-License-Identifier: GPL-2.0-only
>+#
>+# Generic DPLL drivers configuration
>+#
>+
>+config DPLL
>+  bool
>diff --git a/drivers/dpll/Makefile b/drivers/dpll/Makefile
>new file mode 100644
>index 000000000000..d3926f2a733d
>--- /dev/null
>+++ b/drivers/dpll/Makefile
>@@ -0,0 +1,10 @@
>+# SPDX-License-Identifier: GPL-2.0
>+#
>+# Makefile for DPLL drivers.
>+#
>+
>+obj-$(CONFIG_DPLL)          += dpll_sys.o

What's "sys" and why is it here?


>+dpll_sys-y                  += dpll_core.o
>+dpll_sys-y                  += dpll_netlink.o
>+dpll_sys-y                  += dpll_nl.o
>+
>diff --git a/drivers/dpll/dpll_core.c b/drivers/dpll/dpll_core.c
>new file mode 100644
>index 000000000000..3fc151e16751
>--- /dev/null
>+++ b/drivers/dpll/dpll_core.c
>@@ -0,0 +1,835 @@
>+// SPDX-License-Identifier: GPL-2.0
>+/*
>+ *  dpll_core.c - Generic DPLL Management class support.
>+ *
>+ *  Copyright (c) 2021 Meta Platforms, Inc. and affiliates

IIUC, there's a lot of Intel work behind this, I think some credits
should go here as well.

Also, it is 2023, you live in the past :)


>+ */
>+
>+#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
>+
>+#include <linux/device.h>
>+#include <linux/err.h>
>+#include <linux/slab.h>
>+#include <linux/string.h>
>+
>+#include "dpll_core.h"
>+
>+DEFINE_MUTEX(dpll_device_xa_lock);
>+DEFINE_MUTEX(dpll_pin_xa_lock);
>+
>+DEFINE_XARRAY_FLAGS(dpll_device_xa, XA_FLAGS_ALLOC);
>+DEFINE_XARRAY_FLAGS(dpll_pin_xa, XA_FLAGS_ALLOC);
>+
>+#define ASSERT_DPLL_REGISTERED(d)                                          \
>+	WARN_ON_ONCE(!xa_get_mark(&dpll_device_xa, (d)->id, DPLL_REGISTERED))
>+#define ASSERT_DPLL_NOT_REGISTERED(d)                                      \
>+	WARN_ON_ONCE(xa_get_mark(&dpll_device_xa, (d)->id, DPLL_REGISTERED))
>+
>+static struct class dpll_class = {
>+	.name = "dpll",
>+};
>+
>+/**
>+ * dpll_device_get_by_id - find dpll device by it's id
>+ * @id: id of searched dpll
>+ *
>+ * Return:
>+ * * dpll_device struct if found
>+ * * NULL otherwise
>+ */
>+struct dpll_device *dpll_device_get_by_id(int id)
>+{
>+	struct dpll_device *dpll = NULL;
>+
>+	if (xa_get_mark(&dpll_device_xa, id, DPLL_REGISTERED))
>+		dpll = xa_load(&dpll_device_xa, id);
		return xa_load

>+
	return NULL

>+	return dpll;

Anyway, I believe this fuction should go away, see below.


>+}
>+
>+/**
>+ * dpll_device_get_by_name - find dpll device by it's id
>+ * @bus_name: bus name of searched dpll
>+ * @dev_name: dev name of searched dpll
>+ *
>+ * Return:
>+ * * dpll_device struct if found
>+ * * NULL otherwise
>+ */
>+struct dpll_device *
>+dpll_device_get_by_name(const char *bus_name, const char *device_name)
>+{
>+	struct dpll_device *dpll, *ret = NULL;
>+	unsigned long index;
>+
>+	xa_for_each_marked(&dpll_device_xa, index, dpll, DPLL_REGISTERED) {
>+		if (!strcmp(dev_bus_name(&dpll->dev), bus_name) &&
>+		    !strcmp(dev_name(&dpll->dev), device_name)) {
>+			ret = dpll;
>+			break;
>+		}
>+	}
>+
>+	return ret;
>+}
>+
>+/**
>+ * dpll_xa_ref_pin_add - add pin reference to a given xarray
>+ * @xa_pins: dpll_pin_ref xarray holding pins
>+ * @pin: pin being added
>+ * @ops: ops for a pin
>+ * @priv: pointer to private data of owner
>+ *
>+ * Allocate and create reference of a pin or increase refcount on existing pin
>+ * reference on given xarray.
>+ *
>+ * Return:
>+ * * 0 on success
>+ * * -ENOMEM on failed allocation
>+ */
>+static int
>+dpll_xa_ref_pin_add(struct xarray *xa_pins, struct dpll_pin *pin,
>+		    struct dpll_pin_ops *ops, void *priv)
>+{
>+	struct dpll_pin_ref *ref;
>+	unsigned long i;

In previous function you use "index" name for the same variable. Be
consistent. I think "i" is actually better.


>+	u32 idx;
>+	int ret;
>+
>+	xa_for_each(xa_pins, i, ref) {
>+		if (ref->pin == pin) {
>+			refcount_inc(&ref->refcount);
>+			return 0;
>+		}
>+	}
>+
>+	ref = kzalloc(sizeof(*ref), GFP_KERNEL);
>+	if (!ref)
>+		return -ENOMEM;
>+	ref->pin = pin;
>+	ref->ops = ops;
>+	ref->priv = priv;
>+	ret = xa_alloc(xa_pins, &idx, ref, xa_limit_16b, GFP_KERNEL);
>+	if (!ret)
>+		refcount_set(&ref->refcount, 1);
>+	else
>+		kfree(ref);
>+
>+	return ret;
>+}
>+
>+/**
>+ * dpll_xa_ref_pin_del - remove reference of a pin from xarray
>+ * @xa_pins: dpll_pin_ref xarray holding pins
>+ * @pin: pointer to a pin
>+ *
>+ * Decrement refcount of existing pin reference on given xarray.
>+ * If all references are dropped, delete the reference and free its memory.
>+ *
>+ * Return:
>+ * * 0 on success
>+ * * -EINVAL if reference to a pin was not found
>+ */
>+static int dpll_xa_ref_pin_del(struct xarray *xa_pins, struct dpll_pin *pin)
>+{
>+	struct dpll_pin_ref *ref;
>+	unsigned long i;
>+
>+	xa_for_each(xa_pins, i, ref) {
>+		if (ref->pin == pin) {
>+			if (refcount_dec_and_test(&ref->refcount)) {
>+				xa_erase(xa_pins, i);
>+				kfree(ref);
>+			}
>+			return 0;
>+		}
>+	}
>+
>+	return -EINVAL;
>+}
>+
>+/**
>+ * dpll_xa_ref_pin_find - find pin reference on xarray
>+ * @xa_pins: dpll_pin_ref xarray holding pins
>+ * @pin: pointer to a pin
>+ *
>+ * Search for pin reference struct of a given pin on given xarray.
>+ *
>+ * Return:
>+ * * pin reference struct pointer on success
>+ * * NULL - reference to a pin was not found
>+ */
>+struct dpll_pin_ref *
>+dpll_xa_ref_pin_find(struct xarray *xa_pins, const struct dpll_pin *pin)
>+{
>+	struct dpll_pin_ref *ref;
>+	unsigned long i;
>+
>+	xa_for_each(xa_pins, i, ref) {
>+		if (ref->pin == pin)
>+			return ref;
>+	}
>+
>+	return NULL;
>+}
>+
>+/**
>+ * dpll_xa_ref_dpll_add - add dpll reference to a given xarray
>+ * @xa_dplls: dpll_pin_ref xarray holding dplls
>+ * @dpll: dpll being added
>+ * @ops: pin-reference ops for a dpll
>+ * @priv: pointer to private data of owner
>+ *
>+ * Allocate and create reference of a dpll-pin ops or increase refcount
>+ * on existing dpll reference on given xarray.
>+ *
>+ * Return:
>+ * * 0 on success
>+ * * -ENOMEM on failed allocation
>+ */
>+static int
>+dpll_xa_ref_dpll_add(struct xarray *xa_dplls, struct dpll_device *dpll,
>+		     struct dpll_pin_ops *ops, void *priv)
>+{
>+	struct dpll_pin_ref *ref;
>+	unsigned long i;
>+	u32 idx;
>+	int ret;
>+
>+	xa_for_each(xa_dplls, i, ref) {
>+		if (ref->dpll == dpll) {
>+			refcount_inc(&ref->refcount);
>+			return 0;
>+		}
>+	}
>+	ref = kzalloc(sizeof(*ref), GFP_KERNEL);
>+	if (!ref)
>+		return -ENOMEM;
>+	ref->dpll = dpll;
>+	ref->ops = ops;
>+	ref->priv = priv;
>+	ret = xa_alloc(xa_dplls, &idx, ref, xa_limit_16b, GFP_KERNEL);
>+	if (!ret)
>+		refcount_set(&ref->refcount, 1);
>+	else
>+		kfree(ref);
>+
>+	return ret;
>+}
>+
>+/**
>+ * dpll_xa_ref_dpll_del - remove reference of a dpll from xarray
>+ * @xa_dplls: dpll_pin_ref xarray holding dplls
>+ * @dpll: pointer to a dpll to remove
>+ *
>+ * Decrement refcount of existing dpll reference on given xarray.
>+ * If all references are dropped, delete the reference and free its memory.
>+ */
>+static void
>+dpll_xa_ref_dpll_del(struct xarray *xa_dplls, struct dpll_device *dpll)
>+{
>+	struct dpll_pin_ref *ref;
>+	unsigned long i;
>+
>+	xa_for_each(xa_dplls, i, ref) {
>+		if (ref->dpll == dpll) {
>+			if (refcount_dec_and_test(&ref->refcount)) {
>+				xa_erase(xa_dplls, i);
>+				kfree(ref);
>+			}
>+			break;
>+		}
>+	}
>+}
>+
>+/**
>+ * dpll_xa_ref_dpll_find - find dpll reference on xarray
>+ * @xa_dplls: dpll_pin_ref xarray holding dplls
>+ * @dpll: pointer to a dpll
>+ *
>+ * Search for dpll-pin ops reference struct of a given dpll on given xarray.
>+ *
>+ * Return:
>+ * * pin reference struct pointer on success
>+ * * NULL - reference to a pin was not found
>+ */
>+struct dpll_pin_ref *
>+dpll_xa_ref_dpll_find(struct xarray *xa_refs, const struct dpll_device *dpll)
>+{
>+	struct dpll_pin_ref *ref;
>+	unsigned long i;
>+
>+	xa_for_each(xa_refs, i, ref) {
>+		if (ref->dpll == dpll)
>+			return ref;
>+	}
>+
>+	return NULL;
>+}
>+
>+
>+/**
>+ * dpll_device_alloc - allocate the memory for dpll device
>+ * @clock_id: clock_id of creator
>+ * @dev_driver_id: id given by dev driver
>+ * @module: reference to registering module
>+ *
>+ * Allocates memory and initialize dpll device, hold its reference on global
>+ * xarray.
>+ *
>+ * Return:
>+ * * dpll_device struct pointer if succeeded
>+ * * ERR_PTR(X) - failed allocation
>+ */
>+struct dpll_device *
>+dpll_device_alloc(const u64 clock_id, u32 dev_driver_id, struct module *module)

This should be static. Didn't you get warn?


>+{
>+	struct dpll_device *dpll;
>+	int ret;
>+
>+	dpll = kzalloc(sizeof(*dpll), GFP_KERNEL);
>+	if (!dpll)
>+		return ERR_PTR(-ENOMEM);
>+	refcount_set(&dpll->refcount, 1);
>+	dpll->dev.class = &dpll_class;
>+	dpll->dev_driver_id = dev_driver_id;
>+	dpll->clock_id = clock_id;
>+	dpll->module = module;
>+	ret = xa_alloc(&dpll_device_xa, &dpll->id, dpll,
>+		       xa_limit_16b, GFP_KERNEL);
>+	if (ret) {
>+		kfree(dpll);
>+		return ERR_PTR(ret);
>+	}
>+	xa_init_flags(&dpll->pin_refs, XA_FLAGS_ALLOC);
>+
>+	return dpll;
>+}
>+
>+/**
>+ * dpll_device_get - find existing or create new dpll device
>+ * @clock_id: clock_id of creator
>+ * @dev_driver_id: id given by dev driver
>+ * @module: reference to registering module
>+ *
>+ * Get existing object of a dpll device, unique for given arguments.
>+ * Create new if doesn't exist yet.
>+ *
>+ * Return:
>+ * * valid dpll_device struct pointer if succeeded
>+ * * ERR_PTR of an error
>+ */
>+struct dpll_device *
>+dpll_device_get(u64 clock_id, u32 dev_driver_id, struct module *module)
>+{
>+	struct dpll_device *dpll, *ret = NULL;
>+	unsigned long index;
>+
>+	mutex_lock(&dpll_device_xa_lock);
>+	xa_for_each(&dpll_device_xa, index, dpll) {
>+		if (dpll->clock_id == clock_id &&
>+		    dpll->dev_driver_id == dev_driver_id &&

Why you need "dev_driver_id"? clock_id is here for the purpose of
identification, isn't that enough for you.

Plus, the name is odd. "dev_driver" should certainly be avoided.


>+		    dpll->module == module) {
>+			ret = dpll;
>+			refcount_inc(&ret->refcount);
>+			break;
>+		}
>+	}
>+	if (!ret)
>+		ret = dpll_device_alloc(clock_id, dev_driver_id, module);
>+	mutex_unlock(&dpll_device_xa_lock);
>+
>+	return ret;
>+}
>+EXPORT_SYMBOL_GPL(dpll_device_get);
>+
>+/**
>+ * dpll_device_put - decrease the refcount and free memory if possible
>+ * @dpll: dpll_device struct pointer
>+ *
>+ * Drop reference for a dpll device, if all references are gone, delete
>+ * dpll device object.
>+ */
>+void dpll_device_put(struct dpll_device *dpll)
>+{
>+	if (!dpll)
>+		return;

Remove this check. The driver should not call this with NULL.


>+	mutex_lock(&dpll_device_xa_lock);
>+	if (refcount_dec_and_test(&dpll->refcount)) {
>+		WARN_ON_ONCE(!xa_empty(&dpll->pin_refs));

ASSERT_DPLL_NOT_REGISTERED(dpll);


>+		xa_destroy(&dpll->pin_refs);
>+		xa_erase(&dpll_device_xa, dpll->id);
>+		kfree(dpll);
>+	}
>+	mutex_unlock(&dpll_device_xa_lock);
>+}
>+EXPORT_SYMBOL_GPL(dpll_device_put);
>+
>+/**
>+ * dpll_device_register - register the dpll device in the subsystem
>+ * @dpll: pointer to a dpll
>+ * @type: type of a dpll
>+ * @ops: ops for a dpll device
>+ * @priv: pointer to private information of owner
>+ * @owner: pointer to owner device
>+ *
>+ * Make dpll device available for user space.
>+ *
>+ * Return:
>+ * * 0 on success
>+ * * -EINVAL on failure
>+ */
>+int dpll_device_register(struct dpll_device *dpll, enum dpll_type type,
>+			 struct dpll_device_ops *ops, void *priv,
>+			 struct device *owner)
>+{
>+	if (WARN_ON(!ops || !owner))
>+		return -EINVAL;
>+	if (WARN_ON(type <= DPLL_TYPE_UNSPEC || type > DPLL_TYPE_MAX))
>+		return -EINVAL;
>+	mutex_lock(&dpll_device_xa_lock);
>+	if (ASSERT_DPLL_NOT_REGISTERED(dpll)) {
>+		mutex_unlock(&dpll_device_xa_lock);
>+		return -EEXIST;
>+	}
>+	dpll->dev.bus = owner->bus;
>+	dpll->parent = owner;
>+	dpll->type = type;
>+	dpll->ops = ops;
>+	dev_set_name(&dpll->dev, "%s_%d", dev_name(owner),
>+		     dpll->dev_driver_id);

This is really odd. As a result, the user would see something like:
pci/0000:01:00.0_1
pci/0000:01:00.0_2

I have to say it is confusing. In devlink, is bus/name and the user
could use this info to look trough sysfs. Here, 0000:01:00.0_1 is not
there. Also, "_" might have some meaning on some bus. Should not
concatename dev_name() with anything.

Thinking about this some more, the module/clock_id tuple should be
uniqueue and stable. It is used for dpll_device_get(), it could be used
as the user handle, can't it?
Example:
ice/c92d02a7129f4747
mlx5/90265d8bf6e6df56

If you really need the "dev_driver_id" (as I believe clock_id should be
enough), you can put it here as well:
ice/c92d02a7129f4747/1
ice/c92d02a7129f4747/2

This would also be beneficial for mlx5, as mlx5 with 2 PFs would like to
share instance of DPLL equally, there is no "one clock master".



>+	dpll->priv = priv;
>+	xa_set_mark(&dpll_device_xa, dpll->id, DPLL_REGISTERED);
>+	mutex_unlock(&dpll_device_xa_lock);
>+	dpll_notify_device_create(dpll);
>+
>+	return 0;
>+}
>+EXPORT_SYMBOL_GPL(dpll_device_register);
>+
>+/**
>+ * dpll_device_unregister - deregister dpll device
>+ * @dpll: registered dpll pointer
>+ *
>+ * Deregister device, make it unavailable for userspace.
>+ * Note: It does not free the memory
>+ */
>+void dpll_device_unregister(struct dpll_device *dpll)
>+{
>+	mutex_lock(&dpll_device_xa_lock);
>+	ASSERT_DPLL_REGISTERED(dpll);
>+	xa_clear_mark(&dpll_device_xa, dpll->id, DPLL_REGISTERED);
>+	mutex_unlock(&dpll_device_xa_lock);
>+	dpll_notify_device_delete(dpll);
>+}
>+EXPORT_SYMBOL_GPL(dpll_device_unregister);
>+
>+/**
>+ * dpll_pin_alloc - allocate the memory for dpll pin
>+ * @clock_id: clock_id of creator
>+ * @dev_driver_id: id given by dev driver
>+ * @module: reference to registering module
>+ * @prop: dpll pin properties
>+ *
>+ * Return:
>+ * * valid allocated dpll_pin struct pointer if succeeded
>+ * * ERR_PTR of an error

Extra "*"'s


>+ */
>+struct dpll_pin *
>+dpll_pin_alloc(u64 clock_id, u8 device_drv_id,	struct module *module,

Odd whitespace.

Also, func should be static.


>+	       const struct dpll_pin_properties *prop)
>+{
>+	struct dpll_pin *pin;
>+	int ret;
>+
>+	pin = kzalloc(sizeof(*pin), GFP_KERNEL);
>+	if (!pin)
>+		return ERR_PTR(-ENOMEM);
>+	pin->dev_driver_id = device_drv_id;

Name inconsistency: driver/drv
you have it on multiple places


>+	pin->clock_id = clock_id;
>+	pin->module = module;
>+	refcount_set(&pin->refcount, 1);
>+	if (WARN_ON(!prop->description)) {
>+		ret = -EINVAL;
>+		goto release;
>+	}
>+	pin->prop.description = kstrdup(prop->description, GFP_KERNEL);
>+	if (!pin->prop.description) {
>+		ret = -ENOMEM;
>+		goto release;
>+	}
>+	if (WARN_ON(prop->type <= DPLL_PIN_TYPE_UNSPEC ||
>+		    prop->type > DPLL_PIN_TYPE_MAX)) {
>+		ret = -EINVAL;
>+		goto release;
>+	}
>+	pin->prop.type = prop->type;
>+	pin->prop.capabilities = prop->capabilities;
>+	pin->prop.freq_supported = prop->freq_supported;
>+	pin->prop.any_freq_min = prop->any_freq_min;
>+	pin->prop.any_freq_max = prop->any_freq_max;

Make sure that the driver maintains prop (static const) and just save
the pointer. Prop does not need to be something driver needs to change.


>+	xa_init_flags(&pin->dpll_refs, XA_FLAGS_ALLOC);
>+	xa_init_flags(&pin->parent_refs, XA_FLAGS_ALLOC);
>+	ret = xa_alloc(&dpll_pin_xa, &pin->idx, pin,
>+		       xa_limit_16b, GFP_KERNEL);
>+release:
>+	if (ret) {
>+		xa_destroy(&pin->dpll_refs);
>+		xa_destroy(&pin->parent_refs);
>+		kfree(pin->prop.description);
>+		kfree(pin->rclk_dev_name);
>+		kfree(pin);
>+		return ERR_PTR(ret);
>+	}
>+
>+	return pin;
>+}
>+
>+/**
>+ * dpll_pin_get - find existing or create new dpll pin
>+ * @clock_id: clock_id of creator
>+ * @dev_driver_id: id given by dev driver
>+ * @module: reference to registering module
>+ * @prop: dpll pin properties
>+ *
>+ * Get existing object of a pin (unique for given arguments) or create new
>+ * if doesn't exist yet.
>+ *
>+ * Return:
>+ * * valid allocated dpll_pin struct pointer if succeeded
>+ * * ERR_PTR of an error

This is one example, I'm pretty sure that there are others, when you
have text inconsistencies in func doc for the same function in .c and .h
Have it please only on one place. .c is the usual place.


>+ */
>+struct dpll_pin *
>+dpll_pin_get(u64 clock_id, u32 device_drv_id, struct module *module,

Again, why do you need this device_drv_id? Clock id should be enough.


>+	     const struct dpll_pin_properties *prop)
>+{
>+	struct dpll_pin *pos, *ret = NULL;
>+	unsigned long index;
>+
>+	mutex_lock(&dpll_pin_xa_lock);
>+	xa_for_each(&dpll_pin_xa, index, pos) {
>+		if (pos->clock_id == clock_id &&
>+		    pos->dev_driver_id == device_drv_id &&
>+		    pos->module == module) {

Compare prop as well.

Can't the driver_id (pin index) be something const as well? I think it
should. And therefore it could be easily put inside.


>+			ret = pos;
>+			refcount_inc(&ret->refcount);
>+			break;
>+		}
>+	}
>+	if (!ret)
>+		ret = dpll_pin_alloc(clock_id, device_drv_id, module, prop);
>+	mutex_unlock(&dpll_pin_xa_lock);
>+
>+	return ret;
>+}
>+EXPORT_SYMBOL_GPL(dpll_pin_get);
>+
>+/**
>+ * dpll_pin_put - decrease the refcount and free memory if possible
>+ * @dpll: dpll_device struct pointer
>+ *
>+ * Drop reference for a pin, if all references are gone, delete pin object.
>+ */
>+void dpll_pin_put(struct dpll_pin *pin)
>+{
>+	if (!pin)
>+		return;
>+	mutex_lock(&dpll_pin_xa_lock);
>+	if (refcount_dec_and_test(&pin->refcount)) {
>+		xa_destroy(&pin->dpll_refs);
>+		xa_destroy(&pin->parent_refs);
>+		xa_erase(&dpll_pin_xa, pin->idx);
>+		kfree(pin->prop.description);
>+		kfree(pin->rclk_dev_name);
>+		kfree(pin);
>+	}
>+	mutex_unlock(&dpll_pin_xa_lock);
>+}
>+EXPORT_SYMBOL_GPL(dpll_pin_put);
>+
>+static int
>+__dpll_pin_register(struct dpll_device *dpll, struct dpll_pin *pin,
>+		    struct dpll_pin_ops *ops, void *priv,
>+		    const char *rclk_device_name)
>+{
>+	int ret;
>+
>+	if (rclk_device_name && !pin->rclk_dev_name) {
>+		pin->rclk_dev_name = kstrdup(rclk_device_name, GFP_KERNEL);
>+		if (!pin->rclk_dev_name)
>+			return -ENOMEM;
>+	}
>+	ret = dpll_xa_ref_pin_add(&dpll->pin_refs, pin, ops, priv);
>+	if (ret)
>+		goto rclk_free;
>+	ret = dpll_xa_ref_dpll_add(&pin->dpll_refs, dpll, ops, priv);
>+	if (ret)
>+		goto ref_pin_del;
>+	else
>+		dpll_pin_notify(dpll, pin, DPLL_A_PIN_IDX);
>+
>+	return ret;
>+
>+ref_pin_del:
>+	dpll_xa_ref_pin_del(&dpll->pin_refs, pin);
>+rclk_free:
>+	kfree(pin->rclk_dev_name);
>+	return ret;
>+}
>+
>+/**
>+ * dpll_pin_register - register the dpll pin in the subsystem
>+ * @dpll: pointer to a dpll
>+ * @pin: pointer to a dpll pin
>+ * @ops: ops for a dpll pin ops
>+ * @priv: pointer to private information of owner
>+ * @rclk_device: pointer to recovered clock device
>+ *
>+ * Return:
>+ * * 0 on success
>+ * * -EINVAL - missing dpll or pin
>+ * * -ENOMEM - failed to allocate memory
>+ */
>+int
>+dpll_pin_register(struct dpll_device *dpll, struct dpll_pin *pin,
>+		  struct dpll_pin_ops *ops, void *priv,
>+		  struct device *rclk_device)

Wait a second, what is this "struct device *"? Looks very odd.


>+{
>+	const char *rclk_name = rclk_device ? dev_name(rclk_device) : NULL;

If you need to store something here, store the pointer to the device
directly. But this rclk_device seems odd to me.
Dev_name is in case of PCI device for example 0000:01:00.0? That alone
is incomplete. What should it server for?



>+	int ret;
>+
>+	if (WARN_ON(!dpll))
>+		return -EINVAL;
>+	if (WARN_ON(!pin))
>+		return -EINVAL;

Remove these checks and other similar checks in the code. It should rely
on basic driver sanity.


>+
>+	mutex_lock(&dpll_device_xa_lock);
>+	mutex_lock(&dpll_pin_xa_lock);
>+	ret = __dpll_pin_register(dpll, pin, ops, priv, rclk_name);
>+	mutex_unlock(&dpll_pin_xa_lock);
>+	mutex_unlock(&dpll_device_xa_lock);
>+
>+	return ret;
>+}
>+EXPORT_SYMBOL_GPL(dpll_pin_register);
>+
>+static void
>+__dpll_pin_unregister(struct dpll_device *dpll, struct dpll_pin *pin)
>+{
>+	dpll_xa_ref_pin_del(&dpll->pin_refs, pin);
>+	dpll_xa_ref_dpll_del(&pin->dpll_refs, dpll);
>+}
>+
>+/**
>+ * dpll_pin_unregister - deregister dpll pin from dpll device
>+ * @dpll: registered dpll pointer
>+ * @pin: pointer to a pin
>+ *
>+ * Note: It does not free the memory
>+ */
>+int dpll_pin_unregister(struct dpll_device *dpll, struct dpll_pin *pin)
>+{
>+	if (WARN_ON(xa_empty(&dpll->pin_refs)))
>+		return -ENOENT;
>+
>+	mutex_lock(&dpll_device_xa_lock);
>+	mutex_lock(&dpll_pin_xa_lock);
>+	__dpll_pin_unregister(dpll, pin);
>+	mutex_unlock(&dpll_pin_xa_lock);
>+	mutex_unlock(&dpll_device_xa_lock);
>+
>+	return 0;
>+}
>+EXPORT_SYMBOL_GPL(dpll_pin_unregister);
>+
>+/**
>+ * dpll_pin_on_pin_register - register a pin with a parent pin
>+ * @parent: pointer to a parent pin
>+ * @pin: pointer to a pin
>+ * @ops: ops for a dpll pin
>+ * @priv: pointer to private information of owner
>+ * @rclk_device: pointer to recovered clock device
>+ *
>+ * Register a pin with a parent pin, create references between them and
>+ * between newly registered pin and dplls connected with a parent pin.
>+ *
>+ * Return:
>+ * * 0 on success
>+ * * -EINVAL missing pin or parent
>+ * * -ENOMEM failed allocation
>+ * * -EPERM if parent is not allowed
>+ */
>+int
>+dpll_pin_on_pin_register(struct dpll_pin *parent, struct dpll_pin *pin,
>+			 struct dpll_pin_ops *ops, void *priv,
>+			 struct device *rclk_device)
>+{
>+	struct dpll_pin_ref *ref;
>+	unsigned long i, stop;
>+	int ret;
>+
>+	if (WARN_ON(!pin || !parent))
>+		return -EINVAL;
>+	if (WARN_ON(parent->prop.type != DPLL_PIN_TYPE_MUX))
>+		return -EPERM;

I don't think that EPERM is suitable for this. Just use EINVAL. The
driver is buggy in this case anyway.


>+	mutex_lock(&dpll_pin_xa_lock);
>+	ret = dpll_xa_ref_pin_add(&pin->parent_refs, parent, ops, priv);
>+	if (ret)
>+		goto unlock;
>+	refcount_inc(&pin->refcount);
>+	xa_for_each(&parent->dpll_refs, i, ref) {
>+		mutex_lock(&dpll_device_xa_lock);
>+		ret = __dpll_pin_register(ref->dpll, pin, ops, priv,
>+					  rclk_device ?
>+					  dev_name(rclk_device) : NULL);
>+		mutex_unlock(&dpll_device_xa_lock);
>+		if (ret) {
>+			stop = i;
>+			goto dpll_unregister;
>+		}
>+		dpll_pin_parent_notify(ref->dpll, pin, parent, DPLL_A_PIN_IDX);
>+	}
>+	mutex_unlock(&dpll_pin_xa_lock);
>+
>+	return ret;
>+
>+dpll_unregister:
>+	xa_for_each(&parent->dpll_refs, i, ref) {
>+		if (i < stop) {
>+			mutex_lock(&dpll_device_xa_lock);
>+			__dpll_pin_unregister(ref->dpll, pin);
>+			mutex_unlock(&dpll_device_xa_lock);
>+		}
>+	}
>+	refcount_dec(&pin->refcount);
>+	dpll_xa_ref_pin_del(&pin->parent_refs, parent);
>+unlock:
>+	mutex_unlock(&dpll_pin_xa_lock);
>+	return ret;
>+}
>+EXPORT_SYMBOL_GPL(dpll_pin_on_pin_register);
>+
>+/**
>+ * dpll_pin_on_pin_unregister - deregister dpll pin from a parent pin
>+ * @parent: pointer to a parent pin
>+ * @pin: pointer to a pin
>+ *
>+ * Note: It does not free the memory
>+ */
>+void dpll_pin_on_pin_unregister(struct dpll_pin *parent, struct dpll_pin *pin)
>+{
>+	struct dpll_pin_ref *ref;
>+	unsigned long i;
>+
>+	mutex_lock(&dpll_device_xa_lock);
>+	mutex_lock(&dpll_pin_xa_lock);
>+	dpll_xa_ref_pin_del(&pin->parent_refs, parent);
>+	refcount_dec(&pin->refcount);
>+	xa_for_each(&pin->dpll_refs, i, ref) {
>+		__dpll_pin_unregister(ref->dpll, pin);
>+		dpll_pin_parent_notify(ref->dpll, pin, parent,
>+				       DPLL_A_PIN_IDX);
>+	}
>+	mutex_unlock(&dpll_pin_xa_lock);
>+	mutex_unlock(&dpll_device_xa_lock);
>+}
>+EXPORT_SYMBOL_GPL(dpll_pin_on_pin_unregister);
>+
>+/**
>+ * dpll_pin_get_by_idx - find a pin ref on dpll by pin index
>+ * @dpll: dpll device pointer
>+ * @idx: index of pin
>+ *
>+ * Find a reference to a pin registered with given dpll and return its pointer.
>+ *
>+ * Return:
>+ * * valid pointer if pin was found
>+ * * NULL if not found
>+ */
>+struct dpll_pin *dpll_pin_get_by_idx(struct dpll_device *dpll, u32 idx)
>+{
>+	struct dpll_pin_ref *pos;
>+	unsigned long i;
>+
>+	xa_for_each(&dpll->pin_refs, i, pos) {
>+		if (pos && pos->pin && pos->pin->dev_driver_id == idx)

How exactly pos->pin could be NULL?

Also, you are degrading the xarray to a mere list here with lookup like
this. Why can't you use the pin index coming from driver and
insert/lookup based on this index?


>+			return pos->pin;
>+	}
>+
>+	return NULL;
>+}
>+
>+/**
>+ * dpll_priv - get the dpll device private owner data
>+ * @dpll:	registered dpll pointer
>+ *
>+ * Return: pointer to the data
>+ */
>+void *dpll_priv(const struct dpll_device *dpll)
>+{
>+	return dpll->priv;
>+}
>+EXPORT_SYMBOL_GPL(dpll_priv);
>+
>+/**
>+ * dpll_pin_on_dpll_priv - get the dpll device private owner data
>+ * @dpll:	registered dpll pointer
>+ * @pin:	pointer to a pin
>+ *
>+ * Return: pointer to the data
>+ */
>+void *dpll_pin_on_dpll_priv(const struct dpll_device *dpll,

IIUC, you use this helper from dpll ops in drivers to get per dpll priv.
Just pass the priv directly to the op and avoid need for this helper,
no? Same goes to the rest of the priv helpers.


>+			    const struct dpll_pin *pin)
>+{
>+	struct dpll_pin_ref *ref;
>+
>+	ref = dpll_xa_ref_pin_find((struct xarray *)&dpll->pin_refs, pin);

Why cast is needed here? You have this on multiple places.


>+	if (!ref)
>+		return NULL;
>+
>+	return ref->priv;
>+}
>+EXPORT_SYMBOL_GPL(dpll_pin_on_dpll_priv);
>+
>+/**
>+ * dpll_pin_on_pin_priv - get the dpll pin private owner data
>+ * @parent: pointer to a parent pin
>+ * @pin: pointer to a pin
>+ *
>+ * Return: pointer to the data
>+ */
>+void *dpll_pin_on_pin_priv(const struct dpll_pin *parent,
>+			   const struct dpll_pin *pin)
>+{
>+	struct dpll_pin_ref *ref;
>+
>+	ref = dpll_xa_ref_pin_find((struct xarray *)&pin->parent_refs, parent);
>+	if (!ref)
>+		return NULL;
>+
>+	return ref->priv;
>+}
>+EXPORT_SYMBOL_GPL(dpll_pin_on_pin_priv);
>+
>+static int __init dpll_init(void)
>+{
>+	int ret;
>+
>+	ret = dpll_netlink_init();
>+	if (ret)
>+		goto error;
>+
>+	ret = class_register(&dpll_class);

Why exactly do you need this? I asked to remove this previously, IIRC
you said you would check if this needed. Why?


>+	if (ret)
>+		goto unregister_netlink;
>+
>+	return 0;
>+
>+unregister_netlink:
>+	dpll_netlink_finish();
>+error:
>+	mutex_destroy(&dpll_device_xa_lock);
>+	mutex_destroy(&dpll_pin_xa_lock);
>+	return ret;
>+}
>+subsys_initcall(dpll_init);
>diff --git a/drivers/dpll/dpll_core.h b/drivers/dpll/dpll_core.h
>new file mode 100644
>index 000000000000..876b6ac6f3a0
>--- /dev/null
>+++ b/drivers/dpll/dpll_core.h
>@@ -0,0 +1,99 @@
>+/* SPDX-License-Identifier: GPL-2.0 */
>+/*
>+ *  Copyright (c) 2021 Meta Platforms, Inc. and affiliates
>+ */
>+
>+#ifndef __DPLL_CORE_H__
>+#define __DPLL_CORE_H__
>+
>+#include <linux/dpll.h>
>+#include <linux/refcount.h>
>+#include "dpll_netlink.h"
>+
>+#define DPLL_REGISTERED		XA_MARK_1
>+
>+/**
>+ * struct dpll_device - structure for a DPLL device
>+ * @id:			unique id number for each device
>+ * @dev_driver_id:	id given by dev driver
>+ * @dev:		struct device for this dpll device
>+ * @parent:		parent device
>+ * @module:		module of creator
>+ * @ops:		operations this &dpll_device supports
>+ * @lock:		mutex to serialize operations
>+ * @type:		type of a dpll
>+ * @priv:		pointer to private information of owner
>+ * @pins:		list of pointers to pins registered with this dpll
>+ * @clock_id:		unique identifier (clock_id) of a dpll
>+ * @mode_supported_mask: mask of supported modes
>+ * @refcount:		refcount
>+ **/
>+struct dpll_device {
>+	u32 id;
>+	u32 dev_driver_id;
>+	struct device dev;
>+	struct device *parent;
>+	struct module *module;
>+	struct dpll_device_ops *ops;
>+	enum dpll_type type;
>+	void *priv;
>+	struct xarray pin_refs;
>+	u64 clock_id;
>+	unsigned long mode_supported_mask;
>+	refcount_t refcount;
>+};
>+
>+/**
>+ * struct dpll_pin - structure for a dpll pin
>+ * @idx:		unique idx given by alloc on global pin's XA
>+ * @dev_driver_id:	id given by dev driver
>+ * @clock_id:		clock_id of creator
>+ * @module:		module of creator
>+ * @dpll_refs:		hold referencees to dplls that pin is registered with
>+ * @pin_refs:		hold references to pins that pin is registered with
>+ * @prop:		properties given by registerer
>+ * @rclk_dev_name:	holds name of device when pin can recover clock from it
>+ * @refcount:		refcount
>+ **/
>+struct dpll_pin {
>+	u32 idx;
>+	u32 dev_driver_id;
>+	u64 clock_id;
>+	struct module *module;

Have the ordering of common fields, like clock_id and module consistent
with struct dpll_device


>+	struct xarray dpll_refs;
>+	struct xarray parent_refs;
>+	struct dpll_pin_properties prop;
>+	char *rclk_dev_name;
>+	refcount_t refcount;
>+};
>+
>+/**
>+ * struct dpll_pin_ref - structure for referencing either dpll or pins
>+ * @dpll:		pointer to a dpll
>+ * @pin:		pointer to a pin
>+ * @ops:		ops for a dpll pin
>+ * @priv:		pointer to private information of owner
>+ **/
>+struct dpll_pin_ref {
>+	union {
>+		struct dpll_device *dpll;
>+		struct dpll_pin *pin;
>+	};
>+	struct dpll_pin_ops *ops;
>+	void *priv;
>+	refcount_t refcount;
>+};
>+
>+struct dpll_device *dpll_device_get_by_id(int id);
>+struct dpll_device *dpll_device_get_by_name(const char *bus_name,
>+					    const char *dev_name);
>+struct dpll_pin *dpll_pin_get_by_idx(struct dpll_device *dpll, u32 idx);
>+struct dpll_pin_ref *
>+dpll_xa_ref_pin_find(struct xarray *xa_refs, const struct dpll_pin *pin);
>+struct dpll_pin_ref *
>+dpll_xa_ref_dpll_find(struct xarray *xa_refs, const struct dpll_device *dpll);
>+extern struct xarray dpll_device_xa;
>+extern struct xarray dpll_pin_xa;
>+extern struct mutex dpll_device_xa_lock;
>+extern struct mutex dpll_pin_xa_lock;
>+#endif
>diff --git a/drivers/dpll/dpll_netlink.c b/drivers/dpll/dpll_netlink.c
>new file mode 100644
>index 000000000000..46aefeb1ac93
>--- /dev/null
>+++ b/drivers/dpll/dpll_netlink.c
>@@ -0,0 +1,1065 @@
>+// SPDX-License-Identifier: GPL-2.0
>+/*
>+ * Generic netlink for DPLL management framework
>+ *
>+ * Copyright (c) 2021 Meta Platforms, Inc. and affiliates
>+ *
>+ */
>+#include <linux/module.h>
>+#include <linux/kernel.h>
>+#include <net/genetlink.h>
>+#include "dpll_core.h"
>+#include "dpll_nl.h"
>+#include <uapi/linux/dpll.h>
>+
>+static u32 dpll_pin_freq_value[] = {
>+	[DPLL_PIN_FREQ_SUPP_1_HZ] = DPLL_PIN_FREQ_1_HZ,
>+	[DPLL_PIN_FREQ_SUPP_10_MHZ] = DPLL_PIN_FREQ_10_MHZ,
>+};
>+
>+static int
>+dpll_msg_add_dev_handle(struct sk_buff *msg, const struct dpll_device *dpll)
>+{
>+	if (nla_put_u32(msg, DPLL_A_ID, dpll->id))

Why exactly do we need this dua--handle scheme? Why do you need
unpredictable DPLL_A_ID to be exposed to userspace?
It's just confusing.



>+		return -EMSGSIZE;
>+	if (nla_put_string(msg, DPLL_A_BUS_NAME, dev_bus_name(&dpll->dev)))
>+		return -EMSGSIZE;
>+	if (nla_put_string(msg, DPLL_A_DEV_NAME, dev_name(&dpll->dev)))
>+		return -EMSGSIZE;
>+
>+	return 0;
>+}
>+
>+static int
>+dpll_msg_add_mode(struct sk_buff *msg, const struct dpll_device *dpll,
>+		  struct netlink_ext_ack *extack)
>+{
>+	enum dpll_mode mode;
>+
>+	if (WARN_ON(!dpll->ops->mode_get))
>+		return -EOPNOTSUPP;
>+	if (dpll->ops->mode_get(dpll, &mode, extack))
>+		return -EFAULT;
>+	if (nla_put_u8(msg, DPLL_A_MODE, mode))
>+		return -EMSGSIZE;
>+
>+	return 0;
>+}
>+
>+static int
>+dpll_msg_add_source_pin_idx(struct sk_buff *msg, struct dpll_device *dpll,
>+			    struct netlink_ext_ack *extack)
>+{
>+	u32 source_pin_idx;
>+
>+	if (WARN_ON(!dpll->ops->source_pin_idx_get))
>+		return -EOPNOTSUPP;
>+	if (dpll->ops->source_pin_idx_get(dpll, &source_pin_idx, extack))
>+		return -EFAULT;
>+	if (nla_put_u32(msg, DPLL_A_SOURCE_PIN_IDX, source_pin_idx))
>+		return -EMSGSIZE;
>+
>+	return 0;
>+}
>+
>+static int
>+dpll_msg_add_lock_status(struct sk_buff *msg, struct dpll_device *dpll,
>+			 struct netlink_ext_ack *extack)
>+{
>+	enum dpll_lock_status status;
>+
>+	if (WARN_ON(!dpll->ops->lock_status_get))
>+		return -EOPNOTSUPP;
>+	if (dpll->ops->lock_status_get(dpll, &status, extack))
>+		return -EFAULT;
>+	if (nla_put_u8(msg, DPLL_A_LOCK_STATUS, status))
>+		return -EMSGSIZE;
>+
>+	return 0;
>+}
>+
>+static int
>+dpll_msg_add_temp(struct sk_buff *msg, struct dpll_device *dpll,
>+		  struct netlink_ext_ack *extack)
>+{
>+	s32 temp;
>+
>+	if (!dpll->ops->temp_get)
>+		return -EOPNOTSUPP;
>+	if (dpll->ops->temp_get(dpll, &temp, extack))
>+		return -EFAULT;
>+	if (nla_put_s32(msg, DPLL_A_TEMP, temp))
>+		return -EMSGSIZE;
>+
>+	return 0;
>+}
>+
>+static int
>+dpll_msg_add_pin_prio(struct sk_buff *msg, const struct dpll_pin *pin,
>+		      struct dpll_pin_ref *ref,
>+		      struct netlink_ext_ack *extack)
>+{
>+	u32 prio;
>+
>+	if (!ref->ops->prio_get)
>+		return -EOPNOTSUPP;
>+	if (ref->ops->prio_get(pin, ref->dpll, &prio, extack))
>+		return -EFAULT;
>+	if (nla_put_u32(msg, DPLL_A_PIN_PRIO, prio))
>+		return -EMSGSIZE;
>+
>+	return 0;
>+}
>+
>+static int
>+dpll_msg_add_pin_on_dpll_state(struct sk_buff *msg, const struct dpll_pin *pin,
>+			       struct dpll_pin_ref *ref,
>+			       struct netlink_ext_ack *extack)
>+{
>+	enum dpll_pin_state state;
>+
>+	if (!ref->ops->state_on_dpll_get)
>+		return -EOPNOTSUPP;
>+	if (ref->ops->state_on_dpll_get(pin, ref->dpll, &state, extack))
>+		return -EFAULT;
>+	if (nla_put_u8(msg, DPLL_A_PIN_STATE, state))
>+		return -EMSGSIZE;
>+
>+	return 0;
>+}
>+
>+static int
>+dpll_msg_add_pin_direction(struct sk_buff *msg, const struct dpll_pin *pin,
>+			   struct netlink_ext_ack *extack)
>+{
>+	enum dpll_pin_direction direction;
>+	struct dpll_pin_ref *ref;
>+	unsigned long i;
>+
>+	xa_for_each((struct xarray *)&pin->dpll_refs, i, ref) {
>+		if (ref && ref->ops && ref->dpll)
>+			break;
>+	}
>+	if (!ref || !ref->ops || !ref->dpll)
>+		return -ENODEV;
>+	if (!ref->ops->direction_get)
>+		return -EOPNOTSUPP;
>+	if (ref->ops->direction_get(pin, ref->dpll, &direction, extack))
>+		return -EFAULT;
>+	if (nla_put_u8(msg, DPLL_A_PIN_DIRECTION, direction))
>+		return -EMSGSIZE;
>+
>+	return 0;
>+}
>+
>+static int
>+dpll_msg_add_pin_freq(struct sk_buff *msg, const struct dpll_pin *pin,
>+		      struct netlink_ext_ack *extack, bool dump_any_freq)
>+{
>+	enum dpll_pin_freq_supp fs;
>+	struct dpll_pin_ref *ref;
>+	unsigned long i;
>+	u32 freq;
>+
>+	xa_for_each((struct xarray *)&pin->dpll_refs, i, ref) {
>+		if (ref && ref->ops && ref->dpll)
>+			break;
>+	}
>+	if (!ref || !ref->ops || !ref->dpll)
>+		return -ENODEV;
>+	if (!ref->ops->frequency_get)
>+		return -EOPNOTSUPP;
>+	if (ref->ops->frequency_get(pin, ref->dpll, &freq, extack))
>+		return -EFAULT;
>+	if (nla_put_u32(msg, DPLL_A_PIN_FREQUENCY, freq))
>+		return -EMSGSIZE;
>+	if (!dump_any_freq)
>+		return 0;
>+	for (fs = DPLL_PIN_FREQ_SUPP_UNSPEC + 1;
>+	     fs <= DPLL_PIN_FREQ_SUPP_MAX; fs++) {
>+		if (test_bit(fs, &pin->prop.freq_supported)) {
>+			if (nla_put_u32(msg, DPLL_A_PIN_FREQUENCY_SUPPORTED,
>+			    dpll_pin_freq_value[fs]))
>+				return -EMSGSIZE;
>+		}
>+	}
>+	if (pin->prop.any_freq_min && pin->prop.any_freq_max) {
>+		if (nla_put_u32(msg, DPLL_A_PIN_ANY_FREQUENCY_MIN,
>+				pin->prop.any_freq_min))
>+			return -EMSGSIZE;
>+		if (nla_put_u32(msg, DPLL_A_PIN_ANY_FREQUENCY_MAX,
>+				pin->prop.any_freq_max))
>+			return -EMSGSIZE;
>+	}
>+
>+	return 0;
>+}
>+
>+static int
>+dpll_msg_add_pin_parents(struct sk_buff *msg, struct dpll_pin *pin,
>+			 struct netlink_ext_ack *extack)
>+{
>+	struct dpll_pin_ref *ref_parent;
>+	enum dpll_pin_state state;
>+	struct nlattr *nest;
>+	unsigned long index;
>+	int ret;
>+
>+	xa_for_each(&pin->parent_refs, index, ref_parent) {
>+		if (WARN_ON(!ref_parent->ops->state_on_pin_get))
>+			return -EFAULT;
>+		ret = ref_parent->ops->state_on_pin_get(pin, ref_parent->pin,
>+							&state, extack);
>+		if (ret)
>+			return -EFAULT;
>+		nest = nla_nest_start(msg, DPLL_A_PIN_PARENT);
>+		if (!nest)
>+			return -EMSGSIZE;
>+		if (nla_put_u32(msg, DPLL_A_PIN_PARENT_IDX,
>+				ref_parent->pin->dev_driver_id)) {
>+			ret = -EMSGSIZE;
>+			goto nest_cancel;
>+		}
>+		if (nla_put_u8(msg, DPLL_A_PIN_STATE, state)) {
>+			ret = -EMSGSIZE;
>+			goto nest_cancel;
>+		}
>+		nla_nest_end(msg, nest);
>+	}
>+
>+	return 0;
>+
>+nest_cancel:
>+	nla_nest_cancel(msg, nest);
>+	return ret;
>+}
>+
>+static int
>+dpll_msg_add_pins_on_pin(struct sk_buff *msg, struct dpll_pin *pin,
>+			 struct netlink_ext_ack *extack)
>+{
>+	struct dpll_pin_ref *ref = NULL;

Why this needs to be initialized?


>+	enum dpll_pin_state state;
>+	struct nlattr *nest;
>+	unsigned long index;
>+	int ret;
>+
>+	xa_for_each(&pin->parent_refs, index, ref) {
>+		if (WARN_ON(!ref->ops->state_on_pin_get))
>+			return -EFAULT;
>+		ret = ref->ops->state_on_pin_get(pin, ref->pin, &state,
>+						 extack);
>+		if (ret)
>+			return -EFAULT;
>+		nest = nla_nest_start(msg, DPLL_A_PIN_PARENT);
>+		if (!nest)
>+			return -EMSGSIZE;
>+		if (nla_put_u32(msg, DPLL_A_PIN_PARENT_IDX,
>+				ref->pin->dev_driver_id)) {
>+			ret = -EMSGSIZE;
>+			goto nest_cancel;
>+		}
>+		if (nla_put_u8(msg, DPLL_A_PIN_STATE, state)) {
>+			ret = -EMSGSIZE;
>+			goto nest_cancel;
>+		}
>+		nla_nest_end(msg, nest);
>+	}

How is this function different to dpll_msg_add_pin_parents()?
Am I lost? To be honest, this x_on_pin/dpll, parent, refs dance is quite
hard to follow for me :/

Did you get lost here as well? If yes, this needs some serious think
through :)



>+
>+	return 0;
>+
>+nest_cancel:
>+	nla_nest_cancel(msg, nest);
>+	return ret;
>+}
>+
>+static int
>+dpll_msg_add_pin_dplls(struct sk_buff *msg, struct dpll_pin *pin,
>+		       struct netlink_ext_ack *extack)
>+{
>+	struct dpll_pin_ref *ref;
>+	struct nlattr *attr;
>+	unsigned long index;
>+	int ret;
>+
>+	xa_for_each(&pin->dpll_refs, index, ref) {
>+		attr = nla_nest_start(msg, DPLL_A_DEVICE);
>+		if (!attr)
>+			return -EMSGSIZE;
>+		ret = dpll_msg_add_dev_handle(msg, ref->dpll);
>+		if (ret)
>+			goto nest_cancel;
>+		ret = dpll_msg_add_pin_on_dpll_state(msg, pin, ref, extack);
>+		if (ret && ret != -EOPNOTSUPP)
>+			goto nest_cancel;
>+		ret = dpll_msg_add_pin_prio(msg, pin, ref, extack);
>+		if (ret && ret != -EOPNOTSUPP)
>+			goto nest_cancel;
>+		nla_nest_end(msg, attr);
>+	}
>+
>+	return 0;
>+
>+nest_cancel:
>+	nla_nest_end(msg, attr);
>+	return ret;
>+}
>+
>+static int
>+dpll_cmd_pin_on_dpll_get(struct sk_buff *msg, struct dpll_pin *pin,
>+			 struct dpll_device *dpll,
>+			 struct netlink_ext_ack *extack)
>+{
>+	struct dpll_pin_ref *ref;
>+	int ret;
>+
>+	if (nla_put_u32(msg, DPLL_A_PIN_IDX, pin->dev_driver_id))

Is it "ID" or "INDEX" (IDX). Please make this consistent in the whole
code.

>+		return -EMSGSIZE;
>+	if (nla_put_string(msg, DPLL_A_PIN_DESCRIPTION, pin->prop.description))
>+		return -EMSGSIZE;
>+	if (nla_put_u8(msg, DPLL_A_PIN_TYPE, pin->prop.type))
>+		return -EMSGSIZE;
>+	if (nla_put_u32(msg, DPLL_A_PIN_DPLL_CAPS, pin->prop.capabilities))
>+		return -EMSGSIZE;
>+	ret = dpll_msg_add_pin_direction(msg, pin, extack);
>+	if (ret)
>+		return ret;
>+	ret = dpll_msg_add_pin_freq(msg, pin, extack, true);
>+	if (ret && ret != -EOPNOTSUPP)
>+		return ret;
>+	ref = dpll_xa_ref_dpll_find(&pin->dpll_refs, dpll);
>+	if (!ref)
>+		return -EFAULT;
>+	ret = dpll_msg_add_pin_prio(msg, pin, ref, extack);
>+	if (ret && ret != -EOPNOTSUPP)
>+		return ret;
>+	ret = dpll_msg_add_pin_on_dpll_state(msg, pin, ref, extack);
>+	if (ret && ret != -EOPNOTSUPP)
>+		return ret;
>+	ret = dpll_msg_add_pin_parents(msg, pin, extack);
>+	if (ret)
>+		return ret;
>+	if (pin->rclk_dev_name)
>+		if (nla_put_string(msg, DPLL_A_PIN_RCLK_DEVICE,
>+				   pin->rclk_dev_name))
>+			return -EMSGSIZE;
>+
>+	return 0;
>+}
>+
>+static int
>+__dpll_cmd_pin_dump_one(struct sk_buff *msg, struct dpll_pin *pin,
>+			struct netlink_ext_ack *extack, bool dump_dpll)
>+{
>+	int ret;
>+
>+	if (nla_put_u32(msg, DPLL_A_PIN_IDX, pin->dev_driver_id))
>+		return -EMSGSIZE;
>+	if (nla_put_string(msg, DPLL_A_PIN_DESCRIPTION, pin->prop.description))
>+		return -EMSGSIZE;
>+	if (nla_put_u8(msg, DPLL_A_PIN_TYPE, pin->prop.type))
>+		return -EMSGSIZE;
>+	ret = dpll_msg_add_pin_direction(msg, pin, extack);
>+	if (ret)
>+		return ret;
>+	ret = dpll_msg_add_pin_freq(msg, pin, extack, true);
>+	if (ret && ret != -EOPNOTSUPP)
>+		return ret;
>+	ret = dpll_msg_add_pins_on_pin(msg, pin, extack);
>+	if (ret)
>+		return ret;
>+	if (!xa_empty(&pin->dpll_refs) && dump_dpll) {
>+		ret = dpll_msg_add_pin_dplls(msg, pin, extack);
>+		if (ret)
>+			return ret;
>+	}
>+	if (pin->rclk_dev_name)
>+		if (nla_put_string(msg, DPLL_A_PIN_RCLK_DEVICE,
>+				   pin->rclk_dev_name))
>+			return -EMSGSIZE;

Lots of code duplication with the previous functions, please unify.


>+
>+	return 0;
>+}
>+
>+static int
>+dpll_device_get_one(struct dpll_device *dpll, struct sk_buff *msg,
>+		     struct netlink_ext_ack *extack)
>+{
>+	struct dpll_pin_ref *ref;
>+	enum dpll_mode mode;
>+	unsigned long i;
>+	int ret;
>+
>+	ret = dpll_msg_add_dev_handle(msg, dpll);
>+	if (ret)
>+		return ret;
>+	ret = dpll_msg_add_source_pin_idx(msg, dpll, extack);
>+	if (ret)
>+		return ret;
>+	ret = dpll_msg_add_temp(msg, dpll, extack);
>+	if (ret && ret != -EOPNOTSUPP)
>+		return ret;
>+	ret = dpll_msg_add_lock_status(msg, dpll, extack);
>+	if (ret)
>+		return ret;
>+	ret = dpll_msg_add_mode(msg, dpll, extack);
>+	if (ret)
>+		return ret;
>+	for (mode = DPLL_MODE_UNSPEC + 1; mode <= DPLL_MODE_MAX; mode++)
>+		if (test_bit(mode, &dpll->mode_supported_mask))
>+			if (nla_put_s32(msg, DPLL_A_MODE_SUPPORTED, mode))
>+				return -EMSGSIZE;
>+	if (nla_put_64bit(msg, DPLL_A_CLOCK_ID, sizeof(dpll->clock_id),
>+			  &dpll->clock_id, 0))
>+		return -EMSGSIZE;
>+	if (nla_put_u8(msg, DPLL_A_TYPE, dpll->type))
>+		return -EMSGSIZE;
>+	xa_for_each(&dpll->pin_refs, i, ref) {
>+		struct nlattr *nest = nla_nest_start(msg, DPLL_A_PIN);
>+
>+		if (!nest) {
>+			ret = -EMSGSIZE;
>+			break;
>+		}
>+		ret = dpll_cmd_pin_on_dpll_get(msg, ref->pin, dpll, extack);
>+		if (ret) {
>+			nla_nest_cancel(msg, nest);
>+			break;
>+		}
>+		nla_nest_end(msg, nest);
>+	}
>+
>+	return ret;
>+}
>+
>+static bool dpll_pin_is_freq_supported(struct dpll_pin *pin, u32 freq)
>+{
>+	enum dpll_pin_freq_supp fs;
>+
>+	if (freq >= pin->prop.any_freq_min && freq <= pin->prop.any_freq_max)
>+		return true;
>+	for (fs = DPLL_PIN_FREQ_SUPP_UNSPEC + 1;
>+	     fs <= DPLL_PIN_FREQ_SUPP_MAX; fs++)
>+		if (test_bit(fs, &pin->prop.freq_supported))
>+			if (freq == dpll_pin_freq_value[fs])
>+				return true;
>+	return false;
>+}
>+
>+static int
>+dpll_pin_freq_set(struct dpll_pin *pin, struct nlattr *a,
>+		  struct netlink_ext_ack *extack)
>+{
>+	u32 freq = nla_get_u32(a);
>+	struct dpll_pin_ref *ref;
>+	unsigned long i;
>+	int ret;
>+
>+	if (!dpll_pin_is_freq_supported(pin, freq))
>+		return -EINVAL;
>+
>+	xa_for_each(&pin->dpll_refs, i, ref) {
>+		ret = ref->ops->frequency_set(pin, ref->dpll, freq, extack);
>+		if (ret)
>+			return -EFAULT;
>+		dpll_pin_notify(ref->dpll, pin, DPLL_A_PIN_FREQUENCY);
>+	}
>+
>+	return 0;
>+}
>+
>+static int
>+dpll_pin_on_pin_state_set(struct dpll_device *dpll, struct dpll_pin *pin,
>+			  u32 parent_idx, enum dpll_pin_state state,
>+			  struct netlink_ext_ack *extack)
>+{
>+	struct dpll_pin_ref *ref;
>+	struct dpll_pin *parent;
>+
>+	if (!(DPLL_PIN_CAPS_STATE_CAN_CHANGE & pin->prop.capabilities))
>+		return -EOPNOTSUPP;
>+	parent = dpll_pin_get_by_idx(dpll, parent_idx);
>+	if (!parent)
>+		return -EINVAL;
>+	ref = dpll_xa_ref_pin_find(&pin->parent_refs, parent);
>+	if (!ref)
>+		return -EINVAL;
>+	if (!ref->ops || !ref->ops->state_on_pin_set)
>+		return -EOPNOTSUPP;
>+	if (ref->ops->state_on_pin_set(pin, parent, state, extack))
>+		return -EFAULT;
>+	dpll_pin_parent_notify(dpll, pin, parent, DPLL_A_PIN_STATE);
>+
>+	return 0;
>+}
>+
>+static int
>+dpll_pin_state_set(struct dpll_device *dpll, struct dpll_pin *pin,
>+		   enum dpll_pin_state state,
>+		   struct netlink_ext_ack *extack)
>+{
>+	struct dpll_pin_ref *ref;
>+
>+	if (!(DPLL_PIN_CAPS_STATE_CAN_CHANGE & pin->prop.capabilities))
>+		return -EOPNOTSUPP;
>+	ref = dpll_xa_ref_dpll_find(&pin->dpll_refs, dpll);
>+	if (!ref)
>+		return -EFAULT;
>+	if (!ref->ops || !ref->ops->state_on_dpll_set)
>+		return -EOPNOTSUPP;
>+	if (ref->ops->state_on_dpll_set(pin, ref->dpll, state, extack))
>+		return -EINVAL;
>+	dpll_pin_notify(ref->dpll, pin, DPLL_A_PIN_STATE);
>+
>+	return 0;
>+}
>+
>+static int
>+dpll_pin_prio_set(struct dpll_device *dpll, struct dpll_pin *pin,
>+		  struct nlattr *prio_attr, struct netlink_ext_ack *extack)
>+{
>+	struct dpll_pin_ref *ref;
>+	u32 prio = nla_get_u8(prio_attr);
>+
>+	if (!(DPLL_PIN_CAPS_PRIORITY_CAN_CHANGE & pin->prop.capabilities))
>+		return -EOPNOTSUPP;
>+	ref = dpll_xa_ref_dpll_find(&pin->dpll_refs, dpll);
>+	if (!ref)
>+		return -EFAULT;
>+	if (!ref->ops || !ref->ops->prio_set)
>+		return -EOPNOTSUPP;
>+	if (ref->ops->prio_set(pin, dpll, prio, extack))
>+		return -EINVAL;
>+	dpll_pin_notify(dpll, pin, DPLL_A_PIN_PRIO);
>+
>+	return 0;
>+}
>+
>+static int
>+dpll_pin_direction_set(struct dpll_pin *pin, struct nlattr *a,
>+		       struct netlink_ext_ack *extack)
>+{
>+	enum dpll_pin_direction direction = nla_get_u8(a);
>+	struct dpll_pin_ref *ref;
>+	unsigned long i;
>+
>+	if (!(DPLL_PIN_CAPS_DIRECTION_CAN_CHANGE & pin->prop.capabilities))
>+		return -EOPNOTSUPP;
>+
>+	xa_for_each(&pin->dpll_refs, i, ref) {
>+		if (ref->ops->direction_set(pin, ref->dpll, direction, extack))
>+			return -EFAULT;
>+		dpll_pin_notify(ref->dpll, pin, DPLL_A_PIN_DIRECTION);
>+	}
>+
>+	return 0;
>+}
>+
>+static int
>+dpll_pin_set_from_nlattr(struct dpll_device *dpll,
>+			 struct dpll_pin *pin, struct genl_info *info)
>+{
>+	enum dpll_pin_state state = DPLL_PIN_STATE_UNSPEC;
>+	u32 parent_idx = PIN_IDX_INVALID;

You just need this PIN_IDX_INVALID define internally in this function,
change the flow to avoid a need for it.


>+	int rem, ret = -EINVAL;
>+	struct nlattr *a;
>+
>+	nla_for_each_attr(a, genlmsg_data(info->genlhdr),
>+			  genlmsg_len(info->genlhdr), rem) {

This is odd. Why you iterace over attrs? Why don't you just access them
directly, like attrs[DPLL_A_PIN_FREQUENCY] for example?


>+		switch (nla_type(a)) {
>+		case DPLL_A_PIN_FREQUENCY:
>+			ret = dpll_pin_freq_set(pin, a, info->extack);
>+			if (ret)
>+				return ret;
>+			break;
>+		case DPLL_A_PIN_DIRECTION:
>+			ret = dpll_pin_direction_set(pin, a, info->extack);
>+			if (ret)
>+				return ret;
>+			break;
>+		case DPLL_A_PIN_PRIO:
>+			ret = dpll_pin_prio_set(dpll, pin, a, info->extack);
>+			if (ret)
>+				return ret;
>+			break;
>+		case DPLL_A_PIN_PARENT_IDX:
>+			parent_idx = nla_get_u32(a);
>+			break;
>+		case DPLL_A_PIN_STATE:
>+			state = nla_get_u8(a);
>+			break;
>+		default:
>+			break;
>+		}
>+	}
>+	if (state != DPLL_PIN_STATE_UNSPEC) {

Again, change the flow to:
	if (attrs[DPLL_A_PIN_STATE]) {

and avoid need for this value set/check.


>+		if (parent_idx == PIN_IDX_INVALID) {
>+			ret = dpll_pin_state_set(dpll, pin, state,
>+						 info->extack);
>+			if (ret)
>+				return ret;
>+		} else {
>+			ret = dpll_pin_on_pin_state_set(dpll, pin, parent_idx,
>+							state, info->extack);
>+			if (ret)
>+				return ret;
>+		}
>+	}
>+
>+	return ret;
>+}
>+
>+int dpll_nl_pin_set_doit(struct sk_buff *skb, struct genl_info *info)
>+{
>+	struct dpll_device *dpll = info->user_ptr[0];
>+	struct dpll_pin *pin = info->user_ptr[1];
>+
>+	return dpll_pin_set_from_nlattr(dpll, pin, info);
>+}
>+
>+int dpll_nl_pin_get_doit(struct sk_buff *skb, struct genl_info *info)
>+{
>+	struct dpll_pin *pin = info->user_ptr[1];
>+	struct nlattr *hdr, *nest;
>+	struct sk_buff *msg;
>+	int ret;
>+
>+	if (!pin)
>+		return -ENODEV;
>+	msg = genlmsg_new(NLMSG_GOODSIZE, GFP_KERNEL);
>+	if (!msg)
>+		return -ENOMEM;
>+	hdr = genlmsg_put_reply(msg, info, &dpll_nl_family, 0,
>+				DPLL_CMD_PIN_GET);
>+	if (!hdr)
>+		return -EMSGSIZE;
>+	nest = nla_nest_start(msg, DPLL_A_PIN);
>+	if (!nest)
>+		return -EMSGSIZE;
>+	ret = __dpll_cmd_pin_dump_one(msg, pin, info->extack, true);
>+	if (ret) {
>+		nlmsg_free(msg);
>+		return ret;
>+	}
>+	nla_nest_end(msg, nest);
>+	genlmsg_end(msg, hdr);
>+
>+	return genlmsg_reply(msg, info);
>+}
>+
>+int dpll_nl_pin_get_dumpit(struct sk_buff *skb, struct netlink_callback *cb)
>+{
>+	struct nlattr *hdr, *nest;
>+	struct dpll_pin *pin;
>+	unsigned long i;
>+	int ret;
>+
>+	hdr = genlmsg_put(skb, NETLINK_CB(cb->skb).portid, cb->nlh->nlmsg_seq,
>+			  &dpll_nl_family, 0, DPLL_CMD_PIN_GET);
>+	if (!hdr)
>+		return -EMSGSIZE;
>+
>+	xa_for_each(&dpll_pin_xa, i, pin) {
>+		if (xa_empty(&pin->dpll_refs))
>+			continue;
>+		nest = nla_nest_start(skb, DPLL_A_PIN);
>+		if (!nest) {
>+			ret = -EMSGSIZE;
>+			break;
>+		}
>+		ret = __dpll_cmd_pin_dump_one(skb, pin, cb->extack, true);
>+		if (ret) {
>+			nla_nest_cancel(skb, nest);
>+			break;
>+		}
>+		nla_nest_end(skb, nest);
>+	}
>+
>+	if (ret)
>+		genlmsg_cancel(skb, hdr);
>+	else
>+		genlmsg_end(skb, hdr);
>+
>+	return ret;
>+}
>+
>+static int
>+dpll_set_from_nlattr(struct dpll_device *dpll, struct genl_info *info)
>+{
>+	struct nlattr *attr;
>+	enum dpll_mode mode;
>+	int rem, ret = 0;
>+
>+	nla_for_each_attr(attr, genlmsg_data(info->genlhdr),
>+			  genlmsg_len(info->genlhdr), rem) {
>+		switch (nla_type(attr)) {
>+		case DPLL_A_MODE:
>+			mode = nla_get_u8(attr);
>+
>+			if (!dpll->ops || !dpll->ops->mode_set)
>+				return -EOPNOTSUPP;
>+			ret = dpll->ops->mode_set(dpll, mode, info->extack);
>+			if (ret)
>+				return ret;
>+			break;
>+		default:
>+			break;
>+		}
>+	}
>+
>+	return ret;
>+}
>+
>+int dpll_nl_device_set_doit(struct sk_buff *skb, struct genl_info *info)
>+{
>+	struct dpll_device *dpll = info->user_ptr[0];
>+
>+	return dpll_set_from_nlattr(dpll, info);
>+}
>+
>+int dpll_nl_device_get_doit(struct sk_buff *skb, struct genl_info *info)
>+{
>+	struct dpll_device *dpll = info->user_ptr[0];
>+	struct nlattr *hdr, *nest;
>+	struct sk_buff *msg;
>+	int ret;
>+
>+	msg = genlmsg_new(NLMSG_GOODSIZE, GFP_KERNEL);
>+	if (!msg)
>+		return -ENOMEM;
>+	hdr = genlmsg_put_reply(msg, info, &dpll_nl_family, 0,
>+				DPLL_CMD_DEVICE_GET);
>+	if (!hdr)
>+		return -EMSGSIZE;
>+
>+	nest = nla_nest_start(msg, DPLL_A_DEVICE);
>+	ret = dpll_device_get_one(dpll, msg, info->extack);
>+	if (ret) {
>+		nlmsg_free(msg);
>+		return ret;
>+	}
>+	nla_nest_end(msg, nest);
>+	genlmsg_end(msg, hdr);
>+
>+	return genlmsg_reply(msg, info);
>+}
>+
>+int dpll_nl_device_get_dumpit(struct sk_buff *skb, struct netlink_callback *cb)
>+{
>+	struct nlattr *hdr, *nest;
>+	struct dpll_device *dpll;
>+	unsigned long i;
>+	int ret;
>+
>+	hdr = genlmsg_put(skb, NETLINK_CB(cb->skb).portid, cb->nlh->nlmsg_seq,
>+			  &dpll_nl_family, 0, DPLL_CMD_DEVICE_GET);
>+	if (!hdr)
>+		return -EMSGSIZE;
>+
>+	xa_for_each_marked(&dpll_device_xa, i, dpll, DPLL_REGISTERED) {
>+		nest = nla_nest_start(skb, DPLL_A_DEVICE);
>+		ret = dpll_msg_add_dev_handle(skb, dpll);
>+		if (ret) {
>+			nla_nest_cancel(skb, nest);
>+			break;
>+		}
>+		nla_nest_end(skb, nest);
>+	}
>+	if (ret)
>+		genlmsg_cancel(skb, hdr);
>+	else
>+		genlmsg_end(skb, hdr);
>+
>+	return ret;
>+}
>+
>+int dpll_pre_doit(const struct genl_split_ops *ops, struct sk_buff *skb,
>+		  struct genl_info *info)
>+{
>+	struct dpll_device *dpll_id = NULL, *dpll_name = NULL;
>+	int ret = -ENODEV;
>+
>+	if (!info->attrs[DPLL_A_ID] &&
>+	    !(info->attrs[DPLL_A_BUS_NAME] && info->attrs[DPLL_A_DEV_NAME]))
>+		return -EINVAL;
>+
>+	mutex_lock(&dpll_device_xa_lock);
>+	if (info->attrs[DPLL_A_ID]) {
>+		u32 id = nla_get_u32(info->attrs[DPLL_A_ID]);
>+
>+		dpll_id = dpll_device_get_by_id(id);
>+		if (!dpll_id)
>+			goto unlock;
>+		info->user_ptr[0] = dpll_id;
>+	}
>+	if (info->attrs[DPLL_A_BUS_NAME] &&
>+	    info->attrs[DPLL_A_DEV_NAME]) {
>+		const char *bus_name = nla_data(info->attrs[DPLL_A_BUS_NAME]);
>+		const char *dev_name = nla_data(info->attrs[DPLL_A_DEV_NAME]);
>+
>+		dpll_name = dpll_device_get_by_name(bus_name, dev_name);
>+		if (!dpll_name) {
>+			ret = -ENODEV;
>+			goto unlock;
>+		}
>+
>+		if (dpll_id && dpll_name != dpll_id)
>+			goto unlock;
>+		info->user_ptr[0] = dpll_name;
>+	}
>+
>+	return 0;
>+unlock:
>+	mutex_unlock(&dpll_device_xa_lock);
>+	return ret;
>+}
>+
>+void dpll_post_doit(const struct genl_split_ops *ops, struct sk_buff *skb,
>+		    struct genl_info *info)
>+{
>+	mutex_unlock(&dpll_device_xa_lock);
>+}
>+
>+int dpll_pre_dumpit(struct netlink_callback *cb)
>+{
>+	mutex_lock(&dpll_device_xa_lock);
>+
>+	return 0;
>+}
>+
>+int dpll_post_dumpit(struct netlink_callback *cb)
>+{
>+	mutex_unlock(&dpll_device_xa_lock);
>+
>+	return 0;
>+}
>+
>+int dpll_pin_pre_doit(const struct genl_split_ops *ops, struct sk_buff *skb,
>+		      struct genl_info *info)
>+{
>+	int ret = dpll_pre_doit(ops, skb, info);
>+	struct dpll_device *dpll;
>+	struct dpll_pin *pin;
>+
>+	if (ret)
>+		return ret;
>+	dpll = info->user_ptr[0];
>+	if (!info->attrs[DPLL_A_PIN_IDX]) {
>+		ret = -EINVAL;
>+		goto unlock_dev;
>+	}
>+	mutex_lock(&dpll_pin_xa_lock);
>+	pin = dpll_pin_get_by_idx(dpll,
>+				  nla_get_u32(info->attrs[DPLL_A_PIN_IDX]));
>+	if (!pin) {
>+		ret = -ENODEV;
>+		goto unlock_pin;
>+	}
>+	info->user_ptr[1] = pin;
>+
>+	return 0;
>+
>+unlock_pin:
>+	mutex_unlock(&dpll_pin_xa_lock);
>+unlock_dev:
>+	mutex_unlock(&dpll_device_xa_lock);
>+	return ret;
>+}
>+
>+void dpll_pin_post_doit(const struct genl_split_ops *ops, struct sk_buff *skb,
>+			struct genl_info *info)
>+{
>+	mutex_unlock(&dpll_pin_xa_lock);
>+	dpll_post_doit(ops, skb, info);
>+}
>+
>+int dpll_pin_pre_dumpit(struct netlink_callback *cb)
>+{
>+	mutex_lock(&dpll_pin_xa_lock);

ABBA deadlock here, see dpll_pin_register() for example where the lock
taking order is opposite.


>+
>+	return dpll_pre_dumpit(cb);
>+}
>+
>+int dpll_pin_post_dumpit(struct netlink_callback *cb)
>+{
>+	mutex_unlock(&dpll_pin_xa_lock);
>+
>+	return dpll_post_dumpit(cb);
>+}
>+
>+static int
>+dpll_event_device_change(struct sk_buff *msg, struct dpll_device *dpll,
>+			 struct dpll_pin *pin, struct dpll_pin *parent,
>+			 enum dplla attr)
>+{
>+	int ret = dpll_msg_add_dev_handle(msg, dpll);
>+	struct dpll_pin_ref *ref = NULL;
>+	enum dpll_pin_state state;
>+
>+	if (ret)
>+		return ret;
>+	if (pin && nla_put_u32(msg, DPLL_A_PIN_IDX, pin->dev_driver_id))
>+		return -EMSGSIZE;

I don't really understand why you are trying figure something new and
interesting with the change notifications. This object mix and random
attrs fillup is something very wrong and makes userspace completely
fuzzy about what it is getting. And yet it is so simple:
You have 2 objects, dpll and pin, please just have:
dpll_notify()
dpll_pin_notify()
and share the attrs fillup code with pin_get() and dpll_get() callbacks.
No need for any smartness. Have this dumb and simple.

Think about it more as about "object-state-snapshot" than "atomic-change"


>+
>+	switch (attr) {
>+	case DPLL_A_MODE:
>+		ret = dpll_msg_add_mode(msg, dpll, NULL);
>+		break;
>+	case DPLL_A_SOURCE_PIN_IDX:
>+		ret = dpll_msg_add_source_pin_idx(msg, dpll, NULL);
>+		break;
>+	case DPLL_A_LOCK_STATUS:
>+		ret = dpll_msg_add_lock_status(msg, dpll, NULL);
>+		break;
>+	case DPLL_A_TEMP:
>+		ret = dpll_msg_add_temp(msg, dpll, NULL);
>+		break;
>+	case DPLL_A_PIN_FREQUENCY:
>+		ret = dpll_msg_add_pin_freq(msg, pin, NULL, false);
>+		break;
>+	case DPLL_A_PIN_PRIO:
>+		ref = dpll_xa_ref_dpll_find(&pin->dpll_refs, dpll);
>+		if (!ref)
>+			return -EFAULT;
>+		ret = dpll_msg_add_pin_prio(msg, pin, ref, NULL);
>+		break;
>+	case DPLL_A_PIN_STATE:
>+		if (parent) {
>+			ref = dpll_xa_ref_pin_find(&pin->parent_refs, parent);
>+			if (!ref)
>+				return -EFAULT;
>+			if (!ref->ops || !ref->ops->state_on_pin_get)
>+				return -EOPNOTSUPP;
>+			ret = ref->ops->state_on_pin_get(pin, parent, &state,
>+							 NULL);
>+			if (ret)
>+				return ret;
>+			if (nla_put_u32(msg, DPLL_A_PIN_PARENT_IDX,
>+					parent->dev_driver_id))
>+				return -EMSGSIZE;
>+		} else {
>+			ref = dpll_xa_ref_dpll_find(&pin->dpll_refs, dpll);
>+			if (!ref)
>+				return -EFAULT;
>+			ret = dpll_msg_add_pin_on_dpll_state(msg, pin, ref,
>+							     NULL);
>+			if (ret)
>+				return ret;
>+		}
>+		break;
>+	default:
>+		break;
>+	}
>+
>+	return ret;
>+}
>+
>+static int
>+dpll_send_event_create(enum dpll_event event, struct dpll_device *dpll)
>+{
>+	struct sk_buff *msg;
>+	int ret = -EMSGSIZE;
>+	void *hdr;
>+
>+	msg = genlmsg_new(NLMSG_GOODSIZE, GFP_KERNEL);
>+	if (!msg)
>+		return -ENOMEM;
>+
>+	hdr = genlmsg_put(msg, 0, 0, &dpll_nl_family, 0, event);
>+	if (!hdr)
>+		goto out_free_msg;
>+
>+	ret = dpll_msg_add_dev_handle(msg, dpll);
>+	if (ret)
>+		goto out_cancel_msg;
>+	genlmsg_end(msg, hdr);
>+	genlmsg_multicast(&dpll_nl_family, msg, 0, 0, GFP_KERNEL);
>+
>+	return 0;
>+
>+out_cancel_msg:
>+	genlmsg_cancel(msg, hdr);
>+out_free_msg:
>+	nlmsg_free(msg);
>+
>+	return ret;
>+}
>+
>+static int
>+dpll_send_event_change(struct dpll_device *dpll, struct dpll_pin *pin,
>+		       struct dpll_pin *parent, enum dplla attr)
>+{
>+	struct sk_buff *msg;
>+	int ret = -EMSGSIZE;
>+	void *hdr;
>+
>+	msg = genlmsg_new(NLMSG_GOODSIZE, GFP_KERNEL);
>+	if (!msg)
>+		return -ENOMEM;
>+
>+	hdr = genlmsg_put(msg, 0, 0, &dpll_nl_family, 0,
>+			  DPLL_EVENT_DEVICE_CHANGE);

I don't really get it. Why exactly you keep having this *EVENT* cmds?
Why per-object NEW/GET/DEL cmds shared with get genl op are not enough?
I have to be missing something.


>+	if (!hdr)
>+		goto out_free_msg;
>+
>+	ret = dpll_event_device_change(msg, dpll, pin, parent, attr);
>+	if (ret)
>+		goto out_cancel_msg;
>+	genlmsg_end(msg, hdr);
>+	genlmsg_multicast(&dpll_nl_family, msg, 0, 0, GFP_KERNEL);
>+
>+	return 0;
>+
>+out_cancel_msg:
>+	genlmsg_cancel(msg, hdr);
>+out_free_msg:
>+	nlmsg_free(msg);
>+
>+	return ret;
>+}
>+
>+int dpll_notify_device_create(struct dpll_device *dpll)
>+{
>+	return dpll_send_event_create(DPLL_EVENT_DEVICE_CREATE, dpll);
>+}
>+
>+int dpll_notify_device_delete(struct dpll_device *dpll)
>+{
>+	return dpll_send_event_create(DPLL_EVENT_DEVICE_DELETE, dpll);
>+}
>+
>+int dpll_device_notify(struct dpll_device *dpll, enum dplla attr)
>+{
>+	if (WARN_ON(!dpll))
>+		return -EINVAL;
>+
>+	return dpll_send_event_change(dpll, NULL, NULL, attr);
>+}
>+EXPORT_SYMBOL_GPL(dpll_device_notify);
>+
>+int dpll_pin_notify(struct dpll_device *dpll, struct dpll_pin *pin,
>+		    enum dplla attr)
>+{
>+	return dpll_send_event_change(dpll, pin, NULL, attr);
>+}
>+
>+int dpll_pin_parent_notify(struct dpll_device *dpll, struct dpll_pin *pin,
>+			   struct dpll_pin *parent, enum dplla attr)
>+{
>+	return dpll_send_event_change(dpll, pin, parent, attr);
>+}
>+
>+int __init dpll_netlink_init(void)
>+{
>+	return genl_register_family(&dpll_nl_family);
>+}
>+
>+void dpll_netlink_finish(void)
>+{
>+	genl_unregister_family(&dpll_nl_family);
>+}
>+
>+void __exit dpll_netlink_fini(void)
>+{
>+	dpll_netlink_finish();
>+}
>diff --git a/drivers/dpll/dpll_netlink.h b/drivers/dpll/dpll_netlink.h
>new file mode 100644
>index 000000000000..072efa10f0e6
>--- /dev/null
>+++ b/drivers/dpll/dpll_netlink.h
>@@ -0,0 +1,30 @@
>+/* SPDX-License-Identifier: GPL-2.0 */
>+/*
>+ *  Copyright (c) 2021 Meta Platforms, Inc. and affiliates
>+ */
>+
>+/**
>+ * dpll_notify_device_create - notify that the device has been created
>+ * @dpll: registered dpll pointer
>+ *
>+ * Return: 0 if succeeds, error code otherwise.
>+ */
>+int dpll_notify_device_create(struct dpll_device *dpll);
>+
>+
>+/**
>+ * dpll_notify_device_delete - notify that the device has been deleted
>+ * @dpll: registered dpll pointer
>+ *
>+ * Return: 0 if succeeds, error code otherwise.
>+ */
>+int dpll_notify_device_delete(struct dpll_device *dpll);
>+
>+int dpll_pin_notify(struct dpll_device *dpll, struct dpll_pin *pin,
>+		    enum dplla attr);
>+
>+int dpll_pin_parent_notify(struct dpll_device *dpll, struct dpll_pin *pin,
>+			   struct dpll_pin *parent, enum dplla attr);
>+
>+int __init dpll_netlink_init(void);
>+void dpll_netlink_finish(void);
>diff --git a/include/linux/dpll.h b/include/linux/dpll.h
>new file mode 100644
>index 000000000000..db98b6d4bb73
>--- /dev/null
>+++ b/include/linux/dpll.h
>@@ -0,0 +1,284 @@
>+/* SPDX-License-Identifier: GPL-2.0 */
>+/*
>+ *  Copyright (c) 2021 Meta Platforms, Inc. and affiliates
>+ */
>+
>+#ifndef __DPLL_H__
>+#define __DPLL_H__
>+
>+#include <uapi/linux/dpll.h>
>+#include <linux/device.h>
>+#include <linux/netlink.h>
>+
>+struct dpll_device;
>+struct dpll_pin;
>+
>+#define PIN_IDX_INVALID		((u32)ULONG_MAX)
>+
>+struct dpll_device_ops {
>+	int (*mode_get)(const struct dpll_device *dpll, enum dpll_mode *mode,
>+			struct netlink_ext_ack *extack);
>+	int (*mode_set)(const struct dpll_device *dpll,
>+			const enum dpll_mode mode,
>+			struct netlink_ext_ack *extack);
>+	bool (*mode_supported)(const struct dpll_device *dpll,
>+			       const enum dpll_mode mode,
>+			       struct netlink_ext_ack *extack);
>+	int (*source_pin_idx_get)(const struct dpll_device *dpll,
>+				  u32 *pin_idx,
>+				  struct netlink_ext_ack *extack);
>+	int (*lock_status_get)(const struct dpll_device *dpll,
>+			       enum dpll_lock_status *status,
>+			       struct netlink_ext_ack *extack);
>+	int (*temp_get)(const struct dpll_device *dpll, s32 *temp,
>+			struct netlink_ext_ack *extack);
>+};
>+
>+struct dpll_pin_ops {
>+	int (*frequency_set)(const struct dpll_pin *pin,
>+			     const struct dpll_device *dpll,
>+			     const u32 frequency,
>+			     struct netlink_ext_ack *extack);
>+	int (*frequency_get)(const struct dpll_pin *pin,
>+			     const struct dpll_device *dpll,
>+			     u32 *frequency, struct netlink_ext_ack *extack);
>+	int (*direction_set)(const struct dpll_pin *pin,
>+			     const struct dpll_device *dpll,
>+			     const enum dpll_pin_direction direction,
>+			     struct netlink_ext_ack *extack);
>+	int (*direction_get)(const struct dpll_pin *pin,
>+			     const struct dpll_device *dpll,
>+			     enum dpll_pin_direction *direction,
>+			     struct netlink_ext_ack *extack);
>+	int (*state_on_pin_get)(const struct dpll_pin *pin,
>+				const struct dpll_pin *parent_pin,
>+				enum dpll_pin_state *state,
>+				struct netlink_ext_ack *extack);
>+	int (*state_on_dpll_get)(const struct dpll_pin *pin,
>+				 const struct dpll_device *dpll,
>+				 enum dpll_pin_state *state,
>+				 struct netlink_ext_ack *extack);
>+	int (*state_on_pin_set)(const struct dpll_pin *pin,
>+				const struct dpll_pin *parent_pin,
>+				const enum dpll_pin_state state,
>+				struct netlink_ext_ack *extack);
>+	int (*state_on_dpll_set)(const struct dpll_pin *pin,
>+				 const struct dpll_device *dpll,
>+				 const enum dpll_pin_state state,
>+				 struct netlink_ext_ack *extack);
>+	int (*prio_get)(const struct dpll_pin *pin,
>+			const struct dpll_device *dpll,
>+			u32 *prio, struct netlink_ext_ack *extack);
>+	int (*prio_set)(const struct dpll_pin *pin,
>+			const struct dpll_device *dpll,
>+			const u32 prio, struct netlink_ext_ack *extack);
>+};
>+
>+struct dpll_pin_properties {
>+	const char *description;
>+	enum dpll_pin_type type;
>+	unsigned long freq_supported;
>+	u32 any_freq_min;
>+	u32 any_freq_max;
>+	unsigned long capabilities;
>+};
>+
>+enum dpll_pin_freq_supp {
>+	DPLL_PIN_FREQ_SUPP_UNSPEC = 0,
>+	DPLL_PIN_FREQ_SUPP_1_HZ,
>+	DPLL_PIN_FREQ_SUPP_10_MHZ,
>+
>+	__DPLL_PIN_FREQ_SUPP_MAX,
>+	DPLL_PIN_FREQ_SUPP_MAX = (__DPLL_PIN_FREQ_SUPP_MAX - 1)
>+};
>+
>+/**
>+ * dpll_device_get - find or create dpll_device object
>+ * @clock_id: a system unique number for a device
>+ * @dev_driver_idx: index of dpll device on parent device
>+ * @module: register module
>+ *
>+ * Returns:
>+ * * pointer to initialized dpll - success
>+ * * NULL - memory allocation fail
>+ */
>+struct dpll_device
>+*dpll_device_get(u64 clock_id, u32 dev_driver_id, struct module *module);
>+
>+/**
>+ * dpll_device_put - caller drops reference to the device, free resources
>+ * @dpll: dpll device pointer
>+ *
>+ * If all dpll_device_get callers drops their reference, the dpll device
>+ * resources are freed.
>+ */
>+void dpll_device_put(struct dpll_device *dpll);
>+
>+/**
>+ * dpll_device_register - register device, make it visible in the subsystem.
>+ * @dpll: reference previously allocated with dpll_device_get
>+ * @type: type of dpll
>+ * @ops: callbacks
>+ * @priv: private data of registerer
>+ * @owner: device struct of the owner
>+ *
>+ */
>+int dpll_device_register(struct dpll_device *dpll, enum dpll_type type,
>+			 struct dpll_device_ops *ops, void *priv,
>+			 struct device *owner);
>+
>+/**
>+ * dpll_device_unregister - deregister registered dpll
>+ * @dpll: pointer to dpll
>+ *
>+ * Unregister the dpll from the subsystem, make it unavailable for netlink
>+ * API users.
>+ */
>+void dpll_device_unregister(struct dpll_device *dpll);
>+
>+/**
>+ * dpll_priv - get dpll private data
>+ * @dpll: pointer to dpll
>+ *
>+ * Obtain private data pointer passed to dpll subsystem when allocating
>+ * device with ``dpll_device_alloc(..)``
>+ */
>+void *dpll_priv(const struct dpll_device *dpll);
>+
>+/**
>+ * dpll_pin_on_pin_priv - get pin on pin pair private data
>+ * @parent: pointer to a parent pin
>+ * @pin: pointer to a dpll_pin
>+ *
>+ * Obtain private pin data pointer passed to dpll subsystem when pin
>+ * was registered with parent pin.
>+ */
>+void *dpll_pin_on_pin_priv(const struct dpll_pin *parent, const struct dpll_pin *pin);
>+
>+/**
>+ * dpll_pin_on_dpll_priv - get pin on dpll pair private data
>+ * @dpll: pointer to dpll
>+ * @pin: pointer to a dpll_pin
>+ *
>+ * Obtain private pin-dpll pair data pointer passed to dpll subsystem when pin
>+ * was registered with a dpll.
>+ */
>+void *dpll_pin_on_dpll_priv(const struct dpll_device *dpll, const struct dpll_pin *pin);
>+
>+/**
>+ * dpll_pin_get - get reference or create new pin object
>+ * @clock_id: a system unique number of a device
>+ * @dev_driver_idx: index of dpll device on parent device
>+ * @module: register module
>+ * @pin_prop: constant properities of a pin
>+ *
>+ * find existing pin with given clock_id, dev_driver_idx and module, or create new
>+ * and returen its reference.
>+ *
>+ * Returns:
>+ * * pointer to initialized pin - success
>+ * * NULL - memory allocation fail
>+ */
>+struct dpll_pin
>+*dpll_pin_get(u64 clock_id, u32 dev_driver_id, struct module *module,

Name inconsistency: dev_driver_id/idx in comment


>+	      const struct dpll_pin_properties *pin_prop);

In .c you call this "prop", be consistent.


>+
>+/**
>+ * dpll_pin_register - register pin with a dpll device
>+ * @dpll: pointer to dpll object to register pin with
>+ * @pin: pointer to allocated pin object being registered with dpll
>+ * @ops: struct with pin ops callbacks
>+ * @priv: private data pointer passed when calling callback ops
>+ * @rclk_device: pointer to device struct if pin is used for recovery of a clock
>+ * from that device
>+ *
>+ * Register previously allocated pin object with a dpll device.
>+ *
>+ * Return:
>+ * * 0 - if pin was registered with a parent pin,
>+ * * -ENOMEM - failed to allocate memory,
>+ * * -EEXIST - pin already registered with this dpll,
>+ * * -EBUSY - couldn't allocate id for a pin.
>+ */
>+int dpll_pin_register(struct dpll_device *dpll, struct dpll_pin *pin,
>+		      struct dpll_pin_ops *ops, void *priv,
>+		      struct device *rclk_device);
>+
>+/**
>+ * dpll_pin_unregister - deregister pin from a dpll device
>+ * @dpll: pointer to dpll object to deregister pin from
>+ * @pin: pointer to allocated pin object being deregistered from dpll
>+ *
>+ * Deregister previously registered pin object from a dpll device.
>+ *
>+ * Return:
>+ * * 0 - pin was successfully deregistered from this dpll device,
>+ * * -ENXIO - given pin was not registered with this dpll device,
>+ * * -EINVAL - pin pointer is not valid.
>+ */
>+int dpll_pin_unregister(struct dpll_device *dpll, struct dpll_pin *pin);
>+
>+/**
>+ * dpll_pin_put - drop reference to a pin acquired with dpll_pin_get
>+ * @pin: pointer to allocated pin
>+ *
>+ * Pins shall be deregistered from all dpll devices before putting them,
>+ * otherwise the memory won't be freed.
>+ */
>+void dpll_pin_put(struct dpll_pin *pin);
>+
>+/**
>+ * dpll_pin_on_pin_register - register a pin to a muxed-type pin
>+ * @parent: parent pin pointer
>+ * @pin: pointer to allocated pin object being registered with a parent pin
>+ * @ops: struct with pin ops callbacks
>+ * @priv: private data pointer passed when calling callback ops
>+ * @rclk_device: pointer to device struct if pin is used for recovery of a clock
>+ * from that device
>+ *
>+ * In case of multiplexed pins, allows registring them under a single
>+ * parent pin.
>+ *
>+ * Return:
>+ * * 0 - if pin was registered with a parent pin,
>+ * * -ENOMEM - failed to allocate memory,
>+ * * -EEXIST - pin already registered with this parent pin,
>+ */
>+int dpll_pin_on_pin_register(struct dpll_pin *parent, struct dpll_pin *pin,
>+			     struct dpll_pin_ops *ops, void *priv,
>+			     struct device *rclk_device);
>+
>+/**
>+ * dpll_pin_on_pin_register - register a pin to a muxed-type pin
>+ * @parent: parent pin pointer
>+ * @pin: pointer to allocated pin object being registered with a parent pin
>+ * @ops: struct with pin ops callbacks
>+ * @priv: private data pointer passed when calling callback ops
>+ * @rclk_device: pointer to device struct if pin is used for recovery of a clock
>+ * from that device
>+ *
>+ * In case of multiplexed pins, allows registring them under a single
>+ * parent pin.
>+ *
>+ * Return:
>+ * * 0 - if pin was registered with a parent pin,
>+ * * -ENOMEM - failed to allocate memory,
>+ * * -EEXIST - pin already registered with this parent pin,
>+ */
>+void dpll_pin_on_pin_unregister(struct dpll_pin *parent, struct dpll_pin *pin);
>+
>+/**
>+ * dpll_device_notify - notify on dpll device change
>+ * @dpll: dpll device pointer
>+ * @attr: changed attribute
>+ *
>+ * Broadcast event to the netlink multicast registered listeners.
>+ *
>+ * Return:
>+ * * 0 - success
>+ * * negative - error
>+ */
>+int dpll_device_notify(struct dpll_device *dpll, enum dplla attr);
>+
>+
>+#endif
>-- 
>2.34.1
>

Vadim Fedorenko March 13, 2023, 10:59 p.m. UTC | #2

On 13.03.2023 16:21, Jiri Pirko wrote:
> Sun, Mar 12, 2023 at 03:28:03AM CET, [e-mail] wrote:
>> DPLL framework is used to represent and configure DPLL devices
>> in systems. Each device that has DPLL and can configure sources
>> and outputs can use this framework. Netlink interface is used to
>> provide configuration data and to receive notification messages
>> about changes in the configuration or status of DPLL device.
>> Inputs and outputs of the DPLL device are represented as special
>> objects which could be dynamically added to and removed from DPLL
>> device.
>>
>> Changes:
>> dpll: adapt changes after introduction of dpll yaml spec
>> dpll: redesign after review comments, fix minor issues
>> dpll: add get pin command
>> dpll: _get/_put approach for creating and realesing pin or dpll objects
>> dpll: lock access to dplls with global lock
>> dpll: lock access to pins with global lock
>>
>> dpll: replace cookie with clock id
>> dpll: add clock class
> 
> Looks like some leftover in patch description.

Yeah, it's a changelog from the previous iteration.

>>
>> Provide userspace with clock class value of DPLL with dpll device dump
>> netlink request. Clock class is assigned by driver allocating a dpll
>> device. Clock class values are defined as specified in:
>> ITU-T G.8273.2/Y.1368.2 recommendation.
>>
>> dpll: follow one naming schema in dpll subsys
>>
>> dpll: fix dpll device naming scheme
>>
>> Fix dpll device naming scheme by use of new pattern.
>> "dpll_%s_%d_%d", where:
>> - %s - dev_name(parent) of parent device,
>> - %d (1) - enum value of dpll type,
>> - %d (2) - device index provided by parent device.
>>
>> dpll: remove description length parameter
>>
>> dpll: fix muxed/shared pin registration
>>
>> Let the kernel module to register a shared or muxed pin without finding
>> it or its parent. Instead use a parent/shared pin description to find
>> correct pin internally in dpll_core, simplifing a dpll API.
>>
>> dpll: move function comments to dpll_core.c, fix exports
>> dpll: remove single-use helper functions
>> dpll: merge device register with alloc
>> dpll: lock and unlock mutex on dpll device release
>> dpll: move dpll_type to uapi header
>> dpll: rename DPLLA_DUMP_FILTER to DPLLA_FILTER
>> dpll: rename dpll_pin_state to dpll_pin_mode
>> dpll: rename DPLL_MODE_FORCED to DPLL_MODE_MANUAL
>> dpll: remove DPLL_CHANGE_PIN_TYPE enum value
> 
> I'm confused, some messed-up squash?

Yep, from previous iteration. I'll remove it next time.

>> Co-developed-by: Milena Olech <[e-mail]>
>> Signed-off-by: Milena Olech <[e-mail]>
>> Co-developed-by: Michal Michalik <[e-mail]>
>> Signed-off-by: Michal Michalik <[e-mail]>
>> Co-developed-by: Arkadiusz Kubalewski <[e-mail]>
>> Signed-off-by: Arkadiusz Kubalewski <[e-mail]>
>> Signed-off-by: Vadim Fedorenko <[e-mail]>
>> ---
>> MAINTAINERS                 |    9 +
>> drivers/Kconfig             |    2 +
>> drivers/Makefile            |    1 +
>> drivers/dpll/Kconfig        |    7 +
>> drivers/dpll/Makefile       |   10 +
>> drivers/dpll/dpll_core.c    |  835 +++++++++++++++++++++++++++
>> drivers/dpll/dpll_core.h    |   99 ++++
>> drivers/dpll/dpll_netlink.c | 1065 +++++++++++++++++++++++++++++++++++
>> drivers/dpll/dpll_netlink.h |   30 +
>> include/linux/dpll.h        |  284 ++++++++++
>> 10 files changed, 2342 insertions(+)
>> create mode 100644 drivers/dpll/Kconfig
>> create mode 100644 drivers/dpll/Makefile
>> create mode 100644 drivers/dpll/dpll_core.c
>> create mode 100644 drivers/dpll/dpll_core.h
>> create mode 100644 drivers/dpll/dpll_netlink.c
>> create mode 100644 drivers/dpll/dpll_netlink.h
>> create mode 100644 include/linux/dpll.h
>>
>> diff --git a/MAINTAINERS b/MAINTAINERS
>> index edd3d562beee..0222b19af545 100644
>> --- a/MAINTAINERS
>> +++ b/MAINTAINERS
>> @@ -6289,6 +6289,15 @@ F:	Documentation/networking/device_drivers/ethernet/freescale/dpaa2/switch-drive
>> F:	drivers/net/ethernet/freescale/dpaa2/dpaa2-switch*
>> F:	drivers/net/ethernet/freescale/dpaa2/dpsw*
>>
>> +DPLL CLOCK SUBSYSTEM
>> +M:	Vadim Fedorenko <[e-mail]>
>> +M:	Arkadiusz Kubalewski <[e-mail]>
>> +L:	[e-mail]
>> +S:	Maintained
>> +F:	drivers/dpll/*
>> +F:	include/net/dpll.h
>> +F:	include/uapi/linux/dpll.h
>> +
>> DRBD DRIVER
>> M:	Philipp Reisner <[e-mail]>
>> M:	Lars Ellenberg <[e-mail]>
>> diff --git a/drivers/Kconfig b/drivers/Kconfig
>> index 968bd0a6fd78..453df9e1210d 100644
>> --- a/drivers/Kconfig
>> +++ b/drivers/Kconfig
>> @@ -241,4 +241,6 @@ source "drivers/peci/Kconfig"
>>
>> source "drivers/hte/Kconfig"
>>
>> +source "drivers/dpll/Kconfig"
>> +
>> endmenu
>> diff --git a/drivers/Makefile b/drivers/Makefile
>> index 20b118dca999..9ffb554507ef 100644
>> --- a/drivers/Makefile
>> +++ b/drivers/Makefile
>> @@ -194,3 +194,4 @@ obj-$(CONFIG_MOST)		+= most/
>> obj-$(CONFIG_PECI)		+= peci/
>> obj-$(CONFIG_HTE)		+= hte/
>> obj-$(CONFIG_DRM_ACCEL)		+= accel/
>> +obj-$(CONFIG_DPLL)		+= dpll/
>> diff --git a/drivers/dpll/Kconfig b/drivers/dpll/Kconfig
>> new file mode 100644
>> index 000000000000..a4cae73f20d3
>> --- /dev/null
>> +++ b/drivers/dpll/Kconfig
>> @@ -0,0 +1,7 @@
>> +# SPDX-License-Identifier: GPL-2.0-only
>> +#
>> +# Generic DPLL drivers configuration
>> +#
>> +
>> +config DPLL
>> +  bool
>> diff --git a/drivers/dpll/Makefile b/drivers/dpll/Makefile
>> new file mode 100644
>> index 000000000000..d3926f2a733d
>> --- /dev/null
>> +++ b/drivers/dpll/Makefile
>> @@ -0,0 +1,10 @@
>> +# SPDX-License-Identifier: GPL-2.0
>> +#
>> +# Makefile for DPLL drivers.
>> +#
>> +
>> +obj-$(CONFIG_DPLL)          += dpll_sys.o
> 
> What's "sys" and why is it here?

It's an object file for the subsystem. Could be useful if we will have drivers
for DPLL-only devices.

>> +dpll_sys-y                  += dpll_core.o
>> +dpll_sys-y                  += dpll_netlink.o
>> +dpll_sys-y                  += dpll_nl.o
>> +
>> diff --git a/drivers/dpll/dpll_core.c b/drivers/dpll/dpll_core.c
>> new file mode 100644
>> index 000000000000..3fc151e16751
>> --- /dev/null
>> +++ b/drivers/dpll/dpll_core.c
>> @@ -0,0 +1,835 @@
>> +// SPDX-License-Identifier: GPL-2.0
>> +/*
>> + *  dpll_core.c - Generic DPLL Management class support.
>> + *
>> + *  Copyright (c) 2021 Meta Platforms, Inc. and affiliates
> 
> IIUC, there's a lot of Intel work behind this, I think some credits
> should go here as well.
> 
> Also, it is 2023, you live in the past :)

Yeah, that's true. Have to improve it. As well as in other files.

>> + */
>> +
>> +#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
>> +
>> +#include <linux/device.h>
>> +#include <linux/err.h>
>> +#include <linux/slab.h>
>> +#include <linux/string.h>
>> +
>> +#include "dpll_core.h"
>> +
>> +DEFINE_MUTEX(dpll_device_xa_lock);
>> +DEFINE_MUTEX(dpll_pin_xa_lock);
>> +
>> +DEFINE_XARRAY_FLAGS(dpll_device_xa, XA_FLAGS_ALLOC);
>> +DEFINE_XARRAY_FLAGS(dpll_pin_xa, XA_FLAGS_ALLOC);
>> +
>> +#define ASSERT_DPLL_REGISTERED(d)                                          \
>> +	WARN_ON_ONCE(!xa_get_mark(&dpll_device_xa, (d)->id, DPLL_REGISTERED))
>> +#define ASSERT_DPLL_NOT_REGISTERED(d)                                      \
>> +	WARN_ON_ONCE(xa_get_mark(&dpll_device_xa, (d)->id, DPLL_REGISTERED))
>> +
>> +static struct class dpll_class = {
>> +	.name = "dpll",
>> +};
>> +
>> +/**
>> + * dpll_device_get_by_id - find dpll device by it's id
>> + * @id: id of searched dpll
>> + *
>> + * Return:
>> + * * dpll_device struct if found
>> + * * NULL otherwise
>> + */
>> +struct dpll_device *dpll_device_get_by_id(int id)
>> +{
>> +	struct dpll_device *dpll = NULL;
>> +
>> +	if (xa_get_mark(&dpll_device_xa, id, DPLL_REGISTERED))
>> +		dpll = xa_load(&dpll_device_xa, id);
> 		return xa_load
> 
>> +
> 	return NULL
> 
>> +	return dpll;
> 
> Anyway, I believe this fuction should go away, see below.

Don't think it matters, but happy to remove the function.

>> +}
>> +
>> +/**
>> + * dpll_device_get_by_name - find dpll device by it's id
>> + * @bus_name: bus name of searched dpll
>> + * @dev_name: dev name of searched dpll
>> + *
>> + * Return:
>> + * * dpll_device struct if found
>> + * * NULL otherwise
>> + */
>> +struct dpll_device *
>> +dpll_device_get_by_name(const char *bus_name, const char *device_name)
>> +{
>> +	struct dpll_device *dpll, *ret = NULL;
>> +	unsigned long index;
>> +
>> +	xa_for_each_marked(&dpll_device_xa, index, dpll, DPLL_REGISTERED) {
>> +		if (!strcmp(dev_bus_name(&dpll->dev), bus_name) &&
>> +		    !strcmp(dev_name(&dpll->dev), device_name)) {
>> +			ret = dpll;
>> +			break;
>> +		}
>> +	}
>> +
>> +	return ret;
>> +}
>> +
>> +/**
>> + * dpll_xa_ref_pin_add - add pin reference to a given xarray
>> + * @xa_pins: dpll_pin_ref xarray holding pins
>> + * @pin: pin being added
>> + * @ops: ops for a pin
>> + * @priv: pointer to private data of owner
>> + *
>> + * Allocate and create reference of a pin or increase refcount on existing pin
>> + * reference on given xarray.
>> + *
>> + * Return:
>> + * * 0 on success
>> + * * -ENOMEM on failed allocation
>> + */
>> +static int
>> +dpll_xa_ref_pin_add(struct xarray *xa_pins, struct dpll_pin *pin,
>> +		    struct dpll_pin_ops *ops, void *priv)
>> +{
>> +	struct dpll_pin_ref *ref;
>> +	unsigned long i;
> 
> In previous function you use "index" name for the same variable. Be
> consistent. I think "i" is actually better.
> 

Yep, agree.

> 
>> +	u32 idx;
>> +	int ret;
>> +
>> +	xa_for_each(xa_pins, i, ref) {
>> +		if (ref->pin == pin) {
>> +			refcount_inc(&ref->refcount);
>> +			return 0;
>> +		}
>> +	}
>> +
>> +	ref = kzalloc(sizeof(*ref), GFP_KERNEL);
>> +	if (!ref)
>> +		return -ENOMEM;
>> +	ref->pin = pin;
>> +	ref->ops = ops;
>> +	ref->priv = priv;
>> +	ret = xa_alloc(xa_pins, &idx, ref, xa_limit_16b, GFP_KERNEL);
>> +	if (!ret)
>> +		refcount_set(&ref->refcount, 1);
>> +	else
>> +		kfree(ref);
>> +
>> +	return ret;
>> +}
>> +
>> +/**
>> + * dpll_xa_ref_pin_del - remove reference of a pin from xarray
>> + * @xa_pins: dpll_pin_ref xarray holding pins
>> + * @pin: pointer to a pin
>> + *
>> + * Decrement refcount of existing pin reference on given xarray.
>> + * If all references are dropped, delete the reference and free its memory.
>> + *
>> + * Return:
>> + * * 0 on success
>> + * * -EINVAL if reference to a pin was not found
>> + */

[...]

>> +/**
>> + * dpll_device_alloc - allocate the memory for dpll device
>> + * @clock_id: clock_id of creator
>> + * @dev_driver_id: id given by dev driver
>> + * @module: reference to registering module
>> + *
>> + * Allocates memory and initialize dpll device, hold its reference on global
>> + * xarray.
>> + *
>> + * Return:
>> + * * dpll_device struct pointer if succeeded
>> + * * ERR_PTR(X) - failed allocation
>> + */
>> +struct dpll_device *
>> +dpll_device_alloc(const u64 clock_id, u32 dev_driver_id, struct module *module)
> 
> This should be static. Didn't you get warn?

I did get an email from kernel test robot. I'll make it static. As well as 
dpll_pin_alloc()

>> +{
>> +	struct dpll_device *dpll;
>> +	int ret;
>> +
>> +	dpll = kzalloc(sizeof(*dpll), GFP_KERNEL);
>> +	if (!dpll)
>> +		return ERR_PTR(-ENOMEM);
>> +	refcount_set(&dpll->refcount, 1);
>> +	dpll->dev.class = &dpll_class;
>> +	dpll->dev_driver_id = dev_driver_id;
>> +	dpll->clock_id = clock_id;
>> +	dpll->module = module;
>> +	ret = xa_alloc(&dpll_device_xa, &dpll->id, dpll,
>> +		       xa_limit_16b, GFP_KERNEL);
>> +	if (ret) {
>> +		kfree(dpll);
>> +		return ERR_PTR(ret);
>> +	}
>> +	xa_init_flags(&dpll->pin_refs, XA_FLAGS_ALLOC);
>> +
>> +	return dpll;
>> +}
>> +
>> +/**
>> + * dpll_device_get - find existing or create new dpll device
>> + * @clock_id: clock_id of creator
>> + * @dev_driver_id: id given by dev driver
>> + * @module: reference to registering module
>> + *
>> + * Get existing object of a dpll device, unique for given arguments.
>> + * Create new if doesn't exist yet.
>> + *
>> + * Return:
>> + * * valid dpll_device struct pointer if succeeded
>> + * * ERR_PTR of an error
>> + */
>> +struct dpll_device *
>> +dpll_device_get(u64 clock_id, u32 dev_driver_id, struct module *module)
>> +{
>> +	struct dpll_device *dpll, *ret = NULL;
>> +	unsigned long index;
>> +
>> +	mutex_lock(&dpll_device_xa_lock);
>> +	xa_for_each(&dpll_device_xa, index, dpll) {
>> +		if (dpll->clock_id == clock_id &&
>> +		    dpll->dev_driver_id == dev_driver_id &&
> 
> Why you need "dev_driver_id"? clock_id is here for the purpose of
> identification, isn't that enough for you.

dev_driver_id is needed to provide several DPLLs from one device. In ice driver
implementation there are 2 different DPLLs - to recover from PPS input and to 
recover from Sync-E. I believe there is only one clock, that's why clock id is 
the same for both of them. But Arkadiusz can tell more about it.
> 
> Plus, the name is odd. "dev_driver" should certainly be avoided.

Simply id doesn't tell anything either. dpll_dev_id?

>> +		    dpll->module == module) {
>> +			ret = dpll;
>> +			refcount_inc(&ret->refcount);
>> +			break;
>> +		}
>> +	}
>> +	if (!ret)
>> +		ret = dpll_device_alloc(clock_id, dev_driver_id, module);
>> +	mutex_unlock(&dpll_device_xa_lock);
>> +
>> +	return ret;
>> +}
>> +EXPORT_SYMBOL_GPL(dpll_device_get);
>> +
>> +/**
>> + * dpll_device_put - decrease the refcount and free memory if possible
>> + * @dpll: dpll_device struct pointer
>> + *
>> + * Drop reference for a dpll device, if all references are gone, delete
>> + * dpll device object.
>> + */
>> +void dpll_device_put(struct dpll_device *dpll)
>> +{
>> +	if (!dpll)
>> +		return;
> 
> Remove this check. The driver should not call this with NULL.

Well, netdev_put() has this kind of check. As well as spi_dev_put() or
i2c_put_adapter() at least. Not sure I would like to avoid a bit of safety.

>> +	mutex_lock(&dpll_device_xa_lock);
>> +	if (refcount_dec_and_test(&dpll->refcount)) {
>> +		WARN_ON_ONCE(!xa_empty(&dpll->pin_refs));
> 
> ASSERT_DPLL_NOT_REGISTERED(dpll);

Good point!

>> +		xa_destroy(&dpll->pin_refs);
>> +		xa_erase(&dpll_device_xa, dpll->id);
>> +		kfree(dpll);
>> +	}
>> +	mutex_unlock(&dpll_device_xa_lock);
>> +}
>> +EXPORT_SYMBOL_GPL(dpll_device_put);
>> +
>> +/**
>> + * dpll_device_register - register the dpll device in the subsystem
>> + * @dpll: pointer to a dpll
>> + * @type: type of a dpll
>> + * @ops: ops for a dpll device
>> + * @priv: pointer to private information of owner
>> + * @owner: pointer to owner device
>> + *
>> + * Make dpll device available for user space.
>> + *
>> + * Return:
>> + * * 0 on success
>> + * * -EINVAL on failure
>> + */
>> +int dpll_device_register(struct dpll_device *dpll, enum dpll_type type,
>> +			 struct dpll_device_ops *ops, void *priv,
>> +			 struct device *owner)
>> +{
>> +	if (WARN_ON(!ops || !owner))
>> +		return -EINVAL;
>> +	if (WARN_ON(type <= DPLL_TYPE_UNSPEC || type > DPLL_TYPE_MAX))
>> +		return -EINVAL;
>> +	mutex_lock(&dpll_device_xa_lock);
>> +	if (ASSERT_DPLL_NOT_REGISTERED(dpll)) {
>> +		mutex_unlock(&dpll_device_xa_lock);
>> +		return -EEXIST;
>> +	}
>> +	dpll->dev.bus = owner->bus;
>> +	dpll->parent = owner;
>> +	dpll->type = type;
>> +	dpll->ops = ops;
>> +	dev_set_name(&dpll->dev, "%s_%d", dev_name(owner),
>> +		     dpll->dev_driver_id);
> 
> This is really odd. As a result, the user would see something like:
> pci/0000:01:00.0_1
> pci/0000:01:00.0_2
> 
> I have to say it is confusing. In devlink, is bus/name and the user
> could use this info to look trough sysfs. Here, 0000:01:00.0_1 is not
> there. Also, "_" might have some meaning on some bus. Should not
> concatename dev_name() with anything.
> 
> Thinking about this some more, the module/clock_id tuple should be
> uniqueue and stable. It is used for dpll_device_get(), it could be used
> as the user handle, can't it?
> Example:
> ice/c92d02a7129f4747
> mlx5/90265d8bf6e6df56
> 
> If you really need the "dev_driver_id" (as I believe clock_id should be
> enough), you can put it here as well:
> ice/c92d02a7129f4747/1
> ice/c92d02a7129f4747/2
>

Looks good, will change it

> This would also be beneficial for mlx5, as mlx5 with 2 PFs would like to
> share instance of DPLL equally, there is no "one clock master". >
>> +	dpll->priv = priv;
>> +	xa_set_mark(&dpll_device_xa, dpll->id, DPLL_REGISTERED);
>> +	mutex_unlock(&dpll_device_xa_lock);
>> +	dpll_notify_device_create(dpll);
>> +
>> +	return 0;
>> +}
>> +EXPORT_SYMBOL_GPL(dpll_device_register);
>> +
>> +/**
>> + * dpll_device_unregister - deregister dpll device
>> + * @dpll: registered dpll pointer
>> + *
>> + * Deregister device, make it unavailable for userspace.
>> + * Note: It does not free the memory
>> + */
>> +void dpll_device_unregister(struct dpll_device *dpll)
>> +{
>> +	mutex_lock(&dpll_device_xa_lock);
>> +	ASSERT_DPLL_REGISTERED(dpll);
>> +	xa_clear_mark(&dpll_device_xa, dpll->id, DPLL_REGISTERED);
>> +	mutex_unlock(&dpll_device_xa_lock);
>> +	dpll_notify_device_delete(dpll);
>> +}
>> +EXPORT_SYMBOL_GPL(dpll_device_unregister);
>> +
>> +/**
>> + * dpll_pin_alloc - allocate the memory for dpll pin
>> + * @clock_id: clock_id of creator
>> + * @dev_driver_id: id given by dev driver
>> + * @module: reference to registering module
>> + * @prop: dpll pin properties
>> + *
>> + * Return:
>> + * * valid allocated dpll_pin struct pointer if succeeded
>> + * * ERR_PTR of an error
> 
> Extra "*"'s

Ok, I can re-format the comments across the files.

>> + */
>> +struct dpll_pin *
>> +dpll_pin_alloc(u64 clock_id, u8 device_drv_id,	struct module *module,
> 
> Odd whitespace.
> 
> Also, func should be static.
> 

Fixed.

> 
>> +	       const struct dpll_pin_properties *prop)
>> +{
>> +	struct dpll_pin *pin;
>> +	int ret;
>> +
>> +	pin = kzalloc(sizeof(*pin), GFP_KERNEL);
>> +	if (!pin)
>> +		return ERR_PTR(-ENOMEM);
>> +	pin->dev_driver_id = device_drv_id;
> 
> Name inconsistency: driver/drv
> you have it on multiple places
> 

Changed it every where, thanks for spotting.

> 
>> +	pin->clock_id = clock_id;
>> +	pin->module = module;
>> +	refcount_set(&pin->refcount, 1);
>> +	if (WARN_ON(!prop->description)) {
>> +		ret = -EINVAL;
>> +		goto release;
>> +	}
>> +	pin->prop.description = kstrdup(prop->description, GFP_KERNEL);
>> +	if (!pin->prop.description) {
>> +		ret = -ENOMEM;
>> +		goto release;
>> +	}
>> +	if (WARN_ON(prop->type <= DPLL_PIN_TYPE_UNSPEC ||
>> +		    prop->type > DPLL_PIN_TYPE_MAX)) {
>> +		ret = -EINVAL;
>> +		goto release;
>> +	}
>> +	pin->prop.type = prop->type;
>> +	pin->prop.capabilities = prop->capabilities;
>> +	pin->prop.freq_supported = prop->freq_supported;
>> +	pin->prop.any_freq_min = prop->any_freq_min;
>> +	pin->prop.any_freq_max = prop->any_freq_max;
> 
> Make sure that the driver maintains prop (static const) and just save
> the pointer. Prop does not need to be something driver needs to change.
> 

What's the difference? For ptp_ocp, we have the same configuration for all
ext pins and the allocator only changes the name of the pin. Properties of
the DPLL pins are stored within the pin object, not the driver, in this case.
Not sure if the pointer way is much better...

> 
>> +	xa_init_flags(&pin->dpll_refs, XA_FLAGS_ALLOC);
>> +	xa_init_flags(&pin->parent_refs, XA_FLAGS_ALLOC);
>> +	ret = xa_alloc(&dpll_pin_xa, &pin->idx, pin,
>> +		       xa_limit_16b, GFP_KERNEL);
>> +release:
>> +	if (ret) {
>> +		xa_destroy(&pin->dpll_refs);
>> +		xa_destroy(&pin->parent_refs);
>> +		kfree(pin->prop.description);
>> +		kfree(pin->rclk_dev_name);
>> +		kfree(pin);
>> +		return ERR_PTR(ret);
>> +	}
>> +
>> +	return pin;
>> +}
>> +
>> +/**
>> + * dpll_pin_get - find existing or create new dpll pin
>> + * @clock_id: clock_id of creator
>> + * @dev_driver_id: id given by dev driver
>> + * @module: reference to registering module
>> + * @prop: dpll pin properties
>> + *
>> + * Get existing object of a pin (unique for given arguments) or create new
>> + * if doesn't exist yet.
>> + *
>> + * Return:
>> + * * valid allocated dpll_pin struct pointer if succeeded
>> + * * ERR_PTR of an error
> 
> This is one example, I'm pretty sure that there are others, when you
> have text inconsistencies in func doc for the same function in .c and .h
> Have it please only on one place. .c is the usual place.
> 

Yep, will clear .h files.

> 
>> + */
>> +struct dpll_pin *
>> +dpll_pin_get(u64 clock_id, u32 device_drv_id, struct module *module,
> 
> Again, why do you need this device_drv_id? Clock id should be enough.
> 
I explained the reason earlier, but the naming is fixed.

> 
>> +	     const struct dpll_pin_properties *prop)
>> +{
>> +	struct dpll_pin *pos, *ret = NULL;
>> +	unsigned long index;
>> +
>> +	mutex_lock(&dpll_pin_xa_lock);
>> +	xa_for_each(&dpll_pin_xa, index, pos) {
>> +		if (pos->clock_id == clock_id &&
>> +		    pos->dev_driver_id == device_drv_id &&
>> +		    pos->module == module) {
> 
> Compare prop as well.
> 
> Can't the driver_id (pin index) be something const as well? I think it
> should. And therefore it could be easily put inside.
> 

I think clock_id + dev_driver_id + module should identify the pin exactly. And 
now I think that *prop is not needed here at all. Arkadiusz, any thoughts?
> 
>> +			ret = pos;
>> +			refcount_inc(&ret->refcount);
>> +			break;
>> +		}
>> +	}
>> +	if (!ret)
>> +		ret = dpll_pin_alloc(clock_id, device_drv_id, module, prop);
>> +	mutex_unlock(&dpll_pin_xa_lock);
>> +
>> +	return ret;
>> +}
>> +EXPORT_SYMBOL_GPL(dpll_pin_get);
>> +
>> +/**
>> + * dpll_pin_put - decrease the refcount and free memory if possible
>> + * @dpll: dpll_device struct pointer
>> + *
>> + * Drop reference for a pin, if all references are gone, delete pin object.
>> + */
>> +void dpll_pin_put(struct dpll_pin *pin)
>> +{
>> +	if (!pin)
>> +		return;
>> +	mutex_lock(&dpll_pin_xa_lock);
>> +	if (refcount_dec_and_test(&pin->refcount)) {
>> +		xa_destroy(&pin->dpll_refs);
>> +		xa_destroy(&pin->parent_refs);
>> +		xa_erase(&dpll_pin_xa, pin->idx);
>> +		kfree(pin->prop.description);
>> +		kfree(pin->rclk_dev_name);
>> +		kfree(pin);
>> +	}
>> +	mutex_unlock(&dpll_pin_xa_lock);
>> +}
>> +EXPORT_SYMBOL_GPL(dpll_pin_put);
>> +
>> +static int
>> +__dpll_pin_register(struct dpll_device *dpll, struct dpll_pin *pin,
>> +		    struct dpll_pin_ops *ops, void *priv,
>> +		    const char *rclk_device_name)
>> +{
>> +	int ret;
>> +
>> +	if (rclk_device_name && !pin->rclk_dev_name) {
>> +		pin->rclk_dev_name = kstrdup(rclk_device_name, GFP_KERNEL);
>> +		if (!pin->rclk_dev_name)
>> +			return -ENOMEM;
>> +	}
>> +	ret = dpll_xa_ref_pin_add(&dpll->pin_refs, pin, ops, priv);
>> +	if (ret)
>> +		goto rclk_free;
>> +	ret = dpll_xa_ref_dpll_add(&pin->dpll_refs, dpll, ops, priv);
>> +	if (ret)
>> +		goto ref_pin_del;
>> +	else
>> +		dpll_pin_notify(dpll, pin, DPLL_A_PIN_IDX);
>> +
>> +	return ret;
>> +
>> +ref_pin_del:
>> +	dpll_xa_ref_pin_del(&dpll->pin_refs, pin);
>> +rclk_free:
>> +	kfree(pin->rclk_dev_name);
>> +	return ret;
>> +}
>> +
>> +/**
>> + * dpll_pin_register - register the dpll pin in the subsystem
>> + * @dpll: pointer to a dpll
>> + * @pin: pointer to a dpll pin
>> + * @ops: ops for a dpll pin ops
>> + * @priv: pointer to private information of owner
>> + * @rclk_device: pointer to recovered clock device
>> + *
>> + * Return:
>> + * * 0 on success
>> + * * -EINVAL - missing dpll or pin
>> + * * -ENOMEM - failed to allocate memory
>> + */
>> +int
>> +dpll_pin_register(struct dpll_device *dpll, struct dpll_pin *pin,
>> +		  struct dpll_pin_ops *ops, void *priv,
>> +		  struct device *rclk_device)
> 
> Wait a second, what is this "struct device *"? Looks very odd.
> 
> 
>> +{
>> +	const char *rclk_name = rclk_device ? dev_name(rclk_device) : NULL;
> 
> If you need to store something here, store the pointer to the device
> directly. But this rclk_device seems odd to me.
> Dev_name is in case of PCI device for example 0000:01:00.0? That alone
> is incomplete. What should it server for?
> 

Well, these questions go to Arkadiusz...

> 
> 
>> +	int ret;
>> +
>> +	if (WARN_ON(!dpll))
>> +		return -EINVAL;
>> +	if (WARN_ON(!pin))
>> +		return -EINVAL;
> 
> Remove these checks and other similar checks in the code. It should rely
> on basic driver sanity.
> 

Ok...

> 
>> +
>> +	mutex_lock(&dpll_device_xa_lock);
>> +	mutex_lock(&dpll_pin_xa_lock);
>> +	ret = __dpll_pin_register(dpll, pin, ops, priv, rclk_name);
>> +	mutex_unlock(&dpll_pin_xa_lock);
>> +	mutex_unlock(&dpll_device_xa_lock);
>> +
>> +	return ret;
>> +}
>> +EXPORT_SYMBOL_GPL(dpll_pin_register);
>> +
>> +static void
>> +__dpll_pin_unregister(struct dpll_device *dpll, struct dpll_pin *pin)
>> +{
>> +	dpll_xa_ref_pin_del(&dpll->pin_refs, pin);
>> +	dpll_xa_ref_dpll_del(&pin->dpll_refs, dpll);
>> +}
>> +
>> +/**
>> + * dpll_pin_unregister - deregister dpll pin from dpll device
>> + * @dpll: registered dpll pointer
>> + * @pin: pointer to a pin
>> + *
>> + * Note: It does not free the memory
>> + */
>> +int dpll_pin_unregister(struct dpll_device *dpll, struct dpll_pin *pin)
>> +{
>> +	if (WARN_ON(xa_empty(&dpll->pin_refs)))
>> +		return -ENOENT;
>> +
>> +	mutex_lock(&dpll_device_xa_lock);
>> +	mutex_lock(&dpll_pin_xa_lock);
>> +	__dpll_pin_unregister(dpll, pin);
>> +	mutex_unlock(&dpll_pin_xa_lock);
>> +	mutex_unlock(&dpll_device_xa_lock);
>> +
>> +	return 0;
>> +}
>> +EXPORT_SYMBOL_GPL(dpll_pin_unregister);
>> +
>> +/**
>> + * dpll_pin_on_pin_register - register a pin with a parent pin
>> + * @parent: pointer to a parent pin
>> + * @pin: pointer to a pin
>> + * @ops: ops for a dpll pin
>> + * @priv: pointer to private information of owner
>> + * @rclk_device: pointer to recovered clock device
>> + *
>> + * Register a pin with a parent pin, create references between them and
>> + * between newly registered pin and dplls connected with a parent pin.
>> + *
>> + * Return:
>> + * * 0 on success
>> + * * -EINVAL missing pin or parent
>> + * * -ENOMEM failed allocation
>> + * * -EPERM if parent is not allowed
>> + */
>> +int
>> +dpll_pin_on_pin_register(struct dpll_pin *parent, struct dpll_pin *pin,
>> +			 struct dpll_pin_ops *ops, void *priv,
>> +			 struct device *rclk_device)
>> +{
>> +	struct dpll_pin_ref *ref;
>> +	unsigned long i, stop;
>> +	int ret;
>> +
>> +	if (WARN_ON(!pin || !parent))
>> +		return -EINVAL;
>> +	if (WARN_ON(parent->prop.type != DPLL_PIN_TYPE_MUX))
>> +		return -EPERM;
> 
> I don't think that EPERM is suitable for this. Just use EINVAL. The
> driver is buggy in this case anyway.
> 
   Ok.

> 
>> +	mutex_lock(&dpll_pin_xa_lock);
>> +	ret = dpll_xa_ref_pin_add(&pin->parent_refs, parent, ops, priv);
>> +	if (ret)
>> +		goto unlock;
>> +	refcount_inc(&pin->refcount);
>> +	xa_for_each(&parent->dpll_refs, i, ref) {
>> +		mutex_lock(&dpll_device_xa_lock);
>> +		ret = __dpll_pin_register(ref->dpll, pin, ops, priv,
>> +					  rclk_device ?
>> +					  dev_name(rclk_device) : NULL);
>> +		mutex_unlock(&dpll_device_xa_lock);
>> +		if (ret) {
>> +			stop = i;
>> +			goto dpll_unregister;
>> +		}
>> +		dpll_pin_parent_notify(ref->dpll, pin, parent, DPLL_A_PIN_IDX);
>> +	}
>> +	mutex_unlock(&dpll_pin_xa_lock);
>> +
>> +	return ret;
>> +
>> +dpll_unregister:
>> +	xa_for_each(&parent->dpll_refs, i, ref) {
>> +		if (i < stop) {
>> +			mutex_lock(&dpll_device_xa_lock);
>> +			__dpll_pin_unregister(ref->dpll, pin);
>> +			mutex_unlock(&dpll_device_xa_lock);
>> +		}
>> +	}
>> +	refcount_dec(&pin->refcount);
>> +	dpll_xa_ref_pin_del(&pin->parent_refs, parent);
>> +unlock:
>> +	mutex_unlock(&dpll_pin_xa_lock);
>> +	return ret;
>> +}
>> +EXPORT_SYMBOL_GPL(dpll_pin_on_pin_register);

Now I realised that this function can bring us to ABBA deadlock with mutexes..


>> +
>> +/**
>> + * dpll_pin_on_pin_unregister - deregister dpll pin from a parent pin
>> + * @parent: pointer to a parent pin
>> + * @pin: pointer to a pin
>> + *
>> + * Note: It does not free the memory
>> + */
>> +void dpll_pin_on_pin_unregister(struct dpll_pin *parent, struct dpll_pin *pin)
>> +{
>> +	struct dpll_pin_ref *ref;
>> +	unsigned long i;
>> +
>> +	mutex_lock(&dpll_device_xa_lock);
>> +	mutex_lock(&dpll_pin_xa_lock);
>> +	dpll_xa_ref_pin_del(&pin->parent_refs, parent);
>> +	refcount_dec(&pin->refcount);
>> +	xa_for_each(&pin->dpll_refs, i, ref) {
>> +		__dpll_pin_unregister(ref->dpll, pin);
>> +		dpll_pin_parent_notify(ref->dpll, pin, parent,
>> +				       DPLL_A_PIN_IDX);
>> +	}
>> +	mutex_unlock(&dpll_pin_xa_lock);
>> +	mutex_unlock(&dpll_device_xa_lock);
>> +}
>> +EXPORT_SYMBOL_GPL(dpll_pin_on_pin_unregister);
>> +
>> +/**
>> + * dpll_pin_get_by_idx - find a pin ref on dpll by pin index
>> + * @dpll: dpll device pointer
>> + * @idx: index of pin
>> + *
>> + * Find a reference to a pin registered with given dpll and return its pointer.
>> + *
>> + * Return:
>> + * * valid pointer if pin was found
>> + * * NULL if not found
>> + */
>> +struct dpll_pin *dpll_pin_get_by_idx(struct dpll_device *dpll, u32 idx)
>> +{
>> +	struct dpll_pin_ref *pos;
>> +	unsigned long i;
>> +
>> +	xa_for_each(&dpll->pin_refs, i, pos) {
>> +		if (pos && pos->pin && pos->pin->dev_driver_id == idx)
> 
> How exactly pos->pin could be NULL?
> 
> Also, you are degrading the xarray to a mere list here with lookup like
> this. Why can't you use the pin index coming from driver and
> insert/lookup based on this index?
> 
Good point. We just have to be sure, that drivers provide 0-based indexes for 
their pins. I'll re-think it.


> 
>> +			return pos->pin;
>> +	}
>> +
>> +	return NULL;
>> +}
>> +
>> +/**
>> + * dpll_priv - get the dpll device private owner data
>> + * @dpll:	registered dpll pointer
>> + *
>> + * Return: pointer to the data
>> + */
>> +void *dpll_priv(const struct dpll_device *dpll)
>> +{
>> +	return dpll->priv;
>> +}
>> +EXPORT_SYMBOL_GPL(dpll_priv);
>> +
>> +/**
>> + * dpll_pin_on_dpll_priv - get the dpll device private owner data
>> + * @dpll:	registered dpll pointer
>> + * @pin:	pointer to a pin
>> + *
>> + * Return: pointer to the data
>> + */
>> +void *dpll_pin_on_dpll_priv(const struct dpll_device *dpll,
> 
> IIUC, you use this helper from dpll ops in drivers to get per dpll priv.
> Just pass the priv directly to the op and avoid need for this helper,
> no? Same goes to the rest of the priv helpers.
> 
> 
>> +			    const struct dpll_pin *pin)
>> +{
>> +	struct dpll_pin_ref *ref;
>> +
>> +	ref = dpll_xa_ref_pin_find((struct xarray *)&dpll->pin_refs, pin);
> 
> Why cast is needed here? You have this on multiple places.
> 
> 
>> +	if (!ref)
>> +		return NULL;
>> +
>> +	return ref->priv;
>> +}
>> +EXPORT_SYMBOL_GPL(dpll_pin_on_dpll_priv);
>> +
>> +/**
>> + * dpll_pin_on_pin_priv - get the dpll pin private owner data
>> + * @parent: pointer to a parent pin
>> + * @pin: pointer to a pin
>> + *
>> + * Return: pointer to the data
>> + */
>> +void *dpll_pin_on_pin_priv(const struct dpll_pin *parent,
>> +			   const struct dpll_pin *pin)
>> +{
>> +	struct dpll_pin_ref *ref;
>> +
>> +	ref = dpll_xa_ref_pin_find((struct xarray *)&pin->parent_refs, parent);
>> +	if (!ref)
>> +		return NULL;
>> +
>> +	return ref->priv;
>> +}
>> +EXPORT_SYMBOL_GPL(dpll_pin_on_pin_priv);
>> +
>> +static int __init dpll_init(void)
>> +{
>> +	int ret;
>> +
>> +	ret = dpll_netlink_init();
>> +	if (ret)
>> +		goto error;
>> +
>> +	ret = class_register(&dpll_class);
> 
> Why exactly do you need this? I asked to remove this previously, IIRC
> you said you would check if this needed. Why?
> 
Ah, sorry. Removed it now.


> 
>> +	if (ret)
>> +		goto unregister_netlink;
>> +
>> +	return 0;
>> +
>> +unregister_netlink:
>> +	dpll_netlink_finish();
>> +error:
>> +	mutex_destroy(&dpll_device_xa_lock);
>> +	mutex_destroy(&dpll_pin_xa_lock);
>> +	return ret;
>> +}
>> +subsys_initcall(dpll_init);
>> diff --git a/drivers/dpll/dpll_core.h b/drivers/dpll/dpll_core.h
>> new file mode 100644
>> index 000000000000..876b6ac6f3a0
>> --- /dev/null
>> +++ b/drivers/dpll/dpll_core.h
>> @@ -0,0 +1,99 @@
>> +/* SPDX-License-Identifier: GPL-2.0 */
>> +/*
>> + *  Copyright (c) 2021 Meta Platforms, Inc. and affiliates
>> + */
>> +
>> +#ifndef __DPLL_CORE_H__
>> +#define __DPLL_CORE_H__
>> +
>> +#include <linux/dpll.h>
>> +#include <linux/refcount.h>
>> +#include "dpll_netlink.h"
>> +
>> +#define DPLL_REGISTERED		XA_MARK_1
>> +
>> +/**
>> + * struct dpll_device - structure for a DPLL device
>> + * @id:			unique id number for each device
>> + * @dev_driver_id:	id given by dev driver
>> + * @dev:		struct device for this dpll device
>> + * @parent:		parent device
>> + * @module:		module of creator
>> + * @ops:		operations this &dpll_device supports
>> + * @lock:		mutex to serialize operations
>> + * @type:		type of a dpll
>> + * @priv:		pointer to private information of owner
>> + * @pins:		list of pointers to pins registered with this dpll
>> + * @clock_id:		unique identifier (clock_id) of a dpll
>> + * @mode_supported_mask: mask of supported modes
>> + * @refcount:		refcount
>> + **/
>> +struct dpll_device {
>> +	u32 id;
>> +	u32 dev_driver_id;
>> +	struct device dev;
>> +	struct device *parent;
>> +	struct module *module;
>> +	struct dpll_device_ops *ops;
>> +	enum dpll_type type;
>> +	void *priv;
>> +	struct xarray pin_refs;
>> +	u64 clock_id;
>> +	unsigned long mode_supported_mask;
>> +	refcount_t refcount;
>> +};
>> +
>> +/**
>> + * struct dpll_pin - structure for a dpll pin
>> + * @idx:		unique idx given by alloc on global pin's XA
>> + * @dev_driver_id:	id given by dev driver
>> + * @clock_id:		clock_id of creator
>> + * @module:		module of creator
>> + * @dpll_refs:		hold referencees to dplls that pin is registered with
>> + * @pin_refs:		hold references to pins that pin is registered with
>> + * @prop:		properties given by registerer
>> + * @rclk_dev_name:	holds name of device when pin can recover clock from it
>> + * @refcount:		refcount
>> + **/
>> +struct dpll_pin {
>> +	u32 idx;
>> +	u32 dev_driver_id;
>> +	u64 clock_id;
>> +	struct module *module;
> 
> Have the ordering of common fields, like clock_id and module consistent
> with struct dpll_device
> 

Re-arranged it a bit.

> 
>> +	struct xarray dpll_refs;
>> +	struct xarray parent_refs;
>> +	struct dpll_pin_properties prop;
>> +	char *rclk_dev_name;
>> +	refcount_t refcount;
>> +};
>> +
>> +/**
>> + * struct dpll_pin_ref - structure for referencing either dpll or pins
>> + * @dpll:		pointer to a dpll
>> + * @pin:		pointer to a pin
>> + * @ops:		ops for a dpll pin
>> + * @priv:		pointer to private information of owner
>> + **/
>> +struct dpll_pin_ref {
>> +	union {
>> +		struct dpll_device *dpll;
>> +		struct dpll_pin *pin;
>> +	};
>> +	struct dpll_pin_ops *ops;
>> +	void *priv;
>> +	refcount_t refcount;
>> +};
>> +
>> +struct dpll_device *dpll_device_get_by_id(int id);
>> +struct dpll_device *dpll_device_get_by_name(const char *bus_name,
>> +					    const char *dev_name);
>> +struct dpll_pin *dpll_pin_get_by_idx(struct dpll_device *dpll, u32 idx);
>> +struct dpll_pin_ref *
>> +dpll_xa_ref_pin_find(struct xarray *xa_refs, const struct dpll_pin *pin);
>> +struct dpll_pin_ref *
>> +dpll_xa_ref_dpll_find(struct xarray *xa_refs, const struct dpll_device *dpll);
>> +extern struct xarray dpll_device_xa;
>> +extern struct xarray dpll_pin_xa;
>> +extern struct mutex dpll_device_xa_lock;
>> +extern struct mutex dpll_pin_xa_lock;
>> +#endif
>> diff --git a/drivers/dpll/dpll_netlink.c b/drivers/dpll/dpll_netlink.c
>> new file mode 100644
>> index 000000000000..46aefeb1ac93
>> --- /dev/null
>> +++ b/drivers/dpll/dpll_netlink.c
>> @@ -0,0 +1,1065 @@
>> +// SPDX-License-Identifier: GPL-2.0
>> +/*
>> + * Generic netlink for DPLL management framework
>> + *
>> + * Copyright (c) 2021 Meta Platforms, Inc. and affiliates
>> + *
>> + */
>> +#include <linux/module.h>
>> +#include <linux/kernel.h>
>> +#include <net/genetlink.h>
>> +#include "dpll_core.h"
>> +#include "dpll_nl.h"
>> +#include <uapi/linux/dpll.h>
>> +
>> +static u32 dpll_pin_freq_value[] = {
>> +	[DPLL_PIN_FREQ_SUPP_1_HZ] = DPLL_PIN_FREQ_1_HZ,
>> +	[DPLL_PIN_FREQ_SUPP_10_MHZ] = DPLL_PIN_FREQ_10_MHZ,
>> +};
>> +
>> +static int
>> +dpll_msg_add_dev_handle(struct sk_buff *msg, const struct dpll_device *dpll)
>> +{
>> +	if (nla_put_u32(msg, DPLL_A_ID, dpll->id))
> 
> Why exactly do we need this dua--handle scheme? Why do you need
> unpredictable DPLL_A_ID to be exposed to userspace?
> It's just confusing.
> 
To be able to work with DPLL per integer after iterator on the list deducts
which DPLL device is needed. It can reduce the amount of memory copies and
simplify comparisons. Not sure why it's confusing.

> 
>> +		return -EMSGSIZE;
>> +	if (nla_put_string(msg, DPLL_A_BUS_NAME, dev_bus_name(&dpll->dev)))
>> +		return -EMSGSIZE;
>> +	if (nla_put_string(msg, DPLL_A_DEV_NAME, dev_name(&dpll->dev)))
>> +		return -EMSGSIZE;
>> +
>> +	return 0;
>> +}

[...]

>> +
>> +static int
>> +dpll_msg_add_pins_on_pin(struct sk_buff *msg, struct dpll_pin *pin,
>> +			 struct netlink_ext_ack *extack)
>> +{
>> +	struct dpll_pin_ref *ref = NULL;
> 
> Why this needs to be initialized?
> 
No need, fixed.


> 
>> +	enum dpll_pin_state state;
>> +	struct nlattr *nest;
>> +	unsigned long index;
>> +	int ret;
>> +
>> +	xa_for_each(&pin->parent_refs, index, ref) {
>> +		if (WARN_ON(!ref->ops->state_on_pin_get))
>> +			return -EFAULT;
>> +		ret = ref->ops->state_on_pin_get(pin, ref->pin, &state,
>> +						 extack);
>> +		if (ret)
>> +			return -EFAULT;
>> +		nest = nla_nest_start(msg, DPLL_A_PIN_PARENT);
>> +		if (!nest)
>> +			return -EMSGSIZE;
>> +		if (nla_put_u32(msg, DPLL_A_PIN_PARENT_IDX,
>> +				ref->pin->dev_driver_id)) {
>> +			ret = -EMSGSIZE;
>> +			goto nest_cancel;
>> +		}
>> +		if (nla_put_u8(msg, DPLL_A_PIN_STATE, state)) {
>> +			ret = -EMSGSIZE;
>> +			goto nest_cancel;
>> +		}
>> +		nla_nest_end(msg, nest);
>> +	}
> 
> How is this function different to dpll_msg_add_pin_parents()?
> Am I lost? To be honest, this x_on_pin/dpll, parent, refs dance is quite
> hard to follow for me :/
> 
> Did you get lost here as well? If yes, this needs some serious think
> through :)
> 

Let's re-think it again. Arkadiuzs, do you have clear explanation of the
relationship between these things?

> 
>> +
>> +	return 0;
>> +
>> +nest_cancel:
>> +	nla_nest_cancel(msg, nest);
>> +	return ret;
>> +}
>> +
>> +static int
>> +dpll_msg_add_pin_dplls(struct sk_buff *msg, struct dpll_pin *pin,
>> +		       struct netlink_ext_ack *extack)
>> +{
>> +	struct dpll_pin_ref *ref;
>> +	struct nlattr *attr;
>> +	unsigned long index;
>> +	int ret;
>> +
>> +	xa_for_each(&pin->dpll_refs, index, ref) {
>> +		attr = nla_nest_start(msg, DPLL_A_DEVICE);
>> +		if (!attr)
>> +			return -EMSGSIZE;
>> +		ret = dpll_msg_add_dev_handle(msg, ref->dpll);
>> +		if (ret)
>> +			goto nest_cancel;
>> +		ret = dpll_msg_add_pin_on_dpll_state(msg, pin, ref, extack);
>> +		if (ret && ret != -EOPNOTSUPP)
>> +			goto nest_cancel;
>> +		ret = dpll_msg_add_pin_prio(msg, pin, ref, extack);
>> +		if (ret && ret != -EOPNOTSUPP)
>> +			goto nest_cancel;
>> +		nla_nest_end(msg, attr);
>> +	}
>> +
>> +	return 0;
>> +
>> +nest_cancel:
>> +	nla_nest_end(msg, attr);
>> +	return ret;
>> +}
>> +
>> +static int
>> +dpll_cmd_pin_on_dpll_get(struct sk_buff *msg, struct dpll_pin *pin,
>> +			 struct dpll_device *dpll,
>> +			 struct netlink_ext_ack *extack)
>> +{
>> +	struct dpll_pin_ref *ref;
>> +	int ret;
>> +
>> +	if (nla_put_u32(msg, DPLL_A_PIN_IDX, pin->dev_driver_id))
> 
> Is it "ID" or "INDEX" (IDX). Please make this consistent in the whole
> code.
> 

I believe it's INDEX which is provided by the driver. I'll think about renaming,
but suggestions are welcome.

>> +		return -EMSGSIZE;
>> +	if (nla_put_string(msg, DPLL_A_PIN_DESCRIPTION, pin->prop.description))
>> +		return -EMSGSIZE;
>> +	if (nla_put_u8(msg, DPLL_A_PIN_TYPE, pin->prop.type))
>> +		return -EMSGSIZE;
>> +	if (nla_put_u32(msg, DPLL_A_PIN_DPLL_CAPS, pin->prop.capabilities))
>> +		return -EMSGSIZE;
>> +	ret = dpll_msg_add_pin_direction(msg, pin, extack);
>> +	if (ret)
>> +		return ret;
>> +	ret = dpll_msg_add_pin_freq(msg, pin, extack, true);
>> +	if (ret && ret != -EOPNOTSUPP)
>> +		return ret;
>> +	ref = dpll_xa_ref_dpll_find(&pin->dpll_refs, dpll);
>> +	if (!ref)
>> +		return -EFAULT;
>> +	ret = dpll_msg_add_pin_prio(msg, pin, ref, extack);
>> +	if (ret && ret != -EOPNOTSUPP)
>> +		return ret;
>> +	ret = dpll_msg_add_pin_on_dpll_state(msg, pin, ref, extack);
>> +	if (ret && ret != -EOPNOTSUPP)
>> +		return ret;
>> +	ret = dpll_msg_add_pin_parents(msg, pin, extack);
>> +	if (ret)
>> +		return ret;
>> +	if (pin->rclk_dev_name)
>> +		if (nla_put_string(msg, DPLL_A_PIN_RCLK_DEVICE,
>> +				   pin->rclk_dev_name))
>> +			return -EMSGSIZE;
>> +
>> +	return 0;
>> +}
>> +
>> +static int
>> +__dpll_cmd_pin_dump_one(struct sk_buff *msg, struct dpll_pin *pin,
>> +			struct netlink_ext_ack *extack, bool dump_dpll)
>> +{
>> +	int ret;
>> +
>> +	if (nla_put_u32(msg, DPLL_A_PIN_IDX, pin->dev_driver_id))
>> +		return -EMSGSIZE;
>> +	if (nla_put_string(msg, DPLL_A_PIN_DESCRIPTION, pin->prop.description))
>> +		return -EMSGSIZE;
>> +	if (nla_put_u8(msg, DPLL_A_PIN_TYPE, pin->prop.type))
>> +		return -EMSGSIZE;
>> +	ret = dpll_msg_add_pin_direction(msg, pin, extack);
>> +	if (ret)
>> +		return ret;
>> +	ret = dpll_msg_add_pin_freq(msg, pin, extack, true);
>> +	if (ret && ret != -EOPNOTSUPP)
>> +		return ret;
>> +	ret = dpll_msg_add_pins_on_pin(msg, pin, extack);
>> +	if (ret)
>> +		return ret;
>> +	if (!xa_empty(&pin->dpll_refs) && dump_dpll) {
>> +		ret = dpll_msg_add_pin_dplls(msg, pin, extack);
>> +		if (ret)
>> +			return ret;
>> +	}
>> +	if (pin->rclk_dev_name)
>> +		if (nla_put_string(msg, DPLL_A_PIN_RCLK_DEVICE,
>> +				   pin->rclk_dev_name))
>> +			return -EMSGSIZE;
> 
> Lots of code duplication with the previous functions, please unify.
> 

Agree, have done it.

> 
>> +
>> +	return 0;
>> +}
>> +

[...]

>> +static int
>> +dpll_pin_set_from_nlattr(struct dpll_device *dpll,
>> +			 struct dpll_pin *pin, struct genl_info *info)
>> +{
>> +	enum dpll_pin_state state = DPLL_PIN_STATE_UNSPEC;
>> +	u32 parent_idx = PIN_IDX_INVALID;
> 
> You just need this PIN_IDX_INVALID define internally in this function,
> change the flow to avoid a need for it.
> 

I'll re-think it, thanks.

> 
>> +	int rem, ret = -EINVAL;
>> +	struct nlattr *a;
>> +
>> +	nla_for_each_attr(a, genlmsg_data(info->genlhdr),
>> +			  genlmsg_len(info->genlhdr), rem) {
> 
> This is odd. Why you iterace over attrs? Why don't you just access them
> directly, like attrs[DPLL_A_PIN_FREQUENCY] for example?
> 

I had some unknown crashes when I was using such access. I might have lost some
checks, will try it again.

> 
>> +		switch (nla_type(a)) {
>> +		case DPLL_A_PIN_FREQUENCY:
>> +			ret = dpll_pin_freq_set(pin, a, info->extack);
>> +			if (ret)
>> +				return ret;
>> +			break;
>> +		case DPLL_A_PIN_DIRECTION:
>> +			ret = dpll_pin_direction_set(pin, a, info->extack);
>> +			if (ret)
>> +				return ret;
>> +			break;
>> +		case DPLL_A_PIN_PRIO:
>> +			ret = dpll_pin_prio_set(dpll, pin, a, info->extack);
>> +			if (ret)
>> +				return ret;
>> +			break;
>> +		case DPLL_A_PIN_PARENT_IDX:
>> +			parent_idx = nla_get_u32(a);
>> +			break;
>> +		case DPLL_A_PIN_STATE:
>> +			state = nla_get_u8(a);
>> +			break;
>> +		default:
>> +			break;
>> +		}
>> +	}
>> +	if (state != DPLL_PIN_STATE_UNSPEC) {
> 
> Again, change the flow to:
> 	if (attrs[DPLL_A_PIN_STATE]) {
> 
> and avoid need for this value set/check.
> 

Yep, will try.

> 
>> +		if (parent_idx == PIN_IDX_INVALID) {
>> +			ret = dpll_pin_state_set(dpll, pin, state,
>> +						 info->extack);
>> +			if (ret)
>> +				return ret;
>> +		} else {
>> +			ret = dpll_pin_on_pin_state_set(dpll, pin, parent_idx,
>> +							state, info->extack);
>> +			if (ret)
>> +				return ret;
>> +		}
>> +	}
>> +
>> +	return ret;
>> +}
>> +

[...]

>> +int dpll_pre_dumpit(struct netlink_callback *cb)
>> +{
>> +	mutex_lock(&dpll_device_xa_lock);
>> +
>> +	return 0;
>> +}
>> +
>> +int dpll_post_dumpit(struct netlink_callback *cb)
>> +{
>> +	mutex_unlock(&dpll_device_xa_lock);
>> +
>> +	return 0;
>> +}
>> +
>> +int dpll_pin_pre_doit(const struct genl_split_ops *ops, struct sk_buff *skb,
>> +		      struct genl_info *info)
>> +{
>> +	int ret = dpll_pre_doit(ops, skb, info);
>> +	struct dpll_device *dpll;
>> +	struct dpll_pin *pin;
>> +
>> +	if (ret)
>> +		return ret;
>> +	dpll = info->user_ptr[0];
>> +	if (!info->attrs[DPLL_A_PIN_IDX]) {
>> +		ret = -EINVAL;
>> +		goto unlock_dev;
>> +	}
>> +	mutex_lock(&dpll_pin_xa_lock);
>> +	pin = dpll_pin_get_by_idx(dpll,
>> +				  nla_get_u32(info->attrs[DPLL_A_PIN_IDX]));
>> +	if (!pin) {
>> +		ret = -ENODEV;
>> +		goto unlock_pin;
>> +	}
>> +	info->user_ptr[1] = pin;
>> +
>> +	return 0;
>> +
>> +unlock_pin:
>> +	mutex_unlock(&dpll_pin_xa_lock);
>> +unlock_dev:
>> +	mutex_unlock(&dpll_device_xa_lock);
>> +	return ret;
>> +}
>> +
>> +void dpll_pin_post_doit(const struct genl_split_ops *ops, struct sk_buff *skb,
>> +			struct genl_info *info)
>> +{
>> +	mutex_unlock(&dpll_pin_xa_lock);
>> +	dpll_post_doit(ops, skb, info);
>> +}
>> +
>> +int dpll_pin_pre_dumpit(struct netlink_callback *cb)
>> +{
>> +	mutex_lock(&dpll_pin_xa_lock);
> 
> ABBA deadlock here, see dpll_pin_register() for example where the lock
> taking order is opposite.
> 

Now I see an ABBA deadlock here, as well as in function before. Not sure how to
solve it here. Any thoughts?

> 
>> +
>> +	return dpll_pre_dumpit(cb);
>> +}
>> +
>> +int dpll_pin_post_dumpit(struct netlink_callback *cb)
>> +{
>> +	mutex_unlock(&dpll_pin_xa_lock);
>> +
>> +	return dpll_post_dumpit(cb);
>> +}
>> +
>> +static int
>> +dpll_event_device_change(struct sk_buff *msg, struct dpll_device *dpll,
>> +			 struct dpll_pin *pin, struct dpll_pin *parent,
>> +			 enum dplla attr)
>> +{
>> +	int ret = dpll_msg_add_dev_handle(msg, dpll);
>> +	struct dpll_pin_ref *ref = NULL;
>> +	enum dpll_pin_state state;
>> +
>> +	if (ret)
>> +		return ret;
>> +	if (pin && nla_put_u32(msg, DPLL_A_PIN_IDX, pin->dev_driver_id))
>> +		return -EMSGSIZE;
> 
> I don't really understand why you are trying figure something new and
> interesting with the change notifications. This object mix and random
> attrs fillup is something very wrong and makes userspace completely
> fuzzy about what it is getting. And yet it is so simple:
> You have 2 objects, dpll and pin, please just have:
> dpll_notify()
> dpll_pin_notify()
> and share the attrs fillup code with pin_get() and dpll_get() callbacks.
> No need for any smartness. Have this dumb and simple.
> 
> Think about it more as about "object-state-snapshot" than "atomic-change" 

But with full object-snapshot user space app will lose the information about
what exactly has changed. The reason to have this event is to provide the 
attributes which have changed. Otherwise, the app should have full snapshot and
compare all attributes to figure out changes and that's might not be great idea.

> 
>> +
>> +	switch (attr) {
>> +	case DPLL_A_MODE:
>> +		ret = dpll_msg_add_mode(msg, dpll, NULL);
>> +		break;
>> +	case DPLL_A_SOURCE_PIN_IDX:
>> +		ret = dpll_msg_add_source_pin_idx(msg, dpll, NULL);
>> +		break;
>> +	case DPLL_A_LOCK_STATUS:
>> +		ret = dpll_msg_add_lock_status(msg, dpll, NULL);
>> +		break;
>> +	case DPLL_A_TEMP:
>> +		ret = dpll_msg_add_temp(msg, dpll, NULL);
>> +		break;
>> +	case DPLL_A_PIN_FREQUENCY:
>> +		ret = dpll_msg_add_pin_freq(msg, pin, NULL, false);
>> +		break;
>> +	case DPLL_A_PIN_PRIO:
>> +		ref = dpll_xa_ref_dpll_find(&pin->dpll_refs, dpll);
>> +		if (!ref)
>> +			return -EFAULT;
>> +		ret = dpll_msg_add_pin_prio(msg, pin, ref, NULL);
>> +		break;
>> +	case DPLL_A_PIN_STATE:
>> +		if (parent) {
>> +			ref = dpll_xa_ref_pin_find(&pin->parent_refs, parent);
>> +			if (!ref)
>> +				return -EFAULT;
>> +			if (!ref->ops || !ref->ops->state_on_pin_get)
>> +				return -EOPNOTSUPP;
>> +			ret = ref->ops->state_on_pin_get(pin, parent, &state,
>> +							 NULL);
>> +			if (ret)
>> +				return ret;
>> +			if (nla_put_u32(msg, DPLL_A_PIN_PARENT_IDX,
>> +					parent->dev_driver_id))
>> +				return -EMSGSIZE;
>> +		} else {
>> +			ref = dpll_xa_ref_dpll_find(&pin->dpll_refs, dpll);
>> +			if (!ref)
>> +				return -EFAULT;
>> +			ret = dpll_msg_add_pin_on_dpll_state(msg, pin, ref,
>> +							     NULL);
>> +			if (ret)
>> +				return ret;
>> +		}
>> +		break;
>> +	default:
>> +		break;
>> +	}
>> +
>> +	return ret;
>> +}
>> +
>> +static int
>> +dpll_send_event_create(enum dpll_event event, struct dpll_device *dpll)
>> +{
>> +	struct sk_buff *msg;
>> +	int ret = -EMSGSIZE;
>> +	void *hdr;
>> +
>> +	msg = genlmsg_new(NLMSG_GOODSIZE, GFP_KERNEL);
>> +	if (!msg)
>> +		return -ENOMEM;
>> +
>> +	hdr = genlmsg_put(msg, 0, 0, &dpll_nl_family, 0, event);
>> +	if (!hdr)
>> +		goto out_free_msg;
>> +
>> +	ret = dpll_msg_add_dev_handle(msg, dpll);
>> +	if (ret)
>> +		goto out_cancel_msg;
>> +	genlmsg_end(msg, hdr);
>> +	genlmsg_multicast(&dpll_nl_family, msg, 0, 0, GFP_KERNEL);
>> +
>> +	return 0;
>> +
>> +out_cancel_msg:
>> +	genlmsg_cancel(msg, hdr);
>> +out_free_msg:
>> +	nlmsg_free(msg);
>> +
>> +	return ret;
>> +}
>> +
>> +static int
>> +dpll_send_event_change(struct dpll_device *dpll, struct dpll_pin *pin,
>> +		       struct dpll_pin *parent, enum dplla attr)
>> +{
>> +	struct sk_buff *msg;
>> +	int ret = -EMSGSIZE;
>> +	void *hdr;
>> +
>> +	msg = genlmsg_new(NLMSG_GOODSIZE, GFP_KERNEL);
>> +	if (!msg)
>> +		return -ENOMEM;
>> +
>> +	hdr = genlmsg_put(msg, 0, 0, &dpll_nl_family, 0,
>> +			  DPLL_EVENT_DEVICE_CHANGE);
> 
> I don't really get it. Why exactly you keep having this *EVENT* cmds?
> Why per-object NEW/GET/DEL cmds shared with get genl op are not enough?
> I have to be missing something.

Changes might come from other places, but will affect the DPLL device and we
have to notify users in this case.
> 
> 
>> +	if (!hdr)
>> +		goto out_free_msg;
>> +
>> +	ret = dpll_event_device_change(msg, dpll, pin, parent, attr);
>> +	if (ret)
>> +		goto out_cancel_msg;
>> +	genlmsg_end(msg, hdr);
>> +	genlmsg_multicast(&dpll_nl_family, msg, 0, 0, GFP_KERNEL);
>> +
>> +	return 0;
>> +
>> +out_cancel_msg:
>> +	genlmsg_cancel(msg, hdr);
>> +out_free_msg:
>> +	nlmsg_free(msg);
>> +
>> +	return ret;
>> +}
>> +
>> +int dpll_notify_device_create(struct dpll_device *dpll)
>> +{
>> +	return dpll_send_event_create(DPLL_EVENT_DEVICE_CREATE, dpll);
>> +}
>> +
>> +int dpll_notify_device_delete(struct dpll_device *dpll)
>> +{
>> +	return dpll_send_event_create(DPLL_EVENT_DEVICE_DELETE, dpll);
>> +}
>> +
>> +int dpll_device_notify(struct dpll_device *dpll, enum dplla attr)
>> +{
>> +	if (WARN_ON(!dpll))
>> +		return -EINVAL;
>> +
>> +	return dpll_send_event_change(dpll, NULL, NULL, attr);
>> +}
>> +EXPORT_SYMBOL_GPL(dpll_device_notify);
>> +
>> +int dpll_pin_notify(struct dpll_device *dpll, struct dpll_pin *pin,
>> +		    enum dplla attr)
>> +{
>> +	return dpll_send_event_change(dpll, pin, NULL, attr);
>> +}
>> +
>> +int dpll_pin_parent_notify(struct dpll_device *dpll, struct dpll_pin *pin,
>> +			   struct dpll_pin *parent, enum dplla attr)
>> +{
>> +	return dpll_send_event_change(dpll, pin, parent, attr);
>> +}
>> +
>> +int __init dpll_netlink_init(void)
>> +{
>> +	return genl_register_family(&dpll_nl_family);
>> +}
>> +
>> +void dpll_netlink_finish(void)
>> +{
>> +	genl_unregister_family(&dpll_nl_family);
>> +}
>> +
>> +void __exit dpll_netlink_fini(void)
>> +{
>> +	dpll_netlink_finish();
>> +}
>> diff --git a/drivers/dpll/dpll_netlink.h b/drivers/dpll/dpll_netlink.h
>> new file mode 100644
>> index 000000000000..072efa10f0e6
>> --- /dev/null
>> +++ b/drivers/dpll/dpll_netlink.h
>> @@ -0,0 +1,30 @@
>> +/* SPDX-License-Identifier: GPL-2.0 */
>> +/*
>> + *  Copyright (c) 2021 Meta Platforms, Inc. and affiliates
>> + */
>> +
>> +/**
>> + * dpll_notify_device_create - notify that the device has been created
>> + * @dpll: registered dpll pointer
>> + *
>> + * Return: 0 if succeeds, error code otherwise.
>> + */
>> +int dpll_notify_device_create(struct dpll_device *dpll);
>> +
>> +
>> +/**
>> + * dpll_notify_device_delete - notify that the device has been deleted
>> + * @dpll: registered dpll pointer
>> + *
>> + * Return: 0 if succeeds, error code otherwise.
>> + */
>> +int dpll_notify_device_delete(struct dpll_device *dpll);
>> +
>> +int dpll_pin_notify(struct dpll_device *dpll, struct dpll_pin *pin,
>> +		    enum dplla attr);
>> +
>> +int dpll_pin_parent_notify(struct dpll_device *dpll, struct dpll_pin *pin,
>> +			   struct dpll_pin *parent, enum dplla attr);
>> +
>> +int __init dpll_netlink_init(void);
>> +void dpll_netlink_finish(void);
>> diff --git a/include/linux/dpll.h b/include/linux/dpll.h
>> new file mode 100644
>> index 000000000000..db98b6d4bb73
>> --- /dev/null
>> +++ b/include/linux/dpll.h
>> @@ -0,0 +1,284 @@
>> +/* SPDX-License-Identifier: GPL-2.0 */
>> +/*
>> + *  Copyright (c) 2021 Meta Platforms, Inc. and affiliates
>> + */
>> +
>> +#ifndef __DPLL_H__
>> +#define __DPLL_H__
>> +
>> +#include <uapi/linux/dpll.h>
>> +#include <linux/device.h>
>> +#include <linux/netlink.h>
>> +
>> +struct dpll_device;
>> +struct dpll_pin;
>> +
>> +#define PIN_IDX_INVALID		((u32)ULONG_MAX)
>> +
>> +struct dpll_device_ops {
>> +	int (*mode_get)(const struct dpll_device *dpll, enum dpll_mode *mode,
>> +			struct netlink_ext_ack *extack);
>> +	int (*mode_set)(const struct dpll_device *dpll,
>> +			const enum dpll_mode mode,
>> +			struct netlink_ext_ack *extack);
>> +	bool (*mode_supported)(const struct dpll_device *dpll,
>> +			       const enum dpll_mode mode,
>> +			       struct netlink_ext_ack *extack);
>> +	int (*source_pin_idx_get)(const struct dpll_device *dpll,
>> +				  u32 *pin_idx,
>> +				  struct netlink_ext_ack *extack);
>> +	int (*lock_status_get)(const struct dpll_device *dpll,
>> +			       enum dpll_lock_status *status,
>> +			       struct netlink_ext_ack *extack);
>> +	int (*temp_get)(const struct dpll_device *dpll, s32 *temp,
>> +			struct netlink_ext_ack *extack);
>> +};
>> +
>> +struct dpll_pin_ops {
>> +	int (*frequency_set)(const struct dpll_pin *pin,
>> +			     const struct dpll_device *dpll,
>> +			     const u32 frequency,
>> +			     struct netlink_ext_ack *extack);
>> +	int (*frequency_get)(const struct dpll_pin *pin,
>> +			     const struct dpll_device *dpll,
>> +			     u32 *frequency, struct netlink_ext_ack *extack);
>> +	int (*direction_set)(const struct dpll_pin *pin,
>> +			     const struct dpll_device *dpll,
>> +			     const enum dpll_pin_direction direction,
>> +			     struct netlink_ext_ack *extack);
>> +	int (*direction_get)(const struct dpll_pin *pin,
>> +			     const struct dpll_device *dpll,
>> +			     enum dpll_pin_direction *direction,
>> +			     struct netlink_ext_ack *extack);
>> +	int (*state_on_pin_get)(const struct dpll_pin *pin,
>> +				const struct dpll_pin *parent_pin,
>> +				enum dpll_pin_state *state,
>> +				struct netlink_ext_ack *extack);
>> +	int (*state_on_dpll_get)(const struct dpll_pin *pin,
>> +				 const struct dpll_device *dpll,
>> +				 enum dpll_pin_state *state,
>> +				 struct netlink_ext_ack *extack);
>> +	int (*state_on_pin_set)(const struct dpll_pin *pin,
>> +				const struct dpll_pin *parent_pin,
>> +				const enum dpll_pin_state state,
>> +				struct netlink_ext_ack *extack);
>> +	int (*state_on_dpll_set)(const struct dpll_pin *pin,
>> +				 const struct dpll_device *dpll,
>> +				 const enum dpll_pin_state state,
>> +				 struct netlink_ext_ack *extack);
>> +	int (*prio_get)(const struct dpll_pin *pin,
>> +			const struct dpll_device *dpll,
>> +			u32 *prio, struct netlink_ext_ack *extack);
>> +	int (*prio_set)(const struct dpll_pin *pin,
>> +			const struct dpll_device *dpll,
>> +			const u32 prio, struct netlink_ext_ack *extack);
>> +};
>> +
>> +struct dpll_pin_properties {
>> +	const char *description;
>> +	enum dpll_pin_type type;
>> +	unsigned long freq_supported;
>> +	u32 any_freq_min;
>> +	u32 any_freq_max;
>> +	unsigned long capabilities;
>> +};
>> +
>> +enum dpll_pin_freq_supp {
>> +	DPLL_PIN_FREQ_SUPP_UNSPEC = 0,
>> +	DPLL_PIN_FREQ_SUPP_1_HZ,
>> +	DPLL_PIN_FREQ_SUPP_10_MHZ,
>> +
>> +	__DPLL_PIN_FREQ_SUPP_MAX,
>> +	DPLL_PIN_FREQ_SUPP_MAX = (__DPLL_PIN_FREQ_SUPP_MAX - 1)
>> +};
>> +
>> +/**
>> + * dpll_device_get - find or create dpll_device object
>> + * @clock_id: a system unique number for a device
>> + * @dev_driver_idx: index of dpll device on parent device
>> + * @module: register module
>> + *
>> + * Returns:
>> + * * pointer to initialized dpll - success
>> + * * NULL - memory allocation fail
>> + */
>> +struct dpll_device
>> +*dpll_device_get(u64 clock_id, u32 dev_driver_id, struct module *module);
>> +
>> +/**
>> + * dpll_device_put - caller drops reference to the device, free resources
>> + * @dpll: dpll device pointer
>> + *
>> + * If all dpll_device_get callers drops their reference, the dpll device
>> + * resources are freed.
>> + */
>> +void dpll_device_put(struct dpll_device *dpll);
>> +
>> +/**
>> + * dpll_device_register - register device, make it visible in the subsystem.
>> + * @dpll: reference previously allocated with dpll_device_get
>> + * @type: type of dpll
>> + * @ops: callbacks
>> + * @priv: private data of registerer
>> + * @owner: device struct of the owner
>> + *
>> + */
>> +int dpll_device_register(struct dpll_device *dpll, enum dpll_type type,
>> +			 struct dpll_device_ops *ops, void *priv,
>> +			 struct device *owner);
>> +
>> +/**
>> + * dpll_device_unregister - deregister registered dpll
>> + * @dpll: pointer to dpll
>> + *
>> + * Unregister the dpll from the subsystem, make it unavailable for netlink
>> + * API users.
>> + */
>> +void dpll_device_unregister(struct dpll_device *dpll);
>> +
>> +/**
>> + * dpll_priv - get dpll private data
>> + * @dpll: pointer to dpll
>> + *
>> + * Obtain private data pointer passed to dpll subsystem when allocating
>> + * device with ``dpll_device_alloc(..)``
>> + */
>> +void *dpll_priv(const struct dpll_device *dpll);
>> +
>> +/**
>> + * dpll_pin_on_pin_priv - get pin on pin pair private data
>> + * @parent: pointer to a parent pin
>> + * @pin: pointer to a dpll_pin
>> + *
>> + * Obtain private pin data pointer passed to dpll subsystem when pin
>> + * was registered with parent pin.
>> + */
>> +void *dpll_pin_on_pin_priv(const struct dpll_pin *parent, const struct dpll_pin *pin);
>> +
>> +/**
>> + * dpll_pin_on_dpll_priv - get pin on dpll pair private data
>> + * @dpll: pointer to dpll
>> + * @pin: pointer to a dpll_pin
>> + *
>> + * Obtain private pin-dpll pair data pointer passed to dpll subsystem when pin
>> + * was registered with a dpll.
>> + */
>> +void *dpll_pin_on_dpll_priv(const struct dpll_device *dpll, const struct dpll_pin *pin);
>> +
>> +/**
>> + * dpll_pin_get - get reference or create new pin object
>> + * @clock_id: a system unique number of a device
>> + * @dev_driver_idx: index of dpll device on parent device
>> + * @module: register module
>> + * @pin_prop: constant properities of a pin
>> + *
>> + * find existing pin with given clock_id, dev_driver_idx and module, or create new
>> + * and returen its reference.
>> + *
>> + * Returns:
>> + * * pointer to initialized pin - success
>> + * * NULL - memory allocation fail
>> + */
>> +struct dpll_pin
>> +*dpll_pin_get(u64 clock_id, u32 dev_driver_id, struct module *module,
> 
> Name inconsistency: dev_driver_id/idx in comment
> 
> 
>> +	      const struct dpll_pin_properties *pin_prop);
> 
> In .c you call this "prop", be consistent.
> 

Fixed both.

> 
>> +
>> +/**
>> + * dpll_pin_register - register pin with a dpll device
>> + * @dpll: pointer to dpll object to register pin with
>> + * @pin: pointer to allocated pin object being registered with dpll
>> + * @ops: struct with pin ops callbacks
>> + * @priv: private data pointer passed when calling callback ops
>> + * @rclk_device: pointer to device struct if pin is used for recovery of a clock
>> + * from that device
>> + *
>> + * Register previously allocated pin object with a dpll device.
>> + *
>> + * Return:
>> + * * 0 - if pin was registered with a parent pin,
>> + * * -ENOMEM - failed to allocate memory,
>> + * * -EEXIST - pin already registered with this dpll,
>> + * * -EBUSY - couldn't allocate id for a pin.
>> + */
>> +int dpll_pin_register(struct dpll_device *dpll, struct dpll_pin *pin,
>> +		      struct dpll_pin_ops *ops, void *priv,
>> +		      struct device *rclk_device);
>> +
>> +/**
>> + * dpll_pin_unregister - deregister pin from a dpll device
>> + * @dpll: pointer to dpll object to deregister pin from
>> + * @pin: pointer to allocated pin object being deregistered from dpll
>> + *
>> + * Deregister previously registered pin object from a dpll device.
>> + *
>> + * Return:
>> + * * 0 - pin was successfully deregistered from this dpll device,
>> + * * -ENXIO - given pin was not registered with this dpll device,
>> + * * -EINVAL - pin pointer is not valid.
>> + */
>> +int dpll_pin_unregister(struct dpll_device *dpll, struct dpll_pin *pin);
>> +
>> +/**
>> + * dpll_pin_put - drop reference to a pin acquired with dpll_pin_get
>> + * @pin: pointer to allocated pin
>> + *
>> + * Pins shall be deregistered from all dpll devices before putting them,
>> + * otherwise the memory won't be freed.
>> + */
>> +void dpll_pin_put(struct dpll_pin *pin);
>> +
>> +/**
>> + * dpll_pin_on_pin_register - register a pin to a muxed-type pin
>> + * @parent: parent pin pointer
>> + * @pin: pointer to allocated pin object being registered with a parent pin
>> + * @ops: struct with pin ops callbacks
>> + * @priv: private data pointer passed when calling callback ops
>> + * @rclk_device: pointer to device struct if pin is used for recovery of a clock
>> + * from that device
>> + *
>> + * In case of multiplexed pins, allows registring them under a single
>> + * parent pin.
>> + *
>> + * Return:
>> + * * 0 - if pin was registered with a parent pin,
>> + * * -ENOMEM - failed to allocate memory,
>> + * * -EEXIST - pin already registered with this parent pin,
>> + */
>> +int dpll_pin_on_pin_register(struct dpll_pin *parent, struct dpll_pin *pin,
>> +			     struct dpll_pin_ops *ops, void *priv,
>> +			     struct device *rclk_device);
>> +
>> +/**
>> + * dpll_pin_on_pin_register - register a pin to a muxed-type pin
>> + * @parent: parent pin pointer
>> + * @pin: pointer to allocated pin object being registered with a parent pin
>> + * @ops: struct with pin ops callbacks
>> + * @priv: private data pointer passed when calling callback ops
>> + * @rclk_device: pointer to device struct if pin is used for recovery of a clock
>> + * from that device
>> + *
>> + * In case of multiplexed pins, allows registring them under a single
>> + * parent pin.
>> + *
>> + * Return:
>> + * * 0 - if pin was registered with a parent pin,
>> + * * -ENOMEM - failed to allocate memory,
>> + * * -EEXIST - pin already registered with this parent pin,
>> + */
>> +void dpll_pin_on_pin_unregister(struct dpll_pin *parent, struct dpll_pin *pin);
>> +
>> +/**
>> + * dpll_device_notify - notify on dpll device change
>> + * @dpll: dpll device pointer
>> + * @attr: changed attribute
>> + *
>> + * Broadcast event to the netlink multicast registered listeners.
>> + *
>> + * Return:
>> + * * 0 - success
>> + * * negative - error
>> + */
>> +int dpll_device_notify(struct dpll_device *dpll, enum dplla attr);
>> +
>> +
>> +#endif
>> -- 
>> 2.34.1
>>

Jiri Pirko March 14, 2023, 3:45 p.m. UTC | #3

Sun, Mar 12, 2023 at 03:28:03AM CET, [e-mail] wrote:

[...]


>diff --git a/MAINTAINERS b/MAINTAINERS
>index edd3d562beee..0222b19af545 100644
>--- a/MAINTAINERS
>+++ b/MAINTAINERS
>@@ -6289,6 +6289,15 @@ F:	Documentation/networking/device_drivers/ethernet/freescale/dpaa2/switch-drive
> F:	drivers/net/ethernet/freescale/dpaa2/dpaa2-switch*
> F:	drivers/net/ethernet/freescale/dpaa2/dpsw*
> 
>+DPLL CLOCK SUBSYSTEM

Why "clock"? You don't mention "clock" anywhere else.

[...]


>diff --git a/drivers/dpll/dpll_core.c b/drivers/dpll/dpll_core.c
>new file mode 100644
>index 000000000000..3fc151e16751
>--- /dev/null
>+++ b/drivers/dpll/dpll_core.c
>@@ -0,0 +1,835 @@
>+// SPDX-License-Identifier: GPL-2.0
>+/*
>+ *  dpll_core.c - Generic DPLL Management class support.

Why "class" ?

[...]


>+static int
>+dpll_msg_add_pin_freq(struct sk_buff *msg, const struct dpll_pin *pin,
>+		      struct netlink_ext_ack *extack, bool dump_any_freq)
>+{
>+	enum dpll_pin_freq_supp fs;
>+	struct dpll_pin_ref *ref;
>+	unsigned long i;
>+	u32 freq;
>+
>+	xa_for_each((struct xarray *)&pin->dpll_refs, i, ref) {
>+		if (ref && ref->ops && ref->dpll)
>+			break;
>+	}
>+	if (!ref || !ref->ops || !ref->dpll)
>+		return -ENODEV;
>+	if (!ref->ops->frequency_get)
>+		return -EOPNOTSUPP;
>+	if (ref->ops->frequency_get(pin, ref->dpll, &freq, extack))
>+		return -EFAULT;
>+	if (nla_put_u32(msg, DPLL_A_PIN_FREQUENCY, freq))
>+		return -EMSGSIZE;
>+	if (!dump_any_freq)
>+		return 0;
>+	for (fs = DPLL_PIN_FREQ_SUPP_UNSPEC + 1;
>+	     fs <= DPLL_PIN_FREQ_SUPP_MAX; fs++) {
>+		if (test_bit(fs, &pin->prop.freq_supported)) {
>+			if (nla_put_u32(msg, DPLL_A_PIN_FREQUENCY_SUPPORTED,
>+			    dpll_pin_freq_value[fs]))

This is odd. As I suggested in the yaml patch, better to treat all
supported frequencies the same, no matter if it is range or not. The you
don't need this weird bitfield.

You can have a macro to help driver to assemble array of supported
frequencies and ranges.


>+				return -EMSGSIZE;
>+		}
>+	}
>+	if (pin->prop.any_freq_min && pin->prop.any_freq_max) {
>+		if (nla_put_u32(msg, DPLL_A_PIN_ANY_FREQUENCY_MIN,
>+				pin->prop.any_freq_min))
>+			return -EMSGSIZE;
>+		if (nla_put_u32(msg, DPLL_A_PIN_ANY_FREQUENCY_MAX,
>+				pin->prop.any_freq_max))
>+			return -EMSGSIZE;
>+	}
>+
>+	return 0;
>+}
>+

[...]


>+static int
>+dpll_cmd_pin_on_dpll_get(struct sk_buff *msg, struct dpll_pin *pin,
>+			 struct dpll_device *dpll,
>+			 struct netlink_ext_ack *extack)
>+{
>+	struct dpll_pin_ref *ref;
>+	int ret;
>+
>+	if (nla_put_u32(msg, DPLL_A_PIN_IDX, pin->dev_driver_id))
>+		return -EMSGSIZE;
>+	if (nla_put_string(msg, DPLL_A_PIN_DESCRIPTION, pin->prop.description))
>+		return -EMSGSIZE;
>+	if (nla_put_u8(msg, DPLL_A_PIN_TYPE, pin->prop.type))
>+		return -EMSGSIZE;
>+	if (nla_put_u32(msg, DPLL_A_PIN_DPLL_CAPS, pin->prop.capabilities))
>+		return -EMSGSIZE;
>+	ret = dpll_msg_add_pin_direction(msg, pin, extack);
>+	if (ret)
>+		return ret;
>+	ret = dpll_msg_add_pin_freq(msg, pin, extack, true);
>+	if (ret && ret != -EOPNOTSUPP)
>+		return ret;
>+	ref = dpll_xa_ref_dpll_find(&pin->dpll_refs, dpll);
>+	if (!ref)

How exactly this can happen? Looks to me like only in case of a bug.
WARN_ON() perhaps (put directly into dpll_xa_ref_dpll_find()?


>+		return -EFAULT;
>+	ret = dpll_msg_add_pin_prio(msg, pin, ref, extack);
>+	if (ret && ret != -EOPNOTSUPP)
>+		return ret;
>+	ret = dpll_msg_add_pin_on_dpll_state(msg, pin, ref, extack);
>+	if (ret && ret != -EOPNOTSUPP)
>+		return ret;
>+	ret = dpll_msg_add_pin_parents(msg, pin, extack);
>+	if (ret)
>+		return ret;
>+	if (pin->rclk_dev_name)

Use && and single if


>+		if (nla_put_string(msg, DPLL_A_PIN_RCLK_DEVICE,
>+				   pin->rclk_dev_name))
>+			return -EMSGSIZE;
>+
>+	return 0;
>+}
>+

[...]


>+static int
>+dpll_pin_freq_set(struct dpll_pin *pin, struct nlattr *a,
>+		  struct netlink_ext_ack *extack)
>+{
>+	u32 freq = nla_get_u32(a);
>+	struct dpll_pin_ref *ref;
>+	unsigned long i;
>+	int ret;
>+
>+	if (!dpll_pin_is_freq_supported(pin, freq))
>+		return -EINVAL;
>+
>+	xa_for_each(&pin->dpll_refs, i, ref) {
>+		ret = ref->ops->frequency_set(pin, ref->dpll, freq, extack);
>+		if (ret)
>+			return -EFAULT;

return what the op returns: ret


>+		dpll_pin_notify(ref->dpll, pin, DPLL_A_PIN_FREQUENCY);
>+	}
>+
>+	return 0;
>+}
>+

[...]


>+static int
>+dpll_pin_direction_set(struct dpll_pin *pin, struct nlattr *a,
>+		       struct netlink_ext_ack *extack)
>+{
>+	enum dpll_pin_direction direction = nla_get_u8(a);
>+	struct dpll_pin_ref *ref;
>+	unsigned long i;
>+
>+	if (!(DPLL_PIN_CAPS_DIRECTION_CAN_CHANGE & pin->prop.capabilities))
>+		return -EOPNOTSUPP;
>+
>+	xa_for_each(&pin->dpll_refs, i, ref) {
>+		if (ref->ops->direction_set(pin, ref->dpll, direction, extack))

ret = ..
if (ret)
	return ret;

Please use this pattern in other ops call code as well.


>+			return -EFAULT;
>+		dpll_pin_notify(ref->dpll, pin, DPLL_A_PIN_DIRECTION);
>+	}
>+
>+	return 0;

[...]

Kubalewski, Arkadiusz March 14, 2023, 4:43 p.m. UTC | #4

>From: Vadim Fedorenko <[e-mail]>
>Sent: Tuesday, March 14, 2023 12:00 AM
>

[...]

>>> +
>>> +/**
>>> + * dpll_device_get - find existing or create new dpll device
>>> + * @clock_id: clock_id of creator
>>> + * @dev_driver_id: id given by dev driver
>>> + * @module: reference to registering module
>>> + *
>>> + * Get existing object of a dpll device, unique for given arguments.
>>> + * Create new if doesn't exist yet.
>>> + *
>>> + * Return:
>>> + * * valid dpll_device struct pointer if succeeded
>>> + * * ERR_PTR of an error
>>> + */
>>> +struct dpll_device *
>>> +dpll_device_get(u64 clock_id, u32 dev_driver_id, struct module *module)
>>> +{
>>> +	struct dpll_device *dpll, *ret = NULL;
>>> +	unsigned long index;
>>> +
>>> +	mutex_lock(&dpll_device_xa_lock);
>>> +	xa_for_each(&dpll_device_xa, index, dpll) {
>>> +		if (dpll->clock_id == clock_id &&
>>> +		    dpll->dev_driver_id == dev_driver_id &&
>>
>> Why you need "dev_driver_id"? clock_id is here for the purpose of
>> identification, isn't that enough for you.
>
>dev_driver_id is needed to provide several DPLLs from one device. In ice
>driver
>implementation there are 2 different DPLLs - to recover from PPS input and
>to
>recover from Sync-E. I believe there is only one clock, that's why clock id
>is
>the same for both of them. But Arkadiusz can tell more about it.

Yes, exactly.
One driver can have multiple dplls with the same clock id.
Actually dev_driver_id makes dpll objects unique.

>>
>> Plus, the name is odd. "dev_driver" should certainly be avoided.
>
>Simply id doesn't tell anything either. dpll_dev_id?

Looks good to me.

>
>>> +		    dpll->module == module) {
>>> +			ret = dpll;
>>> +			refcount_inc(&ret->refcount);
>>> +			break;
>>> +		}
>>> +	}
>>> +	if (!ret)
>>> +		ret = dpll_device_alloc(clock_id, dev_driver_id, module);
>>> +	mutex_unlock(&dpll_device_xa_lock);
>>> +
>>> +	return ret;
>>> +}
>>> +EXPORT_SYMBOL_GPL(dpll_device_get);
>>> +
>>> +/**
>>> + * dpll_device_put - decrease the refcount and free memory if possible
>>> + * @dpll: dpll_device struct pointer
>>> + *
>>> + * Drop reference for a dpll device, if all references are gone, delete
>>> + * dpll device object.
>>> + */
>>> +void dpll_device_put(struct dpll_device *dpll)
>>> +{
>>> +	if (!dpll)
>>> +		return;
>>
>> Remove this check. The driver should not call this with NULL.
>
>Well, netdev_put() has this kind of check. As well as spi_dev_put() or
>i2c_put_adapter() at least. Not sure I would like to avoid a bit of safety.
>

I agree, IMHO it is better to have safety checks :)

>>> +	mutex_lock(&dpll_device_xa_lock);
>>> +	if (refcount_dec_and_test(&dpll->refcount)) {
>>> +		WARN_ON_ONCE(!xa_empty(&dpll->pin_refs));
>>
>> ASSERT_DPLL_NOT_REGISTERED(dpll);
>
>Good point!
>

Yes, great point!

>>> +		xa_destroy(&dpll->pin_refs);
>>> +		xa_erase(&dpll_device_xa, dpll->id);
>>> +		kfree(dpll);
>>> +	}
>>> +	mutex_unlock(&dpll_device_xa_lock);
>>> +}
>>> +EXPORT_SYMBOL_GPL(dpll_device_put);
>>> +
>>> +/**
>>> + * dpll_device_register - register the dpll device in the subsystem
>>> + * @dpll: pointer to a dpll
>>> + * @type: type of a dpll
>>> + * @ops: ops for a dpll device
>>> + * @priv: pointer to private information of owner
>>> + * @owner: pointer to owner device
>>> + *
>>> + * Make dpll device available for user space.
>>> + *
>>> + * Return:
>>> + * * 0 on success
>>> + * * -EINVAL on failure
>>> + */
>>> +int dpll_device_register(struct dpll_device *dpll, enum dpll_type type,
>>> +			 struct dpll_device_ops *ops, void *priv,
>>> +			 struct device *owner)
>>> +{
>>> +	if (WARN_ON(!ops || !owner))
>>> +		return -EINVAL;
>>> +	if (WARN_ON(type <= DPLL_TYPE_UNSPEC || type > DPLL_TYPE_MAX))
>>> +		return -EINVAL;
>>> +	mutex_lock(&dpll_device_xa_lock);
>>> +	if (ASSERT_DPLL_NOT_REGISTERED(dpll)) {
>>> +		mutex_unlock(&dpll_device_xa_lock);
>>> +		return -EEXIST;
>>> +	}
>>> +	dpll->dev.bus = owner->bus;
>>> +	dpll->parent = owner;
>>> +	dpll->type = type;
>>> +	dpll->ops = ops;
>>> +	dev_set_name(&dpll->dev, "%s_%d", dev_name(owner),
>>> +		     dpll->dev_driver_id);
>>
>> This is really odd. As a result, the user would see something like:
>> pci/0000:01:00.0_1
>> pci/0000:01:00.0_2
>>
>> I have to say it is confusing. In devlink, is bus/name and the user
>> could use this info to look trough sysfs. Here, 0000:01:00.0_1 is not
>> there. Also, "_" might have some meaning on some bus. Should not
>> concatename dev_name() with anything.
>>
>> Thinking about this some more, the module/clock_id tuple should be
>> uniqueue and stable. It is used for dpll_device_get(), it could be used
>> as the user handle, can't it?
>> Example:
>> ice/c92d02a7129f4747
>> mlx5/90265d8bf6e6df56
>>
>> If you really need the "dev_driver_id" (as I believe clock_id should be
>> enough), you can put it here as well:
>> ice/c92d02a7129f4747/1
>> ice/c92d02a7129f4747/2
>>
>
>Looks good, will change it

Makes sense to me.

>
>> This would also be beneficial for mlx5, as mlx5 with 2 PFs would like to
>> share instance of DPLL equally, there is no "one clock master". >
>>> +	dpll->priv = priv;
>>> +	xa_set_mark(&dpll_device_xa, dpll->id, DPLL_REGISTERED);
>>> +	mutex_unlock(&dpll_device_xa_lock);
>>> +	dpll_notify_device_create(dpll);
>>> +
>>> +	return 0;
>>> +}
>>> +EXPORT_SYMBOL_GPL(dpll_device_register);
>>> +
>>> +/**
>>> + * dpll_device_unregister - deregister dpll device
>>> + * @dpll: registered dpll pointer
>>> + *
>>> + * Deregister device, make it unavailable for userspace.
>>> + * Note: It does not free the memory
>>> + */
>>> +void dpll_device_unregister(struct dpll_device *dpll)
>>> +{
>>> +	mutex_lock(&dpll_device_xa_lock);
>>> +	ASSERT_DPLL_REGISTERED(dpll);
>>> +	xa_clear_mark(&dpll_device_xa, dpll->id, DPLL_REGISTERED);
>>> +	mutex_unlock(&dpll_device_xa_lock);
>>> +	dpll_notify_device_delete(dpll);
>>> +}
>>> +EXPORT_SYMBOL_GPL(dpll_device_unregister);
>>> +
>>> +/**
>>> + * dpll_pin_alloc - allocate the memory for dpll pin
>>> + * @clock_id: clock_id of creator
>>> + * @dev_driver_id: id given by dev driver
>>> + * @module: reference to registering module
>>> + * @prop: dpll pin properties
>>> + *
>>> + * Return:
>>> + * * valid allocated dpll_pin struct pointer if succeeded
>>> + * * ERR_PTR of an error
>>
>> Extra "*"'s
>
>Ok, I can re-format the comments across the files.

This is expected on enumerating return values in kernel-docs:
https://www.kernel.org/doc/html/latest/doc-guide/kernel-doc.html#Return%20values

>
>>> + */
>>> +struct dpll_pin *
>>> +dpll_pin_alloc(u64 clock_id, u8 device_drv_id,	struct module
>*module,
>>
>> Odd whitespace.
>>
>> Also, func should be static.
>>
>
>Fixed.
>
>>
>>> +	       const struct dpll_pin_properties *prop)
>>> +{
>>> +	struct dpll_pin *pin;
>>> +	int ret;
>>> +
>>> +	pin = kzalloc(sizeof(*pin), GFP_KERNEL);
>>> +	if (!pin)
>>> +		return ERR_PTR(-ENOMEM);
>>> +	pin->dev_driver_id = device_drv_id;
>>
>> Name inconsistency: driver/drv
>> you have it on multiple places
>>
>
>Changed it every where, thanks for spotting.
>
>>
>>> +	pin->clock_id = clock_id;
>>> +	pin->module = module;
>>> +	refcount_set(&pin->refcount, 1);
>>> +	if (WARN_ON(!prop->description)) {
>>> +		ret = -EINVAL;
>>> +		goto release;
>>> +	}
>>> +	pin->prop.description = kstrdup(prop->description, GFP_KERNEL);
>>> +	if (!pin->prop.description) {
>>> +		ret = -ENOMEM;
>>> +		goto release;
>>> +	}
>>> +	if (WARN_ON(prop->type <= DPLL_PIN_TYPE_UNSPEC ||
>>> +		    prop->type > DPLL_PIN_TYPE_MAX)) {
>>> +		ret = -EINVAL;
>>> +		goto release;
>>> +	}
>>> +	pin->prop.type = prop->type;
>>> +	pin->prop.capabilities = prop->capabilities;
>>> +	pin->prop.freq_supported = prop->freq_supported;
>>> +	pin->prop.any_freq_min = prop->any_freq_min;
>>> +	pin->prop.any_freq_max = prop->any_freq_max;
>>
>> Make sure that the driver maintains prop (static const) and just save
>> the pointer. Prop does not need to be something driver needs to change.
>>
>
>What's the difference? For ptp_ocp, we have the same configuration for all
>ext pins and the allocator only changes the name of the pin. Properties of
>the DPLL pins are stored within the pin object, not the driver, in this
>case.
>Not sure if the pointer way is much better...
>

I also don't feel it.
Dpll subsystem directly using memory of different driver doesn't look like a
great design.

>>
>>> +	xa_init_flags(&pin->dpll_refs, XA_FLAGS_ALLOC);
>>> +	xa_init_flags(&pin->parent_refs, XA_FLAGS_ALLOC);
>>> +	ret = xa_alloc(&dpll_pin_xa, &pin->idx, pin,
>>> +		       xa_limit_16b, GFP_KERNEL);
>>> +release:
>>> +	if (ret) {
>>> +		xa_destroy(&pin->dpll_refs);
>>> +		xa_destroy(&pin->parent_refs);
>>> +		kfree(pin->prop.description);
>>> +		kfree(pin->rclk_dev_name);
>>> +		kfree(pin);
>>> +		return ERR_PTR(ret);
>>> +	}
>>> +
>>> +	return pin;
>>> +}
>>> +
>>> +/**
>>> + * dpll_pin_get - find existing or create new dpll pin
>>> + * @clock_id: clock_id of creator
>>> + * @dev_driver_id: id given by dev driver
>>> + * @module: reference to registering module
>>> + * @prop: dpll pin properties
>>> + *
>>> + * Get existing object of a pin (unique for given arguments) or create
>>>new
>>> + * if doesn't exist yet.
>>> + *
>>> + * Return:
>>> + * * valid allocated dpll_pin struct pointer if succeeded
>>> + * * ERR_PTR of an error
>>
>> This is one example, I'm pretty sure that there are others, when you
>> have text inconsistencies in func doc for the same function in .c and .h
>> Have it please only on one place. .c is the usual place.
>>
>
>Yep, will clear .h files.
>

There might be some issues, and certainly makes sense to have them in one
place.

>>
>>> + */
>>> +struct dpll_pin *
>>> +dpll_pin_get(u64 clock_id, u32 device_drv_id, struct module *module,
>>
>> Again, why do you need this device_drv_id? Clock id should be enough.
>>
>I explained the reason earlier, but the naming is fixed.

Yes, this device_drv_id is id of a pin not a dpll device id.
Maybe worth to rename it to make it more clear.

>
>>
>>> +	     const struct dpll_pin_properties *prop)
>>> +{
>>> +	struct dpll_pin *pos, *ret = NULL;
>>> +	unsigned long index;
>>> +
>>> +	mutex_lock(&dpll_pin_xa_lock);
>>> +	xa_for_each(&dpll_pin_xa, index, pos) {
>>> +		if (pos->clock_id == clock_id &&
>>> +		    pos->dev_driver_id == device_drv_id &&
>>> +		    pos->module == module) {
>>
>> Compare prop as well.
>>
>> Can't the driver_id (pin index) be something const as well? I think it
>> should. And therefore it could be easily put inside.
>>
>
>I think clock_id + dev_driver_id + module should identify the pin exactly.

Yes, they should be unique for a single pin object, and enough for getting
pin reference.
Basically if driver would call twice this with different props, it would be
broken.
We could have props compared here as well.

>And
>now I think that *prop is not needed here at all. Arkadiusz, any thoughts?

As dpll_pin_alloc is having them assigned to a pin object, thus had to put it
here. Not sure how to solve it differently.
Jiri's suggestion to put dev_driver_id inside of pin props would also work
AFAIU.

[...]

>>> +
>>> +/**
>>> + * dpll_pin_register - register the dpll pin in the subsystem
>>> + * @dpll: pointer to a dpll
>>> + * @pin: pointer to a dpll pin
>>> + * @ops: ops for a dpll pin ops
>>> + * @priv: pointer to private information of owner
>>> + * @rclk_device: pointer to recovered clock device
>>> + *
>>> + * Return:
>>> + * * 0 on success
>>> + * * -EINVAL - missing dpll or pin
>>> + * * -ENOMEM - failed to allocate memory
>>> + */
>>> +int
>>> +dpll_pin_register(struct dpll_device *dpll, struct dpll_pin *pin,
>>> +		  struct dpll_pin_ops *ops, void *priv,
>>> +		  struct device *rclk_device)
>>
>> Wait a second, what is this "struct device *"? Looks very odd.
>>
>>
>>> +{
>>> +	const char *rclk_name = rclk_device ? dev_name(rclk_device) : NULL;
>>
>> If you need to store something here, store the pointer to the device
>> directly. But this rclk_device seems odd to me.
>> Dev_name is in case of PCI device for example 0000:01:00.0? That alone
>> is incomplete. What should it server for?
>>
>
>Well, these questions go to Arkadiusz...
>

If pin is able to recover signal from some device this shall convey that
device struct pointer.
Name of that device is later passed to the user with DPLL_A_PIN_RCLK_DEVICE
attribute.
Sure we can have pointer to device and use dev_name (each do/dump) on netlink
part. But isn't it better to have the name ready to use there?

It might be incomplete only if one device would have some kind of access to
a different bus? I don't think it is valid use case.

Basically the driver will refer only to the devices handled by that driver,
which means if dpll is on some bus, also all the pins are there, didn't notice
the need to have bus here as well.

[...]

>>> +struct dpll_pin *dpll_pin_get_by_idx(struct dpll_device *dpll, u32 idx)
>>> +{
>>> +	struct dpll_pin_ref *pos;
>>> +	unsigned long i;
>>> +
>>> +	xa_for_each(&dpll->pin_refs, i, pos) {
>>> +		if (pos && pos->pin && pos->pin->dev_driver_id == idx)
>>
>> How exactly pos->pin could be NULL?
>>

I believe if proper synchronization in place it shall not be NULL, I left it
after fixing some issue with access to the pin that was already removed..

>> Also, you are degrading the xarray to a mere list here with lookup like
>> this. Why can't you use the pin index coming from driver and
>> insert/lookup based on this index?
>>
>Good point. We just have to be sure, that drivers provide 0-based indexes
>for their pins. I'll re-think it.
>

After quick thinking, it might be doable, storing pin being registered on
dpll->pin_refs under given by driver pin's dev_driver_idx or whatever it would
be named.

>
>>
>>> +			return pos->pin;
>>> +	}
>>> +
>>> +	return NULL;
>>> +}
>>> +
>>> +/**
>>> + * dpll_priv - get the dpll device private owner data
>>> + * @dpll:	registered dpll pointer
>>> + *
>>> + * Return: pointer to the data
>>> + */
>>> +void *dpll_priv(const struct dpll_device *dpll)
>>> +{
>>> +	return dpll->priv;
>>> +}
>>> +EXPORT_SYMBOL_GPL(dpll_priv);
>>> +
>>> +/**
>>> + * dpll_pin_on_dpll_priv - get the dpll device private owner data
>>> + * @dpll:	registered dpll pointer
>>> + * @pin:	pointer to a pin
>>> + *
>>> + * Return: pointer to the data
>>> + */
>>> +void *dpll_pin_on_dpll_priv(const struct dpll_device *dpll,
>>
>> IIUC, you use this helper from dpll ops in drivers to get per dpll priv.
>> Just pass the priv directly to the op and avoid need for this helper,
>> no? Same goes to the rest of the priv helpers.
>>

No strong opinion, probably doable.

>>
>>> +			    const struct dpll_pin *pin)
>>> +{
>>> +	struct dpll_pin_ref *ref;
>>> +
>>> +	ref = dpll_xa_ref_pin_find((struct xarray *)&dpll->pin_refs, pin);
>>
>> Why cast is needed here? You have this on multiple places.
>>

`const` of struct dpll_pin *pin makes a warning/error there, there is something
broken on xarray implementation of for_each I believe.

[...]

>>> +
>>> +static int
>>> +dpll_msg_add_pins_on_pin(struct sk_buff *msg, struct dpll_pin *pin,
>>> +			 struct netlink_ext_ack *extack)
>>> +{
>>> +	struct dpll_pin_ref *ref = NULL;
>>
>> Why this needs to be initialized?
>>
>No need, fixed.
>
>
>>
>>> +	enum dpll_pin_state state;
>>> +	struct nlattr *nest;
>>> +	unsigned long index;
>>> +	int ret;
>>> +
>>> +	xa_for_each(&pin->parent_refs, index, ref) {
>>> +		if (WARN_ON(!ref->ops->state_on_pin_get))
>>> +			return -EFAULT;
>>> +		ret = ref->ops->state_on_pin_get(pin, ref->pin, &state,
>>> +						 extack);
>>> +		if (ret)
>>> +			return -EFAULT;
>>> +		nest = nla_nest_start(msg, DPLL_A_PIN_PARENT);
>>> +		if (!nest)
>>> +			return -EMSGSIZE;
>>> +		if (nla_put_u32(msg, DPLL_A_PIN_PARENT_IDX,
>>> +				ref->pin->dev_driver_id)) {
>>> +			ret = -EMSGSIZE;
>>> +			goto nest_cancel;
>>> +		}
>>> +		if (nla_put_u8(msg, DPLL_A_PIN_STATE, state)) {
>>> +			ret = -EMSGSIZE;
>>> +			goto nest_cancel;
>>> +		}
>>> +		nla_nest_end(msg, nest);
>>> +	}
>>
>> How is this function different to dpll_msg_add_pin_parents()?
>> Am I lost? To be honest, this x_on_pin/dpll, parent, refs dance is quite
>> hard to follow for me :/
>>
>> Did you get lost here as well? If yes, this needs some serious think
>> through :)
>>
>
>Let's re-think it again. Arkadiuzs, do you have clear explanation of the
>relationship between these things?
>

No, it is just leftover I didn't catch, we can leave one function and use it in
both cases. Sorry about that, great catch!

>>
>>> +
>>> +	return 0;
>>> +
>>> +nest_cancel:
>>> +	nla_nest_cancel(msg, nest);
>>> +	return ret;
>>> +}
>>> +
>>> +static int
>>> +dpll_msg_add_pin_dplls(struct sk_buff *msg, struct dpll_pin *pin,
>>> +		       struct netlink_ext_ack *extack)
>>> +{
>>> +	struct dpll_pin_ref *ref;
>>> +	struct nlattr *attr;
>>> +	unsigned long index;
>>> +	int ret;
>>> +
>>> +	xa_for_each(&pin->dpll_refs, index, ref) {
>>> +		attr = nla_nest_start(msg, DPLL_A_DEVICE);
>>> +		if (!attr)
>>> +			return -EMSGSIZE;
>>> +		ret = dpll_msg_add_dev_handle(msg, ref->dpll);
>>> +		if (ret)
>>> +			goto nest_cancel;
>>> +		ret = dpll_msg_add_pin_on_dpll_state(msg, pin, ref, extack);
>>> +		if (ret && ret != -EOPNOTSUPP)
>>> +			goto nest_cancel;
>>> +		ret = dpll_msg_add_pin_prio(msg, pin, ref, extack);
>>> +		if (ret && ret != -EOPNOTSUPP)
>>> +			goto nest_cancel;
>>> +		nla_nest_end(msg, attr);
>>> +	}
>>> +
>>> +	return 0;
>>> +
>>> +nest_cancel:
>>> +	nla_nest_end(msg, attr);
>>> +	return ret;
>>> +}
>>> +
>>> +static int
>>> +dpll_cmd_pin_on_dpll_get(struct sk_buff *msg, struct dpll_pin *pin,
>>> +			 struct dpll_device *dpll,
>>> +			 struct netlink_ext_ack *extack)
>>> +{
>>> +	struct dpll_pin_ref *ref;
>>> +	int ret;
>>> +
>>> +	if (nla_put_u32(msg, DPLL_A_PIN_IDX, pin->dev_driver_id))
>>
>> Is it "ID" or "INDEX" (IDX). Please make this consistent in the whole
>> code.
>>
>
>I believe it's INDEX which is provided by the driver. I'll think about
>renaming,
>but suggestions are welcome.

Yes, confusing a bit, I agree we need a fix of this.
Isn't it that dpll has an INDEX assigned by dpll subsystem on its allocation
and pin have ID given by the driver?

[...]

>>> +static int
>>> +dpll_pin_set_from_nlattr(struct dpll_device *dpll,
>>> +			 struct dpll_pin *pin, struct genl_info *info)
>>> +{
>>> +	enum dpll_pin_state state = DPLL_PIN_STATE_UNSPEC;
>>> +	u32 parent_idx = PIN_IDX_INVALID;
>>
>> You just need this PIN_IDX_INVALID define internally in this function,
>> change the flow to avoid a need for it.
>>
>
>I'll re-think it, thanks.
>
>>
>>> +	int rem, ret = -EINVAL;
>>> +	struct nlattr *a;
>>> +
>>> +	nla_for_each_attr(a, genlmsg_data(info->genlhdr),
>>> +			  genlmsg_len(info->genlhdr), rem) {
>>
>> This is odd. Why you iterace over attrs? Why don't you just access them
>> directly, like attrs[DPLL_A_PIN_FREQUENCY] for example?
>>
>
>I had some unknown crashes when I was using such access. I might have lost
>some
>checks, will try it again.
>
>>
>>> +		switch (nla_type(a)) {
>>> +		case DPLL_A_PIN_FREQUENCY:
>>> +			ret = dpll_pin_freq_set(pin, a, info->extack);
>>> +			if (ret)
>>> +				return ret;
>>> +			break;
>>> +		case DPLL_A_PIN_DIRECTION:
>>> +			ret = dpll_pin_direction_set(pin, a, info->extack);
>>> +			if (ret)
>>> +				return ret;
>>> +			break;
>>> +		case DPLL_A_PIN_PRIO:
>>> +			ret = dpll_pin_prio_set(dpll, pin, a, info->extack);
>>> +			if (ret)
>>> +				return ret;
>>> +			break;
>>> +		case DPLL_A_PIN_PARENT_IDX:
>>> +			parent_idx = nla_get_u32(a);
>>> +			break;
>>> +		case DPLL_A_PIN_STATE:
>>> +			state = nla_get_u8(a);
>>> +			break;
>>> +		default:
>>> +			break;
>>> +		}
>>> +	}
>>> +	if (state != DPLL_PIN_STATE_UNSPEC) {
>>
>> Again, change the flow to:
>> 	if (attrs[DPLL_A_PIN_STATE]) {
>>
>> and avoid need for this value set/check.
>>
>
>Yep, will try.

Yes, this shall work now, as long as there are no multiple nested attributes
coming from userspace.

[...]

>>> +void dpll_pin_post_doit(const struct genl_split_ops *ops, struct
>>>sk_buff *skb,
>>> +			struct genl_info *info)
>>> +{
>>> +	mutex_unlock(&dpll_pin_xa_lock);
>>> +	dpll_post_doit(ops, skb, info);
>>> +}
>>> +
>>> +int dpll_pin_pre_dumpit(struct netlink_callback *cb)
>>> +{
>>> +	mutex_lock(&dpll_pin_xa_lock);
>>
>> ABBA deadlock here, see dpll_pin_register() for example where the lock
>> taking order is opposite.
>>
>
>Now I see an ABBA deadlock here, as well as in function before. Not sure
>how to
>solve it here. Any thoughts?
>

Not really, this is why it is there :(
A single global lock for whole dpll subsystem?

>>
>>> +
>>> +	return dpll_pre_dumpit(cb);
>>> +}
>>> +
>>> +int dpll_pin_post_dumpit(struct netlink_callback *cb)
>>> +{
>>> +	mutex_unlock(&dpll_pin_xa_lock);
>>> +
>>> +	return dpll_post_dumpit(cb);
>>> +}
>>> +
>>> +static int
>>> +dpll_event_device_change(struct sk_buff *msg, struct dpll_device *dpll,
>>> +			 struct dpll_pin *pin, struct dpll_pin *parent,
>>> +			 enum dplla attr)
>>> +{
>>> +	int ret = dpll_msg_add_dev_handle(msg, dpll);
>>> +	struct dpll_pin_ref *ref = NULL;
>>> +	enum dpll_pin_state state;
>>> +
>>> +	if (ret)
>>> +		return ret;
>>> +	if (pin && nla_put_u32(msg, DPLL_A_PIN_IDX, pin->dev_driver_id))
>>> +		return -EMSGSIZE;
>>
>> I don't really understand why you are trying figure something new and
>> interesting with the change notifications. This object mix and random
>> attrs fillup is something very wrong and makes userspace completely
>> fuzzy about what it is getting. And yet it is so simple:
>> You have 2 objects, dpll and pin, please just have:
>> dpll_notify()
>> dpll_pin_notify()
>> and share the attrs fillup code with pin_get() and dpll_get() callbacks.
>> No need for any smartness. Have this dumb and simple.
>>
>> Think about it more as about "object-state-snapshot" than "atomic-change"
>
>But with full object-snapshot user space app will lose the information
>about
>what exactly has changed. The reason to have this event is to provide the
>attributes which have changed. Otherwise, the app should have full snapshot
>and
>compare all attributes to figure out changes and that's might not be great
>idea.
>

I agree that from functional perspective it is better to have on userspace a
reason of the notification.

[...]

Thank you,
Arkadiusz

Kubalewski, Arkadiusz March 14, 2023, 5:50 p.m. UTC | #5

>From: Jiri Pirko <[e-mail]>
>Sent: Tuesday, March 14, 2023 10:22 AM
>
>Mon, Mar 13, 2023 at 11:59:32PM CET, [e-mail] wrote:
>>On 13.03.2023 16:21, Jiri Pirko wrote:
>>> Sun, Mar 12, 2023 at 03:28:03AM CET, [e-mail] wrote:
>
>[...]
>
>
>>> > diff --git a/drivers/dpll/Makefile b/drivers/dpll/Makefile
>>> > new file mode 100644
>>> > index 000000000000..d3926f2a733d
>>> > --- /dev/null
>>> > +++ b/drivers/dpll/Makefile
>>> > @@ -0,0 +1,10 @@
>>> > +# SPDX-License-Identifier: GPL-2.0
>>> > +#
>>> > +# Makefile for DPLL drivers.
>>> > +#
>>> > +
>>> > +obj-$(CONFIG_DPLL)          += dpll_sys.o
>>>
>>> What's "sys" and why is it here?
>>
>>It's an object file for the subsystem. Could be useful if we will have
>>drivers
>>for DPLL-only devices.
>
>Yeah, but why "sys"? I don't get what "sys" means here.
>Can't this be just "dpll.o"?
>
>
>>
>>> > +dpll_sys-y                  += dpll_core.o
>>> > +dpll_sys-y                  += dpll_netlink.o
>>> > +dpll_sys-y                  += dpll_nl.o
>>> > +
>
>[...]
>
>
>>> > +struct dpll_device *
>>> > +dpll_device_get(u64 clock_id, u32 dev_driver_id, struct module
>>> > *module)
>>> > +{
>>> > +	struct dpll_device *dpll, *ret = NULL;
>>> > +	unsigned long index;
>>> > +
>>> > +	mutex_lock(&dpll_device_xa_lock);
>>> > +	xa_for_each(&dpll_device_xa, index, dpll) {
>>> > +		if (dpll->clock_id == clock_id &&
>>> > +		    dpll->dev_driver_id == dev_driver_id &&
>>>
>>> Why you need "dev_driver_id"? clock_id is here for the purpose of
>>> identification, isn't that enough for you.
>>
>>dev_driver_id is needed to provide several DPLLs from one device. In ice
>>driver
>>implementation there are 2 different DPLLs - to recover from PPS input and
>>to
>>recover from Sync-E. I believe there is only one clock, that's why clock id
>>is the same for both of them. But Arkadiusz can tell more about it.
>
>Okay, I see. Clock_id is the same. Could we have index for pin, could
>this be index too:
>
>dpll_device_get(u64 clock_id, u32 device_index, struct module *module);
>dpll_pin_get(u64 clock_id, u32 pin_index, struct module *module,
>	     const struct dpll_pin_properties *prop);
>
>This way it is consistent, driver provides custom index for both dpll
>device and dpll pin.
>
>Makes sense?
>

IMHO, Yes this better shows the intentions.

>
>>>
>>> Plus, the name is odd. "dev_driver" should certainly be avoided.
>>
>>Simply id doesn't tell anything either. dpll_dev_id?
>
>Yeah, see above.
>
>
>>
>>> > +		    dpll->module == module) {
>>> > +			ret = dpll;
>>> > +			refcount_inc(&ret->refcount);
>>> > +			break;
>>> > +		}
>>> > +	}
>>> > +	if (!ret)
>>> > +		ret = dpll_device_alloc(clock_id, dev_driver_id, module);
>>> > +	mutex_unlock(&dpll_device_xa_lock);
>>> > +
>>> > +	return ret;
>>> > +}
>>> > +EXPORT_SYMBOL_GPL(dpll_device_get);
>>> > +
>>> > +/**
>>> > + * dpll_device_put - decrease the refcount and free memory if possible
>>> > + * @dpll: dpll_device struct pointer
>>> > + *
>>> > + * Drop reference for a dpll device, if all references are gone, delete
>>> > + * dpll device object.
>>> > + */
>>> > +void dpll_device_put(struct dpll_device *dpll)
>>> > +{
>>> > +	if (!dpll)
>>> > +		return;
>>>
>>> Remove this check. The driver should not call this with NULL.
>>
>>Well, netdev_put() has this kind of check. As well as spi_dev_put() or
>>i2c_put_adapter() at least. Not sure I would like to avoid a bit of safety.
>
>IDK, maybe for historical reasons. My point is, id driver is callin
>this with NULL, there is something odd in the driver flow. Lets not
>allow that for new code.
>
>
>>
>>> > +	mutex_lock(&dpll_device_xa_lock);
>>> > +	if (refcount_dec_and_test(&dpll->refcount)) {
>>> > +		WARN_ON_ONCE(!xa_empty(&dpll->pin_refs));
>>>
>>> ASSERT_DPLL_NOT_REGISTERED(dpll);
>>
>>Good point!
>>
>>> > +		xa_destroy(&dpll->pin_refs);
>>> > +		xa_erase(&dpll_device_xa, dpll->id);
>>> > +		kfree(dpll);
>>> > +	}
>>> > +	mutex_unlock(&dpll_device_xa_lock);
>>> > +}
>>> > +EXPORT_SYMBOL_GPL(dpll_device_put);
>>> > +
>>> > +/**
>>> > + * dpll_device_register - register the dpll device in the subsystem
>>> > + * @dpll: pointer to a dpll
>>> > + * @type: type of a dpll
>>> > + * @ops: ops for a dpll device
>>> > + * @priv: pointer to private information of owner
>>> > + * @owner: pointer to owner device
>>> > + *
>>> > + * Make dpll device available for user space.
>>> > + *
>>> > + * Return:
>>> > + * * 0 on success
>>> > + * * -EINVAL on failure
>>> > + */
>>> > +int dpll_device_register(struct dpll_device *dpll, enum dpll_type type,
>>> > +			 struct dpll_device_ops *ops, void *priv,
>>> > +			 struct device *owner)
>>> > +{
>>> > +	if (WARN_ON(!ops || !owner))
>>> > +		return -EINVAL;
>>> > +	if (WARN_ON(type <= DPLL_TYPE_UNSPEC || type > DPLL_TYPE_MAX))
>>> > +		return -EINVAL;
>>> > +	mutex_lock(&dpll_device_xa_lock);
>>> > +	if (ASSERT_DPLL_NOT_REGISTERED(dpll)) {
>>> > +		mutex_unlock(&dpll_device_xa_lock);
>>> > +		return -EEXIST;
>>> > +	}
>>> > +	dpll->dev.bus = owner->bus;
>>> > +	dpll->parent = owner;
>>> > +	dpll->type = type;
>>> > +	dpll->ops = ops;
>>> > +	dev_set_name(&dpll->dev, "%s_%d", dev_name(owner),
>>> > +		     dpll->dev_driver_id);
>>>
>>> This is really odd. As a result, the user would see something like:
>>> pci/0000:01:00.0_1
>>> pci/0000:01:00.0_2
>>>
>>> I have to say it is confusing. In devlink, is bus/name and the user
>>> could use this info to look trough sysfs. Here, 0000:01:00.0_1 is not
>>> there. Also, "_" might have some meaning on some bus. Should not
>>> concatename dev_name() with anything.
>>>
>>> Thinking about this some more, the module/clock_id tuple should be
>>> uniqueue and stable. It is used for dpll_device_get(), it could be used
>>> as the user handle, can't it?
>>> Example:
>>> ice/c92d02a7129f4747
>>> mlx5/90265d8bf6e6df56
>>>
>>> If you really need the "dev_driver_id" (as I believe clock_id should be
>>> enough), you can put it here as well:
>>> ice/c92d02a7129f4747/1
>>> ice/c92d02a7129f4747/2
>>>
>>
>>Looks good, will change it
>
>Great.
>
>
>>
>>> This would also be beneficial for mlx5, as mlx5 with 2 PFs would like to
>>> share instance of DPLL equally, there is no "one clock master". >
>
>[...]
>
>
>>> > +	pin->prop.description = kstrdup(prop->description, GFP_KERNEL);
>>> > +	if (!pin->prop.description) {
>>> > +		ret = -ENOMEM;
>>> > +		goto release;
>>> > +	}
>>> > +	if (WARN_ON(prop->type <= DPLL_PIN_TYPE_UNSPEC ||
>>> > +		    prop->type > DPLL_PIN_TYPE_MAX)) {
>>> > +		ret = -EINVAL;
>>> > +		goto release;
>>> > +	}
>>> > +	pin->prop.type = prop->type;
>>> > +	pin->prop.capabilities = prop->capabilities;
>>> > +	pin->prop.freq_supported = prop->freq_supported;
>>> > +	pin->prop.any_freq_min = prop->any_freq_min;
>>> > +	pin->prop.any_freq_max = prop->any_freq_max;
>>>
>>> Make sure that the driver maintains prop (static const) and just save
>>> the pointer. Prop does not need to be something driver needs to change.
>>>
>>
>>What's the difference? For ptp_ocp, we have the same configuration for all
>>ext pins and the allocator only changes the name of the pin. Properties of
>>the DPLL pins are stored within the pin object, not the driver, in this
>case.
>>Not sure if the pointer way is much better...
>
>For things like this it is common to have static const array in the
>driver, like:
>
>static const struct dpll_pin_properties dpll_pin_props[] = {
>	{
>		.description = "SMA0",
>		.freq_supported = DPLL_PIN_FREQ_SUPP_MAX,
>		.type = DPLL_PIN_TYPE_EXT,
>		.any_freq_max = 10000000,
>		.capabilities = DPLL_PIN_CAPS_DIRECTION_CAN_CHANGE,
>	},
>	{
>		.description = "SMA1",
>		.freq_supported = DPLL_PIN_FREQ_SUPP_MAX,
>		.type = DPLL_PIN_TYPE_EXT,
>		.any_freq_max = 10000000,
>		.capabilities = DPLL_PIN_CAPS_DIRECTION_CAN_CHANGE,
>	},
>	{
>		.description = "SMA2",
>		.freq_supported = DPLL_PIN_FREQ_SUPP_MAX,
>		.type = DPLL_PIN_TYPE_EXT,
>		.any_freq_max = 10000000,
>		.capabilities = DPLL_PIN_CAPS_DIRECTION_CAN_CHANGE,
>	},
>	{
>		.description = "SMA3",
>		.freq_supported = DPLL_PIN_FREQ_SUPP_MAX,
>		.type = DPLL_PIN_TYPE_EXT,
>		.any_freq_max = 10000000,
>		.capabilities = DPLL_PIN_CAPS_DIRECTION_CAN_CHANGE,
>	},
>};
>
>Here you have very nice list of pins, the reader knows right away what
>is happening.
>
>Thinking about "description" name, I think would be more appropriate to
>name this "label" as it represents user-facing label on the connector,
>isn't it? Does not describe anything.
>

"label" seems good.

>
>>
>>>
>
>[...]
>
>
>>
>>>
>>> > +	     const struct dpll_pin_properties *prop)
>>> > +{
>>> > +	struct dpll_pin *pos, *ret = NULL;
>>> > +	unsigned long index;
>>> > +
>>> > +	mutex_lock(&dpll_pin_xa_lock);
>>> > +	xa_for_each(&dpll_pin_xa, index, pos) {
>>> > +		if (pos->clock_id == clock_id &&
>>> > +		    pos->dev_driver_id == device_drv_id &&
>>> > +		    pos->module == module) {
>>>
>>> Compare prop as well.
>>>
>>> Can't the driver_id (pin index) be something const as well? I think it
>>> should. And therefore it could be easily put inside.
>>>
>>
>>I think clock_id + dev_driver_id + module should identify the pin exactly.
>>And now I think that *prop is not needed here at all. Arkadiusz, any
>>thoughts?
>
>IDK, no strong opinion on this. I just thought it may help to identify
>the pin and avoid potential driver bugs. (Like registering 2 pins with
>the same properties).
>

It would make most sense if pin_index would be a part of *prop.

>[...]
>
>
>>> > +/**
>>> > + * dpll_pin_register - register the dpll pin in the subsystem
>>> > + * @dpll: pointer to a dpll
>>> > + * @pin: pointer to a dpll pin
>>> > + * @ops: ops for a dpll pin ops
>>> > + * @priv: pointer to private information of owner
>>> > + * @rclk_device: pointer to recovered clock device
>>> > + *
>>> > + * Return:
>>> > + * * 0 on success
>>> > + * * -EINVAL - missing dpll or pin
>>> > + * * -ENOMEM - failed to allocate memory
>>> > + */
>>> > +int
>>> > +dpll_pin_register(struct dpll_device *dpll, struct dpll_pin *pin,
>>> > +		  struct dpll_pin_ops *ops, void *priv,
>>> > +		  struct device *rclk_device)
>>>
>>> Wait a second, what is this "struct device *"? Looks very odd.
>>>
>>>
>>> > +{
>>> > +	const char *rclk_name = rclk_device ? dev_name(rclk_device) :
>>> >NULL;
>>>
>>> If you need to store something here, store the pointer to the device
>>> directly. But this rclk_device seems odd to me.
>>> Dev_name is in case of PCI device for example 0000:01:00.0? That alone
>>> is incomplete. What should it server for?
>>>
>>
>>Well, these questions go to Arkadiusz...
>


[ copy paste my answer from previous response ]
If pin is able to recover signal from some device this shall convey that
device struct pointer.
Name of that device is later passed to the user with DPLL_A_PIN_RCLK_DEVICE
attribute.
Sure we can have pointer to device and use dev_name (each do/dump) on netlink
part. But isn't it better to have the name ready to use there?

It might be incomplete only if one device would have some kind of access to
a different bus? I don't think it is valid use case.

Basically the driver will refer only to the devices handled by that driver,
which means if dpll is on some bus, also all the pins are there, didn't notice
the need to have bus here as well.

>Okay.
>
>[...]
>
>
>>> > + * dpll_pin_get_by_idx - find a pin ref on dpll by pin index
>>> > + * @dpll: dpll device pointer
>>> > + * @idx: index of pin
>>> > + *
>>> > + * Find a reference to a pin registered with given dpll and return
>>> > its pointer.
>>> > + *
>>> > + * Return:
>>> > + * * valid pointer if pin was found
>>> > + * * NULL if not found
>>> > + */
>>> > +struct dpll_pin *dpll_pin_get_by_idx(struct dpll_device *dpll, u32 idx)
>>> > +{
>>> > +	struct dpll_pin_ref *pos;
>>> > +	unsigned long i;
>>> > +
>>> > +	xa_for_each(&dpll->pin_refs, i, pos) {
>>> > +		if (pos && pos->pin && pos->pin->dev_driver_id == idx)
>>>
>>> How exactly pos->pin could be NULL?
>>>
>>> Also, you are degrading the xarray to a mere list here with lookup like
>>> this. Why can't you use the pin index coming from driver and
>>> insert/lookup based on this index?
>>>
>>Good point. We just have to be sure, that drivers provide 0-based indexes for
>>their pins. I'll re-think it.
>
>No, driver can provide indexing which is completely up to his decision.
>You should use xa_insert() to insert the entry at specific index. See
>devl_port_register for inspiration where it is done exactly like this.
>
>And this should be done in exactly the same way for both pin and device.
>

Yes, I agree seems doable and better then it is now.

[...]

>>> > +static int
>>> > +dpll_msg_add_dev_handle(struct sk_buff *msg, const struct dpll_device
>>> >*dpll)
>>> > +{
>>> > +	if (nla_put_u32(msg, DPLL_A_ID, dpll->id))
>>>
>>> Why exactly do we need this dua--handle scheme? Why do you need
>>> unpredictable DPLL_A_ID to be exposed to userspace?
>>> It's just confusing.
>>>
>>To be able to work with DPLL per integer after iterator on the list deducts
>>which DPLL device is needed. It can reduce the amount of memory copies and
>>simplify comparisons. Not sure why it's confusing.
>
>Wait, I don't get it. Could you please explain a bit more?
>
>My point is, there should be not such ID exposed over netlink
>You don't need to expose it to userspace. The user has well defined
>handle as you agreed with above. For example:
>
>ice/c92d02a7129f4747/1
>ice/c92d02a7129f4747/2
>
>This is shown in dpll device GET/DUMP outputs.
>Also user passes it during SET operation:
>$ dplltool set ice/c92d02a7129f4747/1 mode auto
>
>Isn't that enough stable and nice?
>

I agree with Vadim, this is rather to be used by a daemon tools, which
would get the index once, then could use it as long as device is there.

>
>>
>>>
>>> > +		return -EMSGSIZE;
>>> > +	if (nla_put_string(msg, DPLL_A_BUS_NAME, dev_bus_name(&dpll-
>>> > dev)))
>>> > +		return -EMSGSIZE;
>>> > +	if (nla_put_string(msg, DPLL_A_DEV_NAME, dev_name(&dpll->dev)))
>>> > +		return -EMSGSIZE;
>>> > +
>>> > +	return 0;
>>> > +}
>>
>>[...]
>>
>>> > +
>>> > +static int
>>> > +dpll_msg_add_pins_on_pin(struct sk_buff *msg, struct dpll_pin *pin,
>>> > +			 struct netlink_ext_ack *extack)
>>> > +{
>>> > +	struct dpll_pin_ref *ref = NULL;
>>>
>>> Why this needs to be initialized?
>>>
>>No need, fixed.
>>
>>
>>>
>>> > +	enum dpll_pin_state state;
>>> > +	struct nlattr *nest;
>>> > +	unsigned long index;
>>> > +	int ret;
>>> > +
>>> > +	xa_for_each(&pin->parent_refs, index, ref) {
>>> > +		if (WARN_ON(!ref->ops->state_on_pin_get))
>>> > +			return -EFAULT;
>>> > +		ret = ref->ops->state_on_pin_get(pin, ref->pin, &state,
>>> > +						 extack);
>>> > +		if (ret)
>>> > +			return -EFAULT;
>>> > +		nest = nla_nest_start(msg, DPLL_A_PIN_PARENT);
>>> > +		if (!nest)
>>> > +			return -EMSGSIZE;
>>> > +		if (nla_put_u32(msg, DPLL_A_PIN_PARENT_IDX,
>>> > +				ref->pin->dev_driver_id)) {
>>> > +			ret = -EMSGSIZE;
>>> > +			goto nest_cancel;
>>> > +		}
>>> > +		if (nla_put_u8(msg, DPLL_A_PIN_STATE, state)) {
>>> > +			ret = -EMSGSIZE;
>>> > +			goto nest_cancel;
>>> > +		}
>>> > +		nla_nest_end(msg, nest);
>>> > +	}
>>>
>>> How is this function different to dpll_msg_add_pin_parents()?
>>> Am I lost? To be honest, this x_on_pin/dpll, parent, refs dance is quite
>>> hard to follow for me :/
>>>
>>> Did you get lost here as well? If yes, this needs some serious think
>>> through :)
>>>
>>
>>Let's re-think it again. Arkadiuzs, do you have clear explanation of the
>>relationship between these things?
>

[ copy paste my answer from previous response ]
No, it is just leftover I didn't catch, we can leave one function and use it in
both cases. Sorry about that, great catch!

[...]

>>> > +	if (nla_put_u32(msg, DPLL_A_PIN_IDX, pin->dev_driver_id))
>>>
>>> Is it "ID" or "INDEX" (IDX). Please make this consistent in the whole
>>> code.
>>>
>>
>>I believe it's INDEX which is provided by the driver. I'll think about
>>renaming,
>>but suggestions are welcome.
>
>Let's use "index" and "INDEX" internalla and in Netlink attr names as
>well then.
>

For me makes sense to have a common name instead of origin-based one.

>[...]
>
>
>>
>>>
>>> > +	int rem, ret = -EINVAL;
>>> > +	struct nlattr *a;
>>> > +
>>> > +	nla_for_each_attr(a, genlmsg_data(info->genlhdr),
>>> > +			  genlmsg_len(info->genlhdr), rem) {
>>>
>>> This is odd. Why you iterace over attrs? Why don't you just access them
>>> directly, like attrs[DPLL_A_PIN_FREQUENCY] for example?
>>>
>>
>>I had some unknown crashes when I was using such access. I might have lost
>>some checks, will try it again.
>
>Odd, yet definitelly debuggable though :)
>
>[...]
>
>
>>> > +int dpll_pin_pre_dumpit(struct netlink_callback *cb)
>>> > +{
>>> > +	mutex_lock(&dpll_pin_xa_lock);
>>>
>>> ABBA deadlock here, see dpll_pin_register() for example where the lock
>>> taking order is opposite.
>>>
>>
>>Now I see an ABBA deadlock here, as well as in function before. Not sure
>>how to
>>solve it here. Any thoughts?
>
>Well, here you can just call dpll_pre_dumpit() before
>mutex_lock(&dpll_pin_xa_lock)
>to take the locks in the same order.
>

This double lock doesn't really improve anything.
Any objections on having single mutex/lock for access the dpll subsystem?

>
>>
>>>
>>> > +
>>> > +	return dpll_pre_dumpit(cb);
>>> > +}
>>> > +
>>> > +int dpll_pin_post_dumpit(struct netlink_callback *cb)
>>> > +{
>>> > +	mutex_unlock(&dpll_pin_xa_lock);
>>> > +
>>> > +	return dpll_post_dumpit(cb);
>>> > +}
>>> > +
>>> > +static int
>>> > +dpll_event_device_change(struct sk_buff *msg, struct dpll_device
>>> > *dpll,
>>> > +			 struct dpll_pin *pin, struct dpll_pin *parent,
>>> > +			 enum dplla attr)
>>> > +{
>>> > +	int ret = dpll_msg_add_dev_handle(msg, dpll);
>>> > +	struct dpll_pin_ref *ref = NULL;
>>> > +	enum dpll_pin_state state;
>>> > +
>>> > +	if (ret)
>>> > +		return ret;
>>> > +	if (pin && nla_put_u32(msg, DPLL_A_PIN_IDX, pin-
>>> > dev_driver_id))
>>> > +		return -EMSGSIZE;
>>>
>>> I don't really understand why you are trying figure something new and
>>> interesting with the change notifications. This object mix and random
>>> attrs fillup is something very wrong and makes userspace completely
>>> fuzzy about what it is getting. And yet it is so simple:
>>> You have 2 objects, dpll and pin, please just have:
>>> dpll_notify()
>>> dpll_pin_notify()
>>> and share the attrs fillup code with pin_get() and dpll_get() callbacks.
>>> No need for any smartness. Have this dumb and simple.
>>>
>>> Think about it more as about "object-state-snapshot" than "atomic-change"
>>
>>But with full object-snapshot user space app will lose the information about
>>what exactly has changed. The reason to have this event is to provide the
>>attributes which have changed. Otherwise, the app should have full snapshot
>>and
>>compare all attributes to figure out changes and that's might not be great
>>idea.
>
>Wait, are you saying that the app is stateless? Could you provide
>example use cases?
>
>From what I see, the app managing dpll knows the state of the device and
>pins, it monitors for the changes and saves new state with appropriate
>reaction (might be some action or maybe just log entry).
>

It depends on the use case, right? App developer having those information knows
what has changed, thus can react in a way it thinks is most suitable.
IMHO from user perspective it is good to have a notification which actually
shows it's reason, so proper flow could be assigned to handle the reaction.

>
>>
>>>
>>> > +
>>> > +	switch (attr) {
>>> > +	case DPLL_A_MODE:
>>> > +		ret = dpll_msg_add_mode(msg, dpll, NULL);
>>> > +		break;
>>> > +	case DPLL_A_SOURCE_PIN_IDX:
>>> > +		ret = dpll_msg_add_source_pin_idx(msg, dpll, NULL);
>>> > +		break;
>>> > +	case DPLL_A_LOCK_STATUS:
>>> > +		ret = dpll_msg_add_lock_status(msg, dpll, NULL);
>>> > +		break;
>>> > +	case DPLL_A_TEMP:
>>> > +		ret = dpll_msg_add_temp(msg, dpll, NULL);
>>> > +		break;
>>> > +	case DPLL_A_PIN_FREQUENCY:
>>> > +		ret = dpll_msg_add_pin_freq(msg, pin, NULL, false);
>>> > +		break;
>>> > +	case DPLL_A_PIN_PRIO:
>>> > +		ref = dpll_xa_ref_dpll_find(&pin->dpll_refs, dpll);
>>> > +		if (!ref)
>>> > +			return -EFAULT;
>>> > +		ret = dpll_msg_add_pin_prio(msg, pin, ref, NULL);
>>> > +		break;
>>> > +	case DPLL_A_PIN_STATE:
>>> > +		if (parent) {
>>> > +			ref = dpll_xa_ref_pin_find(&pin->parent_refs, parent);
>>> > +			if (!ref)
>>> > +				return -EFAULT;
>>> > +			if (!ref->ops || !ref->ops->state_on_pin_get)
>>> > +				return -EOPNOTSUPP;
>>> > +			ret = ref->ops->state_on_pin_get(pin, parent, &state,
>>> > +							 NULL);
>>> > +			if (ret)
>>> > +				return ret;
>>> > +			if (nla_put_u32(msg, DPLL_A_PIN_PARENT_IDX,
>>> > +					parent->dev_driver_id))
>>> > +				return -EMSGSIZE;
>>> > +		} else {
>>> > +			ref = dpll_xa_ref_dpll_find(&pin->dpll_refs, dpll);
>>> > +			if (!ref)
>>> > +				return -EFAULT;
>>> > +			ret = dpll_msg_add_pin_on_dpll_state(msg, pin, ref,
>>> > +							     NULL);
>>> > +			if (ret)
>>> > +				return ret;
>>> > +		}
>>> > +		break;
>>> > +	default:
>>> > +		break;
>>> > +	}
>>> > +
>>> > +	return ret;
>>> > +}
>>> > +
>>> > +static int
>>> > +dpll_send_event_create(enum dpll_event event, struct dpll_device *dpll)
>>> > +{
>>> > +	struct sk_buff *msg;
>>> > +	int ret = -EMSGSIZE;
>>> > +	void *hdr;
>>> > +
>>> > +	msg = genlmsg_new(NLMSG_GOODSIZE, GFP_KERNEL);
>>> > +	if (!msg)
>>> > +		return -ENOMEM;
>>> > +
>>> > +	hdr = genlmsg_put(msg, 0, 0, &dpll_nl_family, 0, event);
>>> > +	if (!hdr)
>>> > +		goto out_free_msg;
>>> > +
>>> > +	ret = dpll_msg_add_dev_handle(msg, dpll);
>>> > +	if (ret)
>>> > +		goto out_cancel_msg;
>>> > +	genlmsg_end(msg, hdr);
>>> > +	genlmsg_multicast(&dpll_nl_family, msg, 0, 0, GFP_KERNEL);
>>> > +
>>> > +	return 0;
>>> > +
>>> > +out_cancel_msg:
>>> > +	genlmsg_cancel(msg, hdr);
>>> > +out_free_msg:
>>> > +	nlmsg_free(msg);
>>> > +
>>> > +	return ret;
>>> > +}
>>> > +
>>> > +static int
>>> > +dpll_send_event_change(struct dpll_device *dpll, struct dpll_pin *pin,
>>> > +		       struct dpll_pin *parent, enum dplla attr)
>>> > +{
>>> > +	struct sk_buff *msg;
>>> > +	int ret = -EMSGSIZE;
>>> > +	void *hdr;
>>> > +
>>> > +	msg = genlmsg_new(NLMSG_GOODSIZE, GFP_KERNEL);
>>> > +	if (!msg)
>>> > +		return -ENOMEM;
>>> > +
>>> > +	hdr = genlmsg_put(msg, 0, 0, &dpll_nl_family, 0,
>>> > +			  DPLL_EVENT_DEVICE_CHANGE);
>>>
>>> I don't really get it. Why exactly you keep having this *EVENT* cmds?
>>> Why per-object NEW/GET/DEL cmds shared with get genl op are not enough?
>>> I have to be missing something.
>>
>>Changes might come from other places, but will affect the DPLL device and we
>>have to notify users in this case.
>
>I'm not sure I follow. There are 2 scenarios for change:
>1) user originated - user issues set of something
>2) driver originated - something changes in HW, driver propagates that
>
>With what I suggest, both scenarios work of course. My point is, user
>knows very well the objects: device and pin, he knows the format or
>messages that are related to GET/DUMP/SET operations of both. The
>notification should have the same format, that is all I say.
>
>Btw, you can see devlink code for example how the notifications like
>this are implemented and work.
>

Devlink packs all the info into a netlink message and notifies with it, isn't
it that it has all the info "buffered" in its structures?
A dpll subsystem keeps only some of the info in its internal structures, but
for most of it it has to question the driver. It would do it multiple times
i.e. if there would be a change on a pin connected to multiple dpll devices.

With current approach the notifications are pretty light to the subsystem as
well as for the registered clients, and as you suggested with having info of
what actually changed the userspace could implement a stateless daemon right
away.

Thank you,
Arkadiusz

Kubalewski, Arkadiusz March 14, 2023, 6:35 p.m. UTC | #6

>From: Jiri Pirko <[e-mail]>
>Sent: Tuesday, March 14, 2023 4:45 PM
>
>[...]
>
>
>>diff --git a/MAINTAINERS b/MAINTAINERS
>>index edd3d562beee..0222b19af545 100644
>>--- a/MAINTAINERS
>>+++ b/MAINTAINERS
>>@@ -6289,6 +6289,15 @@ F:
>	Documentation/networking/device_drivers/ethernet/freescale/dpaa2/swit
>ch-drive
>> F:	drivers/net/ethernet/freescale/dpaa2/dpaa2-switch*
>> F:	drivers/net/ethernet/freescale/dpaa2/dpsw*
>>
>>+DPLL CLOCK SUBSYSTEM
>
>Why "clock"? You don't mention "clock" anywhere else.
>
>[...]
>
>
>>diff --git a/drivers/dpll/dpll_core.c b/drivers/dpll/dpll_core.c
>>new file mode 100644
>>index 000000000000..3fc151e16751
>>--- /dev/null
>>+++ b/drivers/dpll/dpll_core.c
>>@@ -0,0 +1,835 @@
>>+// SPDX-License-Identifier: GPL-2.0
>>+/*
>>+ *  dpll_core.c - Generic DPLL Management class support.
>
>Why "class" ?
>
>[...]
>
>
>>+static int
>>+dpll_msg_add_pin_freq(struct sk_buff *msg, const struct dpll_pin *pin,
>>+		      struct netlink_ext_ack *extack, bool dump_any_freq)
>>+{
>>+	enum dpll_pin_freq_supp fs;
>>+	struct dpll_pin_ref *ref;
>>+	unsigned long i;
>>+	u32 freq;
>>+
>>+	xa_for_each((struct xarray *)&pin->dpll_refs, i, ref) {
>>+		if (ref && ref->ops && ref->dpll)
>>+			break;
>>+	}
>>+	if (!ref || !ref->ops || !ref->dpll)
>>+		return -ENODEV;
>>+	if (!ref->ops->frequency_get)
>>+		return -EOPNOTSUPP;
>>+	if (ref->ops->frequency_get(pin, ref->dpll, &freq, extack))
>>+		return -EFAULT;
>>+	if (nla_put_u32(msg, DPLL_A_PIN_FREQUENCY, freq))
>>+		return -EMSGSIZE;
>>+	if (!dump_any_freq)
>>+		return 0;
>>+	for (fs = DPLL_PIN_FREQ_SUPP_UNSPEC + 1;
>>+	     fs <= DPLL_PIN_FREQ_SUPP_MAX; fs++) {
>>+		if (test_bit(fs, &pin->prop.freq_supported)) {
>>+			if (nla_put_u32(msg, DPLL_A_PIN_FREQUENCY_SUPPORTED,
>>+			    dpll_pin_freq_value[fs]))
>
>This is odd. As I suggested in the yaml patch, better to treat all
>supported frequencies the same, no matter if it is range or not. The you
>don't need this weird bitfield.
>
>You can have a macro to help driver to assemble array of supported
>frequencies and ranges.
>

I understand suggestion on yaml, but here I am confused.
How do they relate to the supported frequency passed between driver and dpll
subsystem?
This bitfield is not visible to the userspace, and sure probably adding macro
can be useful.

>
>>+				return -EMSGSIZE;
>>+		}
>>+	}
>>+	if (pin->prop.any_freq_min && pin->prop.any_freq_max) {
>>+		if (nla_put_u32(msg, DPLL_A_PIN_ANY_FREQUENCY_MIN,
>>+				pin->prop.any_freq_min))
>>+			return -EMSGSIZE;
>>+		if (nla_put_u32(msg, DPLL_A_PIN_ANY_FREQUENCY_MAX,
>>+				pin->prop.any_freq_max))
>>+			return -EMSGSIZE;
>>+	}
>>+
>>+	return 0;
>>+}
>>+
>
>[...]
>
>
>>+static int
>>+dpll_cmd_pin_on_dpll_get(struct sk_buff *msg, struct dpll_pin *pin,
>>+			 struct dpll_device *dpll,
>>+			 struct netlink_ext_ack *extack)
>>+{
>>+	struct dpll_pin_ref *ref;
>>+	int ret;
>>+
>>+	if (nla_put_u32(msg, DPLL_A_PIN_IDX, pin->dev_driver_id))
>>+		return -EMSGSIZE;
>>+	if (nla_put_string(msg, DPLL_A_PIN_DESCRIPTION, pin->prop.description))
>>+		return -EMSGSIZE;
>>+	if (nla_put_u8(msg, DPLL_A_PIN_TYPE, pin->prop.type))
>>+		return -EMSGSIZE;
>>+	if (nla_put_u32(msg, DPLL_A_PIN_DPLL_CAPS, pin->prop.capabilities))
>>+		return -EMSGSIZE;
>>+	ret = dpll_msg_add_pin_direction(msg, pin, extack);
>>+	if (ret)
>>+		return ret;
>>+	ret = dpll_msg_add_pin_freq(msg, pin, extack, true);
>>+	if (ret && ret != -EOPNOTSUPP)
>>+		return ret;
>>+	ref = dpll_xa_ref_dpll_find(&pin->dpll_refs, dpll);
>>+	if (!ref)
>
>How exactly this can happen? Looks to me like only in case of a bug.
>WARN_ON() perhaps (put directly into dpll_xa_ref_dpll_find()?

Yes, makes sense.

>
>
>>+		return -EFAULT;
>>+	ret = dpll_msg_add_pin_prio(msg, pin, ref, extack);
>>+	if (ret && ret != -EOPNOTSUPP)
>>+		return ret;
>>+	ret = dpll_msg_add_pin_on_dpll_state(msg, pin, ref, extack);
>>+	if (ret && ret != -EOPNOTSUPP)
>>+		return ret;
>>+	ret = dpll_msg_add_pin_parents(msg, pin, extack);
>>+	if (ret)
>>+		return ret;
>>+	if (pin->rclk_dev_name)
>
>Use && and single if
>

Make sense to me.

>
>>+		if (nla_put_string(msg, DPLL_A_PIN_RCLK_DEVICE,
>>+				   pin->rclk_dev_name))
>>+			return -EMSGSIZE;
>>+
>>+	return 0;
>>+}
>>+
>
>[...]
>
>
>>+static int
>>+dpll_pin_freq_set(struct dpll_pin *pin, struct nlattr *a,
>>+		  struct netlink_ext_ack *extack)
>>+{
>>+	u32 freq = nla_get_u32(a);
>>+	struct dpll_pin_ref *ref;
>>+	unsigned long i;
>>+	int ret;
>>+
>>+	if (!dpll_pin_is_freq_supported(pin, freq))
>>+		return -EINVAL;
>>+
>>+	xa_for_each(&pin->dpll_refs, i, ref) {
>>+		ret = ref->ops->frequency_set(pin, ref->dpll, freq, extack);
>>+		if (ret)
>>+			return -EFAULT;
>
>return what the op returns: ret

Why would we return here a driver return code, userspace can have this info
from extack. IMHO return values of dpll subsystem shall be not dependent on
what is returned from the driver.

>
>
>>+		dpll_pin_notify(ref->dpll, pin, DPLL_A_PIN_FREQUENCY);
>>+	}
>>+
>>+	return 0;
>>+}
>>+
>
>[...]
>
>
>>+static int
>>+dpll_pin_direction_set(struct dpll_pin *pin, struct nlattr *a,
>>+		       struct netlink_ext_ack *extack)
>>+{
>>+	enum dpll_pin_direction direction = nla_get_u8(a);
>>+	struct dpll_pin_ref *ref;
>>+	unsigned long i;
>>+
>>+	if (!(DPLL_PIN_CAPS_DIRECTION_CAN_CHANGE & pin->prop.capabilities))
>>+		return -EOPNOTSUPP;
>>+
>>+	xa_for_each(&pin->dpll_refs, i, ref) {
>>+		if (ref->ops->direction_set(pin, ref->dpll, direction, extack))
>
>ret = ..
>if (ret)
>	return ret;
>
>Please use this pattern in other ops call code as well.
>

This is the same as above (return code by driver) explanation.

>
>>+			return -EFAULT;
>>+		dpll_pin_notify(ref->dpll, pin, DPLL_A_PIN_DIRECTION);
>>+	}
>>+
>>+	return 0;
>
>[...]

Thanks,
Arkadiusz

Jiri Pirko March 15, 2023, 9:22 a.m. UTC | #7

Tue, Mar 14, 2023 at 06:50:57PM CET, [e-mail] wrote:
>>From: Jiri Pirko <[e-mail]>
>>Sent: Tuesday, March 14, 2023 10:22 AM
>>
>>Mon, Mar 13, 2023 at 11:59:32PM CET, [e-mail] wrote:
>>>On 13.03.2023 16:21, Jiri Pirko wrote:
>>>> Sun, Mar 12, 2023 at 03:28:03AM CET, [e-mail] wrote:
>>
>>[...]
>>
>>
>>>> > diff --git a/drivers/dpll/Makefile b/drivers/dpll/Makefile
>>>> > new file mode 100644
>>>> > index 000000000000..d3926f2a733d
>>>> > --- /dev/null
>>>> > +++ b/drivers/dpll/Makefile
>>>> > @@ -0,0 +1,10 @@
>>>> > +# SPDX-License-Identifier: GPL-2.0
>>>> > +#
>>>> > +# Makefile for DPLL drivers.
>>>> > +#
>>>> > +
>>>> > +obj-$(CONFIG_DPLL)          += dpll_sys.o
>>>>
>>>> What's "sys" and why is it here?
>>>
>>>It's an object file for the subsystem. Could be useful if we will have
>>>drivers
>>>for DPLL-only devices.
>>
>>Yeah, but why "sys"? I don't get what "sys" means here.
>>Can't this be just "dpll.o"?
>>
>>
>>>
>>>> > +dpll_sys-y                  += dpll_core.o
>>>> > +dpll_sys-y                  += dpll_netlink.o
>>>> > +dpll_sys-y                  += dpll_nl.o
>>>> > +
>>
>>[...]
>>
>>
>>>> > +struct dpll_device *
>>>> > +dpll_device_get(u64 clock_id, u32 dev_driver_id, struct module
>>>> > *module)
>>>> > +{
>>>> > +	struct dpll_device *dpll, *ret = NULL;
>>>> > +	unsigned long index;
>>>> > +
>>>> > +	mutex_lock(&dpll_device_xa_lock);
>>>> > +	xa_for_each(&dpll_device_xa, index, dpll) {
>>>> > +		if (dpll->clock_id == clock_id &&
>>>> > +		    dpll->dev_driver_id == dev_driver_id &&
>>>>
>>>> Why you need "dev_driver_id"? clock_id is here for the purpose of
>>>> identification, isn't that enough for you.
>>>
>>>dev_driver_id is needed to provide several DPLLs from one device. In ice
>>>driver
>>>implementation there are 2 different DPLLs - to recover from PPS input and
>>>to
>>>recover from Sync-E. I believe there is only one clock, that's why clock id
>>>is the same for both of them. But Arkadiusz can tell more about it.
>>
>>Okay, I see. Clock_id is the same. Could we have index for pin, could
>>this be index too:
>>
>>dpll_device_get(u64 clock_id, u32 device_index, struct module *module);
>>dpll_pin_get(u64 clock_id, u32 pin_index, struct module *module,
>>	     const struct dpll_pin_properties *prop);
>>
>>This way it is consistent, driver provides custom index for both dpll
>>device and dpll pin.
>>
>>Makes sense?
>>
>
>IMHO, Yes this better shows the intentions.

Ok.


>
>>
>>>>
>>>> Plus, the name is odd. "dev_driver" should certainly be avoided.
>>>
>>>Simply id doesn't tell anything either. dpll_dev_id?
>>
>>Yeah, see above.
>>
>>
>>>
>>>> > +		    dpll->module == module) {
>>>> > +			ret = dpll;
>>>> > +			refcount_inc(&ret->refcount);
>>>> > +			break;
>>>> > +		}
>>>> > +	}
>>>> > +	if (!ret)
>>>> > +		ret = dpll_device_alloc(clock_id, dev_driver_id, module);
>>>> > +	mutex_unlock(&dpll_device_xa_lock);
>>>> > +
>>>> > +	return ret;
>>>> > +}
>>>> > +EXPORT_SYMBOL_GPL(dpll_device_get);
>>>> > +
>>>> > +/**
>>>> > + * dpll_device_put - decrease the refcount and free memory if possible
>>>> > + * @dpll: dpll_device struct pointer
>>>> > + *
>>>> > + * Drop reference for a dpll device, if all references are gone, delete
>>>> > + * dpll device object.
>>>> > + */
>>>> > +void dpll_device_put(struct dpll_device *dpll)
>>>> > +{
>>>> > +	if (!dpll)
>>>> > +		return;
>>>>
>>>> Remove this check. The driver should not call this with NULL.
>>>
>>>Well, netdev_put() has this kind of check. As well as spi_dev_put() or
>>>i2c_put_adapter() at least. Not sure I would like to avoid a bit of safety.
>>
>>IDK, maybe for historical reasons. My point is, id driver is callin
>>this with NULL, there is something odd in the driver flow. Lets not
>>allow that for new code.
>>
>>
>>>
>>>> > +	mutex_lock(&dpll_device_xa_lock);
>>>> > +	if (refcount_dec_and_test(&dpll->refcount)) {
>>>> > +		WARN_ON_ONCE(!xa_empty(&dpll->pin_refs));
>>>>
>>>> ASSERT_DPLL_NOT_REGISTERED(dpll);
>>>
>>>Good point!
>>>
>>>> > +		xa_destroy(&dpll->pin_refs);
>>>> > +		xa_erase(&dpll_device_xa, dpll->id);
>>>> > +		kfree(dpll);
>>>> > +	}
>>>> > +	mutex_unlock(&dpll_device_xa_lock);
>>>> > +}
>>>> > +EXPORT_SYMBOL_GPL(dpll_device_put);
>>>> > +
>>>> > +/**
>>>> > + * dpll_device_register - register the dpll device in the subsystem
>>>> > + * @dpll: pointer to a dpll
>>>> > + * @type: type of a dpll
>>>> > + * @ops: ops for a dpll device
>>>> > + * @priv: pointer to private information of owner
>>>> > + * @owner: pointer to owner device
>>>> > + *
>>>> > + * Make dpll device available for user space.
>>>> > + *
>>>> > + * Return:
>>>> > + * * 0 on success
>>>> > + * * -EINVAL on failure
>>>> > + */
>>>> > +int dpll_device_register(struct dpll_device *dpll, enum dpll_type type,
>>>> > +			 struct dpll_device_ops *ops, void *priv,
>>>> > +			 struct device *owner)
>>>> > +{
>>>> > +	if (WARN_ON(!ops || !owner))
>>>> > +		return -EINVAL;
>>>> > +	if (WARN_ON(type <= DPLL_TYPE_UNSPEC || type > DPLL_TYPE_MAX))
>>>> > +		return -EINVAL;
>>>> > +	mutex_lock(&dpll_device_xa_lock);
>>>> > +	if (ASSERT_DPLL_NOT_REGISTERED(dpll)) {
>>>> > +		mutex_unlock(&dpll_device_xa_lock);
>>>> > +		return -EEXIST;
>>>> > +	}
>>>> > +	dpll->dev.bus = owner->bus;
>>>> > +	dpll->parent = owner;
>>>> > +	dpll->type = type;
>>>> > +	dpll->ops = ops;
>>>> > +	dev_set_name(&dpll->dev, "%s_%d", dev_name(owner),
>>>> > +		     dpll->dev_driver_id);
>>>>
>>>> This is really odd. As a result, the user would see something like:
>>>> pci/0000:01:00.0_1
>>>> pci/0000:01:00.0_2
>>>>
>>>> I have to say it is confusing. In devlink, is bus/name and the user
>>>> could use this info to look trough sysfs. Here, 0000:01:00.0_1 is not
>>>> there. Also, "_" might have some meaning on some bus. Should not
>>>> concatename dev_name() with anything.
>>>>
>>>> Thinking about this some more, the module/clock_id tuple should be
>>>> uniqueue and stable. It is used for dpll_device_get(), it could be used
>>>> as the user handle, can't it?
>>>> Example:
>>>> ice/c92d02a7129f4747
>>>> mlx5/90265d8bf6e6df56
>>>>
>>>> If you really need the "dev_driver_id" (as I believe clock_id should be
>>>> enough), you can put it here as well:
>>>> ice/c92d02a7129f4747/1
>>>> ice/c92d02a7129f4747/2
>>>>
>>>
>>>Looks good, will change it
>>
>>Great.
>>
>>
>>>
>>>> This would also be beneficial for mlx5, as mlx5 with 2 PFs would like to
>>>> share instance of DPLL equally, there is no "one clock master". >
>>
>>[...]
>>
>>
>>>> > +	pin->prop.description = kstrdup(prop->description, GFP_KERNEL);
>>>> > +	if (!pin->prop.description) {
>>>> > +		ret = -ENOMEM;
>>>> > +		goto release;
>>>> > +	}
>>>> > +	if (WARN_ON(prop->type <= DPLL_PIN_TYPE_UNSPEC ||
>>>> > +		    prop->type > DPLL_PIN_TYPE_MAX)) {
>>>> > +		ret = -EINVAL;
>>>> > +		goto release;
>>>> > +	}
>>>> > +	pin->prop.type = prop->type;
>>>> > +	pin->prop.capabilities = prop->capabilities;
>>>> > +	pin->prop.freq_supported = prop->freq_supported;
>>>> > +	pin->prop.any_freq_min = prop->any_freq_min;
>>>> > +	pin->prop.any_freq_max = prop->any_freq_max;
>>>>
>>>> Make sure that the driver maintains prop (static const) and just save
>>>> the pointer. Prop does not need to be something driver needs to change.
>>>>
>>>
>>>What's the difference? For ptp_ocp, we have the same configuration for all
>>>ext pins and the allocator only changes the name of the pin. Properties of
>>>the DPLL pins are stored within the pin object, not the driver, in this
>>case.
>>>Not sure if the pointer way is much better...
>>
>>For things like this it is common to have static const array in the
>>driver, like:
>>
>>static const struct dpll_pin_properties dpll_pin_props[] = {
>>	{
>>		.description = "SMA0",
>>		.freq_supported = DPLL_PIN_FREQ_SUPP_MAX,
>>		.type = DPLL_PIN_TYPE_EXT,
>>		.any_freq_max = 10000000,
>>		.capabilities = DPLL_PIN_CAPS_DIRECTION_CAN_CHANGE,
>>	},
>>	{
>>		.description = "SMA1",
>>		.freq_supported = DPLL_PIN_FREQ_SUPP_MAX,
>>		.type = DPLL_PIN_TYPE_EXT,
>>		.any_freq_max = 10000000,
>>		.capabilities = DPLL_PIN_CAPS_DIRECTION_CAN_CHANGE,
>>	},
>>	{
>>		.description = "SMA2",
>>		.freq_supported = DPLL_PIN_FREQ_SUPP_MAX,
>>		.type = DPLL_PIN_TYPE_EXT,
>>		.any_freq_max = 10000000,
>>		.capabilities = DPLL_PIN_CAPS_DIRECTION_CAN_CHANGE,
>>	},
>>	{
>>		.description = "SMA3",
>>		.freq_supported = DPLL_PIN_FREQ_SUPP_MAX,
>>		.type = DPLL_PIN_TYPE_EXT,
>>		.any_freq_max = 10000000,
>>		.capabilities = DPLL_PIN_CAPS_DIRECTION_CAN_CHANGE,
>>	},
>>};
>>
>>Here you have very nice list of pins, the reader knows right away what
>>is happening.
>>
>>Thinking about "description" name, I think would be more appropriate to
>>name this "label" as it represents user-facing label on the connector,
>>isn't it? Does not describe anything.
>>
>
>"label" seems good.

Ok.


>
>>
>>>
>>>>
>>
>>[...]
>>
>>
>>>
>>>>
>>>> > +	     const struct dpll_pin_properties *prop)
>>>> > +{
>>>> > +	struct dpll_pin *pos, *ret = NULL;
>>>> > +	unsigned long index;
>>>> > +
>>>> > +	mutex_lock(&dpll_pin_xa_lock);
>>>> > +	xa_for_each(&dpll_pin_xa, index, pos) {
>>>> > +		if (pos->clock_id == clock_id &&
>>>> > +		    pos->dev_driver_id == device_drv_id &&
>>>> > +		    pos->module == module) {
>>>>
>>>> Compare prop as well.
>>>>
>>>> Can't the driver_id (pin index) be something const as well? I think it
>>>> should. And therefore it could be easily put inside.
>>>>
>>>
>>>I think clock_id + dev_driver_id + module should identify the pin exactly.
>>>And now I think that *prop is not needed here at all. Arkadiusz, any
>>>thoughts?
>>
>>IDK, no strong opinion on this. I just thought it may help to identify
>>the pin and avoid potential driver bugs. (Like registering 2 pins with
>>the same properties).
>>
>
>It would make most sense if pin_index would be a part of *prop.

Hmm. I see one example where it would not be suitable:
NIC with N physical ports. You, as a driver, register one pin per
physical port. You have one static const prop and pass the pointer to if
for every registration call of N pins, each time with different
pin_index.

So from what I see, the prop describes a flavour of pin in the driver.
In the exaple above, it is still the same SyncE pin, with the same ops.
Only multiple instances of that distinguished by pin_index and priv.

Makes sense?


>
>>[...]
>>
>>
>>>> > +/**
>>>> > + * dpll_pin_register - register the dpll pin in the subsystem
>>>> > + * @dpll: pointer to a dpll
>>>> > + * @pin: pointer to a dpll pin
>>>> > + * @ops: ops for a dpll pin ops
>>>> > + * @priv: pointer to private information of owner
>>>> > + * @rclk_device: pointer to recovered clock device
>>>> > + *
>>>> > + * Return:
>>>> > + * * 0 on success
>>>> > + * * -EINVAL - missing dpll or pin
>>>> > + * * -ENOMEM - failed to allocate memory
>>>> > + */
>>>> > +int
>>>> > +dpll_pin_register(struct dpll_device *dpll, struct dpll_pin *pin,
>>>> > +		  struct dpll_pin_ops *ops, void *priv,
>>>> > +		  struct device *rclk_device)
>>>>
>>>> Wait a second, what is this "struct device *"? Looks very odd.
>>>>
>>>>
>>>> > +{
>>>> > +	const char *rclk_name = rclk_device ? dev_name(rclk_device) :
>>>> >NULL;
>>>>
>>>> If you need to store something here, store the pointer to the device
>>>> directly. But this rclk_device seems odd to me.
>>>> Dev_name is in case of PCI device for example 0000:01:00.0? That alone
>>>> is incomplete. What should it server for?
>>>>
>>>
>>>Well, these questions go to Arkadiusz...
>>
>
>
>[ copy paste my answer from previous response ]

Does not help, see below.


>If pin is able to recover signal from some device this shall convey that
>device struct pointer.

But one device (struct device instance) could easily have multiple
netdev instances attached and therefore mutiple sources of recovered
signal.

mlxsw driver is one of the examples of this 1:N mapping between device
and netdev.


>Name of that device is later passed to the user with DPLL_A_PIN_RCLK_DEVICE
>attribute.
>Sure we can have pointer to device and use dev_name (each do/dump) on netlink
>part. But isn't it better to have the name ready to use there?
>
>It might be incomplete only if one device would have some kind of access to
>a different bus? I don't think it is valid use case.

Very easily, auxiliary_bus. That is actually the case already for mlx5.


>
>Basically the driver will refer only to the devices handled by that driver,
>which means if dpll is on some bus, also all the pins are there, didn't notice
>the need to have bus here as well.

What is the motivation exactly? Is it only SyncE? In case it is, the
link to the pin should be added from the other side, carried over
RTNetlink msg of a netdev associated with pin. I will add a patch for
this.

Do you need to expose any other recovered clock device source?


>
>>Okay.
>>
>>[...]
>>
>>
>>>> > + * dpll_pin_get_by_idx - find a pin ref on dpll by pin index
>>>> > + * @dpll: dpll device pointer
>>>> > + * @idx: index of pin
>>>> > + *
>>>> > + * Find a reference to a pin registered with given dpll and return
>>>> > its pointer.
>>>> > + *
>>>> > + * Return:
>>>> > + * * valid pointer if pin was found
>>>> > + * * NULL if not found
>>>> > + */
>>>> > +struct dpll_pin *dpll_pin_get_by_idx(struct dpll_device *dpll, u32 idx)
>>>> > +{
>>>> > +	struct dpll_pin_ref *pos;
>>>> > +	unsigned long i;
>>>> > +
>>>> > +	xa_for_each(&dpll->pin_refs, i, pos) {
>>>> > +		if (pos && pos->pin && pos->pin->dev_driver_id == idx)
>>>>
>>>> How exactly pos->pin could be NULL?
>>>>
>>>> Also, you are degrading the xarray to a mere list here with lookup like
>>>> this. Why can't you use the pin index coming from driver and
>>>> insert/lookup based on this index?
>>>>
>>>Good point. We just have to be sure, that drivers provide 0-based indexes for
>>>their pins. I'll re-think it.
>>
>>No, driver can provide indexing which is completely up to his decision.
>>You should use xa_insert() to insert the entry at specific index. See
>>devl_port_register for inspiration where it is done exactly like this.
>>
>>And this should be done in exactly the same way for both pin and device.
>>
>
>Yes, I agree seems doable and better then it is now.
>
>[...]
>
>>>> > +static int
>>>> > +dpll_msg_add_dev_handle(struct sk_buff *msg, const struct dpll_device
>>>> >*dpll)
>>>> > +{
>>>> > +	if (nla_put_u32(msg, DPLL_A_ID, dpll->id))
>>>>
>>>> Why exactly do we need this dua--handle scheme? Why do you need
>>>> unpredictable DPLL_A_ID to be exposed to userspace?
>>>> It's just confusing.
>>>>
>>>To be able to work with DPLL per integer after iterator on the list deducts
>>>which DPLL device is needed. It can reduce the amount of memory copies and
>>>simplify comparisons. Not sure why it's confusing.
>>
>>Wait, I don't get it. Could you please explain a bit more?
>>
>>My point is, there should be not such ID exposed over netlink
>>You don't need to expose it to userspace. The user has well defined
>>handle as you agreed with above. For example:
>>
>>ice/c92d02a7129f4747/1
>>ice/c92d02a7129f4747/2
>>
>>This is shown in dpll device GET/DUMP outputs.
>>Also user passes it during SET operation:
>>$ dplltool set ice/c92d02a7129f4747/1 mode auto
>>
>>Isn't that enough stable and nice?
>>
>
>I agree with Vadim, this is rather to be used by a daemon tools, which
>would get the index once, then could use it as long as device is there.

So basically you say, you can have 2 approaches in app:
1)
id = dpll_device_get_id("ice/c92d02a7129f4747/1")
dpll_device_set(id, something);
dpll_device_set(id, something);
dpll_device_set(id, something);
2):
dpll_device_set("ice/c92d02a7129f4747/1, something);
dpll_device_set("ice/c92d02a7129f4747/1, something);
dpll_device_set("ice/c92d02a7129f4747/1, something);

What is exactly benefit of the first one? Why to have 2 handles? Devlink
is a nice example of 2) approach, no problem there.

Perhaps I'm missing something, but looks like you want the id for no
good reason and this dual-handle scheme just makes things more
complicated with 0 added value.


>
>>
>>>
>>>>
>>>> > +		return -EMSGSIZE;
>>>> > +	if (nla_put_string(msg, DPLL_A_BUS_NAME, dev_bus_name(&dpll-
>>>> > dev)))
>>>> > +		return -EMSGSIZE;
>>>> > +	if (nla_put_string(msg, DPLL_A_DEV_NAME, dev_name(&dpll->dev)))
>>>> > +		return -EMSGSIZE;
>>>> > +
>>>> > +	return 0;
>>>> > +}
>>>
>>>[...]
>>>
>>>> > +
>>>> > +static int
>>>> > +dpll_msg_add_pins_on_pin(struct sk_buff *msg, struct dpll_pin *pin,
>>>> > +			 struct netlink_ext_ack *extack)
>>>> > +{
>>>> > +	struct dpll_pin_ref *ref = NULL;
>>>>
>>>> Why this needs to be initialized?
>>>>
>>>No need, fixed.
>>>
>>>
>>>>
>>>> > +	enum dpll_pin_state state;
>>>> > +	struct nlattr *nest;
>>>> > +	unsigned long index;
>>>> > +	int ret;
>>>> > +
>>>> > +	xa_for_each(&pin->parent_refs, index, ref) {
>>>> > +		if (WARN_ON(!ref->ops->state_on_pin_get))
>>>> > +			return -EFAULT;
>>>> > +		ret = ref->ops->state_on_pin_get(pin, ref->pin, &state,
>>>> > +						 extack);
>>>> > +		if (ret)
>>>> > +			return -EFAULT;
>>>> > +		nest = nla_nest_start(msg, DPLL_A_PIN_PARENT);
>>>> > +		if (!nest)
>>>> > +			return -EMSGSIZE;
>>>> > +		if (nla_put_u32(msg, DPLL_A_PIN_PARENT_IDX,
>>>> > +				ref->pin->dev_driver_id)) {
>>>> > +			ret = -EMSGSIZE;
>>>> > +			goto nest_cancel;
>>>> > +		}
>>>> > +		if (nla_put_u8(msg, DPLL_A_PIN_STATE, state)) {
>>>> > +			ret = -EMSGSIZE;
>>>> > +			goto nest_cancel;
>>>> > +		}
>>>> > +		nla_nest_end(msg, nest);
>>>> > +	}
>>>>
>>>> How is this function different to dpll_msg_add_pin_parents()?
>>>> Am I lost? To be honest, this x_on_pin/dpll, parent, refs dance is quite
>>>> hard to follow for me :/
>>>>
>>>> Did you get lost here as well? If yes, this needs some serious think
>>>> through :)
>>>>
>>>
>>>Let's re-think it again. Arkadiuzs, do you have clear explanation of the
>>>relationship between these things?
>>
>
>[ copy paste my answer from previous response ]
>No, it is just leftover I didn't catch, we can leave one function and use it in
>both cases. Sorry about that, great catch!

Hmm, ok. I'm still lost a bit with all the referencing and cross
referencing, I wonder if it could be made a bit simpler and/or easier to
read and follow.


>
>[...]
>
>>>> > +	if (nla_put_u32(msg, DPLL_A_PIN_IDX, pin->dev_driver_id))
>>>>
>>>> Is it "ID" or "INDEX" (IDX). Please make this consistent in the whole
>>>> code.
>>>>
>>>
>>>I believe it's INDEX which is provided by the driver. I'll think about
>>>renaming,
>>>but suggestions are welcome.
>>
>>Let's use "index" and "INDEX" internalla and in Netlink attr names as
>>well then.
>>
>
>For me makes sense to have a common name instead of origin-based one.

What do you mean by this?


>
>>[...]
>>
>>
>>>
>>>>
>>>> > +	int rem, ret = -EINVAL;
>>>> > +	struct nlattr *a;
>>>> > +
>>>> > +	nla_for_each_attr(a, genlmsg_data(info->genlhdr),
>>>> > +			  genlmsg_len(info->genlhdr), rem) {
>>>>
>>>> This is odd. Why you iterace over attrs? Why don't you just access them
>>>> directly, like attrs[DPLL_A_PIN_FREQUENCY] for example?
>>>>
>>>
>>>I had some unknown crashes when I was using such access. I might have lost
>>>some checks, will try it again.
>>
>>Odd, yet definitelly debuggable though :)
>>
>>[...]
>>
>>
>>>> > +int dpll_pin_pre_dumpit(struct netlink_callback *cb)
>>>> > +{
>>>> > +	mutex_lock(&dpll_pin_xa_lock);
>>>>
>>>> ABBA deadlock here, see dpll_pin_register() for example where the lock
>>>> taking order is opposite.
>>>>
>>>
>>>Now I see an ABBA deadlock here, as well as in function before. Not sure
>>>how to
>>>solve it here. Any thoughts?
>>
>>Well, here you can just call dpll_pre_dumpit() before
>>mutex_lock(&dpll_pin_xa_lock)
>>to take the locks in the same order.
>>
>
>This double lock doesn't really improve anything.
>Any objections on having single mutex/lock for access the dpll subsystem?

Yeah, we had it in devlink and then we spent a lot of a time to get rid
of it. Now we have per-instance locking. Here, it would be
per-dpll-device locking.

No strong opinion, perhaps on case of dpll one master lock is enough.
Perhaps Jakub would have some opinion on this.


>
>>
>>>
>>>>
>>>> > +
>>>> > +	return dpll_pre_dumpit(cb);
>>>> > +}
>>>> > +
>>>> > +int dpll_pin_post_dumpit(struct netlink_callback *cb)
>>>> > +{
>>>> > +	mutex_unlock(&dpll_pin_xa_lock);
>>>> > +
>>>> > +	return dpll_post_dumpit(cb);
>>>> > +}
>>>> > +
>>>> > +static int
>>>> > +dpll_event_device_change(struct sk_buff *msg, struct dpll_device
>>>> > *dpll,
>>>> > +			 struct dpll_pin *pin, struct dpll_pin *parent,
>>>> > +			 enum dplla attr)
>>>> > +{
>>>> > +	int ret = dpll_msg_add_dev_handle(msg, dpll);
>>>> > +	struct dpll_pin_ref *ref = NULL;
>>>> > +	enum dpll_pin_state state;
>>>> > +
>>>> > +	if (ret)
>>>> > +		return ret;
>>>> > +	if (pin && nla_put_u32(msg, DPLL_A_PIN_IDX, pin-
>>>> > dev_driver_id))
>>>> > +		return -EMSGSIZE;
>>>>
>>>> I don't really understand why you are trying figure something new and
>>>> interesting with the change notifications. This object mix and random
>>>> attrs fillup is something very wrong and makes userspace completely
>>>> fuzzy about what it is getting. And yet it is so simple:
>>>> You have 2 objects, dpll and pin, please just have:
>>>> dpll_notify()
>>>> dpll_pin_notify()
>>>> and share the attrs fillup code with pin_get() and dpll_get() callbacks.
>>>> No need for any smartness. Have this dumb and simple.
>>>>
>>>> Think about it more as about "object-state-snapshot" than "atomic-change"
>>>
>>>But with full object-snapshot user space app will lose the information about
>>>what exactly has changed. The reason to have this event is to provide the
>>>attributes which have changed. Otherwise, the app should have full snapshot
>>>and
>>>compare all attributes to figure out changes and that's might not be great
>>>idea.
>>
>>Wait, are you saying that the app is stateless? Could you provide
>>example use cases?
>>
>>From what I see, the app managing dpll knows the state of the device and
>>pins, it monitors for the changes and saves new state with appropriate
>>reaction (might be some action or maybe just log entry).
>>
>
>It depends on the use case, right? App developer having those information knows
>what has changed, thus can react in a way it thinks is most suitable.
>IMHO from user perspective it is good to have a notification which actually
>shows it's reason, so proper flow could be assigned to handle the reaction.

Again, could you provide me specific example in which this is needed?
I may be missing something, but I don't see how it can bring
and benefit. It just makes the live of the app harder because it has to
treat the get and notify messages differently.

It is quite common for app to:
init:
1) get object state
2) store it
3) apply configuration
runtime:
1) listen to object state change
2) store it
3) apply configuration

Same code for both.




>
>>
>>>
>>>>
>>>> > +
>>>> > +	switch (attr) {
>>>> > +	case DPLL_A_MODE:
>>>> > +		ret = dpll_msg_add_mode(msg, dpll, NULL);
>>>> > +		break;
>>>> > +	case DPLL_A_SOURCE_PIN_IDX:
>>>> > +		ret = dpll_msg_add_source_pin_idx(msg, dpll, NULL);
>>>> > +		break;
>>>> > +	case DPLL_A_LOCK_STATUS:
>>>> > +		ret = dpll_msg_add_lock_status(msg, dpll, NULL);
>>>> > +		break;
>>>> > +	case DPLL_A_TEMP:
>>>> > +		ret = dpll_msg_add_temp(msg, dpll, NULL);
>>>> > +		break;
>>>> > +	case DPLL_A_PIN_FREQUENCY:
>>>> > +		ret = dpll_msg_add_pin_freq(msg, pin, NULL, false);
>>>> > +		break;
>>>> > +	case DPLL_A_PIN_PRIO:
>>>> > +		ref = dpll_xa_ref_dpll_find(&pin->dpll_refs, dpll);
>>>> > +		if (!ref)
>>>> > +			return -EFAULT;
>>>> > +		ret = dpll_msg_add_pin_prio(msg, pin, ref, NULL);
>>>> > +		break;
>>>> > +	case DPLL_A_PIN_STATE:
>>>> > +		if (parent) {
>>>> > +			ref = dpll_xa_ref_pin_find(&pin->parent_refs, parent);
>>>> > +			if (!ref)
>>>> > +				return -EFAULT;
>>>> > +			if (!ref->ops || !ref->ops->state_on_pin_get)
>>>> > +				return -EOPNOTSUPP;
>>>> > +			ret = ref->ops->state_on_pin_get(pin, parent, &state,
>>>> > +							 NULL);
>>>> > +			if (ret)
>>>> > +				return ret;
>>>> > +			if (nla_put_u32(msg, DPLL_A_PIN_PARENT_IDX,
>>>> > +					parent->dev_driver_id))
>>>> > +				return -EMSGSIZE;
>>>> > +		} else {
>>>> > +			ref = dpll_xa_ref_dpll_find(&pin->dpll_refs, dpll);
>>>> > +			if (!ref)
>>>> > +				return -EFAULT;
>>>> > +			ret = dpll_msg_add_pin_on_dpll_state(msg, pin, ref,
>>>> > +							     NULL);
>>>> > +			if (ret)
>>>> > +				return ret;
>>>> > +		}
>>>> > +		break;
>>>> > +	default:
>>>> > +		break;
>>>> > +	}
>>>> > +
>>>> > +	return ret;
>>>> > +}
>>>> > +
>>>> > +static int
>>>> > +dpll_send_event_create(enum dpll_event event, struct dpll_device *dpll)
>>>> > +{
>>>> > +	struct sk_buff *msg;
>>>> > +	int ret = -EMSGSIZE;
>>>> > +	void *hdr;
>>>> > +
>>>> > +	msg = genlmsg_new(NLMSG_GOODSIZE, GFP_KERNEL);
>>>> > +	if (!msg)
>>>> > +		return -ENOMEM;
>>>> > +
>>>> > +	hdr = genlmsg_put(msg, 0, 0, &dpll_nl_family, 0, event);
>>>> > +	if (!hdr)
>>>> > +		goto out_free_msg;
>>>> > +
>>>> > +	ret = dpll_msg_add_dev_handle(msg, dpll);
>>>> > +	if (ret)
>>>> > +		goto out_cancel_msg;
>>>> > +	genlmsg_end(msg, hdr);
>>>> > +	genlmsg_multicast(&dpll_nl_family, msg, 0, 0, GFP_KERNEL);
>>>> > +
>>>> > +	return 0;
>>>> > +
>>>> > +out_cancel_msg:
>>>> > +	genlmsg_cancel(msg, hdr);
>>>> > +out_free_msg:
>>>> > +	nlmsg_free(msg);
>>>> > +
>>>> > +	return ret;
>>>> > +}
>>>> > +
>>>> > +static int
>>>> > +dpll_send_event_change(struct dpll_device *dpll, struct dpll_pin *pin,
>>>> > +		       struct dpll_pin *parent, enum dplla attr)
>>>> > +{
>>>> > +	struct sk_buff *msg;
>>>> > +	int ret = -EMSGSIZE;
>>>> > +	void *hdr;
>>>> > +
>>>> > +	msg = genlmsg_new(NLMSG_GOODSIZE, GFP_KERNEL);
>>>> > +	if (!msg)
>>>> > +		return -ENOMEM;
>>>> > +
>>>> > +	hdr = genlmsg_put(msg, 0, 0, &dpll_nl_family, 0,
>>>> > +			  DPLL_EVENT_DEVICE_CHANGE);
>>>>
>>>> I don't really get it. Why exactly you keep having this *EVENT* cmds?
>>>> Why per-object NEW/GET/DEL cmds shared with get genl op are not enough?
>>>> I have to be missing something.
>>>
>>>Changes might come from other places, but will affect the DPLL device and we
>>>have to notify users in this case.
>>
>>I'm not sure I follow. There are 2 scenarios for change:
>>1) user originated - user issues set of something
>>2) driver originated - something changes in HW, driver propagates that
>>
>>With what I suggest, both scenarios work of course. My point is, user
>>knows very well the objects: device and pin, he knows the format or
>>messages that are related to GET/DUMP/SET operations of both. The
>>notification should have the same format, that is all I say.
>>
>>Btw, you can see devlink code for example how the notifications like
>>this are implemented and work.
>>
>
>Devlink packs all the info into a netlink message and notifies with it, isn't
>it that it has all the info "buffered" in its structures?

Not really. Ops are there as well.


>A dpll subsystem keeps only some of the info in its internal structures, but
>for most of it it has to question the driver. It would do it multiple times
>i.e. if there would be a change on a pin connected to multiple dpll devices.
>
>With current approach the notifications are pretty light to the subsystem as
>well as for the registered clients, and as you suggested with having info of
>what actually changed the userspace could implement a stateless daemon right
>away.

Okay, examples? I really can't see how this stateless deamon could be
implemented, but perhaps I lack better imagination :/


>
>Thank you,
>Arkadiusz

Jiri Pirko March 15, 2023, 12:14 p.m. UTC | #8

Tue, Mar 14, 2023 at 05:43:18PM CET, [e-mail] wrote:
>>From: Vadim Fedorenko <[e-mail]>
>>Sent: Tuesday, March 14, 2023 12:00 AM
>>
>
>[...]
>
>>>> +
>>>> +/**
>>>> + * dpll_device_get - find existing or create new dpll device
>>>> + * @clock_id: clock_id of creator
>>>> + * @dev_driver_id: id given by dev driver
>>>> + * @module: reference to registering module
>>>> + *
>>>> + * Get existing object of a dpll device, unique for given arguments.
>>>> + * Create new if doesn't exist yet.
>>>> + *
>>>> + * Return:
>>>> + * * valid dpll_device struct pointer if succeeded
>>>> + * * ERR_PTR of an error
>>>> + */
>>>> +struct dpll_device *
>>>> +dpll_device_get(u64 clock_id, u32 dev_driver_id, struct module *module)
>>>> +{
>>>> +	struct dpll_device *dpll, *ret = NULL;
>>>> +	unsigned long index;
>>>> +
>>>> +	mutex_lock(&dpll_device_xa_lock);
>>>> +	xa_for_each(&dpll_device_xa, index, dpll) {
>>>> +		if (dpll->clock_id == clock_id &&
>>>> +		    dpll->dev_driver_id == dev_driver_id &&
>>>
>>> Why you need "dev_driver_id"? clock_id is here for the purpose of
>>> identification, isn't that enough for you.
>>
>>dev_driver_id is needed to provide several DPLLs from one device. In ice
>>driver
>>implementation there are 2 different DPLLs - to recover from PPS input and
>>to
>>recover from Sync-E. I believe there is only one clock, that's why clock id
>>is
>>the same for both of them. But Arkadiusz can tell more about it.
>
>Yes, exactly.
>One driver can have multiple dplls with the same clock id.
>Actually dev_driver_id makes dpll objects unique.
>
>>>
>>> Plus, the name is odd. "dev_driver" should certainly be avoided.
>>
>>Simply id doesn't tell anything either. dpll_dev_id?
>
>Looks good to me.

Let's call this "device_index" and "pin_index" for the pin getter as I
suggested in the other email.


>
>>
>>>> +		    dpll->module == module) {
>>>> +			ret = dpll;
>>>> +			refcount_inc(&ret->refcount);
>>>> +			break;
>>>> +		}
>>>> +	}
>>>> +	if (!ret)
>>>> +		ret = dpll_device_alloc(clock_id, dev_driver_id, module);
>>>> +	mutex_unlock(&dpll_device_xa_lock);
>>>> +
>>>> +	return ret;
>>>> +}
>>>> +EXPORT_SYMBOL_GPL(dpll_device_get);
>>>> +
>>>> +/**
>>>> + * dpll_device_put - decrease the refcount and free memory if possible
>>>> + * @dpll: dpll_device struct pointer
>>>> + *
>>>> + * Drop reference for a dpll device, if all references are gone, delete
>>>> + * dpll device object.
>>>> + */
>>>> +void dpll_device_put(struct dpll_device *dpll)
>>>> +{
>>>> +	if (!dpll)
>>>> +		return;
>>>
>>> Remove this check. The driver should not call this with NULL.
>>
>>Well, netdev_put() has this kind of check. As well as spi_dev_put() or
>>i2c_put_adapter() at least. Not sure I would like to avoid a bit of safety.
>>
>
>I agree, IMHO it is better to have safety checks :)

IDK, we should rely on basic driver sanity. Let least put a WARN_ON() to
these checks. But try to reduce them at least.



>
>>>> +	mutex_lock(&dpll_device_xa_lock);
>>>> +	if (refcount_dec_and_test(&dpll->refcount)) {
>>>> +		WARN_ON_ONCE(!xa_empty(&dpll->pin_refs));
>>>
>>> ASSERT_DPLL_NOT_REGISTERED(dpll);
>>
>>Good point!
>>
>
>Yes, great point!
>
>>>> +		xa_destroy(&dpll->pin_refs);
>>>> +		xa_erase(&dpll_device_xa, dpll->id);
>>>> +		kfree(dpll);
>>>> +	}
>>>> +	mutex_unlock(&dpll_device_xa_lock);
>>>> +}
>>>> +EXPORT_SYMBOL_GPL(dpll_device_put);
>>>> +
>>>> +/**
>>>> + * dpll_device_register - register the dpll device in the subsystem
>>>> + * @dpll: pointer to a dpll
>>>> + * @type: type of a dpll
>>>> + * @ops: ops for a dpll device
>>>> + * @priv: pointer to private information of owner
>>>> + * @owner: pointer to owner device
>>>> + *
>>>> + * Make dpll device available for user space.
>>>> + *
>>>> + * Return:
>>>> + * * 0 on success
>>>> + * * -EINVAL on failure
>>>> + */
>>>> +int dpll_device_register(struct dpll_device *dpll, enum dpll_type type,
>>>> +			 struct dpll_device_ops *ops, void *priv,
>>>> +			 struct device *owner)
>>>> +{
>>>> +	if (WARN_ON(!ops || !owner))
>>>> +		return -EINVAL;
>>>> +	if (WARN_ON(type <= DPLL_TYPE_UNSPEC || type > DPLL_TYPE_MAX))
>>>> +		return -EINVAL;
>>>> +	mutex_lock(&dpll_device_xa_lock);
>>>> +	if (ASSERT_DPLL_NOT_REGISTERED(dpll)) {
>>>> +		mutex_unlock(&dpll_device_xa_lock);
>>>> +		return -EEXIST;
>>>> +	}
>>>> +	dpll->dev.bus = owner->bus;
>>>> +	dpll->parent = owner;
>>>> +	dpll->type = type;
>>>> +	dpll->ops = ops;
>>>> +	dev_set_name(&dpll->dev, "%s_%d", dev_name(owner),
>>>> +		     dpll->dev_driver_id);
>>>
>>> This is really odd. As a result, the user would see something like:
>>> pci/0000:01:00.0_1
>>> pci/0000:01:00.0_2
>>>
>>> I have to say it is confusing. In devlink, is bus/name and the user
>>> could use this info to look trough sysfs. Here, 0000:01:00.0_1 is not
>>> there. Also, "_" might have some meaning on some bus. Should not
>>> concatename dev_name() with anything.
>>>
>>> Thinking about this some more, the module/clock_id tuple should be
>>> uniqueue and stable. It is used for dpll_device_get(), it could be used
>>> as the user handle, can't it?
>>> Example:
>>> ice/c92d02a7129f4747
>>> mlx5/90265d8bf6e6df56
>>>
>>> If you really need the "dev_driver_id" (as I believe clock_id should be
>>> enough), you can put it here as well:
>>> ice/c92d02a7129f4747/1
>>> ice/c92d02a7129f4747/2
>>>
>>
>>Looks good, will change it
>
>Makes sense to me.

Good. Fits the mlx5 model nicely.


>
>>
>>> This would also be beneficial for mlx5, as mlx5 with 2 PFs would like to
>>> share instance of DPLL equally, there is no "one clock master". >
>>>> +	dpll->priv = priv;
>>>> +	xa_set_mark(&dpll_device_xa, dpll->id, DPLL_REGISTERED);
>>>> +	mutex_unlock(&dpll_device_xa_lock);
>>>> +	dpll_notify_device_create(dpll);
>>>> +
>>>> +	return 0;
>>>> +}
>>>> +EXPORT_SYMBOL_GPL(dpll_device_register);
>>>> +
>>>> +/**
>>>> + * dpll_device_unregister - deregister dpll device
>>>> + * @dpll: registered dpll pointer
>>>> + *
>>>> + * Deregister device, make it unavailable for userspace.
>>>> + * Note: It does not free the memory
>>>> + */
>>>> +void dpll_device_unregister(struct dpll_device *dpll)
>>>> +{
>>>> +	mutex_lock(&dpll_device_xa_lock);
>>>> +	ASSERT_DPLL_REGISTERED(dpll);
>>>> +	xa_clear_mark(&dpll_device_xa, dpll->id, DPLL_REGISTERED);
>>>> +	mutex_unlock(&dpll_device_xa_lock);
>>>> +	dpll_notify_device_delete(dpll);
>>>> +}
>>>> +EXPORT_SYMBOL_GPL(dpll_device_unregister);
>>>> +
>>>> +/**
>>>> + * dpll_pin_alloc - allocate the memory for dpll pin
>>>> + * @clock_id: clock_id of creator
>>>> + * @dev_driver_id: id given by dev driver
>>>> + * @module: reference to registering module
>>>> + * @prop: dpll pin properties
>>>> + *
>>>> + * Return:
>>>> + * * valid allocated dpll_pin struct pointer if succeeded
>>>> + * * ERR_PTR of an error
>>>
>>> Extra "*"'s
>>
>>Ok, I can re-format the comments across the files.
>
>This is expected on enumerating return values in kernel-docs:
>https://www.kernel.org/doc/html/latest/doc-guide/kernel-doc.html#Return%20values

Ah, okay. I missed that. Sorry.


>
>>
>>>> + */
>>>> +struct dpll_pin *
>>>> +dpll_pin_alloc(u64 clock_id, u8 device_drv_id,	struct module
>>*module,
>>>
>>> Odd whitespace.
>>>
>>> Also, func should be static.
>>>
>>
>>Fixed.
>>
>>>
>>>> +	       const struct dpll_pin_properties *prop)
>>>> +{
>>>> +	struct dpll_pin *pin;
>>>> +	int ret;
>>>> +
>>>> +	pin = kzalloc(sizeof(*pin), GFP_KERNEL);
>>>> +	if (!pin)
>>>> +		return ERR_PTR(-ENOMEM);
>>>> +	pin->dev_driver_id = device_drv_id;
>>>
>>> Name inconsistency: driver/drv
>>> you have it on multiple places
>>>
>>
>>Changed it every where, thanks for spotting.
>>
>>>
>>>> +	pin->clock_id = clock_id;
>>>> +	pin->module = module;
>>>> +	refcount_set(&pin->refcount, 1);
>>>> +	if (WARN_ON(!prop->description)) {
>>>> +		ret = -EINVAL;
>>>> +		goto release;
>>>> +	}
>>>> +	pin->prop.description = kstrdup(prop->description, GFP_KERNEL);
>>>> +	if (!pin->prop.description) {
>>>> +		ret = -ENOMEM;
>>>> +		goto release;
>>>> +	}
>>>> +	if (WARN_ON(prop->type <= DPLL_PIN_TYPE_UNSPEC ||
>>>> +		    prop->type > DPLL_PIN_TYPE_MAX)) {
>>>> +		ret = -EINVAL;
>>>> +		goto release;
>>>> +	}
>>>> +	pin->prop.type = prop->type;
>>>> +	pin->prop.capabilities = prop->capabilities;
>>>> +	pin->prop.freq_supported = prop->freq_supported;
>>>> +	pin->prop.any_freq_min = prop->any_freq_min;
>>>> +	pin->prop.any_freq_max = prop->any_freq_max;
>>>
>>> Make sure that the driver maintains prop (static const) and just save
>>> the pointer. Prop does not need to be something driver needs to change.
>>>
>>
>>What's the difference? For ptp_ocp, we have the same configuration for all
>>ext pins and the allocator only changes the name of the pin. Properties of
>>the DPLL pins are stored within the pin object, not the driver, in this
>>case.
>>Not sure if the pointer way is much better...
>>
>
>I also don't feel it.
>Dpll subsystem directly using memory of different driver doesn't look like a
>great design.

Wait. This is done all the time in kernel. Almost every ops for example.
Lots of other examples like that as well:
git grep "static const struct" drivers/net/


>
>>>
>>>> +	xa_init_flags(&pin->dpll_refs, XA_FLAGS_ALLOC);
>>>> +	xa_init_flags(&pin->parent_refs, XA_FLAGS_ALLOC);
>>>> +	ret = xa_alloc(&dpll_pin_xa, &pin->idx, pin,
>>>> +		       xa_limit_16b, GFP_KERNEL);
>>>> +release:
>>>> +	if (ret) {
>>>> +		xa_destroy(&pin->dpll_refs);
>>>> +		xa_destroy(&pin->parent_refs);
>>>> +		kfree(pin->prop.description);
>>>> +		kfree(pin->rclk_dev_name);
>>>> +		kfree(pin);
>>>> +		return ERR_PTR(ret);
>>>> +	}
>>>> +
>>>> +	return pin;
>>>> +}
>>>> +
>>>> +/**
>>>> + * dpll_pin_get - find existing or create new dpll pin
>>>> + * @clock_id: clock_id of creator
>>>> + * @dev_driver_id: id given by dev driver
>>>> + * @module: reference to registering module
>>>> + * @prop: dpll pin properties
>>>> + *
>>>> + * Get existing object of a pin (unique for given arguments) or create
>>>>new
>>>> + * if doesn't exist yet.
>>>> + *
>>>> + * Return:
>>>> + * * valid allocated dpll_pin struct pointer if succeeded
>>>> + * * ERR_PTR of an error
>>>
>>> This is one example, I'm pretty sure that there are others, when you
>>> have text inconsistencies in func doc for the same function in .c and .h
>>> Have it please only on one place. .c is the usual place.
>>>
>>
>>Yep, will clear .h files.
>>
>
>There might be some issues, and certainly makes sense to have them in one
>place.
>
>>>
>>>> + */
>>>> +struct dpll_pin *
>>>> +dpll_pin_get(u64 clock_id, u32 device_drv_id, struct module *module,
>>>
>>> Again, why do you need this device_drv_id? Clock id should be enough.
>>>
>>I explained the reason earlier, but the naming is fixed.
>
>Yes, this device_drv_id is id of a pin not a dpll device id.
>Maybe worth to rename it to make it more clear.

Yeah, device_index and pin_index


>
>>
>>>
>>>> +	     const struct dpll_pin_properties *prop)
>>>> +{
>>>> +	struct dpll_pin *pos, *ret = NULL;
>>>> +	unsigned long index;
>>>> +
>>>> +	mutex_lock(&dpll_pin_xa_lock);
>>>> +	xa_for_each(&dpll_pin_xa, index, pos) {
>>>> +		if (pos->clock_id == clock_id &&
>>>> +		    pos->dev_driver_id == device_drv_id &&
>>>> +		    pos->module == module) {
>>>
>>> Compare prop as well.
>>>
>>> Can't the driver_id (pin index) be something const as well? I think it
>>> should. And therefore it could be easily put inside.
>>>
>>
>>I think clock_id + dev_driver_id + module should identify the pin exactly.
>
>Yes, they should be unique for a single pin object, and enough for getting
>pin reference.
>Basically if driver would call twice this with different props, it would be
>broken.
>We could have props compared here as well.

Let's leave it as it is for now, it's ok.


>
>>And
>>now I think that *prop is not needed here at all. Arkadiusz, any thoughts?
>
>As dpll_pin_alloc is having them assigned to a pin object, thus had to put it
>here. Not sure how to solve it differently.
>Jiri's suggestion to put dev_driver_id inside of pin props would also work
>AFAIU.

Let's have it as device_get()/pin_get() args as you have it right now.
Makes more sense as the prop could be static const as I described in the
other email.


>
>[...]
>
>>>> +
>>>> +/**
>>>> + * dpll_pin_register - register the dpll pin in the subsystem
>>>> + * @dpll: pointer to a dpll
>>>> + * @pin: pointer to a dpll pin
>>>> + * @ops: ops for a dpll pin ops
>>>> + * @priv: pointer to private information of owner
>>>> + * @rclk_device: pointer to recovered clock device
>>>> + *
>>>> + * Return:
>>>> + * * 0 on success
>>>> + * * -EINVAL - missing dpll or pin
>>>> + * * -ENOMEM - failed to allocate memory
>>>> + */
>>>> +int
>>>> +dpll_pin_register(struct dpll_device *dpll, struct dpll_pin *pin,
>>>> +		  struct dpll_pin_ops *ops, void *priv,
>>>> +		  struct device *rclk_device)
>>>
>>> Wait a second, what is this "struct device *"? Looks very odd.
>>>
>>>
>>>> +{
>>>> +	const char *rclk_name = rclk_device ? dev_name(rclk_device) : NULL;
>>>
>>> If you need to store something here, store the pointer to the device
>>> directly. But this rclk_device seems odd to me.
>>> Dev_name is in case of PCI device for example 0000:01:00.0? That alone
>>> is incomplete. What should it server for?
>>>
>>
>>Well, these questions go to Arkadiusz...
>>
>
>If pin is able to recover signal from some device this shall convey that
>device struct pointer.
>Name of that device is later passed to the user with DPLL_A_PIN_RCLK_DEVICE
>attribute.
>Sure we can have pointer to device and use dev_name (each do/dump) on netlink
>part. But isn't it better to have the name ready to use there?
>
>It might be incomplete only if one device would have some kind of access to
>a different bus? I don't think it is valid use case.

Very valid as I explained in the other email.


>
>Basically the driver will refer only to the devices handled by that driver,
>which means if dpll is on some bus, also all the pins are there, didn't notice
>the need to have bus here as well.
>
>[...]
>
>>>> +struct dpll_pin *dpll_pin_get_by_idx(struct dpll_device *dpll, u32 idx)
>>>> +{
>>>> +	struct dpll_pin_ref *pos;
>>>> +	unsigned long i;
>>>> +
>>>> +	xa_for_each(&dpll->pin_refs, i, pos) {
>>>> +		if (pos && pos->pin && pos->pin->dev_driver_id == idx)
>>>
>>> How exactly pos->pin could be NULL?
>>>
>
>I believe if proper synchronization in place it shall not be NULL, I left it
>after fixing some issue with access to the pin that was already removed..

Then don't check it here.


>
>>> Also, you are degrading the xarray to a mere list here with lookup like
>>> this. Why can't you use the pin index coming from driver and
>>> insert/lookup based on this index?
>>>
>>Good point. We just have to be sure, that drivers provide 0-based indexes
>>for their pins. I'll re-think it.
>>
>
>After quick thinking, it might be doable, storing pin being registered on
>dpll->pin_refs under given by driver pin's dev_driver_idx or whatever it would
>be named.

Yep.


>
>>
>>>
>>>> +			return pos->pin;
>>>> +	}
>>>> +
>>>> +	return NULL;
>>>> +}
>>>> +
>>>> +/**
>>>> + * dpll_priv - get the dpll device private owner data
>>>> + * @dpll:	registered dpll pointer
>>>> + *
>>>> + * Return: pointer to the data
>>>> + */
>>>> +void *dpll_priv(const struct dpll_device *dpll)
>>>> +{
>>>> +	return dpll->priv;
>>>> +}
>>>> +EXPORT_SYMBOL_GPL(dpll_priv);
>>>> +
>>>> +/**
>>>> + * dpll_pin_on_dpll_priv - get the dpll device private owner data
>>>> + * @dpll:	registered dpll pointer
>>>> + * @pin:	pointer to a pin
>>>> + *
>>>> + * Return: pointer to the data
>>>> + */
>>>> +void *dpll_pin_on_dpll_priv(const struct dpll_device *dpll,
>>>
>>> IIUC, you use this helper from dpll ops in drivers to get per dpll priv.
>>> Just pass the priv directly to the op and avoid need for this helper,
>>> no? Same goes to the rest of the priv helpers.
>>>
>
>No strong opinion, probably doable.

It is better for the driver writer to actually see right away that
there is a priv so he can use it and not need to get the priv from
some place else (using pin index * lookup). Very nice example where
this would work is the last patch of this set, where Vadim does not
use priv at all.


>
>>>
>>>> +			    const struct dpll_pin *pin)
>>>> +{
>>>> +	struct dpll_pin_ref *ref;
>>>> +
>>>> +	ref = dpll_xa_ref_pin_find((struct xarray *)&dpll->pin_refs, pin);
>>>
>>> Why cast is needed here? You have this on multiple places.
>>>
>
>`const` of struct dpll_pin *pin makes a warning/error there, there is something
>broken on xarray implementation of for_each I believe.

Either fix it or avoid using const arg. Cast's like this always smell
and should be avoided.


>
>[...]
>
>>>> +
>>>> +static int
>>>> +dpll_msg_add_pins_on_pin(struct sk_buff *msg, struct dpll_pin *pin,
>>>> +			 struct netlink_ext_ack *extack)
>>>> +{
>>>> +	struct dpll_pin_ref *ref = NULL;
>>>
>>> Why this needs to be initialized?
>>>
>>No need, fixed.
>>
>>
>>>
>>>> +	enum dpll_pin_state state;
>>>> +	struct nlattr *nest;
>>>> +	unsigned long index;
>>>> +	int ret;
>>>> +
>>>> +	xa_for_each(&pin->parent_refs, index, ref) {
>>>> +		if (WARN_ON(!ref->ops->state_on_pin_get))
>>>> +			return -EFAULT;
>>>> +		ret = ref->ops->state_on_pin_get(pin, ref->pin, &state,
>>>> +						 extack);
>>>> +		if (ret)
>>>> +			return -EFAULT;
>>>> +		nest = nla_nest_start(msg, DPLL_A_PIN_PARENT);
>>>> +		if (!nest)
>>>> +			return -EMSGSIZE;
>>>> +		if (nla_put_u32(msg, DPLL_A_PIN_PARENT_IDX,
>>>> +				ref->pin->dev_driver_id)) {
>>>> +			ret = -EMSGSIZE;
>>>> +			goto nest_cancel;
>>>> +		}
>>>> +		if (nla_put_u8(msg, DPLL_A_PIN_STATE, state)) {
>>>> +			ret = -EMSGSIZE;
>>>> +			goto nest_cancel;
>>>> +		}
>>>> +		nla_nest_end(msg, nest);
>>>> +	}
>>>
>>> How is this function different to dpll_msg_add_pin_parents()?
>>> Am I lost? To be honest, this x_on_pin/dpll, parent, refs dance is quite
>>> hard to follow for me :/
>>>
>>> Did you get lost here as well? If yes, this needs some serious think
>>> through :)
>>>
>>
>>Let's re-think it again. Arkadiuzs, do you have clear explanation of the
>>relationship between these things?
>>
>
>No, it is just leftover I didn't catch, we can leave one function and use it in
>both cases. Sorry about that, great catch!
>
>>>
>>>> +
>>>> +	return 0;
>>>> +
>>>> +nest_cancel:
>>>> +	nla_nest_cancel(msg, nest);
>>>> +	return ret;
>>>> +}
>>>> +
>>>> +static int
>>>> +dpll_msg_add_pin_dplls(struct sk_buff *msg, struct dpll_pin *pin,
>>>> +		       struct netlink_ext_ack *extack)
>>>> +{
>>>> +	struct dpll_pin_ref *ref;
>>>> +	struct nlattr *attr;
>>>> +	unsigned long index;
>>>> +	int ret;
>>>> +
>>>> +	xa_for_each(&pin->dpll_refs, index, ref) {
>>>> +		attr = nla_nest_start(msg, DPLL_A_DEVICE);
>>>> +		if (!attr)
>>>> +			return -EMSGSIZE;
>>>> +		ret = dpll_msg_add_dev_handle(msg, ref->dpll);
>>>> +		if (ret)
>>>> +			goto nest_cancel;
>>>> +		ret = dpll_msg_add_pin_on_dpll_state(msg, pin, ref, extack);
>>>> +		if (ret && ret != -EOPNOTSUPP)
>>>> +			goto nest_cancel;
>>>> +		ret = dpll_msg_add_pin_prio(msg, pin, ref, extack);
>>>> +		if (ret && ret != -EOPNOTSUPP)
>>>> +			goto nest_cancel;
>>>> +		nla_nest_end(msg, attr);
>>>> +	}
>>>> +
>>>> +	return 0;
>>>> +
>>>> +nest_cancel:
>>>> +	nla_nest_end(msg, attr);
>>>> +	return ret;
>>>> +}
>>>> +
>>>> +static int
>>>> +dpll_cmd_pin_on_dpll_get(struct sk_buff *msg, struct dpll_pin *pin,
>>>> +			 struct dpll_device *dpll,
>>>> +			 struct netlink_ext_ack *extack)
>>>> +{
>>>> +	struct dpll_pin_ref *ref;
>>>> +	int ret;
>>>> +
>>>> +	if (nla_put_u32(msg, DPLL_A_PIN_IDX, pin->dev_driver_id))
>>>
>>> Is it "ID" or "INDEX" (IDX). Please make this consistent in the whole
>>> code.
>>>
>>
>>I believe it's INDEX which is provided by the driver. I'll think about
>>renaming,
>>but suggestions are welcome.
>
>Yes, confusing a bit, I agree we need a fix of this.
>Isn't it that dpll has an INDEX assigned by dpll subsystem on its allocation
>and pin have ID given by the driver?

Driver should provice the device_index and pin_index in appropriate
_get() functions. I think it is better to spell out "index". "idx" looks
a bit odd. Either way, please unify the name all along the code, netlink
included.

ID is your odd dual-handle construct, which I don't see a need for and
only adds confusions. Better to remove it.


>
>[...]
>
>>>> +static int
>>>> +dpll_pin_set_from_nlattr(struct dpll_device *dpll,
>>>> +			 struct dpll_pin *pin, struct genl_info *info)
>>>> +{
>>>> +	enum dpll_pin_state state = DPLL_PIN_STATE_UNSPEC;
>>>> +	u32 parent_idx = PIN_IDX_INVALID;
>>>
>>> You just need this PIN_IDX_INVALID define internally in this function,
>>> change the flow to avoid a need for it.
>>>
>>
>>I'll re-think it, thanks.
>>
>>>
>>>> +	int rem, ret = -EINVAL;
>>>> +	struct nlattr *a;
>>>> +
>>>> +	nla_for_each_attr(a, genlmsg_data(info->genlhdr),
>>>> +			  genlmsg_len(info->genlhdr), rem) {
>>>
>>> This is odd. Why you iterace over attrs? Why don't you just access them
>>> directly, like attrs[DPLL_A_PIN_FREQUENCY] for example?
>>>
>>
>>I had some unknown crashes when I was using such access. I might have lost
>>some
>>checks, will try it again.
>>
>>>
>>>> +		switch (nla_type(a)) {
>>>> +		case DPLL_A_PIN_FREQUENCY:
>>>> +			ret = dpll_pin_freq_set(pin, a, info->extack);
>>>> +			if (ret)
>>>> +				return ret;
>>>> +			break;
>>>> +		case DPLL_A_PIN_DIRECTION:
>>>> +			ret = dpll_pin_direction_set(pin, a, info->extack);
>>>> +			if (ret)
>>>> +				return ret;
>>>> +			break;
>>>> +		case DPLL_A_PIN_PRIO:
>>>> +			ret = dpll_pin_prio_set(dpll, pin, a, info->extack);
>>>> +			if (ret)
>>>> +				return ret;
>>>> +			break;
>>>> +		case DPLL_A_PIN_PARENT_IDX:
>>>> +			parent_idx = nla_get_u32(a);
>>>> +			break;
>>>> +		case DPLL_A_PIN_STATE:
>>>> +			state = nla_get_u8(a);
>>>> +			break;
>>>> +		default:
>>>> +			break;
>>>> +		}
>>>> +	}
>>>> +	if (state != DPLL_PIN_STATE_UNSPEC) {
>>>
>>> Again, change the flow to:
>>> 	if (attrs[DPLL_A_PIN_STATE]) {
>>>
>>> and avoid need for this value set/check.
>>>
>>
>>Yep, will try.
>
>Yes, this shall work now, as long as there are no multiple nested attributes
>coming from userspace.
>
>[...]
>
>>>> +void dpll_pin_post_doit(const struct genl_split_ops *ops, struct
>>>>sk_buff *skb,
>>>> +			struct genl_info *info)
>>>> +{
>>>> +	mutex_unlock(&dpll_pin_xa_lock);
>>>> +	dpll_post_doit(ops, skb, info);
>>>> +}
>>>> +
>>>> +int dpll_pin_pre_dumpit(struct netlink_callback *cb)
>>>> +{
>>>> +	mutex_lock(&dpll_pin_xa_lock);
>>>
>>> ABBA deadlock here, see dpll_pin_register() for example where the lock
>>> taking order is opposite.
>>>
>>
>>Now I see an ABBA deadlock here, as well as in function before. Not sure
>>how to
>>solve it here. Any thoughts?
>>
>
>Not really, this is why it is there :(
>A single global lock for whole dpll subsystem?

Either that or per-instance/device lock.


>
>>>
>>>> +
>>>> +	return dpll_pre_dumpit(cb);
>>>> +}
>>>> +
>>>> +int dpll_pin_post_dumpit(struct netlink_callback *cb)
>>>> +{
>>>> +	mutex_unlock(&dpll_pin_xa_lock);
>>>> +
>>>> +	return dpll_post_dumpit(cb);
>>>> +}
>>>> +
>>>> +static int
>>>> +dpll_event_device_change(struct sk_buff *msg, struct dpll_device *dpll,
>>>> +			 struct dpll_pin *pin, struct dpll_pin *parent,
>>>> +			 enum dplla attr)
>>>> +{
>>>> +	int ret = dpll_msg_add_dev_handle(msg, dpll);
>>>> +	struct dpll_pin_ref *ref = NULL;
>>>> +	enum dpll_pin_state state;
>>>> +
>>>> +	if (ret)
>>>> +		return ret;
>>>> +	if (pin && nla_put_u32(msg, DPLL_A_PIN_IDX, pin->dev_driver_id))
>>>> +		return -EMSGSIZE;
>>>
>>> I don't really understand why you are trying figure something new and
>>> interesting with the change notifications. This object mix and random
>>> attrs fillup is something very wrong and makes userspace completely
>>> fuzzy about what it is getting. And yet it is so simple:
>>> You have 2 objects, dpll and pin, please just have:
>>> dpll_notify()
>>> dpll_pin_notify()
>>> and share the attrs fillup code with pin_get() and dpll_get() callbacks.
>>> No need for any smartness. Have this dumb and simple.
>>>
>>> Think about it more as about "object-state-snapshot" than "atomic-change"
>>
>>But with full object-snapshot user space app will lose the information
>>about
>>what exactly has changed. The reason to have this event is to provide the
>>attributes which have changed. Otherwise, the app should have full snapshot
>>and
>>compare all attributes to figure out changes and that's might not be great
>>idea.
>>
>
>I agree that from functional perspective it is better to have on userspace a
>reason of the notification.

I would like to understand why exactly is it better.
But anyway, I was thinking about it a bit more and it might make
sense only the added/changed/deleted attribute to safe some msg space
and getting/parsing cycles. However, it is questionable how much does it
actually save. But you apparently strongly want it, so lets have it

But, the format of the message should be exactly the same as for GET.
Meaning, if some nested attr changes/is added/is removed, the msg should
contain the proper nest same as for the same attr in GET msg. Think of
it as if you assemble the whole GET msg for an object and filter out
things that did not change.

Also need to emphasize strict objects separation, same as for GET.

Also, you have to distinguish between attr being added/removed or the
whole object being added/removed. The userspace has to understand
difference.

So you would have events like:
DEVICE_ADDED
DEVICE_REMOVED
DEVICE_ATTRS_ADDED
DEVICE_ATTRS_CHANGED
DEVICE_ATTRS_REMOVED
PIN_ADDED
PIN_REMOVED
PIN_ATTRS_ADDED
PIN_ATTRS_CHANGED
PIN_ATTRS_REMOVED

This scenario I can imagine working in a sane way.

Makes sense?

Btw, with the whole object snapshot scenario you just have:
DEVICE_NEW (sent for creation and change)
DEVICE_DEL
PIN_NEW (sent for creation and change)
PIN_DEL
And you are fine with it. This is how it's usually done in Netlink.
In fact, do you have examples in kernel code of what you are suggesting?



>
>[...]
>
>Thank you,
>Arkadiusz
>

Jiri Pirko March 15, 2023, 2:43 p.m. UTC | #9

Tue, Mar 14, 2023 at 07:35:55PM CET, [e-mail] wrote:
>>From: Jiri Pirko <[e-mail]>
>>Sent: Tuesday, March 14, 2023 4:45 PM
>>
>>[...]
>>
>>
>>>diff --git a/MAINTAINERS b/MAINTAINERS
>>>index edd3d562beee..0222b19af545 100644
>>>--- a/MAINTAINERS
>>>+++ b/MAINTAINERS
>>>@@ -6289,6 +6289,15 @@ F:
>>	Documentation/networking/device_drivers/ethernet/freescale/dpaa2/swit
>>ch-drive
>>> F:	drivers/net/ethernet/freescale/dpaa2/dpaa2-switch*
>>> F:	drivers/net/ethernet/freescale/dpaa2/dpsw*
>>>
>>>+DPLL CLOCK SUBSYSTEM
>>
>>Why "clock"? You don't mention "clock" anywhere else.
>>
>>[...]
>>
>>
>>>diff --git a/drivers/dpll/dpll_core.c b/drivers/dpll/dpll_core.c
>>>new file mode 100644
>>>index 000000000000..3fc151e16751
>>>--- /dev/null
>>>+++ b/drivers/dpll/dpll_core.c
>>>@@ -0,0 +1,835 @@
>>>+// SPDX-License-Identifier: GPL-2.0
>>>+/*
>>>+ *  dpll_core.c - Generic DPLL Management class support.
>>
>>Why "class" ?
>>
>>[...]
>>
>>
>>>+static int
>>>+dpll_msg_add_pin_freq(struct sk_buff *msg, const struct dpll_pin *pin,
>>>+		      struct netlink_ext_ack *extack, bool dump_any_freq)
>>>+{
>>>+	enum dpll_pin_freq_supp fs;
>>>+	struct dpll_pin_ref *ref;
>>>+	unsigned long i;
>>>+	u32 freq;
>>>+
>>>+	xa_for_each((struct xarray *)&pin->dpll_refs, i, ref) {
>>>+		if (ref && ref->ops && ref->dpll)
>>>+			break;
>>>+	}
>>>+	if (!ref || !ref->ops || !ref->dpll)
>>>+		return -ENODEV;
>>>+	if (!ref->ops->frequency_get)
>>>+		return -EOPNOTSUPP;
>>>+	if (ref->ops->frequency_get(pin, ref->dpll, &freq, extack))
>>>+		return -EFAULT;
>>>+	if (nla_put_u32(msg, DPLL_A_PIN_FREQUENCY, freq))
>>>+		return -EMSGSIZE;
>>>+	if (!dump_any_freq)
>>>+		return 0;
>>>+	for (fs = DPLL_PIN_FREQ_SUPP_UNSPEC + 1;
>>>+	     fs <= DPLL_PIN_FREQ_SUPP_MAX; fs++) {
>>>+		if (test_bit(fs, &pin->prop.freq_supported)) {
>>>+			if (nla_put_u32(msg, DPLL_A_PIN_FREQUENCY_SUPPORTED,
>>>+			    dpll_pin_freq_value[fs]))
>>
>>This is odd. As I suggested in the yaml patch, better to treat all
>>supported frequencies the same, no matter if it is range or not. The you
>>don't need this weird bitfield.
>>
>>You can have a macro to help driver to assemble array of supported
>>frequencies and ranges.
>>
>
>I understand suggestion on yaml, but here I am confused.
>How do they relate to the supported frequency passed between driver and dpll
>subsystem?
>This bitfield is not visible to the userspace, and sure probably adding macro
>can be useful.

My point is to avoid the bitfield and to treat supported frequencies and
ranges in the same way. It can look similar to this:

in dpll.h:

struct struct dpll_pin_frequency {
	u64 min;
	u64 max;
};

#define DPLL_PIN_FREQUENCY_RANGE(_min, _mac)	\
	{					\
		.min = _min,			\
		.max = _max,			\
	}

#define DPLL_PIN_FREQUENCY(_val) DPLL_PIN_FREQUENCY_RANGE(_val, _val)
#define DPLL_PIN_FREQUENCY_1PPS DPLL_PIN_FREQUENCY(1)
#define DPLL_PIN_FREQUENCY_10MHZ DPLL_PIN_FREQUENCY(1000000)


Then in driver you have:

static const struct dpll_pin_frequency pcp_dpll_pin_freqs[] = {
	DPLL_PIN_FREQUENCY_1PPS,
	DPLL_PIN_FREQUENCY_10MHZ,
	DPLL_PIN_FREQUENCY(4000000),
	DPLL_PIN_FREQUENCY_RANGE(500, 1000),
	DPLL_PIN_FREQUENCY_RANGE(9000, 10000),
};

static const struct dpll_pin_properties pcp_dpll_pin_props = {
	.label = "SMA",
	.frequencies_supported = pcp_dpll_pin_freqs,
	.frequencies_supported_count = ARRAY_SIZE(pcp_dpll_pin_freqs),
	.type = DPLL_PIN_TYPE_EXT,
	.capabilities = DPLL_PIN_CAPS_DIRECTION_CAN_CHANGE,
};


Then the dpll core could very easily iterate over .frequencies_supported
array and dump the supported values and ranges to user in uniform way.


>
>>
>>>+				return -EMSGSIZE;
>>>+		}
>>>+	}
>>>+	if (pin->prop.any_freq_min && pin->prop.any_freq_max) {
>>>+		if (nla_put_u32(msg, DPLL_A_PIN_ANY_FREQUENCY_MIN,
>>>+				pin->prop.any_freq_min))
>>>+			return -EMSGSIZE;
>>>+		if (nla_put_u32(msg, DPLL_A_PIN_ANY_FREQUENCY_MAX,
>>>+				pin->prop.any_freq_max))
>>>+			return -EMSGSIZE;
>>>+	}
>>>+
>>>+	return 0;
>>>+}
>>>+
>>
>>[...]
>>
>>
>>>+static int
>>>+dpll_cmd_pin_on_dpll_get(struct sk_buff *msg, struct dpll_pin *pin,
>>>+			 struct dpll_device *dpll,
>>>+			 struct netlink_ext_ack *extack)
>>>+{
>>>+	struct dpll_pin_ref *ref;
>>>+	int ret;
>>>+
>>>+	if (nla_put_u32(msg, DPLL_A_PIN_IDX, pin->dev_driver_id))
>>>+		return -EMSGSIZE;
>>>+	if (nla_put_string(msg, DPLL_A_PIN_DESCRIPTION, pin->prop.description))
>>>+		return -EMSGSIZE;
>>>+	if (nla_put_u8(msg, DPLL_A_PIN_TYPE, pin->prop.type))
>>>+		return -EMSGSIZE;
>>>+	if (nla_put_u32(msg, DPLL_A_PIN_DPLL_CAPS, pin->prop.capabilities))
>>>+		return -EMSGSIZE;
>>>+	ret = dpll_msg_add_pin_direction(msg, pin, extack);
>>>+	if (ret)
>>>+		return ret;
>>>+	ret = dpll_msg_add_pin_freq(msg, pin, extack, true);
>>>+	if (ret && ret != -EOPNOTSUPP)
>>>+		return ret;
>>>+	ref = dpll_xa_ref_dpll_find(&pin->dpll_refs, dpll);
>>>+	if (!ref)
>>
>>How exactly this can happen? Looks to me like only in case of a bug.
>>WARN_ON() perhaps (put directly into dpll_xa_ref_dpll_find()?
>
>Yes, makes sense.
>
>>
>>
>>>+		return -EFAULT;
>>>+	ret = dpll_msg_add_pin_prio(msg, pin, ref, extack);
>>>+	if (ret && ret != -EOPNOTSUPP)
>>>+		return ret;
>>>+	ret = dpll_msg_add_pin_on_dpll_state(msg, pin, ref, extack);
>>>+	if (ret && ret != -EOPNOTSUPP)
>>>+		return ret;
>>>+	ret = dpll_msg_add_pin_parents(msg, pin, extack);
>>>+	if (ret)
>>>+		return ret;
>>>+	if (pin->rclk_dev_name)
>>
>>Use && and single if
>>
>
>Make sense to me.
>
>>
>>>+		if (nla_put_string(msg, DPLL_A_PIN_RCLK_DEVICE,
>>>+				   pin->rclk_dev_name))
>>>+			return -EMSGSIZE;
>>>+
>>>+	return 0;
>>>+}
>>>+
>>
>>[...]
>>
>>
>>>+static int
>>>+dpll_pin_freq_set(struct dpll_pin *pin, struct nlattr *a,
>>>+		  struct netlink_ext_ack *extack)
>>>+{
>>>+	u32 freq = nla_get_u32(a);
>>>+	struct dpll_pin_ref *ref;
>>>+	unsigned long i;
>>>+	int ret;
>>>+
>>>+	if (!dpll_pin_is_freq_supported(pin, freq))
>>>+		return -EINVAL;
>>>+
>>>+	xa_for_each(&pin->dpll_refs, i, ref) {
>>>+		ret = ref->ops->frequency_set(pin, ref->dpll, freq, extack);
>>>+		if (ret)
>>>+			return -EFAULT;
>>
>>return what the op returns: ret
>
>Why would we return here a driver return code, userspace can have this info
>from extack. IMHO return values of dpll subsystem shall be not dependent on
>what is returned from the driver.

Why not to return it? The driver had some problem, errno suggests what
that was. It is completely desired to pass that along and actually,
it's been done like this in the rest of the netlink ops I can think of.
Why would you want to hide it? Extack carries string message,
not related to this directly.


>
>>
>>
>>>+		dpll_pin_notify(ref->dpll, pin, DPLL_A_PIN_FREQUENCY);
>>>+	}
>>>+
>>>+	return 0;
>>>+}
>>>+
>>
>>[...]
>>
>>
>>>+static int
>>>+dpll_pin_direction_set(struct dpll_pin *pin, struct nlattr *a,
>>>+		       struct netlink_ext_ack *extack)
>>>+{
>>>+	enum dpll_pin_direction direction = nla_get_u8(a);
>>>+	struct dpll_pin_ref *ref;
>>>+	unsigned long i;
>>>+
>>>+	if (!(DPLL_PIN_CAPS_DIRECTION_CAN_CHANGE & pin->prop.capabilities))
>>>+		return -EOPNOTSUPP;
>>>+
>>>+	xa_for_each(&pin->dpll_refs, i, ref) {
>>>+		if (ref->ops->direction_set(pin, ref->dpll, direction, extack))
>>
>>ret = ..
>>if (ret)
>>	return ret;
>>
>>Please use this pattern in other ops call code as well.
>>
>
>This is the same as above (return code by driver) explanation.

Same reply as above.


>
>>
>>>+			return -EFAULT;
>>>+		dpll_pin_notify(ref->dpll, pin, DPLL_A_PIN_DIRECTION);
>>>+	}
>>>+
>>>+	return 0;
>>
>>[...]
>
>Thanks,
>Arkadiusz

Jiri Pirko March 15, 2023, 3:29 p.m. UTC | #10

Sun, Mar 12, 2023 at 03:28:03AM CET, [e-mail] wrote:

[...]

>+struct dpll_device
>+*dpll_device_get(u64 clock_id, u32 dev_driver_id, struct module *module);

[...]

>+struct dpll_pin
>+*dpll_pin_get(u64 clock_id, u32 dev_driver_id, struct module *module,
>+	      const struct dpll_pin_properties *pin_prop);

Small tweak, please use the same trick as shown for example here:
/* pci_register_driver() must be a macro so KBUILD_MODNAME can be expanded */
#define pci_register_driver(driver)             \
        __pci_register_driver(driver, THIS_MODULE, KBUILD_MODNAME)

The driver calls header helper which fills up the "THIS_MODULE" for it.

Thanks!

Jiri Pirko March 16, 2023, 12:20 p.m. UTC | #11

Sun, Mar 12, 2023 at 03:28:03AM CET, [e-mail] wrote:

[...]


>+/**
>+ * dpll_pin_unregister - deregister dpll pin from dpll device
>+ * @dpll: registered dpll pointer
>+ * @pin: pointer to a pin
>+ *
>+ * Note: It does not free the memory
>+ */
>+int dpll_pin_unregister(struct dpll_device *dpll, struct dpll_pin *pin)

Make this return void. Function does not report anything useful,
non-0 is only in case of WARN_ON. Nobody is going to check that ever
anyway.


>+{
>+	if (WARN_ON(xa_empty(&dpll->pin_refs)))
>+		return -ENOENT;
>+
>+	mutex_lock(&dpll_device_xa_lock);
>+	mutex_lock(&dpll_pin_xa_lock);
>+	__dpll_pin_unregister(dpll, pin);
>+	mutex_unlock(&dpll_pin_xa_lock);
>+	mutex_unlock(&dpll_device_xa_lock);
>+
>+	return 0;
>+}
>+EXPORT_SYMBOL_GPL(dpll_pin_unregister);

[...]

Jiri Pirko March 16, 2023, 12:31 p.m. UTC | #12

Wed, Mar 15, 2023 at 10:22:33AM CET, [e-mail] wrote:
>Tue, Mar 14, 2023 at 06:50:57PM CET, [e-mail] wrote:
>>>From: Jiri Pirko <[e-mail]>
>>>Sent: Tuesday, March 14, 2023 10:22 AM
>>>
>>>Mon, Mar 13, 2023 at 11:59:32PM CET, [e-mail] wrote:
>>>>On 13.03.2023 16:21, Jiri Pirko wrote:
>>>>> Sun, Mar 12, 2023 at 03:28:03AM CET, [e-mail] wrote:

[...]


>>>>> > +	     const struct dpll_pin_properties *prop)
>>>>> > +{
>>>>> > +	struct dpll_pin *pos, *ret = NULL;
>>>>> > +	unsigned long index;
>>>>> > +
>>>>> > +	mutex_lock(&dpll_pin_xa_lock);
>>>>> > +	xa_for_each(&dpll_pin_xa, index, pos) {
>>>>> > +		if (pos->clock_id == clock_id &&
>>>>> > +		    pos->dev_driver_id == device_drv_id &&
>>>>> > +		    pos->module == module) {
>>>>>
>>>>> Compare prop as well.
>>>>>
>>>>> Can't the driver_id (pin index) be something const as well? I think it
>>>>> should. And therefore it could be easily put inside.
>>>>>
>>>>
>>>>I think clock_id + dev_driver_id + module should identify the pin exactly.
>>>>And now I think that *prop is not needed here at all. Arkadiusz, any
>>>>thoughts?
>>>
>>>IDK, no strong opinion on this. I just thought it may help to identify
>>>the pin and avoid potential driver bugs. (Like registering 2 pins with

Was thinking about this some more, I think that it would be good to
store and check properties in case of pin_get() call. If the driver does
call for the second time pin_get() for the same pin (clockid,pin_index)
with different prop, it is buggy and WARN_ON should be triggered.
WARN_ON check should compare the actual struct, not just pointer.

[...]

Jiri Pirko March 16, 2023, 12:37 p.m. UTC | #13

Sun, Mar 12, 2023 at 03:28:03AM CET, [e-mail] wrote:

[...]


>+struct dpll_pin *
>+dpll_pin_alloc(u64 clock_id, u8 device_drv_id,	struct module *module,
>+	       const struct dpll_pin_properties *prop)
>+{
>+	struct dpll_pin *pin;
>+	int ret;
>+
>+	pin = kzalloc(sizeof(*pin), GFP_KERNEL);
>+	if (!pin)
>+		return ERR_PTR(-ENOMEM);
>+	pin->dev_driver_id = device_drv_id;
>+	pin->clock_id = clock_id;
>+	pin->module = module;
>+	refcount_set(&pin->refcount, 1);
>+	if (WARN_ON(!prop->description)) {

Why is this mandatory? I'm now working on mlx5 implementation, don't
really know what to put here for SyncE. The type of the pin SyncE tells
all I need. I would like to avoid description fill-up.


>+		ret = -EINVAL;
>+		goto release;
>+	}
>+	pin->prop.description = kstrdup(prop->description, GFP_KERNEL);
>+	if (!pin->prop.description) {
>+		ret = -ENOMEM;
>+		goto release;
>+	}

[...]

Jiri Pirko March 16, 2023, 1:53 p.m. UTC | #14

Sun, Mar 12, 2023 at 03:28:03AM CET, [e-mail] wrote:

[...]


>+	int (*state_on_pin_get)(const struct dpll_pin *pin,
>+				const struct dpll_pin *parent_pin,
>+				enum dpll_pin_state *state,
>+				struct netlink_ext_ack *extack);
>+	int (*state_on_dpll_get)(const struct dpll_pin *pin,
>+				 const struct dpll_device *dpll,
>+				 enum dpll_pin_state *state,
>+				 struct netlink_ext_ack *extack);

Could this be rather called "state_on_device_get" or perhaps even better
just "state_get" (in sync with "prio_set/get") ?

This "od dpll" is a bit confusing, there is no such object.
We have "device" and "pin".


>+	int (*state_on_pin_set)(const struct dpll_pin *pin,
>+				const struct dpll_pin *parent_pin,
>+				const enum dpll_pin_state state,
>+				struct netlink_ext_ack *extack);
>+	int (*state_on_dpll_set)(const struct dpll_pin *pin,
>+				 const struct dpll_device *dpll,
>+				 const enum dpll_pin_state state,
>+				 struct netlink_ext_ack *extack);

[...]

Jiri Pirko March 16, 2023, 4:16 p.m. UTC | #15

Sun, Mar 12, 2023 at 03:28:03AM CET, [e-mail] wrote:

[...]


>+int dpll_device_register(struct dpll_device *dpll, enum dpll_type type,
>+			 struct dpll_device_ops *ops, void *priv,

ops should be const


>+			 struct device *owner)

[...]


>+
>+static int
>+__dpll_pin_register(struct dpll_device *dpll, struct dpll_pin *pin,
>+		    struct dpll_pin_ops *ops, void *priv,

ops should be const


>+		    const char *rclk_device_name)
>+{

[...]

Jiri Pirko March 17, 2023, 4:21 p.m. UTC | #16

Sun, Mar 12, 2023 at 03:28:03AM CET, [e-mail] wrote:

[...]


>+int dpll_nl_device_get_dumpit(struct sk_buff *skb, struct netlink_callback *cb)
>+{
>+	struct nlattr *hdr, *nest;
>+	struct dpll_device *dpll;
>+	unsigned long i;
>+	int ret;
>+
>+	hdr = genlmsg_put(skb, NETLINK_CB(cb->skb).portid, cb->nlh->nlmsg_seq,
>+			  &dpll_nl_family, 0, DPLL_CMD_DEVICE_GET);
>+	if (!hdr)
>+		return -EMSGSIZE;
>+
>+	xa_for_each_marked(&dpll_device_xa, i, dpll, DPLL_REGISTERED) {
>+		nest = nla_nest_start(skb, DPLL_A_DEVICE);
>+		ret = dpll_msg_add_dev_handle(skb, dpll);
>+		if (ret) {
>+			nla_nest_cancel(skb, nest);
>+			break;
>+		}

Please fillup the attrs for the object. The format should be exactly the
same as for doit.


>+		nla_nest_end(skb, nest);
>+	}
>+	if (ret)
>+		genlmsg_cancel(skb, hdr);
>+	else
>+		genlmsg_end(skb, hdr);
>+
>+	return ret;
>+}
>+

[...]

Jiri Pirko March 20, 2023, 10:24 a.m. UTC | #17

Sun, Mar 12, 2023 at 03:28:03AM CET, [e-mail] wrote:

[...]


>+static int
>+dpll_msg_add_pins_on_pin(struct sk_buff *msg, struct dpll_pin *pin,
>+			 struct netlink_ext_ack *extack)
>+{
>+	struct dpll_pin_ref *ref = NULL;

Pointless init.

[...]


>+static int
>+dpll_event_device_change(struct sk_buff *msg, struct dpll_device *dpll,
>+			 struct dpll_pin *pin, struct dpll_pin *parent,
>+			 enum dplla attr)
>+{
>+	int ret = dpll_msg_add_dev_handle(msg, dpll);
>+	struct dpll_pin_ref *ref = NULL;

Pointless init.

[...]


>+struct dpll_pin_ops {
>+	int (*frequency_set)(const struct dpll_pin *pin,
>+			     const struct dpll_device *dpll,
>+			     const u32 frequency,
>+			     struct netlink_ext_ack *extack);
>+	int (*frequency_get)(const struct dpll_pin *pin,
>+			     const struct dpll_device *dpll,
>+			     u32 *frequency, struct netlink_ext_ack *extack);
>+	int (*direction_set)(const struct dpll_pin *pin,
>+			     const struct dpll_device *dpll,
>+			     const enum dpll_pin_direction direction,
>+			     struct netlink_ext_ack *extack);
>+	int (*direction_get)(const struct dpll_pin *pin,
>+			     const struct dpll_device *dpll,
>+			     enum dpll_pin_direction *direction,
>+			     struct netlink_ext_ack *extack);


For frequency and direction, why exactly do you need to pass dpll
pointer? For set I can understand you need it to set the same
frequency/direction for all dplls the pin is connected to, but why for
get()?

[...]

Jiri Pirko March 21, 2023, 1:34 p.m. UTC | #18

Sun, Mar 12, 2023 at 03:28:03AM CET, [e-mail] wrote:

[...]

>+static int
>+dpll_event_device_change(struct sk_buff *msg, struct dpll_device *dpll,
>+			 struct dpll_pin *pin, struct dpll_pin *parent,
>+			 enum dplla attr)
>+{
>+	int ret = dpll_msg_add_dev_handle(msg, dpll);
>+	struct dpll_pin_ref *ref = NULL;
>+	enum dpll_pin_state state;
>+
>+	if (ret)
>+		return ret;
>+	if (pin && nla_put_u32(msg, DPLL_A_PIN_IDX, pin->dev_driver_id))
>+		return -EMSGSIZE;
>+
>+	switch (attr) {
>+	case DPLL_A_MODE:
>+		ret = dpll_msg_add_mode(msg, dpll, NULL);
>+		break;
>+	case DPLL_A_SOURCE_PIN_IDX:
>+		ret = dpll_msg_add_source_pin_idx(msg, dpll, NULL);
>+		break;
>+	case DPLL_A_LOCK_STATUS:
>+		ret = dpll_msg_add_lock_status(msg, dpll, NULL);

On top of what I wrote about the notifications, I found another two
issues:
1) You don't take any lock calling this from drivers. You need to hold
   the xarray locks you have now.

   I have to repear, I think that we definitelly need to convert the
   overall locking scheme to have this per-instance, in a similar way
   we did that for devlink. I noted this in another email, but wanted
   to say that again.

2) You have possible race condition:
   1) -> driver gets a state change event
   2) -> driver calls into this function
   3) -> this code does call the driver op to get the state, driver
         queries the state again

   Between 1) and 3) state can easily change, multiple times. That might
   lead to oddities observed by the user (like getting a notification
   of change with the original values)

   I see only 1 solutions to this:
   Pass the value of changed item from the driver here and just pass
   it on over netlink without doing calling into driver again.


>+		break;
>+	case DPLL_A_TEMP:
>+		ret = dpll_msg_add_temp(msg, dpll, NULL);
>+		break;
>+	case DPLL_A_PIN_FREQUENCY:
>+		ret = dpll_msg_add_pin_freq(msg, pin, NULL, false);
>+		break;
>+	case DPLL_A_PIN_PRIO:
>+		ref = dpll_xa_ref_dpll_find(&pin->dpll_refs, dpll);
>+		if (!ref)
>+			return -EFAULT;
>+		ret = dpll_msg_add_pin_prio(msg, pin, ref, NULL);
>+		break;
>+	case DPLL_A_PIN_STATE:
>+		if (parent) {
>+			ref = dpll_xa_ref_pin_find(&pin->parent_refs, parent);
>+			if (!ref)
>+				return -EFAULT;
>+			if (!ref->ops || !ref->ops->state_on_pin_get)
>+				return -EOPNOTSUPP;
>+			ret = ref->ops->state_on_pin_get(pin, parent, &state,
>+							 NULL);
>+			if (ret)
>+				return ret;
>+			if (nla_put_u32(msg, DPLL_A_PIN_PARENT_IDX,
>+					parent->dev_driver_id))
>+				return -EMSGSIZE;
>+		} else {
>+			ref = dpll_xa_ref_dpll_find(&pin->dpll_refs, dpll);
>+			if (!ref)
>+				return -EFAULT;
>+			ret = dpll_msg_add_pin_on_dpll_state(msg, pin, ref,
>+							     NULL);
>+			if (ret)
>+				return ret;
>+		}
>+		break;
>+	default:
>+		break;
>+	}
>+
>+	return ret;
>+}
>+
>+static int
>+dpll_send_event_create(enum dpll_event event, struct dpll_device *dpll)
>+{
>+	struct sk_buff *msg;
>+	int ret = -EMSGSIZE;
>+	void *hdr;
>+
>+	msg = genlmsg_new(NLMSG_GOODSIZE, GFP_KERNEL);
>+	if (!msg)
>+		return -ENOMEM;
>+
>+	hdr = genlmsg_put(msg, 0, 0, &dpll_nl_family, 0, event);
>+	if (!hdr)
>+		goto out_free_msg;
>+
>+	ret = dpll_msg_add_dev_handle(msg, dpll);
>+	if (ret)
>+		goto out_cancel_msg;
>+	genlmsg_end(msg, hdr);
>+	genlmsg_multicast(&dpll_nl_family, msg, 0, 0, GFP_KERNEL);
>+
>+	return 0;
>+
>+out_cancel_msg:
>+	genlmsg_cancel(msg, hdr);
>+out_free_msg:
>+	nlmsg_free(msg);
>+
>+	return ret;
>+}
>+
>+static int
>+dpll_send_event_change(struct dpll_device *dpll, struct dpll_pin *pin,
>+		       struct dpll_pin *parent, enum dplla attr)
>+{
>+	struct sk_buff *msg;
>+	int ret = -EMSGSIZE;
>+	void *hdr;
>+
>+	msg = genlmsg_new(NLMSG_GOODSIZE, GFP_KERNEL);
>+	if (!msg)
>+		return -ENOMEM;
>+
>+	hdr = genlmsg_put(msg, 0, 0, &dpll_nl_family, 0,
>+			  DPLL_EVENT_DEVICE_CHANGE);
>+	if (!hdr)
>+		goto out_free_msg;
>+
>+	ret = dpll_event_device_change(msg, dpll, pin, parent, attr);
>+	if (ret)
>+		goto out_cancel_msg;
>+	genlmsg_end(msg, hdr);
>+	genlmsg_multicast(&dpll_nl_family, msg, 0, 0, GFP_KERNEL);
>+
>+	return 0;
>+
>+out_cancel_msg:
>+	genlmsg_cancel(msg, hdr);
>+out_free_msg:
>+	nlmsg_free(msg);
>+
>+	return ret;
>+}
>+
>+int dpll_notify_device_create(struct dpll_device *dpll)
>+{
>+	return dpll_send_event_create(DPLL_EVENT_DEVICE_CREATE, dpll);
>+}
>+
>+int dpll_notify_device_delete(struct dpll_device *dpll)

Please change the function names to "register/unregister" to be
consistent with the rest of the code.


>+{
>+	return dpll_send_event_create(DPLL_EVENT_DEVICE_DELETE, dpll);
>+}
>+
>+int dpll_device_notify(struct dpll_device *dpll, enum dplla attr)
>+{
>+	if (WARN_ON(!dpll))
>+		return -EINVAL;
>+
>+	return dpll_send_event_change(dpll, NULL, NULL, attr);
>+}
>+EXPORT_SYMBOL_GPL(dpll_device_notify);
>+
>+int dpll_pin_notify(struct dpll_device *dpll, struct dpll_pin *pin,
>+		    enum dplla attr)

The driver should be aware of netlink attributes. Should be
abstracted out.

just have per-item notification like:
dpll_pin_state_notify()
dpll_pin_prio_notify()
...

Then you can easily pass changed value that would allow solution to
the issue 2) I described above.



>+{
>+	return dpll_send_event_change(dpll, pin, NULL, attr);
>+}
>+

[...]

Jiri Pirko March 23, 2023, 11:18 a.m. UTC | #19

Sun, Mar 12, 2023 at 03:28:03AM CET, [e-mail] wrote:

[...]


>+/**
>+ * dpll_xa_ref_pin_del - remove reference of a pin from xarray
>+ * @xa_pins: dpll_pin_ref xarray holding pins
>+ * @pin: pointer to a pin
>+ *
>+ * Decrement refcount of existing pin reference on given xarray.
>+ * If all references are dropped, delete the reference and free its memory.
>+ *

Hmm, came to think about this, why do you do func docs even for static
function? It is customary to do that for exported function. For static
ones, not really needed.


>+ * Return:
>+ * * 0 on success
>+ * * -EINVAL if reference to a pin was not found
>+ */
>+static int dpll_xa_ref_pin_del(struct xarray *xa_pins, struct dpll_pin *pin)

Have this to return void, you don't check the return value anywhere.


>+{
>+	struct dpll_pin_ref *ref;
>+	unsigned long i;
>+
>+	xa_for_each(xa_pins, i, ref) {
>+		if (ref->pin == pin) {
>+			if (refcount_dec_and_test(&ref->refcount)) {
>+				xa_erase(xa_pins, i);
>+				kfree(ref);
>+			}
>+			return 0;
>+		}
>+	}
>+
>+	return -EINVAL;
>+}

[...]


>+/**
>+ * dpll_xa_ref_dpll_find - find dpll reference on xarray
>+ * @xa_dplls: dpll_pin_ref xarray holding dplls
>+ * @dpll: pointer to a dpll
>+ *
>+ * Search for dpll-pin ops reference struct of a given dpll on given xarray.
>+ *
>+ * Return:
>+ * * pin reference struct pointer on success
>+ * * NULL - reference to a pin was not found
>+ */
>+struct dpll_pin_ref *
>+dpll_xa_ref_dpll_find(struct xarray *xa_refs, const struct dpll_device *dpll)

Every caller of this function does fill the first arg by:
&pin->dpll_refs
Could you please change it to "struct dpll_pin *pin" and get the xarray
pointer in this function?

The same applies to other functions passing xarray pointer, like:
dpll_xa_ref_dpll_add
dpll_xa_ref_dpll_del
dpll_xa_ref_pin_find

The point is, always better and easier to read to pass
"struct dpll_device *" and "struct dpll_pin *" as function args.
Passing "struct xarray *" makes the reader uncertain about what
is going on.



>+{
>+	struct dpll_pin_ref *ref;
>+	unsigned long i;
>+
>+	xa_for_each(xa_refs, i, ref) {
>+		if (ref->dpll == dpll)
>+			return ref;
>+	}
>+
>+	return NULL;
>+}
>+
>+

[...]


>+/**
>+ * dpll_device_register - register the dpll device in the subsystem
>+ * @dpll: pointer to a dpll
>+ * @type: type of a dpll
>+ * @ops: ops for a dpll device
>+ * @priv: pointer to private information of owner
>+ * @owner: pointer to owner device
>+ *
>+ * Make dpll device available for user space.
>+ *
>+ * Return:
>+ * * 0 on success
>+ * * -EINVAL on failure

You return more than that. Do you really need to list the error
possiblities in the func docs?


>+ */
>+int dpll_device_register(struct dpll_device *dpll, enum dpll_type type,
>+			 struct dpll_device_ops *ops, void *priv,
>+			 struct device *owner)

[...]


>+static int
>+__dpll_pin_register(struct dpll_device *dpll, struct dpll_pin *pin,
>+		    struct dpll_pin_ops *ops, void *priv,
>+		    const char *rclk_device_name)
>+{
>+	int ret;
>+
>+	if (rclk_device_name && !pin->rclk_dev_name) {
>+		pin->rclk_dev_name = kstrdup(rclk_device_name, GFP_KERNEL);
>+		if (!pin->rclk_dev_name)
>+			return -ENOMEM;
>+	}

Somewhere here, please add a check:
dpll->module == pin->module dpll->clock_id && pin->clock_id
For sanity sake.


>+	ret = dpll_xa_ref_pin_add(&dpll->pin_refs, pin, ops, priv);
>+	if (ret)
>+		goto rclk_free;
>+	ret = dpll_xa_ref_dpll_add(&pin->dpll_refs, dpll, ops, priv);
>+	if (ret)
>+		goto ref_pin_del;
>+	else
>+		dpll_pin_notify(dpll, pin, DPLL_A_PIN_IDX);

Pointless else.


>+
>+	return ret;
>+
>+ref_pin_del:
>+	dpll_xa_ref_pin_del(&dpll->pin_refs, pin);
>+rclk_free:
>+	kfree(pin->rclk_dev_name);
>+	return ret;
>+}

[...]


>+int
>+dpll_pin_on_pin_register(struct dpll_pin *parent, struct dpll_pin *pin,
>+			 struct dpll_pin_ops *ops, void *priv,
>+			 struct device *rclk_device)
>+{
>+	struct dpll_pin_ref *ref;
>+	unsigned long i, stop;
>+	int ret;
>+
>+	if (WARN_ON(!pin || !parent))
>+		return -EINVAL;
>+	if (WARN_ON(parent->prop.type != DPLL_PIN_TYPE_MUX))
>+		return -EPERM;
>+	mutex_lock(&dpll_pin_xa_lock);
>+	ret = dpll_xa_ref_pin_add(&pin->parent_refs, parent, ops, priv);
>+	if (ret)
>+		goto unlock;
>+	refcount_inc(&pin->refcount);
>+	xa_for_each(&parent->dpll_refs, i, ref) {
>+		mutex_lock(&dpll_device_xa_lock);
>+		ret = __dpll_pin_register(ref->dpll, pin, ops, priv,

Why exactly do you need to register the pin over to the dpll of a
parent? Isn't it enough to have the pin registered on a parent?
I mean, there is no direct connection between pin and dpll, the parent
is in the middle. So prio setup, and other things does not make sense to
configure on this child pin, isn't it?

Btw, what is stopping the driver from:
dpll register
pin1 register on dpll
pin2 register on pin1
pin1 unregister
?
The you would have pin2 registered to dpll incorrectly.


>+					  rclk_device ?
>+					  dev_name(rclk_device) : NULL);
>+		mutex_unlock(&dpll_device_xa_lock);
>+		if (ret) {
>+			stop = i;
>+			goto dpll_unregister;
>+		}
>+		dpll_pin_parent_notify(ref->dpll, pin, parent, DPLL_A_PIN_IDX);
>+	}
>+	mutex_unlock(&dpll_pin_xa_lock);
>+
>+	return ret;
>+
>+dpll_unregister:
>+	xa_for_each(&parent->dpll_refs, i, ref) {
>+		if (i < stop) {
>+			mutex_lock(&dpll_device_xa_lock);
>+			__dpll_pin_unregister(ref->dpll, pin);
>+			mutex_unlock(&dpll_device_xa_lock);
>+		}
>+	}
>+	refcount_dec(&pin->refcount);
>+	dpll_xa_ref_pin_del(&pin->parent_refs, parent);
>+unlock:
>+	mutex_unlock(&dpll_pin_xa_lock);
>+	return ret;
>+}

[...]


>+static int
>+dpll_pin_on_pin_state_set(struct dpll_device *dpll, struct dpll_pin *pin,
>+			  u32 parent_idx, enum dpll_pin_state state,
>+			  struct netlink_ext_ack *extack)
>+{
>+	struct dpll_pin_ref *ref;
>+	struct dpll_pin *parent;
>+
>+	if (!(DPLL_PIN_CAPS_STATE_CAN_CHANGE & pin->prop.capabilities))

Hmm, why is this capabilities are any good for internal purposes? I
understand the need to expose it to the user, but internally in kernel,
if the driver implements some _set() op, it is good enough indication of
a support of a certain setter. You can check if the relevant _set()
is not null and expose the appropriate capability to the user.



>+		return -EOPNOTSUPP;
>+	parent = dpll_pin_get_by_idx(dpll, parent_idx);

I don't follow. Why do you need dpll pointer to get the parent pin?
The same handle as pin should be used, you have clock_id and driver name
(in next patchsets implementation) that should be enough.
Pin is a separate entity, attached 0:N dplls.

Please remove dpll pointer from here. Also, please remove
dpll->pins_ref, as you are using this array only for this lookup (here
and in dpll_pin_pre_doit())


>+	if (!parent)
>+		return -EINVAL;
>+	ref = dpll_xa_ref_pin_find(&pin->parent_refs, parent);
>+	if (!ref)
>+		return -EINVAL;
>+	if (!ref->ops || !ref->ops->state_on_pin_set)
>+		return -EOPNOTSUPP;
>+	if (ref->ops->state_on_pin_set(pin, parent, state, extack))
>+		return -EFAULT;
>+	dpll_pin_parent_notify(dpll, pin, parent, DPLL_A_PIN_STATE);
>+
>+	return 0;
>+}
>+
>+static int
>+dpll_pin_state_set(struct dpll_device *dpll, struct dpll_pin *pin,
>+		   enum dpll_pin_state state,
>+		   struct netlink_ext_ack *extack)
>+{
>+	struct dpll_pin_ref *ref;
>+
>+	if (!(DPLL_PIN_CAPS_STATE_CAN_CHANGE & pin->prop.capabilities))
>+		return -EOPNOTSUPP;
>+	ref = dpll_xa_ref_dpll_find(&pin->dpll_refs, dpll);
>+	if (!ref)
>+		return -EFAULT;
>+	if (!ref->ops || !ref->ops->state_on_dpll_set)
>+		return -EOPNOTSUPP;
>+	if (ref->ops->state_on_dpll_set(pin, ref->dpll, state, extack))
>+		return -EINVAL;
>+	dpll_pin_notify(ref->dpll, pin, DPLL_A_PIN_STATE);
>+
>+	return 0;
>+}

[...]


>+static int
>+dpll_set_from_nlattr(struct dpll_device *dpll, struct genl_info *info)
>+{
>+	struct nlattr *attr;
>+	enum dpll_mode mode;
>+	int rem, ret = 0;
>+
>+	nla_for_each_attr(attr, genlmsg_data(info->genlhdr),
>+			  genlmsg_len(info->genlhdr), rem) {
>+		switch (nla_type(attr)) {
>+		case DPLL_A_MODE:
>+			mode = nla_get_u8(attr);
>+
>+			if (!dpll->ops || !dpll->ops->mode_set)

Remove the pointless check of ops. This cannot happen (checked in
dpll_device_register())


>+				return -EOPNOTSUPP;
>+			ret = dpll->ops->mode_set(dpll, mode, info->extack);
>+			if (ret)
>+				return ret;
>+			break;
>+		default:
>+			break;
>+		}
>+	}
>+
>+	return ret;
>+}
>+

[...]

Jiri Pirko March 24, 2023, 9:29 a.m. UTC | #20

Sun, Mar 12, 2023 at 03:28:03AM CET, [e-mail] wrote:

[...]


>+static int
>+dpll_msg_add_pin_direction(struct sk_buff *msg, const struct dpll_pin *pin,
>+			   struct netlink_ext_ack *extack)
>+{
>+	enum dpll_pin_direction direction;
>+	struct dpll_pin_ref *ref;
>+	unsigned long i;
>+
>+	xa_for_each((struct xarray *)&pin->dpll_refs, i, ref) {
>+		if (ref && ref->ops && ref->dpll)
>+			break;
>+	}
>+	if (!ref || !ref->ops || !ref->dpll)
>+		return -ENODEV;
>+	if (!ref->ops->direction_get)
>+		return -EOPNOTSUPP;
>+	if (ref->ops->direction_get(pin, ref->dpll, &direction, extack))
>+		return -EFAULT;
>+	if (nla_put_u8(msg, DPLL_A_PIN_DIRECTION, direction))
>+		return -EMSGSIZE;
>+
>+	return 0;
>+}
>+
>+static int
>+dpll_msg_add_pin_freq(struct sk_buff *msg, const struct dpll_pin *pin,
>+		      struct netlink_ext_ack *extack, bool dump_any_freq)
>+{
>+	enum dpll_pin_freq_supp fs;
>+	struct dpll_pin_ref *ref;
>+	unsigned long i;
>+	u32 freq;
>+
>+	xa_for_each((struct xarray *)&pin->dpll_refs, i, ref) {
>+		if (ref && ref->ops && ref->dpll)

Checking for "ref" is nonsense here, as xa_for_each fills it up
for every iteration.

ref->dpll is always filled. Also pointless check.

Does it make sense to register with ops==NULL? I think we should
forbid it and make this just xa_find(0) to get the first item in the
xarray.

I'm doing this in my patch, as it is dependency on some other patch I do
in this area.


>+			break;
>+	}
>+	if (!ref || !ref->ops || !ref->dpll)
>+		return -ENODEV;
>+	if (!ref->ops->frequency_get)
>+		return -EOPNOTSUPP;
>+	if (ref->ops->frequency_get(pin, ref->dpll, &freq, extack))
>+		return -EFAULT;
>+	if (nla_put_u32(msg, DPLL_A_PIN_FREQUENCY, freq))
>+		return -EMSGSIZE;
>+	if (!dump_any_freq)
>+		return 0;
>+	for (fs = DPLL_PIN_FREQ_SUPP_UNSPEC + 1;
>+	     fs <= DPLL_PIN_FREQ_SUPP_MAX; fs++) {
>+		if (test_bit(fs, &pin->prop.freq_supported)) {
>+			if (nla_put_u32(msg, DPLL_A_PIN_FREQUENCY_SUPPORTED,
>+			    dpll_pin_freq_value[fs]))
>+				return -EMSGSIZE;
>+		}
>+	}
>+	if (pin->prop.any_freq_min && pin->prop.any_freq_max) {
>+		if (nla_put_u32(msg, DPLL_A_PIN_ANY_FREQUENCY_MIN,
>+				pin->prop.any_freq_min))
>+			return -EMSGSIZE;
>+		if (nla_put_u32(msg, DPLL_A_PIN_ANY_FREQUENCY_MAX,
>+				pin->prop.any_freq_max))
>+			return -EMSGSIZE;
>+	}
>+
>+	return 0;
>+}
>+

[...]


>+static int
>+dpll_cmd_pin_on_dpll_get(struct sk_buff *msg, struct dpll_pin *pin,
>+			 struct dpll_device *dpll,
>+			 struct netlink_ext_ack *extack)
>+{
>+	struct dpll_pin_ref *ref;
>+	int ret;
>+
>+	if (nla_put_u32(msg, DPLL_A_PIN_IDX, pin->dev_driver_id))
>+		return -EMSGSIZE;
>+	if (nla_put_string(msg, DPLL_A_PIN_DESCRIPTION, pin->prop.description))
>+		return -EMSGSIZE;
>+	if (nla_put_u8(msg, DPLL_A_PIN_TYPE, pin->prop.type))
>+		return -EMSGSIZE;
>+	if (nla_put_u32(msg, DPLL_A_PIN_DPLL_CAPS, pin->prop.capabilities))
>+		return -EMSGSIZE;
>+	ret = dpll_msg_add_pin_direction(msg, pin, extack);
>+	if (ret)

Why -EOPNOTSUPP here is not ok, as for the others below?


>+		return ret;
>+	ret = dpll_msg_add_pin_freq(msg, pin, extack, true);
>+	if (ret && ret != -EOPNOTSUPP)
>+		return ret;
>+	ref = dpll_xa_ref_dpll_find(&pin->dpll_refs, dpll);
>+	if (!ref)

How this can happen? I don't think it could.


>+		return -EFAULT;
>+	ret = dpll_msg_add_pin_prio(msg, pin, ref, extack);
>+	if (ret && ret != -EOPNOTSUPP)
>+		return ret;
>+	ret = dpll_msg_add_pin_on_dpll_state(msg, pin, ref, extack);
>+	if (ret && ret != -EOPNOTSUPP)
>+		return ret;
>+	ret = dpll_msg_add_pin_parents(msg, pin, extack);
>+	if (ret)
>+		return ret;
>+	if (pin->rclk_dev_name)
>+		if (nla_put_string(msg, DPLL_A_PIN_RCLK_DEVICE,
>+				   pin->rclk_dev_name))
>+			return -EMSGSIZE;
>+
>+	return 0;
>+}
>+
>+static int
>+__dpll_cmd_pin_dump_one(struct sk_buff *msg, struct dpll_pin *pin,
>+			struct netlink_ext_ack *extack, bool dump_dpll)
>+{
>+	int ret;
>+
>+	if (nla_put_u32(msg, DPLL_A_PIN_IDX, pin->dev_driver_id))
>+		return -EMSGSIZE;
>+	if (nla_put_string(msg, DPLL_A_PIN_DESCRIPTION, pin->prop.description))
>+		return -EMSGSIZE;
>+	if (nla_put_u8(msg, DPLL_A_PIN_TYPE, pin->prop.type))
>+		return -EMSGSIZE;
>+	ret = dpll_msg_add_pin_direction(msg, pin, extack);
>+	if (ret)
>+		return ret;
>+	ret = dpll_msg_add_pin_freq(msg, pin, extack, true);
>+	if (ret && ret != -EOPNOTSUPP)
>+		return ret;
>+	ret = dpll_msg_add_pins_on_pin(msg, pin, extack);
>+	if (ret)
>+		return ret;
>+	if (!xa_empty(&pin->dpll_refs) && dump_dpll) {

How dpll refs could be empty? I don't think it is possible.

Overall, whole the code has very odd habit of checking for conditions
that are obviously impossible to happen. Only confuses reader as he
naturally expects that the check is there for a reason.



>+		ret = dpll_msg_add_pin_dplls(msg, pin, extack);
>+		if (ret)
>+			return ret;
>+	}
>+	if (pin->rclk_dev_name)
>+		if (nla_put_string(msg, DPLL_A_PIN_RCLK_DEVICE,
>+				   pin->rclk_dev_name))
>+			return -EMSGSIZE;
>+
>+	return 0;
>+}
>+

Vadim Fedorenko March 28, 2023, 3:22 p.m. UTC | #21

On 15/03/2023 09:22, Jiri Pirko wrote:
> Tue, Mar 14, 2023 at 06:50:57PM CET, [e-mail] wrote:
>>> From: Jiri Pirko <[e-mail]>
>>> Sent: Tuesday, March 14, 2023 10:22 AM
>>>
>>> Mon, Mar 13, 2023 at 11:59:32PM CET, [e-mail] wrote:
>>>> On 13.03.2023 16:21, Jiri Pirko wrote:
>>>>> Sun, Mar 12, 2023 at 03:28:03AM CET, [e-mail] wrote:
>>>
>>> [...]
>>>
>>>
>>>>>> diff --git a/drivers/dpll/Makefile b/drivers/dpll/Makefile
>>>>>> new file mode 100644
>>>>>> index 000000000000..d3926f2a733d
>>>>>> --- /dev/null
>>>>>> +++ b/drivers/dpll/Makefile
>>>>>> @@ -0,0 +1,10 @@
>>>>>> +# SPDX-License-Identifier: GPL-2.0
>>>>>> +#
>>>>>> +# Makefile for DPLL drivers.
>>>>>> +#
>>>>>> +
>>>>>> +obj-$(CONFIG_DPLL)          += dpll_sys.o
>>>>>
>>>>> What's "sys" and why is it here?
>>>>
>>>> It's an object file for the subsystem. Could be useful if we will have
>>>> drivers
>>>> for DPLL-only devices.
>>>
>>> Yeah, but why "sys"? I don't get what "sys" means here.
>>> Can't this be just "dpll.o"?
>>>
>>>
>>>>
>>>>>> +dpll_sys-y                  += dpll_core.o
>>>>>> +dpll_sys-y                  += dpll_netlink.o
>>>>>> +dpll_sys-y                  += dpll_nl.o
>>>>>> +
>>>
>>> [...]
>>>
>>>
>>>>>> +struct dpll_device *
>>>>>> +dpll_device_get(u64 clock_id, u32 dev_driver_id, struct module
>>>>>> *module)
>>>>>> +{
>>>>>> +	struct dpll_device *dpll, *ret = NULL;
>>>>>> +	unsigned long index;
>>>>>> +
>>>>>> +	mutex_lock(&dpll_device_xa_lock);
>>>>>> +	xa_for_each(&dpll_device_xa, index, dpll) {
>>>>>> +		if (dpll->clock_id == clock_id &&
>>>>>> +		    dpll->dev_driver_id == dev_driver_id &&
>>>>>
>>>>> Why you need "dev_driver_id"? clock_id is here for the purpose of
>>>>> identification, isn't that enough for you.
>>>>
>>>> dev_driver_id is needed to provide several DPLLs from one device. In ice
>>>> driver
>>>> implementation there are 2 different DPLLs - to recover from PPS input and
>>>> to
>>>> recover from Sync-E. I believe there is only one clock, that's why clock id
>>>> is the same for both of them. But Arkadiusz can tell more about it.
>>>
>>> Okay, I see. Clock_id is the same. Could we have index for pin, could
>>> this be index too:
>>>
>>> dpll_device_get(u64 clock_id, u32 device_index, struct module *module);
>>> dpll_pin_get(u64 clock_id, u32 pin_index, struct module *module,
>>> 	     const struct dpll_pin_properties *prop);
>>>
>>> This way it is consistent, driver provides custom index for both dpll
>>> device and dpll pin.
>>>
>>> Makes sense?
>>>
>>
>> IMHO, Yes this better shows the intentions.
> 
> Ok.
> 
> 
>>
>>>
>>>>>
>>>>> Plus, the name is odd. "dev_driver" should certainly be avoided.
>>>>
>>>> Simply id doesn't tell anything either. dpll_dev_id?
>>>
>>> Yeah, see above.
>>>
>>>
>>>>
>>>>>> +		    dpll->module == module) {
>>>>>> +			ret = dpll;
>>>>>> +			refcount_inc(&ret->refcount);
>>>>>> +			break;
>>>>>> +		}
>>>>>> +	}
>>>>>> +	if (!ret)
>>>>>> +		ret = dpll_device_alloc(clock_id, dev_driver_id, module);
>>>>>> +	mutex_unlock(&dpll_device_xa_lock);
>>>>>> +
>>>>>> +	return ret;
>>>>>> +}
>>>>>> +EXPORT_SYMBOL_GPL(dpll_device_get);
>>>>>> +
>>>>>> +/**
>>>>>> + * dpll_device_put - decrease the refcount and free memory if possible
>>>>>> + * @dpll: dpll_device struct pointer
>>>>>> + *
>>>>>> + * Drop reference for a dpll device, if all references are gone, delete
>>>>>> + * dpll device object.
>>>>>> + */
>>>>>> +void dpll_device_put(struct dpll_device *dpll)
>>>>>> +{
>>>>>> +	if (!dpll)
>>>>>> +		return;
>>>>>
>>>>> Remove this check. The driver should not call this with NULL.
>>>>
>>>> Well, netdev_put() has this kind of check. As well as spi_dev_put() or
>>>> i2c_put_adapter() at least. Not sure I would like to avoid a bit of safety.
>>>
>>> IDK, maybe for historical reasons. My point is, id driver is callin
>>> this with NULL, there is something odd in the driver flow. Lets not
>>> allow that for new code.
>>>
>>>
>>>>
>>>>>> +	mutex_lock(&dpll_device_xa_lock);
>>>>>> +	if (refcount_dec_and_test(&dpll->refcount)) {
>>>>>> +		WARN_ON_ONCE(!xa_empty(&dpll->pin_refs));
>>>>>
>>>>> ASSERT_DPLL_NOT_REGISTERED(dpll);
>>>>
>>>> Good point!
>>>>
>>>>>> +		xa_destroy(&dpll->pin_refs);
>>>>>> +		xa_erase(&dpll_device_xa, dpll->id);
>>>>>> +		kfree(dpll);
>>>>>> +	}
>>>>>> +	mutex_unlock(&dpll_device_xa_lock);
>>>>>> +}
>>>>>> +EXPORT_SYMBOL_GPL(dpll_device_put);
>>>>>> +
>>>>>> +/**
>>>>>> + * dpll_device_register - register the dpll device in the subsystem
>>>>>> + * @dpll: pointer to a dpll
>>>>>> + * @type: type of a dpll
>>>>>> + * @ops: ops for a dpll device
>>>>>> + * @priv: pointer to private information of owner
>>>>>> + * @owner: pointer to owner device
>>>>>> + *
>>>>>> + * Make dpll device available for user space.
>>>>>> + *
>>>>>> + * Return:
>>>>>> + * * 0 on success
>>>>>> + * * -EINVAL on failure
>>>>>> + */
>>>>>> +int dpll_device_register(struct dpll_device *dpll, enum dpll_type type,
>>>>>> +			 struct dpll_device_ops *ops, void *priv,
>>>>>> +			 struct device *owner)
>>>>>> +{
>>>>>> +	if (WARN_ON(!ops || !owner))
>>>>>> +		return -EINVAL;
>>>>>> +	if (WARN_ON(type <= DPLL_TYPE_UNSPEC || type > DPLL_TYPE_MAX))
>>>>>> +		return -EINVAL;
>>>>>> +	mutex_lock(&dpll_device_xa_lock);
>>>>>> +	if (ASSERT_DPLL_NOT_REGISTERED(dpll)) {
>>>>>> +		mutex_unlock(&dpll_device_xa_lock);
>>>>>> +		return -EEXIST;
>>>>>> +	}
>>>>>> +	dpll->dev.bus = owner->bus;
>>>>>> +	dpll->parent = owner;
>>>>>> +	dpll->type = type;
>>>>>> +	dpll->ops = ops;
>>>>>> +	dev_set_name(&dpll->dev, "%s_%d", dev_name(owner),
>>>>>> +		     dpll->dev_driver_id);
>>>>>
>>>>> This is really odd. As a result, the user would see something like:
>>>>> pci/0000:01:00.0_1
>>>>> pci/0000:01:00.0_2
>>>>>
>>>>> I have to say it is confusing. In devlink, is bus/name and the user
>>>>> could use this info to look trough sysfs. Here, 0000:01:00.0_1 is not
>>>>> there. Also, "_" might have some meaning on some bus. Should not
>>>>> concatename dev_name() with anything.
>>>>>
>>>>> Thinking about this some more, the module/clock_id tuple should be
>>>>> uniqueue and stable. It is used for dpll_device_get(), it could be used
>>>>> as the user handle, can't it?
>>>>> Example:
>>>>> ice/c92d02a7129f4747
>>>>> mlx5/90265d8bf6e6df56
>>>>>
>>>>> If you really need the "dev_driver_id" (as I believe clock_id should be
>>>>> enough), you can put it here as well:
>>>>> ice/c92d02a7129f4747/1
>>>>> ice/c92d02a7129f4747/2
>>>>>
>>>>
>>>> Looks good, will change it
>>>
>>> Great.
>>>
>>>
>>>>
>>>>> This would also be beneficial for mlx5, as mlx5 with 2 PFs would like to
>>>>> share instance of DPLL equally, there is no "one clock master". >
>>>
>>> [...]
>>>
>>>
>>>>>> +	pin->prop.description = kstrdup(prop->description, GFP_KERNEL);
>>>>>> +	if (!pin->prop.description) {
>>>>>> +		ret = -ENOMEM;
>>>>>> +		goto release;
>>>>>> +	}
>>>>>> +	if (WARN_ON(prop->type <= DPLL_PIN_TYPE_UNSPEC ||
>>>>>> +		    prop->type > DPLL_PIN_TYPE_MAX)) {
>>>>>> +		ret = -EINVAL;
>>>>>> +		goto release;
>>>>>> +	}
>>>>>> +	pin->prop.type = prop->type;
>>>>>> +	pin->prop.capabilities = prop->capabilities;
>>>>>> +	pin->prop.freq_supported = prop->freq_supported;
>>>>>> +	pin->prop.any_freq_min = prop->any_freq_min;
>>>>>> +	pin->prop.any_freq_max = prop->any_freq_max;
>>>>>
>>>>> Make sure that the driver maintains prop (static const) and just save
>>>>> the pointer. Prop does not need to be something driver needs to change.
>>>>>
>>>>
>>>> What's the difference? For ptp_ocp, we have the same configuration for all
>>>> ext pins and the allocator only changes the name of the pin. Properties of
>>>> the DPLL pins are stored within the pin object, not the driver, in this
>>> case.
>>>> Not sure if the pointer way is much better...
>>>
>>> For things like this it is common to have static const array in the
>>> driver, like:
>>>
>>> static const struct dpll_pin_properties dpll_pin_props[] = {
>>> 	{
>>> 		.description = "SMA0",
>>> 		.freq_supported = DPLL_PIN_FREQ_SUPP_MAX,
>>> 		.type = DPLL_PIN_TYPE_EXT,
>>> 		.any_freq_max = 10000000,
>>> 		.capabilities = DPLL_PIN_CAPS_DIRECTION_CAN_CHANGE,
>>> 	},
>>> 	{
>>> 		.description = "SMA1",
>>> 		.freq_supported = DPLL_PIN_FREQ_SUPP_MAX,
>>> 		.type = DPLL_PIN_TYPE_EXT,
>>> 		.any_freq_max = 10000000,
>>> 		.capabilities = DPLL_PIN_CAPS_DIRECTION_CAN_CHANGE,
>>> 	},
>>> 	{
>>> 		.description = "SMA2",
>>> 		.freq_supported = DPLL_PIN_FREQ_SUPP_MAX,
>>> 		.type = DPLL_PIN_TYPE_EXT,
>>> 		.any_freq_max = 10000000,
>>> 		.capabilities = DPLL_PIN_CAPS_DIRECTION_CAN_CHANGE,
>>> 	},
>>> 	{
>>> 		.description = "SMA3",
>>> 		.freq_supported = DPLL_PIN_FREQ_SUPP_MAX,
>>> 		.type = DPLL_PIN_TYPE_EXT,
>>> 		.any_freq_max = 10000000,
>>> 		.capabilities = DPLL_PIN_CAPS_DIRECTION_CAN_CHANGE,
>>> 	},
>>> };
>>>
>>> Here you have very nice list of pins, the reader knows right away what
>>> is happening.
>>>
>>> Thinking about "description" name, I think would be more appropriate to
>>> name this "label" as it represents user-facing label on the connector,
>>> isn't it? Does not describe anything.
>>>
>>
>> "label" seems good.
> 
> Ok.
> 
> 
>>
>>>
>>>>
>>>>>
>>>
>>> [...]
>>>
>>>
>>>>
>>>>>
>>>>>> +	     const struct dpll_pin_properties *prop)
>>>>>> +{
>>>>>> +	struct dpll_pin *pos, *ret = NULL;
>>>>>> +	unsigned long index;
>>>>>> +
>>>>>> +	mutex_lock(&dpll_pin_xa_lock);
>>>>>> +	xa_for_each(&dpll_pin_xa, index, pos) {
>>>>>> +		if (pos->clock_id == clock_id &&
>>>>>> +		    pos->dev_driver_id == device_drv_id &&
>>>>>> +		    pos->module == module) {
>>>>>
>>>>> Compare prop as well.
>>>>>
>>>>> Can't the driver_id (pin index) be something const as well? I think it
>>>>> should. And therefore it could be easily put inside.
>>>>>
>>>>
>>>> I think clock_id + dev_driver_id + module should identify the pin exactly.
>>>> And now I think that *prop is not needed here at all. Arkadiusz, any
>>>> thoughts?
>>>
>>> IDK, no strong opinion on this. I just thought it may help to identify
>>> the pin and avoid potential driver bugs. (Like registering 2 pins with
>>> the same properties).
>>>
>>
>> It would make most sense if pin_index would be a part of *prop.
> 
> Hmm. I see one example where it would not be suitable:
> NIC with N physical ports. You, as a driver, register one pin per
> physical port. You have one static const prop and pass the pointer to if
> for every registration call of N pins, each time with different
> pin_index.
> 
> So from what I see, the prop describes a flavour of pin in the driver.
> In the exaple above, it is still the same SyncE pin, with the same ops.
> Only multiple instances of that distinguished by pin_index and priv.
> 
> Makes sense?
> 
> 
>>
>>> [...]
>>>
>>>
>>>>>> +/**
>>>>>> + * dpll_pin_register - register the dpll pin in the subsystem
>>>>>> + * @dpll: pointer to a dpll
>>>>>> + * @pin: pointer to a dpll pin
>>>>>> + * @ops: ops for a dpll pin ops
>>>>>> + * @priv: pointer to private information of owner
>>>>>> + * @rclk_device: pointer to recovered clock device
>>>>>> + *
>>>>>> + * Return:
>>>>>> + * * 0 on success
>>>>>> + * * -EINVAL - missing dpll or pin
>>>>>> + * * -ENOMEM - failed to allocate memory
>>>>>> + */
>>>>>> +int
>>>>>> +dpll_pin_register(struct dpll_device *dpll, struct dpll_pin *pin,
>>>>>> +		  struct dpll_pin_ops *ops, void *priv,
>>>>>> +		  struct device *rclk_device)
>>>>>
>>>>> Wait a second, what is this "struct device *"? Looks very odd.
>>>>>
>>>>>
>>>>>> +{
>>>>>> +	const char *rclk_name = rclk_device ? dev_name(rclk_device) :
>>>>>> NULL;
>>>>>
>>>>> If you need to store something here, store the pointer to the device
>>>>> directly. But this rclk_device seems odd to me.
>>>>> Dev_name is in case of PCI device for example 0000:01:00.0? That alone
>>>>> is incomplete. What should it server for?
>>>>>
>>>>
>>>> Well, these questions go to Arkadiusz...
>>>
>>
>>
>> [ copy paste my answer from previous response ]
> 
> Does not help, see below.
> 
> 
>> If pin is able to recover signal from some device this shall convey that
>> device struct pointer.
> 
> But one device (struct device instance) could easily have multiple
> netdev instances attached and therefore mutiple sources of recovered
> signal.
> 
> mlxsw driver is one of the examples of this 1:N mapping between device
> and netdev.
> 
> 
>> Name of that device is later passed to the user with DPLL_A_PIN_RCLK_DEVICE
>> attribute.
>> Sure we can have pointer to device and use dev_name (each do/dump) on netlink
>> part. But isn't it better to have the name ready to use there?
>>
>> It might be incomplete only if one device would have some kind of access to
>> a different bus? I don't think it is valid use case.
> 
> Very easily, auxiliary_bus. That is actually the case already for mlx5.
> 
> 
>>
>> Basically the driver will refer only to the devices handled by that driver,
>> which means if dpll is on some bus, also all the pins are there, didn't notice
>> the need to have bus here as well.
> 
> What is the motivation exactly? Is it only SyncE? In case it is, the
> link to the pin should be added from the other side, carried over
> RTNetlink msg of a netdev associated with pin. I will add a patch for
> this.
> 
> Do you need to expose any other recovered clock device source?
> 
> 
>>
>>> Okay.
>>>
>>> [...]
>>>
>>>
>>>>>> + * dpll_pin_get_by_idx - find a pin ref on dpll by pin index
>>>>>> + * @dpll: dpll device pointer
>>>>>> + * @idx: index of pin
>>>>>> + *
>>>>>> + * Find a reference to a pin registered with given dpll and return
>>>>>> its pointer.
>>>>>> + *
>>>>>> + * Return:
>>>>>> + * * valid pointer if pin was found
>>>>>> + * * NULL if not found
>>>>>> + */
>>>>>> +struct dpll_pin *dpll_pin_get_by_idx(struct dpll_device *dpll, u32 idx)
>>>>>> +{
>>>>>> +	struct dpll_pin_ref *pos;
>>>>>> +	unsigned long i;
>>>>>> +
>>>>>> +	xa_for_each(&dpll->pin_refs, i, pos) {
>>>>>> +		if (pos && pos->pin && pos->pin->dev_driver_id == idx)
>>>>>
>>>>> How exactly pos->pin could be NULL?
>>>>>
>>>>> Also, you are degrading the xarray to a mere list here with lookup like
>>>>> this. Why can't you use the pin index coming from driver and
>>>>> insert/lookup based on this index?
>>>>>
>>>> Good point. We just have to be sure, that drivers provide 0-based indexes for
>>>> their pins. I'll re-think it.
>>>
>>> No, driver can provide indexing which is completely up to his decision.
>>> You should use xa_insert() to insert the entry at specific index. See
>>> devl_port_register for inspiration where it is done exactly like this.
>>>
>>> And this should be done in exactly the same way for both pin and device.
>>>
>>
>> Yes, I agree seems doable and better then it is now.
>>
>> [...]
>>
>>>>>> +static int
>>>>>> +dpll_msg_add_dev_handle(struct sk_buff *msg, const struct dpll_device
>>>>>> *dpll)
>>>>>> +{
>>>>>> +	if (nla_put_u32(msg, DPLL_A_ID, dpll->id))
>>>>>
>>>>> Why exactly do we need this dua--handle scheme? Why do you need
>>>>> unpredictable DPLL_A_ID to be exposed to userspace?
>>>>> It's just confusing.
>>>>>
>>>> To be able to work with DPLL per integer after iterator on the list deducts
>>>> which DPLL device is needed. It can reduce the amount of memory copies and
>>>> simplify comparisons. Not sure why it's confusing.
>>>
>>> Wait, I don't get it. Could you please explain a bit more?
>>>
>>> My point is, there should be not such ID exposed over netlink
>>> You don't need to expose it to userspace. The user has well defined
>>> handle as you agreed with above. For example:
>>>
>>> ice/c92d02a7129f4747/1
>>> ice/c92d02a7129f4747/2
>>>
>>> This is shown in dpll device GET/DUMP outputs.
>>> Also user passes it during SET operation:
>>> $ dplltool set ice/c92d02a7129f4747/1 mode auto
>>>
>>> Isn't that enough stable and nice?
>>>
>>
>> I agree with Vadim, this is rather to be used by a daemon tools, which
>> would get the index once, then could use it as long as device is there.
> 
> So basically you say, you can have 2 approaches in app:
> 1)
> id = dpll_device_get_id("ice/c92d02a7129f4747/1")
> dpll_device_set(id, something);
> dpll_device_set(id, something);
> dpll_device_set(id, something);
> 2):
> dpll_device_set("ice/c92d02a7129f4747/1, something);
> dpll_device_set("ice/c92d02a7129f4747/1, something);
> dpll_device_set("ice/c92d02a7129f4747/1, something);
> 
> What is exactly benefit of the first one? Why to have 2 handles? Devlink
> is a nice example of 2) approach, no problem there.
> 
> Perhaps I'm missing something, but looks like you want the id for no
> good reason and this dual-handle scheme just makes things more
> complicated with 0 added value.

I would like to avoid any extra memory copies or memory checks when it's 
possible to compare single u32/u64 index value. I might be invisible on 
a single host setup, but running monitoring at scale which will parse 
and compare string on every get/event can burn a bit of compute capacity.

> 
> 
>>
>>>
>>>>
>>>>>
>>>>>> +		return -EMSGSIZE;
>>>>>> +	if (nla_put_string(msg, DPLL_A_BUS_NAME, dev_bus_name(&dpll-
>>>>>> dev)))
>>>>>> +		return -EMSGSIZE;
>>>>>> +	if (nla_put_string(msg, DPLL_A_DEV_NAME, dev_name(&dpll->dev)))
>>>>>> +		return -EMSGSIZE;
>>>>>> +
>>>>>> +	return 0;
>>>>>> +}
>>>>
>>>> [...]
>>>>
>>>>>> +
>>>>>> +static int
>>>>>> +dpll_msg_add_pins_on_pin(struct sk_buff *msg, struct dpll_pin *pin,
>>>>>> +			 struct netlink_ext_ack *extack)
>>>>>> +{
>>>>>> +	struct dpll_pin_ref *ref = NULL;
>>>>>
>>>>> Why this needs to be initialized?
>>>>>
>>>> No need, fixed.
>>>>
>>>>
>>>>>
>>>>>> +	enum dpll_pin_state state;
>>>>>> +	struct nlattr *nest;
>>>>>> +	unsigned long index;
>>>>>> +	int ret;
>>>>>> +
>>>>>> +	xa_for_each(&pin->parent_refs, index, ref) {
>>>>>> +		if (WARN_ON(!ref->ops->state_on_pin_get))
>>>>>> +			return -EFAULT;
>>>>>> +		ret = ref->ops->state_on_pin_get(pin, ref->pin, &state,
>>>>>> +						 extack);
>>>>>> +		if (ret)
>>>>>> +			return -EFAULT;
>>>>>> +		nest = nla_nest_start(msg, DPLL_A_PIN_PARENT);
>>>>>> +		if (!nest)
>>>>>> +			return -EMSGSIZE;
>>>>>> +		if (nla_put_u32(msg, DPLL_A_PIN_PARENT_IDX,
>>>>>> +				ref->pin->dev_driver_id)) {
>>>>>> +			ret = -EMSGSIZE;
>>>>>> +			goto nest_cancel;
>>>>>> +		}
>>>>>> +		if (nla_put_u8(msg, DPLL_A_PIN_STATE, state)) {
>>>>>> +			ret = -EMSGSIZE;
>>>>>> +			goto nest_cancel;
>>>>>> +		}
>>>>>> +		nla_nest_end(msg, nest);
>>>>>> +	}
>>>>>
>>>>> How is this function different to dpll_msg_add_pin_parents()?
>>>>> Am I lost? To be honest, this x_on_pin/dpll, parent, refs dance is quite
>>>>> hard to follow for me :/
>>>>>
>>>>> Did you get lost here as well? If yes, this needs some serious think
>>>>> through :)
>>>>>
>>>>
>>>> Let's re-think it again. Arkadiuzs, do you have clear explanation of the
>>>> relationship between these things?
>>>
>>
>> [ copy paste my answer from previous response ]
>> No, it is just leftover I didn't catch, we can leave one function and use it in
>> both cases. Sorry about that, great catch!
> 
> Hmm, ok. I'm still lost a bit with all the referencing and cross
> referencing, I wonder if it could be made a bit simpler and/or easier to
> read and follow.
> 
> 
>>
>> [...]
>>
>>>>>> +	if (nla_put_u32(msg, DPLL_A_PIN_IDX, pin->dev_driver_id))
>>>>>
>>>>> Is it "ID" or "INDEX" (IDX). Please make this consistent in the whole
>>>>> code.
>>>>>
>>>>
>>>> I believe it's INDEX which is provided by the driver. I'll think about
>>>> renaming,
>>>> but suggestions are welcome.
>>>
>>> Let's use "index" and "INDEX" internalla and in Netlink attr names as
>>> well then.
>>>
>>
>> For me makes sense to have a common name instead of origin-based one.
> 
> What do you mean by this?
> 
> 
>>
>>> [...]
>>>
>>>
>>>>
>>>>>
>>>>>> +	int rem, ret = -EINVAL;
>>>>>> +	struct nlattr *a;
>>>>>> +
>>>>>> +	nla_for_each_attr(a, genlmsg_data(info->genlhdr),
>>>>>> +			  genlmsg_len(info->genlhdr), rem) {
>>>>>
>>>>> This is odd. Why you iterace over attrs? Why don't you just access them
>>>>> directly, like attrs[DPLL_A_PIN_FREQUENCY] for example?
>>>>>
>>>>
>>>> I had some unknown crashes when I was using such access. I might have lost
>>>> some checks, will try it again.
>>>
>>> Odd, yet definitelly debuggable though :)
>>>
>>> [...]
>>>
>>>
>>>>>> +int dpll_pin_pre_dumpit(struct netlink_callback *cb)
>>>>>> +{
>>>>>> +	mutex_lock(&dpll_pin_xa_lock);
>>>>>
>>>>> ABBA deadlock here, see dpll_pin_register() for example where the lock
>>>>> taking order is opposite.
>>>>>
>>>>
>>>> Now I see an ABBA deadlock here, as well as in function before. Not sure
>>>> how to
>>>> solve it here. Any thoughts?
>>>
>>> Well, here you can just call dpll_pre_dumpit() before
>>> mutex_lock(&dpll_pin_xa_lock)
>>> to take the locks in the same order.
>>>
>>
>> This double lock doesn't really improve anything.
>> Any objections on having single mutex/lock for access the dpll subsystem?
> 
> Yeah, we had it in devlink and then we spent a lot of a time to get rid
> of it. Now we have per-instance locking. Here, it would be
> per-dpll-device locking.
> 
> No strong opinion, perhaps on case of dpll one master lock is enough.
> Perhaps Jakub would have some opinion on this.
> 
> 
>>
>>>
>>>>
>>>>>
>>>>>> +
>>>>>> +	return dpll_pre_dumpit(cb);
>>>>>> +}
>>>>>> +
>>>>>> +int dpll_pin_post_dumpit(struct netlink_callback *cb)
>>>>>> +{
>>>>>> +	mutex_unlock(&dpll_pin_xa_lock);
>>>>>> +
>>>>>> +	return dpll_post_dumpit(cb);
>>>>>> +}
>>>>>> +
>>>>>> +static int
>>>>>> +dpll_event_device_change(struct sk_buff *msg, struct dpll_device
>>>>>> *dpll,
>>>>>> +			 struct dpll_pin *pin, struct dpll_pin *parent,
>>>>>> +			 enum dplla attr)
>>>>>> +{
>>>>>> +	int ret = dpll_msg_add_dev_handle(msg, dpll);
>>>>>> +	struct dpll_pin_ref *ref = NULL;
>>>>>> +	enum dpll_pin_state state;
>>>>>> +
>>>>>> +	if (ret)
>>>>>> +		return ret;
>>>>>> +	if (pin && nla_put_u32(msg, DPLL_A_PIN_IDX, pin-
>>>>>> dev_driver_id))
>>>>>> +		return -EMSGSIZE;
>>>>>
>>>>> I don't really understand why you are trying figure something new and
>>>>> interesting with the change notifications. This object mix and random
>>>>> attrs fillup is something very wrong and makes userspace completely
>>>>> fuzzy about what it is getting. And yet it is so simple:
>>>>> You have 2 objects, dpll and pin, please just have:
>>>>> dpll_notify()
>>>>> dpll_pin_notify()
>>>>> and share the attrs fillup code with pin_get() and dpll_get() callbacks.
>>>>> No need for any smartness. Have this dumb and simple.
>>>>>
>>>>> Think about it more as about "object-state-snapshot" than "atomic-change"
>>>>
>>>> But with full object-snapshot user space app will lose the information about
>>>> what exactly has changed. The reason to have this event is to provide the
>>>> attributes which have changed. Otherwise, the app should have full snapshot
>>>> and
>>>> compare all attributes to figure out changes and that's might not be great
>>>> idea.
>>>
>>> Wait, are you saying that the app is stateless? Could you provide
>>> example use cases?
>>>
>> >From what I see, the app managing dpll knows the state of the device and
>>> pins, it monitors for the changes and saves new state with appropriate
>>> reaction (might be some action or maybe just log entry).
>>>
>>
>> It depends on the use case, right? App developer having those information knows
>> what has changed, thus can react in a way it thinks is most suitable.
>> IMHO from user perspective it is good to have a notification which actually
>> shows it's reason, so proper flow could be assigned to handle the reaction.
> 
> Again, could you provide me specific example in which this is needed?
> I may be missing something, but I don't see how it can bring
> and benefit. It just makes the live of the app harder because it has to
> treat the get and notify messages differently.
> 
> It is quite common for app to:
> init:
> 1) get object state
> 2) store it
> 3) apply configuration
> runtime:
> 1) listen to object state change
> 2) store it
> 3) apply configuration
> 
> Same code for both.

Well, I'm thinking about simple monitoring app which will wait for the 
events and create an alert if the changes coming from the event differ 
from the "allowed configs". In this case no real reason to store whole 
object state, but the changes in events are very useful.

> 
> 
> 
> 
>>
>>>
>>>>
>>>>>
>>>>>> +
>>>>>> +	switch (attr) {
>>>>>> +	case DPLL_A_MODE:
>>>>>> +		ret = dpll_msg_add_mode(msg, dpll, NULL);
>>>>>> +		break;
>>>>>> +	case DPLL_A_SOURCE_PIN_IDX:
>>>>>> +		ret = dpll_msg_add_source_pin_idx(msg, dpll, NULL);
>>>>>> +		break;
>>>>>> +	case DPLL_A_LOCK_STATUS:
>>>>>> +		ret = dpll_msg_add_lock_status(msg, dpll, NULL);
>>>>>> +		break;
>>>>>> +	case DPLL_A_TEMP:
>>>>>> +		ret = dpll_msg_add_temp(msg, dpll, NULL);
>>>>>> +		break;
>>>>>> +	case DPLL_A_PIN_FREQUENCY:
>>>>>> +		ret = dpll_msg_add_pin_freq(msg, pin, NULL, false);
>>>>>> +		break;
>>>>>> +	case DPLL_A_PIN_PRIO:
>>>>>> +		ref = dpll_xa_ref_dpll_find(&pin->dpll_refs, dpll);
>>>>>> +		if (!ref)
>>>>>> +			return -EFAULT;
>>>>>> +		ret = dpll_msg_add_pin_prio(msg, pin, ref, NULL);
>>>>>> +		break;
>>>>>> +	case DPLL_A_PIN_STATE:
>>>>>> +		if (parent) {
>>>>>> +			ref = dpll_xa_ref_pin_find(&pin->parent_refs, parent);
>>>>>> +			if (!ref)
>>>>>> +				return -EFAULT;
>>>>>> +			if (!ref->ops || !ref->ops->state_on_pin_get)
>>>>>> +				return -EOPNOTSUPP;
>>>>>> +			ret = ref->ops->state_on_pin_get(pin, parent, &state,
>>>>>> +							 NULL);
>>>>>> +			if (ret)
>>>>>> +				return ret;
>>>>>> +			if (nla_put_u32(msg, DPLL_A_PIN_PARENT_IDX,
>>>>>> +					parent->dev_driver_id))
>>>>>> +				return -EMSGSIZE;
>>>>>> +		} else {
>>>>>> +			ref = dpll_xa_ref_dpll_find(&pin->dpll_refs, dpll);
>>>>>> +			if (!ref)
>>>>>> +				return -EFAULT;
>>>>>> +			ret = dpll_msg_add_pin_on_dpll_state(msg, pin, ref,
>>>>>> +							     NULL);
>>>>>> +			if (ret)
>>>>>> +				return ret;
>>>>>> +		}
>>>>>> +		break;
>>>>>> +	default:
>>>>>> +		break;
>>>>>> +	}
>>>>>> +
>>>>>> +	return ret;
>>>>>> +}
>>>>>> +
>>>>>> +static int
>>>>>> +dpll_send_event_create(enum dpll_event event, struct dpll_device *dpll)
>>>>>> +{
>>>>>> +	struct sk_buff *msg;
>>>>>> +	int ret = -EMSGSIZE;
>>>>>> +	void *hdr;
>>>>>> +
>>>>>> +	msg = genlmsg_new(NLMSG_GOODSIZE, GFP_KERNEL);
>>>>>> +	if (!msg)
>>>>>> +		return -ENOMEM;
>>>>>> +
>>>>>> +	hdr = genlmsg_put(msg, 0, 0, &dpll_nl_family, 0, event);
>>>>>> +	if (!hdr)
>>>>>> +		goto out_free_msg;
>>>>>> +
>>>>>> +	ret = dpll_msg_add_dev_handle(msg, dpll);
>>>>>> +	if (ret)
>>>>>> +		goto out_cancel_msg;
>>>>>> +	genlmsg_end(msg, hdr);
>>>>>> +	genlmsg_multicast(&dpll_nl_family, msg, 0, 0, GFP_KERNEL);
>>>>>> +
>>>>>> +	return 0;
>>>>>> +
>>>>>> +out_cancel_msg:
>>>>>> +	genlmsg_cancel(msg, hdr);
>>>>>> +out_free_msg:
>>>>>> +	nlmsg_free(msg);
>>>>>> +
>>>>>> +	return ret;
>>>>>> +}
>>>>>> +
>>>>>> +static int
>>>>>> +dpll_send_event_change(struct dpll_device *dpll, struct dpll_pin *pin,
>>>>>> +		       struct dpll_pin *parent, enum dplla attr)
>>>>>> +{
>>>>>> +	struct sk_buff *msg;
>>>>>> +	int ret = -EMSGSIZE;
>>>>>> +	void *hdr;
>>>>>> +
>>>>>> +	msg = genlmsg_new(NLMSG_GOODSIZE, GFP_KERNEL);
>>>>>> +	if (!msg)
>>>>>> +		return -ENOMEM;
>>>>>> +
>>>>>> +	hdr = genlmsg_put(msg, 0, 0, &dpll_nl_family, 0,
>>>>>> +			  DPLL_EVENT_DEVICE_CHANGE);
>>>>>
>>>>> I don't really get it. Why exactly you keep having this *EVENT* cmds?
>>>>> Why per-object NEW/GET/DEL cmds shared with get genl op are not enough?
>>>>> I have to be missing something.
>>>>
>>>> Changes might come from other places, but will affect the DPLL device and we
>>>> have to notify users in this case.
>>>
>>> I'm not sure I follow. There are 2 scenarios for change:
>>> 1) user originated - user issues set of something
>>> 2) driver originated - something changes in HW, driver propagates that
>>>
>>> With what I suggest, both scenarios work of course. My point is, user
>>> knows very well the objects: device and pin, he knows the format or
>>> messages that are related to GET/DUMP/SET operations of both. The
>>> notification should have the same format, that is all I say.
>>>
>>> Btw, you can see devlink code for example how the notifications like
>>> this are implemented and work.
>>>
>>
>> Devlink packs all the info into a netlink message and notifies with it, isn't
>> it that it has all the info "buffered" in its structures?
> 
> Not really. Ops are there as well.
> 
> 
>> A dpll subsystem keeps only some of the info in its internal structures, but
>> for most of it it has to question the driver. It would do it multiple times
>> i.e. if there would be a change on a pin connected to multiple dpll devices.
>>
>> With current approach the notifications are pretty light to the subsystem as
>> well as for the registered clients, and as you suggested with having info of
>> what actually changed the userspace could implement a stateless daemon right
>> away.
> 
> Okay, examples? I really can't see how this stateless deamon could be
> implemented, but perhaps I lack better imagination :/
> 
> 
>>
>> Thank you,
>> Arkadiusz

Jiri Pirko April 1, 2023, 12:49 p.m. UTC | #22

Tue, Mar 28, 2023 at 05:22:04PM CEST, [e-mail] wrote:
>On 15/03/2023 09:22, Jiri Pirko wrote:
>> Tue, Mar 14, 2023 at 06:50:57PM CET, [e-mail] wrote:
>> > > From: Jiri Pirko <[e-mail]>
>> > > Sent: Tuesday, March 14, 2023 10:22 AM
>> > > 
>> > > Mon, Mar 13, 2023 at 11:59:32PM CET, [e-mail] wrote:
>> > > > On 13.03.2023 16:21, Jiri Pirko wrote:
>> > > > > Sun, Mar 12, 2023 at 03:28:03AM CET, [e-mail] wrote:

[...]


>> > > > > > +static int
>> > > > > > +dpll_msg_add_dev_handle(struct sk_buff *msg, const struct dpll_device
>> > > > > > *dpll)
>> > > > > > +{
>> > > > > > +	if (nla_put_u32(msg, DPLL_A_ID, dpll->id))
>> > > > > 
>> > > > > Why exactly do we need this dua--handle scheme? Why do you need
>> > > > > unpredictable DPLL_A_ID to be exposed to userspace?
>> > > > > It's just confusing.
>> > > > > 
>> > > > To be able to work with DPLL per integer after iterator on the list deducts
>> > > > which DPLL device is needed. It can reduce the amount of memory copies and
>> > > > simplify comparisons. Not sure why it's confusing.
>> > > 
>> > > Wait, I don't get it. Could you please explain a bit more?
>> > > 
>> > > My point is, there should be not such ID exposed over netlink
>> > > You don't need to expose it to userspace. The user has well defined
>> > > handle as you agreed with above. For example:
>> > > 
>> > > ice/c92d02a7129f4747/1
>> > > ice/c92d02a7129f4747/2
>> > > 
>> > > This is shown in dpll device GET/DUMP outputs.
>> > > Also user passes it during SET operation:
>> > > $ dplltool set ice/c92d02a7129f4747/1 mode auto
>> > > 
>> > > Isn't that enough stable and nice?
>> > > 
>> > 
>> > I agree with Vadim, this is rather to be used by a daemon tools, which
>> > would get the index once, then could use it as long as device is there.
>> 
>> So basically you say, you can have 2 approaches in app:
>> 1)
>> id = dpll_device_get_id("ice/c92d02a7129f4747/1")
>> dpll_device_set(id, something);
>> dpll_device_set(id, something);
>> dpll_device_set(id, something);
>> 2):
>> dpll_device_set("ice/c92d02a7129f4747/1, something);
>> dpll_device_set("ice/c92d02a7129f4747/1, something);
>> dpll_device_set("ice/c92d02a7129f4747/1, something);
>> 
>> What is exactly benefit of the first one? Why to have 2 handles? Devlink
>> is a nice example of 2) approach, no problem there.
>> 
>> Perhaps I'm missing something, but looks like you want the id for no
>> good reason and this dual-handle scheme just makes things more
>> complicated with 0 added value.
>
>I would like to avoid any extra memory copies or memory checks when it's
>possible to compare single u32/u64 index value. I might be invisible on a
>single host setup, but running monitoring at scale which will parse and
>compare string on every get/event can burn a bit of compute capacity.

Wait, that does not make any sense what so ever.
Show me numbers and real usecase. A sane app gets once at start, then
processes notifications. You have a flow where string compare on a
netlink command makes difference to int compare? Like tens of thousands
of netlink cmds per second? I doubt that. If yes, it is a misuse. Btw,
the string compare would be your last problem comparing the overhead of
Netlink processing with ioctl for example.

Your dual handle scheme just adds complexicity, confusion with 0 added
value. Please drop it. I really don't understand the need to defend this
odd approach :/

[...]


>> > > > > > +
>> > > > > > +	return dpll_pre_dumpit(cb);
>> > > > > > +}
>> > > > > > +
>> > > > > > +int dpll_pin_post_dumpit(struct netlink_callback *cb)
>> > > > > > +{
>> > > > > > +	mutex_unlock(&dpll_pin_xa_lock);
>> > > > > > +
>> > > > > > +	return dpll_post_dumpit(cb);
>> > > > > > +}
>> > > > > > +
>> > > > > > +static int
>> > > > > > +dpll_event_device_change(struct sk_buff *msg, struct dpll_device
>> > > > > > *dpll,
>> > > > > > +			 struct dpll_pin *pin, struct dpll_pin *parent,
>> > > > > > +			 enum dplla attr)
>> > > > > > +{
>> > > > > > +	int ret = dpll_msg_add_dev_handle(msg, dpll);
>> > > > > > +	struct dpll_pin_ref *ref = NULL;
>> > > > > > +	enum dpll_pin_state state;
>> > > > > > +
>> > > > > > +	if (ret)
>> > > > > > +		return ret;
>> > > > > > +	if (pin && nla_put_u32(msg, DPLL_A_PIN_IDX, pin-
>> > > > > > dev_driver_id))
>> > > > > > +		return -EMSGSIZE;
>> > > > > 
>> > > > > I don't really understand why you are trying figure something new and
>> > > > > interesting with the change notifications. This object mix and random
>> > > > > attrs fillup is something very wrong and makes userspace completely
>> > > > > fuzzy about what it is getting. And yet it is so simple:
>> > > > > You have 2 objects, dpll and pin, please just have:
>> > > > > dpll_notify()
>> > > > > dpll_pin_notify()
>> > > > > and share the attrs fillup code with pin_get() and dpll_get() callbacks.
>> > > > > No need for any smartness. Have this dumb and simple.
>> > > > > 
>> > > > > Think about it more as about "object-state-snapshot" than "atomic-change"
>> > > > 
>> > > > But with full object-snapshot user space app will lose the information about
>> > > > what exactly has changed. The reason to have this event is to provide the
>> > > > attributes which have changed. Otherwise, the app should have full snapshot
>> > > > and
>> > > > compare all attributes to figure out changes and that's might not be great
>> > > > idea.
>> > > 
>> > > Wait, are you saying that the app is stateless? Could you provide
>> > > example use cases?
>> > > 
>> > >From what I see, the app managing dpll knows the state of the device and
>> > > pins, it monitors for the changes and saves new state with appropriate
>> > > reaction (might be some action or maybe just log entry).
>> > > 
>> > 
>> > It depends on the use case, right? App developer having those information knows
>> > what has changed, thus can react in a way it thinks is most suitable.
>> > IMHO from user perspective it is good to have a notification which actually
>> > shows it's reason, so proper flow could be assigned to handle the reaction.
>> 
>> Again, could you provide me specific example in which this is needed?
>> I may be missing something, but I don't see how it can bring
>> and benefit. It just makes the live of the app harder because it has to
>> treat the get and notify messages differently.
>> 
>> It is quite common for app to:
>> init:
>> 1) get object state
>> 2) store it
>> 3) apply configuration
>> runtime:
>> 1) listen to object state change
>> 2) store it
>> 3) apply configuration
>> 
>> Same code for both.
>
>Well, I'm thinking about simple monitoring app which will wait for the events
>and create an alert if the changes coming from the event differ from the
>"allowed configs". In this case no real reason to store whole object state,
>but the changes in events are very useful.

No problem. But as I wrote elsewhere in this thread, make sure that msg
format of a change message is the same as the format of get cmd message.
Basically think of it as a get cmd message filtered to have only
attrs which changed. Same nesting and everything. Makes it simple for
the app to have same parsing code for get/dump/notification messages.

[...]

Jakub Kicinski April 3, 2023, 6:18 p.m. UTC | #23

On Wed, 15 Mar 2023 10:22:33 +0100 Jiri Pirko wrote:
> So basically you say, you can have 2 approaches in app:
> 1)
> id = dpll_device_get_id("ice/c92d02a7129f4747/1")
> dpll_device_set(id, something);
> dpll_device_set(id, something);
> dpll_device_set(id, something);
> 2):
> dpll_device_set("ice/c92d02a7129f4747/1, something);
> dpll_device_set("ice/c92d02a7129f4747/1, something);
> dpll_device_set("ice/c92d02a7129f4747/1, something);
> 
> What is exactly benefit of the first one? Why to have 2 handles? Devlink
> is a nice example of 2) approach, no problem there.

IMHO for devlink the neatness of using the name came from the fact 
that the device name was meaningful. 

With the advent of auxbus that's no longer the case.

In fact it seems more than likely that changing the name to auxbus
will break FW update scripts. Maybe nobody has complained yet only
because prod adoption of these APIs is generally lacking :(

I agree that supporting both name and ID is pointless, user space can
translate between the two trivially all by itself. But I'd lean towards
deleting the name support not the ID support :(

Jiri Pirko April 9, 2023, 7:51 a.m. UTC | #24

Mon, Apr 03, 2023 at 08:18:12PM CEST, [e-mail] wrote:
>On Wed, 15 Mar 2023 10:22:33 +0100 Jiri Pirko wrote:
>> So basically you say, you can have 2 approaches in app:
>> 1)
>> id = dpll_device_get_id("ice/c92d02a7129f4747/1")
>> dpll_device_set(id, something);
>> dpll_device_set(id, something);
>> dpll_device_set(id, something);
>> 2):
>> dpll_device_set("ice/c92d02a7129f4747/1, something);
>> dpll_device_set("ice/c92d02a7129f4747/1, something);
>> dpll_device_set("ice/c92d02a7129f4747/1, something);
>> 
>> What is exactly benefit of the first one? Why to have 2 handles? Devlink
>> is a nice example of 2) approach, no problem there.
>
>IMHO for devlink the neatness of using the name came from the fact 
>that the device name was meaningful. 
>
>With the advent of auxbus that's no longer the case.
>
>In fact it seems more than likely that changing the name to auxbus
>will break FW update scripts. Maybe nobody has complained yet only
>because prod adoption of these APIs is generally lacking :(
>
>I agree that supporting both name and ID is pointless, user space can
>translate between the two trivially all by itself. But I'd lean towards
>deleting the name support not the ID support :(

Wait, not sure you get the format of the "name". It does not contain any
bus address, so the auxdev issue you pointed out is not applicable.
It is driver/clock_id/index.
All 3 are stable and user can rely on them. Do you see any issue in
that?

Jiri Pirko April 16, 2023, 4:23 p.m. UTC | #25

Tue, Apr 11, 2023 at 12:31:49AM CEST, [e-mail] wrote:
>On Sun, 9 Apr 2023 09:51:48 +0200 Jiri Pirko wrote:
>> Wait, not sure you get the format of the "name". It does not contain any
>> bus address, so the auxdev issue you pointed out is not applicable.
>> It is driver/clock_id/index.
>> All 3 are stable and user can rely on them. Do you see any issue in
>> that?
>
>What is index? I thought you don't want an index and yet there is one,
>just scoped by random attributes :(

Index internal within a single instance. Like Intel guys, they have 1
clock wired up with multiple DPLLs. The driver gives every DPLL index.
This is internal, totally up to the driver decision. Similar concept to
devlink port index.

Vadim Fedorenko April 17, 2023, 3:53 p.m. UTC | #26

On 16/04/2023 17:23, Jiri Pirko wrote:
> Tue, Apr 11, 2023 at 12:31:49AM CEST, [e-mail] wrote:
>> On Sun, 9 Apr 2023 09:51:48 +0200 Jiri Pirko wrote:
>>> Wait, not sure you get the format of the "name". It does not contain any
>>> bus address, so the auxdev issue you pointed out is not applicable.
>>> It is driver/clock_id/index.
>>> All 3 are stable and user can rely on them. Do you see any issue in
>>> that?
>>
>> What is index? I thought you don't want an index and yet there is one,
>> just scoped by random attributes :(
> 
> Index internal within a single instance. Like Intel guys, they have 1
> clock wired up with multiple DPLLs. The driver gives every DPLL index.
> This is internal, totally up to the driver decision. Similar concept to
> devlink port index.

It feels like a dead-lock in conversation here. We have to agree on 
something because for now it's the only blocker to post the next version 
with all the comments from the previous one addressed in the code.
My position here is that I'm ok to have any of the properties being an 
identifier as well as keep both of them, the code already has all the 
lines to support any decision. I just to want to go back to this part 
again in the next iteration.

Paolo Abeni April 27, 2023, 8:05 a.m. UTC | #27

Hi,

On Mon, 2023-04-17 at 12:49 -0700, Jakub Kicinski wrote:
> [resend with fixed CC list]
> 
> On Sun, 16 Apr 2023 18:23:15 +0200 Jiri Pirko wrote:
> > > What is index? I thought you don't want an index and yet there is one,
> > > just scoped by random attributes :(  
> > 
> > Index internal within a single instance. Like Intel guys, they have 1
> > clock wired up with multiple DPLLs. The driver gives every DPLL index.
> > This is internal, totally up to the driver decision. Similar concept to
> > devlink port index.
> 
> devlink port index ended up as a pretty odd beast with drivers encoding
> various information into it, using locally grown schemes.
> 
> Hard no on doing that in dpll, it should not be exposed to the user.

I'm replying here just in case the even the above reply was lost.

I guess the last remark resolved this specific discussion.

@Vadim, @Arkadiusz, even if net-next is currently closed, there are no
restrictions for RFC patches: feel free to share v7 when it fits you
better!

Thanks,

Paolo

Vadim Fedorenko April 27, 2023, 10:20 a.m. UTC | #28

On 27/04/2023 09:05, Paolo Abeni wrote:
> Hi,
> 
> On Mon, 2023-04-17 at 12:49 -0700, Jakub Kicinski wrote:
>> [resend with fixed CC list]
>>
>> On Sun, 16 Apr 2023 18:23:15 +0200 Jiri Pirko wrote:
>>>> What is index? I thought you don't want an index and yet there is one,
>>>> just scoped by random attributes :(
>>>
>>> Index internal within a single instance. Like Intel guys, they have 1
>>> clock wired up with multiple DPLLs. The driver gives every DPLL index.
>>> This is internal, totally up to the driver decision. Similar concept to
>>> devlink port index.
>>
>> devlink port index ended up as a pretty odd beast with drivers encoding
>> various information into it, using locally grown schemes.
>>
>> Hard no on doing that in dpll, it should not be exposed to the user.
> 
> I'm replying here just in case the even the above reply was lost.
> 
> I guess the last remark resolved this specific discussion.
> 
> @Vadim, @Arkadiusz, even if net-next is currently closed, there are no
> restrictions for RFC patches: feel free to share v7 when it fits you
> better!

Hi Paolo!
Thanks for reminder, the new version is almost ready, I was a bit 
distracted by other burning issues. Hopefully I'll publish it till the 
end of the week.

Jiri Pirko May 3, 2023, 7:56 a.m. UTC | #29

Adding back the cclist stripped due to Claws bug.

Tue, May 02, 2023 at 05:32:44PM CEST, [e-mail] wrote:
>On Tue, 2 May 2023 10:52:57 +0200 Jiri Pirko wrote:
>> >> Index internal within a single instance. Like Intel guys, they have 1
>> >> clock wired up with multiple DPLLs. The driver gives every DPLL index.
>> >> This is internal, totally up to the driver decision. Similar concept to
>> >> devlink port index.  
>> >
>> >devlink port index ended up as a pretty odd beast with drivers encoding
>> >various information into it, using locally grown schemes.
>> >
>> >Hard no on doing that in dpll, it should not be exposed to the user.  
>> 
>> So you say to have ID fully dynamic and non deterministic? I'm lost a
>> bit.
>
>Yup, non-deterministic, just a cyclic ID allocated by the core starting
>from 1. Finding the right device / pin needs to be done via
>informational attributes not making assumptions about the ID.

Okay.

When netdev will have pin ID in the RT netlink message (as it is done
in RFCv7), it is easy to get the pin/dpll for netdev. No problem there.

However, for non-SyncE usecase, how do you imagine scripts to work?
I mean, the script have to obtain dpll/pin ID by deterministic
module_name/clock_id/idx tuple.

There are 2 options to do that:
1) dump all dplls/pins and do lookup in userspace
2) get a dpll/pin according to given module_name/clock_id/idx tuple

The first approach is not very nice.
The currently pushed RFCv7 of the patchset does not support 2)

Now if we add support for 2), we basically use module_name/clock_id/idx
as a handle for "get cmd". My point is, why can't we use it for "set
cmd" as well and avoid the ID entirely?

What am I missing here?

Jakub Kicinski May 4, 2023, 2:16 a.m. UTC | #30

On Wed, 3 May 2023 09:56:57 +0200 Jiri Pirko wrote:
> >Yup, non-deterministic, just a cyclic ID allocated by the core starting
> >from 1. Finding the right device / pin needs to be done via
> >informational attributes not making assumptions about the ID.  
> 
> Okay.
> 
> When netdev will have pin ID in the RT netlink message (as it is done
> in RFCv7), it is easy to get the pin/dpll for netdev. No problem there.
> 
> However, for non-SyncE usecase, how do you imagine scripts to work?
> I mean, the script have to obtain dpll/pin ID by deterministic
> module_name/clock_id/idx tuple.

No scoped idx.

> There are 2 options to do that:
> 1) dump all dplls/pins and do lookup in userspace
> 2) get a dpll/pin according to given module_name/clock_id/idx tuple
> 
> The first approach is not very nice.
> The currently pushed RFCv7 of the patchset does not support 2)
> 
> Now if we add support for 2), we basically use module_name/clock_id/idx
> as a handle for "get cmd". My point is, why can't we use it for "set
> cmd" as well and avoid the ID entirely?

Sure, we don't _have_ to have an ID, but it seems go against normal
data normalization rules. And I don't see any harm in it.

But you're asking for per-device "idx" and that's a no-go for me,
given already cited experience.

The user space can look up the ID based on identifying information it
has. IMO it's better to support multiple different intelligible elements
than single integer index into which drivers will start encoding all
sort of info, using locally invented schemes.

Jiri Pirko May 4, 2023, 11 a.m. UTC | #31

Thu, May 04, 2023 at 04:16:43AM CEST, [e-mail] wrote:
>On Wed, 3 May 2023 09:56:57 +0200 Jiri Pirko wrote:
>> >Yup, non-deterministic, just a cyclic ID allocated by the core starting
>> >from 1. Finding the right device / pin needs to be done via
>> >informational attributes not making assumptions about the ID.  
>> 
>> Okay.
>> 
>> When netdev will have pin ID in the RT netlink message (as it is done
>> in RFCv7), it is easy to get the pin/dpll for netdev. No problem there.
>> 
>> However, for non-SyncE usecase, how do you imagine scripts to work?
>> I mean, the script have to obtain dpll/pin ID by deterministic
>> module_name/clock_id/idx tuple.
>
>No scoped idx.

That means, no index defined by a driver if I undestand you correctly,
right?


>
>> There are 2 options to do that:
>> 1) dump all dplls/pins and do lookup in userspace
>> 2) get a dpll/pin according to given module_name/clock_id/idx tuple
>> 
>> The first approach is not very nice.
>> The currently pushed RFCv7 of the patchset does not support 2)
>> 
>> Now if we add support for 2), we basically use module_name/clock_id/idx
>> as a handle for "get cmd". My point is, why can't we use it for "set
>> cmd" as well and avoid the ID entirely?
>
>Sure, we don't _have_ to have an ID, but it seems go against normal
>data normalization rules. And I don't see any harm in it.
>
>But you're asking for per-device "idx" and that's a no-go for me,
>given already cited experience.
>
>The user space can look up the ID based on identifying information it
>has. IMO it's better to support multiple different intelligible elements

Do you mean fixed tuple or variable tuple?

CMD_GET_ID
  -> DPLL_A_MODULE_NAME
     DPLL_A_CLOCK_ID
  <- DPLL_A_ID

CMD_GET_PIN_ID
  -> DPLL_A_MODULE_NAME
     DPLL_A_CLOCK_ID
  <- DPLL_A_PIN_ID



>than single integer index into which drivers will start encoding all
>sort of info, using locally invented schemes.

There could be multiple DPLL and pin instances for a single
module/clock_id tuple we have to distinguish somehow. If the driver
can't pass "index" of DPLL or a pin, how we distinguish them?

Plus is is possible that 2 driver instances share the same dpll
instance, then to get the dpll pointer reference, they do:
INSTANCE A:
dpll_0 = dpll_device_get(clock_id, 0, THIS_MODULE);
dpll_1 = dpll_device_get(clock_id, 1, THIS_MODULE);

INSTANCE B:
dpll_0 = dpll_device_get(clock_id, 0, THIS_MODULE);
dpll_1 = dpll_device_get(clock_id, 1, THIS_MODULE);

My point is, event if we don't expose the index to the userspace,
we need to have it internally.

Jiri Pirko May 4, 2023, 11:14 a.m. UTC | #32

Thu, May 04, 2023 at 01:00:42PM CEST, [e-mail] wrote:
>Thu, May 04, 2023 at 04:16:43AM CEST, [e-mail] wrote:
>>On Wed, 3 May 2023 09:56:57 +0200 Jiri Pirko wrote:
>>> >Yup, non-deterministic, just a cyclic ID allocated by the core starting
>>> >from 1. Finding the right device / pin needs to be done via
>>> >informational attributes not making assumptions about the ID.  
>>> 
>>> Okay.
>>> 
>>> When netdev will have pin ID in the RT netlink message (as it is done
>>> in RFCv7), it is easy to get the pin/dpll for netdev. No problem there.
>>> 
>>> However, for non-SyncE usecase, how do you imagine scripts to work?
>>> I mean, the script have to obtain dpll/pin ID by deterministic
>>> module_name/clock_id/idx tuple.
>>
>>No scoped idx.
>
>That means, no index defined by a driver if I undestand you correctly,
>right?
>
>
>>
>>> There are 2 options to do that:
>>> 1) dump all dplls/pins and do lookup in userspace
>>> 2) get a dpll/pin according to given module_name/clock_id/idx tuple
>>> 
>>> The first approach is not very nice.
>>> The currently pushed RFCv7 of the patchset does not support 2)
>>> 
>>> Now if we add support for 2), we basically use module_name/clock_id/idx
>>> as a handle for "get cmd". My point is, why can't we use it for "set
>>> cmd" as well and avoid the ID entirely?
>>
>>Sure, we don't _have_ to have an ID, but it seems go against normal
>>data normalization rules. And I don't see any harm in it.
>>
>>But you're asking for per-device "idx" and that's a no-go for me,
>>given already cited experience.
>>
>>The user space can look up the ID based on identifying information it
>>has. IMO it's better to support multiple different intelligible elements
>
>Do you mean fixed tuple or variable tuple?
>
>CMD_GET_ID
>  -> DPLL_A_MODULE_NAME
>     DPLL_A_CLOCK_ID

Sorry, I hit the send button by a mistake.
I ment to add a question here:
What is the next intelligible element to identify DPLL device here?

>  <- DPLL_A_ID
>
>CMD_GET_PIN_ID
>  -> DPLL_A_MODULE_NAME
>     DPLL_A_CLOCK_ID

What is the next intelligible element to identify a pin here?

>  <- DPLL_A_PIN_ID
>
>
>
>>than single integer index into which drivers will start encoding all
>>sort of info, using locally invented schemes.
>
>There could be multiple DPLL and pin instances for a single
>module/clock_id tuple we have to distinguish somehow. If the driver
>can't pass "index" of DPLL or a pin, how we distinguish them?
>
>Plus is is possible that 2 driver instances share the same dpll
>instance, then to get the dpll pointer reference, they do:
>INSTANCE A:
>dpll_0 = dpll_device_get(clock_id, 0, THIS_MODULE);
>dpll_1 = dpll_device_get(clock_id, 1, THIS_MODULE);
>
>INSTANCE B:
>dpll_0 = dpll_device_get(clock_id, 0, THIS_MODULE);
>dpll_1 = dpll_device_get(clock_id, 1, THIS_MODULE);
>
>My point is, event if we don't expose the index to the userspace,
>we need to have it internally.
>

Jakub Kicinski May 4, 2023, 4:04 p.m. UTC | #33

On Thu, 4 May 2023 13:00:42 +0200 Jiri Pirko wrote:
> Thu, May 04, 2023 at 04:16:43AM CEST, [e-mail] wrote:
> >On Wed, 3 May 2023 09:56:57 +0200 Jiri Pirko wrote:  
> >> Okay.
> >> 
> >> When netdev will have pin ID in the RT netlink message (as it is done
> >> in RFCv7), it is easy to get the pin/dpll for netdev. No problem there.
> >> 
> >> However, for non-SyncE usecase, how do you imagine scripts to work?
> >> I mean, the script have to obtain dpll/pin ID by deterministic
> >> module_name/clock_id/idx tuple.  
> >
> >No scoped idx.  
> 
> That means, no index defined by a driver if I undestand you correctly,
> right?

Yes, my suggestion did not include a scoped index with no
globally defined semantics.
 
> >> There are 2 options to do that:
> >> 1) dump all dplls/pins and do lookup in userspace
> >> 2) get a dpll/pin according to given module_name/clock_id/idx tuple
> >> 
> >> The first approach is not very nice.
> >> The currently pushed RFCv7 of the patchset does not support 2)
> >> 
> >> Now if we add support for 2), we basically use module_name/clock_id/idx
> >> as a handle for "get cmd". My point is, why can't we use it for "set
> >> cmd" as well and avoid the ID entirely?  
> >
> >Sure, we don't _have_ to have an ID, but it seems go against normal
> >data normalization rules. And I don't see any harm in it.
> >
> >But you're asking for per-device "idx" and that's a no-go for me,
> >given already cited experience.
> >
> >The user space can look up the ID based on identifying information it
> >has. IMO it's better to support multiple different intelligible elements  
> 
> Do you mean fixed tuple or variable tuple?
> 
> CMD_GET_ID
>   -> DPLL_A_MODULE_NAME  
>      DPLL_A_CLOCK_ID

> What is the next intelligible element to identify DPLL device here?

I don't know. We can always add more as needed.
We presuppose that the devices are identifiable, so whatever info
is used to identify them goes here.

>   <- DPLL_A_ID
> 
> CMD_GET_PIN_ID
>   -> DPLL_A_MODULE_NAME  
>      DPLL_A_CLOCK_ID

> What is the next intelligible element to identify a pin here?

Same answer. Could be a name of the pin according to ASIC docs.
Could be the ball name for a BGA package. Anything that's meaningful.

My point is that we don't want a field simply called "index". Because
then for one vendor it will mean Ethernet port, for another SMA
connector number and for the third pin of the package. Those are
different attributes.

>   <- DPLL_A_PIN_ID
> 
> >than single integer index into which drivers will start encoding all
> >sort of info, using locally invented schemes.  
> 
> There could be multiple DPLL and pin instances for a single
> module/clock_id tuple we have to distinguish somehow. If the driver
> can't pass "index" of DPLL or a pin, how we distinguish them?
> 
> Plus is is possible that 2 driver instances share the same dpll
> instance, then to get the dpll pointer reference, they do:
> INSTANCE A:
> dpll_0 = dpll_device_get(clock_id, 0, THIS_MODULE);
> dpll_1 = dpll_device_get(clock_id, 1, THIS_MODULE);
> 
> INSTANCE B:
> dpll_0 = dpll_device_get(clock_id, 0, THIS_MODULE);
> dpll_1 = dpll_device_get(clock_id, 1, THIS_MODULE);
> 
> My point is, event if we don't expose the index to the userspace,
> we need to have it internally.

That's fine, I guess. I'd prefer driver matching to be the same as user
space matching to force driver authors to have the same perspective as
the user. But a "driver coookie" not visible to user space it probably
fine.

Jiri Pirko May 4, 2023, 5:51 p.m. UTC | #34

Thu, May 04, 2023 at 06:04:01PM CEST, [e-mail] wrote:
>On Thu, 4 May 2023 13:00:42 +0200 Jiri Pirko wrote:
>> Thu, May 04, 2023 at 04:16:43AM CEST, [e-mail] wrote:
>> >On Wed, 3 May 2023 09:56:57 +0200 Jiri Pirko wrote:  
>> >> Okay.
>> >> 
>> >> When netdev will have pin ID in the RT netlink message (as it is done
>> >> in RFCv7), it is easy to get the pin/dpll for netdev. No problem there.
>> >> 
>> >> However, for non-SyncE usecase, how do you imagine scripts to work?
>> >> I mean, the script have to obtain dpll/pin ID by deterministic
>> >> module_name/clock_id/idx tuple.  
>> >
>> >No scoped idx.  
>> 
>> That means, no index defined by a driver if I undestand you correctly,
>> right?
>
>Yes, my suggestion did not include a scoped index with no
>globally defined semantics.

Okay, makes sense. Devlink port index didn't end up well :/


> 
>> >> There are 2 options to do that:
>> >> 1) dump all dplls/pins and do lookup in userspace
>> >> 2) get a dpll/pin according to given module_name/clock_id/idx tuple
>> >> 
>> >> The first approach is not very nice.
>> >> The currently pushed RFCv7 of the patchset does not support 2)
>> >> 
>> >> Now if we add support for 2), we basically use module_name/clock_id/idx
>> >> as a handle for "get cmd". My point is, why can't we use it for "set
>> >> cmd" as well and avoid the ID entirely?  
>> >
>> >Sure, we don't _have_ to have an ID, but it seems go against normal
>> >data normalization rules. And I don't see any harm in it.
>> >
>> >But you're asking for per-device "idx" and that's a no-go for me,
>> >given already cited experience.
>> >
>> >The user space can look up the ID based on identifying information it
>> >has. IMO it's better to support multiple different intelligible elements  
>> 
>> Do you mean fixed tuple or variable tuple?
>> 
>> CMD_GET_ID
>>   -> DPLL_A_MODULE_NAME  
>>      DPLL_A_CLOCK_ID
>
>> What is the next intelligible element to identify DPLL device here?
>
>I don't know. We can always add more as needed.
>We presuppose that the devices are identifiable, so whatever info
>is used to identify them goes here.

Allright. So in case of ptp_ocp and mlx5, module_name and clock_id
are enough. In case of ice, DPLL_A_TYPE, attr is the one to make
distinction between the 2 dpll instances there

So for now, we can have:
 CMD_GET_ID
   -> DPLL_A_MODULE_NAME
      DPLL_A_CLOCK_ID
      DPLL_A_TYPE
   <- DPLL_A_ID


if user passes a subset which would not provide a single match, we error
out with -EINVAL and proper exack message. Makes sense?





>
>>   <- DPLL_A_ID
>> 
>> CMD_GET_PIN_ID
>>   -> DPLL_A_MODULE_NAME  
>>      DPLL_A_CLOCK_ID
>
>> What is the next intelligible element to identify a pin here?
>
>Same answer. Could be a name of the pin according to ASIC docs.
>Could be the ball name for a BGA package. Anything that's meaningful.

Okay, for pin, the type and label would probably do:
 CMD_GET_PIN_ID
   -> DPLL_A_MODULE_NAME
      DPLL_A_CLOCK_ID
      DPLL_A_PIN_TYPE
      DPLL_A_PIN_LABEL
   <- DPLL_A_PIN_ID

Again, if user passes a subset which would not provide a single match,
we error out with -EINVAL and proper exack message.

If there is only one pin for example, user query of DPLL_A_MODULE_NAME
and DPLL_A_CLOCK_ID would do return a single match. No need to pass
anything else.

I think this could work with both ice and ptp_ocp, correct guys?

For mlx5, I will have 2 or more pins with same module name, clock id
and type. For these SyncE pins the label does not really make sense.
But I don't have to query, because the PIN_ID is going to be exposed for
netdev over RT netlink. Clicks.

Makes sense?


>
>My point is that we don't want a field simply called "index". Because
>then for one vendor it will mean Ethernet port, for another SMA
>connector number and for the third pin of the package. Those are
>different attributes.

Got you and agree.


>
>>   <- DPLL_A_PIN_ID
>> 
>> >than single integer index into which drivers will start encoding all
>> >sort of info, using locally invented schemes.  
>> 
>> There could be multiple DPLL and pin instances for a single
>> module/clock_id tuple we have to distinguish somehow. If the driver
>> can't pass "index" of DPLL or a pin, how we distinguish them?
>> 
>> Plus is is possible that 2 driver instances share the same dpll
>> instance, then to get the dpll pointer reference, they do:
>> INSTANCE A:
>> dpll_0 = dpll_device_get(clock_id, 0, THIS_MODULE);
>> dpll_1 = dpll_device_get(clock_id, 1, THIS_MODULE);
>> 
>> INSTANCE B:
>> dpll_0 = dpll_device_get(clock_id, 0, THIS_MODULE);
>> dpll_1 = dpll_device_get(clock_id, 1, THIS_MODULE);
>> 
>> My point is, event if we don't expose the index to the userspace,
>> we need to have it internally.
>
>That's fine, I guess. I'd prefer driver matching to be the same as user
>space matching to force driver authors to have the same perspective as
>the user. But a "driver coookie" not visible to user space it probably
>fine.

Allright, lets leave them for now. As internal kernel API, could be
changed in the future if needed.

Arkadiusz, Vadim, are you following this?

Jakub Kicinski May 4, 2023, 6:44 p.m. UTC | #35

On Thu, 4 May 2023 19:51:38 +0200 Jiri Pirko wrote:
> >> What is the next intelligible element to identify DPLL device here?  
> >
> >I don't know. We can always add more as needed.
> >We presuppose that the devices are identifiable, so whatever info
> >is used to identify them goes here.  
> 
> Allright. So in case of ptp_ocp and mlx5, module_name and clock_id
> are enough. In case of ice, DPLL_A_TYPE, attr is the one to make
> distinction between the 2 dpll instances there
> 
> So for now, we can have:
>  CMD_GET_ID
>    -> DPLL_A_MODULE_NAME  
>       DPLL_A_CLOCK_ID
>       DPLL_A_TYPE
>    <- DPLL_A_ID
> 
> 
> if user passes a subset which would not provide a single match, we error
> out with -EINVAL and proper exack message. Makes sense?

Yup, that sounds good to me.

> >Same answer. Could be a name of the pin according to ASIC docs.
> >Could be the ball name for a BGA package. Anything that's meaningful.  
> 
> Okay, for pin, the type and label would probably do:
>  CMD_GET_PIN_ID
>    -> DPLL_A_MODULE_NAME  
>       DPLL_A_CLOCK_ID
>       DPLL_A_PIN_TYPE
>       DPLL_A_PIN_LABEL

Label sounds dangerously open ended, too. Would that be the SMA
connector label (i.e. front panel label)? Or also applicable to
internal pins? It'd be easier to talk details if we had the user
facing documentation that ships with these products.

>    <- DPLL_A_PIN_ID
> 
> Again, if user passes a subset which would not provide a single match,
> we error out with -EINVAL and proper exack message.
> 
> If there is only one pin for example, user query of DPLL_A_MODULE_NAME
> and DPLL_A_CLOCK_ID would do return a single match. No need to pass
> anything else.
> 
> I think this could work with both ice and ptp_ocp, correct guys?
> 
> For mlx5, I will have 2 or more pins with same module name, clock id
> and type. For these SyncE pins the label does not really make sense.
> But I don't have to query, because the PIN_ID is going to be exposed for
> netdev over RT netlink. Clicks.
> 
> Makes sense?

Jiri Pirko May 5, 2023, 10:41 a.m. UTC | #36

Thu, May 04, 2023 at 08:44:21PM CEST, [e-mail] wrote:
>On Thu, 4 May 2023 19:51:38 +0200 Jiri Pirko wrote:
>> >> What is the next intelligible element to identify DPLL device here?  
>> >
>> >I don't know. We can always add more as needed.
>> >We presuppose that the devices are identifiable, so whatever info
>> >is used to identify them goes here.  
>> 
>> Allright. So in case of ptp_ocp and mlx5, module_name and clock_id
>> are enough. In case of ice, DPLL_A_TYPE, attr is the one to make
>> distinction between the 2 dpll instances there
>> 
>> So for now, we can have:
>>  CMD_GET_ID
>>    -> DPLL_A_MODULE_NAME  
>>       DPLL_A_CLOCK_ID
>>       DPLL_A_TYPE
>>    <- DPLL_A_ID
>> 
>> 
>> if user passes a subset which would not provide a single match, we error
>> out with -EINVAL and proper exack message. Makes sense?
>
>Yup, that sounds good to me.
>
>> >Same answer. Could be a name of the pin according to ASIC docs.
>> >Could be the ball name for a BGA package. Anything that's meaningful.  
>> 
>> Okay, for pin, the type and label would probably do:
>>  CMD_GET_PIN_ID
>>    -> DPLL_A_MODULE_NAME  
>>       DPLL_A_CLOCK_ID
>>       DPLL_A_PIN_TYPE
>>       DPLL_A_PIN_LABEL
>
>Label sounds dangerously open ended, too. Would that be the SMA

Well, every string is. And past RFCs of this patchset demonstrated guys
did serialize a lot of stuff in strings.


>connector label (i.e. front panel label)? Or also applicable to
>internal pins? It'd be easier to talk details if we had the user
>facing documentation that ships with these products.

I think is is use case specific. Some of the pins face the user over
physical port, they it is a front panel label. Others are internal
names. I have no clue how to define and mainly enforce rules here.

But as an example, if you have 2 pins of the same type, only difference
is they are connected to front panel connector "A" and "B", this is the
label you have to pass to the ID query. Do you see any other way?


>
>>    <- DPLL_A_PIN_ID
>> 
>> Again, if user passes a subset which would not provide a single match,
>> we error out with -EINVAL and proper exack message.
>> 
>> If there is only one pin for example, user query of DPLL_A_MODULE_NAME
>> and DPLL_A_CLOCK_ID would do return a single match. No need to pass
>> anything else.
>> 
>> I think this could work with both ice and ptp_ocp, correct guys?
>> 
>> For mlx5, I will have 2 or more pins with same module name, clock id
>> and type. For these SyncE pins the label does not really make sense.
>> But I don't have to query, because the PIN_ID is going to be exposed for
>> netdev over RT netlink. Clicks.
>> 
>> Makes sense?

Jakub Kicinski May 5, 2023, 3:35 p.m. UTC | #37

On Fri, 5 May 2023 12:41:11 +0200 Jiri Pirko wrote:
> >connector label (i.e. front panel label)? Or also applicable to
> >internal pins? It'd be easier to talk details if we had the user
> >facing documentation that ships with these products.  
> 
> I think is is use case specific. Some of the pins face the user over
> physical port, they it is a front panel label. Others are internal
> names. I have no clue how to define and mainly enforce rules here.

It should be pretty easy to judge if we see the user-facing
documentation vendors have.

> But as an example, if you have 2 pins of the same type, only difference
> is they are connected to front panel connector "A" and "B", this is the
> label you have to pass to the ID query. Do you see any other way?

Sound perfectly fine, if it's a front panel label, let's call 
the attribute DPLL_A_PIN_FRONT_PANEL_LABEL. If the pin is not
brought out to the front panel it will not have this attr.
For other type of labels we should have different attributes.

Jiri Pirko May 7, 2023, 7:58 a.m. UTC | #38

Fri, May 05, 2023 at 05:35:31PM CEST, [e-mail] wrote:
>On Fri, 5 May 2023 12:41:11 +0200 Jiri Pirko wrote:
>> >connector label (i.e. front panel label)? Or also applicable to
>> >internal pins? It'd be easier to talk details if we had the user
>> >facing documentation that ships with these products.  
>> 
>> I think is is use case specific. Some of the pins face the user over
>> physical port, they it is a front panel label. Others are internal
>> names. I have no clue how to define and mainly enforce rules here.
>
>It should be pretty easy to judge if we see the user-facing
>documentation vendors have.

Intel, Vadim, do you have such documentation?
As I wrote, for mlx5 the label is not really applicable as the link
netdev->pin is defining what the pin is.


>
>> But as an example, if you have 2 pins of the same type, only difference
>> is they are connected to front panel connector "A" and "B", this is the
>> label you have to pass to the ID query. Do you see any other way?
>
>Sound perfectly fine, if it's a front panel label, let's call 
>the attribute DPLL_A_PIN_FRONT_PANEL_LABEL. If the pin is not
>brought out to the front panel it will not have this attr.
>For other type of labels we should have different attributes.

Hmm, that would kind of embed the pin type into attr which feels wrong.
We already have the pin type exposed:
enum dpll_pin_type {
	DPLL_PIN_TYPE_UNSPEC,
	DPLL_PIN_TYPE_MUX,
	DPLL_PIN_TYPE_EXT,
	DPLL_PIN_TYPE_SYNCE_ETH_PORT,
	DPLL_PIN_TYPE_INT_OSCILLATOR,
	DPLL_PIN_TYPE_GNSS,

       __DPLL_PIN_TYPE_MAX,
       DPLL_PIN_TYPE_MAX = (__DPLL_PIN_TYPE_MAX - 1)
};

It case of front panel pin label, the type is "EXT" as for external pin.

Patch

diff --git a/MAINTAINERS b/MAINTAINERS
index edd3d562beee..0222b19af545 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -6289,6 +6289,15 @@  F:	Documentation/networking/device_drivers/ethernet/freescale/dpaa2/switch-drive
 F:	drivers/net/ethernet/freescale/dpaa2/dpaa2-switch*
 F:	drivers/net/ethernet/freescale/dpaa2/dpsw*
 
+DPLL CLOCK SUBSYSTEM
+M:	Vadim Fedorenko <[e-mail]>
+M:	Arkadiusz Kubalewski <[e-mail]>
+L:	[e-mail]
+S:	Maintained
+F:	drivers/dpll/*
+F:	include/net/dpll.h
+F:	include/uapi/linux/dpll.h
+
 DRBD DRIVER
 M:	Philipp Reisner <[e-mail]>
 M:	Lars Ellenberg <[e-mail]>
diff --git a/drivers/Kconfig b/drivers/Kconfig
index 968bd0a6fd78..453df9e1210d 100644
--- a/drivers/Kconfig
+++ b/drivers/Kconfig
@@ -241,4 +241,6 @@  source "drivers/peci/Kconfig"
 
 source "drivers/hte/Kconfig"
 
+source "drivers/dpll/Kconfig"
+
 endmenu
diff --git a/drivers/Makefile b/drivers/Makefile
index 20b118dca999..9ffb554507ef 100644
--- a/drivers/Makefile
+++ b/drivers/Makefile
@@ -194,3 +194,4 @@  obj-$(CONFIG_MOST)		+= most/
 obj-$(CONFIG_PECI)		+= peci/
 obj-$(CONFIG_HTE)		+= hte/
 obj-$(CONFIG_DRM_ACCEL)		+= accel/
+obj-$(CONFIG_DPLL)		+= dpll/
diff --git a/drivers/dpll/Kconfig b/drivers/dpll/Kconfig
new file mode 100644
index 000000000000..a4cae73f20d3
--- /dev/null
+++ b/drivers/dpll/Kconfig
@@ -0,0 +1,7 @@ 
+# SPDX-License-Identifier: GPL-2.0-only
+#
+# Generic DPLL drivers configuration
+#
+
+config DPLL
+  bool
diff --git a/drivers/dpll/Makefile b/drivers/dpll/Makefile
new file mode 100644
index 000000000000..d3926f2a733d
--- /dev/null
+++ b/drivers/dpll/Makefile
@@ -0,0 +1,10 @@ 
+# SPDX-License-Identifier: GPL-2.0
+#
+# Makefile for DPLL drivers.
+#
+
+obj-$(CONFIG_DPLL)          += dpll_sys.o
+dpll_sys-y                  += dpll_core.o
+dpll_sys-y                  += dpll_netlink.o
+dpll_sys-y                  += dpll_nl.o
+
diff --git a/drivers/dpll/dpll_core.c b/drivers/dpll/dpll_core.c
new file mode 100644
index 000000000000..3fc151e16751
--- /dev/null
+++ b/drivers/dpll/dpll_core.c
@@ -0,0 +1,835 @@ 
+// SPDX-License-Identifier: GPL-2.0
+/*
+ *  dpll_core.c - Generic DPLL Management class support.
+ *
+ *  Copyright (c) 2021 Meta Platforms, Inc. and affiliates
+ */
+
+#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
+
+#include <linux/device.h>
+#include <linux/err.h>
+#include <linux/slab.h>
+#include <linux/string.h>
+
+#include "dpll_core.h"
+
+DEFINE_MUTEX(dpll_device_xa_lock);
+DEFINE_MUTEX(dpll_pin_xa_lock);
+
+DEFINE_XARRAY_FLAGS(dpll_device_xa, XA_FLAGS_ALLOC);
+DEFINE_XARRAY_FLAGS(dpll_pin_xa, XA_FLAGS_ALLOC);
+
+#define ASSERT_DPLL_REGISTERED(d)                                          \
+	WARN_ON_ONCE(!xa_get_mark(&dpll_device_xa, (d)->id, DPLL_REGISTERED))
+#define ASSERT_DPLL_NOT_REGISTERED(d)                                      \
+	WARN_ON_ONCE(xa_get_mark(&dpll_device_xa, (d)->id, DPLL_REGISTERED))
+
+static struct class dpll_class = {
+	.name = "dpll",
+};
+
+/**
+ * dpll_device_get_by_id - find dpll device by it's id
+ * @id: id of searched dpll
+ *
+ * Return:
+ * * dpll_device struct if found
+ * * NULL otherwise
+ */
+struct dpll_device *dpll_device_get_by_id(int id)
+{
+	struct dpll_device *dpll = NULL;
+
+	if (xa_get_mark(&dpll_device_xa, id, DPLL_REGISTERED))
+		dpll = xa_load(&dpll_device_xa, id);
+
+	return dpll;
+}
+
+/**
+ * dpll_device_get_by_name - find dpll device by it's id
+ * @bus_name: bus name of searched dpll
+ * @dev_name: dev name of searched dpll
+ *
+ * Return:
+ * * dpll_device struct if found
+ * * NULL otherwise
+ */
+struct dpll_device *
+dpll_device_get_by_name(const char *bus_name, const char *device_name)
+{
+	struct dpll_device *dpll, *ret = NULL;
+	unsigned long index;
+
+	xa_for_each_marked(&dpll_device_xa, index, dpll, DPLL_REGISTERED) {
+		if (!strcmp(dev_bus_name(&dpll->dev), bus_name) &&
+		    !strcmp(dev_name(&dpll->dev), device_name)) {
+			ret = dpll;
+			break;
+		}
+	}
+
+	return ret;
+}
+
+/**
+ * dpll_xa_ref_pin_add - add pin reference to a given xarray
+ * @xa_pins: dpll_pin_ref xarray holding pins
+ * @pin: pin being added
+ * @ops: ops for a pin
+ * @priv: pointer to private data of owner
+ *
+ * Allocate and create reference of a pin or increase refcount on existing pin
+ * reference on given xarray.
+ *
+ * Return:
+ * * 0 on success
+ * * -ENOMEM on failed allocation
+ */
+static int
+dpll_xa_ref_pin_add(struct xarray *xa_pins, struct dpll_pin *pin,
+		    struct dpll_pin_ops *ops, void *priv)
+{
+	struct dpll_pin_ref *ref;
+	unsigned long i;
+	u32 idx;
+	int ret;
+
+	xa_for_each(xa_pins, i, ref) {
+		if (ref->pin == pin) {
+			refcount_inc(&ref->refcount);
+			return 0;
+		}
+	}
+
+	ref = kzalloc(sizeof(*ref), GFP_KERNEL);
+	if (!ref)
+		return -ENOMEM;
+	ref->pin = pin;
+	ref->ops = ops;
+	ref->priv = priv;
+	ret = xa_alloc(xa_pins, &idx, ref, xa_limit_16b, GFP_KERNEL);
+	if (!ret)
+		refcount_set(&ref->refcount, 1);
+	else
+		kfree(ref);
+
+	return ret;
+}
+
+/**
+ * dpll_xa_ref_pin_del - remove reference of a pin from xarray
+ * @xa_pins: dpll_pin_ref xarray holding pins
+ * @pin: pointer to a pin
+ *
+ * Decrement refcount of existing pin reference on given xarray.
+ * If all references are dropped, delete the reference and free its memory.
+ *
+ * Return:
+ * * 0 on success
+ * * -EINVAL if reference to a pin was not found
+ */
+static int dpll_xa_ref_pin_del(struct xarray *xa_pins, struct dpll_pin *pin)
+{
+	struct dpll_pin_ref *ref;
+	unsigned long i;
+
+	xa_for_each(xa_pins, i, ref) {
+		if (ref->pin == pin) {
+			if (refcount_dec_and_test(&ref->refcount)) {
+				xa_erase(xa_pins, i);
+				kfree(ref);
+			}
+			return 0;
+		}
+	}
+
+	return -EINVAL;
+}
+
+/**
+ * dpll_xa_ref_pin_find - find pin reference on xarray
+ * @xa_pins: dpll_pin_ref xarray holding pins
+ * @pin: pointer to a pin
+ *
+ * Search for pin reference struct of a given pin on given xarray.
+ *
+ * Return:
+ * * pin reference struct pointer on success
+ * * NULL - reference to a pin was not found
+ */
+struct dpll_pin_ref *
+dpll_xa_ref_pin_find(struct xarray *xa_pins, const struct dpll_pin *pin)
+{
+	struct dpll_pin_ref *ref;
+	unsigned long i;
+
+	xa_for_each(xa_pins, i, ref) {
+		if (ref->pin == pin)
+			return ref;
+	}
+
+	return NULL;
+}
+
+/**
+ * dpll_xa_ref_dpll_add - add dpll reference to a given xarray
+ * @xa_dplls: dpll_pin_ref xarray holding dplls
+ * @dpll: dpll being added
+ * @ops: pin-reference ops for a dpll
+ * @priv: pointer to private data of owner
+ *
+ * Allocate and create reference of a dpll-pin ops or increase refcount
+ * on existing dpll reference on given xarray.
+ *
+ * Return:
+ * * 0 on success
+ * * -ENOMEM on failed allocation
+ */
+static int
+dpll_xa_ref_dpll_add(struct xarray *xa_dplls, struct dpll_device *dpll,
+		     struct dpll_pin_ops *ops, void *priv)
+{
+	struct dpll_pin_ref *ref;
+	unsigned long i;
+	u32 idx;
+	int ret;
+
+	xa_for_each(xa_dplls, i, ref) {
+		if (ref->dpll == dpll) {
+			refcount_inc(&ref->refcount);
+			return 0;
+		}
+	}
+	ref = kzalloc(sizeof(*ref), GFP_KERNEL);
+	if (!ref)
+		return -ENOMEM;
+	ref->dpll = dpll;
+	ref->ops = ops;
+	ref->priv = priv;
+	ret = xa_alloc(xa_dplls, &idx, ref, xa_limit_16b, GFP_KERNEL);
+	if (!ret)
+		refcount_set(&ref->refcount, 1);
+	else
+		kfree(ref);
+
+	return ret;
+}
+
+/**
+ * dpll_xa_ref_dpll_del - remove reference of a dpll from xarray
+ * @xa_dplls: dpll_pin_ref xarray holding dplls
+ * @dpll: pointer to a dpll to remove
+ *
+ * Decrement refcount of existing dpll reference on given xarray.
+ * If all references are dropped, delete the reference and free its memory.
+ */
+static void
+dpll_xa_ref_dpll_del(struct xarray *xa_dplls, struct dpll_device *dpll)
+{
+	struct dpll_pin_ref *ref;
+	unsigned long i;
+
+	xa_for_each(xa_dplls, i, ref) {
+		if (ref->dpll == dpll) {
+			if (refcount_dec_and_test(&ref->refcount)) {
+				xa_erase(xa_dplls, i);
+				kfree(ref);
+			}
+			break;
+		}
+	}
+}
+
+/**
+ * dpll_xa_ref_dpll_find - find dpll reference on xarray
+ * @xa_dplls: dpll_pin_ref xarray holding dplls
+ * @dpll: pointer to a dpll
+ *
+ * Search for dpll-pin ops reference struct of a given dpll on given xarray.
+ *
+ * Return:
+ * * pin reference struct pointer on success
+ * * NULL - reference to a pin was not found
+ */
+struct dpll_pin_ref *
+dpll_xa_ref_dpll_find(struct xarray *xa_refs, const struct dpll_device *dpll)
+{
+	struct dpll_pin_ref *ref;
+	unsigned long i;
+
+	xa_for_each(xa_refs, i, ref) {
+		if (ref->dpll == dpll)
+			return ref;
+	}
+
+	return NULL;
+}
+
+
+/**
+ * dpll_device_alloc - allocate the memory for dpll device
+ * @clock_id: clock_id of creator
+ * @dev_driver_id: id given by dev driver
+ * @module: reference to registering module
+ *
+ * Allocates memory and initialize dpll device, hold its reference on global
+ * xarray.
+ *
+ * Return:
+ * * dpll_device struct pointer if succeeded
+ * * ERR_PTR(X) - failed allocation
+ */
+struct dpll_device *
+dpll_device_alloc(const u64 clock_id, u32 dev_driver_id, struct module *module)
+{
+	struct dpll_device *dpll;
+	int ret;
+
+	dpll = kzalloc(sizeof(*dpll), GFP_KERNEL);
+	if (!dpll)
+		return ERR_PTR(-ENOMEM);
+	refcount_set(&dpll->refcount, 1);
+	dpll->dev.class = &dpll_class;
+	dpll->dev_driver_id = dev_driver_id;
+	dpll->clock_id = clock_id;
+	dpll->module = module;
+	ret = xa_alloc(&dpll_device_xa, &dpll->id, dpll,
+		       xa_limit_16b, GFP_KERNEL);
+	if (ret) {
+		kfree(dpll);
+		return ERR_PTR(ret);
+	}
+	xa_init_flags(&dpll->pin_refs, XA_FLAGS_ALLOC);
+
+	return dpll;
+}
+
+/**
+ * dpll_device_get - find existing or create new dpll device
+ * @clock_id: clock_id of creator
+ * @dev_driver_id: id given by dev driver
+ * @module: reference to registering module
+ *
+ * Get existing object of a dpll device, unique for given arguments.
+ * Create new if doesn't exist yet.
+ *
+ * Return:
+ * * valid dpll_device struct pointer if succeeded
+ * * ERR_PTR of an error
+ */
+struct dpll_device *
+dpll_device_get(u64 clock_id, u32 dev_driver_id, struct module *module)
+{
+	struct dpll_device *dpll, *ret = NULL;
+	unsigned long index;
+
+	mutex_lock(&dpll_device_xa_lock);
+	xa_for_each(&dpll_device_xa, index, dpll) {
+		if (dpll->clock_id == clock_id &&
+		    dpll->dev_driver_id == dev_driver_id &&
+		    dpll->module == module) {
+			ret = dpll;
+			refcount_inc(&ret->refcount);
+			break;
+		}
+	}
+	if (!ret)
+		ret = dpll_device_alloc(clock_id, dev_driver_id, module);
+	mutex_unlock(&dpll_device_xa_lock);
+
+	return ret;
+}
+EXPORT_SYMBOL_GPL(dpll_device_get);
+
+/**
+ * dpll_device_put - decrease the refcount and free memory if possible
+ * @dpll: dpll_device struct pointer
+ *
+ * Drop reference for a dpll device, if all references are gone, delete
+ * dpll device object.
+ */
+void dpll_device_put(struct dpll_device *dpll)
+{
+	if (!dpll)
+		return;
+	mutex_lock(&dpll_device_xa_lock);
+	if (refcount_dec_and_test(&dpll->refcount)) {
+		WARN_ON_ONCE(!xa_empty(&dpll->pin_refs));
+		xa_destroy(&dpll->pin_refs);
+		xa_erase(&dpll_device_xa, dpll->id);
+		kfree(dpll);
+	}
+	mutex_unlock(&dpll_device_xa_lock);
+}
+EXPORT_SYMBOL_GPL(dpll_device_put);
+
+/**
+ * dpll_device_register - register the dpll device in the subsystem
+ * @dpll: pointer to a dpll
+ * @type: type of a dpll
+ * @ops: ops for a dpll device
+ * @priv: pointer to private information of owner
+ * @owner: pointer to owner device
+ *
+ * Make dpll device available for user space.
+ *
+ * Return:
+ * * 0 on success
+ * * -EINVAL on failure
+ */
+int dpll_device_register(struct dpll_device *dpll, enum dpll_type type,
+			 struct dpll_device_ops *ops, void *priv,
+			 struct device *owner)
+{
+	if (WARN_ON(!ops || !owner))
+		return -EINVAL;
+	if (WARN_ON(type <= DPLL_TYPE_UNSPEC || type > DPLL_TYPE_MAX))
+		return -EINVAL;
+	mutex_lock(&dpll_device_xa_lock);
+	if (ASSERT_DPLL_NOT_REGISTERED(dpll)) {
+		mutex_unlock(&dpll_device_xa_lock);
+		return -EEXIST;
+	}
+	dpll->dev.bus = owner->bus;
+	dpll->parent = owner;
+	dpll->type = type;
+	dpll->ops = ops;
+	dev_set_name(&dpll->dev, "%s_%d", dev_name(owner),
+		     dpll->dev_driver_id);
+	dpll->priv = priv;
+	xa_set_mark(&dpll_device_xa, dpll->id, DPLL_REGISTERED);
+	mutex_unlock(&dpll_device_xa_lock);
+	dpll_notify_device_create(dpll);
+
+	return 0;
+}
+EXPORT_SYMBOL_GPL(dpll_device_register);
+
+/**
+ * dpll_device_unregister - deregister dpll device
+ * @dpll: registered dpll pointer
+ *
+ * Deregister device, make it unavailable for userspace.
+ * Note: It does not free the memory
+ */
+void dpll_device_unregister(struct dpll_device *dpll)
+{
+	mutex_lock(&dpll_device_xa_lock);
+	ASSERT_DPLL_REGISTERED(dpll);
+	xa_clear_mark(&dpll_device_xa, dpll->id, DPLL_REGISTERED);
+	mutex_unlock(&dpll_device_xa_lock);
+	dpll_notify_device_delete(dpll);
+}
+EXPORT_SYMBOL_GPL(dpll_device_unregister);
+
+/**
+ * dpll_pin_alloc - allocate the memory for dpll pin
+ * @clock_id: clock_id of creator
+ * @dev_driver_id: id given by dev driver
+ * @module: reference to registering module
+ * @prop: dpll pin properties
+ *
+ * Return:
+ * * valid allocated dpll_pin struct pointer if succeeded
+ * * ERR_PTR of an error
+ */
+struct dpll_pin *
+dpll_pin_alloc(u64 clock_id, u8 device_drv_id,	struct module *module,
+	       const struct dpll_pin_properties *prop)
+{
+	struct dpll_pin *pin;
+	int ret;
+
+	pin = kzalloc(sizeof(*pin), GFP_KERNEL);
+	if (!pin)
+		return ERR_PTR(-ENOMEM);
+	pin->dev_driver_id = device_drv_id;
+	pin->clock_id = clock_id;
+	pin->module = module;
+	refcount_set(&pin->refcount, 1);
+	if (WARN_ON(!prop->description)) {
+		ret = -EINVAL;
+		goto release;
+	}
+	pin->prop.description = kstrdup(prop->description, GFP_KERNEL);
+	if (!pin->prop.description) {
+		ret = -ENOMEM;
+		goto release;
+	}
+	if (WARN_ON(prop->type <= DPLL_PIN_TYPE_UNSPEC ||
+		    prop->type > DPLL_PIN_TYPE_MAX)) {
+		ret = -EINVAL;
+		goto release;
+	}
+	pin->prop.type = prop->type;
+	pin->prop.capabilities = prop->capabilities;
+	pin->prop.freq_supported = prop->freq_supported;
+	pin->prop.any_freq_min = prop->any_freq_min;
+	pin->prop.any_freq_max = prop->any_freq_max;
+	xa_init_flags(&pin->dpll_refs, XA_FLAGS_ALLOC);
+	xa_init_flags(&pin->parent_refs, XA_FLAGS_ALLOC);
+	ret = xa_alloc(&dpll_pin_xa, &pin->idx, pin,
+		       xa_limit_16b, GFP_KERNEL);
+release:
+	if (ret) {
+		xa_destroy(&pin->dpll_refs);
+		xa_destroy(&pin->parent_refs);
+		kfree(pin->prop.description);
+		kfree(pin->rclk_dev_name);
+		kfree(pin);
+		return ERR_PTR(ret);
+	}
+
+	return pin;
+}
+
+/**
+ * dpll_pin_get - find existing or create new dpll pin
+ * @clock_id: clock_id of creator
+ * @dev_driver_id: id given by dev driver
+ * @module: reference to registering module
+ * @prop: dpll pin properties
+ *
+ * Get existing object of a pin (unique for given arguments) or create new
+ * if doesn't exist yet.
+ *
+ * Return:
+ * * valid allocated dpll_pin struct pointer if succeeded
+ * * ERR_PTR of an error
+ */
+struct dpll_pin *
+dpll_pin_get(u64 clock_id, u32 device_drv_id, struct module *module,
+	     const struct dpll_pin_properties *prop)
+{
+	struct dpll_pin *pos, *ret = NULL;
+	unsigned long index;
+
+	mutex_lock(&dpll_pin_xa_lock);
+	xa_for_each(&dpll_pin_xa, index, pos) {
+		if (pos->clock_id == clock_id &&
+		    pos->dev_driver_id == device_drv_id &&
+		    pos->module == module) {
+			ret = pos;
+			refcount_inc(&ret->refcount);
+			break;
+		}
+	}
+	if (!ret)
+		ret = dpll_pin_alloc(clock_id, device_drv_id, module, prop);
+	mutex_unlock(&dpll_pin_xa_lock);
+
+	return ret;
+}
+EXPORT_SYMBOL_GPL(dpll_pin_get);
+
+/**
+ * dpll_pin_put - decrease the refcount and free memory if possible
+ * @dpll: dpll_device struct pointer
+ *
+ * Drop reference for a pin, if all references are gone, delete pin object.
+ */
+void dpll_pin_put(struct dpll_pin *pin)
+{
+	if (!pin)
+		return;
+	mutex_lock(&dpll_pin_xa_lock);
+	if (refcount_dec_and_test(&pin->refcount)) {
+		xa_destroy(&pin->dpll_refs);
+		xa_destroy(&pin->parent_refs);
+		xa_erase(&dpll_pin_xa, pin->idx);
+		kfree(pin->prop.description);
+		kfree(pin->rclk_dev_name);
+		kfree(pin);
+	}
+	mutex_unlock(&dpll_pin_xa_lock);
+}
+EXPORT_SYMBOL_GPL(dpll_pin_put);
+
+static int
+__dpll_pin_register(struct dpll_device *dpll, struct dpll_pin *pin,
+		    struct dpll_pin_ops *ops, void *priv,
+		    const char *rclk_device_name)
+{
+	int ret;
+
+	if (rclk_device_name && !pin->rclk_dev_name) {
+		pin->rclk_dev_name = kstrdup(rclk_device_name, GFP_KERNEL);
+		if (!pin->rclk_dev_name)
+			return -ENOMEM;
+	}
+	ret = dpll_xa_ref_pin_add(&dpll->pin_refs, pin, ops, priv);
+	if (ret)
+		goto rclk_free;
+	ret = dpll_xa_ref_dpll_add(&pin->dpll_refs, dpll, ops, priv);
+	if (ret)
+		goto ref_pin_del;
+	else
+		dpll_pin_notify(dpll, pin, DPLL_A_PIN_IDX);
+
+	return ret;
+
+ref_pin_del:
+	dpll_xa_ref_pin_del(&dpll->pin_refs, pin);
+rclk_free:
+	kfree(pin->rclk_dev_name);
+	return ret;
+}
+
+/**
+ * dpll_pin_register - register the dpll pin in the subsystem
+ * @dpll: pointer to a dpll
+ * @pin: pointer to a dpll pin
+ * @ops: ops for a dpll pin ops
+ * @priv: pointer to private information of owner
+ * @rclk_device: pointer to recovered clock device
+ *
+ * Return:
+ * * 0 on success
+ * * -EINVAL - missing dpll or pin
+ * * -ENOMEM - failed to allocate memory
+ */
+int
+dpll_pin_register(struct dpll_device *dpll, struct dpll_pin *pin,
+		  struct dpll_pin_ops *ops, void *priv,
+		  struct device *rclk_device)
+{
+	const char *rclk_name = rclk_device ? dev_name(rclk_device) : NULL;
+	int ret;
+
+	if (WARN_ON(!dpll))
+		return -EINVAL;
+	if (WARN_ON(!pin))
+		return -EINVAL;
+
+	mutex_lock(&dpll_device_xa_lock);
+	mutex_lock(&dpll_pin_xa_lock);
+	ret = __dpll_pin_register(dpll, pin, ops, priv, rclk_name);
+	mutex_unlock(&dpll_pin_xa_lock);
+	mutex_unlock(&dpll_device_xa_lock);
+
+	return ret;
+}
+EXPORT_SYMBOL_GPL(dpll_pin_register);
+
+static void
+__dpll_pin_unregister(struct dpll_device *dpll, struct dpll_pin *pin)
+{
+	dpll_xa_ref_pin_del(&dpll->pin_refs, pin);
+	dpll_xa_ref_dpll_del(&pin->dpll_refs, dpll);
+}
+
+/**
+ * dpll_pin_unregister - deregister dpll pin from dpll device
+ * @dpll: registered dpll pointer
+ * @pin: pointer to a pin
+ *
+ * Note: It does not free the memory
+ */
+int dpll_pin_unregister(struct dpll_device *dpll, struct dpll_pin *pin)
+{
+	if (WARN_ON(xa_empty(&dpll->pin_refs)))
+		return -ENOENT;
+
+	mutex_lock(&dpll_device_xa_lock);
+	mutex_lock(&dpll_pin_xa_lock);
+	__dpll_pin_unregister(dpll, pin);
+	mutex_unlock(&dpll_pin_xa_lock);
+	mutex_unlock(&dpll_device_xa_lock);
+
+	return 0;
+}
+EXPORT_SYMBOL_GPL(dpll_pin_unregister);
+
+/**
+ * dpll_pin_on_pin_register - register a pin with a parent pin
+ * @parent: pointer to a parent pin
+ * @pin: pointer to a pin
+ * @ops: ops for a dpll pin
+ * @priv: pointer to private information of owner
+ * @rclk_device: pointer to recovered clock device
+ *
+ * Register a pin with a parent pin, create references between them and
+ * between newly registered pin and dplls connected with a parent pin.
+ *
+ * Return:
+ * * 0 on success
+ * * -EINVAL missing pin or parent
+ * * -ENOMEM failed allocation
+ * * -EPERM if parent is not allowed
+ */
+int
+dpll_pin_on_pin_register(struct dpll_pin *parent, struct dpll_pin *pin,
+			 struct dpll_pin_ops *ops, void *priv,
+			 struct device *rclk_device)
+{
+	struct dpll_pin_ref *ref;
+	unsigned long i, stop;
+	int ret;
+
+	if (WARN_ON(!pin || !parent))
+		return -EINVAL;
+	if (WARN_ON(parent->prop.type != DPLL_PIN_TYPE_MUX))
+		return -EPERM;
+	mutex_lock(&dpll_pin_xa_lock);
+	ret = dpll_xa_ref_pin_add(&pin->parent_refs, parent, ops, priv);
+	if (ret)
+		goto unlock;
+	refcount_inc(&pin->refcount);
+	xa_for_each(&parent->dpll_refs, i, ref) {
+		mutex_lock(&dpll_device_xa_lock);
+		ret = __dpll_pin_register(ref->dpll, pin, ops, priv,
+					  rclk_device ?
+					  dev_name(rclk_device) : NULL);
+		mutex_unlock(&dpll_device_xa_lock);
+		if (ret) {
+			stop = i;
+			goto dpll_unregister;
+		}
+		dpll_pin_parent_notify(ref->dpll, pin, parent, DPLL_A_PIN_IDX);
+	}
+	mutex_unlock(&dpll_pin_xa_lock);
+
+	return ret;
+
+dpll_unregister:
+	xa_for_each(&parent->dpll_refs, i, ref) {
+		if (i < stop) {
+			mutex_lock(&dpll_device_xa_lock);
+			__dpll_pin_unregister(ref->dpll, pin);
+			mutex_unlock(&dpll_device_xa_lock);
+		}
+	}
+	refcount_dec(&pin->refcount);
+	dpll_xa_ref_pin_del(&pin->parent_refs, parent);
+unlock:
+	mutex_unlock(&dpll_pin_xa_lock);
+	return ret;
+}
+EXPORT_SYMBOL_GPL(dpll_pin_on_pin_register);
+
+/**
+ * dpll_pin_on_pin_unregister - deregister dpll pin from a parent pin
+ * @parent: pointer to a parent pin
+ * @pin: pointer to a pin
+ *
+ * Note: It does not free the memory
+ */
+void dpll_pin_on_pin_unregister(struct dpll_pin *parent, struct dpll_pin *pin)
+{
+	struct dpll_pin_ref *ref;
+	unsigned long i;
+
+	mutex_lock(&dpll_device_xa_lock);
+	mutex_lock(&dpll_pin_xa_lock);
+	dpll_xa_ref_pin_del(&pin->parent_refs, parent);
+	refcount_dec(&pin->refcount);
+	xa_for_each(&pin->dpll_refs, i, ref) {
+		__dpll_pin_unregister(ref->dpll, pin);
+		dpll_pin_parent_notify(ref->dpll, pin, parent,
+				       DPLL_A_PIN_IDX);
+	}
+	mutex_unlock(&dpll_pin_xa_lock);
+	mutex_unlock(&dpll_device_xa_lock);
+}
+EXPORT_SYMBOL_GPL(dpll_pin_on_pin_unregister);
+
+/**
+ * dpll_pin_get_by_idx - find a pin ref on dpll by pin index
+ * @dpll: dpll device pointer
+ * @idx: index of pin
+ *
+ * Find a reference to a pin registered with given dpll and return its pointer.
+ *
+ * Return:
+ * * valid pointer if pin was found
+ * * NULL if not found
+ */
+struct dpll_pin *dpll_pin_get_by_idx(struct dpll_device *dpll, u32 idx)
+{
+	struct dpll_pin_ref *pos;
+	unsigned long i;
+
+	xa_for_each(&dpll->pin_refs, i, pos) {
+		if (pos && pos->pin && pos->pin->dev_driver_id == idx)
+			return pos->pin;
+	}
+
+	return NULL;
+}
+
+/**
+ * dpll_priv - get the dpll device private owner data
+ * @dpll:	registered dpll pointer
+ *
+ * Return: pointer to the data
+ */
+void *dpll_priv(const struct dpll_device *dpll)
+{
+	return dpll->priv;
+}
+EXPORT_SYMBOL_GPL(dpll_priv);
+
+/**
+ * dpll_pin_on_dpll_priv - get the dpll device private owner data
+ * @dpll:	registered dpll pointer
+ * @pin:	pointer to a pin
+ *
+ * Return: pointer to the data
+ */
+void *dpll_pin_on_dpll_priv(const struct dpll_device *dpll,
+			    const struct dpll_pin *pin)
+{
+	struct dpll_pin_ref *ref;
+
+	ref = dpll_xa_ref_pin_find((struct xarray *)&dpll->pin_refs, pin);
+	if (!ref)
+		return NULL;
+
+	return ref->priv;
+}
+EXPORT_SYMBOL_GPL(dpll_pin_on_dpll_priv);
+
+/**
+ * dpll_pin_on_pin_priv - get the dpll pin private owner data
+ * @parent: pointer to a parent pin
+ * @pin: pointer to a pin
+ *
+ * Return: pointer to the data
+ */
+void *dpll_pin_on_pin_priv(const struct dpll_pin *parent,
+			   const struct dpll_pin *pin)
+{
+	struct dpll_pin_ref *ref;
+
+	ref = dpll_xa_ref_pin_find((struct xarray *)&pin->parent_refs, parent);
+	if (!ref)
+		return NULL;
+
+	return ref->priv;
+}
+EXPORT_SYMBOL_GPL(dpll_pin_on_pin_priv);
+
+static int __init dpll_init(void)
+{
+	int ret;
+
+	ret = dpll_netlink_init();
+	if (ret)
+		goto error;
+
+	ret = class_register(&dpll_class);
+	if (ret)
+		goto unregister_netlink;
+
+	return 0;
+
+unregister_netlink:
+	dpll_netlink_finish();
+error:
+	mutex_destroy(&dpll_device_xa_lock);
+	mutex_destroy(&dpll_pin_xa_lock);
+	return ret;
+}
+subsys_initcall(dpll_init);
diff --git a/drivers/dpll/dpll_core.h b/drivers/dpll/dpll_core.h
new file mode 100644
index 000000000000..876b6ac6f3a0
--- /dev/null
+++ b/drivers/dpll/dpll_core.h
@@ -0,0 +1,99 @@ 
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ *  Copyright (c) 2021 Meta Platforms, Inc. and affiliates
+ */
+
+#ifndef __DPLL_CORE_H__
+#define __DPLL_CORE_H__
+
+#include <linux/dpll.h>
+#include <linux/refcount.h>
+#include "dpll_netlink.h"
+
+#define DPLL_REGISTERED		XA_MARK_1
+
+/**
+ * struct dpll_device - structure for a DPLL device
+ * @id:			unique id number for each device
+ * @dev_driver_id:	id given by dev driver
+ * @dev:		struct device for this dpll device
+ * @parent:		parent device
+ * @module:		module of creator
+ * @ops:		operations this &dpll_device supports
+ * @lock:		mutex to serialize operations
+ * @type:		type of a dpll
+ * @priv:		pointer to private information of owner
+ * @pins:		list of pointers to pins registered with this dpll
+ * @clock_id:		unique identifier (clock_id) of a dpll
+ * @mode_supported_mask: mask of supported modes
+ * @refcount:		refcount
+ **/
+struct dpll_device {
+	u32 id;
+	u32 dev_driver_id;
+	struct device dev;
+	struct device *parent;
+	struct module *module;
+	struct dpll_device_ops *ops;
+	enum dpll_type type;
+	void *priv;
+	struct xarray pin_refs;
+	u64 clock_id;
+	unsigned long mode_supported_mask;
+	refcount_t refcount;
+};
+
+/**
+ * struct dpll_pin - structure for a dpll pin
+ * @idx:		unique idx given by alloc on global pin's XA
+ * @dev_driver_id:	id given by dev driver
+ * @clock_id:		clock_id of creator
+ * @module:		module of creator
+ * @dpll_refs:		hold referencees to dplls that pin is registered with
+ * @pin_refs:		hold references to pins that pin is registered with
+ * @prop:		properties given by registerer
+ * @rclk_dev_name:	holds name of device when pin can recover clock from it
+ * @refcount:		refcount
+ **/
+struct dpll_pin {
+	u32 idx;
+	u32 dev_driver_id;
+	u64 clock_id;
+	struct module *module;
+	struct xarray dpll_refs;
+	struct xarray parent_refs;
+	struct dpll_pin_properties prop;
+	char *rclk_dev_name;
+	refcount_t refcount;
+};
+
+/**
+ * struct dpll_pin_ref - structure for referencing either dpll or pins
+ * @dpll:		pointer to a dpll
+ * @pin:		pointer to a pin
+ * @ops:		ops for a dpll pin
+ * @priv:		pointer to private information of owner
+ **/
+struct dpll_pin_ref {
+	union {
+		struct dpll_device *dpll;
+		struct dpll_pin *pin;
+	};
+	struct dpll_pin_ops *ops;
+	void *priv;
+	refcount_t refcount;
+};
+
+struct dpll_device *dpll_device_get_by_id(int id);
+struct dpll_device *dpll_device_get_by_name(const char *bus_name,
+					    const char *dev_name);
+struct dpll_pin *dpll_pin_get_by_idx(struct dpll_device *dpll, u32 idx);
+struct dpll_pin_ref *
+dpll_xa_ref_pin_find(struct xarray *xa_refs, const struct dpll_pin *pin);
+struct dpll_pin_ref *
+dpll_xa_ref_dpll_find(struct xarray *xa_refs, const struct dpll_device *dpll);
+extern struct xarray dpll_device_xa;
+extern struct xarray dpll_pin_xa;
+extern struct mutex dpll_device_xa_lock;
+extern struct mutex dpll_pin_xa_lock;
+#endif
diff --git a/drivers/dpll/dpll_netlink.c b/drivers/dpll/dpll_netlink.c
new file mode 100644
index 000000000000..46aefeb1ac93
--- /dev/null
+++ b/drivers/dpll/dpll_netlink.c
@@ -0,0 +1,1065 @@ 
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Generic netlink for DPLL management framework
+ *
+ * Copyright (c) 2021 Meta Platforms, Inc. and affiliates
+ *
+ */
+#include <linux/module.h>
+#include <linux/kernel.h>
+#include <net/genetlink.h>
+#include "dpll_core.h"
+#include "dpll_nl.h"
+#include <uapi/linux/dpll.h>
+
+static u32 dpll_pin_freq_value[] = {
+	[DPLL_PIN_FREQ_SUPP_1_HZ] = DPLL_PIN_FREQ_1_HZ,
+	[DPLL_PIN_FREQ_SUPP_10_MHZ] = DPLL_PIN_FREQ_10_MHZ,
+};
+
+static int
+dpll_msg_add_dev_handle(struct sk_buff *msg, const struct dpll_device *dpll)
+{
+	if (nla_put_u32(msg, DPLL_A_ID, dpll->id))
+		return -EMSGSIZE;
+	if (nla_put_string(msg, DPLL_A_BUS_NAME, dev_bus_name(&dpll->dev)))
+		return -EMSGSIZE;
+	if (nla_put_string(msg, DPLL_A_DEV_NAME, dev_name(&dpll->dev)))
+		return -EMSGSIZE;
+
+	return 0;
+}
+
+static int
+dpll_msg_add_mode(struct sk_buff *msg, const struct dpll_device *dpll,
+		  struct netlink_ext_ack *extack)
+{
+	enum dpll_mode mode;
+
+	if (WARN_ON(!dpll->ops->mode_get))
+		return -EOPNOTSUPP;
+	if (dpll->ops->mode_get(dpll, &mode, extack))
+		return -EFAULT;
+	if (nla_put_u8(msg, DPLL_A_MODE, mode))
+		return -EMSGSIZE;
+
+	return 0;
+}
+
+static int
+dpll_msg_add_source_pin_idx(struct sk_buff *msg, struct dpll_device *dpll,
+			    struct netlink_ext_ack *extack)
+{
+	u32 source_pin_idx;
+
+	if (WARN_ON(!dpll->ops->source_pin_idx_get))
+		return -EOPNOTSUPP;
+	if (dpll->ops->source_pin_idx_get(dpll, &source_pin_idx, extack))
+		return -EFAULT;
+	if (nla_put_u32(msg, DPLL_A_SOURCE_PIN_IDX, source_pin_idx))
+		return -EMSGSIZE;
+
+	return 0;
+}
+
+static int
+dpll_msg_add_lock_status(struct sk_buff *msg, struct dpll_device *dpll,
+			 struct netlink_ext_ack *extack)
+{
+	enum dpll_lock_status status;
+
+	if (WARN_ON(!dpll->ops->lock_status_get))
+		return -EOPNOTSUPP;
+	if (dpll->ops->lock_status_get(dpll, &status, extack))
+		return -EFAULT;
+	if (nla_put_u8(msg, DPLL_A_LOCK_STATUS, status))
+		return -EMSGSIZE;
+
+	return 0;
+}
+
+static int
+dpll_msg_add_temp(struct sk_buff *msg, struct dpll_device *dpll,
+		  struct netlink_ext_ack *extack)
+{
+	s32 temp;
+
+	if (!dpll->ops->temp_get)
+		return -EOPNOTSUPP;
+	if (dpll->ops->temp_get(dpll, &temp, extack))
+		return -EFAULT;
+	if (nla_put_s32(msg, DPLL_A_TEMP, temp))
+		return -EMSGSIZE;
+
+	return 0;
+}
+
+static int
+dpll_msg_add_pin_prio(struct sk_buff *msg, const struct dpll_pin *pin,
+		      struct dpll_pin_ref *ref,
+		      struct netlink_ext_ack *extack)
+{
+	u32 prio;
+
+	if (!ref->ops->prio_get)
+		return -EOPNOTSUPP;
+	if (ref->ops->prio_get(pin, ref->dpll, &prio, extack))
+		return -EFAULT;
+	if (nla_put_u32(msg, DPLL_A_PIN_PRIO, prio))
+		return -EMSGSIZE;
+
+	return 0;
+}
+
+static int
+dpll_msg_add_pin_on_dpll_state(struct sk_buff *msg, const struct dpll_pin *pin,
+			       struct dpll_pin_ref *ref,
+			       struct netlink_ext_ack *extack)
+{
+	enum dpll_pin_state state;
+
+	if (!ref->ops->state_on_dpll_get)
+		return -EOPNOTSUPP;
+	if (ref->ops->state_on_dpll_get(pin, ref->dpll, &state, extack))
+		return -EFAULT;
+	if (nla_put_u8(msg, DPLL_A_PIN_STATE, state))
+		return -EMSGSIZE;
+
+	return 0;
+}
+
+static int
+dpll_msg_add_pin_direction(struct sk_buff *msg, const struct dpll_pin *pin,
+			   struct netlink_ext_ack *extack)
+{
+	enum dpll_pin_direction direction;
+	struct dpll_pin_ref *ref;
+	unsigned long i;
+
+	xa_for_each((struct xarray *)&pin->dpll_refs, i, ref) {
+		if (ref && ref->ops && ref->dpll)
+			break;
+	}
+	if (!ref || !ref->ops || !ref->dpll)
+		return -ENODEV;
+	if (!ref->ops->direction_get)
+		return -EOPNOTSUPP;
+	if (ref->ops->direction_get(pin, ref->dpll, &direction, extack))
+		return -EFAULT;
+	if (nla_put_u8(msg, DPLL_A_PIN_DIRECTION, direction))
+		return -EMSGSIZE;
+
+	return 0;
+}
+
+static int
+dpll_msg_add_pin_freq(struct sk_buff *msg, const struct dpll_pin *pin,
+		      struct netlink_ext_ack *extack, bool dump_any_freq)
+{
+	enum dpll_pin_freq_supp fs;
+	struct dpll_pin_ref *ref;
+	unsigned long i;
+	u32 freq;
+
+	xa_for_each((struct xarray *)&pin->dpll_refs, i, ref) {
+		if (ref && ref->ops && ref->dpll)
+			break;
+	}
+	if (!ref || !ref->ops || !ref->dpll)
+		return -ENODEV;
+	if (!ref->ops->frequency_get)
+		return -EOPNOTSUPP;
+	if (ref->ops->frequency_get(pin, ref->dpll, &freq, extack))
+		return -EFAULT;
+	if (nla_put_u32(msg, DPLL_A_PIN_FREQUENCY, freq))
+		return -EMSGSIZE;
+	if (!dump_any_freq)
+		return 0;
+	for (fs = DPLL_PIN_FREQ_SUPP_UNSPEC + 1;
+	     fs <= DPLL_PIN_FREQ_SUPP_MAX; fs++) {
+		if (test_bit(fs, &pin->prop.freq_supported)) {
+			if (nla_put_u32(msg, DPLL_A_PIN_FREQUENCY_SUPPORTED,
+			    dpll_pin_freq_value[fs]))
+				return -EMSGSIZE;
+		}
+	}
+	if (pin->prop.any_freq_min && pin->prop.any_freq_max) {
+		if (nla_put_u32(msg, DPLL_A_PIN_ANY_FREQUENCY_MIN,
+				pin->prop.any_freq_min))
+			return -EMSGSIZE;
+		if (nla_put_u32(msg, DPLL_A_PIN_ANY_FREQUENCY_MAX,
+				pin->prop.any_freq_max))
+			return -EMSGSIZE;
+	}
+
+	return 0;
+}
+
+static int
+dpll_msg_add_pin_parents(struct sk_buff *msg, struct dpll_pin *pin,
+			 struct netlink_ext_ack *extack)
+{
+	struct dpll_pin_ref *ref_parent;
+	enum dpll_pin_state state;
+	struct nlattr *nest;
+	unsigned long index;
+	int ret;
+
+	xa_for_each(&pin->parent_refs, index, ref_parent) {
+		if (WARN_ON(!ref_parent->ops->state_on_pin_get))
+			return -EFAULT;
+		ret = ref_parent->ops->state_on_pin_get(pin, ref_parent->pin,
+							&state, extack);
+		if (ret)
+			return -EFAULT;
+		nest = nla_nest_start(msg, DPLL_A_PIN_PARENT);
+		if (!nest)
+			return -EMSGSIZE;
+		if (nla_put_u32(msg, DPLL_A_PIN_PARENT_IDX,
+				ref_parent->pin->dev_driver_id)) {
+			ret = -EMSGSIZE;
+			goto nest_cancel;
+		}
+		if (nla_put_u8(msg, DPLL_A_PIN_STATE, state)) {
+			ret = -EMSGSIZE;
+			goto nest_cancel;
+		}
+		nla_nest_end(msg, nest);
+	}
+
+	return 0;
+
+nest_cancel:
+	nla_nest_cancel(msg, nest);
+	return ret;
+}
+
+static int
+dpll_msg_add_pins_on_pin(struct sk_buff *msg, struct dpll_pin *pin,
+			 struct netlink_ext_ack *extack)
+{
+	struct dpll_pin_ref *ref = NULL;
+	enum dpll_pin_state state;
+	struct nlattr *nest;
+	unsigned long index;
+	int ret;
+
+	xa_for_each(&pin->parent_refs, index, ref) {
+		if (WARN_ON(!ref->ops->state_on_pin_get))
+			return -EFAULT;
+		ret = ref->ops->state_on_pin_get(pin, ref->pin, &state,
+						 extack);
+		if (ret)
+			return -EFAULT;
+		nest = nla_nest_start(msg, DPLL_A_PIN_PARENT);
+		if (!nest)
+			return -EMSGSIZE;
+		if (nla_put_u32(msg, DPLL_A_PIN_PARENT_IDX,
+				ref->pin->dev_driver_id)) {
+			ret = -EMSGSIZE;
+			goto nest_cancel;
+		}
+		if (nla_put_u8(msg, DPLL_A_PIN_STATE, state)) {
+			ret = -EMSGSIZE;
+			goto nest_cancel;
+		}
+		nla_nest_end(msg, nest);
+	}
+
+	return 0;
+
+nest_cancel:
+	nla_nest_cancel(msg, nest);
+	return ret;
+}
+
+static int
+dpll_msg_add_pin_dplls(struct sk_buff *msg, struct dpll_pin *pin,
+		       struct netlink_ext_ack *extack)
+{
+	struct dpll_pin_ref *ref;
+	struct nlattr *attr;
+	unsigned long index;
+	int ret;
+
+	xa_for_each(&pin->dpll_refs, index, ref) {
+		attr = nla_nest_start(msg, DPLL_A_DEVICE);
+		if (!attr)
+			return -EMSGSIZE;
+		ret = dpll_msg_add_dev_handle(msg, ref->dpll);
+		if (ret)
+			goto nest_cancel;
+		ret = dpll_msg_add_pin_on_dpll_state(msg, pin, ref, extack);
+		if (ret && ret != -EOPNOTSUPP)
+			goto nest_cancel;
+		ret = dpll_msg_add_pin_prio(msg, pin, ref, extack);
+		if (ret && ret != -EOPNOTSUPP)
+			goto nest_cancel;
+		nla_nest_end(msg, attr);
+	}
+
+	return 0;
+
+nest_cancel:
+	nla_nest_end(msg, attr);
+	return ret;
+}
+
+static int
+dpll_cmd_pin_on_dpll_get(struct sk_buff *msg, struct dpll_pin *pin,
+			 struct dpll_device *dpll,
+			 struct netlink_ext_ack *extack)
+{
+	struct dpll_pin_ref *ref;
+	int ret;
+
+	if (nla_put_u32(msg, DPLL_A_PIN_IDX, pin->dev_driver_id))
+		return -EMSGSIZE;
+	if (nla_put_string(msg, DPLL_A_PIN_DESCRIPTION, pin->prop.description))
+		return -EMSGSIZE;
+	if (nla_put_u8(msg, DPLL_A_PIN_TYPE, pin->prop.type))
+		return -EMSGSIZE;
+	if (nla_put_u32(msg, DPLL_A_PIN_DPLL_CAPS, pin->prop.capabilities))
+		return -EMSGSIZE;
+	ret = dpll_msg_add_pin_direction(msg, pin, extack);
+	if (ret)
+		return ret;
+	ret = dpll_msg_add_pin_freq(msg, pin, extack, true);
+	if (ret && ret != -EOPNOTSUPP)
+		return ret;
+	ref = dpll_xa_ref_dpll_find(&pin->dpll_refs, dpll);
+	if (!ref)
+		return -EFAULT;
+	ret = dpll_msg_add_pin_prio(msg, pin, ref, extack);
+	if (ret && ret != -EOPNOTSUPP)
+		return ret;
+	ret = dpll_msg_add_pin_on_dpll_state(msg, pin, ref, extack);
+	if (ret && ret != -EOPNOTSUPP)
+		return ret;
+	ret = dpll_msg_add_pin_parents(msg, pin, extack);
+	if (ret)
+		return ret;
+	if (pin->rclk_dev_name)
+		if (nla_put_string(msg, DPLL_A_PIN_RCLK_DEVICE,
+				   pin->rclk_dev_name))
+			return -EMSGSIZE;
+
+	return 0;
+}
+
+static int
+__dpll_cmd_pin_dump_one(struct sk_buff *msg, struct dpll_pin *pin,
+			struct netlink_ext_ack *extack, bool dump_dpll)
+{
+	int ret;
+
+	if (nla_put_u32(msg, DPLL_A_PIN_IDX, pin->dev_driver_id))
+		return -EMSGSIZE;
+	if (nla_put_string(msg, DPLL_A_PIN_DESCRIPTION, pin->prop.description))
+		return -EMSGSIZE;
+	if (nla_put_u8(msg, DPLL_A_PIN_TYPE, pin->prop.type))
+		return -EMSGSIZE;
+	ret = dpll_msg_add_pin_direction(msg, pin, extack);
+	if (ret)
+		return ret;
+	ret = dpll_msg_add_pin_freq(msg, pin, extack, true);
+	if (ret && ret != -EOPNOTSUPP)
+		return ret;
+	ret = dpll_msg_add_pins_on_pin(msg, pin, extack);
+	if (ret)
+		return ret;
+	if (!xa_empty(&pin->dpll_refs) && dump_dpll) {
+		ret = dpll_msg_add_pin_dplls(msg, pin, extack);
+		if (ret)
+			return ret;
+	}
+	if (pin->rclk_dev_name)
+		if (nla_put_string(msg, DPLL_A_PIN_RCLK_DEVICE,
+				   pin->rclk_dev_name))
+			return -EMSGSIZE;
+
+	return 0;
+}
+
+static int
+dpll_device_get_one(struct dpll_device *dpll, struct sk_buff *msg,
+		     struct netlink_ext_ack *extack)
+{
+	struct dpll_pin_ref *ref;
+	enum dpll_mode mode;
+	unsigned long i;
+	int ret;
+
+	ret = dpll_msg_add_dev_handle(msg, dpll);
+	if (ret)
+		return ret;
+	ret = dpll_msg_add_source_pin_idx(msg, dpll, extack);
+	if (ret)
+		return ret;
+	ret = dpll_msg_add_temp(msg, dpll, extack);
+	if (ret && ret != -EOPNOTSUPP)
+		return ret;
+	ret = dpll_msg_add_lock_status(msg, dpll, extack);
+	if (ret)
+		return ret;
+	ret = dpll_msg_add_mode(msg, dpll, extack);
+	if (ret)
+		return ret;
+	for (mode = DPLL_MODE_UNSPEC + 1; mode <= DPLL_MODE_MAX; mode++)
+		if (test_bit(mode, &dpll->mode_supported_mask))
+			if (nla_put_s32(msg, DPLL_A_MODE_SUPPORTED, mode))
+				return -EMSGSIZE;
+	if (nla_put_64bit(msg, DPLL_A_CLOCK_ID, sizeof(dpll->clock_id),
+			  &dpll->clock_id, 0))
+		return -EMSGSIZE;
+	if (nla_put_u8(msg, DPLL_A_TYPE, dpll->type))
+		return -EMSGSIZE;
+	xa_for_each(&dpll->pin_refs, i, ref) {
+		struct nlattr *nest = nla_nest_start(msg, DPLL_A_PIN);
+
+		if (!nest) {
+			ret = -EMSGSIZE;
+			break;
+		}
+		ret = dpll_cmd_pin_on_dpll_get(msg, ref->pin, dpll, extack);
+		if (ret) {
+			nla_nest_cancel(msg, nest);
+			break;
+		}
+		nla_nest_end(msg, nest);
+	}
+
+	return ret;
+}
+
+static bool dpll_pin_is_freq_supported(struct dpll_pin *pin, u32 freq)
+{
+	enum dpll_pin_freq_supp fs;
+
+	if (freq >= pin->prop.any_freq_min && freq <= pin->prop.any_freq_max)
+		return true;
+	for (fs = DPLL_PIN_FREQ_SUPP_UNSPEC + 1;
+	     fs <= DPLL_PIN_FREQ_SUPP_MAX; fs++)
+		if (test_bit(fs, &pin->prop.freq_supported))
+			if (freq == dpll_pin_freq_value[fs])
+				return true;
+	return false;
+}
+
+static int
+dpll_pin_freq_set(struct dpll_pin *pin, struct nlattr *a,
+		  struct netlink_ext_ack *extack)
+{
+	u32 freq = nla_get_u32(a);
+	struct dpll_pin_ref *ref;
+	unsigned long i;
+	int ret;
+
+	if (!dpll_pin_is_freq_supported(pin, freq))
+		return -EINVAL;
+
+	xa_for_each(&pin->dpll_refs, i, ref) {
+		ret = ref->ops->frequency_set(pin, ref->dpll, freq, extack);
+		if (ret)
+			return -EFAULT;
+		dpll_pin_notify(ref->dpll, pin, DPLL_A_PIN_FREQUENCY);
+	}
+
+	return 0;
+}
+
+static int
+dpll_pin_on_pin_state_set(struct dpll_device *dpll, struct dpll_pin *pin,
+			  u32 parent_idx, enum dpll_pin_state state,
+			  struct netlink_ext_ack *extack)
+{
+	struct dpll_pin_ref *ref;
+	struct dpll_pin *parent;
+
+	if (!(DPLL_PIN_CAPS_STATE_CAN_CHANGE & pin->prop.capabilities))
+		return -EOPNOTSUPP;
+	parent = dpll_pin_get_by_idx(dpll, parent_idx);
+	if (!parent)
+		return -EINVAL;
+	ref = dpll_xa_ref_pin_find(&pin->parent_refs, parent);
+	if (!ref)
+		return -EINVAL;
+	if (!ref->ops || !ref->ops->state_on_pin_set)
+		return -EOPNOTSUPP;
+	if (ref->ops->state_on_pin_set(pin, parent, state, extack))
+		return -EFAULT;
+	dpll_pin_parent_notify(dpll, pin, parent, DPLL_A_PIN_STATE);
+
+	return 0;
+}
+
+static int
+dpll_pin_state_set(struct dpll_device *dpll, struct dpll_pin *pin,
+		   enum dpll_pin_state state,
+		   struct netlink_ext_ack *extack)
+{
+	struct dpll_pin_ref *ref;
+
+	if (!(DPLL_PIN_CAPS_STATE_CAN_CHANGE & pin->prop.capabilities))
+		return -EOPNOTSUPP;
+	ref = dpll_xa_ref_dpll_find(&pin->dpll_refs, dpll);
+	if (!ref)
+		return -EFAULT;
+	if (!ref->ops || !ref->ops->state_on_dpll_set)
+		return -EOPNOTSUPP;
+	if (ref->ops->state_on_dpll_set(pin, ref->dpll, state, extack))
+		return -EINVAL;
+	dpll_pin_notify(ref->dpll, pin, DPLL_A_PIN_STATE);
+
+	return 0;
+}
+
+static int
+dpll_pin_prio_set(struct dpll_device *dpll, struct dpll_pin *pin,
+		  struct nlattr *prio_attr, struct netlink_ext_ack *extack)
+{
+	struct dpll_pin_ref *ref;
+	u32 prio = nla_get_u8(prio_attr);
+
+	if (!(DPLL_PIN_CAPS_PRIORITY_CAN_CHANGE & pin->prop.capabilities))
+		return -EOPNOTSUPP;
+	ref = dpll_xa_ref_dpll_find(&pin->dpll_refs, dpll);
+	if (!ref)
+		return -EFAULT;
+	if (!ref->ops || !ref->ops->prio_set)
+		return -EOPNOTSUPP;
+	if (ref->ops->prio_set(pin, dpll, prio, extack))
+		return -EINVAL;
+	dpll_pin_notify(dpll, pin, DPLL_A_PIN_PRIO);
+
+	return 0;
+}
+
+static int
+dpll_pin_direction_set(struct dpll_pin *pin, struct nlattr *a,
+		       struct netlink_ext_ack *extack)
+{
+	enum dpll_pin_direction direction = nla_get_u8(a);
+	struct dpll_pin_ref *ref;
+	unsigned long i;
+
+	if (!(DPLL_PIN_CAPS_DIRECTION_CAN_CHANGE & pin->prop.capabilities))
+		return -EOPNOTSUPP;
+
+	xa_for_each(&pin->dpll_refs, i, ref) {
+		if (ref->ops->direction_set(pin, ref->dpll, direction, extack))
+			return -EFAULT;
+		dpll_pin_notify(ref->dpll, pin, DPLL_A_PIN_DIRECTION);
+	}
+
+	return 0;
+}
+
+static int
+dpll_pin_set_from_nlattr(struct dpll_device *dpll,
+			 struct dpll_pin *pin, struct genl_info *info)
+{
+	enum dpll_pin_state state = DPLL_PIN_STATE_UNSPEC;
+	u32 parent_idx = PIN_IDX_INVALID;
+	int rem, ret = -EINVAL;
+	struct nlattr *a;
+
+	nla_for_each_attr(a, genlmsg_data(info->genlhdr),
+			  genlmsg_len(info->genlhdr), rem) {
+		switch (nla_type(a)) {
+		case DPLL_A_PIN_FREQUENCY:
+			ret = dpll_pin_freq_set(pin, a, info->extack);
+			if (ret)
+				return ret;
+			break;
+		case DPLL_A_PIN_DIRECTION:
+			ret = dpll_pin_direction_set(pin, a, info->extack);
+			if (ret)
+				return ret;
+			break;
+		case DPLL_A_PIN_PRIO:
+			ret = dpll_pin_prio_set(dpll, pin, a, info->extack);
+			if (ret)
+				return ret;
+			break;
+		case DPLL_A_PIN_PARENT_IDX:
+			parent_idx = nla_get_u32(a);
+			break;
+		case DPLL_A_PIN_STATE:
+			state = nla_get_u8(a);
+			break;
+		default:
+			break;
+		}
+	}
+	if (state != DPLL_PIN_STATE_UNSPEC) {
+		if (parent_idx == PIN_IDX_INVALID) {
+			ret = dpll_pin_state_set(dpll, pin, state,
+						 info->extack);
+			if (ret)
+				return ret;
+		} else {
+			ret = dpll_pin_on_pin_state_set(dpll, pin, parent_idx,
+							state, info->extack);
+			if (ret)
+				return ret;
+		}
+	}
+
+	return ret;
+}
+
+int dpll_nl_pin_set_doit(struct sk_buff *skb, struct genl_info *info)
+{
+	struct dpll_device *dpll = info->user_ptr[0];
+	struct dpll_pin *pin = info->user_ptr[1];
+
+	return dpll_pin_set_from_nlattr(dpll, pin, info);
+}
+
+int dpll_nl_pin_get_doit(struct sk_buff *skb, struct genl_info *info)
+{
+	struct dpll_pin *pin = info->user_ptr[1];
+	struct nlattr *hdr, *nest;
+	struct sk_buff *msg;
+	int ret;
+
+	if (!pin)
+		return -ENODEV;
+	msg = genlmsg_new(NLMSG_GOODSIZE, GFP_KERNEL);
+	if (!msg)
+		return -ENOMEM;
+	hdr = genlmsg_put_reply(msg, info, &dpll_nl_family, 0,
+				DPLL_CMD_PIN_GET);
+	if (!hdr)
+		return -EMSGSIZE;
+	nest = nla_nest_start(msg, DPLL_A_PIN);
+	if (!nest)
+		return -EMSGSIZE;
+	ret = __dpll_cmd_pin_dump_one(msg, pin, info->extack, true);
+	if (ret) {
+		nlmsg_free(msg);
+		return ret;
+	}
+	nla_nest_end(msg, nest);
+	genlmsg_end(msg, hdr);
+
+	return genlmsg_reply(msg, info);
+}
+
+int dpll_nl_pin_get_dumpit(struct sk_buff *skb, struct netlink_callback *cb)
+{
+	struct nlattr *hdr, *nest;
+	struct dpll_pin *pin;
+	unsigned long i;
+	int ret;
+
+	hdr = genlmsg_put(skb, NETLINK_CB(cb->skb).portid, cb->nlh->nlmsg_seq,
+			  &dpll_nl_family, 0, DPLL_CMD_PIN_GET);
+	if (!hdr)
+		return -EMSGSIZE;
+
+	xa_for_each(&dpll_pin_xa, i, pin) {
+		if (xa_empty(&pin->dpll_refs))
+			continue;
+		nest = nla_nest_start(skb, DPLL_A_PIN);
+		if (!nest) {
+			ret = -EMSGSIZE;
+			break;
+		}
+		ret = __dpll_cmd_pin_dump_one(skb, pin, cb->extack, true);
+		if (ret) {
+			nla_nest_cancel(skb, nest);
+			break;
+		}
+		nla_nest_end(skb, nest);
+	}
+
+	if (ret)
+		genlmsg_cancel(skb, hdr);
+	else
+		genlmsg_end(skb, hdr);
+
+	return ret;
+}
+
+static int
+dpll_set_from_nlattr(struct dpll_device *dpll, struct genl_info *info)
+{
+	struct nlattr *attr;
+	enum dpll_mode mode;
+	int rem, ret = 0;
+
+	nla_for_each_attr(attr, genlmsg_data(info->genlhdr),
+			  genlmsg_len(info->genlhdr), rem) {
+		switch (nla_type(attr)) {
+		case DPLL_A_MODE:
+			mode = nla_get_u8(attr);
+
+			if (!dpll->ops || !dpll->ops->mode_set)
+				return -EOPNOTSUPP;
+			ret = dpll->ops->mode_set(dpll, mode, info->extack);
+			if (ret)
+				return ret;
+			break;
+		default:
+			break;
+		}
+	}
+
+	return ret;
+}
+
+int dpll_nl_device_set_doit(struct sk_buff *skb, struct genl_info *info)
+{
+	struct dpll_device *dpll = info->user_ptr[0];
+
+	return dpll_set_from_nlattr(dpll, info);
+}
+
+int dpll_nl_device_get_doit(struct sk_buff *skb, struct genl_info *info)
+{
+	struct dpll_device *dpll = info->user_ptr[0];
+	struct nlattr *hdr, *nest;
+	struct sk_buff *msg;
+	int ret;
+
+	msg = genlmsg_new(NLMSG_GOODSIZE, GFP_KERNEL);
+	if (!msg)
+		return -ENOMEM;
+	hdr = genlmsg_put_reply(msg, info, &dpll_nl_family, 0,
+				DPLL_CMD_DEVICE_GET);
+	if (!hdr)
+		return -EMSGSIZE;
+
+	nest = nla_nest_start(msg, DPLL_A_DEVICE);
+	ret = dpll_device_get_one(dpll, msg, info->extack);
+	if (ret) {
+		nlmsg_free(msg);
+		return ret;
+	}
+	nla_nest_end(msg, nest);
+	genlmsg_end(msg, hdr);
+
+	return genlmsg_reply(msg, info);
+}
+
+int dpll_nl_device_get_dumpit(struct sk_buff *skb, struct netlink_callback *cb)
+{
+	struct nlattr *hdr, *nest;
+	struct dpll_device *dpll;
+	unsigned long i;
+	int ret;
+
+	hdr = genlmsg_put(skb, NETLINK_CB(cb->skb).portid, cb->nlh->nlmsg_seq,
+			  &dpll_nl_family, 0, DPLL_CMD_DEVICE_GET);
+	if (!hdr)
+		return -EMSGSIZE;
+
+	xa_for_each_marked(&dpll_device_xa, i, dpll, DPLL_REGISTERED) {
+		nest = nla_nest_start(skb, DPLL_A_DEVICE);
+		ret = dpll_msg_add_dev_handle(skb, dpll);
+		if (ret) {
+			nla_nest_cancel(skb, nest);
+			break;
+		}
+		nla_nest_end(skb, nest);
+	}
+	if (ret)
+		genlmsg_cancel(skb, hdr);
+	else
+		genlmsg_end(skb, hdr);
+
+	return ret;
+}
+
+int dpll_pre_doit(const struct genl_split_ops *ops, struct sk_buff *skb,
+		  struct genl_info *info)
+{
+	struct dpll_device *dpll_id = NULL, *dpll_name = NULL;
+	int ret = -ENODEV;
+
+	if (!info->attrs[DPLL_A_ID] &&
+	    !(info->attrs[DPLL_A_BUS_NAME] && info->attrs[DPLL_A_DEV_NAME]))
+		return -EINVAL;
+
+	mutex_lock(&dpll_device_xa_lock);
+	if (info->attrs[DPLL_A_ID]) {
+		u32 id = nla_get_u32(info->attrs[DPLL_A_ID]);
+
+		dpll_id = dpll_device_get_by_id(id);
+		if (!dpll_id)
+			goto unlock;
+		info->user_ptr[0] = dpll_id;
+	}
+	if (info->attrs[DPLL_A_BUS_NAME] &&
+	    info->attrs[DPLL_A_DEV_NAME]) {
+		const char *bus_name = nla_data(info->attrs[DPLL_A_BUS_NAME]);
+		const char *dev_name = nla_data(info->attrs[DPLL_A_DEV_NAME]);
+
+		dpll_name = dpll_device_get_by_name(bus_name, dev_name);
+		if (!dpll_name) {
+			ret = -ENODEV;
+			goto unlock;
+		}
+
+		if (dpll_id && dpll_name != dpll_id)
+			goto unlock;
+		info->user_ptr[0] = dpll_name;
+	}
+
+	return 0;
+unlock:
+	mutex_unlock(&dpll_device_xa_lock);
+	return ret;
+}
+
+void dpll_post_doit(const struct genl_split_ops *ops, struct sk_buff *skb,
+		    struct genl_info *info)
+{
+	mutex_unlock(&dpll_device_xa_lock);
+}
+
+int dpll_pre_dumpit(struct netlink_callback *cb)
+{
+	mutex_lock(&dpll_device_xa_lock);
+
+	return 0;
+}
+
+int dpll_post_dumpit(struct netlink_callback *cb)
+{
+	mutex_unlock(&dpll_device_xa_lock);
+
+	return 0;
+}
+
+int dpll_pin_pre_doit(const struct genl_split_ops *ops, struct sk_buff *skb,
+		      struct genl_info *info)
+{
+	int ret = dpll_pre_doit(ops, skb, info);
+	struct dpll_device *dpll;
+	struct dpll_pin *pin;
+
+	if (ret)
+		return ret;
+	dpll = info->user_ptr[0];
+	if (!info->attrs[DPLL_A_PIN_IDX]) {
+		ret = -EINVAL;
+		goto unlock_dev;
+	}
+	mutex_lock(&dpll_pin_xa_lock);
+	pin = dpll_pin_get_by_idx(dpll,
+				  nla_get_u32(info->attrs[DPLL_A_PIN_IDX]));
+	if (!pin) {
+		ret = -ENODEV;
+		goto unlock_pin;
+	}
+	info->user_ptr[1] = pin;
+
+	return 0;
+
+unlock_pin:
+	mutex_unlock(&dpll_pin_xa_lock);
+unlock_dev:
+	mutex_unlock(&dpll_device_xa_lock);
+	return ret;
+}
+
+void dpll_pin_post_doit(const struct genl_split_ops *ops, struct sk_buff *skb,
+			struct genl_info *info)
+{
+	mutex_unlock(&dpll_pin_xa_lock);
+	dpll_post_doit(ops, skb, info);
+}
+
+int dpll_pin_pre_dumpit(struct netlink_callback *cb)
+{
+	mutex_lock(&dpll_pin_xa_lock);
+
+	return dpll_pre_dumpit(cb);
+}
+
+int dpll_pin_post_dumpit(struct netlink_callback *cb)
+{
+	mutex_unlock(&dpll_pin_xa_lock);
+
+	return dpll_post_dumpit(cb);
+}
+
+static int
+dpll_event_device_change(struct sk_buff *msg, struct dpll_device *dpll,
+			 struct dpll_pin *pin, struct dpll_pin *parent,
+			 enum dplla attr)
+{
+	int ret = dpll_msg_add_dev_handle(msg, dpll);
+	struct dpll_pin_ref *ref = NULL;
+	enum dpll_pin_state state;
+
+	if (ret)
+		return ret;
+	if (pin && nla_put_u32(msg, DPLL_A_PIN_IDX, pin->dev_driver_id))
+		return -EMSGSIZE;
+
+	switch (attr) {
+	case DPLL_A_MODE:
+		ret = dpll_msg_add_mode(msg, dpll, NULL);
+		break;
+	case DPLL_A_SOURCE_PIN_IDX:
+		ret = dpll_msg_add_source_pin_idx(msg, dpll, NULL);
+		break;
+	case DPLL_A_LOCK_STATUS:
+		ret = dpll_msg_add_lock_status(msg, dpll, NULL);
+		break;
+	case DPLL_A_TEMP:
+		ret = dpll_msg_add_temp(msg, dpll, NULL);
+		break;
+	case DPLL_A_PIN_FREQUENCY:
+		ret = dpll_msg_add_pin_freq(msg, pin, NULL, false);
+		break;
+	case DPLL_A_PIN_PRIO:
+		ref = dpll_xa_ref_dpll_find(&pin->dpll_refs, dpll);
+		if (!ref)
+			return -EFAULT;
+		ret = dpll_msg_add_pin_prio(msg, pin, ref, NULL);
+		break;
+	case DPLL_A_PIN_STATE:
+		if (parent) {
+			ref = dpll_xa_ref_pin_find(&pin->parent_refs, parent);
+			if (!ref)
+				return -EFAULT;
+			if (!ref->ops || !ref->ops->state_on_pin_get)
+				return -EOPNOTSUPP;
+			ret = ref->ops->state_on_pin_get(pin, parent, &state,
+							 NULL);
+			if (ret)
+				return ret;
+			if (nla_put_u32(msg, DPLL_A_PIN_PARENT_IDX,
+					parent->dev_driver_id))
+				return -EMSGSIZE;
+		} else {
+			ref = dpll_xa_ref_dpll_find(&pin->dpll_refs, dpll);
+			if (!ref)
+				return -EFAULT;
+			ret = dpll_msg_add_pin_on_dpll_state(msg, pin, ref,
+							     NULL);
+			if (ret)
+				return ret;
+		}
+		break;
+	default:
+		break;
+	}
+
+	return ret;
+}
+
+static int
+dpll_send_event_create(enum dpll_event event, struct dpll_device *dpll)
+{
+	struct sk_buff *msg;
+	int ret = -EMSGSIZE;
+	void *hdr;
+
+	msg = genlmsg_new(NLMSG_GOODSIZE, GFP_KERNEL);
+	if (!msg)
+		return -ENOMEM;
+
+	hdr = genlmsg_put(msg, 0, 0, &dpll_nl_family, 0, event);
+	if (!hdr)
+		goto out_free_msg;
+
+	ret = dpll_msg_add_dev_handle(msg, dpll);
+	if (ret)
+		goto out_cancel_msg;
+	genlmsg_end(msg, hdr);
+	genlmsg_multicast(&dpll_nl_family, msg, 0, 0, GFP_KERNEL);
+
+	return 0;
+
+out_cancel_msg:
+	genlmsg_cancel(msg, hdr);
+out_free_msg:
+	nlmsg_free(msg);
+
+	return ret;
+}
+
+static int
+dpll_send_event_change(struct dpll_device *dpll, struct dpll_pin *pin,
+		       struct dpll_pin *parent, enum dplla attr)
+{
+	struct sk_buff *msg;
+	int ret = -EMSGSIZE;
+	void *hdr;
+
+	msg = genlmsg_new(NLMSG_GOODSIZE, GFP_KERNEL);
+	if (!msg)
+		return -ENOMEM;
+
+	hdr = genlmsg_put(msg, 0, 0, &dpll_nl_family, 0,
+			  DPLL_EVENT_DEVICE_CHANGE);
+	if (!hdr)
+		goto out_free_msg;
+
+	ret = dpll_event_device_change(msg, dpll, pin, parent, attr);
+	if (ret)
+		goto out_cancel_msg;
+	genlmsg_end(msg, hdr);
+	genlmsg_multicast(&dpll_nl_family, msg, 0, 0, GFP_KERNEL);
+
+	return 0;
+
+out_cancel_msg:
+	genlmsg_cancel(msg, hdr);
+out_free_msg:
+	nlmsg_free(msg);
+
+	return ret;
+}
+
+int dpll_notify_device_create(struct dpll_device *dpll)
+{
+	return dpll_send_event_create(DPLL_EVENT_DEVICE_CREATE, dpll);
+}
+
+int dpll_notify_device_delete(struct dpll_device *dpll)
+{
+	return dpll_send_event_create(DPLL_EVENT_DEVICE_DELETE, dpll);
+}
+
+int dpll_device_notify(struct dpll_device *dpll, enum dplla attr)
+{
+	if (WARN_ON(!dpll))
+		return -EINVAL;
+
+	return dpll_send_event_change(dpll, NULL, NULL, attr);
+}
+EXPORT_SYMBOL_GPL(dpll_device_notify);
+
+int dpll_pin_notify(struct dpll_device *dpll, struct dpll_pin *pin,
+		    enum dplla attr)
+{
+	return dpll_send_event_change(dpll, pin, NULL, attr);
+}
+
+int dpll_pin_parent_notify(struct dpll_device *dpll, struct dpll_pin *pin,
+			   struct dpll_pin *parent, enum dplla attr)
+{
+	return dpll_send_event_change(dpll, pin, parent, attr);
+}
+
+int __init dpll_netlink_init(void)
+{
+	return genl_register_family(&dpll_nl_family);
+}
+
+void dpll_netlink_finish(void)
+{
+	genl_unregister_family(&dpll_nl_family);
+}
+
+void __exit dpll_netlink_fini(void)
+{
+	dpll_netlink_finish();
+}
diff --git a/drivers/dpll/dpll_netlink.h b/drivers/dpll/dpll_netlink.h
new file mode 100644
index 000000000000..072efa10f0e6
--- /dev/null
+++ b/drivers/dpll/dpll_netlink.h
@@ -0,0 +1,30 @@ 
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ *  Copyright (c) 2021 Meta Platforms, Inc. and affiliates
+ */
+
+/**
+ * dpll_notify_device_create - notify that the device has been created
+ * @dpll: registered dpll pointer
+ *
+ * Return: 0 if succeeds, error code otherwise.
+ */
+int dpll_notify_device_create(struct dpll_device *dpll);
+
+
+/**
+ * dpll_notify_device_delete - notify that the device has been deleted
+ * @dpll: registered dpll pointer
+ *
+ * Return: 0 if succeeds, error code otherwise.
+ */
+int dpll_notify_device_delete(struct dpll_device *dpll);
+
+int dpll_pin_notify(struct dpll_device *dpll, struct dpll_pin *pin,
+		    enum dplla attr);
+
+int dpll_pin_parent_notify(struct dpll_device *dpll, struct dpll_pin *pin,
+			   struct dpll_pin *parent, enum dplla attr);
+
+int __init dpll_netlink_init(void);
+void dpll_netlink_finish(void);
diff --git a/include/linux/dpll.h b/include/linux/dpll.h
new file mode 100644
index 000000000000..db98b6d4bb73
--- /dev/null
+++ b/include/linux/dpll.h
@@ -0,0 +1,284 @@ 
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ *  Copyright (c) 2021 Meta Platforms, Inc. and affiliates
+ */
+
+#ifndef __DPLL_H__
+#define __DPLL_H__
+
+#include <uapi/linux/dpll.h>
+#include <linux/device.h>
+#include <linux/netlink.h>
+
+struct dpll_device;
+struct dpll_pin;
+
+#define PIN_IDX_INVALID		((u32)ULONG_MAX)
+
+struct dpll_device_ops {
+	int (*mode_get)(const struct dpll_device *dpll, enum dpll_mode *mode,
+			struct netlink_ext_ack *extack);
+	int (*mode_set)(const struct dpll_device *dpll,
+			const enum dpll_mode mode,
+			struct netlink_ext_ack *extack);
+	bool (*mode_supported)(const struct dpll_device *dpll,
+			       const enum dpll_mode mode,
+			       struct netlink_ext_ack *extack);
+	int (*source_pin_idx_get)(const struct dpll_device *dpll,
+				  u32 *pin_idx,
+				  struct netlink_ext_ack *extack);
+	int (*lock_status_get)(const struct dpll_device *dpll,
+			       enum dpll_lock_status *status,
+			       struct netlink_ext_ack *extack);
+	int (*temp_get)(const struct dpll_device *dpll, s32 *temp,
+			struct netlink_ext_ack *extack);
+};
+
+struct dpll_pin_ops {
+	int (*frequency_set)(const struct dpll_pin *pin,
+			     const struct dpll_device *dpll,
+			     const u32 frequency,
+			     struct netlink_ext_ack *extack);
+	int (*frequency_get)(const struct dpll_pin *pin,
+			     const struct dpll_device *dpll,
+			     u32 *frequency, struct netlink_ext_ack *extack);
+	int (*direction_set)(const struct dpll_pin *pin,
+			     const struct dpll_device *dpll,
+			     const enum dpll_pin_direction direction,
+			     struct netlink_ext_ack *extack);
+	int (*direction_get)(const struct dpll_pin *pin,
+			     const struct dpll_device *dpll,
+			     enum dpll_pin_direction *direction,
+			     struct netlink_ext_ack *extack);
+	int (*state_on_pin_get)(const struct dpll_pin *pin,
+				const struct dpll_pin *parent_pin,
+				enum dpll_pin_state *state,
+				struct netlink_ext_ack *extack);
+	int (*state_on_dpll_get)(const struct dpll_pin *pin,
+				 const struct dpll_device *dpll,
+				 enum dpll_pin_state *state,
+				 struct netlink_ext_ack *extack);
+	int (*state_on_pin_set)(const struct dpll_pin *pin,
+				const struct dpll_pin *parent_pin,
+				const enum dpll_pin_state state,
+				struct netlink_ext_ack *extack);
+	int (*state_on_dpll_set)(const struct dpll_pin *pin,
+				 const struct dpll_device *dpll,
+				 const enum dpll_pin_state state,
+				 struct netlink_ext_ack *extack);
+	int (*prio_get)(const struct dpll_pin *pin,
+			const struct dpll_device *dpll,
+			u32 *prio, struct netlink_ext_ack *extack);
+	int (*prio_set)(const struct dpll_pin *pin,
+			const struct dpll_device *dpll,
+			const u32 prio, struct netlink_ext_ack *extack);
+};
+
+struct dpll_pin_properties {
+	const char *description;
+	enum dpll_pin_type type;
+	unsigned long freq_supported;
+	u32 any_freq_min;
+	u32 any_freq_max;
+	unsigned long capabilities;
+};
+
+enum dpll_pin_freq_supp {
+	DPLL_PIN_FREQ_SUPP_UNSPEC = 0,
+	DPLL_PIN_FREQ_SUPP_1_HZ,
+	DPLL_PIN_FREQ_SUPP_10_MHZ,
+
+	__DPLL_PIN_FREQ_SUPP_MAX,
+	DPLL_PIN_FREQ_SUPP_MAX = (__DPLL_PIN_FREQ_SUPP_MAX - 1)
+};
+
+/**
+ * dpll_device_get - find or create dpll_device object
+ * @clock_id: a system unique number for a device
+ * @dev_driver_idx: index of dpll device on parent device
+ * @module: register module
+ *
+ * Returns:
+ * * pointer to initialized dpll - success
+ * * NULL - memory allocation fail
+ */
+struct dpll_device
+*dpll_device_get(u64 clock_id, u32 dev_driver_id, struct module *module);
+
+/**
+ * dpll_device_put - caller drops reference to the device, free resources
+ * @dpll: dpll device pointer
+ *
+ * If all dpll_device_get callers drops their reference, the dpll device
+ * resources are freed.
+ */
+void dpll_device_put(struct dpll_device *dpll);
+
+/**
+ * dpll_device_register - register device, make it visible in the subsystem.
+ * @dpll: reference previously allocated with dpll_device_get
+ * @type: type of dpll
+ * @ops: callbacks
+ * @priv: private data of registerer
+ * @owner: device struct of the owner
+ *
+ */
+int dpll_device_register(struct dpll_device *dpll, enum dpll_type type,
+			 struct dpll_device_ops *ops, void *priv,
+			 struct device *owner);
+
+/**
+ * dpll_device_unregister - deregister registered dpll
+ * @dpll: pointer to dpll
+ *
+ * Unregister the dpll from the subsystem, make it unavailable for netlink
+ * API users.
+ */
+void dpll_device_unregister(struct dpll_device *dpll);
+
+/**
+ * dpll_priv - get dpll private data
+ * @dpll: pointer to dpll
+ *
+ * Obtain private data pointer passed to dpll subsystem when allocating
+ * device with ``dpll_device_alloc(..)``
+ */
+void *dpll_priv(const struct dpll_device *dpll);
+
+/**
+ * dpll_pin_on_pin_priv - get pin on pin pair private data
+ * @parent: pointer to a parent pin
+ * @pin: pointer to a dpll_pin
+ *
+ * Obtain private pin data pointer passed to dpll subsystem when pin
+ * was registered with parent pin.
+ */
+void *dpll_pin_on_pin_priv(const struct dpll_pin *parent, const struct dpll_pin *pin);
+
+/**
+ * dpll_pin_on_dpll_priv - get pin on dpll pair private data
+ * @dpll: pointer to dpll
+ * @pin: pointer to a dpll_pin
+ *
+ * Obtain private pin-dpll pair data pointer passed to dpll subsystem when pin
+ * was registered with a dpll.
+ */
+void *dpll_pin_on_dpll_priv(const struct dpll_device *dpll, const struct dpll_pin *pin);
+
+/**
+ * dpll_pin_get - get reference or create new pin object
+ * @clock_id: a system unique number of a device
+ * @dev_driver_idx: index of dpll device on parent device
+ * @module: register module
+ * @pin_prop: constant properities of a pin
+ *
+ * find existing pin with given clock_id, dev_driver_idx and module, or create new
+ * and returen its reference.
+ *
+ * Returns:
+ * * pointer to initialized pin - success
+ * * NULL - memory allocation fail
+ */
+struct dpll_pin
+*dpll_pin_get(u64 clock_id, u32 dev_driver_id, struct module *module,
+	      const struct dpll_pin_properties *pin_prop);
+
+/**
+ * dpll_pin_register - register pin with a dpll device
+ * @dpll: pointer to dpll object to register pin with
+ * @pin: pointer to allocated pin object being registered with dpll
+ * @ops: struct with pin ops callbacks
+ * @priv: private data pointer passed when calling callback ops
+ * @rclk_device: pointer to device struct if pin is used for recovery of a clock
+ * from that device
+ *
+ * Register previously allocated pin object with a dpll device.
+ *
+ * Return:
+ * * 0 - if pin was registered with a parent pin,
+ * * -ENOMEM - failed to allocate memory,
+ * * -EEXIST - pin already registered with this dpll,
+ * * -EBUSY - couldn't allocate id for a pin.
+ */
+int dpll_pin_register(struct dpll_device *dpll, struct dpll_pin *pin,
+		      struct dpll_pin_ops *ops, void *priv,
+		      struct device *rclk_device);
+
+/**
+ * dpll_pin_unregister - deregister pin from a dpll device
+ * @dpll: pointer to dpll object to deregister pin from
+ * @pin: pointer to allocated pin object being deregistered from dpll
+ *
+ * Deregister previously registered pin object from a dpll device.
+ *
+ * Return:
+ * * 0 - pin was successfully deregistered from this dpll device,
+ * * -ENXIO - given pin was not registered with this dpll device,
+ * * -EINVAL - pin pointer is not valid.
+ */
+int dpll_pin_unregister(struct dpll_device *dpll, struct dpll_pin *pin);
+
+/**
+ * dpll_pin_put - drop reference to a pin acquired with dpll_pin_get
+ * @pin: pointer to allocated pin
+ *
+ * Pins shall be deregistered from all dpll devices before putting them,
+ * otherwise the memory won't be freed.
+ */
+void dpll_pin_put(struct dpll_pin *pin);
+
+/**
+ * dpll_pin_on_pin_register - register a pin to a muxed-type pin
+ * @parent: parent pin pointer
+ * @pin: pointer to allocated pin object being registered with a parent pin
+ * @ops: struct with pin ops callbacks
+ * @priv: private data pointer passed when calling callback ops
+ * @rclk_device: pointer to device struct if pin is used for recovery of a clock
+ * from that device
+ *
+ * In case of multiplexed pins, allows registring them under a single
+ * parent pin.
+ *
+ * Return:
+ * * 0 - if pin was registered with a parent pin,
+ * * -ENOMEM - failed to allocate memory,
+ * * -EEXIST - pin already registered with this parent pin,
+ */
+int dpll_pin_on_pin_register(struct dpll_pin *parent, struct dpll_pin *pin,
+			     struct dpll_pin_ops *ops, void *priv,
+			     struct device *rclk_device);
+
+/**
+ * dpll_pin_on_pin_register - register a pin to a muxed-type pin
+ * @parent: parent pin pointer
+ * @pin: pointer to allocated pin object being registered with a parent pin
+ * @ops: struct with pin ops callbacks
+ * @priv: private data pointer passed when calling callback ops
+ * @rclk_device: pointer to device struct if pin is used for recovery of a clock
+ * from that device
+ *
+ * In case of multiplexed pins, allows registring them under a single
+ * parent pin.
+ *
+ * Return:
+ * * 0 - if pin was registered with a parent pin,
+ * * -ENOMEM - failed to allocate memory,
+ * * -EEXIST - pin already registered with this parent pin,
+ */
+void dpll_pin_on_pin_unregister(struct dpll_pin *parent, struct dpll_pin *pin);
+
+/**
+ * dpll_device_notify - notify on dpll device change
+ * @dpll: dpll device pointer
+ * @attr: changed attribute
+ *
+ * Broadcast event to the netlink multicast registered listeners.
+ *
+ * Return:
+ * * 0 - success
+ * * negative - error
+ */
+int dpll_device_notify(struct dpll_device *dpll, enum dplla attr);
+
+
+#endif