| Message ID | 20230308051310.12544-12-shannon.nelson@amd.com (mailing list archive) |
|---|---|
| State | RFC |
| Delegated to: | Netdev Maintainers |
| Headers | show |
| Series | pds_core driver | expand |
On Tue, Mar 07, 2023 at 09:13:08PM -0800, Shannon Nelson wrote: > Add the client API operations for registering, unregistering, > and running adminq commands. We expect to add additional > operations for other clients, including requesting additional > private adminqs and IRQs, but don't have the need yet, > > Signed-off-by: Shannon Nelson <shannon.nelson@amd.com> > --- > drivers/net/ethernet/amd/pds_core/auxbus.c | 134 ++++++++++++++++++++- > include/linux/pds/pds_auxbus.h | 42 +++++++ > 2 files changed, 174 insertions(+), 2 deletions(-) <...> > +static struct pds_core_ops pds_core_ops = { > + .register_client = pds_client_register, > + .unregister_client = pds_client_unregister, > + .adminq_cmd = pds_client_adminq_cmd, > +}; <...> > +/* > + * ptrs to functions to be used by the client for core services > + */ > +struct pds_core_ops { > + /* .register() - register the client with the device > + * padev: ptr to the client device info > + * Register the client with the core and with the DSC. The core > + * will fill in the client padrv->client_id for use in calls > + * to the DSC AdminQ > + */ > + int (*register_client)(struct pds_auxiliary_dev *padev); > + > + /* .unregister() - disconnect the client from the device > + * padev: ptr to the client device info > + * Disconnect the client from the core and with the DSC. > + */ > + int (*unregister_client)(struct pds_auxiliary_dev *padev); > + > + /* .adminq_cmd() - process an adminq request for the client > + * padev: ptr to the client device > + * req: ptr to buffer with request > + * req_len: length of actual struct used for request > + * resp: ptr to buffer where answer is to be copied > + * flags: optional flags defined by enum pds_core_adminq_flags > + * and used for more flexible adminq behvior > + * > + * returns 0 on success, or > + * negative for error > + * Client sends pointers to request and response buffers > + * Core copies request data into pds_core_client_request_cmd > + * Core sets other fields as needed > + * Core posts to AdminQ > + * Core copies completion data into response buffer > + */ > + int (*adminq_cmd)(struct pds_auxiliary_dev *padev, > + union pds_core_adminq_cmd *req, > + size_t req_len, > + union pds_core_adminq_comp *resp, > + u64 flags); > +}; I don't expect to see any register/unregister AUX client code at all. All clients are registered and unregistered through auxiliary_driver_register()/auxiliary_driver_unregister() calls and perform as standalone drivers. Maybe client, register and unregister words means something else in this series.. Thanks > #endif /* _PDSC_AUXBUS_H_ */ > -- > 2.17.1 >
On 3/14/23 5:14 AM, Leon Romanovsky wrote: > On Tue, Mar 07, 2023 at 09:13:08PM -0800, Shannon Nelson wrote: >> Add the client API operations for registering, unregistering, >> and running adminq commands. We expect to add additional >> operations for other clients, including requesting additional >> private adminqs and IRQs, but don't have the need yet, >> >> Signed-off-by: Shannon Nelson <shannon.nelson@amd.com> >> --- >> drivers/net/ethernet/amd/pds_core/auxbus.c | 134 ++++++++++++++++++++- >> include/linux/pds/pds_auxbus.h | 42 +++++++ >> 2 files changed, 174 insertions(+), 2 deletions(-) > > <...> > >> +static struct pds_core_ops pds_core_ops = { >> + .register_client = pds_client_register, >> + .unregister_client = pds_client_unregister, >> + .adminq_cmd = pds_client_adminq_cmd, >> +}; > > <...> > >> +/* >> + * ptrs to functions to be used by the client for core services >> + */ >> +struct pds_core_ops { >> + /* .register() - register the client with the device >> + * padev: ptr to the client device info >> + * Register the client with the core and with the DSC. The core >> + * will fill in the client padrv->client_id for use in calls >> + * to the DSC AdminQ >> + */ >> + int (*register_client)(struct pds_auxiliary_dev *padev); >> + >> + /* .unregister() - disconnect the client from the device >> + * padev: ptr to the client device info >> + * Disconnect the client from the core and with the DSC. >> + */ >> + int (*unregister_client)(struct pds_auxiliary_dev *padev); >> + >> + /* .adminq_cmd() - process an adminq request for the client >> + * padev: ptr to the client device >> + * req: ptr to buffer with request >> + * req_len: length of actual struct used for request >> + * resp: ptr to buffer where answer is to be copied >> + * flags: optional flags defined by enum pds_core_adminq_flags >> + * and used for more flexible adminq behvior >> + * >> + * returns 0 on success, or >> + * negative for error >> + * Client sends pointers to request and response buffers >> + * Core copies request data into pds_core_client_request_cmd >> + * Core sets other fields as needed >> + * Core posts to AdminQ >> + * Core copies completion data into response buffer >> + */ >> + int (*adminq_cmd)(struct pds_auxiliary_dev *padev, >> + union pds_core_adminq_cmd *req, >> + size_t req_len, >> + union pds_core_adminq_comp *resp, >> + u64 flags); >> +}; > > I don't expect to see any register/unregister AUX client code at all. > > All clients are registered and unregistered through > auxiliary_driver_register()/auxiliary_driver_unregister() calls and > perform as standalone drivers. > > Maybe client, register and unregister words means something else in this > series.. Yeah, I'm not thrilled with the overlap in nomenclature either. In this case we're talking about the logic in the pds_vdpa module connecting to the services needed in the device FW, and getting a client_id from the FW that is used for tracking client context in the FW. Maybe these names can change to something like "fw_client_reg" and "fw_client_unreg" - would that make it more clear? sln > > Thanks > >> #endif /* _PDSC_AUXBUS_H_ */ >> -- >> 2.17.1 >>
On Tue, Mar 14, 2023 at 09:53:25AM -0700, Shannon Nelson wrote: > On 3/14/23 5:14 AM, Leon Romanovsky wrote: > > On Tue, Mar 07, 2023 at 09:13:08PM -0800, Shannon Nelson wrote: > > > Add the client API operations for registering, unregistering, > > > and running adminq commands. We expect to add additional > > > operations for other clients, including requesting additional > > > private adminqs and IRQs, but don't have the need yet, > > > > > > Signed-off-by: Shannon Nelson <shannon.nelson@amd.com> > > > --- > > > drivers/net/ethernet/amd/pds_core/auxbus.c | 134 ++++++++++++++++++++- > > > include/linux/pds/pds_auxbus.h | 42 +++++++ > > > 2 files changed, 174 insertions(+), 2 deletions(-) > > > > <...> > > > > > +static struct pds_core_ops pds_core_ops = { > > > + .register_client = pds_client_register, > > > + .unregister_client = pds_client_unregister, > > > + .adminq_cmd = pds_client_adminq_cmd, > > > +}; > > > > <...> > > > > > +/* > > > + * ptrs to functions to be used by the client for core services > > > + */ > > > +struct pds_core_ops { > > > + /* .register() - register the client with the device > > > + * padev: ptr to the client device info > > > + * Register the client with the core and with the DSC. The core > > > + * will fill in the client padrv->client_id for use in calls > > > + * to the DSC AdminQ > > > + */ > > > + int (*register_client)(struct pds_auxiliary_dev *padev); > > > + > > > + /* .unregister() - disconnect the client from the device > > > + * padev: ptr to the client device info > > > + * Disconnect the client from the core and with the DSC. > > > + */ > > > + int (*unregister_client)(struct pds_auxiliary_dev *padev); > > > + > > > + /* .adminq_cmd() - process an adminq request for the client > > > + * padev: ptr to the client device > > > + * req: ptr to buffer with request > > > + * req_len: length of actual struct used for request > > > + * resp: ptr to buffer where answer is to be copied > > > + * flags: optional flags defined by enum pds_core_adminq_flags > > > + * and used for more flexible adminq behvior > > > + * > > > + * returns 0 on success, or > > > + * negative for error > > > + * Client sends pointers to request and response buffers > > > + * Core copies request data into pds_core_client_request_cmd > > > + * Core sets other fields as needed > > > + * Core posts to AdminQ > > > + * Core copies completion data into response buffer > > > + */ > > > + int (*adminq_cmd)(struct pds_auxiliary_dev *padev, > > > + union pds_core_adminq_cmd *req, > > > + size_t req_len, > > > + union pds_core_adminq_comp *resp, > > > + u64 flags); > > > +}; > > > > I don't expect to see any register/unregister AUX client code at all. > > > > All clients are registered and unregistered through > > auxiliary_driver_register()/auxiliary_driver_unregister() calls and > > perform as standalone drivers. > > > > Maybe client, register and unregister words means something else in this > > series.. > > Yeah, I'm not thrilled with the overlap in nomenclature either. In this > case we're talking about the logic in the pds_vdpa module connecting to the > services needed in the device FW, and getting a client_id from the FW that > is used for tracking client context in the FW. Maybe these names can change > to something like "fw_client_reg" and "fw_client_unreg" - would that make it > more clear? I feel that such ops are not needed at all. Once you create aux devices (vdpa, eth, e.t.c), you would be able to connect only one driver with one such device. It means context is already known at that point. In addition, user controls if he/she wants aux specific devices by relevant devlink *_enable knob. Thanks > > sln > > > > > > Thanks > > > > > #endif /* _PDSC_AUXBUS_H_ */ > > > -- > > > 2.17.1 > > >
On 3/15/23 1:21 AM, Leon Romanovsky wrote: > On Tue, Mar 14, 2023 at 09:53:25AM -0700, Shannon Nelson wrote: >> On 3/14/23 5:14 AM, Leon Romanovsky wrote: >>> On Tue, Mar 07, 2023 at 09:13:08PM -0800, Shannon Nelson wrote: >>>> Add the client API operations for registering, unregistering, >>>> and running adminq commands. We expect to add additional >>>> operations for other clients, including requesting additional >>>> private adminqs and IRQs, but don't have the need yet, >>>> >>>> Signed-off-by: Shannon Nelson <shannon.nelson@amd.com> >>>> --- >>>> drivers/net/ethernet/amd/pds_core/auxbus.c | 134 ++++++++++++++++++++- >>>> include/linux/pds/pds_auxbus.h | 42 +++++++ >>>> 2 files changed, 174 insertions(+), 2 deletions(-) >>> >>> <...> >>> >>>> +static struct pds_core_ops pds_core_ops = { >>>> + .register_client = pds_client_register, >>>> + .unregister_client = pds_client_unregister, >>>> + .adminq_cmd = pds_client_adminq_cmd, >>>> +}; >>> >>> <...> >>> >>>> +/* >>>> + * ptrs to functions to be used by the client for core services >>>> + */ >>>> +struct pds_core_ops { >>>> + /* .register() - register the client with the device >>>> + * padev: ptr to the client device info >>>> + * Register the client with the core and with the DSC. The core >>>> + * will fill in the client padrv->client_id for use in calls >>>> + * to the DSC AdminQ >>>> + */ >>>> + int (*register_client)(struct pds_auxiliary_dev *padev); >>>> + >>>> + /* .unregister() - disconnect the client from the device >>>> + * padev: ptr to the client device info >>>> + * Disconnect the client from the core and with the DSC. >>>> + */ >>>> + int (*unregister_client)(struct pds_auxiliary_dev *padev); >>>> + >>>> + /* .adminq_cmd() - process an adminq request for the client >>>> + * padev: ptr to the client device >>>> + * req: ptr to buffer with request >>>> + * req_len: length of actual struct used for request >>>> + * resp: ptr to buffer where answer is to be copied >>>> + * flags: optional flags defined by enum pds_core_adminq_flags >>>> + * and used for more flexible adminq behvior >>>> + * >>>> + * returns 0 on success, or >>>> + * negative for error >>>> + * Client sends pointers to request and response buffers >>>> + * Core copies request data into pds_core_client_request_cmd >>>> + * Core sets other fields as needed >>>> + * Core posts to AdminQ >>>> + * Core copies completion data into response buffer >>>> + */ >>>> + int (*adminq_cmd)(struct pds_auxiliary_dev *padev, >>>> + union pds_core_adminq_cmd *req, >>>> + size_t req_len, >>>> + union pds_core_adminq_comp *resp, >>>> + u64 flags); >>>> +}; >>> >>> I don't expect to see any register/unregister AUX client code at all. >>> >>> All clients are registered and unregistered through >>> auxiliary_driver_register()/auxiliary_driver_unregister() calls and >>> perform as standalone drivers. >>> >>> Maybe client, register and unregister words means something else in this >>> series.. >> >> Yeah, I'm not thrilled with the overlap in nomenclature either. In this >> case we're talking about the logic in the pds_vdpa module connecting to the >> services needed in the device FW, and getting a client_id from the FW that >> is used for tracking client context in the FW. Maybe these names can change >> to something like "fw_client_reg" and "fw_client_unreg" - would that make it >> more clear? > > I feel that such ops are not needed at all. Once you create aux devices > (vdpa, eth, e.t.c), you would be able to connect only one driver with one > such device. It means context is already known at that point. In addition, > user controls if he/she wants aux specific devices by relevant devlink *_enable > knob. > > Thanks As this code matures here, as well as the internal gyrations happening, this makes more and more sense. Some of the original work was there for greater flexibility in what we might need "soon", but much has gone away. This is one of those things that we can whittle down. The registration is still needed to tell the FW about the client, but can be moved out of the clients' responsibility and into the VF startup. Thanks, sln
diff --git a/drivers/net/ethernet/amd/pds_core/auxbus.c b/drivers/net/ethernet/amd/pds_core/auxbus.c index 535fee627874..3f0e0bd13184 100644 --- a/drivers/net/ethernet/amd/pds_core/auxbus.c +++ b/drivers/net/ethernet/amd/pds_core/auxbus.c @@ -5,6 +5,132 @@ #include <linux/pds/pds_adminq.h> #include <linux/pds/pds_auxbus.h> +/** + * pds_client_register - Register the client with the device + * @padev: ptr to the client device info + * + * Register the client with the core and with the DSC. The core + * will fill in the client padev->client_id for use in calls + * to the DSC AdminQ + */ +static int pds_client_register(struct pds_auxiliary_dev *padev) +{ + union pds_core_adminq_comp comp = { 0 }; + union pds_core_adminq_cmd cmd = { 0 }; + struct device *dev = padev->pf->dev; + struct pdsc *pf = padev->pf; + int err; + u16 ci; + + if (pf->state) + return -ENXIO; + + cmd.client_reg.opcode = PDS_AQ_CMD_CLIENT_REG; + strscpy(cmd.client_reg.devname, dev_name(&padev->aux_dev.dev), + sizeof(cmd.client_reg.devname)); + + err = pdsc_adminq_post(pf, &pf->adminqcq, &cmd, &comp, false); + if (err) { + dev_info(dev, "register dev_name %s with DSC failed, status %d: %pe\n", + dev_name(&padev->aux_dev.dev), comp.status, ERR_PTR(err)); + return err; + } + + ci = le16_to_cpu(comp.client_reg.client_id); + if (!ci) { + dev_err(dev, "%s: device returned null client_id\n", __func__); + return -EIO; + } + + padev->client_id = ci; + + return 0; +} + +/** + * pds_client_unregister - Disconnect the client from the device + * @padev: ptr to the client device info + * + * Disconnect the client from the core and with the DSC. + */ +static int pds_client_unregister(struct pds_auxiliary_dev *padev) +{ + union pds_core_adminq_comp comp = { 0 }; + union pds_core_adminq_cmd cmd = { 0 }; + struct device *dev = padev->pf->dev; + struct pdsc *pf = padev->pf; + int err; + + if (pf->state) + return -ENXIO; + + cmd.client_unreg.opcode = PDS_AQ_CMD_CLIENT_UNREG; + cmd.client_unreg.client_id = cpu_to_le16(padev->client_id); + + err = pdsc_adminq_post(pf, &pf->adminqcq, &cmd, &comp, false); + if (err) + dev_info(dev, "unregister dev_name %s failed, status %d: %pe\n", + dev_name(&padev->aux_dev.dev), comp.status, ERR_PTR(err)); + + padev->client_id = 0; + + return err; +} + +/** + * pds_client_adminq_cmd - Process an adminq request for the client + * @padev: ptr to the client device + * @req: ptr to buffer with request + * @req_len: length of actual struct used for request + * @resp: ptr to buffer where answer is to be copied + * @flags: optional flags from pds_core_adminq_flags + * + * Return: 0 on success, or + * negative for error + * + * Client sends pointers to request and response buffers + * Core copies request data into pds_core_client_request_cmd + * Core sets other fields as needed + * Core posts to AdminQ + * Core copies completion data into response buffer + */ +static int pds_client_adminq_cmd(struct pds_auxiliary_dev *padev, + union pds_core_adminq_cmd *req, + size_t req_len, + union pds_core_adminq_comp *resp, + u64 flags) +{ + union pds_core_adminq_cmd cmd = { 0 }; + struct device *dev = padev->pf->dev; + struct pdsc *pf = padev->pf; + size_t cp_len; + int err; + + dev_dbg(dev, "%s: %s opcode %d\n", + __func__, dev_name(&padev->aux_dev.dev), req->opcode); + + if (pf->state) + return -ENXIO; + + /* Wrap the client's request */ + cmd.client_request.opcode = PDS_AQ_CMD_CLIENT_CMD; + cmd.client_request.client_id = cpu_to_le16(padev->client_id); + cp_len = min_t(size_t, req_len, sizeof(cmd.client_request.client_cmd)); + memcpy(cmd.client_request.client_cmd, req, cp_len); + + err = pdsc_adminq_post(pf, &pf->adminqcq, &cmd, resp, !!(flags & PDS_AQ_FLAG_FASTPOLL)); + if (err && err != -EAGAIN) + dev_info(dev, "client admin cmd failed: %pe\n", ERR_PTR(err)); + + return err; +} + +static struct pds_core_ops pds_core_ops = { + .register_client = pds_client_register, + .unregister_client = pds_client_unregister, + .adminq_cmd = pds_client_adminq_cmd, +}; + static void pdsc_auxbus_dev_release(struct device *dev) { struct pds_auxiliary_dev *padev = @@ -15,7 +141,8 @@ static void pdsc_auxbus_dev_release(struct device *dev) static struct pds_auxiliary_dev *pdsc_auxbus_dev_register(struct pdsc *vf, struct pdsc *pf, - char *name) + char *name, + struct pds_core_ops *ops) { struct auxiliary_device *aux_dev; struct pds_auxiliary_dev *padev; @@ -27,6 +154,7 @@ static struct pds_auxiliary_dev *pdsc_auxbus_dev_register(struct pdsc *vf, padev->vf = vf; padev->pf = pf; + padev->ops = ops; aux_dev = &padev->aux_dev; aux_dev->name = name; @@ -94,7 +222,9 @@ int pdsc_auxbus_dev_add_vf(struct pdsc *vf, struct pdsc *pf) pf->viftype_status[vt].enabled)) continue; - padev = pdsc_auxbus_dev_register(vf, pf, pf->viftype_status[vt].name); + padev = pdsc_auxbus_dev_register(vf, pf, + pf->viftype_status[vt].name, + &pds_core_ops); pf->vfs[vf->vf_id].padev = padev; /* We only support a single type per VF, so jump out here */ diff --git a/include/linux/pds/pds_auxbus.h b/include/linux/pds/pds_auxbus.h index 30c643878a2b..e3e99fe58d0e 100644 --- a/include/linux/pds/pds_auxbus.h +++ b/include/linux/pds/pds_auxbus.h @@ -10,7 +10,49 @@ struct pds_auxiliary_dev { struct auxiliary_device aux_dev; struct pdsc *vf; struct pdsc *pf; + struct pds_core_ops *ops; u16 client_id; void *priv; }; + +/* + * ptrs to functions to be used by the client for core services + */ +struct pds_core_ops { + /* .register() - register the client with the device + * padev: ptr to the client device info + * Register the client with the core and with the DSC. The core + * will fill in the client padrv->client_id for use in calls + * to the DSC AdminQ + */ + int (*register_client)(struct pds_auxiliary_dev *padev); + + /* .unregister() - disconnect the client from the device + * padev: ptr to the client device info + * Disconnect the client from the core and with the DSC. + */ + int (*unregister_client)(struct pds_auxiliary_dev *padev); + + /* .adminq_cmd() - process an adminq request for the client + * padev: ptr to the client device + * req: ptr to buffer with request + * req_len: length of actual struct used for request + * resp: ptr to buffer where answer is to be copied + * flags: optional flags defined by enum pds_core_adminq_flags + * and used for more flexible adminq behvior + * + * returns 0 on success, or + * negative for error + * Client sends pointers to request and response buffers + * Core copies request data into pds_core_client_request_cmd + * Core sets other fields as needed + * Core posts to AdminQ + * Core copies completion data into response buffer + */ + int (*adminq_cmd)(struct pds_auxiliary_dev *padev, + union pds_core_adminq_cmd *req, + size_t req_len, + union pds_core_adminq_comp *resp, + u64 flags); +}; #endif /* _PDSC_AUXBUS_H_ */
Add the client API operations for registering, unregistering, and running adminq commands. We expect to add additional operations for other clients, including requesting additional private adminqs and IRQs, but don't have the need yet, Signed-off-by: Shannon Nelson <shannon.nelson@amd.com> --- drivers/net/ethernet/amd/pds_core/auxbus.c | 134 ++++++++++++++++++++- include/linux/pds/pds_auxbus.h | 42 +++++++ 2 files changed, 174 insertions(+), 2 deletions(-)