| Message ID | 20240415-pme_msg-v6-5-56dad968ad3a@nxp.com (mailing list archive) |
|---|---|
| State | Superseded |
| Headers | show |
| Series | PCI: dwc: Add common pme_turn_off message by using outbound iATU | expand |
On Mon, Apr 15, 2024 at 03:33:29PM -0400, Frank Li wrote: > Instead of relying on the vendor specific implementations to send the > PME_Turn_Off message, let's introduce a generic way of sending the message > using the MSG TLP. > > This is achieved by reserving a region for MSG TLP of size > 'pci->region_align', at the end of the first IORESOURCE_MEM window of the > host bridge. And then sending the PME_Turn_Off message during system > suspend with the help of iATU. > > The reserve space at end is a little bit better than alloc_resource() > because the below reasons. > - alloc_resource() will allocate space at begin of IORESOURCE_MEM window. > There will be a hole when first Endpoint Device (EP) try to alloc a bigger > space then 'region_align' size. > - Keep EP device's IORESOURCE_MEM space unchange with/without this patch. > I'd rewrite the above sentence as: 'The reason for reserving the MSG TLP region at the end of the IORESOURCE_MEM is to avoid generating holes in between. Because, when the region is allocated using allocate_resource(), memory will be allocated from the start of the window. Later, if memory gets allocated for an endpoint of size bigger than 'region_align', there will be a hole between MSG TLP region and endpoint memory.' > It should be noted that this generic implementation is optional for the > glue drivers and can be overridden by a custom 'pme_turn_off' callback. > > Signed-off-by: Frank Li <Frank.Li@nxp.com> > --- > drivers/pci/controller/dwc/pcie-designware-host.c | 94 ++++++++++++++++++++++- > drivers/pci/controller/dwc/pcie-designware.h | 3 + > 2 files changed, 93 insertions(+), 4 deletions(-) > > diff --git a/drivers/pci/controller/dwc/pcie-designware-host.c b/drivers/pci/controller/dwc/pcie-designware-host.c > index 3a9cb4be22ab2..5001cdf220123 100644 > --- a/drivers/pci/controller/dwc/pcie-designware-host.c > +++ b/drivers/pci/controller/dwc/pcie-designware-host.c > @@ -398,6 +398,31 @@ static int dw_pcie_msi_host_init(struct dw_pcie_rp *pp) > return 0; > } > > +static void dw_pcie_host_request_msg_tlp_res(struct dw_pcie_rp *pp) > +{ > + struct dw_pcie *pci = to_dw_pcie_from_pp(pp); > + struct resource_entry *win; > + struct resource *res; > + > + win = resource_list_first_type(&pp->bridge->windows, IORESOURCE_MEM); > + if (win) { > + res = devm_kzalloc(pci->dev, sizeof(*res), GFP_KERNEL); > + if (!res) > + return; > + > + /* Reserve last region_align block as message TLP space */ /* * Allocate MSG TLP region of size 'region_align' at the end of * the host bridge window. */ > + res->start = win->res->end - pci->region_align + 1; > + res->end = win->res->end; > + res->name = "msg"; > + res->flags = win->res->flags | IORESOURCE_BUSY; > + > + if (!devm_request_resource(pci->dev, win->res, res)) > + pp->msg_res = res; > + else > + devm_kfree(pci->dev, res); You are explicitly freeing 'msg_res' everywhere. So either drop devm_ or rely on devm to free the memory. > + } > +} > + > int dw_pcie_host_init(struct dw_pcie_rp *pp) > { > struct dw_pcie *pci = to_dw_pcie_from_pp(pp); > @@ -484,6 +509,16 @@ int dw_pcie_host_init(struct dw_pcie_rp *pp) > > dw_pcie_iatu_detect(pci); > > + /* > + * Allocate the resource for MSG TLP before programming the iATU outbound window in > + * dw_pcie_setup_rc(). Since the allocation depends on the value of 'region_align', this has > + * to be done after dw_pcie_iatu_detect(). Please wrap the comments to 80 columns. > + * > + * Glue driver need set use_atu_msg before dw_pcie_host_init() if want MSG TLP feature. How about, * Glue drivers need to set 'use_atu_msg' before dw_pcie_host_init() to * make use of the generic MSG TLP implementation. > + */ > + if (pp->use_atu_msg) > + dw_pcie_host_request_msg_tlp_res(pp); > + > ret = dw_pcie_edma_detect(pci); > if (ret) > goto err_free_msi; > @@ -541,6 +576,11 @@ void dw_pcie_host_deinit(struct dw_pcie_rp *pp) > > dw_pcie_edma_remove(pci); > > + if (pp->msg_res) { > + release_resource(pp->msg_res); > + devm_kfree(pci->dev, pp->msg_res); > + } > + > if (pp->has_msi_ctrl) > dw_pcie_free_msi(pp); > > @@ -702,6 +742,10 @@ static int dw_pcie_iatu_setup(struct dw_pcie_rp *pp) > atu.pci_addr = entry->res->start - entry->offset; > atu.size = resource_size(entry->res); > > + /* MSG TLB window resource reserve at one of end of IORESOURCE_MEM resource */ How about, /* Adjust iATU size if MSG TLP region was allocated before */ if (pp->msg_res && pp->msg_res->parent == entry->res) atu.size = resource_size(entry->res) - resource_size(pp->msg_res); else atu.size = resource_size(entry->res); > + if (pp->msg_res && pp->msg_res->parent == entry->res) > + atu.size -= resource_size(pp->msg_res); > + > ret = dw_pcie_prog_outbound_atu(pci, &atu); > if (ret) { > dev_err(pci->dev, "Failed to set MEM range %pr\n", > @@ -733,6 +777,8 @@ static int dw_pcie_iatu_setup(struct dw_pcie_rp *pp) > dev_warn(pci->dev, "Ranges exceed outbound iATU size (%d)\n", > pci->num_ob_windows); > > + pp->msg_atu_index = i; > + > i = 0; > resource_list_for_each_entry(entry, &pp->bridge->dma_ranges) { > if (resource_type(entry->res) != IORESOURCE_MEM) > @@ -838,11 +884,48 @@ int dw_pcie_setup_rc(struct dw_pcie_rp *pp) > } > EXPORT_SYMBOL_GPL(dw_pcie_setup_rc); > > +/* Using message outbound ATU to send out PME_Turn_Off message for dwc PCIe controller */ No need of this comment. > +static int dw_pcie_pme_turn_off(struct dw_pcie *pci) > +{ > + struct dw_pcie_ob_atu_cfg atu = { 0 }; > + void __iomem *mem; > + int ret; > + > + if (pci->num_ob_windows <= pci->pp.msg_atu_index) > + return -EINVAL; return -ENOSPC; > + > + if (!pci->pp.msg_res) > + return -EINVAL; return -ENOSPC; - Mani
On Wed, Apr 17, 2024 at 03:49:44PM +0530, Manivannan Sadhasivam wrote: > On Mon, Apr 15, 2024 at 03:33:29PM -0400, Frank Li wrote: > > Instead of relying on the vendor specific implementations to send the > > PME_Turn_Off message, let's introduce a generic way of sending the message > > using the MSG TLP. > > > > This is achieved by reserving a region for MSG TLP of size > > 'pci->region_align', at the end of the first IORESOURCE_MEM window of the > > host bridge. And then sending the PME_Turn_Off message during system > > suspend with the help of iATU. > > > > The reserve space at end is a little bit better than alloc_resource() > > because the below reasons. > > - alloc_resource() will allocate space at begin of IORESOURCE_MEM window. > > There will be a hole when first Endpoint Device (EP) try to alloc a bigger > > space then 'region_align' size. > > - Keep EP device's IORESOURCE_MEM space unchange with/without this patch. > > > > I'd rewrite the above sentence as: > > 'The reason for reserving the MSG TLP region at the end of the > IORESOURCE_MEM is to avoid generating holes in between. Because, when the region > is allocated using allocate_resource(), memory will be allocated from the start > of the window. Later, if memory gets allocated for an endpoint of size bigger > than 'region_align', there will be a hole between MSG TLP region and endpoint > memory.' > > > It should be noted that this generic implementation is optional for the > > glue drivers and can be overridden by a custom 'pme_turn_off' callback. > > > > Signed-off-by: Frank Li <Frank.Li@nxp.com> > > --- > > drivers/pci/controller/dwc/pcie-designware-host.c | 94 ++++++++++++++++++++++- > > drivers/pci/controller/dwc/pcie-designware.h | 3 + > > 2 files changed, 93 insertions(+), 4 deletions(-) > > > > diff --git a/drivers/pci/controller/dwc/pcie-designware-host.c b/drivers/pci/controller/dwc/pcie-designware-host.c > > index 3a9cb4be22ab2..5001cdf220123 100644 > > --- a/drivers/pci/controller/dwc/pcie-designware-host.c > > +++ b/drivers/pci/controller/dwc/pcie-designware-host.c > > @@ -398,6 +398,31 @@ static int dw_pcie_msi_host_init(struct dw_pcie_rp *pp) > > return 0; > > } > > > > +static void dw_pcie_host_request_msg_tlp_res(struct dw_pcie_rp *pp) > > +{ > > + struct dw_pcie *pci = to_dw_pcie_from_pp(pp); > > + struct resource_entry *win; > > + struct resource *res; > > + > > + win = resource_list_first_type(&pp->bridge->windows, IORESOURCE_MEM); > > + if (win) { > > + res = devm_kzalloc(pci->dev, sizeof(*res), GFP_KERNEL); > > + if (!res) > > + return; > > + > > + /* Reserve last region_align block as message TLP space */ > > /* > * Allocate MSG TLP region of size 'region_align' at the end of > * the host bridge window. > */ > > > + res->start = win->res->end - pci->region_align + 1; > > + res->end = win->res->end; > > + res->name = "msg"; > > + res->flags = win->res->flags | IORESOURCE_BUSY; > > + > > + if (!devm_request_resource(pci->dev, win->res, res)) > > + pp->msg_res = res; > > + else > > + devm_kfree(pci->dev, res); > > You are explicitly freeing 'msg_res' everywhere. So either drop devm_ or rely > on devm to free the memory. > > > + } > > +} > > + > > int dw_pcie_host_init(struct dw_pcie_rp *pp) > > { > > struct dw_pcie *pci = to_dw_pcie_from_pp(pp); > > @@ -484,6 +509,16 @@ int dw_pcie_host_init(struct dw_pcie_rp *pp) > > > > dw_pcie_iatu_detect(pci); > > > > + /* > > + * Allocate the resource for MSG TLP before programming the iATU outbound window in > > + * dw_pcie_setup_rc(). Since the allocation depends on the value of 'region_align', this has > > + * to be done after dw_pcie_iatu_detect(). > > Please wrap the comments to 80 columns. New code style is 100 columns. does comments still stick to 80 columns? > > > + * > > + * Glue driver need set use_atu_msg before dw_pcie_host_init() if want MSG TLP feature. > > How about, > > * Glue drivers need to set 'use_atu_msg' before dw_pcie_host_init() to > * make use of the generic MSG TLP implementation. > > > + */ > > + if (pp->use_atu_msg) > > + dw_pcie_host_request_msg_tlp_res(pp); > > + > > ret = dw_pcie_edma_detect(pci); > > if (ret) > > goto err_free_msi; > > @@ -541,6 +576,11 @@ void dw_pcie_host_deinit(struct dw_pcie_rp *pp) > > > > dw_pcie_edma_remove(pci); > > > > + if (pp->msg_res) { > > + release_resource(pp->msg_res); > > + devm_kfree(pci->dev, pp->msg_res); > > + } > > + > > if (pp->has_msi_ctrl) > > dw_pcie_free_msi(pp); > > > > @@ -702,6 +742,10 @@ static int dw_pcie_iatu_setup(struct dw_pcie_rp *pp) > > atu.pci_addr = entry->res->start - entry->offset; > > atu.size = resource_size(entry->res); > > > > + /* MSG TLB window resource reserve at one of end of IORESOURCE_MEM resource */ > > How about, > > /* Adjust iATU size if MSG TLP region was allocated before */ > if (pp->msg_res && pp->msg_res->parent == entry->res) > atu.size = resource_size(entry->res) - > resource_size(pp->msg_res); > else > atu.size = resource_size(entry->res); > > > + if (pp->msg_res && pp->msg_res->parent == entry->res) > > + atu.size -= resource_size(pp->msg_res); > > + > > ret = dw_pcie_prog_outbound_atu(pci, &atu); > > if (ret) { > > dev_err(pci->dev, "Failed to set MEM range %pr\n", > > @@ -733,6 +777,8 @@ static int dw_pcie_iatu_setup(struct dw_pcie_rp *pp) > > dev_warn(pci->dev, "Ranges exceed outbound iATU size (%d)\n", > > pci->num_ob_windows); > > > > + pp->msg_atu_index = i; > > + > > i = 0; > > resource_list_for_each_entry(entry, &pp->bridge->dma_ranges) { > > if (resource_type(entry->res) != IORESOURCE_MEM) > > @@ -838,11 +884,48 @@ int dw_pcie_setup_rc(struct dw_pcie_rp *pp) > > } > > EXPORT_SYMBOL_GPL(dw_pcie_setup_rc); > > > > +/* Using message outbound ATU to send out PME_Turn_Off message for dwc PCIe controller */ > > No need of this comment. > > > +static int dw_pcie_pme_turn_off(struct dw_pcie *pci) > > +{ > > + struct dw_pcie_ob_atu_cfg atu = { 0 }; > > + void __iomem *mem; > > + int ret; > > + > > + if (pci->num_ob_windows <= pci->pp.msg_atu_index) > > + return -EINVAL; > > return -ENOSPC; > > > + > > + if (!pci->pp.msg_res) > > + return -EINVAL; > > return -ENOSPC; > - Mani > > -- > மணிவண்ணன் சதாசிவம்
On Wed, Apr 17, 2024 at 10:38:04AM -0400, Frank Li wrote: > On Wed, Apr 17, 2024 at 03:49:44PM +0530, Manivannan Sadhasivam wrote: > > On Mon, Apr 15, 2024 at 03:33:29PM -0400, Frank Li wrote: > > > Instead of relying on the vendor specific implementations to send the > > > PME_Turn_Off message, let's introduce a generic way of sending the message > > > using the MSG TLP. > > > > > > This is achieved by reserving a region for MSG TLP of size > > > 'pci->region_align', at the end of the first IORESOURCE_MEM window of the > > > host bridge. And then sending the PME_Turn_Off message during system > > > suspend with the help of iATU. > > > > > > The reserve space at end is a little bit better than alloc_resource() > > > because the below reasons. > > > - alloc_resource() will allocate space at begin of IORESOURCE_MEM window. > > > There will be a hole when first Endpoint Device (EP) try to alloc a bigger > > > space then 'region_align' size. > > > - Keep EP device's IORESOURCE_MEM space unchange with/without this patch. > > > > > > > I'd rewrite the above sentence as: > > > > 'The reason for reserving the MSG TLP region at the end of the > > IORESOURCE_MEM is to avoid generating holes in between. Because, when the region > > is allocated using allocate_resource(), memory will be allocated from the start > > of the window. Later, if memory gets allocated for an endpoint of size bigger > > than 'region_align', there will be a hole between MSG TLP region and endpoint > > memory.' > > > > > It should be noted that this generic implementation is optional for the > > > glue drivers and can be overridden by a custom 'pme_turn_off' callback. > > > > > > Signed-off-by: Frank Li <Frank.Li@nxp.com> > > > --- > > > drivers/pci/controller/dwc/pcie-designware-host.c | 94 ++++++++++++++++++++++- > > > drivers/pci/controller/dwc/pcie-designware.h | 3 + > > > 2 files changed, 93 insertions(+), 4 deletions(-) > > > > > > diff --git a/drivers/pci/controller/dwc/pcie-designware-host.c b/drivers/pci/controller/dwc/pcie-designware-host.c > > > index 3a9cb4be22ab2..5001cdf220123 100644 > > > --- a/drivers/pci/controller/dwc/pcie-designware-host.c > > > +++ b/drivers/pci/controller/dwc/pcie-designware-host.c > > > @@ -398,6 +398,31 @@ static int dw_pcie_msi_host_init(struct dw_pcie_rp *pp) > > > return 0; > > > } > > > > > > +static void dw_pcie_host_request_msg_tlp_res(struct dw_pcie_rp *pp) > > > +{ > > > + struct dw_pcie *pci = to_dw_pcie_from_pp(pp); > > > + struct resource_entry *win; > > > + struct resource *res; > > > + > > > + win = resource_list_first_type(&pp->bridge->windows, IORESOURCE_MEM); > > > + if (win) { > > > + res = devm_kzalloc(pci->dev, sizeof(*res), GFP_KERNEL); > > > + if (!res) > > > + return; > > > + > > > + /* Reserve last region_align block as message TLP space */ > > > > /* > > * Allocate MSG TLP region of size 'region_align' at the end of > > * the host bridge window. > > */ > > > > > + res->start = win->res->end - pci->region_align + 1; > > > + res->end = win->res->end; > > > + res->name = "msg"; > > > + res->flags = win->res->flags | IORESOURCE_BUSY; > > > + > > > + if (!devm_request_resource(pci->dev, win->res, res)) > > > + pp->msg_res = res; > > > + else > > > + devm_kfree(pci->dev, res); > > > > You are explicitly freeing 'msg_res' everywhere. So either drop devm_ or rely > > on devm to free the memory. > > > > > + } > > > +} > > > + > > > int dw_pcie_host_init(struct dw_pcie_rp *pp) > > > { > > > struct dw_pcie *pci = to_dw_pcie_from_pp(pp); > > > @@ -484,6 +509,16 @@ int dw_pcie_host_init(struct dw_pcie_rp *pp) > > > > > > dw_pcie_iatu_detect(pci); > > > > > > + /* > > > + * Allocate the resource for MSG TLP before programming the iATU outbound window in > > > + * dw_pcie_setup_rc(). Since the allocation depends on the value of 'region_align', this has > > > + * to be done after dw_pcie_iatu_detect(). > > > > Please wrap the comments to 80 columns. > > New code style is 100 columns. does comments still stick to 80 columns? > That's only applicable for code and not comments. I also learned that some time back. - Mani > > > > > + * > > > + * Glue driver need set use_atu_msg before dw_pcie_host_init() if want MSG TLP feature. > > > > How about, > > > > * Glue drivers need to set 'use_atu_msg' before dw_pcie_host_init() to > > * make use of the generic MSG TLP implementation. > > > > > + */ > > > + if (pp->use_atu_msg) > > > + dw_pcie_host_request_msg_tlp_res(pp); > > > + > > > ret = dw_pcie_edma_detect(pci); > > > if (ret) > > > goto err_free_msi; > > > @@ -541,6 +576,11 @@ void dw_pcie_host_deinit(struct dw_pcie_rp *pp) > > > > > > dw_pcie_edma_remove(pci); > > > > > > + if (pp->msg_res) { > > > + release_resource(pp->msg_res); > > > + devm_kfree(pci->dev, pp->msg_res); > > > + } > > > + > > > if (pp->has_msi_ctrl) > > > dw_pcie_free_msi(pp); > > > > > > @@ -702,6 +742,10 @@ static int dw_pcie_iatu_setup(struct dw_pcie_rp *pp) > > > atu.pci_addr = entry->res->start - entry->offset; > > > atu.size = resource_size(entry->res); > > > > > > + /* MSG TLB window resource reserve at one of end of IORESOURCE_MEM resource */ > > > > How about, > > > > /* Adjust iATU size if MSG TLP region was allocated before */ > > if (pp->msg_res && pp->msg_res->parent == entry->res) > > atu.size = resource_size(entry->res) - > > resource_size(pp->msg_res); > > else > > atu.size = resource_size(entry->res); > > > > > + if (pp->msg_res && pp->msg_res->parent == entry->res) > > > + atu.size -= resource_size(pp->msg_res); > > > + > > > ret = dw_pcie_prog_outbound_atu(pci, &atu); > > > if (ret) { > > > dev_err(pci->dev, "Failed to set MEM range %pr\n", > > > @@ -733,6 +777,8 @@ static int dw_pcie_iatu_setup(struct dw_pcie_rp *pp) > > > dev_warn(pci->dev, "Ranges exceed outbound iATU size (%d)\n", > > > pci->num_ob_windows); > > > > > > + pp->msg_atu_index = i; > > > + > > > i = 0; > > > resource_list_for_each_entry(entry, &pp->bridge->dma_ranges) { > > > if (resource_type(entry->res) != IORESOURCE_MEM) > > > @@ -838,11 +884,48 @@ int dw_pcie_setup_rc(struct dw_pcie_rp *pp) > > > } > > > EXPORT_SYMBOL_GPL(dw_pcie_setup_rc); > > > > > > +/* Using message outbound ATU to send out PME_Turn_Off message for dwc PCIe controller */ > > > > No need of this comment. > > > > > +static int dw_pcie_pme_turn_off(struct dw_pcie *pci) > > > +{ > > > + struct dw_pcie_ob_atu_cfg atu = { 0 }; > > > + void __iomem *mem; > > > + int ret; > > > + > > > + if (pci->num_ob_windows <= pci->pp.msg_atu_index) > > > + return -EINVAL; > > > > return -ENOSPC; > > > > > + > > > + if (!pci->pp.msg_res) > > > + return -EINVAL; > > > > return -ENOSPC; > > - Mani > > > > -- > > மணிவண்ணன் சதாசிவம்
On Wed, Apr 17, 2024 at 10:38:04AM -0400, Frank Li wrote: > On Wed, Apr 17, 2024 at 03:49:44PM +0530, Manivannan Sadhasivam wrote: > > On Mon, Apr 15, 2024 at 03:33:29PM -0400, Frank Li wrote: > > > + /* > > > + * Allocate the resource for MSG TLP before programming the iATU outbound window in > > > + * dw_pcie_setup_rc(). Since the allocation depends on the value of 'region_align', this has > > > + * to be done after dw_pcie_iatu_detect(). > > > > Please wrap the comments to 80 columns. > > New code style is 100 columns. does comments still stick to 80 columns? I care more about files being consistent internally than I do about adhering to a new 100 column rule. With very few exceptions, pcie-designware-host.c fits in 80 columns. I think there's no point in adding a comment using 100 columns. That leadds to either a window sized to accommodate 100 columns, where 20 columns are wasted for all the existing code and comments, or a window sized for 80 columns, where the new comment looks ugly. Neither is attractive. Bjorn
diff --git a/drivers/pci/controller/dwc/pcie-designware-host.c b/drivers/pci/controller/dwc/pcie-designware-host.c index 3a9cb4be22ab2..5001cdf220123 100644 --- a/drivers/pci/controller/dwc/pcie-designware-host.c +++ b/drivers/pci/controller/dwc/pcie-designware-host.c @@ -398,6 +398,31 @@ static int dw_pcie_msi_host_init(struct dw_pcie_rp *pp) return 0; } +static void dw_pcie_host_request_msg_tlp_res(struct dw_pcie_rp *pp) +{ + struct dw_pcie *pci = to_dw_pcie_from_pp(pp); + struct resource_entry *win; + struct resource *res; + + win = resource_list_first_type(&pp->bridge->windows, IORESOURCE_MEM); + if (win) { + res = devm_kzalloc(pci->dev, sizeof(*res), GFP_KERNEL); + if (!res) + return; + + /* Reserve last region_align block as message TLP space */ + res->start = win->res->end - pci->region_align + 1; + res->end = win->res->end; + res->name = "msg"; + res->flags = win->res->flags | IORESOURCE_BUSY; + + if (!devm_request_resource(pci->dev, win->res, res)) + pp->msg_res = res; + else + devm_kfree(pci->dev, res); + } +} + int dw_pcie_host_init(struct dw_pcie_rp *pp) { struct dw_pcie *pci = to_dw_pcie_from_pp(pp); @@ -484,6 +509,16 @@ int dw_pcie_host_init(struct dw_pcie_rp *pp) dw_pcie_iatu_detect(pci); + /* + * Allocate the resource for MSG TLP before programming the iATU outbound window in + * dw_pcie_setup_rc(). Since the allocation depends on the value of 'region_align', this has + * to be done after dw_pcie_iatu_detect(). + * + * Glue driver need set use_atu_msg before dw_pcie_host_init() if want MSG TLP feature. + */ + if (pp->use_atu_msg) + dw_pcie_host_request_msg_tlp_res(pp); + ret = dw_pcie_edma_detect(pci); if (ret) goto err_free_msi; @@ -541,6 +576,11 @@ void dw_pcie_host_deinit(struct dw_pcie_rp *pp) dw_pcie_edma_remove(pci); + if (pp->msg_res) { + release_resource(pp->msg_res); + devm_kfree(pci->dev, pp->msg_res); + } + if (pp->has_msi_ctrl) dw_pcie_free_msi(pp); @@ -702,6 +742,10 @@ static int dw_pcie_iatu_setup(struct dw_pcie_rp *pp) atu.pci_addr = entry->res->start - entry->offset; atu.size = resource_size(entry->res); + /* MSG TLB window resource reserve at one of end of IORESOURCE_MEM resource */ + if (pp->msg_res && pp->msg_res->parent == entry->res) + atu.size -= resource_size(pp->msg_res); + ret = dw_pcie_prog_outbound_atu(pci, &atu); if (ret) { dev_err(pci->dev, "Failed to set MEM range %pr\n", @@ -733,6 +777,8 @@ static int dw_pcie_iatu_setup(struct dw_pcie_rp *pp) dev_warn(pci->dev, "Ranges exceed outbound iATU size (%d)\n", pci->num_ob_windows); + pp->msg_atu_index = i; + i = 0; resource_list_for_each_entry(entry, &pp->bridge->dma_ranges) { if (resource_type(entry->res) != IORESOURCE_MEM) @@ -838,11 +884,48 @@ int dw_pcie_setup_rc(struct dw_pcie_rp *pp) } EXPORT_SYMBOL_GPL(dw_pcie_setup_rc); +/* Using message outbound ATU to send out PME_Turn_Off message for dwc PCIe controller */ +static int dw_pcie_pme_turn_off(struct dw_pcie *pci) +{ + struct dw_pcie_ob_atu_cfg atu = { 0 }; + void __iomem *mem; + int ret; + + if (pci->num_ob_windows <= pci->pp.msg_atu_index) + return -EINVAL; + + if (!pci->pp.msg_res) + return -EINVAL; + + atu.code = PCIE_MSG_CODE_PME_TURN_OFF; + atu.routing = PCIE_MSG_TYPE_R_BC; + atu.type = PCIE_ATU_TYPE_MSG; + atu.size = resource_size(pci->pp.msg_res); + atu.index = pci->pp.msg_atu_index; + + atu.cpu_addr = pci->pp.msg_res->start; + + ret = dw_pcie_prog_outbound_atu(pci, &atu); + if (ret) + return ret; + + mem = ioremap(atu.cpu_addr, pci->region_align); + if (!mem) + return -ENOMEM; + + /* A dummy write is converted to a Msg TLP */ + writel(0, mem); + + iounmap(mem); + + return 0; +} + int dw_pcie_suspend_noirq(struct dw_pcie *pci) { u8 offset = dw_pcie_find_capability(pci, PCI_CAP_ID_EXP); u32 val; - int ret; + int ret = 0; /* * If L1SS is supported, then do not put the link into L2 as some @@ -854,10 +937,13 @@ int dw_pcie_suspend_noirq(struct dw_pcie *pci) if (dw_pcie_get_ltssm(pci) <= DW_PCIE_LTSSM_DETECT_ACT) return 0; - if (!pci->pp.ops->pme_turn_off) - return 0; + if (pci->pp.ops->pme_turn_off) + pci->pp.ops->pme_turn_off(&pci->pp); + else + ret = dw_pcie_pme_turn_off(pci); - pci->pp.ops->pme_turn_off(&pci->pp); + if (ret) + return ret; ret = read_poll_timeout(dw_pcie_get_ltssm, val, val == DW_PCIE_LTSSM_L2_IDLE, PCIE_PME_TO_L2_TIMEOUT_US/10, diff --git a/drivers/pci/controller/dwc/pcie-designware.h b/drivers/pci/controller/dwc/pcie-designware.h index 703b50bc5e0f1..dca5de4c6e877 100644 --- a/drivers/pci/controller/dwc/pcie-designware.h +++ b/drivers/pci/controller/dwc/pcie-designware.h @@ -341,6 +341,9 @@ struct dw_pcie_rp { struct pci_host_bridge *bridge; raw_spinlock_t lock; DECLARE_BITMAP(msi_irq_in_use, MAX_MSI_IRQS); + bool use_atu_msg; + int msg_atu_index; + struct resource *msg_res; }; struct dw_pcie_ep_ops {
Instead of relying on the vendor specific implementations to send the PME_Turn_Off message, let's introduce a generic way of sending the message using the MSG TLP. This is achieved by reserving a region for MSG TLP of size 'pci->region_align', at the end of the first IORESOURCE_MEM window of the host bridge. And then sending the PME_Turn_Off message during system suspend with the help of iATU. The reserve space at end is a little bit better than alloc_resource() because the below reasons. - alloc_resource() will allocate space at begin of IORESOURCE_MEM window. There will be a hole when first Endpoint Device (EP) try to alloc a bigger space then 'region_align' size. - Keep EP device's IORESOURCE_MEM space unchange with/without this patch. It should be noted that this generic implementation is optional for the glue drivers and can be overridden by a custom 'pme_turn_off' callback. Signed-off-by: Frank Li <Frank.Li@nxp.com> --- drivers/pci/controller/dwc/pcie-designware-host.c | 94 ++++++++++++++++++++++- drivers/pci/controller/dwc/pcie-designware.h | 3 + 2 files changed, 93 insertions(+), 4 deletions(-)