diff mbox series

[V2] PCI: set BAR size bits correctly in Resize BAR control register

Message ID 20190725192552.24295-1-sumit.saxena@broadcom.com (mailing list archive)
State New, archived
Delegated to: Bjorn Helgaas
Headers show
Series [V2] PCI: set BAR size bits correctly in Resize BAR control register | expand

Commit Message

Sumit Saxena July 25, 2019, 7:25 p.m. UTC
In Resize BAR control register, bits[8:12] represents size of BAR.
As per PCIe specification, below is encoded values in register bits
to actual BAR size table:

Bits  BAR size
0     1 MB
1     2 MB
2     4 MB
3     8 MB
--

For 1 MB BAR size, BAR size bits should be set to 0 but incorrectly
these bits are set to "1f". Latest megaraid_sas and mpt3sas adapters
which support Resizable BAR with 1 MB BAR size fails to initialize
during system resume from S3 sleep.

Fix: Correctly calculate BAR size bits for Resize BAR control register.

V2:
-Simplified calculation of BAR size bits as suggested by Christian Koenig.

CC: stable@vger.kernel.org # v4.16+
Bugzilla: https://bugzilla.kernel.org/show_bug.cgi?id=203939
Fixes: d3252ace0bc652a1a244455556b6a549f969bf99 ("PCI: Restore resized BAR state on resume")
Signed-off-by: Sumit Saxena <sumit.saxena@broadcom.com>
---
 drivers/pci/pci.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

Comments

Christian König July 25, 2019, 11:58 a.m. UTC | #1
Am 25.07.19 um 21:25 schrieb Sumit Saxena:
> In Resize BAR control register, bits[8:12] represents size of BAR.
> As per PCIe specification, below is encoded values in register bits
> to actual BAR size table:
>
> Bits  BAR size
> 0     1 MB
> 1     2 MB
> 2     4 MB
> 3     8 MB
> --
>
> For 1 MB BAR size, BAR size bits should be set to 0 but incorrectly
> these bits are set to "1f". Latest megaraid_sas and mpt3sas adapters
> which support Resizable BAR with 1 MB BAR size fails to initialize
> during system resume from S3 sleep.
>
> Fix: Correctly calculate BAR size bits for Resize BAR control register.
>
> V2:
> -Simplified calculation of BAR size bits as suggested by Christian Koenig.
>
> CC: stable@vger.kernel.org # v4.16+
> Bugzilla: https://bugzilla.kernel.org/show_bug.cgi?id=203939
> Fixes: d3252ace0bc652a1a244455556b6a549f969bf99 ("PCI: Restore resized BAR state on resume")
> Signed-off-by: Sumit Saxena <sumit.saxena@broadcom.com>

Reviewed-by: Christian König <christian.koenig@amd.com>

> ---
>   drivers/pci/pci.c | 2 +-
>   1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c
> index 29ed5ec1ac27..e59921296125 100644
> --- a/drivers/pci/pci.c
> +++ b/drivers/pci/pci.c
> @@ -1438,7 +1438,7 @@ static void pci_restore_rebar_state(struct pci_dev *pdev)
>   		pci_read_config_dword(pdev, pos + PCI_REBAR_CTRL, &ctrl);
>   		bar_idx = ctrl & PCI_REBAR_CTRL_BAR_IDX;
>   		res = pdev->resource + bar_idx;
> -		size = order_base_2((resource_size(res) >> 20) | 1) - 1;
> +		size = order_base_2(resource_size(res) >> 20);
>   		ctrl &= ~PCI_REBAR_CTRL_BAR_SIZE;
>   		ctrl |= size << PCI_REBAR_CTRL_BAR_SHIFT;
>   		pci_write_config_dword(pdev, pos + PCI_REBAR_CTRL, ctrl);
Bjorn Helgaas Aug. 7, 2019, 11:01 p.m. UTC | #2
On Fri, Jul 26, 2019 at 12:55:52AM +0530, Sumit Saxena wrote:
> In Resize BAR control register, bits[8:12] represents size of BAR.
> As per PCIe specification, below is encoded values in register bits
> to actual BAR size table:
> 
> Bits  BAR size
> 0     1 MB
> 1     2 MB
> 2     4 MB
> 3     8 MB
> --
> 
> For 1 MB BAR size, BAR size bits should be set to 0 but incorrectly
> these bits are set to "1f". Latest megaraid_sas and mpt3sas adapters
> which support Resizable BAR with 1 MB BAR size fails to initialize
> during system resume from S3 sleep.
> 
> Fix: Correctly calculate BAR size bits for Resize BAR control register.
> 
> V2:
> -Simplified calculation of BAR size bits as suggested by Christian Koenig.
> 
> CC: stable@vger.kernel.org # v4.16+
> Bugzilla: https://bugzilla.kernel.org/show_bug.cgi?id=203939
> Fixes: d3252ace0bc652a1a244455556b6a549f969bf99 ("PCI: Restore resized BAR state on resume")
> Signed-off-by: Sumit Saxena <sumit.saxena@broadcom.com>
> ---
>  drivers/pci/pci.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c
> index 29ed5ec1ac27..e59921296125 100644
> --- a/drivers/pci/pci.c
> +++ b/drivers/pci/pci.c
> @@ -1438,7 +1438,7 @@ static void pci_restore_rebar_state(struct pci_dev *pdev)
>  		pci_read_config_dword(pdev, pos + PCI_REBAR_CTRL, &ctrl);
>  		bar_idx = ctrl & PCI_REBAR_CTRL_BAR_IDX;
>  		res = pdev->resource + bar_idx;
> -		size = order_base_2((resource_size(res) >> 20) | 1) - 1;
> +		size = order_base_2(resource_size(res) >> 20);

Since BAR sizes are always powers of 2, wouldn't this be simpler as:

		size = ilog2(resource_size(res)) - 20;

which nicely matches the table in PCIe r5.0, sec 7.8.6.3?

>  		ctrl &= ~PCI_REBAR_CTRL_BAR_SIZE;
>  		ctrl |= size << PCI_REBAR_CTRL_BAR_SHIFT;
>  		pci_write_config_dword(pdev, pos + PCI_REBAR_CTRL, ctrl);
> -- 
> 2.18.1
>
Bjorn Helgaas Aug. 7, 2019, 11:17 p.m. UTC | #3
On Fri, Jul 26, 2019 at 12:55:52AM +0530, Sumit Saxena wrote:
> In Resize BAR control register, bits[8:12] represents size of BAR.
> As per PCIe specification, below is encoded values in register bits
> to actual BAR size table:
> 
> Bits  BAR size
> 0     1 MB
> 1     2 MB
> 2     4 MB
> 3     8 MB
> --
> 
> For 1 MB BAR size, BAR size bits should be set to 0 but incorrectly
> these bits are set to "1f". Latest megaraid_sas and mpt3sas adapters
> which support Resizable BAR with 1 MB BAR size fails to initialize
> during system resume from S3 sleep.
> 
> Fix: Correctly calculate BAR size bits for Resize BAR control register.
> 
> V2:
> -Simplified calculation of BAR size bits as suggested by Christian Koenig.
> 
> CC: stable@vger.kernel.org # v4.16+

Also, d3252ace0bc6 ("PCI: Restore resized BAR state on resume") didn't
appear until v4.19.  I updated this to "v4.19+"; let me know if that's
wrong.

> Bugzilla: https://bugzilla.kernel.org/show_bug.cgi?id=203939
> Fixes: d3252ace0bc652a1a244455556b6a549f969bf99 ("PCI: Restore resized BAR state on resume")

I updated this to conventional format as above (12-char SHA1).

> Signed-off-by: Sumit Saxena <sumit.saxena@broadcom.com>
> ---
>  drivers/pci/pci.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c
> index 29ed5ec1ac27..e59921296125 100644
> --- a/drivers/pci/pci.c
> +++ b/drivers/pci/pci.c
> @@ -1438,7 +1438,7 @@ static void pci_restore_rebar_state(struct pci_dev *pdev)
>  		pci_read_config_dword(pdev, pos + PCI_REBAR_CTRL, &ctrl);
>  		bar_idx = ctrl & PCI_REBAR_CTRL_BAR_IDX;
>  		res = pdev->resource + bar_idx;
> -		size = order_base_2((resource_size(res) >> 20) | 1) - 1;
> +		size = order_base_2(resource_size(res) >> 20);
>  		ctrl &= ~PCI_REBAR_CTRL_BAR_SIZE;
>  		ctrl |= size << PCI_REBAR_CTRL_BAR_SHIFT;
>  		pci_write_config_dword(pdev, pos + PCI_REBAR_CTRL, ctrl);
> -- 
> 2.18.1
>
Bjorn Helgaas Aug. 7, 2019, 11:27 p.m. UTC | #4
On Fri, Jul 26, 2019 at 12:55:52AM +0530, Sumit Saxena wrote:
> In Resize BAR control register, bits[8:12] represents size of BAR.
> As per PCIe specification, below is encoded values in register bits
> to actual BAR size table:
> 
> Bits  BAR size
> 0     1 MB
> 1     2 MB
> 2     4 MB
> 3     8 MB
> --
> 
> For 1 MB BAR size, BAR size bits should be set to 0 but incorrectly
> these bits are set to "1f". Latest megaraid_sas and mpt3sas adapters
> which support Resizable BAR with 1 MB BAR size fails to initialize
> during system resume from S3 sleep.
> 
> Fix: Correctly calculate BAR size bits for Resize BAR control register.
> 
> V2:
> -Simplified calculation of BAR size bits as suggested by Christian Koenig.
> 
> CC: stable@vger.kernel.org # v4.16+
> Bugzilla: https://bugzilla.kernel.org/show_bug.cgi?id=203939
> Fixes: d3252ace0bc652a1a244455556b6a549f969bf99 ("PCI: Restore resized BAR state on resume")
> Signed-off-by: Sumit Saxena <sumit.saxena@broadcom.com>

I applied this to pci/misc for v5.4 with the tweaks I mentioned (see
below).

Christian, I didn't add your Reviewed-by since I changed the patch,
but I'll be glad to update the branch if you take another look.

> ---
>  drivers/pci/pci.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c
> index 29ed5ec1ac27..e59921296125 100644
> --- a/drivers/pci/pci.c
> +++ b/drivers/pci/pci.c
> @@ -1438,7 +1438,7 @@ static void pci_restore_rebar_state(struct pci_dev *pdev)
>  		pci_read_config_dword(pdev, pos + PCI_REBAR_CTRL, &ctrl);
>  		bar_idx = ctrl & PCI_REBAR_CTRL_BAR_IDX;
>  		res = pdev->resource + bar_idx;
> -		size = order_base_2((resource_size(res) >> 20) | 1) - 1;
> +		size = order_base_2(resource_size(res) >> 20);
>  		ctrl &= ~PCI_REBAR_CTRL_BAR_SIZE;
>  		ctrl |= size << PCI_REBAR_CTRL_BAR_SHIFT;
>  		pci_write_config_dword(pdev, pos + PCI_REBAR_CTRL, ctrl);

commit 614b04644b57
Author: Sumit Saxena <sumit.saxena@broadcom.com>
Date:   Fri Jul 26 00:55:52 2019 +0530

    PCI: Set BAR size bits correctly in Resize BAR control register
    
    In a Resizable BAR Control Register, bits 13:8 control the size of the BAR.
    The encoded values of these bits are as follows (see PCIe r5.0, sec
    7.8.6.3):
    
      Value    BAR size
         0     1 MB (2^20 bytes)
         1     2 MB (2^21 bytes)
         2     4 MB (2^22 bytes)
       ...
        43     8 EB (2^63 bytes)
    
    Previously we incorrectly set the BAR size bits for a 1 MB BAR to 0x1f
    instead of 0, so devices that support that size, e.g., new megaraid_sas and
    mpt3sas adapters, fail to initialize during resume from S3 sleep.
    
    Correctly calculate the BAR size bits for Resizable BAR control registers.
    
    Link: https://lore.kernel.org/r/20190725192552.24295-1-sumit.saxena@broadcom.com
    Bugzilla: https://bugzilla.kernel.org/show_bug.cgi?id=203939
    Fixes: d3252ace0bc6 ("PCI: Restore resized BAR state on resume")
    Signed-off-by: Sumit Saxena <sumit.saxena@broadcom.com>
    Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
    Cc: stable@vger.kernel.org      # v4.19+

diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c
index da3241bb4479..5836eb576d96 100644
--- a/drivers/pci/pci.c
+++ b/drivers/pci/pci.c
@@ -1438,7 +1438,7 @@ static void pci_restore_rebar_state(struct pci_dev *pdev)
 		pci_read_config_dword(pdev, pos + PCI_REBAR_CTRL, &ctrl);
 		bar_idx = ctrl & PCI_REBAR_CTRL_BAR_IDX;
 		res = pdev->resource + bar_idx;
-		size = order_base_2((resource_size(res) >> 20) | 1) - 1;
+		size = ilog2(resource_size(res)) - 20;
 		ctrl &= ~PCI_REBAR_CTRL_BAR_SIZE;
 		ctrl |= size << PCI_REBAR_CTRL_BAR_SHIFT;
 		pci_write_config_dword(pdev, pos + PCI_REBAR_CTRL, ctrl);
Christian König Aug. 8, 2019, 7:01 a.m. UTC | #5
Am 08.08.19 um 01:01 schrieb Bjorn Helgaas:
> On Fri, Jul 26, 2019 at 12:55:52AM +0530, Sumit Saxena wrote:
>> In Resize BAR control register, bits[8:12] represents size of BAR.
>> As per PCIe specification, below is encoded values in register bits
>> to actual BAR size table:
>>
>> Bits  BAR size
>> 0     1 MB
>> 1     2 MB
>> 2     4 MB
>> 3     8 MB
>> --
>>
>> For 1 MB BAR size, BAR size bits should be set to 0 but incorrectly
>> these bits are set to "1f". Latest megaraid_sas and mpt3sas adapters
>> which support Resizable BAR with 1 MB BAR size fails to initialize
>> during system resume from S3 sleep.
>>
>> Fix: Correctly calculate BAR size bits for Resize BAR control register.
>>
>> V2:
>> -Simplified calculation of BAR size bits as suggested by Christian Koenig.
>>
>> CC: stable@vger.kernel.org # v4.16+
>> Bugzilla: https://bugzilla.kernel.org/show_bug.cgi?id=203939
>> Fixes: d3252ace0bc652a1a244455556b6a549f969bf99 ("PCI: Restore resized BAR state on resume")
>> Signed-off-by: Sumit Saxena <sumit.saxena@broadcom.com>
>> ---
>>   drivers/pci/pci.c | 2 +-
>>   1 file changed, 1 insertion(+), 1 deletion(-)
>>
>> diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c
>> index 29ed5ec1ac27..e59921296125 100644
>> --- a/drivers/pci/pci.c
>> +++ b/drivers/pci/pci.c
>> @@ -1438,7 +1438,7 @@ static void pci_restore_rebar_state(struct pci_dev *pdev)
>>   		pci_read_config_dword(pdev, pos + PCI_REBAR_CTRL, &ctrl);
>>   		bar_idx = ctrl & PCI_REBAR_CTRL_BAR_IDX;
>>   		res = pdev->resource + bar_idx;
>> -		size = order_base_2((resource_size(res) >> 20) | 1) - 1;
>> +		size = order_base_2(resource_size(res) >> 20);
> Since BAR sizes are always powers of 2, wouldn't this be simpler as:
>
> 		size = ilog2(resource_size(res)) - 20;
>
> which nicely matches the table in PCIe r5.0, sec 7.8.6.3?

Yeah, that should obviously work as well.

We would have a serious problem in the resource management if the 
resource size is smaller than 1MB or not a power of two.

Feel free to add my r-b.

Regards,
Christian.

>
>>   		ctrl &= ~PCI_REBAR_CTRL_BAR_SIZE;
>>   		ctrl |= size << PCI_REBAR_CTRL_BAR_SHIFT;
>>   		pci_write_config_dword(pdev, pos + PCI_REBAR_CTRL, ctrl);
>> -- 
>> 2.18.1
>>
Bjorn Helgaas Aug. 8, 2019, 12:31 p.m. UTC | #6
On Thu, Aug 08, 2019 at 07:01:03AM +0000, Koenig, Christian wrote:
> Am 08.08.19 um 01:01 schrieb Bjorn Helgaas:
> > On Fri, Jul 26, 2019 at 12:55:52AM +0530, Sumit Saxena wrote:
> >> In Resize BAR control register, bits[8:12] represents size of BAR.
> >> As per PCIe specification, below is encoded values in register bits
> >> to actual BAR size table:
> >>
> >> Bits  BAR size
> >> 0     1 MB
> >> 1     2 MB
> >> 2     4 MB
> >> 3     8 MB
> >> --
> >>
> >> For 1 MB BAR size, BAR size bits should be set to 0 but incorrectly
> >> these bits are set to "1f". Latest megaraid_sas and mpt3sas adapters
> >> which support Resizable BAR with 1 MB BAR size fails to initialize
> >> during system resume from S3 sleep.
> >>
> >> Fix: Correctly calculate BAR size bits for Resize BAR control register.
> >>
> >> V2:
> >> -Simplified calculation of BAR size bits as suggested by Christian Koenig.
> >>
> >> CC: stable@vger.kernel.org # v4.16+
> >> Bugzilla: https://bugzilla.kernel.org/show_bug.cgi?id=203939
> >> Fixes: d3252ace0bc652a1a244455556b6a549f969bf99 ("PCI: Restore resized BAR state on resume")
> >> Signed-off-by: Sumit Saxena <sumit.saxena@broadcom.com>
> >> ---
> >>   drivers/pci/pci.c | 2 +-
> >>   1 file changed, 1 insertion(+), 1 deletion(-)
> >>
> >> diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c
> >> index 29ed5ec1ac27..e59921296125 100644
> >> --- a/drivers/pci/pci.c
> >> +++ b/drivers/pci/pci.c
> >> @@ -1438,7 +1438,7 @@ static void pci_restore_rebar_state(struct pci_dev *pdev)
> >>   		pci_read_config_dword(pdev, pos + PCI_REBAR_CTRL, &ctrl);
> >>   		bar_idx = ctrl & PCI_REBAR_CTRL_BAR_IDX;
> >>   		res = pdev->resource + bar_idx;
> >> -		size = order_base_2((resource_size(res) >> 20) | 1) - 1;
> >> +		size = order_base_2(resource_size(res) >> 20);
> > Since BAR sizes are always powers of 2, wouldn't this be simpler as:
> >
> > 		size = ilog2(resource_size(res)) - 20;
> >
> > which nicely matches the table in PCIe r5.0, sec 7.8.6.3?
> 
> Yeah, that should obviously work as well.
> 
> We would have a serious problem in the resource management if the 
> resource size is smaller than 1MB or not a power of two.

Yes, definitely.  Resizable BARs are required by spec to be 1MB or
larger, but this does niggle at me a little bit, too.  It probably
saves a few bits in pci_dev to recompute this at restore-time, but
honestly, I think it would be more obviously correct to just do the
simple-minded thing of saving and restoring the entire register.

> Feel free to add my r-b.

Done, thanks!

> Regards,
> Christian.
> 
> >
> >>   		ctrl &= ~PCI_REBAR_CTRL_BAR_SIZE;
> >>   		ctrl |= size << PCI_REBAR_CTRL_BAR_SHIFT;
> >>   		pci_write_config_dword(pdev, pos + PCI_REBAR_CTRL, ctrl);
> >> -- 
> >> 2.18.1
> >>
>
diff mbox series

Patch

diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c
index 29ed5ec1ac27..e59921296125 100644
--- a/drivers/pci/pci.c
+++ b/drivers/pci/pci.c
@@ -1438,7 +1438,7 @@  static void pci_restore_rebar_state(struct pci_dev *pdev)
 		pci_read_config_dword(pdev, pos + PCI_REBAR_CTRL, &ctrl);
 		bar_idx = ctrl & PCI_REBAR_CTRL_BAR_IDX;
 		res = pdev->resource + bar_idx;
-		size = order_base_2((resource_size(res) >> 20) | 1) - 1;
+		size = order_base_2(resource_size(res) >> 20);
 		ctrl &= ~PCI_REBAR_CTRL_BAR_SIZE;
 		ctrl |= size << PCI_REBAR_CTRL_BAR_SHIFT;
 		pci_write_config_dword(pdev, pos + PCI_REBAR_CTRL, ctrl);