| Message ID | 20190614142012.31384-1-srinivas.kandagatla@linaro.org (mailing list archive) |
|---|---|
| State | Not Applicable |
| Headers | show |
| Series | dmaengine: qcom-bam: fix circular buffer handling | expand |
On Fri, Jun 14, 2019 at 03:20:12PM +0100, Srinivas Kandagatla wrote: > For some reason arguments to most of the circular buffers > macros are used in reverse, tail is used for head and vice versa. > > This leads to bam thinking that there is an extra descriptor at the > end and leading to retransmitting descriptor which was not scheduled > by any driver. This happens after MAX_DESCRIPTORS (4096) are scheduled > and done, so most of the drivers would not notice this, unless they are > heavily using bam dma. Originally found this issue while testing > SoundWire over SlimBus on DB845c which uses DMA very heavily for > read/writes. > > Signed-off-by: Srinivas Kandagatla <srinivas.kandagatla@linaro.org> Reviewed-by: Andy Gross <agross@kernel.org>
On 14-06-19, 15:20, Srinivas Kandagatla wrote: > For some reason arguments to most of the circular buffers > macros are used in reverse, tail is used for head and vice versa. > > This leads to bam thinking that there is an extra descriptor at the > end and leading to retransmitting descriptor which was not scheduled > by any driver. This happens after MAX_DESCRIPTORS (4096) are scheduled > and done, so most of the drivers would not notice this, unless they are > heavily using bam dma. Originally found this issue while testing > SoundWire over SlimBus on DB845c which uses DMA very heavily for > read/writes. Applied and copied stable, thanks
Hi Srini, On 6/14/2019 7:50 PM, Srinivas Kandagatla wrote: > For some reason arguments to most of the circular buffers > macros are used in reverse, tail is used for head and vice versa. > > This leads to bam thinking that there is an extra descriptor at the > end and leading to retransmitting descriptor which was not scheduled > by any driver. This happens after MAX_DESCRIPTORS (4096) are scheduled > and done, so most of the drivers would not notice this, unless they are > heavily using bam dma. Originally found this issue while testing > SoundWire over SlimBus on DB845c which uses DMA very heavily for > read/writes. > > Signed-off-by: Srinivas Kandagatla <srinivas.kandagatla@linaro.org> > --- > drivers/dma/qcom/bam_dma.c | 9 ++++----- > 1 file changed, 4 insertions(+), 5 deletions(-) > > diff --git a/drivers/dma/qcom/bam_dma.c b/drivers/dma/qcom/bam_dma.c > index cb860cb53c27..43d7b0a9713a 100644 > --- a/drivers/dma/qcom/bam_dma.c > +++ b/drivers/dma/qcom/bam_dma.c > @@ -350,8 +350,8 @@ static const struct reg_offset_data bam_v1_7_reg_info[] = { > #define BAM_DESC_FIFO_SIZE SZ_32K > #define MAX_DESCRIPTORS (BAM_DESC_FIFO_SIZE / sizeof(struct bam_desc_hw) - 1) > #define BAM_FIFO_SIZE (SZ_32K - 8) > -#define IS_BUSY(chan) (CIRC_SPACE(bchan->tail, bchan->head,\ > - MAX_DESCRIPTORS + 1) == 0) > +#define IS_BUSY(chan) (CIRC_SPACE(bchan->head, bchan->tail,\ > + MAX_DESCRIPTORS) == 0) > > struct bam_chan { > struct virt_dma_chan vc; > @@ -806,7 +806,7 @@ static u32 process_channel_irqs(struct bam_device *bdev) > offset /= sizeof(struct bam_desc_hw); > > /* Number of bytes available to read */ > - avail = CIRC_CNT(offset, bchan->head, MAX_DESCRIPTORS + 1); > + avail = CIRC_CNT(bchan->head, offset, MAX_DESCRIPTORS); > one question, so MAX_DESCRIPTORS is already a mask, #define MAX_DESCRIPTORS (BAM_DESC_FIFO_SIZE / sizeof(struct bam_desc_hw) - 1) CIRC_CNT/SPACE macros also does a size - 1, so would it not be a problem if we just pass MAX_DESCRIPTORS ? Regards, Sricharan > list_for_each_entry_safe(async_desc, tmp, > &bchan->desc_list, desc_node) { > @@ -997,8 +997,7 @@ static void bam_start_dma(struct bam_chan *bchan) > bam_apply_new_config(bchan, async_desc->dir); > > desc = async_desc->curr_desc; > - avail = CIRC_SPACE(bchan->tail, bchan->head, > - MAX_DESCRIPTORS + 1); > + avail = CIRC_SPACE(bchan->head, bchan->tail, MAX_DESCRIPTORS); > > if (async_desc->num_desc > avail) > async_desc->xfer_len = avail; >
Hi Sricharan, On 18/06/2019 08:13, Sricharan R wrote: > Hi Srini, > > On 6/14/2019 7:50 PM, Srinivas Kandagatla wrote: >> For some reason arguments to most of the circular buffers >> macros are used in reverse, tail is used for head and vice versa. >> >> This leads to bam thinking that there is an extra descriptor at the >> end and leading to retransmitting descriptor which was not scheduled >> by any driver. This happens after MAX_DESCRIPTORS (4096) are scheduled >> and done, so most of the drivers would not notice this, unless they are >> heavily using bam dma. Originally found this issue while testing >> SoundWire over SlimBus on DB845c which uses DMA very heavily for >> read/writes. >> >> Signed-off-by: Srinivas Kandagatla <srinivas.kandagatla@linaro.org> >> --- >> drivers/dma/qcom/bam_dma.c | 9 ++++----- >> 1 file changed, 4 insertions(+), 5 deletions(-) >> >> diff --git a/drivers/dma/qcom/bam_dma.c b/drivers/dma/qcom/bam_dma.c >> index cb860cb53c27..43d7b0a9713a 100644 >> --- a/drivers/dma/qcom/bam_dma.c >> +++ b/drivers/dma/qcom/bam_dma.c >> @@ -350,8 +350,8 @@ static const struct reg_offset_data bam_v1_7_reg_info[] = { >> #define BAM_DESC_FIFO_SIZE SZ_32K >> #define MAX_DESCRIPTORS (BAM_DESC_FIFO_SIZE / sizeof(struct bam_desc_hw) - 1) >> #define BAM_FIFO_SIZE (SZ_32K - 8) >> -#define IS_BUSY(chan) (CIRC_SPACE(bchan->tail, bchan->head,\ >> - MAX_DESCRIPTORS + 1) == 0) >> +#define IS_BUSY(chan) (CIRC_SPACE(bchan->head, bchan->tail,\ >> + MAX_DESCRIPTORS) == 0) >> >> struct bam_chan { >> struct virt_dma_chan vc; >> @@ -806,7 +806,7 @@ static u32 process_channel_irqs(struct bam_device *bdev) >> offset /= sizeof(struct bam_desc_hw); >> >> /* Number of bytes available to read */ >> - avail = CIRC_CNT(offset, bchan->head, MAX_DESCRIPTORS + 1); >> + avail = CIRC_CNT(bchan->head, offset, MAX_DESCRIPTORS); >> > one question, so MAX_DESCRIPTORS is already a mask, > #define MAX_DESCRIPTORS (BAM_DESC_FIFO_SIZE / sizeof(struct bam_desc_hw) - 1) > > CIRC_CNT/SPACE macros also does a size - 1, so would it not be a problem if we > just pass MAX_DESCRIPTORS ? Thanks for looking at this, TBH, usage of CIRC_* macros is only valid for power-of-2 buffers, In bam case MAX_DESCRIPTORS is 4095. Am really not sure why 8 bytes have been removed from fifo data buffer size. So basically usage of these macros is incorrect in bam case, this need to be fixed properly. Do you agree? Vinod, can you hold off with this patch, I will try to find some time this week to cook up a better patch removing the usage of these macros. thanks, srini > > Regards, > Sricharan > >> list_for_each_entry_safe(async_desc, tmp, >> &bchan->desc_list, desc_node) { >> @@ -997,8 +997,7 @@ static void bam_start_dma(struct bam_chan *bchan) >> bam_apply_new_config(bchan, async_desc->dir); >> >> desc = async_desc->curr_desc; >> - avail = CIRC_SPACE(bchan->tail, bchan->head, >> - MAX_DESCRIPTORS + 1); >> + avail = CIRC_SPACE(bchan->head, bchan->tail, MAX_DESCRIPTORS); >> >> if (async_desc->num_desc > avail) >> async_desc->xfer_len = avail; >> >
Hi Sricharan, On 18/06/2019 08:13, Sricharan R wrote: > Hi Srini, > > On 6/14/2019 7:50 PM, Srinivas Kandagatla wrote: >> For some reason arguments to most of the circular buffers >> macros are used in reverse, tail is used for head and vice versa. >> >> This leads to bam thinking that there is an extra descriptor at the >> end and leading to retransmitting descriptor which was not scheduled >> by any driver. This happens after MAX_DESCRIPTORS (4096) are scheduled >> and done, so most of the drivers would not notice this, unless they are >> heavily using bam dma. Originally found this issue while testing >> SoundWire over SlimBus on DB845c which uses DMA very heavily for >> read/writes. >> >> Signed-off-by: Srinivas Kandagatla <srinivas.kandagatla@linaro.org> >> --- >> drivers/dma/qcom/bam_dma.c | 9 ++++----- >> 1 file changed, 4 insertions(+), 5 deletions(-) >> >> diff --git a/drivers/dma/qcom/bam_dma.c b/drivers/dma/qcom/bam_dma.c >> index cb860cb53c27..43d7b0a9713a 100644 >> --- a/drivers/dma/qcom/bam_dma.c >> +++ b/drivers/dma/qcom/bam_dma.c >> @@ -350,8 +350,8 @@ static const struct reg_offset_data bam_v1_7_reg_info[] = { >> #define BAM_DESC_FIFO_SIZE SZ_32K >> #define MAX_DESCRIPTORS (BAM_DESC_FIFO_SIZE / sizeof(struct bam_desc_hw) - 1) >> #define BAM_FIFO_SIZE (SZ_32K - 8) >> -#define IS_BUSY(chan) (CIRC_SPACE(bchan->tail, bchan->head,\ >> - MAX_DESCRIPTORS + 1) == 0) >> +#define IS_BUSY(chan) (CIRC_SPACE(bchan->head, bchan->tail,\ >> + MAX_DESCRIPTORS) == 0) >> >> struct bam_chan { >> struct virt_dma_chan vc; >> @@ -806,7 +806,7 @@ static u32 process_channel_irqs(struct bam_device *bdev) >> offset /= sizeof(struct bam_desc_hw); >> >> /* Number of bytes available to read */ >> - avail = CIRC_CNT(offset, bchan->head, MAX_DESCRIPTORS + 1); >> + avail = CIRC_CNT(bchan->head, offset, MAX_DESCRIPTORS); >> > one question, so MAX_DESCRIPTORS is already a mask, > #define MAX_DESCRIPTORS (BAM_DESC_FIFO_SIZE / sizeof(struct bam_desc_hw) - 1) > > CIRC_CNT/SPACE macros also does a size - 1, so would it not be a problem if we > just pass MAX_DESCRIPTORS ? Thanks for looking at this, TBH, usage of CIRC_* macros is only valid for power-of-2 buffers, In bam case MAX_DESCRIPTORS is 4095. Am really not sure why 8 bytes have been removed from fifo data buffer size. So basically usage of these macros is incorrect in bam case, this need to be fixed properly. Do you agree? Vinod, can you hold off with this patch, I will try to find some time this week to cook up a better patch removing the usage of these macros. thanks, srini > > Regards, > Sricharan > >> list_for_each_entry_safe(async_desc, tmp, >> &bchan->desc_list, desc_node) { >> @@ -997,8 +997,7 @@ static void bam_start_dma(struct bam_chan *bchan) >> bam_apply_new_config(bchan, async_desc->dir); >> >> desc = async_desc->curr_desc; >> - avail = CIRC_SPACE(bchan->tail, bchan->head, >> - MAX_DESCRIPTORS + 1); >> + avail = CIRC_SPACE(bchan->head, bchan->tail, MAX_DESCRIPTORS); >> >> if (async_desc->num_desc > avail) >> async_desc->xfer_len = avail; >> >
Hi Srini, On 6/18/2019 8:20 PM, Srinivas Kandagatla wrote: > Hi Sricharan, > > On 18/06/2019 08:13, Sricharan R wrote: >> Hi Srini, >> >> On 6/14/2019 7:50 PM, Srinivas Kandagatla wrote: >>> For some reason arguments to most of the circular buffers >>> macros are used in reverse, tail is used for head and vice versa. >>> >>> This leads to bam thinking that there is an extra descriptor at the >>> end and leading to retransmitting descriptor which was not scheduled >>> by any driver. This happens after MAX_DESCRIPTORS (4096) are scheduled >>> and done, so most of the drivers would not notice this, unless they are >>> heavily using bam dma. Originally found this issue while testing >>> SoundWire over SlimBus on DB845c which uses DMA very heavily for >>> read/writes. >>> >>> Signed-off-by: Srinivas Kandagatla <srinivas.kandagatla@linaro.org> >>> --- >>> drivers/dma/qcom/bam_dma.c | 9 ++++----- >>> 1 file changed, 4 insertions(+), 5 deletions(-) >>> >>> diff --git a/drivers/dma/qcom/bam_dma.c b/drivers/dma/qcom/bam_dma.c >>> index cb860cb53c27..43d7b0a9713a 100644 >>> --- a/drivers/dma/qcom/bam_dma.c >>> +++ b/drivers/dma/qcom/bam_dma.c >>> @@ -350,8 +350,8 @@ static const struct reg_offset_data bam_v1_7_reg_info[] = { >>> #define BAM_DESC_FIFO_SIZE SZ_32K >>> #define MAX_DESCRIPTORS (BAM_DESC_FIFO_SIZE / sizeof(struct bam_desc_hw) - 1) >>> #define BAM_FIFO_SIZE (SZ_32K - 8) >>> -#define IS_BUSY(chan) (CIRC_SPACE(bchan->tail, bchan->head,\ >>> - MAX_DESCRIPTORS + 1) == 0) >>> +#define IS_BUSY(chan) (CIRC_SPACE(bchan->head, bchan->tail,\ >>> + MAX_DESCRIPTORS) == 0) >>> struct bam_chan { >>> struct virt_dma_chan vc; >>> @@ -806,7 +806,7 @@ static u32 process_channel_irqs(struct bam_device *bdev) >>> offset /= sizeof(struct bam_desc_hw); >>> /* Number of bytes available to read */ >>> - avail = CIRC_CNT(offset, bchan->head, MAX_DESCRIPTORS + 1); >>> + avail = CIRC_CNT(bchan->head, offset, MAX_DESCRIPTORS); >>> >> one question, so MAX_DESCRIPTORS is already a mask, >> #define MAX_DESCRIPTORS (BAM_DESC_FIFO_SIZE / sizeof(struct bam_desc_hw) - 1) >> >> CIRC_CNT/SPACE macros also does a size - 1, so would it not be a problem if we >> just pass MAX_DESCRIPTORS ? > > Thanks for looking at this, > TBH, usage of CIRC_* macros is only valid for power-of-2 buffers, > In bam case MAX_DESCRIPTORS is 4095. > Am really not sure why 8 bytes have been removed from fifo data buffer size. > So basically usage of these macros is incorrect in bam case, this need to be fixed properly. > > Do you agree? > So MAX_DESCRIPTORS is used in driver for masking head/tail pointers. That's why we have to pass MAX_DESCRIPTORS + 1 so that it works when the Macros does a size - 1 Regards, Sricharan > Vinod, can you hold off with this patch, I will try to find some time this week to cook up a better patch removing the usage of these macros. > > > > thanks, > srini > >> >> Regards, >> Sricharan >> >>> list_for_each_entry_safe(async_desc, tmp, >>> &bchan->desc_list, desc_node) { >>> @@ -997,8 +997,7 @@ static void bam_start_dma(struct bam_chan *bchan) >>> bam_apply_new_config(bchan, async_desc->dir); >>> desc = async_desc->curr_desc; >>> - avail = CIRC_SPACE(bchan->tail, bchan->head, >>> - MAX_DESCRIPTORS + 1); >>> + avail = CIRC_SPACE(bchan->head, bchan->tail, MAX_DESCRIPTORS); >>> if (async_desc->num_desc > avail) >>> async_desc->xfer_len = avail; >>> >>
On 18/06/2019 15:56, Sricharan R wrote: > So MAX_DESCRIPTORS is used in driver for masking head/tail pointers. > That's why we have to pass MAX_DESCRIPTORS + 1 so that it works > when the Macros does a size - 1 Isn't that incorrect to do that, pretending to have more descriptors than we actually have? --srini
On 6/18/2019 8:42 PM, Srinivas Kandagatla wrote: > > > On 18/06/2019 15:56, Sricharan R wrote: >> So MAX_DESCRIPTORS is used in driver for masking head/tail pointers. >> That's why we have to pass MAX_DESCRIPTORS + 1 so that it works >> when the Macros does a size - 1 > Isn't that incorrect to do that, pretending to have more descriptors than we actually have? > The Macro's expect that buffer size is power of 2. So we are infact passing the actual correct size ( MAX_DESCRIPTORS + 1 = 4096) Regards, Sricharan
On 18/06/2019 17:27, Sricharan R wrote: > The Macro's expect that buffer size is power of 2. So we are infact passing the actual correct > size ( MAX_DESCRIPTORS + 1 = 4096) This will make the circular buffer macros happy but question is that do we actually have that many descriptor buffers? This is what is in the driver: #define BAM_DESC_FIFO_SIZE SZ_32K #define MAX_DESCRIPTORS (BAM_DESC_FIFO_SIZE / sizeof(struct bam_desc_hw) - 1) #define BAM_FIFO_SIZE (SZ_32K - 8) Wouldn't having MAX_DESCRIPTORS + 1 = 4096 lead to overflow the actual descriptor memory size of (SZ_32K - 8) ? --srini
Hi Srini, On 6/18/2019 10:20 PM, Srinivas Kandagatla wrote: > > > On 18/06/2019 17:27, Sricharan R wrote: >> The Macro's expect that buffer size is power of 2. So we are infact passing the actual correct >> size ( MAX_DESCRIPTORS + 1 = 4096) > This will make the circular buffer macros happy but question is that do we actually have that many descriptor buffers? > > This is what is in the driver: > > #define BAM_DESC_FIFO_SIZE SZ_32K > #define MAX_DESCRIPTORS (BAM_DESC_FIFO_SIZE / sizeof(struct bam_desc_hw) - 1) > #define BAM_FIFO_SIZE (SZ_32K - 8) > > Wouldn't having MAX_DESCRIPTORS + 1 = 4096 lead to overflow the actual descriptor memory size of (SZ_32K - 8) ? > Right, but the CIRC_SPACE macro assumes there is 1 space less than the actual size. That said, agree there is an issue on the boundary. I will also do some testing tomorrow on this and get back. Regards, Sricharan
diff --git a/drivers/dma/qcom/bam_dma.c b/drivers/dma/qcom/bam_dma.c index cb860cb53c27..43d7b0a9713a 100644 --- a/drivers/dma/qcom/bam_dma.c +++ b/drivers/dma/qcom/bam_dma.c @@ -350,8 +350,8 @@ static const struct reg_offset_data bam_v1_7_reg_info[] = { #define BAM_DESC_FIFO_SIZE SZ_32K #define MAX_DESCRIPTORS (BAM_DESC_FIFO_SIZE / sizeof(struct bam_desc_hw) - 1) #define BAM_FIFO_SIZE (SZ_32K - 8) -#define IS_BUSY(chan) (CIRC_SPACE(bchan->tail, bchan->head,\ - MAX_DESCRIPTORS + 1) == 0) +#define IS_BUSY(chan) (CIRC_SPACE(bchan->head, bchan->tail,\ + MAX_DESCRIPTORS) == 0) struct bam_chan { struct virt_dma_chan vc; @@ -806,7 +806,7 @@ static u32 process_channel_irqs(struct bam_device *bdev) offset /= sizeof(struct bam_desc_hw); /* Number of bytes available to read */ - avail = CIRC_CNT(offset, bchan->head, MAX_DESCRIPTORS + 1); + avail = CIRC_CNT(bchan->head, offset, MAX_DESCRIPTORS); list_for_each_entry_safe(async_desc, tmp, &bchan->desc_list, desc_node) { @@ -997,8 +997,7 @@ static void bam_start_dma(struct bam_chan *bchan) bam_apply_new_config(bchan, async_desc->dir); desc = async_desc->curr_desc; - avail = CIRC_SPACE(bchan->tail, bchan->head, - MAX_DESCRIPTORS + 1); + avail = CIRC_SPACE(bchan->head, bchan->tail, MAX_DESCRIPTORS); if (async_desc->num_desc > avail) async_desc->xfer_len = avail;
For some reason arguments to most of the circular buffers macros are used in reverse, tail is used for head and vice versa. This leads to bam thinking that there is an extra descriptor at the end and leading to retransmitting descriptor which was not scheduled by any driver. This happens after MAX_DESCRIPTORS (4096) are scheduled and done, so most of the drivers would not notice this, unless they are heavily using bam dma. Originally found this issue while testing SoundWire over SlimBus on DB845c which uses DMA very heavily for read/writes. Signed-off-by: Srinivas Kandagatla <srinivas.kandagatla@linaro.org> --- drivers/dma/qcom/bam_dma.c | 9 ++++----- 1 file changed, 4 insertions(+), 5 deletions(-)