| Message ID | 20240610235808.22173-33-quic_wcheng@quicinc.com (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
| Series | Introduce QC USB SND audio offloading support | expand |
On Mon, Jun 10, 2024 at 04:58:08PM -0700, Wesley Cheng wrote: > +Overview > +======== > +In order to leverage the existing USB sound device support in ALSA, the > +introduction of the ASoC USB APIs, allow for the entities to communicate > +with one another. "... ASoC USB APIs are introduced to allow for ..." > +USB Audio Device Connection Flow > +-------------------------------- > +USB devices can be hotplugged into the USB root hub at any point in time. > +The BE DAI link should be aware of the current state of the physical USB > +port, i.e. if there are any USB devices with audio interface(s) connected. > +The following callback can be used to notify the BE DAI link of any change: > + > + **connection_status_cb()** "... connection_status_cb() can be used to ..." Thanks.
On 6/11/2024 1:58 AM, Wesley Cheng wrote: (...) > +In the case where the USB offload driver is unbounded, while USB SND is unbounded -> unbound (...) > +SOC USB and USB Sound Kcontrols > +=============================== > +Details > +------- > +SOC USB and USB sound expose a set of SND kcontrols for applications to select > +and fetch the current offloading status for the ASoC platform sound card. Kcontrols > +are split between two layers: > + > + - USB sound - Notifies the sound card number for the ASoC platform sound > + card that it is registered to for supporting audio offload. > + > + - SOC USB - Maintains the current status of the offload path, and device > + (USB sound card and PCM device) information. This would be the main > + card that applications can read to determine offloading capabilities. > + > +Implementation > +-------------- > + > +**Example:** > + > + **Sound Cards**: > + > + :: > + > + 0 [SM8250MTPWCD938]: sm8250 - SM8250-MTP-WCD9380-WSA8810-VA-D > + SM8250-MTP-WCD9380-WSA8810-VA-DMIC > + 1 [C320M ]: USB-Audio - Plantronics C320-M > + Plantronics Plantronics C320-M at usb-xhci-hcd.1.auto-1, full speed > + > + > + **Platform Sound Card** - card#0: > + > + :: > + > + USB Offload Playback Route Card Select 1 (range -1->32) > + USB Offload Playback Route PCM Select 0 (range -1->255) > + USB Offload Playback Route Card Status -1 (range -1->32) > + USB Offload Playback Route PCM Status -1 (range -1->255) > + > + > + **USB Sound Card** - card#1: > + > + :: > + > + USB Offload Playback Capable Card 0 (range -1->32) > + > + > +The platform sound card(card#0) kcontrols are created as part of adding the SOC > +USB device using **snd_soc_usb_add_port()**. The following kcontrols are defined > +as: > + > + - ``USB Offload Playback Route Card Status`` **(R)**: USB sound card device index > + that defines which USB SND resources are currently offloaded. If -1 is seen, it > + signifies that offload is not active. > + - ``USB Offload Playback Route PCM Status`` **(R)**: USB PCM device index > + that defines which USB SND resources are currently offloaded. If -1 is seen, it > + signifies that offload is not active. > + - ``USB Offload Playback Route Card Select`` **(R/W)**: USB sound card index which > + selects the USB device to initiate offloading on. If no value is written to the > + kcontrol, then the last USB device discovered card index will be chosen. I see only one kcontrol, what if hardware is capable of offloading on more cards, is it possible to do offloading on more than one device? > + - ``USB Offload Playback Route PCM Select`` **(R/W)**: USB PCM index which selects > + the USB device to initiate offloading on. If no value is written to the > + kcontrol, then the last USB device discovered PCM zero index will be chosen. > + > +The USB sound card(card#1) kcontrols are created as USB audio devices are plugged > +into the physical USB port and enumerated. The kcontrols are defined as: > + > + - ``USB Offload Playback Capable Card`` **(R)**: Provides the sound card > + number/index that supports USB offloading. Further/follow up queries about > + the current offload state can be handled by reading the offload status > + kcontrol exposed by the platform card. > + Why do we need to some magic between cards? I feel like whole kcontrol thing is overengineered a bit - I'm not sure I understand the need to do linking between cards. It would feel a lot simpler if USB card exposed one "USB Offload" kcontrol on USB card if USB controller supports offloading and allowed to set it to true/false to allow user to choose if they want to do offloading on device. (...) > +Mixer Examples > +-------------- > + > + :: > + > + tinymix -D 0 set 'USB Offload Playback Route Card Select' 2 > + tinymix -D 0 set 'USB Offload Playback Route PCM Select' 0 > + > + > + :: > + > + tinymix -D 0 get 'USB Offload Playback Route Card Select' > + --> 2 (range -1->32) > + tinymix -D 0 get 'USB Offload Playback Route PCM Select' > + --> 0 (range -1->255) > + > + :: > + > + tinymix -D 0 get 'USB Offload Playback Route Card Status' > + --> 2 (range -1->32) [OFFLD active] > + --> -1 (range -1->32) [OFFLD idle] > + tinymix -D 0 get 'USB Offload Playback Route PCM Status' > + --> 0 (range -1->255) [OFFLD active] > + --> -1 (range -1->255) [OFFLD idle] > + > + :: > + > + tinymix -D 1 get 'USB Offload Playback Capable Card' > + --> 0 (range -1->32) > Yes, looking at examples again, I'm still not sure I understand. There are two cards and you do linking between them, this feels broken by design. From my point of view USB Offload should be property of USB card and not involve any other card in a system.
Hi Amadeusz, On 6/12/2024 7:47 AM, Amadeusz Sławiński wrote: > On 6/11/2024 1:58 AM, Wesley Cheng wrote: > > (...) > >> +In the case where the USB offload driver is unbounded, while USB SND is > > unbounded -> unbound > > (...) > >> +SOC USB and USB Sound Kcontrols >> +=============================== >> +Details >> +------- >> +SOC USB and USB sound expose a set of SND kcontrols for applications >> to select >> +and fetch the current offloading status for the ASoC platform sound >> card. Kcontrols >> +are split between two layers: >> + >> + - USB sound - Notifies the sound card number for the ASoC >> platform sound >> + card that it is registered to for supporting audio offload. >> + >> + - SOC USB - Maintains the current status of the offload path, and >> device >> + (USB sound card and PCM device) information. This would be the >> main >> + card that applications can read to determine offloading >> capabilities. >> + >> +Implementation >> +-------------- >> + >> +**Example:** >> + >> + **Sound Cards**: >> + >> + :: >> + >> + 0 [SM8250MTPWCD938]: sm8250 - SM8250-MTP-WCD9380-WSA8810-VA-D >> + SM8250-MTP-WCD9380-WSA8810-VA-DMIC >> + 1 [C320M ]: USB-Audio - Plantronics C320-M >> + Plantronics Plantronics C320-M at >> usb-xhci-hcd.1.auto-1, full speed >> + >> + >> + **Platform Sound Card** - card#0: >> + >> + :: >> + >> + USB Offload Playback Route Card Select 1 (range -1->32) >> + USB Offload Playback Route PCM Select 0 (range -1->255) >> + USB Offload Playback Route Card Status -1 (range -1->32) >> + USB Offload Playback Route PCM Status -1 (range -1->255) >> + >> + >> + **USB Sound Card** - card#1: >> + >> + :: >> + >> + USB Offload Playback Capable Card 0 (range -1->32) >> + >> + >> +The platform sound card(card#0) kcontrols are created as part of >> adding the SOC >> +USB device using **snd_soc_usb_add_port()**. The following kcontrols >> are defined >> +as: >> + >> + - ``USB Offload Playback Route Card Status`` **(R)**: USB sound >> card device index >> + that defines which USB SND resources are currently offloaded. If >> -1 is seen, it >> + signifies that offload is not active. >> + - ``USB Offload Playback Route PCM Status`` **(R)**: USB PCM device >> index >> + that defines which USB SND resources are currently offloaded. If >> -1 is seen, it >> + signifies that offload is not active. >> + - ``USB Offload Playback Route Card Select`` **(R/W)**: USB sound >> card index which >> + selects the USB device to initiate offloading on. If no value is >> written to the >> + kcontrol, then the last USB device discovered card index will be >> chosen. > > I see only one kcontrol, what if hardware is capable of offloading on > more cards, is it possible to do offloading on more than one device? > >> + - ``USB Offload Playback Route PCM Select`` **(R/W)**: USB PCM >> index which selects >> + the USB device to initiate offloading on. If no value is written >> to the >> + kcontrol, then the last USB device discovered PCM zero index will >> be chosen. >> + >> +The USB sound card(card#1) kcontrols are created as USB audio devices >> are plugged >> +into the physical USB port and enumerated. The kcontrols are defined >> as: >> + >> + - ``USB Offload Playback Capable Card`` **(R)**: Provides the sound >> card >> + number/index that supports USB offloading. Further/follow up >> queries about >> + the current offload state can be handled by reading the offload >> status >> + kcontrol exposed by the platform card. >> + > > > Why do we need to some magic between cards? I feel like whole kcontrol > thing is overengineered a bit - I'm not sure I understand the need to do > linking between cards. It would feel a lot simpler if USB card exposed > one "USB Offload" kcontrol on USB card if USB controller supports > offloading and allowed to set it to true/false to allow user to choose > if they want to do offloading on device. > > (...) Based on feedback from Pierre, what I understood is that for some applications, there won't be an order on which sound card is queried/opened first. So the end use case example given was if an application opened the USB sound card first, it can see if there is an offload path available. If there is then it can enable the offload path on the corresponding card if desired. >> +Mixer Examples >> +-------------- >> + >> + :: >> + >> + tinymix -D 0 set 'USB Offload Playback Route Card Select' 2 >> + tinymix -D 0 set 'USB Offload Playback Route PCM Select' 0 >> + >> + >> + :: >> + >> + tinymix -D 0 get 'USB Offload Playback Route Card Select' >> + --> 2 (range -1->32) >> + tinymix -D 0 get 'USB Offload Playback Route PCM Select' >> + --> 0 (range -1->255) >> + >> + :: >> + >> + tinymix -D 0 get 'USB Offload Playback Route Card Status' >> + --> 2 (range -1->32) [OFFLD active] >> + --> -1 (range -1->32) [OFFLD idle] >> + tinymix -D 0 get 'USB Offload Playback Route PCM Status' >> + --> 0 (range -1->255) [OFFLD active] >> + --> -1 (range -1->255) [OFFLD idle] >> + >> + :: >> + >> + tinymix -D 1 get 'USB Offload Playback Capable Card' >> + --> 0 (range -1->32) >> > > Yes, looking at examples again, I'm still not sure I understand. There > are two cards and you do linking between them, this feels broken by > design. From my point of view USB Offload should be property of USB card > and not involve any other card in a system. > Main benefit to having two cards (keeping one for USB SND and another for the ASoC platform sound card) is that current applications won't break. The behavior is the same, in that if something opens the USB sound card, it will go through the same non-offloaded path. During initial reviews, I think this was a big point where folks wanted the USB PCM path to still be an option. If applications want to add the offload capabilities to its environment, they can enable it as an additional feature. Thanks Wesley Cheng
Hi Bagas, On 6/12/2024 5:25 AM, Bagas Sanjaya wrote: > On Mon, Jun 10, 2024 at 04:58:08PM -0700, Wesley Cheng wrote: >> +Overview >> +======== >> +In order to leverage the existing USB sound device support in ALSA, the >> +introduction of the ASoC USB APIs, allow for the entities to communicate >> +with one another. > "... ASoC USB APIs are introduced to allow for ..." > >> +USB Audio Device Connection Flow >> +-------------------------------- >> +USB devices can be hotplugged into the USB root hub at any point in time. >> +The BE DAI link should be aware of the current state of the physical USB >> +port, i.e. if there are any USB devices with audio interface(s) connected. >> +The following callback can be used to notify the BE DAI link of any change: >> + >> + **connection_status_cb()** > "... connection_status_cb() can be used to ..." > > Thanks. > Thanks for the review, will update Thanks Wesley Cheng
On 6/12/2024 9:28 PM, Wesley Cheng wrote: > Hi Amadeusz, > > On 6/12/2024 7:47 AM, Amadeusz Sławiński wrote: >> On 6/11/2024 1:58 AM, Wesley Cheng wrote: >> >> (...) >> >>> +In the case where the USB offload driver is unbounded, while USB SND is >> >> unbounded -> unbound >> >> (...) >> >>> +SOC USB and USB Sound Kcontrols >>> +=============================== >>> +Details >>> +------- >>> +SOC USB and USB sound expose a set of SND kcontrols for applications >>> to select >>> +and fetch the current offloading status for the ASoC platform sound >>> card. Kcontrols >>> +are split between two layers: >>> + >>> + - USB sound - Notifies the sound card number for the ASoC >>> platform sound >>> + card that it is registered to for supporting audio offload. >>> + >>> + - SOC USB - Maintains the current status of the offload path, >>> and device >>> + (USB sound card and PCM device) information. This would be >>> the main >>> + card that applications can read to determine offloading >>> capabilities. >>> + >>> +Implementation >>> +-------------- >>> + >>> +**Example:** >>> + >>> + **Sound Cards**: >>> + >>> + :: >>> + >>> + 0 [SM8250MTPWCD938]: sm8250 - SM8250-MTP-WCD9380-WSA8810-VA-D >>> + SM8250-MTP-WCD9380-WSA8810-VA-DMIC >>> + 1 [C320M ]: USB-Audio - Plantronics C320-M >>> + Plantronics Plantronics C320-M at >>> usb-xhci-hcd.1.auto-1, full speed >>> + >>> + >>> + **Platform Sound Card** - card#0: >>> + >>> + :: >>> + >>> + USB Offload Playback Route Card Select 1 (range -1->32) >>> + USB Offload Playback Route PCM Select 0 (range -1->255) >>> + USB Offload Playback Route Card Status -1 (range -1->32) >>> + USB Offload Playback Route PCM Status -1 (range -1->255) >>> + >>> + >>> + **USB Sound Card** - card#1: >>> + >>> + :: >>> + >>> + USB Offload Playback Capable Card 0 (range -1->32) >>> + >>> + >>> +The platform sound card(card#0) kcontrols are created as part of >>> adding the SOC >>> +USB device using **snd_soc_usb_add_port()**. The following >>> kcontrols are defined >>> +as: >>> + >>> + - ``USB Offload Playback Route Card Status`` **(R)**: USB sound >>> card device index >>> + that defines which USB SND resources are currently offloaded. >>> If -1 is seen, it >>> + signifies that offload is not active. >>> + - ``USB Offload Playback Route PCM Status`` **(R)**: USB PCM >>> device index >>> + that defines which USB SND resources are currently offloaded. >>> If -1 is seen, it >>> + signifies that offload is not active. >>> + - ``USB Offload Playback Route Card Select`` **(R/W)**: USB sound >>> card index which >>> + selects the USB device to initiate offloading on. If no value >>> is written to the >>> + kcontrol, then the last USB device discovered card index will be >>> chosen. >> >> I see only one kcontrol, what if hardware is capable of offloading on >> more cards, is it possible to do offloading on more than one device? >> >>> + - ``USB Offload Playback Route PCM Select`` **(R/W)**: USB PCM >>> index which selects >>> + the USB device to initiate offloading on. If no value is >>> written to the >>> + kcontrol, then the last USB device discovered PCM zero index >>> will be chosen. >>> + >>> +The USB sound card(card#1) kcontrols are created as USB audio >>> devices are plugged >>> +into the physical USB port and enumerated. The kcontrols are >>> defined as: >>> + >>> + - ``USB Offload Playback Capable Card`` **(R)**: Provides the >>> sound card >>> + number/index that supports USB offloading. Further/follow up >>> queries about >>> + the current offload state can be handled by reading the offload >>> status >>> + kcontrol exposed by the platform card. >>> + >> >> >> Why do we need to some magic between cards? I feel like whole kcontrol >> thing is overengineered a bit - I'm not sure I understand the need to >> do linking between cards. It would feel a lot simpler if USB card >> exposed one "USB Offload" kcontrol on USB card if USB controller >> supports offloading and allowed to set it to true/false to allow user >> to choose if they want to do offloading on device. >> >> (...) > > Based on feedback from Pierre, what I understood is that for some > applications, there won't be an order on which sound card is > queried/opened first. > Yes if you have multiple cards, they are probed in random order. > So the end use case example given was if an application opened the USB > sound card first, it can see if there is an offload path available. If > there is then it can enable the offload path on the corresponding card > if desired. > This still doesn't explain why you need to link cards using controls. What would not work with simple "Enable Offload" with true/false values on USB card that works while you do have above routing controls? >>> +Mixer Examples >>> +-------------- >>> + >>> + :: >>> + >>> + tinymix -D 0 set 'USB Offload Playback Route Card Select' 2 >>> + tinymix -D 0 set 'USB Offload Playback Route PCM Select' 0 >>> + >>> + >>> + :: >>> + >>> + tinymix -D 0 get 'USB Offload Playback Route Card Select' >>> + --> 2 (range -1->32) >>> + tinymix -D 0 get 'USB Offload Playback Route PCM Select' >>> + --> 0 (range -1->255) >>> + >>> + :: >>> + >>> + tinymix -D 0 get 'USB Offload Playback Route Card Status' >>> + --> 2 (range -1->32) [OFFLD active] >>> + --> -1 (range -1->32) [OFFLD idle] >>> + tinymix -D 0 get 'USB Offload Playback Route PCM Status' >>> + --> 0 (range -1->255) [OFFLD active] >>> + --> -1 (range -1->255) [OFFLD idle] >>> + >>> + :: >>> + >>> + tinymix -D 1 get 'USB Offload Playback Capable Card' >>> + --> 0 (range -1->32) >>> >> >> Yes, looking at examples again, I'm still not sure I understand. There >> are two cards and you do linking between them, this feels broken by >> design. From my point of view USB Offload should be property of USB >> card and not involve any other card in a system. >> > > Main benefit to having two cards (keeping one for USB SND and another > for the ASoC platform sound card) is that current applications won't > break. The behavior is the same, in that if something opens the USB > sound card, it will go through the same non-offloaded path. During > initial reviews, I think this was a big point where folks wanted the USB > PCM path to still be an option. > I'm not against having two cards, in fact I hope that USB card looks and behaves the same as before this patch set, with only difference being controls for enabling offload. > If applications want to add the offload capabilities to its environment, > they can enable it as an additional feature. That sounds fine to me.
Hi Amadeusz, On 6/13/2024 12:46 AM, Amadeusz Sławiński wrote: > On 6/12/2024 9:28 PM, Wesley Cheng wrote: >> Hi Amadeusz, >> >> On 6/12/2024 7:47 AM, Amadeusz Sławiński wrote: >>> On 6/11/2024 1:58 AM, Wesley Cheng wrote: >>> >>> (...) >>> >>>> +In the case where the USB offload driver is unbounded, while USB >>>> SND is >>> >>> unbounded -> unbound >>> >>> (...) >>> >>>> +SOC USB and USB Sound Kcontrols >>>> +=============================== >>>> +Details >>>> +------- >>>> +SOC USB and USB sound expose a set of SND kcontrols for >>>> applications to select >>>> +and fetch the current offloading status for the ASoC platform sound >>>> card. Kcontrols >>>> +are split between two layers: >>>> + >>>> + - USB sound - Notifies the sound card number for the ASoC >>>> platform sound >>>> + card that it is registered to for supporting audio offload. >>>> + >>>> + - SOC USB - Maintains the current status of the offload path, >>>> and device >>>> + (USB sound card and PCM device) information. This would be >>>> the main >>>> + card that applications can read to determine offloading >>>> capabilities. >>>> + >>>> +Implementation >>>> +-------------- >>>> + >>>> +**Example:** >>>> + >>>> + **Sound Cards**: >>>> + >>>> + :: >>>> + >>>> + 0 [SM8250MTPWCD938]: sm8250 - SM8250-MTP-WCD9380-WSA8810-VA-D >>>> + SM8250-MTP-WCD9380-WSA8810-VA-DMIC >>>> + 1 [C320M ]: USB-Audio - Plantronics C320-M >>>> + Plantronics Plantronics C320-M at >>>> usb-xhci-hcd.1.auto-1, full speed >>>> + >>>> + >>>> + **Platform Sound Card** - card#0: >>>> + >>>> + :: >>>> + >>>> + USB Offload Playback Route Card Select 1 (range -1->32) >>>> + USB Offload Playback Route PCM Select 0 (range -1->255) >>>> + USB Offload Playback Route Card Status -1 (range -1->32) >>>> + USB Offload Playback Route PCM Status -1 (range -1->255) >>>> + >>>> + >>>> + **USB Sound Card** - card#1: >>>> + >>>> + :: >>>> + >>>> + USB Offload Playback Capable Card 0 (range -1->32) >>>> + >>>> + >>>> +The platform sound card(card#0) kcontrols are created as part of >>>> adding the SOC >>>> +USB device using **snd_soc_usb_add_port()**. The following >>>> kcontrols are defined >>>> +as: >>>> + >>>> + - ``USB Offload Playback Route Card Status`` **(R)**: USB sound >>>> card device index >>>> + that defines which USB SND resources are currently offloaded. >>>> If -1 is seen, it >>>> + signifies that offload is not active. >>>> + - ``USB Offload Playback Route PCM Status`` **(R)**: USB PCM >>>> device index >>>> + that defines which USB SND resources are currently offloaded. >>>> If -1 is seen, it >>>> + signifies that offload is not active. >>>> + - ``USB Offload Playback Route Card Select`` **(R/W)**: USB sound >>>> card index which >>>> + selects the USB device to initiate offloading on. If no value >>>> is written to the >>>> + kcontrol, then the last USB device discovered card index will >>>> be chosen. >>> >>> I see only one kcontrol, what if hardware is capable of offloading on >>> more cards, is it possible to do offloading on more than one device? >>> >>>> + - ``USB Offload Playback Route PCM Select`` **(R/W)**: USB PCM >>>> index which selects >>>> + the USB device to initiate offloading on. If no value is >>>> written to the >>>> + kcontrol, then the last USB device discovered PCM zero index >>>> will be chosen. >>>> + >>>> +The USB sound card(card#1) kcontrols are created as USB audio >>>> devices are plugged >>>> +into the physical USB port and enumerated. The kcontrols are >>>> defined as: >>>> + >>>> + - ``USB Offload Playback Capable Card`` **(R)**: Provides the >>>> sound card >>>> + number/index that supports USB offloading. Further/follow up >>>> queries about >>>> + the current offload state can be handled by reading the offload >>>> status >>>> + kcontrol exposed by the platform card. >>>> + >>> >>> >>> Why do we need to some magic between cards? I feel like whole >>> kcontrol thing is overengineered a bit - I'm not sure I understand >>> the need to do linking between cards. It would feel a lot simpler if >>> USB card exposed one "USB Offload" kcontrol on USB card if USB >>> controller supports offloading and allowed to set it to true/false to >>> allow user to choose if they want to do offloading on device. >>> >>> (...) >> >> Based on feedback from Pierre, what I understood is that for some >> applications, there won't be an order on which sound card is >> queried/opened first. >> > > Yes if you have multiple cards, they are probed in random order. > >> So the end use case example given was if an application opened the USB >> sound card first, it can see if there is an offload path available. >> If there is then it can enable the offload path on the corresponding >> card if desired. >> > > This still doesn't explain why you need to link cards using controls. > What would not work with simple "Enable Offload" with true/false values > on USB card that works while you do have above routing controls? > Sorry for the late response. I think either way, even with the "Enable Offload" kcontrol in USB SND, we'd need a way to link these cards, because if you have multiple USB audio devices connected, and say... your offload mechanism only supports one stream. Then I assume we'd still need to way to determine if that stream can be enabled for that USB SND device or not. Since the USB SND isn't really the entity maintaining the offload path, I went with the decision to add that route selection to the ASoC platform card. It would have access to all the parameters supported by the audio DSP. Thanks Wesley Cheng >>>> +Mixer Examples >>>> +-------------- >>>> + >>>> + :: >>>> + >>>> + tinymix -D 0 set 'USB Offload Playback Route Card Select' 2 >>>> + tinymix -D 0 set 'USB Offload Playback Route PCM Select' 0 >>>> + >>>> + >>>> + :: >>>> + >>>> + tinymix -D 0 get 'USB Offload Playback Route Card Select' >>>> + --> 2 (range -1->32) >>>> + tinymix -D 0 get 'USB Offload Playback Route PCM Select' >>>> + --> 0 (range -1->255) >>>> + >>>> + :: >>>> + >>>> + tinymix -D 0 get 'USB Offload Playback Route Card Status' >>>> + --> 2 (range -1->32) [OFFLD active] >>>> + --> -1 (range -1->32) [OFFLD idle] >>>> + tinymix -D 0 get 'USB Offload Playback Route PCM Status' >>>> + --> 0 (range -1->255) [OFFLD active] >>>> + --> -1 (range -1->255) [OFFLD idle] >>>> + >>>> + :: >>>> + >>>> + tinymix -D 1 get 'USB Offload Playback Capable Card' >>>> + --> 0 (range -1->32) >>>> >>> >>> Yes, looking at examples again, I'm still not sure I understand. >>> There are two cards and you do linking between them, this feels >>> broken by design. From my point of view USB Offload should be >>> property of USB card and not involve any other card in a system. >>> >> >> Main benefit to having two cards (keeping one for USB SND and another >> for the ASoC platform sound card) is that current applications won't >> break. The behavior is the same, in that if something opens the USB >> sound card, it will go through the same non-offloaded path. During >> initial reviews, I think this was a big point where folks wanted the >> USB PCM path to still be an option. >> > > I'm not against having two cards, in fact I hope that USB card looks and > behaves the same as before this patch set, with only difference being > controls for enabling offload. > >> If applications want to add the offload capabilities to its >> environment, they can enable it as an additional feature. > > That sounds fine to me. >
On 6/17/2024 7:02 PM, Wesley Cheng wrote: > Hi Amadeusz, > > On 6/13/2024 12:46 AM, Amadeusz Sławiński wrote: >> On 6/12/2024 9:28 PM, Wesley Cheng wrote: >>> Hi Amadeusz, >>> >>> On 6/12/2024 7:47 AM, Amadeusz Sławiński wrote: >>>> On 6/11/2024 1:58 AM, Wesley Cheng wrote: >>>> >>>> (...) >>>> >>>>> +In the case where the USB offload driver is unbounded, while USB >>>>> SND is >>>> >>>> unbounded -> unbound >>>> >>>> (...) >>>> >>>>> +SOC USB and USB Sound Kcontrols >>>>> +=============================== >>>>> +Details >>>>> +------- >>>>> +SOC USB and USB sound expose a set of SND kcontrols for >>>>> applications to select >>>>> +and fetch the current offloading status for the ASoC platform >>>>> sound card. Kcontrols >>>>> +are split between two layers: >>>>> + >>>>> + - USB sound - Notifies the sound card number for the ASoC >>>>> platform sound >>>>> + card that it is registered to for supporting audio offload. >>>>> + >>>>> + - SOC USB - Maintains the current status of the offload path, >>>>> and device >>>>> + (USB sound card and PCM device) information. This would be >>>>> the main >>>>> + card that applications can read to determine offloading >>>>> capabilities. >>>>> + >>>>> +Implementation >>>>> +-------------- >>>>> + >>>>> +**Example:** >>>>> + >>>>> + **Sound Cards**: >>>>> + >>>>> + :: >>>>> + >>>>> + 0 [SM8250MTPWCD938]: sm8250 - SM8250-MTP-WCD9380-WSA8810-VA-D >>>>> + SM8250-MTP-WCD9380-WSA8810-VA-DMIC >>>>> + 1 [C320M ]: USB-Audio - Plantronics C320-M >>>>> + Plantronics Plantronics C320-M at >>>>> usb-xhci-hcd.1.auto-1, full speed >>>>> + >>>>> + >>>>> + **Platform Sound Card** - card#0: >>>>> + >>>>> + :: >>>>> + >>>>> + USB Offload Playback Route Card Select 1 (range -1->32) >>>>> + USB Offload Playback Route PCM Select 0 (range -1->255) >>>>> + USB Offload Playback Route Card Status -1 (range -1->32) >>>>> + USB Offload Playback Route PCM Status -1 (range -1->255) >>>>> + >>>>> + >>>>> + **USB Sound Card** - card#1: >>>>> + >>>>> + :: >>>>> + >>>>> + USB Offload Playback Capable Card 0 (range -1->32) >>>>> + >>>>> + >>>>> +The platform sound card(card#0) kcontrols are created as part of >>>>> adding the SOC >>>>> +USB device using **snd_soc_usb_add_port()**. The following >>>>> kcontrols are defined >>>>> +as: >>>>> + >>>>> + - ``USB Offload Playback Route Card Status`` **(R)**: USB sound >>>>> card device index >>>>> + that defines which USB SND resources are currently offloaded. >>>>> If -1 is seen, it >>>>> + signifies that offload is not active. >>>>> + - ``USB Offload Playback Route PCM Status`` **(R)**: USB PCM >>>>> device index >>>>> + that defines which USB SND resources are currently offloaded. >>>>> If -1 is seen, it >>>>> + signifies that offload is not active. >>>>> + - ``USB Offload Playback Route Card Select`` **(R/W)**: USB >>>>> sound card index which >>>>> + selects the USB device to initiate offloading on. If no value >>>>> is written to the >>>>> + kcontrol, then the last USB device discovered card index will >>>>> be chosen. >>>> >>>> I see only one kcontrol, what if hardware is capable of offloading >>>> on more cards, is it possible to do offloading on more than one device? >>>> >>>>> + - ``USB Offload Playback Route PCM Select`` **(R/W)**: USB PCM >>>>> index which selects >>>>> + the USB device to initiate offloading on. If no value is >>>>> written to the >>>>> + kcontrol, then the last USB device discovered PCM zero index >>>>> will be chosen. >>>>> + >>>>> +The USB sound card(card#1) kcontrols are created as USB audio >>>>> devices are plugged >>>>> +into the physical USB port and enumerated. The kcontrols are >>>>> defined as: >>>>> + >>>>> + - ``USB Offload Playback Capable Card`` **(R)**: Provides the >>>>> sound card >>>>> + number/index that supports USB offloading. Further/follow up >>>>> queries about >>>>> + the current offload state can be handled by reading the >>>>> offload status >>>>> + kcontrol exposed by the platform card. >>>>> + >>>> >>>> >>>> Why do we need to some magic between cards? I feel like whole >>>> kcontrol thing is overengineered a bit - I'm not sure I understand >>>> the need to do linking between cards. It would feel a lot simpler if >>>> USB card exposed one "USB Offload" kcontrol on USB card if USB >>>> controller supports offloading and allowed to set it to true/false >>>> to allow user to choose if they want to do offloading on device. >>>> >>>> (...) >>> >>> Based on feedback from Pierre, what I understood is that for some >>> applications, there won't be an order on which sound card is >>> queried/opened first. >>> >> >> Yes if you have multiple cards, they are probed in random order. >> >>> So the end use case example given was if an application opened the >>> USB sound card first, it can see if there is an offload path >>> available. If there is then it can enable the offload path on the >>> corresponding card if desired. >>> >> >> This still doesn't explain why you need to link cards using controls. >> What would not work with simple "Enable Offload" with true/false >> values on USB card that works while you do have above routing controls? >> > > Sorry for the late response. > > I think either way, even with the "Enable Offload" kcontrol in USB SND, > we'd need a way to link these cards, because if you have multiple USB > audio devices connected, and say... your offload mechanism only supports > one stream. Then I assume we'd still need to way to determine if that > stream can be enabled for that USB SND device or not. > > Since the USB SND isn't really the entity maintaining the offload path, > I went with the decision to add that route selection to the ASoC > platform card. It would have access to all the parameters supported by > the audio DSP. > Problem with card selection is that it will most likely work in pretty random way during reboots and similar scenarios. Taking from your example: USB Offload Playback Route Card Select 1 (range -1->32) USB Offload Playback Route PCM Select 0 (range -1->255) USB Offload Playback Route Card Status -1 (range -1->32) USB Offload Playback Route PCM Status -1 (range -1->255) This tells that hw:1,0 will be offloaded USB card. What happens if after reboot the USB card and offload card change places, the control will be pointing at its owner... Another scenario to consider is that user attaches two USB cards and only first one does offload. Now what happens when they enumerate in different order after reboot (swapping places)? Taking into the account that most systems restore previous values of controls in some way - this will point at wrong card. In my opinion Offload capability should be the capability of the endpoint - in this case USB card (even if in the background it needs to talk to some other device) and it should be exposed as such. Currently you are mixing capabilities of your audio card with capabilities of USB card. And adding more controls will not make it easy to use from end user perspective. Most users will most likely want for the devices to perform offload automatically if possible to save power and just have control to disable it in case they want to test if it works better without it in case of some problems. Additional question what happens if you want to offload two usb cards, currently the above set of controls allows you to only point at one card, will you be adding additional set of above controls dynamically for each USB card attached? Thanks, Amadeusz
Hi Amadeusz, On 6/18/2024 4:42 AM, Amadeusz Sławiński wrote: > On 6/17/2024 7:02 PM, Wesley Cheng wrote: >> Hi Amadeusz, >> >> On 6/13/2024 12:46 AM, Amadeusz Sławiński wrote: >>> On 6/12/2024 9:28 PM, Wesley Cheng wrote: >>>> Hi Amadeusz, >>>> >>>> On 6/12/2024 7:47 AM, Amadeusz Sławiński wrote: >>>>> On 6/11/2024 1:58 AM, Wesley Cheng wrote: >>>>> >>>>> (...) >>>>> >>>>>> +In the case where the USB offload driver is unbounded, while USB >>>>>> SND is >>>>> >>>>> unbounded -> unbound >>>>> >>>>> (...) >>>>> >>>>>> +SOC USB and USB Sound Kcontrols >>>>>> +=============================== >>>>>> +Details >>>>>> +------- >>>>>> +SOC USB and USB sound expose a set of SND kcontrols for >>>>>> applications to select >>>>>> +and fetch the current offloading status for the ASoC platform >>>>>> sound card. Kcontrols >>>>>> +are split between two layers: >>>>>> + >>>>>> + - USB sound - Notifies the sound card number for the ASoC >>>>>> platform sound >>>>>> + card that it is registered to for supporting audio offload. >>>>>> + >>>>>> + - SOC USB - Maintains the current status of the offload path, >>>>>> and device >>>>>> + (USB sound card and PCM device) information. This would be >>>>>> the main >>>>>> + card that applications can read to determine offloading >>>>>> capabilities. >>>>>> + >>>>>> +Implementation >>>>>> +-------------- >>>>>> + >>>>>> +**Example:** >>>>>> + >>>>>> + **Sound Cards**: >>>>>> + >>>>>> + :: >>>>>> + >>>>>> + 0 [SM8250MTPWCD938]: sm8250 - SM8250-MTP-WCD9380-WSA8810-VA-D >>>>>> + SM8250-MTP-WCD9380-WSA8810-VA-DMIC >>>>>> + 1 [C320M ]: USB-Audio - Plantronics C320-M >>>>>> + Plantronics Plantronics C320-M at >>>>>> usb-xhci-hcd.1.auto-1, full speed >>>>>> + >>>>>> + >>>>>> + **Platform Sound Card** - card#0: >>>>>> + >>>>>> + :: >>>>>> + >>>>>> + USB Offload Playback Route Card Select 1 (range -1->32) >>>>>> + USB Offload Playback Route PCM Select 0 (range -1->255) >>>>>> + USB Offload Playback Route Card Status -1 (range -1->32) >>>>>> + USB Offload Playback Route PCM Status -1 (range -1->255) >>>>>> + >>>>>> + >>>>>> + **USB Sound Card** - card#1: >>>>>> + >>>>>> + :: >>>>>> + >>>>>> + USB Offload Playback Capable Card 0 (range -1->32) >>>>>> + >>>>>> + >>>>>> +The platform sound card(card#0) kcontrols are created as part of >>>>>> adding the SOC >>>>>> +USB device using **snd_soc_usb_add_port()**. The following >>>>>> kcontrols are defined >>>>>> +as: >>>>>> + >>>>>> + - ``USB Offload Playback Route Card Status`` **(R)**: USB sound >>>>>> card device index >>>>>> + that defines which USB SND resources are currently offloaded. >>>>>> If -1 is seen, it >>>>>> + signifies that offload is not active. >>>>>> + - ``USB Offload Playback Route PCM Status`` **(R)**: USB PCM >>>>>> device index >>>>>> + that defines which USB SND resources are currently offloaded. >>>>>> If -1 is seen, it >>>>>> + signifies that offload is not active. >>>>>> + - ``USB Offload Playback Route Card Select`` **(R/W)**: USB >>>>>> sound card index which >>>>>> + selects the USB device to initiate offloading on. If no >>>>>> value is written to the >>>>>> + kcontrol, then the last USB device discovered card index will >>>>>> be chosen. >>>>> >>>>> I see only one kcontrol, what if hardware is capable of offloading >>>>> on more cards, is it possible to do offloading on more than one >>>>> device? >>>>> >>>>>> + - ``USB Offload Playback Route PCM Select`` **(R/W)**: USB PCM >>>>>> index which selects >>>>>> + the USB device to initiate offloading on. If no value is >>>>>> written to the >>>>>> + kcontrol, then the last USB device discovered PCM zero index >>>>>> will be chosen. >>>>>> + >>>>>> +The USB sound card(card#1) kcontrols are created as USB audio >>>>>> devices are plugged >>>>>> +into the physical USB port and enumerated. The kcontrols are >>>>>> defined as: >>>>>> + >>>>>> + - ``USB Offload Playback Capable Card`` **(R)**: Provides the >>>>>> sound card >>>>>> + number/index that supports USB offloading. Further/follow up >>>>>> queries about >>>>>> + the current offload state can be handled by reading the >>>>>> offload status >>>>>> + kcontrol exposed by the platform card. >>>>>> + >>>>> >>>>> >>>>> Why do we need to some magic between cards? I feel like whole >>>>> kcontrol thing is overengineered a bit - I'm not sure I understand >>>>> the need to do linking between cards. It would feel a lot simpler >>>>> if USB card exposed one "USB Offload" kcontrol on USB card if USB >>>>> controller supports offloading and allowed to set it to true/false >>>>> to allow user to choose if they want to do offloading on device. >>>>> >>>>> (...) >>>> >>>> Based on feedback from Pierre, what I understood is that for some >>>> applications, there won't be an order on which sound card is >>>> queried/opened first. >>>> >>> >>> Yes if you have multiple cards, they are probed in random order. >>> >>>> So the end use case example given was if an application opened the >>>> USB sound card first, it can see if there is an offload path >>>> available. If there is then it can enable the offload path on the >>>> corresponding card if desired. >>>> >>> >>> This still doesn't explain why you need to link cards using controls. >>> What would not work with simple "Enable Offload" with true/false >>> values on USB card that works while you do have above routing controls? >>> >> >> Sorry for the late response. >> >> I think either way, even with the "Enable Offload" kcontrol in USB >> SND, we'd need a way to link these cards, because if you have multiple >> USB audio devices connected, and say... your offload mechanism only >> supports one stream. Then I assume we'd still need to way to >> determine if that stream can be enabled for that USB SND device or not. >> >> Since the USB SND isn't really the entity maintaining the offload >> path, I went with the decision to add that route selection to the ASoC >> platform card. It would have access to all the parameters supported by >> the audio DSP. >> > > Problem with card selection is that it will most likely work in pretty > random way during reboots and similar scenarios. > > Taking from your example: > USB Offload Playback Route Card Select 1 (range -1->32) > USB Offload Playback Route PCM Select 0 (range -1->255) > USB Offload Playback Route Card Status -1 (range -1->32) > USB Offload Playback Route PCM Status -1 (range -1->255) > > This tells that hw:1,0 will be offloaded USB card. What happens if after > reboot the USB card and offload card change places, the control will be > pointing at its owner... Another scenario to consider is that user > attaches two USB cards and only first one does offload. Now what happens > when they enumerate in different order after reboot (swapping places)? > Taking into the account that most systems restore previous values of > controls in some way - this will point at wrong card. That sounds like a problem that would exist with current USB SND implementation too? Removing the offloading perspective, how does the system ensure that the previous setting stays persistent? For example, as you mentioned, depending on which USB device enumerates first, the sound card may be different so cards will be switched. I think I mentioned this previously in another discussion, but I think the idea was that with the USB Offload Playback Capable Card kcontrol, would allow the system to at least know there is an offload capable path pointing to the ASoC platform card, and fetch more detailed information about which device is selected for offloading, etc... > > In my opinion Offload capability should be the capability of the > endpoint - in this case USB card (even if in the background it needs to > talk to some other device) and it should be exposed as such. Currently > you are mixing capabilities of your audio card with capabilities of USB > card. > > And adding more controls will not make it easy to use from end user > perspective. Most users will most likely want for the devices to perform > offload automatically if possible to save power and just have control to > disable it in case they want to test if it works better without it in > case of some problems. I agree with you that we need to keep the controls at a minimum, but I think what I have in place is fairly reasonable. If we switch to having the USB SND controlling things, we'd save maybe one control? I think keeping the offload status controls are still fairly valuable in both scenarios, as userspace may need to verify which USB SND card is being offloaded. > > Additional question what happens if you want to offload two usb cards, > currently the above set of controls allows you to only point at one > card, will you be adding additional set of above controls dynamically > for each USB card attached? > It would depend on the number of offload streams that folks may be supporting on their platform. In our case we only have one available stream, so applications would need to switch between the two devices using the card/pcm selector. In this case, there will be only one set of controls to select the card/pcm device. As of now (I think I'll change to to add another separate set of controls per stream) if you did support multiple streams, then the current card/PCM device selector would take in multiple arugments. (ie for two streams the kcontrol can take in two values) Thanks Wesley Cheng
On 6/18/2024 10:52 PM, Wesley Cheng wrote: > Hi Amadeusz, > > On 6/18/2024 4:42 AM, Amadeusz Sławiński wrote: >> On 6/17/2024 7:02 PM, Wesley Cheng wrote: >>> Hi Amadeusz, >>> >>> On 6/13/2024 12:46 AM, Amadeusz Sławiński wrote: >>>> On 6/12/2024 9:28 PM, Wesley Cheng wrote: >>>>> Hi Amadeusz, >>>>> >>>>> On 6/12/2024 7:47 AM, Amadeusz Sławiński wrote: >>>>>> On 6/11/2024 1:58 AM, Wesley Cheng wrote: >>>>>> >>>>>> (...) >>>>>> >>>>>>> +In the case where the USB offload driver is unbounded, while USB >>>>>>> SND is >>>>>> >>>>>> unbounded -> unbound >>>>>> >>>>>> (...) >>>>>> >>>>>>> +SOC USB and USB Sound Kcontrols >>>>>>> +=============================== >>>>>>> +Details >>>>>>> +------- >>>>>>> +SOC USB and USB sound expose a set of SND kcontrols for >>>>>>> applications to select >>>>>>> +and fetch the current offloading status for the ASoC platform >>>>>>> sound card. Kcontrols >>>>>>> +are split between two layers: >>>>>>> + >>>>>>> + - USB sound - Notifies the sound card number for the ASoC >>>>>>> platform sound >>>>>>> + card that it is registered to for supporting audio offload. >>>>>>> + >>>>>>> + - SOC USB - Maintains the current status of the offload >>>>>>> path, and device >>>>>>> + (USB sound card and PCM device) information. This would >>>>>>> be the main >>>>>>> + card that applications can read to determine offloading >>>>>>> capabilities. >>>>>>> + >>>>>>> +Implementation >>>>>>> +-------------- >>>>>>> + >>>>>>> +**Example:** >>>>>>> + >>>>>>> + **Sound Cards**: >>>>>>> + >>>>>>> + :: >>>>>>> + >>>>>>> + 0 [SM8250MTPWCD938]: sm8250 - SM8250-MTP-WCD9380-WSA8810-VA-D >>>>>>> + SM8250-MTP-WCD9380-WSA8810-VA-DMIC >>>>>>> + 1 [C320M ]: USB-Audio - Plantronics C320-M >>>>>>> + Plantronics Plantronics C320-M at >>>>>>> usb-xhci-hcd.1.auto-1, full speed >>>>>>> + >>>>>>> + >>>>>>> + **Platform Sound Card** - card#0: >>>>>>> + >>>>>>> + :: >>>>>>> + >>>>>>> + USB Offload Playback Route Card Select 1 (range -1->32) >>>>>>> + USB Offload Playback Route PCM Select 0 (range -1->255) >>>>>>> + USB Offload Playback Route Card Status -1 (range -1->32) >>>>>>> + USB Offload Playback Route PCM Status -1 (range -1->255) >>>>>>> + >>>>>>> + >>>>>>> + **USB Sound Card** - card#1: >>>>>>> + >>>>>>> + :: >>>>>>> + >>>>>>> + USB Offload Playback Capable Card 0 (range -1->32) >>>>>>> + >>>>>>> + >>>>>>> +The platform sound card(card#0) kcontrols are created as part of >>>>>>> adding the SOC >>>>>>> +USB device using **snd_soc_usb_add_port()**. The following >>>>>>> kcontrols are defined >>>>>>> +as: >>>>>>> + >>>>>>> + - ``USB Offload Playback Route Card Status`` **(R)**: USB >>>>>>> sound card device index >>>>>>> + that defines which USB SND resources are currently >>>>>>> offloaded. If -1 is seen, it >>>>>>> + signifies that offload is not active. >>>>>>> + - ``USB Offload Playback Route PCM Status`` **(R)**: USB PCM >>>>>>> device index >>>>>>> + that defines which USB SND resources are currently >>>>>>> offloaded. If -1 is seen, it >>>>>>> + signifies that offload is not active. >>>>>>> + - ``USB Offload Playback Route Card Select`` **(R/W)**: USB >>>>>>> sound card index which >>>>>>> + selects the USB device to initiate offloading on. If no >>>>>>> value is written to the >>>>>>> + kcontrol, then the last USB device discovered card index >>>>>>> will be chosen. >>>>>> >>>>>> I see only one kcontrol, what if hardware is capable of offloading >>>>>> on more cards, is it possible to do offloading on more than one >>>>>> device? >>>>>> >>>>>>> + - ``USB Offload Playback Route PCM Select`` **(R/W)**: USB PCM >>>>>>> index which selects >>>>>>> + the USB device to initiate offloading on. If no value is >>>>>>> written to the >>>>>>> + kcontrol, then the last USB device discovered PCM zero index >>>>>>> will be chosen. >>>>>>> + >>>>>>> +The USB sound card(card#1) kcontrols are created as USB audio >>>>>>> devices are plugged >>>>>>> +into the physical USB port and enumerated. The kcontrols are >>>>>>> defined as: >>>>>>> + >>>>>>> + - ``USB Offload Playback Capable Card`` **(R)**: Provides the >>>>>>> sound card >>>>>>> + number/index that supports USB offloading. Further/follow >>>>>>> up queries about >>>>>>> + the current offload state can be handled by reading the >>>>>>> offload status >>>>>>> + kcontrol exposed by the platform card. >>>>>>> + >>>>>> >>>>>> >>>>>> Why do we need to some magic between cards? I feel like whole >>>>>> kcontrol thing is overengineered a bit - I'm not sure I understand >>>>>> the need to do linking between cards. It would feel a lot simpler >>>>>> if USB card exposed one "USB Offload" kcontrol on USB card if USB >>>>>> controller supports offloading and allowed to set it to true/false >>>>>> to allow user to choose if they want to do offloading on device. >>>>>> >>>>>> (...) >>>>> >>>>> Based on feedback from Pierre, what I understood is that for some >>>>> applications, there won't be an order on which sound card is >>>>> queried/opened first. >>>>> >>>> >>>> Yes if you have multiple cards, they are probed in random order. >>>> >>>>> So the end use case example given was if an application opened the >>>>> USB sound card first, it can see if there is an offload path >>>>> available. If there is then it can enable the offload path on the >>>>> corresponding card if desired. >>>>> >>>> >>>> This still doesn't explain why you need to link cards using >>>> controls. What would not work with simple "Enable Offload" with >>>> true/false values on USB card that works while you do have above >>>> routing controls? >>>> >>> >>> Sorry for the late response. >>> >>> I think either way, even with the "Enable Offload" kcontrol in USB >>> SND, we'd need a way to link these cards, because if you have >>> multiple USB audio devices connected, and say... your offload >>> mechanism only supports one stream. Then I assume we'd still need to >>> way to determine if that stream can be enabled for that USB SND >>> device or not. >>> >>> Since the USB SND isn't really the entity maintaining the offload >>> path, I went with the decision to add that route selection to the >>> ASoC platform card. It would have access to all the parameters >>> supported by the audio DSP. >>> >> >> Problem with card selection is that it will most likely work in pretty >> random way during reboots and similar scenarios. >> >> Taking from your example: >> USB Offload Playback Route Card Select 1 (range -1->32) >> USB Offload Playback Route PCM Select 0 (range -1->255) >> USB Offload Playback Route Card Status -1 (range -1->32) >> USB Offload Playback Route PCM Status -1 (range -1->255) >> >> This tells that hw:1,0 will be offloaded USB card. What happens if >> after reboot the USB card and offload card change places, the control >> will be pointing at its owner... Another scenario to consider is that >> user attaches two USB cards and only first one does offload. Now what >> happens when they enumerate in different order after reboot (swapping >> places)? Taking into the account that most systems restore previous >> values of controls in some way - this will point at wrong card. > > That sounds like a problem that would exist with current USB SND > implementation too? Removing the offloading perspective, how does the > system ensure that the previous setting stays persistent? For example, > as you mentioned, depending on which USB device enumerates first, the > sound card may be different so cards will be switched. > It works because there is no control pointing at other card. My main problem is with controls which have card and pcm id of other card in it. > I think I mentioned this previously in another discussion, but I think > the idea was that with the > USB Offload Playback Capable Card > > kcontrol, would allow the system to at least know there is an offload > capable path pointing to the ASoC platform card, and fetch more detailed > information about which device is selected for offloading, etc... > This works only in your design, where USB Offload is backed by card, what happens if it is backed by something else? >> >> In my opinion Offload capability should be the capability of the >> endpoint - in this case USB card (even if in the background it needs >> to talk to some other device) and it should be exposed as such. >> Currently you are mixing capabilities of your audio card with >> capabilities of USB card. >> >> And adding more controls will not make it easy to use from end user >> perspective. Most users will most likely want for the devices to >> perform offload automatically if possible to save power and just have >> control to disable it in case they want to test if it works better >> without it in case of some problems. > > I agree with you that we need to keep the controls at a minimum, but I > think what I have in place is fairly reasonable. If we switch to having > the USB SND controlling things, we'd save maybe one control? I think > keeping the offload status controls are still fairly valuable in both > scenarios, as userspace may need to verify which USB SND card is being > offloaded. > It should be able to tell which one is being offloaded by examining which USB card has Offload control set to true. I would assume that USB cards that cannot perform Offload have no control at all, as it is unneeded. And ones that can do, have Offload control. And ones actively being Offloaded have it set to true, otherwise to false. End user has no need to know where it is offloaded. I'm not HW person, but I would assume that it is even unlikely that someone will design HW, where it is possible to Offload one endpoint to two different places, as this complicates things a lot, but if it were possible, from design perspective it would make a lot more sense to set it in Offloaded USB card settings, instead of some seemingly unrelated controller card device. And that is assuming that all solutions use some other card device to perform Offload. >> >> Additional question what happens if you want to offload two usb cards, >> currently the above set of controls allows you to only point at one >> card, will you be adding additional set of above controls dynamically >> for each USB card attached? >> > > It would depend on the number of offload streams that folks may be > supporting on their platform. In our case we only have one available > stream, so applications would need to switch between the two devices > using the card/pcm selector. > > In this case, there will be only one set of controls to select the > card/pcm device. As of now (I think I'll change to to add another > separate set of controls per stream) if you did support multiple > streams, then the current card/PCM device selector would take in > multiple arugments. (ie for two streams the kcontrol can take in two > values) > Then it is implementation detail of your device, and it should be implemented as controls in your device instead of as part of generic API. Thanks, Amadeusz
Hi Amaduesz, On 6/19/2024 12:52 AM, Amadeusz Sławiński wrote: > On 6/18/2024 10:52 PM, Wesley Cheng wrote: >> Hi Amadeusz, >> >> On 6/18/2024 4:42 AM, Amadeusz Sławiński wrote: >>> On 6/17/2024 7:02 PM, Wesley Cheng wrote: >>>> Hi Amadeusz, >>>> >>>> On 6/13/2024 12:46 AM, Amadeusz Sławiński wrote: >>>>> On 6/12/2024 9:28 PM, Wesley Cheng wrote: >>>>>> Hi Amadeusz, >>>>>> >>>>>> On 6/12/2024 7:47 AM, Amadeusz Sławiński wrote: >>>>>>> On 6/11/2024 1:58 AM, Wesley Cheng wrote: >>>>>>> >>>>>>> (...) >>>>>>> >>>>>>>> +In the case where the USB offload driver is unbounded, while >>>>>>>> USB SND is >>>>>>> >>>>>>> unbounded -> unbound >>>>>>> >>>>>>> (...) >>>>>>> >>>>>>>> +SOC USB and USB Sound Kcontrols >>>>>>>> +=============================== >>>>>>>> +Details >>>>>>>> +------- >>>>>>>> +SOC USB and USB sound expose a set of SND kcontrols for >>>>>>>> applications to select >>>>>>>> +and fetch the current offloading status for the ASoC platform >>>>>>>> sound card. Kcontrols >>>>>>>> +are split between two layers: >>>>>>>> + >>>>>>>> + - USB sound - Notifies the sound card number for the ASoC >>>>>>>> platform sound >>>>>>>> + card that it is registered to for supporting audio offload. >>>>>>>> + >>>>>>>> + - SOC USB - Maintains the current status of the offload >>>>>>>> path, and device >>>>>>>> + (USB sound card and PCM device) information. This would >>>>>>>> be the main >>>>>>>> + card that applications can read to determine offloading >>>>>>>> capabilities. >>>>>>>> + >>>>>>>> +Implementation >>>>>>>> +-------------- >>>>>>>> + >>>>>>>> +**Example:** >>>>>>>> + >>>>>>>> + **Sound Cards**: >>>>>>>> + >>>>>>>> + :: >>>>>>>> + >>>>>>>> + 0 [SM8250MTPWCD938]: sm8250 - >>>>>>>> SM8250-MTP-WCD9380-WSA8810-VA-D >>>>>>>> + SM8250-MTP-WCD9380-WSA8810-VA-DMIC >>>>>>>> + 1 [C320M ]: USB-Audio - Plantronics C320-M >>>>>>>> + Plantronics Plantronics C320-M at >>>>>>>> usb-xhci-hcd.1.auto-1, full speed >>>>>>>> + >>>>>>>> + >>>>>>>> + **Platform Sound Card** - card#0: >>>>>>>> + >>>>>>>> + :: >>>>>>>> + >>>>>>>> + USB Offload Playback Route Card Select 1 (range -1->32) >>>>>>>> + USB Offload Playback Route PCM Select 0 (range -1->255) >>>>>>>> + USB Offload Playback Route Card Status -1 (range -1->32) >>>>>>>> + USB Offload Playback Route PCM Status -1 (range -1->255) >>>>>>>> + >>>>>>>> + >>>>>>>> + **USB Sound Card** - card#1: >>>>>>>> + >>>>>>>> + :: >>>>>>>> + >>>>>>>> + USB Offload Playback Capable Card 0 (range -1->32) >>>>>>>> + >>>>>>>> + >>>>>>>> +The platform sound card(card#0) kcontrols are created as part >>>>>>>> of adding the SOC >>>>>>>> +USB device using **snd_soc_usb_add_port()**. The following >>>>>>>> kcontrols are defined >>>>>>>> +as: >>>>>>>> + >>>>>>>> + - ``USB Offload Playback Route Card Status`` **(R)**: USB >>>>>>>> sound card device index >>>>>>>> + that defines which USB SND resources are currently >>>>>>>> offloaded. If -1 is seen, it >>>>>>>> + signifies that offload is not active. >>>>>>>> + - ``USB Offload Playback Route PCM Status`` **(R)**: USB PCM >>>>>>>> device index >>>>>>>> + that defines which USB SND resources are currently >>>>>>>> offloaded. If -1 is seen, it >>>>>>>> + signifies that offload is not active. >>>>>>>> + - ``USB Offload Playback Route Card Select`` **(R/W)**: USB >>>>>>>> sound card index which >>>>>>>> + selects the USB device to initiate offloading on. If no >>>>>>>> value is written to the >>>>>>>> + kcontrol, then the last USB device discovered card index >>>>>>>> will be chosen. >>>>>>> >>>>>>> I see only one kcontrol, what if hardware is capable of >>>>>>> offloading on more cards, is it possible to do offloading on more >>>>>>> than one device? >>>>>>> >>>>>>>> + - ``USB Offload Playback Route PCM Select`` **(R/W)**: USB >>>>>>>> PCM index which selects >>>>>>>> + the USB device to initiate offloading on. If no value is >>>>>>>> written to the >>>>>>>> + kcontrol, then the last USB device discovered PCM zero >>>>>>>> index will be chosen. >>>>>>>> + >>>>>>>> +The USB sound card(card#1) kcontrols are created as USB audio >>>>>>>> devices are plugged >>>>>>>> +into the physical USB port and enumerated. The kcontrols are >>>>>>>> defined as: >>>>>>>> + >>>>>>>> + - ``USB Offload Playback Capable Card`` **(R)**: Provides the >>>>>>>> sound card >>>>>>>> + number/index that supports USB offloading. Further/follow >>>>>>>> up queries about >>>>>>>> + the current offload state can be handled by reading the >>>>>>>> offload status >>>>>>>> + kcontrol exposed by the platform card. >>>>>>>> + >>>>>>> >>>>>>> >>>>>>> Why do we need to some magic between cards? I feel like whole >>>>>>> kcontrol thing is overengineered a bit - I'm not sure I >>>>>>> understand the need to do linking between cards. It would feel a >>>>>>> lot simpler if USB card exposed one "USB Offload" kcontrol on USB >>>>>>> card if USB controller supports offloading and allowed to set it >>>>>>> to true/false to allow user to choose if they want to do >>>>>>> offloading on device. >>>>>>> >>>>>>> (...) >>>>>> >>>>>> Based on feedback from Pierre, what I understood is that for some >>>>>> applications, there won't be an order on which sound card is >>>>>> queried/opened first. >>>>>> >>>>> >>>>> Yes if you have multiple cards, they are probed in random order. >>>>> >>>>>> So the end use case example given was if an application opened the >>>>>> USB sound card first, it can see if there is an offload path >>>>>> available. If there is then it can enable the offload path on the >>>>>> corresponding card if desired. >>>>>> >>>>> >>>>> This still doesn't explain why you need to link cards using >>>>> controls. What would not work with simple "Enable Offload" with >>>>> true/false values on USB card that works while you do have above >>>>> routing controls? >>>>> >>>> >>>> Sorry for the late response. >>>> >>>> I think either way, even with the "Enable Offload" kcontrol in USB >>>> SND, we'd need a way to link these cards, because if you have >>>> multiple USB audio devices connected, and say... your offload >>>> mechanism only supports one stream. Then I assume we'd still need >>>> to way to determine if that stream can be enabled for that USB SND >>>> device or not. >>>> >>>> Since the USB SND isn't really the entity maintaining the offload >>>> path, I went with the decision to add that route selection to the >>>> ASoC platform card. It would have access to all the parameters >>>> supported by the audio DSP. >>>> >>> >>> Problem with card selection is that it will most likely work in >>> pretty random way during reboots and similar scenarios. >>> >>> Taking from your example: >>> USB Offload Playback Route Card Select 1 (range -1->32) >>> USB Offload Playback Route PCM Select 0 (range -1->255) >>> USB Offload Playback Route Card Status -1 (range -1->32) >>> USB Offload Playback Route PCM Status -1 (range -1->255) >>> >>> This tells that hw:1,0 will be offloaded USB card. What happens if >>> after reboot the USB card and offload card change places, the control >>> will be pointing at its owner... Another scenario to consider is that >>> user attaches two USB cards and only first one does offload. Now what >>> happens when they enumerate in different order after reboot (swapping >>> places)? Taking into the account that most systems restore previous >>> values of controls in some way - this will point at wrong card. >> >> That sounds like a problem that would exist with current USB SND >> implementation too? Removing the offloading perspective, how does the >> system ensure that the previous setting stays persistent? For >> example, as you mentioned, depending on which USB device enumerates >> first, the sound card may be different so cards will be switched. >> > > It works because there is no control pointing at other card. My main > problem is with controls which have card and pcm id of other card in it. > >> I think I mentioned this previously in another discussion, but I think >> the idea was that with the >> USB Offload Playback Capable Card >> >> kcontrol, would allow the system to at least know there is an offload >> capable path pointing to the ASoC platform card, and fetch more >> detailed information about which device is selected for offloading, >> etc... >> > > This works only in your design, where USB Offload is backed by card, > what happens if it is backed by something else? > >>> >>> In my opinion Offload capability should be the capability of the >>> endpoint - in this case USB card (even if in the background it needs >>> to talk to some other device) and it should be exposed as such. >>> Currently you are mixing capabilities of your audio card with >>> capabilities of USB card. >>> >>> And adding more controls will not make it easy to use from end user >>> perspective. Most users will most likely want for the devices to >>> perform offload automatically if possible to save power and just have >>> control to disable it in case they want to test if it works better >>> without it in case of some problems. >> >> I agree with you that we need to keep the controls at a minimum, but I >> think what I have in place is fairly reasonable. If we switch to >> having the USB SND controlling things, we'd save maybe one control? I >> think keeping the offload status controls are still fairly valuable in >> both scenarios, as userspace may need to verify which USB SND card is >> being offloaded. >> > > It should be able to tell which one is being offloaded by examining > which USB card has Offload control set to true. > > I would assume that USB cards that cannot perform Offload have no > control at all, as it is unneeded. And ones that can do, have Offload > control. And ones actively being Offloaded have it set to true, > otherwise to false. > > End user has no need to know where it is offloaded. I'm not HW person, > but I would assume that it is even unlikely that someone will design HW, > where it is possible to Offload one endpoint to two different places, as > this complicates things a lot, but if it were possible, from design > perspective it would make a lot more sense to set it in Offloaded USB > card settings, instead of some seemingly unrelated controller card > device. And that is assuming that all solutions use some other card > device to perform Offload. > >>> >>> Additional question what happens if you want to offload two usb >>> cards, currently the above set of controls allows you to only point >>> at one card, will you be adding additional set of above controls >>> dynamically for each USB card attached? >>> >> >> It would depend on the number of offload streams that folks may be >> supporting on their platform. In our case we only have one available >> stream, so applications would need to switch between the two devices >> using the card/pcm selector. >> >> In this case, there will be only one set of controls to select the >> card/pcm device. As of now (I think I'll change to to add another >> separate set of controls per stream) if you did support multiple >> streams, then the current card/PCM device selector would take in >> multiple arugments. (ie for two streams the kcontrol can take in two >> values) >> > > Then it is implementation detail of your device, and it should be > implemented as controls in your device instead of as part of generic API. > I initially had it as part of the device specific files, but I did get some feedback on [1], it might be better to have generic control names, hence the reason for moving into soc-usb. I'll spend some time to evaluate your suggestion about moving the logic to control the offloading from USB SND versus ASoC, since there are valid points. However, before I do that, I just want to make sure folks are also inline with that thinking. I've had to put a lot of effort moving things around such as the previous example, and now you've suggested to move it back to the vendor specific drivers. @Pierre, since you've helped with providing a lot of valuable input in the previous revisions on the kcontrol uses, what do you think about the proposal from Amadeusz? Basically shifting the offload device selection into USB SND from the ASoC USB BE driver, and having this per USB SND device. [1] https://lore.kernel.org/linux-usb/20231017200109.11407-30-quic_wcheng@quicinc.com/
> I'll spend some time to evaluate your suggestion about moving the logic > to control the offloading from USB SND versus ASoC, since there are > valid points. However, before I do that, I just want to make sure folks > are also inline with that thinking. I've had to put a lot of effort > moving things around such as the previous example, and now you've > suggested to move it back to the vendor specific drivers. > > @Pierre, since you've helped with providing a lot of valuable input in > the previous revisions on the kcontrol uses, what do you think about the > proposal from Amadeusz? Basically shifting the offload device selection > into USB SND from the ASoC USB BE driver, and having this per USB SND > device. > > [1] > https://lore.kernel.org/linux-usb/20231017200109.11407-30-quic_wcheng@quicinc.com/ This thread is very hard to follow, I am not sure I fully understood the initial proposal, and I am not sure I follow Amadeusz' either. There are really multiple layers to deal with a) is the controller able to support the offload path? IIRC this is embedded in an obscure XHCI property, it would make sense to expose it as a control, or component string, of the USB card. b) is there a companion card capable of dealing with the offload path? Since the presence of this card may depend on driver probe, there should be a control on the USB card. userspace could detect changes to this control and detect if that path is or is no longer enabled. c) which PCM device is actually offloaded? This could be plural for some implementations. The mapping between PCM devices exposed by the USB card, and those exposed by the companion card, should be known to userspace. I am not sure how this would be done though, a variable number of controls is a sure way to confuse userspace. At any rate, I would put all the controls under the USB generic card, because it's always present no matter what the controller or DSP configurations are.
Hi Pierre, On 6/21/2024 1:27 AM, Pierre-Louis Bossart wrote: > > >> I'll spend some time to evaluate your suggestion about moving the logic >> to control the offloading from USB SND versus ASoC, since there are >> valid points. However, before I do that, I just want to make sure folks >> are also inline with that thinking. I've had to put a lot of effort >> moving things around such as the previous example, and now you've >> suggested to move it back to the vendor specific drivers. >> >> @Pierre, since you've helped with providing a lot of valuable input in >> the previous revisions on the kcontrol uses, what do you think about the >> proposal from Amadeusz? Basically shifting the offload device selection >> into USB SND from the ASoC USB BE driver, and having this per USB SND >> device. >> >> [1] >> https://lore.kernel.org/linux-usb/20231017200109.11407-30-quic_wcheng@quicinc.com/ > This thread is very hard to follow, I am not sure I fully understood the > initial proposal, and I am not sure I follow Amadeusz' either. > > There are really multiple layers to deal with > > a) is the controller able to support the offload path? IIRC this is > embedded in an obscure XHCI property, it would make sense to expose it > as a control, or component string, of the USB card. If a component string/tag is desired, I already have some way of doing that. I can add it to the USB card. > > b) is there a companion card capable of dealing with the offload path? > Since the presence of this card may depend on driver probe, there should > be a control on the USB card. userspace could detect changes to this > control and detect if that path is or is no longer enabled. So currently, the "USB Offload Playback Capable Card" kcontrol (on the USB card) will determine if there is an offload path. However, this differs than what Amadeusz is suggesting, in that he wants a single kcontrol created for EACH USB card identified (per USB audio device), and a simple enable/disable control to determine if the offload path is enabled for that card/pcm stream. It would be a simpler approach for the userspace, and if the card that handles the offload card isn't present, then these enable/disable control will be set to "disabled," and even if users attempt to set the control, it won't go through. > > c) which PCM device is actually offloaded? This could be plural for some > implementations. The mapping between PCM devices exposed by the USB > card, and those exposed by the companion card, should be known to > userspace. I am not sure how this would be done though, a variable > number of controls is a sure way to confuse userspace. Expanding on Amadeusz's suggestion, my idea is to have an enable/disable kcontrol per USB stream. For example, one USB card could have multiple PCM devices (USB streams). So we would have something like: PCM Offload Playback Enable Stream#0 enable/disable PCM Offload Playback Enable Stream#1 enable/disable .... So we'd know which USB card and PCM device is selected for USB SND. However, I see what you're getting at in case there are multiple supported streams, because userspace needs to know which ASoC card/pcm combination corresponds to which USB device/combination. What do you think about having a USB card kcontrol to display the mapped ASoC card and PCM indexes? PCM Offload Playback Enable Stream Mapping#0 0, 1 (ASoC card#0, PCM device#1) To summarize, if we did this, I'd plan to remove all the kcontrols in ASoC card, and have the following in the USB card for an USB audio device that supports one USB stream: PCM Offload Playback Enable Stream#0 enable/disable PCM Offload Playback Enable Stream Mapping#0 0, 1 (ASoC card#0, PCM device#1) Thanks Wesley Cheng
>> There are really multiple layers to deal with >> >> a) is the controller able to support the offload path? IIRC this is >> embedded in an obscure XHCI property, it would make sense to expose it >> as a control, or component string, of the USB card. > > If a component string/tag is desired, I already have some way of doing that. I can add it to the USB card. > >> >> b) is there a companion card capable of dealing with the offload path? >> Since the presence of this card may depend on driver probe, there should >> be a control on the USB card. userspace could detect changes to this >> control and detect if that path is or is no longer enabled. > > So currently, the "USB Offload Playback Capable Card" kcontrol (on the USB card) will determine if there is an offload path. However, this differs than what Amadeusz is suggesting, in that he wants a single kcontrol created for EACH USB card identified (per USB audio device), and a simple enable/disable control to determine if the offload path is enabled for that card/pcm stream. > > It would be a simpler approach for the userspace, and if the card that handles the offload card isn't present, then these enable/disable control will be set to "disabled," and even if users attempt to set the control, it won't go through. Not following. Are you suggesting userspace would modify the enable/disable status? I would just have a read-only control that reports what the hardware can do and which other card can deal with offload. It's up to userspace to select the offloaded PCM device or not. >> c) which PCM device is actually offloaded? This could be plural for some >> implementations. The mapping between PCM devices exposed by the USB >> card, and those exposed by the companion card, should be known to >> userspace. I am not sure how this would be done though, a variable >> number of controls is a sure way to confuse userspace. > > Expanding on Amadeusz's suggestion, my idea is to have an enable/disable kcontrol per USB stream. For example, one USB card could have multiple PCM devices (USB streams). So we would have something like: > > PCM Offload Playback Enable Stream#0 enable/disable > > PCM Offload Playback Enable Stream#1 enable/disable > > .... are those read-only or not? > So we'd know which USB card and PCM device is selected for USB SND. However, I see what you're getting at in case there are multiple supported streams, because userspace needs to know which ASoC card/pcm combination corresponds to which USB device/combination. I don't understand how this would help map the two parts? There's got to be an additional mapping... > What do you think about having a USB card kcontrol to display the mapped ASoC card and PCM indexes? > > PCM Offload Playback Enable Stream Mapping#0 0, 1 (ASoC card#0, PCM device#1) > > To summarize, if we did this, I'd plan to remove all the kcontrols in ASoC card, and have the following in the USB card for an USB audio device that supports one USB stream: > > PCM Offload Playback Enable Stream#0 enable/disable > > PCM Offload Playback Enable Stream Mapping#0 0, 1 (ASoC card#0, PCM device#1) ... which is suggested here. Assuming these are read-only controls, we would need to know which PCM device on the USB card can be optimized with the use of which PCM device on the ASoC card. That means a set of three values. You would also want a number of streams to make the guesswork on controls less painful.
Hi Pierre/Amadeusz, On 7/2/2024 1:30 AM, Pierre-Louis Bossart wrote: >>> There are really multiple layers to deal with >>> >>> a) is the controller able to support the offload path? IIRC this is >>> embedded in an obscure XHCI property, it would make sense to expose it >>> as a control, or component string, of the USB card. >> If a component string/tag is desired, I already have some way of doing that. I can add it to the USB card. >> >>> b) is there a companion card capable of dealing with the offload path? >>> Since the presence of this card may depend on driver probe, there should >>> be a control on the USB card. userspace could detect changes to this >>> control and detect if that path is or is no longer enabled. >> So currently, the "USB Offload Playback Capable Card" kcontrol (on the USB card) will determine if there is an offload path. However, this differs than what Amadeusz is suggesting, in that he wants a single kcontrol created for EACH USB card identified (per USB audio device), and a simple enable/disable control to determine if the offload path is enabled for that card/pcm stream. >> >> It would be a simpler approach for the userspace, and if the card that handles the offload card isn't present, then these enable/disable control will be set to "disabled," and even if users attempt to set the control, it won't go through. > Not following. Are you suggesting userspace would modify the > enable/disable status? Yes, this is the suggestion. One writeable kcontrol on the USB SND audio device that will control if that USB audio device is going to be offloaded. If the kcontrol reads back "enabled" (or 1) then userspace knows that the offload path is active. Else, if it reads "disabled" (or 0) after the attempt to set the kcontrol, then the offload path was unsuccessfully enabled, ie maybe due to no available offload streams. > I would just have a read-only control that reports what the hardware can > do and which other card can deal with offload. It's up to userspace to > select the offloaded PCM device or not. > That is what I have implemented in the previous patch series. One USB SND kcontrol within each USB audio device, which points to the ASoC platform card that supports offloading: "USB Offload Playback Capable Card" --> returns the card index to the ASoC platform card From there the offloading control is all within the ASoC platform card. This is opposite to what Amaduesz suggested in that, the offload control of which USB device to offload should be within USB SND (not ASoC) > >>> c) which PCM device is actually offloaded? This could be plural for some >>> implementations. The mapping between PCM devices exposed by the USB >>> card, and those exposed by the companion card, should be known to >>> userspace. I am not sure how this would be done though, a variable >>> number of controls is a sure way to confuse userspace. >> Expanding on Amadeusz's suggestion, my idea is to have an enable/disable kcontrol per USB stream. For example, one USB card could have multiple PCM devices (USB streams). So we would have something like: >> >> PCM Offload Playback Enable Stream#0 enable/disable >> >> PCM Offload Playback Enable Stream#1 enable/disable >> >> .... > are those read-only or not? No, writable and readable. > >> So we'd know which USB card and PCM device is selected for USB SND. However, I see what you're getting at in case there are multiple supported streams, because userspace needs to know which ASoC card/pcm combination corresponds to which USB device/combination. > I don't understand how this would help map the two parts? There's got to > be an additional mapping... It won't help with the mapping. That is something which we'd need to add, suggestion below. >> What do you think about having a USB card kcontrol to display the mapped ASoC card and PCM indexes? >> >> PCM Offload Playback Enable Stream Mapping#0 0, 1 (ASoC card#0, PCM device#1) >> >> To summarize, if we did this, I'd plan to remove all the kcontrols in ASoC card, and have the following in the USB card for an USB audio device that supports one USB stream: >> >> PCM Offload Playback Enable Stream#0 enable/disable >> >> PCM Offload Playback Enable Stream Mapping#0 0, 1 (ASoC card#0, PCM device#1) > ... which is suggested here. > > Assuming these are read-only controls, we would need to know which PCM > device on the USB card can be optimized with the use of which PCM device > on the ASoC card. That means a set of three values. You would also want > a number of streams to make the guesswork on controls less painful. OK, so now to just figuring out something that both you and Amadeusz can agree on before I put time implementing it. So I've implemented the "enable/disable" path that Amadeusz suggested, which is highlighted in my previous response, for evaluation purposes. The overall question is which layer should control the devices that will be offloaded. In my submissions up until now, the control was given to the ASoC platform card to determine which USB device to offload. Amadeusz mentioned that it might be beneficial to move the control to the USB SND devices, because what if the offloading is NOT backed by ASoC. (highlighted in [1]) However, IMO the current implementation assumes there is ASoC involved, which should mean that there is some platform "card" that is backing the offload path. Please let me know if my understanding is incorrect, @Amadeusz. [1] - Re: [PATCH v23 32/32] ASoC: doc: Add documentation for SOC USB - Amadeusz Sławiński (kernel.org) <https://lore.kernel.org/linux-usb/510468c7-b181-48d0-bf2d-3e478b2f2aca@linux.intel.com/> Thanks Wesley Cheng
>>>> There are really multiple layers to deal with >>>> >>>> a) is the controller able to support the offload path? IIRC this is >>>> embedded in an obscure XHCI property, it would make sense to expose it >>>> as a control, or component string, of the USB card. >>> If a component string/tag is desired, I already have some way of doing that. I can add it to the USB card. >>> >>>> b) is there a companion card capable of dealing with the offload path? >>>> Since the presence of this card may depend on driver probe, there should >>>> be a control on the USB card. userspace could detect changes to this >>>> control and detect if that path is or is no longer enabled. >>> So currently, the "USB Offload Playback Capable Card" kcontrol (on the USB card) will determine if there is an offload path. However, this differs than what Amadeusz is suggesting, in that he wants a single kcontrol created for EACH USB card identified (per USB audio device), and a simple enable/disable control to determine if the offload path is enabled for that card/pcm stream. >>> >>> It would be a simpler approach for the userspace, and if the card that handles the offload card isn't present, then these enable/disable control will be set to "disabled," and even if users attempt to set the control, it won't go through. >> Not following. Are you suggesting userspace would modify the >> enable/disable status? > > Yes, this is the suggestion. One writeable kcontrol on the USB SND audio device that will control if that USB audio device is going to be offloaded. If the kcontrol reads back "enabled" (or 1) then userspace knows that the offload path is active. Else, if it reads "disabled" (or 0) after the attempt to set the kcontrol, then the offload path was unsuccessfully enabled, ie maybe due to no available offload streams. It's a bit over-engineered IMHO. My alternate suggestion is a read-only control reporting that offload is possible. Then userspace attempts to open a PCM device on the ASoC card, any failures due to resources would be handled at that point. >> I would just have a read-only control that reports what the hardware can >> do and which other card can deal with offload. It's up to userspace to >> select the offloaded PCM device or not. >> > That is what I have implemented in the previous patch series. One USB SND kcontrol within each USB audio device, which points to the ASoC platform card that supports offloading: > > "USB Offload Playback Capable Card" --> returns the card index to the ASoC platform card > >>From there the offloading control is all within the ASoC platform card. This is opposite to what Amaduesz suggested in that, the offload control of which USB device to offload should be within USB SND (not ASoC) It's very hard to follow, I don't understand what userspace needs to 'control' - in the modify sense. What userspace needs is a place to read from, and then select the PCM device and follow usual ALSA configuration with hw_params. >>>> c) which PCM device is actually offloaded? This could be plural for some >>>> implementations. The mapping between PCM devices exposed by the USB >>>> card, and those exposed by the companion card, should be known to >>>> userspace. I am not sure how this would be done though, a variable >>>> number of controls is a sure way to confuse userspace. >>> Expanding on Amadeusz's suggestion, my idea is to have an enable/disable kcontrol per USB stream. For example, one USB card could have multiple PCM devices (USB streams). So we would have something like: >>> >>> PCM Offload Playback Enable Stream#0 enable/disable >>> >>> PCM Offload Playback Enable Stream#1 enable/disable >>> >>> .... >> are those read-only or not? > > No, writable and readable. The writable part introduces a complicated error handling, e.g. what happens if you have an offloaded stream and then this control is changed with amixer while streaming? >>> So we'd know which USB card and PCM device is selected for USB SND. However, I see what you're getting at in case there are multiple supported streams, because userspace needs to know which ASoC card/pcm combination corresponds to which USB device/combination. >> I don't understand how this would help map the two parts? There's got to >> be an additional mapping... > It won't help with the mapping. That is something which we'd need to add, suggestion below. >>> What do you think about having a USB card kcontrol to display the mapped ASoC card and PCM indexes? >>> >>> PCM Offload Playback Enable Stream Mapping#0 0, 1 (ASoC card#0, PCM device#1) >>> >>> To summarize, if we did this, I'd plan to remove all the kcontrols in ASoC card, and have the following in the USB card for an USB audio device that supports one USB stream: >>> >>> PCM Offload Playback Enable Stream#0 enable/disable >>> >>> PCM Offload Playback Enable Stream Mapping#0 0, 1 (ASoC card#0, PCM device#1) >> ... which is suggested here. >> >> Assuming these are read-only controls, we would need to know which PCM >> device on the USB card can be optimized with the use of which PCM device >> on the ASoC card. That means a set of three values. You would also want >> a number of streams to make the guesswork on controls less painful. > > OK, so now to just figuring out something that both you and Amadeusz can agree on before I put time implementing it. So I've implemented the "enable/disable" path that Amadeusz suggested, which is highlighted in my previous response, for evaluation purposes. The overall question is which layer should control the devices that will be offloaded. In my submissions up until now, the control was given to the ASoC platform card to determine which USB device to offload. Amadeusz mentioned that it might be beneficial to move the control to the USB SND devices, because what if the offloading is NOT backed by ASoC. (highlighted in [1]) However, IMO the current implementation assumes there is ASoC involved, which should mean that there is some platform "card" that is backing the offload path. Please let me know if my understanding is incorrect, @Amadeusz. I still fundamentally don't get why userspace would try and modify any controls, this makes the flows more complicated IMHO since you also have the PCM open/hw_params stages. I really think it'd be more than enough if the USB card exposed read-only values showing that offload is possible and which card/device to map to. Then userspace uses the ASoC PCM device and errors are handled at that level.
On 7/3/2024 10:49 AM, Pierre-Louis Bossart wrote: > > >>>>> There are really multiple layers to deal with >>>>> >>>>> a) is the controller able to support the offload path? IIRC this is >>>>> embedded in an obscure XHCI property, it would make sense to expose it >>>>> as a control, or component string, of the USB card. >>>> If a component string/tag is desired, I already have some way of doing that. I can add it to the USB card. >>>> >>>>> b) is there a companion card capable of dealing with the offload path? >>>>> Since the presence of this card may depend on driver probe, there should >>>>> be a control on the USB card. userspace could detect changes to this >>>>> control and detect if that path is or is no longer enabled. >>>> So currently, the "USB Offload Playback Capable Card" kcontrol (on the USB card) will determine if there is an offload path. However, this differs than what Amadeusz is suggesting, in that he wants a single kcontrol created for EACH USB card identified (per USB audio device), and a simple enable/disable control to determine if the offload path is enabled for that card/pcm stream. >>>> >>>> It would be a simpler approach for the userspace, and if the card that handles the offload card isn't present, then these enable/disable control will be set to "disabled," and even if users attempt to set the control, it won't go through. >>> Not following. Are you suggesting userspace would modify the >>> enable/disable status? >> >> Yes, this is the suggestion. One writeable kcontrol on the USB SND audio device that will control if that USB audio device is going to be offloaded. If the kcontrol reads back "enabled" (or 1) then userspace knows that the offload path is active. Else, if it reads "disabled" (or 0) after the attempt to set the kcontrol, then the offload path was unsuccessfully enabled, ie maybe due to no available offload streams. > > It's a bit over-engineered IMHO. > > My alternate suggestion is a read-only control reporting that offload is > possible. Then userspace attempts to open a PCM device on the ASoC card, > any failures due to resources would be handled at that point. > >>> I would just have a read-only control that reports what the hardware can >>> do and which other card can deal with offload. It's up to userspace to >>> select the offloaded PCM device or not. >>> >> That is what I have implemented in the previous patch series. One USB SND kcontrol within each USB audio device, which points to the ASoC platform card that supports offloading: >> >> "USB Offload Playback Capable Card" --> returns the card index to the ASoC platform card >> >> >From there the offloading control is all within the ASoC platform card. This is opposite to what Amaduesz suggested in that, the offload control of which USB device to offload should be within USB SND (not ASoC) > > It's very hard to follow, I don't understand what userspace needs to > 'control' - in the modify sense. What userspace needs is a place to read > from, and then select the PCM device and follow usual ALSA configuration > with hw_params. > From what I've seen I assumed that goal is to allow Offloading of specific stream from USB card. Otherwise I would say controls are not needed at all, as more user friendly solution is Offloading streams in order they are used as long as resources are available. >>>>> c) which PCM device is actually offloaded? This could be plural for some >>>>> implementations. The mapping between PCM devices exposed by the USB >>>>> card, and those exposed by the companion card, should be known to >>>>> userspace. I am not sure how this would be done though, a variable >>>>> number of controls is a sure way to confuse userspace. >>>> Expanding on Amadeusz's suggestion, my idea is to have an enable/disable kcontrol per USB stream. For example, one USB card could have multiple PCM devices (USB streams). So we would have something like: >>>> >>>> PCM Offload Playback Enable Stream#0 enable/disable >>>> >>>> PCM Offload Playback Enable Stream#1 enable/disable >>>> >>>> .... >>> are those read-only or not? >> >> No, writable and readable. > > The writable part introduces a complicated error handling, e.g. what > happens if you have an offloaded stream and then this control is changed > with amixer while streaming? > -EBUSY? and keep old value >>>> So we'd know which USB card and PCM device is selected for USB SND. However, I see what you're getting at in case there are multiple supported streams, because userspace needs to know which ASoC card/pcm combination corresponds to which USB device/combination. >>> I don't understand how this would help map the two parts? There's got to >>> be an additional mapping... >> It won't help with the mapping. That is something which we'd need to add, suggestion below. >>>> What do you think about having a USB card kcontrol to display the mapped ASoC card and PCM indexes? >>>> >>>> PCM Offload Playback Enable Stream Mapping#0 0, 1 (ASoC card#0, PCM device#1) >>>> >>>> To summarize, if we did this, I'd plan to remove all the kcontrols in ASoC card, and have the following in the USB card for an USB audio device that supports one USB stream: >>>> >>>> PCM Offload Playback Enable Stream#0 enable/disable >>>> >>>> PCM Offload Playback Enable Stream Mapping#0 0, 1 (ASoC card#0, PCM device#1) >>> ... which is suggested here. >>> >>> Assuming these are read-only controls, we would need to know which PCM >>> device on the USB card can be optimized with the use of which PCM device >>> on the ASoC card. That means a set of three values. You would also want >>> a number of streams to make the guesswork on controls less painful. >> >> OK, so now to just figuring out something that both you and Amadeusz can agree on before I put time implementing it. So I've implemented the "enable/disable" path that Amadeusz suggested, which is highlighted in my previous response, for evaluation purposes. The overall question is which layer should control the devices that will be offloaded. In my submissions up until now, the control was given to the ASoC platform card to determine which USB device to offload. Amadeusz mentioned that it might be beneficial to move the control to the USB SND devices, because what if the offloading is NOT backed by ASoC. (highlighted in [1]) However, IMO the current implementation assumes there is ASoC involved, which should mean that there is some platform "card" that is backing the offload path. Please let me know if my understanding is incorrect, @Amadeusz. > > I still fundamentally don't get why userspace would try and modify any > controls, this makes the flows more complicated IMHO since you also have > the PCM open/hw_params stages. > I really think it'd be more than enough if the USB card exposed > read-only values showing that offload is possible and which card/device > to map to. Then userspace uses the ASoC PCM device and errors are > handled at that level. I tend to agree, less values to change, less chance something breaks. However I think that there should be some way to disable Offload in case something doesn't work properly. (It doesn't have to be control, one can go with module parameter or sysfs toggle or something.)
>>>>>> There are really multiple layers to deal with >>>>>> >>>>>> a) is the controller able to support the offload path? IIRC this is >>>>>> embedded in an obscure XHCI property, it would make sense to >>>>>> expose it >>>>>> as a control, or component string, of the USB card. >>>>> If a component string/tag is desired, I already have some way of >>>>> doing that. I can add it to the USB card. >>>>> >>>>>> b) is there a companion card capable of dealing with the offload >>>>>> path? >>>>>> Since the presence of this card may depend on driver probe, there >>>>>> should >>>>>> be a control on the USB card. userspace could detect changes to this >>>>>> control and detect if that path is or is no longer enabled. >>>>> So currently, the "USB Offload Playback Capable Card" kcontrol (on >>>>> the USB card) will determine if there is an offload path. However, >>>>> this differs than what Amadeusz is suggesting, in that he wants a >>>>> single kcontrol created for EACH USB card identified (per USB audio >>>>> device), and a simple enable/disable control to determine if the >>>>> offload path is enabled for that card/pcm stream. >>>>> >>>>> It would be a simpler approach for the userspace, and if the card >>>>> that handles the offload card isn't present, then these >>>>> enable/disable control will be set to "disabled," and even if users >>>>> attempt to set the control, it won't go through. >>>> Not following. Are you suggesting userspace would modify the >>>> enable/disable status? >>> >>> Yes, this is the suggestion. One writeable kcontrol on the USB SND >>> audio device that will control if that USB audio device is going to >>> be offloaded. If the kcontrol reads back "enabled" (or 1) then >>> userspace knows that the offload path is active. Else, if it reads >>> "disabled" (or 0) after the attempt to set the kcontrol, then the >>> offload path was unsuccessfully enabled, ie maybe due to no available >>> offload streams. >> >> It's a bit over-engineered IMHO. >> >> My alternate suggestion is a read-only control reporting that offload is >> possible. Then userspace attempts to open a PCM device on the ASoC card, >> any failures due to resources would be handled at that point. >> >>>> I would just have a read-only control that reports what the hardware >>>> can >>>> do and which other card can deal with offload. It's up to userspace to >>>> select the offloaded PCM device or not. >>>> >>> That is what I have implemented in the previous patch series. One >>> USB SND kcontrol within each USB audio device, which points to the >>> ASoC platform card that supports offloading: >>> >>> "USB Offload Playback Capable Card" --> returns the card index to the >>> ASoC platform card >>> >>> >From there the offloading control is all within the ASoC platform >>> card. This is opposite to what Amaduesz suggested in that, the >>> offload control of which USB device to offload should be within USB >>> SND (not ASoC) >> >> It's very hard to follow, I don't understand what userspace needs to >> 'control' - in the modify sense. What userspace needs is a place to read >> from, and then select the PCM device and follow usual ALSA configuration >> with hw_params. >> > > From what I've seen I assumed that goal is to allow Offloading of > specific stream from USB card. Otherwise I would say controls are not > needed at all, as more user friendly solution is Offloading streams in > order they are used as long as resources are available. That's not great in terms of audio routing, you'd really want more rules or controlled behavior where the order in which devices are used does not matter. >>>>>> c) which PCM device is actually offloaded? This could be plural >>>>>> for some >>>>>> implementations. The mapping between PCM devices exposed by the USB >>>>>> card, and those exposed by the companion card, should be known to >>>>>> userspace. I am not sure how this would be done though, a variable >>>>>> number of controls is a sure way to confuse userspace. >>>>> Expanding on Amadeusz's suggestion, my idea is to have an >>>>> enable/disable kcontrol per USB stream. For example, one USB card >>>>> could have multiple PCM devices (USB streams). So we would have >>>>> something like: >>>>> >>>>> PCM Offload Playback Enable Stream#0 enable/disable >>>>> >>>>> PCM Offload Playback Enable Stream#1 enable/disable >>>>> >>>>> .... >>>> are those read-only or not? >>> >>> No, writable and readable. >> >> The writable part introduces a complicated error handling, e.g. what >> happens if you have an offloaded stream and then this control is changed >> with amixer while streaming? >> > > -EBUSY? and keep old value That would require a stop, fw_free, close, reopening of the non-offloaded device and restart. Best to avoid interrupting streams, if there are no resources that should be detected with an early fail during open/hw_params. Once the stream is flowing, it should not be interrupted - unless the USB device is removed of course. >>>>> So we'd know which USB card and PCM device is selected for USB >>>>> SND. However, I see what you're getting at in case there are >>>>> multiple supported streams, because userspace needs to know which >>>>> ASoC card/pcm combination corresponds to which USB device/combination. >>>> I don't understand how this would help map the two parts? There's >>>> got to >>>> be an additional mapping... >>> It won't help with the mapping. That is something which we'd need to >>> add, suggestion below. >>>>> What do you think about having a USB card kcontrol to display the >>>>> mapped ASoC card and PCM indexes? >>>>> >>>>> PCM Offload Playback Enable Stream Mapping#0 0, 1 (ASoC card#0, >>>>> PCM device#1) >>>>> >>>>> To summarize, if we did this, I'd plan to remove all the kcontrols >>>>> in ASoC card, and have the following in the USB card for an USB >>>>> audio device that supports one USB stream: >>>>> >>>>> PCM Offload Playback Enable Stream#0 enable/disable >>>>> >>>>> PCM Offload Playback Enable Stream Mapping#0 0, 1 (ASoC card#0, >>>>> PCM device#1) >>>> ... which is suggested here. >>>> >>>> Assuming these are read-only controls, we would need to know which PCM >>>> device on the USB card can be optimized with the use of which PCM >>>> device >>>> on the ASoC card. That means a set of three values. You would also want >>>> a number of streams to make the guesswork on controls less painful. >>> >>> OK, so now to just figuring out something that both you and Amadeusz >>> can agree on before I put time implementing it. So I've implemented >>> the "enable/disable" path that Amadeusz suggested, which is >>> highlighted in my previous response, for evaluation purposes. The >>> overall question is which layer should control the devices that will >>> be offloaded. In my submissions up until now, the control was given >>> to the ASoC platform card to determine which USB device to offload. >>> Amadeusz mentioned that it might be beneficial to move the control to >>> the USB SND devices, because what if the offloading is NOT backed by >>> ASoC. (highlighted in [1]) However, IMO the current implementation >>> assumes there is ASoC involved, which should mean that there is some >>> platform "card" that is backing the offload path. Please let me know >>> if my understanding is incorrect, @Amadeusz. >> >> I still fundamentally don't get why userspace would try and modify any >> controls, this makes the flows more complicated IMHO since you also have >> the PCM open/hw_params stages. >> I really think it'd be more than enough if the USB card exposed >> read-only values showing that offload is possible and which card/device >> to map to. Then userspace uses the ASoC PCM device and errors are >> handled at that level. > > I tend to agree, less values to change, less chance something breaks. > However I think that there should be some way to disable Offload in case > something doesn't work properly. (It doesn't have to be control, one can > go with module parameter or sysfs toggle or something.) Agree with this, a 'static' configuration to disable offload would be just fine. Module parameter is fine. A control to read if offload is possible and what the mapping is would be good. From there on, userspace may open the offloaded PCM and deal with all events (hw_params not supported, xruns, removal, etc).
On 7/3/2024 2:50 AM, Pierre-Louis Bossart wrote: > > >>>>>>> There are really multiple layers to deal with >>>>>>> >>>>>>> a) is the controller able to support the offload path? IIRC this is >>>>>>> embedded in an obscure XHCI property, it would make sense to >>>>>>> expose it >>>>>>> as a control, or component string, of the USB card. >>>>>> If a component string/tag is desired, I already have some way of >>>>>> doing that. I can add it to the USB card. >>>>>> >>>>>>> b) is there a companion card capable of dealing with the offload >>>>>>> path? >>>>>>> Since the presence of this card may depend on driver probe, there >>>>>>> should >>>>>>> be a control on the USB card. userspace could detect changes to this >>>>>>> control and detect if that path is or is no longer enabled. >>>>>> So currently, the "USB Offload Playback Capable Card" kcontrol (on >>>>>> the USB card) will determine if there is an offload path. However, >>>>>> this differs than what Amadeusz is suggesting, in that he wants a >>>>>> single kcontrol created for EACH USB card identified (per USB audio >>>>>> device), and a simple enable/disable control to determine if the >>>>>> offload path is enabled for that card/pcm stream. >>>>>> >>>>>> It would be a simpler approach for the userspace, and if the card >>>>>> that handles the offload card isn't present, then these >>>>>> enable/disable control will be set to "disabled," and even if users >>>>>> attempt to set the control, it won't go through. >>>>> Not following. Are you suggesting userspace would modify the >>>>> enable/disable status? >>>> Yes, this is the suggestion. One writeable kcontrol on the USB SND >>>> audio device that will control if that USB audio device is going to >>>> be offloaded. If the kcontrol reads back "enabled" (or 1) then >>>> userspace knows that the offload path is active. Else, if it reads >>>> "disabled" (or 0) after the attempt to set the kcontrol, then the >>>> offload path was unsuccessfully enabled, ie maybe due to no available >>>> offload streams. >>> It's a bit over-engineered IMHO. >>> >>> My alternate suggestion is a read-only control reporting that offload is >>> possible. Then userspace attempts to open a PCM device on the ASoC card, >>> any failures due to resources would be handled at that point. >>> >>>>> I would just have a read-only control that reports what the hardware >>>>> can >>>>> do and which other card can deal with offload. It's up to userspace to >>>>> select the offloaded PCM device or not. >>>>> >>>> That is what I have implemented in the previous patch series. One >>>> USB SND kcontrol within each USB audio device, which points to the >>>> ASoC platform card that supports offloading: >>>> >>>> "USB Offload Playback Capable Card" --> returns the card index to the >>>> ASoC platform card >>>> >>>> >From there the offloading control is all within the ASoC platform >>>> card. This is opposite to what Amaduesz suggested in that, the >>>> offload control of which USB device to offload should be within USB >>>> SND (not ASoC) >>> It's very hard to follow, I don't understand what userspace needs to >>> 'control' - in the modify sense. What userspace needs is a place to read >>> from, and then select the PCM device and follow usual ALSA configuration >>> with hw_params. >>> >> From what I've seen I assumed that goal is to allow Offloading of >> specific stream from USB card. Otherwise I would say controls are not >> needed at all, as more user friendly solution is Offloading streams in >> order they are used as long as resources are available. > That's not great in terms of audio routing, you'd really want more rules > or controlled behavior where the order in which devices are used does > not matter. So to clarify, when I mention to 'control' this is in case userspace does want to modify which USB device is offloaded. By default, the current behavior is that the latest USB headset is going to be enabled for offloading. IF say...userspace has some mechanism to switch which USB device is offloaded, they can do so using: USB Offload Playback Route Card Select USB Offload Playback Route PCM Select The above controls are read/write. read will fetch the current USB card and pcm indexes being enabled for offload. write will set the USB card and pcm index to enable offload on. Both reside on the ASoC card that is associated w/ the USB BE DAI link. >>>>>>> c) which PCM device is actually offloaded? This could be plural >>>>>>> for some >>>>>>> implementations. The mapping between PCM devices exposed by the USB >>>>>>> card, and those exposed by the companion card, should be known to >>>>>>> userspace. I am not sure how this would be done though, a variable >>>>>>> number of controls is a sure way to confuse userspace. >>>>>> Expanding on Amadeusz's suggestion, my idea is to have an >>>>>> enable/disable kcontrol per USB stream. For example, one USB card >>>>>> could have multiple PCM devices (USB streams). So we would have >>>>>> something like: >>>>>> >>>>>> PCM Offload Playback Enable Stream#0 enable/disable >>>>>> >>>>>> PCM Offload Playback Enable Stream#1 enable/disable >>>>>> >>>>>> .... >>>>> are those read-only or not? >>>> No, writable and readable. >>> The writable part introduces a complicated error handling, e.g. what >>> happens if you have an offloaded stream and then this control is changed >>> with amixer while streaming? >>> >> -EBUSY? and keep old value > That would require a stop, fw_free, close, reopening of the > non-offloaded device and restart. Best to avoid interrupting streams, if > there are no resources that should be detected with an early fail during > open/hw_params. Once the stream is flowing, it should not be interrupted > - unless the USB device is removed of course. I agree. We don't want to interrupt streams if some reason someone tries to switch the offload path. If there is already an active offload stream, then any attempt to switch/control which USB card/pcm stream is offload is going to be ignored. > >>>>>> So we'd know which USB card and PCM device is selected for USB >>>>>> SND. However, I see what you're getting at in case there are >>>>>> multiple supported streams, because userspace needs to know which >>>>>> ASoC card/pcm combination corresponds to which USB device/combination. >>>>> I don't understand how this would help map the two parts? There's >>>>> got to >>>>> be an additional mapping... >>>> It won't help with the mapping. That is something which we'd need to >>>> add, suggestion below. >>>>>> What do you think about having a USB card kcontrol to display the >>>>>> mapped ASoC card and PCM indexes? >>>>>> >>>>>> PCM Offload Playback Enable Stream Mapping#0 0, 1 (ASoC card#0, >>>>>> PCM device#1) >>>>>> >>>>>> To summarize, if we did this, I'd plan to remove all the kcontrols >>>>>> in ASoC card, and have the following in the USB card for an USB >>>>>> audio device that supports one USB stream: >>>>>> >>>>>> PCM Offload Playback Enable Stream#0 enable/disable >>>>>> >>>>>> PCM Offload Playback Enable Stream Mapping#0 0, 1 (ASoC card#0, >>>>>> PCM device#1) >>>>> ... which is suggested here. >>>>> >>>>> Assuming these are read-only controls, we would need to know which PCM >>>>> device on the USB card can be optimized with the use of which PCM >>>>> device >>>>> on the ASoC card. That means a set of three values. You would also want >>>>> a number of streams to make the guesswork on controls less painful. >>>> OK, so now to just figuring out something that both you and Amadeusz >>>> can agree on before I put time implementing it. So I've implemented >>>> the "enable/disable" path that Amadeusz suggested, which is >>>> highlighted in my previous response, for evaluation purposes. The >>>> overall question is which layer should control the devices that will >>>> be offloaded. In my submissions up until now, the control was given >>>> to the ASoC platform card to determine which USB device to offload. >>>> Amadeusz mentioned that it might be beneficial to move the control to >>>> the USB SND devices, because what if the offloading is NOT backed by >>>> ASoC. (highlighted in [1]) However, IMO the current implementation >>>> assumes there is ASoC involved, which should mean that there is some >>>> platform "card" that is backing the offload path. Please let me know >>>> if my understanding is incorrect, @Amadeusz. >>> I still fundamentally don't get why userspace would try and modify any >>> controls, this makes the flows more complicated IMHO since you also have >>> the PCM open/hw_params stages. >>> I really think it'd be more than enough if the USB card exposed >>> read-only values showing that offload is possible and which card/device >>> to map to. Then userspace uses the ASoC PCM device and errors are >>> handled at that level. Just so I understand...is it really desired that userspace doesn't have the flexibility to choose which USB device is offloaded? I know it complicates what needs to be done, but it could be just an additional feature that can be added later on. Again, by default, we select the last USB headset plugged in to be enabled for offload by default. >> I tend to agree, less values to change, less chance something breaks. >> However I think that there should be some way to disable Offload in case >> something doesn't work properly. (It doesn't have to be control, one can >> go with module parameter or sysfs toggle or something.) > Agree with this, a 'static' configuration to disable offload would be > just fine. Module parameter is fine. > > A control to read if offload is possible and what the mapping is would > be good. From there on, userspace may open the offloaded PCM and deal > with all events (hw_params not supported, xruns, removal, etc). I can look into adding this mapping part to the USB SND card, so we know which ASoC card/pcm devices are associated to the USB SND device for offloading. Thank you for the discussions and input! Thanks Wesley Cheng
On 7/4/2024 12:05 AM, Wesley Cheng wrote: > > On 7/3/2024 2:50 AM, Pierre-Louis Bossart wrote: >> >> >>>>>>>> There are really multiple layers to deal with >>>>>>>> >>>>>>>> a) is the controller able to support the offload path? IIRC this is >>>>>>>> embedded in an obscure XHCI property, it would make sense to >>>>>>>> expose it >>>>>>>> as a control, or component string, of the USB card. >>>>>>> If a component string/tag is desired, I already have some way of >>>>>>> doing that. I can add it to the USB card. >>>>>>> >>>>>>>> b) is there a companion card capable of dealing with the offload >>>>>>>> path? >>>>>>>> Since the presence of this card may depend on driver probe, there >>>>>>>> should >>>>>>>> be a control on the USB card. userspace could detect changes to this >>>>>>>> control and detect if that path is or is no longer enabled. >>>>>>> So currently, the "USB Offload Playback Capable Card" kcontrol (on >>>>>>> the USB card) will determine if there is an offload path. However, >>>>>>> this differs than what Amadeusz is suggesting, in that he wants a >>>>>>> single kcontrol created for EACH USB card identified (per USB audio >>>>>>> device), and a simple enable/disable control to determine if the >>>>>>> offload path is enabled for that card/pcm stream. >>>>>>> >>>>>>> It would be a simpler approach for the userspace, and if the card >>>>>>> that handles the offload card isn't present, then these >>>>>>> enable/disable control will be set to "disabled," and even if users >>>>>>> attempt to set the control, it won't go through. >>>>>> Not following. Are you suggesting userspace would modify the >>>>>> enable/disable status? >>>>> Yes, this is the suggestion. One writeable kcontrol on the USB SND >>>>> audio device that will control if that USB audio device is going to >>>>> be offloaded. If the kcontrol reads back "enabled" (or 1) then >>>>> userspace knows that the offload path is active. Else, if it reads >>>>> "disabled" (or 0) after the attempt to set the kcontrol, then the >>>>> offload path was unsuccessfully enabled, ie maybe due to no available >>>>> offload streams. >>>> It's a bit over-engineered IMHO. >>>> >>>> My alternate suggestion is a read-only control reporting that offload is >>>> possible. Then userspace attempts to open a PCM device on the ASoC card, >>>> any failures due to resources would be handled at that point. >>>> >>>>>> I would just have a read-only control that reports what the hardware >>>>>> can >>>>>> do and which other card can deal with offload. It's up to userspace to >>>>>> select the offloaded PCM device or not. >>>>>> >>>>> That is what I have implemented in the previous patch series. One >>>>> USB SND kcontrol within each USB audio device, which points to the >>>>> ASoC platform card that supports offloading: >>>>> >>>>> "USB Offload Playback Capable Card" --> returns the card index to the >>>>> ASoC platform card >>>>> >>>>> >From there the offloading control is all within the ASoC platform >>>>> card. This is opposite to what Amaduesz suggested in that, the >>>>> offload control of which USB device to offload should be within USB >>>>> SND (not ASoC) >>>> It's very hard to follow, I don't understand what userspace needs to >>>> 'control' - in the modify sense. What userspace needs is a place to read >>>> from, and then select the PCM device and follow usual ALSA configuration >>>> with hw_params. >>>> >>> From what I've seen I assumed that goal is to allow Offloading of >>> specific stream from USB card. Otherwise I would say controls are not >>> needed at all, as more user friendly solution is Offloading streams in >>> order they are used as long as resources are available. >> That's not great in terms of audio routing, you'd really want more rules >> or controlled behavior where the order in which devices are used does >> not matter. > > So to clarify, when I mention to 'control' this is in case userspace does want to modify which USB device is offloaded. By default, the current behavior is that the latest USB headset is going to be enabled for offloading. IF say...userspace has some mechanism to switch which USB device is offloaded, they can do so using: > > USB Offload Playback Route Card Select > > USB Offload Playback Route PCM Select > > The above controls are read/write. read will fetch the current USB card and pcm indexes being enabled for offload. write will set the USB card and pcm index to enable offload on. Both reside on the ASoC card that is associated w/ the USB BE DAI link. > >>>>>>>> c) which PCM device is actually offloaded? This could be plural >>>>>>>> for some >>>>>>>> implementations. The mapping between PCM devices exposed by the USB >>>>>>>> card, and those exposed by the companion card, should be known to >>>>>>>> userspace. I am not sure how this would be done though, a variable >>>>>>>> number of controls is a sure way to confuse userspace. >>>>>>> Expanding on Amadeusz's suggestion, my idea is to have an >>>>>>> enable/disable kcontrol per USB stream. For example, one USB card >>>>>>> could have multiple PCM devices (USB streams). So we would have >>>>>>> something like: >>>>>>> >>>>>>> PCM Offload Playback Enable Stream#0 enable/disable >>>>>>> >>>>>>> PCM Offload Playback Enable Stream#1 enable/disable >>>>>>> >>>>>>> .... >>>>>> are those read-only or not? >>>>> No, writable and readable. >>>> The writable part introduces a complicated error handling, e.g. what >>>> happens if you have an offloaded stream and then this control is changed >>>> with amixer while streaming? >>>> >>> -EBUSY? and keep old value >> That would require a stop, fw_free, close, reopening of the >> non-offloaded device and restart. Best to avoid interrupting streams, if >> there are no resources that should be detected with an early fail during >> open/hw_params. Once the stream is flowing, it should not be interrupted >> - unless the USB device is removed of course. > I agree. We don't want to interrupt streams if some reason someone tries to switch the offload path. If there is already an active offload stream, then any attempt to switch/control which USB card/pcm stream is offload is going to be ignored. >> >>>>>>> So we'd know which USB card and PCM device is selected for USB >>>>>>> SND. However, I see what you're getting at in case there are >>>>>>> multiple supported streams, because userspace needs to know which >>>>>>> ASoC card/pcm combination corresponds to which USB device/combination. >>>>>> I don't understand how this would help map the two parts? There's >>>>>> got to >>>>>> be an additional mapping... >>>>> It won't help with the mapping. That is something which we'd need to >>>>> add, suggestion below. >>>>>>> What do you think about having a USB card kcontrol to display the >>>>>>> mapped ASoC card and PCM indexes? >>>>>>> >>>>>>> PCM Offload Playback Enable Stream Mapping#0 0, 1 (ASoC card#0, >>>>>>> PCM device#1) >>>>>>> >>>>>>> To summarize, if we did this, I'd plan to remove all the kcontrols >>>>>>> in ASoC card, and have the following in the USB card for an USB >>>>>>> audio device that supports one USB stream: >>>>>>> >>>>>>> PCM Offload Playback Enable Stream#0 enable/disable >>>>>>> >>>>>>> PCM Offload Playback Enable Stream Mapping#0 0, 1 (ASoC card#0, >>>>>>> PCM device#1) >>>>>> ... which is suggested here. >>>>>> >>>>>> Assuming these are read-only controls, we would need to know which PCM >>>>>> device on the USB card can be optimized with the use of which PCM >>>>>> device >>>>>> on the ASoC card. That means a set of three values. You would also want >>>>>> a number of streams to make the guesswork on controls less painful. >>>>> OK, so now to just figuring out something that both you and Amadeusz >>>>> can agree on before I put time implementing it. So I've implemented >>>>> the "enable/disable" path that Amadeusz suggested, which is >>>>> highlighted in my previous response, for evaluation purposes. The >>>>> overall question is which layer should control the devices that will >>>>> be offloaded. In my submissions up until now, the control was given >>>>> to the ASoC platform card to determine which USB device to offload. >>>>> Amadeusz mentioned that it might be beneficial to move the control to >>>>> the USB SND devices, because what if the offloading is NOT backed by >>>>> ASoC. (highlighted in [1]) However, IMO the current implementation >>>>> assumes there is ASoC involved, which should mean that there is some >>>>> platform "card" that is backing the offload path. Please let me know >>>>> if my understanding is incorrect, @Amadeusz. >>>> I still fundamentally don't get why userspace would try and modify any >>>> controls, this makes the flows more complicated IMHO since you also have >>>> the PCM open/hw_params stages. >>>> I really think it'd be more than enough if the USB card exposed >>>> read-only values showing that offload is possible and which card/device >>>> to map to. Then userspace uses the ASoC PCM device and errors are >>>> handled at that level. > Just so I understand...is it really desired that userspace doesn't have the flexibility to choose which USB device is offloaded? I know it complicates what needs to be done, but it could be just an additional feature that can be added later on. Again, by default, we select the last USB headset plugged in to be enabled for offload by default. If it chooses endpoint by itself perhaps you can send patch set without controls first? This has added benefit of less patches in series, making it easier to review and it won't block whole patch set by discussion on controls feature. Controls can be added in followup series. >>> I tend to agree, less values to change, less chance something breaks. >>> However I think that there should be some way to disable Offload in case >>> something doesn't work properly. (It doesn't have to be control, one can >>> go with module parameter or sysfs toggle or something.) >> Agree with this, a 'static' configuration to disable offload would be >> just fine. Module parameter is fine. >> >> A control to read if offload is possible and what the mapping is would >> be good. From there on, userspace may open the offloaded PCM and deal >> with all events (hw_params not supported, xruns, removal, etc). > > I can look into adding this mapping part to the USB SND card, so we know which ASoC card/pcm devices are associated to the USB SND device for offloading. Thank you for the discussions and input! > > Thanks > > Wesley Cheng > >
>> Just so I understand...is it really desired that userspace doesn't >> have the flexibility to choose which USB device is offloaded? I know >> it complicates what needs to be done, but it could be just an >> additional feature that can be added later on. Again, by default, we >> select the last USB headset plugged in to be enabled for offload by >> default. > > If it chooses endpoint by itself perhaps you can send patch set without > controls first? This has added benefit of less patches in series, making > it easier to review and it won't block whole patch set by discussion on > controls feature. Controls can be added in followup series. We do need read-only controls for userspace to know that offload is possible and which card/device to use. That can be done in a first step assuming there's a single device plugged-in. Dealing with multiple devices and defining rules or configuration options to select the offloaded device is a second-level problem. In most cases the only thing that will be offloaded is a headset anyways, so the selection could be rather static based on a vendor/system ID, all other USB devices would be ignored.
On 7/4/2024 4:25 AM, Pierre-Louis Bossart wrote: >>> Just so I understand...is it really desired that userspace doesn't >>> have the flexibility to choose which USB device is offloaded? I know >>> it complicates what needs to be done, but it could be just an >>> additional feature that can be added later on. Again, by default, we >>> select the last USB headset plugged in to be enabled for offload by >>> default. >> If it chooses endpoint by itself perhaps you can send patch set without >> controls first? This has added benefit of less patches in series, making >> it easier to review and it won't block whole patch set by discussion on >> controls feature. Controls can be added in followup series. > We do need read-only controls for userspace to know that offload is > possible and which card/device to use. That can be done in a first step > assuming there's a single device plugged-in. I agree, some kcontrol need to be present to at least determine: 1. USB SND device is offload capable (ASoC card and PCM index)- Fetches associated (mapped) ASoC platform card and PCM index (read only) 2. ASoC platform card offload status - Current offload status (read only) Those would be the minimum kcontrols we could have at this time. I will remove the device selection part, and leave that for future discussions. Does this sound good, Amadeusz/Pierre? > Dealing with multiple devices and defining rules or configuration > options to select the offloaded device is a second-level problem. > > In most cases the only thing that will be offloaded is a headset > anyways, so the selection could be rather static based on a > vendor/system ID, all other USB devices would be ignored. If the USB SND offload driver (ie qc_audio_offload) isn't compiled in, then it would be disabled. Do we need some over-arching mechanism to disable the offload functionality? Although, one thing I can see if I can add is some device classification within the USB offload vendor driver. Thanks Wesley Cheng
Hi Pierre/Amadeusz, On 7/8/2024 4:16 PM, Wesley Cheng wrote: > On 7/4/2024 4:25 AM, Pierre-Louis Bossart wrote: >>>> Just so I understand...is it really desired that userspace doesn't >>>> have the flexibility to choose which USB device is offloaded? I know >>>> it complicates what needs to be done, but it could be just an >>>> additional feature that can be added later on. Again, by default, we >>>> select the last USB headset plugged in to be enabled for offload by >>>> default. >>> If it chooses endpoint by itself perhaps you can send patch set without >>> controls first? This has added benefit of less patches in series, making >>> it easier to review and it won't block whole patch set by discussion on >>> controls feature. Controls can be added in followup series. >> We do need read-only controls for userspace to know that offload is >> possible and which card/device to use. That can be done in a first step >> assuming there's a single device plugged-in. > I agree, some kcontrol need to be present to at least determine: > > 1. USB SND device is offload capable (ASoC card and PCM index)- Fetches associated (mapped) ASoC platform card and PCM index (read only) > > 2. ASoC platform card offload status - Current offload status (read only) > > Those would be the minimum kcontrols we could have at this time. I will remove the device selection part, and leave that for future discussions. Does this sound good, Amadeusz/Pierre? So I reworked the series a bit with respects to the kcontrols that we had, and I simplified it for the next submission. I went ahead and just have a read only kcontrol residing in the USB SND device and will implement #1 above: / # tinymix -D 1 contents Number of controls: 9 ctl type num name value 0 INT 2 Capture Channel Map 0, 0 (range 0->36) 1 INT 2 Playback Channel Map 0, 0 (range 0->36) 2 BOOL 1 Headset Capture Switch On 3 INT 1 Headset Capture Volume 10 (range 0->13) 4 BOOL 1 Sidetone Playback Switch On 5 INT 1 Sidetone Playback Volume 4096 (range 0->8192) 6 BOOL 1 Headset Playback Switch On 7 INT 2 Headset Playback Volume 20, 20 (range 0->24) 8 INT 2 USB Offload Playback Route PCM#0 0, 0 (range -1->255) If there is an available audio offload path, then the value will show the card and pcm index that it is mapped to. That way the application will know which card/pcm device to open from there. In the above example, the offload path is mapped to card#0 pcm#0. If there is no offload path available, it will show -1, -1. For now, I removed the control that allows for explicit selection of which USB card and PCM device to offload, and will take this up on a separate series as we see fit. The codebase I have now will select the last USB headset plugged in for offloading. Will clean up the changes and submit a new revision with the other feedback included as well. Thanks Wesley Cheng > >> Dealing with multiple devices and defining rules or configuration >> options to select the offloaded device is a second-level problem. >> >> In most cases the only thing that will be offloaded is a headset >> anyways, so the selection could be rather static based on a >> vendor/system ID, all other USB devices would be ignored. > If the USB SND offload driver (ie qc_audio_offload) isn't compiled in, then it would be disabled. Do we need some over-arching mechanism to disable the offload functionality? Although, one thing I can see if I can add is some device classification within the USB offload vendor driver. > > Thanks > > Wesley Cheng >
diff --git a/Documentation/sound/soc/index.rst b/Documentation/sound/soc/index.rst index e57df2dab2fd..8bed8f8f48da 100644 --- a/Documentation/sound/soc/index.rst +++ b/Documentation/sound/soc/index.rst @@ -18,3 +18,4 @@ The documentation is spilt into the following sections:- jack dpcm codec-to-codec + usb diff --git a/Documentation/sound/soc/usb.rst b/Documentation/sound/soc/usb.rst new file mode 100644 index 000000000000..c5175af52154 --- /dev/null +++ b/Documentation/sound/soc/usb.rst @@ -0,0 +1,603 @@ +================ +ASoC USB support +================ + +Overview +======== +In order to leverage the existing USB sound device support in ALSA, the +introduction of the ASoC USB APIs, allow for the entities to communicate +with one another. + +One potential use case would be to support USB audio offloading, which is +an implementation that allows for an external DSP on the SoC to handle the +transfer of audio data over the USB bus. This would let the main +processor to stay in lower power modes for longer durations. The following +is an example design of how the ASoC and ALSA pieces can be connected +together to achieve this: + +:: + + USB | ASoC + | _________________________ + | | ASoC Platform card | + | |_________________________| + | | | + | ___V____ ____V____ + | |ASoC BE | |ASoC FE | + | |DAI LNK | |DAI LNK | + | |________| |_________| + | ^ ^ ^ + | | |________| + | ___V____ | + | |SOC-USB | | + ________ ________ | | | + |USB SND |<--->|USBSND |<------------>|________| | + |(card.c)| |offld |<---------- | + |________| |________|___ | | | + ^ ^ | | | ____________V_________ + | | | | | |IPC | + __ V_______________V_____ | | | |______________________| + |USB SND (endpoint.c) | | | | ^ + |_________________________| | | | | + ^ | | | ___________V___________ + | | | |->|audio DSP | + ___________V_____________ | | |_______________________| + |XHCI HCD |<- | + |_________________________| | + + +SOC USB driver +============== +Structures +---------- +``struct snd_soc_usb`` + + - ``list``: list head for SND SOC struct list + - ``dev``: USB backend device reference + - ``component``: reference to ASoC component + - ``active_list``: active sessions + - ``num_supported_streams``: number of supported concurrent sessions + - ``connection_status_cb``: callback to notify connection events + - ``put_offload_dev``: callback to select USB sound card/PCM device + - ``get_offload_dev``: callback to fetch selected USB sound card/PCM device + - ``priv_data``: driver data + +The snd_soc_usb structure can be referenced using the ASoC platform card +device, or a USB device (udev->dev). This is created by the ASoC BE DAI +link, and the USB sound entity will be able to pass information to the +ASoC BE DAI link using this structure. + +``struct snd_soc_usb_device`` + + - ``card_idx``: sound card index associated with USB device + - ``chip_idx``: USB sound chip array index + - ``num_playback``: number of playback streams + - ``num_capture``: number of capture streams + +The struct snd_soc_usb_device is created by the USB sound offload driver. +This will carry basic parameters/limitations that will be used to +determine the possible offloading paths for this USB audio device. + +``struct snd_soc_usb_session`` + + - ``active_card_idx``: active offloaded sound card + - ``active_pcm_idx``: active offloaded PCM device + - ``state``: USB BE DAI link PCM state + +The struct snd_soc_usb_session tracks the current offloading state for a +particular card and PCM combination. This structure is carried/saved as +part of the active_list within struct snd_soc_usb. + +The number of entities in the active list corresponds to the number of +snd_soc_usb_session structures that are allocated. This is controlled +by the num_supported_streams that is reported as part of the SOC USB +structure creation. + +Functions +--------- +.. code-block:: rst + + const char *snd_soc_usb_get_components_tag(bool playback); +.. + + - ``playback``: direction of audio stream + +**snd_soc_usb_get_components_tag()** returns the tag used for describing if USB +offloading is supported for appending to the ASoC platform card's components +string. + +Returns a tag based on the direction of the audio stream. + +.. code-block:: rst + + int snd_soc_usb_find_format(int card_idx, struct snd_pcm_hw_params *params, + int direction) +.. + + - ``card_idx``: the index into the USB sound chip array. + - ``params``: Requested PCM parameters from the USB DPCM BE DAI link + - ``direction``: capture or playback + +**snd_soc_usb_find_format()** ensures that the requested audio profile being +requested by the external DSP is supported by the USB device. + +Returns 0 on success, and -EOPNOTSUPP on failure. + +.. code-block:: rst + + int snd_soc_usb_connect(struct device *usbdev, struct snd_soc_usb_device *sdev) +.. + + - ``usbdev``: the usb device that was discovered + - ``sdev``: capabilities of the device + +**snd_soc_usb_connect()** notifies the ASoC USB DCPM BE DAI link of a USB +audio device detection. This can be utilized in the BE DAI +driver to keep track of available USB audio devices. This is intended +to be called by the USB offload driver residing in USB SND. + +Returns 0 on success, negative error code on failure. + +.. code-block:: rst + + int snd_soc_usb_disconnect(struct device *usbdev, struct snd_soc_usb_device *sdev) +.. + + - ``usbdev``: the usb device that was removed + - ``sdev``: capabilities to free + +**snd_soc_usb_disconnect()** notifies the ASoC USB DCPM BE DAI link of a USB +audio device removal. This is intended to be called by the USB offload +driver that resides in USB SND. + +.. code-block:: rst + + void *snd_soc_usb_find_priv_data(struct device *usbdev) +.. + + - ``usbdev``: the usb device to reference to find private data + +**snd_soc_usb_find_priv_data()** fetches the private data saved to the SOC USB +device. + +Returns pointer to priv_data on success, NULL on failure. + +.. code-block:: rst + + int snd_soc_usb_device_offload_available(struct device *dev) +.. + + - ``dev``: the device to find in SOC USB + +**snd_soc_usb_device_offload_available()** fetch the sound card number associated +to the USB BE DAI link. + +Returns a valid sound card index on success, negative on failure. + +.. code-block:: rst + + int snd_soc_usb_prepare_session(struct snd_soc_usb *usb, int card_idx, int pcm_idx); +.. + + - ``usb``: SOC USB device + - ``card_idx``: USB sound card index + - ``pcm_idx``: USB PCM device index + +**snd_soc_usb_prepare_session()** populates active_list with a 'struct +snd_soc_usb_session.' This will move the session into the SND_SOC_USB_PREPARED +state. State updates will always start here. + +Returns index to active_list on success, -EBUSY on failure. + +.. code-block:: rst + + int snd_soc_usb_shutdown_session(struct snd_soc_usb *usb, int session_id); +.. + + - ``usb``: SOC USB device + - ``session_id``: session id returned by **snd_soc_usb_prepare_session()** + +**snd_soc_usb_shutdown_session()** frees up a slot in active_list, which signals +that there is no longer an active offloading device. This allows for another +session to be started. + +Returns 0 on success, -EINVAL if session index is invalid. + +.. code-block:: rst + + int snd_soc_usb_set_session_state(struct snd_soc_usb *usb, int session_id, + enum snd_soc_usb_dai_state state); +.. + + - ``usb``: SOC USB device + - ``session_id``: session id returned by **snd_soc_usb_prepare_session()** + - ``state``: state to move into + +**snd_soc_usb_set_session_state()** moves an offloading session to the desired +state. + +.. code-block:: rst + + int snd_soc_usb_setup_offload_jack(struct snd_soc_component *component, + struct snd_soc_jack *jack) +.. + + - ``component``: ASoC component to add the jack + - ``jack``: ASoC sound jack to add + +**snd_soc_usb_setup_offload_jack()** is a helper to add a sound jack control to +the platform sound card. This will allow for consistent naming to be used on +designs that support USB audio offloading. + +Returns 0 on success, negative otherwise. + +.. code-block:: rst + + struct snd_soc_usb *snd_soc_usb_allocate_port(struct snd_soc_component *component, + int num_supported_streams, void *data); +.. + + - ``component``: DPCM BE DAI link component + - ``num_supported_streams``: number of active streams supported by external DSP + - ``data``: private data + +**snd_soc_usb_allocate_port()** allocates a SOC USB device and populates standard +parameters that is used for further operations. + +Returns a pointer to struct soc_usb on success, negative on error. + +.. code-block:: rst + + void snd_soc_usb_free_port(struct snd_soc_usb *usb); +.. + + - ``usb``: SOC USB device to free + +**snd_soc_usb_free_port()** frees a SOC USB device. + +.. code-block:: rst + + void snd_soc_usb_add_port(struct snd_soc_usb *usb); +.. + + - ``usb``: SOC USB device to add + +**snd_soc_usb_add_port()** add an allocated SOC USB device to the SOC USB framework. +Once added, this device can be referenced by further operations. + +.. code-block:: rst + + void snd_soc_usb_remove_port(struct snd_soc_usb *usb); +.. + + - ``usb``: SOC USB device to remove + +**snd_soc_usb_remove_port()** removes a SOC USB device from the SOC USB framework. +After removing a device, any SOC USB operations would not be able to reference the +device removed. + +How to Register to SOC USB +-------------------------- +The ASoC DPCM USB BE DAI link is the entity responsible for allocating and +registering the SOC USB device on the component bind. Likewise, it will +also be responsible for freeing the allocated resources. An example can +be shown below: + +.. code-block:: rst + + static int q6usb_component_probe(struct snd_soc_component *component) + { + ... + data->usb = snd_soc_usb_allocate_port(component, 1, &data->priv); + if (!data->usb) + return -ENOMEM; + + usb->connection_status_cb = q6usb_alsa_connection_cb; + + ret = snd_soc_usb_add_port(usb); + if (ret < 0) { + dev_err(component->dev, "failed to add usb port\n"); + goto free_usb; + } + ... + } + + static void q6usb_component_remove(struct snd_soc_component *component) + { + ... + snd_soc_usb_remove_port(data->usb); + snd_soc_usb_free_port(data->usb); + } + + static const struct snd_soc_component_driver q6usb_dai_component = { + .probe = q6usb_component_probe, + .remove = q6usb_component_remove, + .name = "q6usb-dai-component", + ... + }; +.. + +BE DAI links can pass along vendor specific information as part of the +call to allocate the SOC USB device. This will allow any BE DAI link +parameters or settings to be accessed by the USB offload driver that +resides in USB SND. + +USB Audio Device Connection Flow +-------------------------------- +USB devices can be hotplugged into the USB root hub at any point in time. +The BE DAI link should be aware of the current state of the physical USB +port, i.e. if there are any USB devices with audio interface(s) connected. +The following callback can be used to notify the BE DAI link of any change: + + **connection_status_cb()** + +This is called whenever there is a USB SND interface bind or remove event, +using snd_soc_usb_connect() or snd_soc_usb_disconnect(): + +.. code-block:: rst + + static void qc_usb_audio_offload_probe(struct snd_usb_audio *chip) + { + ... + snd_soc_usb_connect(usb_get_usb_backend(udev), sdev); + ... + } + + static void qc_usb_audio_offload_disconnect(struct snd_usb_audio *chip) + { + ... + snd_soc_usb_disconnect(usb_get_usb_backend(chip->dev), dev->sdev); + ... + } +.. + +In order to account for conditions where driver or device existence is +not guaranteed, USB SND exposes snd_usb_rediscover_devices() to resend the +connect events for any identified USB audio interfaces. Consider the +the following situtation: + + **usb_audio_probe()** + | --> USB audio streams allocated and saved to usb_chip[] + | --> Propagate connect event to USB offload driver in USB SND + | --> **snd_soc_usb_connect()** exits as USB BE DAI link is not ready + + BE DAI link component probe + | --> DAI link is probed and SOC USB port is allocated + | --> The USB audio device connect event is missed + +To ensure connection events are not missed, **snd_usb_rediscover_devices()** +is executed when the SOC USB device is registered. Now, when the BE DAI +link component probe occurs, the following highlights the sequence: + + BE DAI link component probe + | --> DAI link is probed and SOC USB port is allocated + | --> SOC USB device added, and **snd_usb_rediscover_devices()** runs + + **snd_usb_rediscover_devices()** + | --> Traverses through usb_chip[] and for non-NULL entries issue + | **connection_status_cb()** + +In the case where the USB offload driver is unbounded, while USB SND is +ready, the **snd_usb_rediscover_devices()** is called during module init. +This allows for the offloading path to also be enabled with the following +flow: + + **usb_audio_probe()** + | --> USB audio streams allocated and saved to usb_chip[] + | --> Propagate connect event to USB offload driver in USB SND + | --> USB offload driver **NOT** ready! + + BE DAI link component probe + | --> DAI link is probed and SOC USB port is allocated + | --> No USB connect event due to missing USB offload driver + + USB offload driver probe + | --> **qc_usb_audio_offload_init()** + | --> Calls **snd_usb_rediscover_devices()** to notify of devices + +SOC USB and USB Sound Kcontrols +=============================== +Details +------- +SOC USB and USB sound expose a set of SND kcontrols for applications to select +and fetch the current offloading status for the ASoC platform sound card. Kcontrols +are split between two layers: + + - USB sound - Notifies the sound card number for the ASoC platform sound + card that it is registered to for supporting audio offload. + + - SOC USB - Maintains the current status of the offload path, and device + (USB sound card and PCM device) information. This would be the main + card that applications can read to determine offloading capabilities. + +Implementation +-------------- + +**Example:** + + **Sound Cards**: + + :: + + 0 [SM8250MTPWCD938]: sm8250 - SM8250-MTP-WCD9380-WSA8810-VA-D + SM8250-MTP-WCD9380-WSA8810-VA-DMIC + 1 [C320M ]: USB-Audio - Plantronics C320-M + Plantronics Plantronics C320-M at usb-xhci-hcd.1.auto-1, full speed + + + **Platform Sound Card** - card#0: + + :: + + USB Offload Playback Route Card Select 1 (range -1->32) + USB Offload Playback Route PCM Select 0 (range -1->255) + USB Offload Playback Route Card Status -1 (range -1->32) + USB Offload Playback Route PCM Status -1 (range -1->255) + + + **USB Sound Card** - card#1: + + :: + + USB Offload Playback Capable Card 0 (range -1->32) + + +The platform sound card(card#0) kcontrols are created as part of adding the SOC +USB device using **snd_soc_usb_add_port()**. The following kcontrols are defined +as: + + - ``USB Offload Playback Route Card Status`` **(R)**: USB sound card device index + that defines which USB SND resources are currently offloaded. If -1 is seen, it + signifies that offload is not active. + - ``USB Offload Playback Route PCM Status`` **(R)**: USB PCM device index + that defines which USB SND resources are currently offloaded. If -1 is seen, it + signifies that offload is not active. + - ``USB Offload Playback Route Card Select`` **(R/W)**: USB sound card index which + selects the USB device to initiate offloading on. If no value is written to the + kcontrol, then the last USB device discovered card index will be chosen. + - ``USB Offload Playback Route PCM Select`` **(R/W)**: USB PCM index which selects + the USB device to initiate offloading on. If no value is written to the + kcontrol, then the last USB device discovered PCM zero index will be chosen. + +The USB sound card(card#1) kcontrols are created as USB audio devices are plugged +into the physical USB port and enumerated. The kcontrols are defined as: + + - ``USB Offload Playback Capable Card`` **(R)**: Provides the sound card + number/index that supports USB offloading. Further/follow up queries about + the current offload state can be handled by reading the offload status + kcontrol exposed by the platform card. + +USB Offload Playback Route Select Kcontrol +------------------------------------------ +In order to allow for vendor specific implementations on audio offloading device +selection, the SOC USB layer exposes the following: + +.. code-block:: rst + + int (*put_offload_dev)(struct snd_kcontrol *kcontrol, + struct snd_ctl_elem_value *ucontrol); + int (*get_offload_dev)(struct snd_kcontrol *kcontrol, + struct snd_ctl_elem_value *ucontrol); +.. + +These are specific for the **USB Offload Playback Route Select** kcontrol. + +When users issue get/put calls to the kcontrol, the registered SOC USB callbacks +will execute the registered function calls to the DPCM BE DAI link. + +**Callback Registration:** + +.. code-block:: rst + + static int q6usb_component_probe(struct snd_soc_component *component) + { + ... + usb = snd_soc_usb_allocate_port(component, 1, &data->priv); + if (IS_ERR(usb)) + return -ENOMEM; + + usb->connection_status_cb = q6usb_alsa_connection_cb; + usb->put_offload_dev = q6usb_put_offload_dev; + usb->get_offload_dev = q6usb_get_offload_dev; + + ret = snd_soc_usb_add_port(usb); +.. + +**PUT Callback:** + +Can be used to track current device selection, and to issue any external DSP +commands that might be required for enabling audio offloading. + +.. code-block:: rst + + static int q6usb_put_offload_dev(struct snd_kcontrol *kcontrol, + struct snd_ctl_elem_value *ucontrol) + { + ... + if ((cardidx >= 0 && test_bit(cardidx, &data->available_card_slot))) { + data->sel_card_idx = cardidx; + changed = 1; + } + ... + if ((pcmidx >= 0 && pcmidx < data->status[cardidx].sdev->num_playback)) { + data->sel_pcm_idx = pcmidx; + changed = 1; + } +.. + +The above is an example of keeping track of what the userspace entity is +selecting as the playback device. This can be later used to pass the information +along to the external DSP. + + +USB Offload Playback Route Status +--------------------------------- +SOC USB exposes APIs for keeping track of the offloading state, and expects this +to be maintained by the BE DAI link that created/added the SOC USB device. + +**SOC USB State Flow Example** + +:: + + PCM Core | BE USB DAI Link | SOC USB + | | + snd_pcm_hw_params --------> dai_link->ops->hw_params --> snd_soc_usb_prepare_session + | | |--> state = SND_SOC_USB_PREPARED + ... | | |--> slot[0] now active + | | + | | + snd_pcm_do_prepare--------> dai_link->ops->prepare ---> snd_soc_usb_set_session_state + | | |--> state = SND_SOC_USB_RUNNING + ... | | + | | + snd_pcm_release_substream-> dai_link->ops->shutdown---> snd_soc_usb_shutdown_session + | | |--> state = SND_SOC_USB_IDLE + | | |--> slot[0] now idle + + +When executing the kcontrol get callback, it will loop across the active_list array +and report to the application for active USB sound card and USB PCM device indexes. + +USB Offload Playback Capable Card +--------------------------------- +USB sound also creates a kcontrol for applications to help determine which platform +sound card USB offloading is linked to. This will allow applications to further +query the platform sound card for specific information about the current USB offload +status. + +This is added as a separate mixer driver: + - mixer_usb_offload.c + - kcontrol: snd_usb_offload_available_ctl + +**snd_usb_offload_available_get()** fetches the associated sound card by utilizing +the **snd_soc_usb_device_offload_available()** API. + +Mixer Examples +-------------- + + :: + + tinymix -D 0 set 'USB Offload Playback Route Card Select' 2 + tinymix -D 0 set 'USB Offload Playback Route PCM Select' 0 + + + :: + + tinymix -D 0 get 'USB Offload Playback Route Card Select' + --> 2 (range -1->32) + tinymix -D 0 get 'USB Offload Playback Route PCM Select' + --> 0 (range -1->255) + + :: + + tinymix -D 0 get 'USB Offload Playback Route Card Status' + --> 2 (range -1->32) [OFFLD active] + --> -1 (range -1->32) [OFFLD idle] + tinymix -D 0 get 'USB Offload Playback Route PCM Status' + --> 0 (range -1->255) [OFFLD active] + --> -1 (range -1->255) [OFFLD idle] + + :: + + tinymix -D 1 get 'USB Offload Playback Capable Card' + --> 0 (range -1->32)
With the introduction of the soc-usb driver, add documentation highlighting details on how to utilize the new driver and how it interacts with different components in USB SND and ASoC. It provides examples on how to implement the drivers that will need to be introduced in order to enable USB audio offloading. Signed-off-by: Wesley Cheng <quic_wcheng@quicinc.com> --- Documentation/sound/soc/index.rst | 1 + Documentation/sound/soc/usb.rst | 603 ++++++++++++++++++++++++++++++ 2 files changed, 604 insertions(+) create mode 100644 Documentation/sound/soc/usb.rst