[-,AAF,PCM,plugin,0/7] Introduce AVTP Audio Format (AAF) plugin
mbox series

Message ID 20181003234547.16839-1-andre.guedes@intel.com
Headers show
Series
  • Introduce AVTP Audio Format (AAF) plugin
Related show

Message

Guedes, Andre Oct. 3, 2018, 11:45 p.m. UTC
Hi all,

The AAF plugin RFC series v2 has been around on alsa-devel ML for more than two
weeks so I'm moving forward and sending it as PATCH series now.

The first 5 patches from this series are pretty much the same from the previous
version. The news are the top 2 patches which add more functionality to the
plugin. Patch 6/7 implements the dump() callback which has been useful for
debugging purposes and Patch 7/7 adds support for direct read/write transfers.

For general context information about the AAF plugin, I'm quoting below the
cover letter from the RFC series v1:

	"This patchset introduces the AAF plugin to the alsa-plugins project
	which enables TSN audio applications on Linux.

	For those not familiarized with TSN, it stands for Time-Sensitive
	Networking, and it's a set of IEEE technologies (enhancements to IEEE
	802.1Q and Ethernet protocols) that provide precise time
	synchronization, bounded latency and application interoperability to
	bridged networks. Those technologies enable time-sensitive applications
	such as audio, video and control loops to run on top of bridged
	networks, co-existing with regular applications. TSN technologies are a
	super set of Audio Video Bridging (AVB) technologies also developed by
	IEEE. AVTP is the protocol defined to transport data in a TSN system,
	and AAF is one of the formats defined by AVTP to encapsulate audio
	data. AAF is specified in Chapter 7 from the AVTP spec [1].

	This work is part of the effort to enable TSN technologies on upstream
	Linux ecosystem. All building-blocks required to enable TSN audio
	applications have been already developed and pushed to upstream
	projects. Time synchronization features are provided by linuxptp
	project [2], bounded latency features are provided by Linux Traffic
	Control subsystem since kernel version 4.15 [3], and AVTP packetization
	is provided by libavtp [4]. What is currently missing in the ecosystem
	to enable TSN audio applications is a piece of software that plumbs it
	all together and interfaces with Linux Audio system properly. That's
	the point of the AAF plugin introduced here.

	The AAF plugin is a PCM plugin that uses AVTP protocol to transmit /
	receive audio data through a TSN capable network.  When operating in
	playback mode, the plugin reads PCM samples from the audio buffer,
	encapsulates into AVTP packets and transmits to the network, mimicking
	a typical AVTP Talker. When operating in capture mode, the plugin
	receives AVTP packets from the network, retrieves the PCM samples, and
	present them to alsa-lib layer at the presentation time, mimicking a
	typical AVTP Listener."

For further information about what has been previously discussed, please refer
to RFC series v1 and v2 archives in [5] and [6]. Finally, all versions of this
series can be also found in my alsa-plugins tree in github [7].

Best regards,

Andre

[1] 1722-2016 - IEEE Standard for a Transport Protocol for Time-Sensitive
    Applications in Bridged Local Area Networks
[2] http://linuxptp.sourceforge.net
[3] https://patchwork.ozlabs.org/cover/826678/
[4] https://github.com/AVnu/libavtp
[5] http://mailman.alsa-project.org/pipermail/alsa-devel/2018-August/139494.html
[6] http://mailman.alsa-project.org/pipermail/alsa-devel/2018-September/140290.html
[7] https://github.com/aguedes/alsa-plugins

Andre Guedes (7):
  aaf: Introduce plugin skeleton
  aaf: Load configuration parameters
  aaf: Implement Playback mode support
  aaf: Prepare for Capture mode support
  aaf: Implement Capture mode support
  aaf: Implement dump() ioplug callback
  aaf: Add support for direct read/write transfers

 Makefile.am     |    3 +
 aaf/Makefile.am |    9 +
 aaf/pcm_aaf.c   | 1197 +++++++++++++++++++++++++++++++++++++++++++++++++++++++
 configure.ac    |    9 +
 doc/aaf.txt     |  142 +++++++
 5 files changed, 1360 insertions(+)
 create mode 100644 aaf/Makefile.am
 create mode 100644 aaf/pcm_aaf.c
 create mode 100644 doc/aaf.txt

Comments

Takashi Iwai Oct. 10, 2018, 10:25 a.m. UTC | #1
On Thu, 04 Oct 2018 01:45:40 +0200,
Andre Guedes wrote:
> 
> Hi all,
> 
> The AAF plugin RFC series v2 has been around on alsa-devel ML for more than two
> weeks so I'm moving forward and sending it as PATCH series now.
> 
> The first 5 patches from this series are pretty much the same from the previous
> version. The news are the top 2 patches which add more functionality to the
> plugin. Patch 6/7 implements the dump() callback which has been useful for
> debugging purposes and Patch 7/7 adds support for direct read/write transfers.
> 
> For general context information about the AAF plugin, I'm quoting below the
> cover letter from the RFC series v1:
> 
> 	"This patchset introduces the AAF plugin to the alsa-plugins project
> 	which enables TSN audio applications on Linux.
> 
> 	For those not familiarized with TSN, it stands for Time-Sensitive
> 	Networking, and it's a set of IEEE technologies (enhancements to IEEE
> 	802.1Q and Ethernet protocols) that provide precise time
> 	synchronization, bounded latency and application interoperability to
> 	bridged networks. Those technologies enable time-sensitive applications
> 	such as audio, video and control loops to run on top of bridged
> 	networks, co-existing with regular applications. TSN technologies are a
> 	super set of Audio Video Bridging (AVB) technologies also developed by
> 	IEEE. AVTP is the protocol defined to transport data in a TSN system,
> 	and AAF is one of the formats defined by AVTP to encapsulate audio
> 	data. AAF is specified in Chapter 7 from the AVTP spec [1].
> 
> 	This work is part of the effort to enable TSN technologies on upstream
> 	Linux ecosystem. All building-blocks required to enable TSN audio
> 	applications have been already developed and pushed to upstream
> 	projects. Time synchronization features are provided by linuxptp
> 	project [2], bounded latency features are provided by Linux Traffic
> 	Control subsystem since kernel version 4.15 [3], and AVTP packetization
> 	is provided by libavtp [4]. What is currently missing in the ecosystem
> 	to enable TSN audio applications is a piece of software that plumbs it
> 	all together and interfaces with Linux Audio system properly. That's
> 	the point of the AAF plugin introduced here.
> 
> 	The AAF plugin is a PCM plugin that uses AVTP protocol to transmit /
> 	receive audio data through a TSN capable network.  When operating in
> 	playback mode, the plugin reads PCM samples from the audio buffer,
> 	encapsulates into AVTP packets and transmits to the network, mimicking
> 	a typical AVTP Talker. When operating in capture mode, the plugin
> 	receives AVTP packets from the network, retrieves the PCM samples, and
> 	present them to alsa-lib layer at the presentation time, mimicking a
> 	typical AVTP Listener."
> 
> For further information about what has been previously discussed, please refer
> to RFC series v1 and v2 archives in [5] and [6]. Finally, all versions of this
> series can be also found in my alsa-plugins tree in github [7].

The codes look mostly good.  I'll post just a new nitpicking later.

But I'd like to postpone the merge until Pierre back from his
vacation and getting some feedback from him.

BTW, what's the good way to test this stuff locally?


Thanks!

Takashi
Guedes, Andre Oct. 10, 2018, 9:19 p.m. UTC | #2
Hi Takashi,

On Wed, 2018-10-10 at 12:25 +0200, Takashi Iwai wrote:
> On Thu, 04 Oct 2018 01:45:40 +0200,
> Andre Guedes wrote:
> > 
> > Hi all,
> > 
> > The AAF plugin RFC series v2 has been around on alsa-devel ML for
> > more than two
> > weeks so I'm moving forward and sending it as PATCH series now.
> > 
> > The first 5 patches from this series are pretty much the same from
> > the previous
> > version. The news are the top 2 patches which add more
> > functionality to the
> > plugin. Patch 6/7 implements the dump() callback which has been
> > useful for
> > debugging purposes and Patch 7/7 adds support for direct read/write
> > transfers.
> > 
> > For general context information about the AAF plugin, I'm quoting
> > below the
> > cover letter from the RFC series v1:
> > 
> > 	"This patchset introduces the AAF plugin to the alsa-plugins
> > project
> > 	which enables TSN audio applications on Linux.
> > 
> > 	For those not familiarized with TSN, it stands for Time-
> > Sensitive
> > 	Networking, and it's a set of IEEE technologies (enhancements
> > to IEEE
> > 	802.1Q and Ethernet protocols) that provide precise time
> > 	synchronization, bounded latency and application
> > interoperability to
> > 	bridged networks. Those technologies enable time-sensitive
> > applications
> > 	such as audio, video and control loops to run on top of bridged
> > 	networks, co-existing with regular applications. TSN
> > technologies are a
> > 	super set of Audio Video Bridging (AVB) technologies also
> > developed by
> > 	IEEE. AVTP is the protocol defined to transport data in a TSN
> > system,
> > 	and AAF is one of the formats defined by AVTP to encapsulate
> > audio
> > 	data. AAF is specified in Chapter 7 from the AVTP spec [1].
> > 
> > 	This work is part of the effort to enable TSN technologies on
> > upstream
> > 	Linux ecosystem. All building-blocks required to enable TSN
> > audio
> > 	applications have been already developed and pushed to upstream
> > 	projects. Time synchronization features are provided by
> > linuxptp
> > 	project [2], bounded latency features are provided by Linux
> > Traffic
> > 	Control subsystem since kernel version 4.15 [3], and AVTP
> > packetization
> > 	is provided by libavtp [4]. What is currently missing in the
> > ecosystem
> > 	to enable TSN audio applications is a piece of software that
> > plumbs it
> > 	all together and interfaces with Linux Audio system properly.
> > That's
> > 	the point of the AAF plugin introduced here.
> > 
> > 	The AAF plugin is a PCM plugin that uses AVTP protocol to
> > transmit /
> > 	receive audio data through a TSN capable network.  When
> > operating in
> > 	playback mode, the plugin reads PCM samples from the audio
> > buffer,
> > 	encapsulates into AVTP packets and transmits to the network,
> > mimicking
> > 	a typical AVTP Talker. When operating in capture mode, the
> > plugin
> > 	receives AVTP packets from the network, retrieves the PCM
> > samples, and
> > 	present them to alsa-lib layer at the presentation time,
> > mimicking a
> > 	typical AVTP Listener."
> > 
> > For further information about what has been previously discussed,
> > please refer
> > to RFC series v1 and v2 archives in [5] and [6]. Finally, all
> > versions of this
> > series can be also found in my alsa-plugins tree in github [7].
> 
> The codes look mostly good.  I'll post just a new nitpicking later.
> 
> But I'd like to postpone the merge until Pierre back from his
> vacation and getting some feedback from him.

Ok, thanks for letting me know. I'll address your comments locally and
wait for his feedback before pushing the v2.

> BTW, what's the good way to test this stuff locally?

The minimal setup to test the plugin requires 2 machines connected
back-to-back. The machines should be equipped with a TSN-capable NIC
that supports PTP and FQTSS features. I'm using Intel i210 NIC.

doc/aaf.txt provides detailed instructions on how to setup the PTP and
FQTSS features from the NIC as well as how to run the plugin. If you
have any trouble with the instructions, please let me know and I'll fix
it for the next series version.

Thanks,

Andre
Pierre-Louis Bossart Oct. 22, 2018, 4:51 p.m. UTC | #3
On 10/10/18 4:19 PM, Guedes, Andre wrote:
> Hi Takashi,
>
> On Wed, 2018-10-10 at 12:25 +0200, Takashi Iwai wrote:
>> On Thu, 04 Oct 2018 01:45:40 +0200,
>> Andre Guedes wrote:
>>> Hi all,
>>>
>>> The AAF plugin RFC series v2 has been around on alsa-devel ML for
>>> more than two
>>> weeks so I'm moving forward and sending it as PATCH series now.
>>>
>>> The first 5 patches from this series are pretty much the same from
>>> the previous
>>> version. The news are the top 2 patches which add more
>>> functionality to the
>>> plugin. Patch 6/7 implements the dump() callback which has been
>>> useful for
>>> debugging purposes and Patch 7/7 adds support for direct read/write
>>> transfers.
>>>
>>> For general context information about the AAF plugin, I'm quoting
>>> below the
>>> cover letter from the RFC series v1:
>>>
>>> 	"This patchset introduces the AAF plugin to the alsa-plugins
>>> project
>>> 	which enables TSN audio applications on Linux.
>>>
>>> 	For those not familiarized with TSN, it stands for Time-
>>> Sensitive
>>> 	Networking, and it's a set of IEEE technologies (enhancements
>>> to IEEE
>>> 	802.1Q and Ethernet protocols) that provide precise time
>>> 	synchronization, bounded latency and application
>>> interoperability to
>>> 	bridged networks. Those technologies enable time-sensitive
>>> applications
>>> 	such as audio, video and control loops to run on top of bridged
>>> 	networks, co-existing with regular applications. TSN
>>> technologies are a
>>> 	super set of Audio Video Bridging (AVB) technologies also
>>> developed by
>>> 	IEEE. AVTP is the protocol defined to transport data in a TSN
>>> system,
>>> 	and AAF is one of the formats defined by AVTP to encapsulate
>>> audio
>>> 	data. AAF is specified in Chapter 7 from the AVTP spec [1].
>>>
>>> 	This work is part of the effort to enable TSN technologies on
>>> upstream
>>> 	Linux ecosystem. All building-blocks required to enable TSN
>>> audio
>>> 	applications have been already developed and pushed to upstream
>>> 	projects. Time synchronization features are provided by
>>> linuxptp
>>> 	project [2], bounded latency features are provided by Linux
>>> Traffic
>>> 	Control subsystem since kernel version 4.15 [3], and AVTP
>>> packetization
>>> 	is provided by libavtp [4]. What is currently missing in the
>>> ecosystem
>>> 	to enable TSN audio applications is a piece of software that
>>> plumbs it
>>> 	all together and interfaces with Linux Audio system properly.
>>> That's
>>> 	the point of the AAF plugin introduced here.
>>>
>>> 	The AAF plugin is a PCM plugin that uses AVTP protocol to
>>> transmit /
>>> 	receive audio data through a TSN capable network.  When
>>> operating in
>>> 	playback mode, the plugin reads PCM samples from the audio
>>> buffer,
>>> 	encapsulates into AVTP packets and transmits to the network,
>>> mimicking
>>> 	a typical AVTP Talker. When operating in capture mode, the
>>> plugin
>>> 	receives AVTP packets from the network, retrieves the PCM
>>> samples, and
>>> 	present them to alsa-lib layer at the presentation time,
>>> mimicking a
>>> 	typical AVTP Listener."
>>>
>>> For further information about what has been previously discussed,
>>> please refer
>>> to RFC series v1 and v2 archives in [5] and [6]. Finally, all
>>> versions of this
>>> series can be also found in my alsa-plugins tree in github [7].
>> The codes look mostly good.  I'll post just a new nitpicking later.
>>
>> But I'd like to postpone the merge until Pierre back from his
>> vacation and getting some feedback from him.

I'm back and will look at those patches this week. I'll probably have to 
talk to Andre to figure out how the clocks are managed, I didn't fully 
understand the answers from August that

"The plugin requires both CLOCK_REALTIME and PTP to be synchronized, and 
this can add some usage scenarios limitation, indeed. However, the 
scenario you described looks still feasible. For instance, at the host 
running as PTP master, we could have NTP disciplining CLOCK_REALTIME 
(ntp daemon) and CLOCK_REALTIME disciplining PTP (phc2sys daemon). At 
the hosts running as PTP slave, we have PTP disciplining CLOCK_REALTIME 
(phc2sys daemon). This way, CLOCK_REALTIME time from all systems is NTP 
time while CLOCK_REALTIME and PTP clock are in sync"

For audio usages we typically have all clocks derived from ART, and we'd 
need the CLOCK_MONOTONIC_RAW to align with the PTP clock to allow for 
loopbacks without ASRC (or ASRC between the audio clock and ART). Not 
sure who uses CLOCK_REALTIME for audio?

> Ok, thanks for letting me know. I'll address your comments locally and
> wait for his feedback before pushing the v2.
>
>> BTW, what's the good way to test this stuff locally?
> The minimal setup to test the plugin requires 2 machines connected
> back-to-back. The machines should be equipped with a TSN-capable NIC
> that supports PTP and FQTSS features. I'm using Intel i210 NIC.
>
> doc/aaf.txt provides detailed instructions on how to setup the PTP and
> FQTSS features from the NIC as well as how to run the plugin. If you
> have any trouble with the instructions, please let me know and I'll fix
> it for the next series version.

I was planning to experiment with two stacked MinnowBoard Turbot quad 
core dual ethernet [1], they have i210 support and would be the 
cheapest/compact way to test all this. You can connect to each board 
from your workstation with one of the two ports for 
updates/command/control and dedicate the second port to AVB/TSN 
transfers between the two boards.

[1] 
https://store.netgate.com/MinnowBoard-Turbot-Dual-Ethernet-Quad-Core-Board-P2780.aspx

>
> Thanks,
>
> Andre
>
> _______________________________________________
> Alsa-devel mailing list
> Alsa-devel@alsa-project.org
> http://mailman.alsa-project.org/mailman/listinfo/alsa-devel
Guedes, Andre Oct. 23, 2018, 8:01 p.m. UTC | #4
Hi Pierre,

On Mon, 2018-10-22 at 11:51 -0500, Pierre-Louis Bossart wrote:
> > > The codes look mostly good.  I'll post just a new nitpicking
> > > later.
> > > 
> > > But I'd like to postpone the merge until Pierre back from his
> > > vacation and getting some feedback from him.
> 
> I'm back and will look at those patches this week.

Thanks for letting me know. I just found a bug when multiple streams
are running at the same time. I have it fixed already, and since
Takashi's comments have been addressed as well, I'm going to push the
v2 so you can review the latest code.

I'm finishing testing the v2 series and should post it by tomorrow.

> I'll probably have to talk to Andre to figure out how the clocks are
> managed, I didn't fully understand the answers from August that 
> "The plugin requires both CLOCK_REALTIME and PTP to be synchronized,
> and this can add some usage scenarios limitation, indeed. However,
> the scenario you described looks still feasible. For instance, at the
> host running as PTP master, we could have NTP disciplining
> CLOCK_REALTIME (ntp daemon) and CLOCK_REALTIME disciplining PTP
> (phc2sys daemon). At the hosts running as PTP slave, we have PTP
> disciplining CLOCK_REALTIME (phc2sys daemon). This way,
> CLOCK_REALTIME time from all systems is NTP time while CLOCK_REALTIME
> and PTP clock are in sync"
> For audio usages we typically have all clocks derived from ART, and
> we'd need the CLOCK_MONOTONIC_RAW to align with the PTP clock to
> allow for loopbacks without ASRC (or ASRC between the audio clock and
> ART). Not sure who uses CLOCK_REALTIME for audio?

Sure, let's follow up on this when you're available.

> > > BTW, what's the good way to test this stuff locally?
> > 
> > The minimal setup to test the plugin requires 2 machines connected
> > back-to-back. The machines should be equipped with a TSN-capable
> > NIC that supports PTP and FQTSS features. I'm using Intel i210 NIC.
> > 
> > doc/aaf.txt provides detailed instructions on how to setup the PTP
> > and FQTSS features from the NIC as well as how to run the plugin.
> > If you have any trouble with the instructions, please let me know
> > and I'll fix it for the next series version.
> 
> I was planning to experiment with two stacked MinnowBoard Turbot quad
> core dual ethernet [1], they have i210 support and would be the
> cheapest/compact way to test all this. You can connect to each board
> from your workstation with one of the two ports for
> updates/command/control and dedicate the second port to AVB/TSN
> transfers between the two boards.

I'm not using MinnowBoard Turbot but the setup you described here is
pretty much what my testing setup is.

Regards,

Andre