diff mbox

[RFCv3] dvb: Add DVBv5 properties for quality parameters

Message ID 20130101130041.52dee65f@redhat.com (mailing list archive)
State New, archived
Headers show

Commit Message

Mauro Carvalho Chehab Jan. 1, 2013, 3 p.m. UTC
Em Sat, 29 Dec 2012 11:36:16 -0500
Devin Heitmueller <dheitmueller@kernellabs.com> escreveu:

> On Fri, Dec 28, 2012 at 6:56 PM, Mauro Carvalho Chehab
> <mchehab@redhat.com> wrote:
> > The DVBv3 quality parameters are limited on several ways:
> >         - Doesn't provide any way to indicate the used measure;
> >         - Userspace need to guess how to calculate the measure;
> >         - Only a limited set of stats are supported;
> >         - Doesn't provide QoS measure for the OFDM TPS/TMCC
> >           carriers, used to detect the network parameters for
> >           DVB-T/ISDB-T;
> >         - Can't be called in a way to require them to be filled
> >           all at once (atomic reads from the hardware), with may
> >           cause troubles on interpreting them on userspace;
> >         - On some OFDM delivery systems, the carriers can be
> >           independently modulated, having different properties.
> >           Currently, there's no way to report per-layer stats;
> > This RFC adds the header definitions meant to solve that issues.
> > After discussed, I'll write a patch for the DocBook and add support
> > for it on some demods. Support for dvbv5-zap and dvbv5-scan tools
> > will also have support for those features.
> 
> Hi Mauro,
<...>
> You have a units field which is "decibels", but in what unit?  1 dB /
> unit?  0.1 db / unit?  1/255 db / unit?  This particular issue is why
> the current snr field varies across even demods where we have the
> datasheets.  Many demods reported in 0.1 dB increments, while others
> reported in 1/255 dB increments.

There was min/max values there to allow userspace to calculate the scale,
but it seems simpler to just define a 0.1dB step.
<...>
> This needs to be defined *in the spec*,

Sure. After we agree on the API we'll be using, I'll write the spec bits
and add the logic on a driver. I did that in the past, but as the discussions
were long and no consensus was reached. So, I opted to just write the changes
at the frontend.h for the discussons.
<...>

Ok, I decided to post a version 4, removing the things that I won't be using
(that got inherited by a previous proposal from another developer sent to
the ML a few years ago).

This version defines 4 types of statistics: signal, SNR, BER, error count
at the TMCC/TPS carrier. There are just 2 possible ranges:
	dB (actually 0.1 dB);
	percentage, from 0 to 100%.

There's a third "range" type that means that a given parameter is not
available. I didn't add any bits to indicate why, as the reason may not
be relevant to userspace. It might make sense to have different types, one
for temporary unavailability and another one for permanent unavailability.

There's also one parameter to enumerate what QoS parameters are supported by
the driver. 

Regards,
Mauro

-

[RFCv4] dvb: Add DVBv5 properties for quality parameters

The DVBv3 quality parameters are limited on several ways:
	- Doesn't provide any way to indicate the used measure;
	- Userspace need to guess how to calculate the measure;
	- Only a limited set of stats are supported;
	- Doesn't provide QoS measure for the OFDM TPS/TMCC
	  carriers, used to detect the network parameters for
	  DVB-T/ISDB-T;
	- Can't be called in a way to require them to be filled
	  all at once (atomic reads from the hardware), with may
	  cause troubles on interpreting them on userspace;
	- On some OFDM delivery systems, the carriers can be
	  independently modulated, having different properties.
	  Currently, there's no way to report per-layer stats;
This RFC adds the header definitions meant to solve that issues.
After discussed, I'll write a patch for the DocBook and add support
for it on some demods. Support for dvbv5-zap and dvbv5-scan tools
will also have support for those features.

Signed-off-by: Mauro Carvalho Chehab <mchehab@redhat.com>

---
include/uapi/linux/dvb/frontend.h | 78 ++++++++++++++++++++++++++++++++++++++-
 include/uapi/linux/dvb/frontend.h |   60 +++++++++++++++++++++++++++++++++++++-
 1 file changed, 59 insertions(+), 1 deletion(-)

v3: Just update http://patchwork.linuxtv.org/patch/9578/ to current tip

v4: API simplified and addressed some issues pointed by Devin


> 
> Devin
>

Comments

Manu Abraham Jan. 1, 2013, 4:48 p.m. UTC | #1
On Tue, Jan 1, 2013 at 8:30 PM, Mauro Carvalho Chehab
<mchehab@redhat.com> wrote:

> [RFCv4] dvb: Add DVBv5 properties for quality parameters
>
> The DVBv3 quality parameters are limited on several ways:
>         - Doesn't provide any way to indicate the used measure;
>         - Userspace need to guess how to calculate the measure;
>         - Only a limited set of stats are supported;
>         - Doesn't provide QoS measure for the OFDM TPS/TMCC
>           carriers, used to detect the network parameters for
>           DVB-T/ISDB-T;
>         - Can't be called in a way to require them to be filled
>           all at once (atomic reads from the hardware), with may
>           cause troubles on interpreting them on userspace;
>         - On some OFDM delivery systems, the carriers can be
>           independently modulated, having different properties.
>           Currently, there's no way to report per-layer stats;

per layer stats is a mythical bird, nothing of that sort does exist. If some
driver states that it is simply due to lack of knowledge at the coding side.

ISDB-T uses hierarchial modulation, just like DVB-S2 or DVB-T2

Once the Outer code is decoded, the OFDM segments are separated
using Hierarchial separation. This is well described by NHK.


"To improve mobile reception and robustness to multipath
interference, the system performs, in symbol units, time
interleaving plus frequency interleaving according to the
arrangement of OFDM segments. Pilot signals for
demodulation and control symbols consisting of TMCC
information are combined with information symbols to an
OFDM frame. Here, information symbols are modulated
by Differential Binary Phase Shift Keying (DBPSK) and
guard intervals are added at the IFFT output.

[3] Hierarchical transmission
A mixture of fixed-reception programs and mobile reception
programs in the transmission system is made
possible through the application of hierarchical
transmission achieved by band division within a channel.
"Hierarchical transmission" means that the three elements
of channel coding, namely, the modulation system, the
coding rate of convolutional correction code, and the time
interleave length, can be independently set. Time and
frequency interleaving are each performed in their
respective hierarchical data segment.
As described earlier, the smallest hierarchical unit in a
frequency spectrum is one OFDM segment."


Please don't muck up existing working things with uber crap.

Manu
--
To unsubscribe from this list: send the line "unsubscribe linux-media" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Mauro Carvalho Chehab Jan. 1, 2013, 5:29 p.m. UTC | #2
Em Tue, 1 Jan 2013 22:18:49 +0530
Manu Abraham <abraham.manu@gmail.com> escreveu:

> On Tue, Jan 1, 2013 at 8:30 PM, Mauro Carvalho Chehab
> <mchehab@redhat.com> wrote:
> 
> > [RFCv4] dvb: Add DVBv5 properties for quality parameters
> >
> > The DVBv3 quality parameters are limited on several ways:
> >         - Doesn't provide any way to indicate the used measure;
> >         - Userspace need to guess how to calculate the measure;
> >         - Only a limited set of stats are supported;
> >         - Doesn't provide QoS measure for the OFDM TPS/TMCC
> >           carriers, used to detect the network parameters for
> >           DVB-T/ISDB-T;
> >         - Can't be called in a way to require them to be filled
> >           all at once (atomic reads from the hardware), with may
> >           cause troubles on interpreting them on userspace;
> >         - On some OFDM delivery systems, the carriers can be
> >           independently modulated, having different properties.
> >           Currently, there's no way to report per-layer stats;
> 
> per layer stats is a mythical bird, nothing of that sort does exist. 

Had you ever read or tried to get stats from an ISDB-T demod? If you
had, you would see that it only provides per-layer stats. Btw, this is
a requirement to follow the ARIB and ABNT ISDB specs.

Cheers,
Mauro
--
To unsubscribe from this list: send the line "unsubscribe linux-media" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Manu Abraham Jan. 1, 2013, 7:08 p.m. UTC | #3
On Tue, Jan 1, 2013 at 10:59 PM, Mauro Carvalho Chehab
<mchehab@redhat.com> wrote:
> Em Tue, 1 Jan 2013 22:18:49 +0530
> Manu Abraham <abraham.manu@gmail.com> escreveu:
>
>> On Tue, Jan 1, 2013 at 8:30 PM, Mauro Carvalho Chehab
>> <mchehab@redhat.com> wrote:
>>
>> > [RFCv4] dvb: Add DVBv5 properties for quality parameters
>> >
>> > The DVBv3 quality parameters are limited on several ways:
>> >         - Doesn't provide any way to indicate the used measure;
>> >         - Userspace need to guess how to calculate the measure;
>> >         - Only a limited set of stats are supported;
>> >         - Doesn't provide QoS measure for the OFDM TPS/TMCC
>> >           carriers, used to detect the network parameters for
>> >           DVB-T/ISDB-T;
>> >         - Can't be called in a way to require them to be filled
>> >           all at once (atomic reads from the hardware), with may
>> >           cause troubles on interpreting them on userspace;
>> >         - On some OFDM delivery systems, the carriers can be
>> >           independently modulated, having different properties.
>> >           Currently, there's no way to report per-layer stats;
>>
>> per layer stats is a mythical bird, nothing of that sort does exist.
>
> Had you ever read or tried to get stats from an ISDB-T demod? If you
> had, you would see that it only provides per-layer stats. Btw, this is
> a requirement to follow the ARIB and ABNT ISDB specs.

I understand you keep writing junk for ages, but nevertheless:

Do you have any idea what's a BBHEADER (DVB-S2) or
PLHEADER (DVB-T2) ? The headers do indicate what MODCOD
(aka Modulation/Coding Standard follows, whatever mode ACM,
VCM or CCM) follows. These MODCOD foolows a TDM approach
with a hierarchial modulation principle. This is exactly what ISDB
does too.

And for your info:

" The TMCC control information is
common to all TMCC carriers and
error correction is performed by using
difference-set cyclic code."

Manu
--
To unsubscribe from this list: send the line "unsubscribe linux-media" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Mauro Carvalho Chehab Jan. 3, 2013, 1:20 p.m. UTC | #4
Em Wed, 2 Jan 2013 00:38:50 +0530
Manu Abraham <abraham.manu@gmail.com> escreveu:

> On Tue, Jan 1, 2013 at 10:59 PM, Mauro Carvalho Chehab
> <mchehab@redhat.com> wrote:
> > Em Tue, 1 Jan 2013 22:18:49 +0530
> > Manu Abraham <abraham.manu@gmail.com> escreveu:
> >
> >> On Tue, Jan 1, 2013 at 8:30 PM, Mauro Carvalho Chehab
> >> <mchehab@redhat.com> wrote:
> >>
> >> > [RFCv4] dvb: Add DVBv5 properties for quality parameters
> >> >
> >> > The DVBv3 quality parameters are limited on several ways:
> >> >         - Doesn't provide any way to indicate the used measure;
> >> >         - Userspace need to guess how to calculate the measure;
> >> >         - Only a limited set of stats are supported;
> >> >         - Doesn't provide QoS measure for the OFDM TPS/TMCC
> >> >           carriers, used to detect the network parameters for
> >> >           DVB-T/ISDB-T;
> >> >         - Can't be called in a way to require them to be filled
> >> >           all at once (atomic reads from the hardware), with may
> >> >           cause troubles on interpreting them on userspace;
> >> >         - On some OFDM delivery systems, the carriers can be
> >> >           independently modulated, having different properties.
> >> >           Currently, there's no way to report per-layer stats;
> >>
> >> per layer stats is a mythical bird, nothing of that sort does exist.
> >
> > Had you ever read or tried to get stats from an ISDB-T demod? If you
> > had, you would see that it only provides per-layer stats. Btw, this is
> > a requirement to follow the ARIB and ABNT ISDB specs.
> 
> I understand you keep writing junk for ages, but nevertheless:
> 
> Do you have any idea what's a BBHEADER (DVB-S2) or
> PLHEADER (DVB-T2) ? The headers do indicate what MODCOD
> (aka Modulation/Coding Standard follows, whatever mode ACM,
> VCM or CCM) follows. These MODCOD foolows a TDM approach
> with a hierarchial modulation principle. This is exactly what ISDB
> does too.

No, I didn't check DVB-S2/T2 specs deeply enough to understand
if they're doing the same thing as ISDB.

Yet, ISDB-T doesn't use a TDM approach for hierarchical modulation.
It uses a FDM (OFDM is a type of Frequency Division Multiplexing).

So, if you're saying that DVB-S2 uses TDM, it is very different than
ISDB-T. As DVB-T2 uses an FDM type of modulation (OFDM), it would
be possible to segment the carriers there, just like ISDB, or to
use TDM hierarchical modulation techniques.

> 
> And for your info:
> 
> " The TMCC control information is
> common to all TMCC carriers and
> error correction is performed by using
> difference-set cyclic code."

Yes, TMCC carriers are equal and they are always modulated using DBPSK.
That is done to make it possible to receive the TMCC carriers even under
worse SNR conditions, where it may not be possible to decode the segment
groups.

It seems that you completely missed the point though. On ISDB-T, the 
carriers that belong to each group of segments (except for the control
carriers - carriers 1 to 107) uses a completely independent modulation.
Also, as they're spaced in frequency, the interference of each segment
is different. So, error indications are different on each segment.

Btw, in any case, the datasheets of ISDB-T demods clearly shows that
the BER measures are per segment group (layer).

For example, for the BER measures before Viterbi, those are the register
names for a certain demod:

	VBERSNUMA Bit count of BER measurement before Viterbi in A layer
	VBERSNUMB Bit count of BER measurement before Viterbi in B layer
	VBERSNUMC Bit count of BER measurement before Viterbi in C layer

It has another set of registers for BER after Viterbi, and for PER after
Viterbi and RS, for bit count errors, etc.

There's no way to get any type of "global" BER measure, simply because
ISDB-T demods don't provide.

Regards,
Mauro
--
To unsubscribe from this list: send the line "unsubscribe linux-media" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Klaus Schmidinger Jan. 3, 2013, 3:18 p.m. UTC | #5
On 03.01.2013 14:20, Mauro Carvalho Chehab wrote:
> Em Wed, 2 Jan 2013 00:38:50 +0530
> Manu Abraham <abraham.manu@gmail.com> escreveu:
>
>> On Tue, Jan 1, 2013 at 10:59 PM, Mauro Carvalho Chehab
>> <mchehab@redhat.com> wrote:
>>> Em Tue, 1 Jan 2013 22:18:49 +0530
>>> Manu Abraham <abraham.manu@gmail.com> escreveu:
>>>
>>>> On Tue, Jan 1, 2013 at 8:30 PM, Mauro Carvalho Chehab
>>>> <mchehab@redhat.com> wrote:
>>>>
>>>>> [RFCv4] dvb: Add DVBv5 properties for quality parameters
>>>>>
>>>>> The DVBv3 quality parameters are limited on several ways:
>>>>>          - Doesn't provide any way to indicate the used measure;
>>>>>          - Userspace need to guess how to calculate the measure;
>>>>>          - Only a limited set of stats are supported;
>>>>>          - Doesn't provide QoS measure for the OFDM TPS/TMCC
>>>>>            carriers, used to detect the network parameters for
>>>>>            DVB-T/ISDB-T;
>>>>>          - Can't be called in a way to require them to be filled
>>>>>            all at once (atomic reads from the hardware), with may
>>>>>            cause troubles on interpreting them on userspace;
>>>>>          - On some OFDM delivery systems, the carriers can be
>>>>>            independently modulated, having different properties.
>>>>>            Currently, there's no way to report per-layer stats;
>>>>
>>>> per layer stats is a mythical bird, nothing of that sort does exist.
>>>
>>> Had you ever read or tried to get stats from an ISDB-T demod? If you
>>> had, you would see that it only provides per-layer stats. Btw, this is
>>> a requirement to follow the ARIB and ABNT ISDB specs.
>>
>> I understand you keep writing junk for ages, but nevertheless:
>>
>> Do you have any idea what's a BBHEADER (DVB-S2) or
>> PLHEADER (DVB-T2) ? The headers do indicate what MODCOD
>> (aka Modulation/Coding Standard follows, whatever mode ACM,
>> VCM or CCM) follows. These MODCOD foolows a TDM approach
>> with a hierarchial modulation principle. This is exactly what ISDB
>> does too.
>
> No, I didn't check DVB-S2/T2 specs deeply enough to understand
> if they're doing the same thing as ISDB.
>
> Yet, ISDB-T doesn't use a TDM approach for hierarchical modulation.
> It uses a FDM (OFDM is a type of Frequency Division Multiplexing).
>
> So, if you're saying that DVB-S2 uses TDM, it is very different than
> ISDB-T. As DVB-T2 uses an FDM type of modulation (OFDM), it would
> be possible to segment the carriers there, just like ISDB, or to
> use TDM hierarchical modulation techniques.
>
>>
>> And for your info:
>>
>> " The TMCC control information is
>> common to all TMCC carriers and
>> error correction is performed by using
>> difference-set cyclic code."
>
> Yes, TMCC carriers are equal and they are always modulated using DBPSK.
> That is done to make it possible to receive the TMCC carriers even under
> worse SNR conditions, where it may not be possible to decode the segment
> groups.
>
> It seems that you completely missed the point though. On ISDB-T, the
> carriers that belong to each group of segments (except for the control
> carriers - carriers 1 to 107) uses a completely independent modulation.
> Also, as they're spaced in frequency, the interference of each segment
> is different. So, error indications are different on each segment.
>
> Btw, in any case, the datasheets of ISDB-T demods clearly shows that
> the BER measures are per segment group (layer).
>
> For example, for the BER measures before Viterbi, those are the register
> names for a certain demod:
>
> 	VBERSNUMA Bit count of BER measurement before Viterbi in A layer
> 	VBERSNUMB Bit count of BER measurement before Viterbi in B layer
> 	VBERSNUMC Bit count of BER measurement before Viterbi in C layer
>
> It has another set of registers for BER after Viterbi, and for PER after
> Viterbi and RS, for bit count errors, etc.
>
> There's no way to get any type of "global" BER measure, simply because
> ISDB-T demods don't provide.

Maybe we should put all this theoretical discussion aside for the moment and
think about what is *really* needed by real world applications. As with any
receiver, VDR simply wants to have some measure of the signal's "strength"
and "quality". These are just two values that should be delivered by each
frontend/demux, using the *same* defined and mandatory range. I don't care
what exactly that is, but it needs to be the same for all devices.
What values a particular driver uses internally to come up with these
is of no interest to VDR. The "signal strength" might just be what is
currently returned through FE_READ_SIGNAL_STRENGTH (however, normalized to
the same range in all drivers, which currently is not the case). The "signal
quality" might use flags like FE_HAS_SIGNAL, FE_HAS_CARRIER, FE_HAS_VITERBI,
FE_HAS_SYNC, as well as SNR, BER and UNC (if available) to form some
value where 0 means no quality at all, and 0xFFFF means excellent quality.
If a particular frontend/demux uses totally different concepts, it can
just use whatever it deems reasonable to form the "strength" and "quality"
values. The important thing here is just that all this needs to be hidden
inside the driver, and the only interface to an application are ioctl()
calls that return these two values.

So I suggest that you define this minimal interface and allow applications
to retrieve what they really need. Once this is done, feel free to implement
whatever theoretical bells and whistles you fell like doing - that's all
fine with me, as long as the really important stuff keeps working ;-)

Klaus
--
To unsubscribe from this list: send the line "unsubscribe linux-media" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Antti Palosaari Jan. 3, 2013, 3:34 p.m. UTC | #6
On 01/01/2013 06:48 PM, Manu Abraham wrote:
> On Tue, Jan 1, 2013 at 8:30 PM, Mauro Carvalho Chehab
> <mchehab@redhat.com> wrote:
>
>> [RFCv4] dvb: Add DVBv5 properties for quality parameters
>>
>> The DVBv3 quality parameters are limited on several ways:
>>          - Doesn't provide any way to indicate the used measure;
>>          - Userspace need to guess how to calculate the measure;
>>          - Only a limited set of stats are supported;
>>          - Doesn't provide QoS measure for the OFDM TPS/TMCC
>>            carriers, used to detect the network parameters for
>>            DVB-T/ISDB-T;
>>          - Can't be called in a way to require them to be filled
>>            all at once (atomic reads from the hardware), with may
>>            cause troubles on interpreting them on userspace;
>>          - On some OFDM delivery systems, the carriers can be
>>            independently modulated, having different properties.
>>            Currently, there's no way to report per-layer stats;
>
> per layer stats is a mythical bird, nothing of that sort does exist. If some
> driver states that it is simply due to lack of knowledge at the coding side.
>
> ISDB-T uses hierarchial modulation, just like DVB-S2 or DVB-T2

Manu, you confused now two concept (which are aimed to resolve same real 
life problem) - hierarchical coding and multiple transport stream. Both 
are quite similar on lower level of radio channel, but differs on upper 
levels.

Hierarchical is a little bit weird baby as it remuxes those lower lever 
radio channels (called layers in case of ISDB-T) to one single mux! 
There is only single TS which demodulator is responsible to remux all 
those 3 physical "layer" channels, which could be modulated differently. 
So after demodulation you really has a TS which contains stream that has 
different statistics. That's opposite to compared for multiple TS 
principle used for DVB-T2/S2. In case of multiple TS you have same 
statistics for whole TS (but naturally there could be multiple TS after 
demodulation).

regards
Antti
Mauro Carvalho Chehab Jan. 3, 2013, 4:14 p.m. UTC | #7
Em Thu, 03 Jan 2013 16:18:26 +0100
Klaus Schmidinger <Klaus.Schmidinger@tvdr.de> escreveu:

> On 03.01.2013 14:20, Mauro Carvalho Chehab wrote:
> > Em Wed, 2 Jan 2013 00:38:50 +0530
> > Manu Abraham <abraham.manu@gmail.com> escreveu:
> >
> >> On Tue, Jan 1, 2013 at 10:59 PM, Mauro Carvalho Chehab
> >> <mchehab@redhat.com> wrote:
> >>> Em Tue, 1 Jan 2013 22:18:49 +0530
> >>> Manu Abraham <abraham.manu@gmail.com> escreveu:
> >>>
> >>>> On Tue, Jan 1, 2013 at 8:30 PM, Mauro Carvalho Chehab
> >>>> <mchehab@redhat.com> wrote:
> >>>>
> >>>>> [RFCv4] dvb: Add DVBv5 properties for quality parameters
> >>>>>
> >>>>> The DVBv3 quality parameters are limited on several ways:
> >>>>>          - Doesn't provide any way to indicate the used measure;
> >>>>>          - Userspace need to guess how to calculate the measure;
> >>>>>          - Only a limited set of stats are supported;
> >>>>>          - Doesn't provide QoS measure for the OFDM TPS/TMCC
> >>>>>            carriers, used to detect the network parameters for
> >>>>>            DVB-T/ISDB-T;
> >>>>>          - Can't be called in a way to require them to be filled
> >>>>>            all at once (atomic reads from the hardware), with may
> >>>>>            cause troubles on interpreting them on userspace;
> >>>>>          - On some OFDM delivery systems, the carriers can be
> >>>>>            independently modulated, having different properties.
> >>>>>            Currently, there's no way to report per-layer stats;
> >>>>
> >>>> per layer stats is a mythical bird, nothing of that sort does exist.
> >>>
> >>> Had you ever read or tried to get stats from an ISDB-T demod? If you
> >>> had, you would see that it only provides per-layer stats. Btw, this is
> >>> a requirement to follow the ARIB and ABNT ISDB specs.
> >>
> >> I understand you keep writing junk for ages, but nevertheless:
> >>
> >> Do you have any idea what's a BBHEADER (DVB-S2) or
> >> PLHEADER (DVB-T2) ? The headers do indicate what MODCOD
> >> (aka Modulation/Coding Standard follows, whatever mode ACM,
> >> VCM or CCM) follows. These MODCOD foolows a TDM approach
> >> with a hierarchial modulation principle. This is exactly what ISDB
> >> does too.
> >
> > No, I didn't check DVB-S2/T2 specs deeply enough to understand
> > if they're doing the same thing as ISDB.
> >
> > Yet, ISDB-T doesn't use a TDM approach for hierarchical modulation.
> > It uses a FDM (OFDM is a type of Frequency Division Multiplexing).
> >
> > So, if you're saying that DVB-S2 uses TDM, it is very different than
> > ISDB-T. As DVB-T2 uses an FDM type of modulation (OFDM), it would
> > be possible to segment the carriers there, just like ISDB, or to
> > use TDM hierarchical modulation techniques.
> >
> >>
> >> And for your info:
> >>
> >> " The TMCC control information is
> >> common to all TMCC carriers and
> >> error correction is performed by using
> >> difference-set cyclic code."
> >
> > Yes, TMCC carriers are equal and they are always modulated using DBPSK.
> > That is done to make it possible to receive the TMCC carriers even under
> > worse SNR conditions, where it may not be possible to decode the segment
> > groups.
> >
> > It seems that you completely missed the point though. On ISDB-T, the
> > carriers that belong to each group of segments (except for the control
> > carriers - carriers 1 to 107) uses a completely independent modulation.
> > Also, as they're spaced in frequency, the interference of each segment
> > is different. So, error indications are different on each segment.
> >
> > Btw, in any case, the datasheets of ISDB-T demods clearly shows that
> > the BER measures are per segment group (layer).
> >
> > For example, for the BER measures before Viterbi, those are the register
> > names for a certain demod:
> >
> > 	VBERSNUMA Bit count of BER measurement before Viterbi in A layer
> > 	VBERSNUMB Bit count of BER measurement before Viterbi in B layer
> > 	VBERSNUMC Bit count of BER measurement before Viterbi in C layer
> >
> > It has another set of registers for BER after Viterbi, and for PER after
> > Viterbi and RS, for bit count errors, etc.
> >
> > There's no way to get any type of "global" BER measure, simply because
> > ISDB-T demods don't provide.
> 
> Maybe we should put all this theoretical discussion aside for the moment and
> think about what is *really* needed by real world applications. As with any
> receiver, VDR simply wants to have some measure of the signal's "strength"
> and "quality". These are just two values that should be delivered by each
> frontend/demux, using the *same* defined and mandatory range. I don't care
> what exactly that is, but it needs to be the same for all devices.
> What values a particular driver uses internally to come up with these
> is of no interest to VDR. The "signal strength" might just be what is
> currently returned through FE_READ_SIGNAL_STRENGTH (however, normalized to
> the same range in all drivers, which currently is not the case). The "signal
> quality" might use flags like FE_HAS_SIGNAL, FE_HAS_CARRIER, FE_HAS_VITERBI,
> FE_HAS_SYNC, as well as SNR, BER and UNC (if available) to form some
> value where 0 means no quality at all, and 0xFFFF means excellent quality.
> If a particular frontend/demux uses totally different concepts, it can
> just use whatever it deems reasonable to form the "strength" and "quality"
> values. The important thing here is just that all this needs to be hidden
> inside the driver, and the only interface to an application are ioctl()
> calls that return these two values.
> 
> So I suggest that you define this minimal interface and allow applications
> to retrieve what they really need. Once this is done, feel free to implement
> whatever theoretical bells and whistles you fell like doing - that's all
> fine with me, as long as the really important stuff keeps working ;-)

Klaus,

On ISDB-T, it splits the TS into (up to) three independent physical channels
(called layers).

Each channel has its own statistics, as they're completely independent:
they use different inner FEC's, use different modulations, etc.

The ISDB demods don't provide a single value for the 3 layers. They
can't, as they're independent. So, signal-strengh and SNR measures are
also independent for each of those 3 layers.

A typical ISDB transmission uses 13 segments of carriers, each segment
using a 4.28 kHz bandwidth, grouped into 3 layers. While it is up to 
the broadcaster to decide how to group the segments, it is typically 
arranged like that:

	layer A - 1 segment for LD programs - modulated using QPSK;
	layer B - 3 segments for SD programs - modulated using 16QAM;
	layer C - 9 segments for HD programs - modulated using 64QAM.

The TDM TS packets from the Transport Stream are broken into those 3
layers, each being an independent transmission channel.

So, all channel level QoS measure are per-layer (SNR, signal strength,
BER, MER, ...).

While the demods I have datasheets here don't provide it, it would be
possible to provide error counts for a given program ID, by summing
the error count that applies to each PID.

So, let's assume, for example, that the UCB count is:
	layer A = 0
	layer B = 12
	layer C = 30

an 1-seg LD program will have 0 uncorrected blocks;
an SD program will have 12 uncorrected error blocks;
a HD program will have 42 uncorrected error blocks.

It shouldn't be that hard to take it into account on userspace, but
doing it at kernel level would be very painful, if possible, as
kernelspace would be required to know what PID's are being shown, in
order to estimate the error count measures for them. Also, it would
require a much more complex kernelspace-userspace interface.

Regards,
Mauro
--
To unsubscribe from this list: send the line "unsubscribe linux-media" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Mauro Carvalho Chehab Jan. 3, 2013, 4:29 p.m. UTC | #8
Em Thu, 3 Jan 2013 14:14:29 -0200
Mauro Carvalho Chehab <mchehab@redhat.com> escreveu:

> Em Thu, 03 Jan 2013 16:18:26 +0100
> Klaus Schmidinger <Klaus.Schmidinger@tvdr.de> escreveu:
> 
> > On 03.01.2013 14:20, Mauro Carvalho Chehab wrote:
> > > Em Wed, 2 Jan 2013 00:38:50 +0530
> > > Manu Abraham <abraham.manu@gmail.com> escreveu:
> > >
> > >> On Tue, Jan 1, 2013 at 10:59 PM, Mauro Carvalho Chehab
> > >> <mchehab@redhat.com> wrote:
> > >>> Em Tue, 1 Jan 2013 22:18:49 +0530
> > >>> Manu Abraham <abraham.manu@gmail.com> escreveu:
> > >>>
> > >>>> On Tue, Jan 1, 2013 at 8:30 PM, Mauro Carvalho Chehab
> > >>>> <mchehab@redhat.com> wrote:
> > >>>>
> > >>>>> [RFCv4] dvb: Add DVBv5 properties for quality parameters
> > >>>>>
> > >>>>> The DVBv3 quality parameters are limited on several ways:
> > >>>>>          - Doesn't provide any way to indicate the used measure;
> > >>>>>          - Userspace need to guess how to calculate the measure;
> > >>>>>          - Only a limited set of stats are supported;
> > >>>>>          - Doesn't provide QoS measure for the OFDM TPS/TMCC
> > >>>>>            carriers, used to detect the network parameters for
> > >>>>>            DVB-T/ISDB-T;
> > >>>>>          - Can't be called in a way to require them to be filled
> > >>>>>            all at once (atomic reads from the hardware), with may
> > >>>>>            cause troubles on interpreting them on userspace;
> > >>>>>          - On some OFDM delivery systems, the carriers can be
> > >>>>>            independently modulated, having different properties.
> > >>>>>            Currently, there's no way to report per-layer stats;
> > >>>>
> > >>>> per layer stats is a mythical bird, nothing of that sort does exist.
> > >>>
> > >>> Had you ever read or tried to get stats from an ISDB-T demod? If you
> > >>> had, you would see that it only provides per-layer stats. Btw, this is
> > >>> a requirement to follow the ARIB and ABNT ISDB specs.
> > >>
> > >> I understand you keep writing junk for ages, but nevertheless:
> > >>
> > >> Do you have any idea what's a BBHEADER (DVB-S2) or
> > >> PLHEADER (DVB-T2) ? The headers do indicate what MODCOD
> > >> (aka Modulation/Coding Standard follows, whatever mode ACM,
> > >> VCM or CCM) follows. These MODCOD foolows a TDM approach
> > >> with a hierarchial modulation principle. This is exactly what ISDB
> > >> does too.
> > >
> > > No, I didn't check DVB-S2/T2 specs deeply enough to understand
> > > if they're doing the same thing as ISDB.
> > >
> > > Yet, ISDB-T doesn't use a TDM approach for hierarchical modulation.
> > > It uses a FDM (OFDM is a type of Frequency Division Multiplexing).
> > >
> > > So, if you're saying that DVB-S2 uses TDM, it is very different than
> > > ISDB-T. As DVB-T2 uses an FDM type of modulation (OFDM), it would
> > > be possible to segment the carriers there, just like ISDB, or to
> > > use TDM hierarchical modulation techniques.
> > >
> > >>
> > >> And for your info:
> > >>
> > >> " The TMCC control information is
> > >> common to all TMCC carriers and
> > >> error correction is performed by using
> > >> difference-set cyclic code."
> > >
> > > Yes, TMCC carriers are equal and they are always modulated using DBPSK.
> > > That is done to make it possible to receive the TMCC carriers even under
> > > worse SNR conditions, where it may not be possible to decode the segment
> > > groups.
> > >
> > > It seems that you completely missed the point though. On ISDB-T, the
> > > carriers that belong to each group of segments (except for the control
> > > carriers - carriers 1 to 107) uses a completely independent modulation.
> > > Also, as they're spaced in frequency, the interference of each segment
> > > is different. So, error indications are different on each segment.
> > >
> > > Btw, in any case, the datasheets of ISDB-T demods clearly shows that
> > > the BER measures are per segment group (layer).
> > >
> > > For example, for the BER measures before Viterbi, those are the register
> > > names for a certain demod:
> > >
> > > 	VBERSNUMA Bit count of BER measurement before Viterbi in A layer
> > > 	VBERSNUMB Bit count of BER measurement before Viterbi in B layer
> > > 	VBERSNUMC Bit count of BER measurement before Viterbi in C layer
> > >
> > > It has another set of registers for BER after Viterbi, and for PER after
> > > Viterbi and RS, for bit count errors, etc.
> > >
> > > There's no way to get any type of "global" BER measure, simply because
> > > ISDB-T demods don't provide.
> > 
> > Maybe we should put all this theoretical discussion aside for the moment and
> > think about what is *really* needed by real world applications. As with any
> > receiver, VDR simply wants to have some measure of the signal's "strength"
> > and "quality". These are just two values that should be delivered by each
> > frontend/demux, using the *same* defined and mandatory range. I don't care
> > what exactly that is, but it needs to be the same for all devices.
> > What values a particular driver uses internally to come up with these
> > is of no interest to VDR. The "signal strength" might just be what is
> > currently returned through FE_READ_SIGNAL_STRENGTH (however, normalized to
> > the same range in all drivers, which currently is not the case). The "signal
> > quality" might use flags like FE_HAS_SIGNAL, FE_HAS_CARRIER, FE_HAS_VITERBI,
> > FE_HAS_SYNC, as well as SNR, BER and UNC (if available) to form some
> > value where 0 means no quality at all, and 0xFFFF means excellent quality.
> > If a particular frontend/demux uses totally different concepts, it can
> > just use whatever it deems reasonable to form the "strength" and "quality"
> > values. The important thing here is just that all this needs to be hidden
> > inside the driver, and the only interface to an application are ioctl()
> > calls that return these two values.
> > 
> > So I suggest that you define this minimal interface and allow applications
> > to retrieve what they really need. Once this is done, feel free to implement
> > whatever theoretical bells and whistles you fell like doing - that's all
> > fine with me, as long as the really important stuff keeps working ;-)
> 
> Klaus,
> 
> On ISDB-T, it splits the TS into (up to) three independent physical channels
> (called layers).
> 
> Each channel has its own statistics, as they're completely independent:
> they use different inner FEC's, use different modulations, etc.
> 
> The ISDB demods don't provide a single value for the 3 layers. They
> can't, as they're independent. So, signal-strengh and SNR measures are
> also independent for each of those 3 layers.
> 
> A typical ISDB transmission uses 13 segments of carriers, each segment
> using a 4.28 kHz bandwidth, grouped into 3 layers. While it is up to 
> the broadcaster to decide how to group the segments, it is typically 
> arranged like that:
> 
> 	layer A - 1 segment for LD programs - modulated using QPSK;
> 	layer B - 3 segments for SD programs - modulated using 16QAM;
> 	layer C - 9 segments for HD programs - modulated using 64QAM.
> 
> The TDM TS packets from the Transport Stream are broken into those 3
> layers, each being an independent transmission channel.
> 
> So, all channel level QoS measure are per-layer (SNR, signal strength,
> BER, MER, ...).
> 
> While the demods I have datasheets here don't provide it, it would be
> possible to provide error counts for a given program ID, by summing
> the error count that applies to each PID.
> 
> So, let's assume, for example, that the UCB count is:
> 	layer A = 0
> 	layer B = 12
> 	layer C = 30
> 
> an 1-seg LD program will have 0 uncorrected blocks;
> an SD program will have 12 uncorrected error blocks;
> a HD program will have 42 uncorrected error blocks.
> 
> It shouldn't be that hard to take it into account on userspace, but
> doing it at kernel level would be very painful, if possible, as
> kernelspace would be required to know what PID's are being shown, in
> order to estimate the error count measures for them. Also, it would
> require a much more complex kernelspace-userspace interface.

Two additional notes:

1) If you want to get further information, it is available on ARIB
	STD-B31 spec:

	http://www.arib.or.jp/english/html/overview/doc/6-STD-B31v1_6-E2.pdf

There, table 3-2 shows the main characteristics of the modulation;
how the 3 independent channels are handled and fig. 3.4
shows a simplified diagram to give an idea on how the hierarchical TS
packets are broken into the 3 layers

2) There are in the market some narrow-band decoders. Those tunes only
1 segment (440kHz), and are meant to be used on mobile devices that can
receive only LD programs. Only for those devices, it is possible to
offer a single set of statistics (SNR, strength, BER, UCB, etc), 
because it can decode just one layer. I have a few of them here,
and we have 2 drivers for those 1-seg devices (s921 and siano).
The full-seg drivers currently provide crappy information or don't
provide any QoS stats at all due to the lack of a proper API.

Regards,
Mauro
--
To unsubscribe from this list: send the line "unsubscribe linux-media" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Manu Abraham Jan. 3, 2013, 7:09 p.m. UTC | #9
Hi Antti,

On Thu, Jan 3, 2013 at 9:04 PM, Antti Palosaari <crope@iki.fi> wrote:
> On 01/01/2013 06:48 PM, Manu Abraham wrote:
>>
>> On Tue, Jan 1, 2013 at 8:30 PM, Mauro Carvalho Chehab
>> <mchehab@redhat.com> wrote:
>>
>>> [RFCv4] dvb: Add DVBv5 properties for quality parameters
>>>
>>> The DVBv3 quality parameters are limited on several ways:
>>>          - Doesn't provide any way to indicate the used measure;
>>>          - Userspace need to guess how to calculate the measure;
>>>          - Only a limited set of stats are supported;
>>>          - Doesn't provide QoS measure for the OFDM TPS/TMCC
>>>            carriers, used to detect the network parameters for
>>>            DVB-T/ISDB-T;
>>>          - Can't be called in a way to require them to be filled
>>>            all at once (atomic reads from the hardware), with may
>>>            cause troubles on interpreting them on userspace;
>>>          - On some OFDM delivery systems, the carriers can be
>>>            independently modulated, having different properties.
>>>            Currently, there's no way to report per-layer stats;
>>
>>
>> per layer stats is a mythical bird, nothing of that sort does exist. If
>> some
>> driver states that it is simply due to lack of knowledge at the coding
>> side.
>>
>> ISDB-T uses hierarchial modulation, just like DVB-S2 or DVB-T2
>
>
> Manu, you confused now two concept (which are aimed to resolve same real
> life problem) - hierarchical coding and multiple transport stream. Both are
> quite similar on lower level of radio channel, but differs on upper levels.
>
> Hierarchical is a little bit weird baby as it remuxes those lower lever
> radio channels (called layers in case of ISDB-T) to one single mux!

That is not really correct. There is one single OFDM channel, the layers
are processed via hierarchial separation. Stuffing exists, to maintain
constant rate.

http://farm9.staticflickr.com/8077/8343296328_e1e375b519_b_d.jpg

When rate is constant within the same channel..
(The only case what I can think parameters could be different with a
constant rate,
is that stuffing frames are unaccounted for. Most likely a bug ?)

Manu
--
To unsubscribe from this list: send the line "unsubscribe linux-media" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Mauro Carvalho Chehab Jan. 3, 2013, 7:27 p.m. UTC | #10
Em Fri, 4 Jan 2013 00:39:25 +0530
Manu Abraham <abraham.manu@gmail.com> escreveu:

> Hi Antti,
> 
> On Thu, Jan 3, 2013 at 9:04 PM, Antti Palosaari <crope@iki.fi> wrote:
> > On 01/01/2013 06:48 PM, Manu Abraham wrote:
> >>
> >> On Tue, Jan 1, 2013 at 8:30 PM, Mauro Carvalho Chehab
> >> <mchehab@redhat.com> wrote:
> >>
> >>> [RFCv4] dvb: Add DVBv5 properties for quality parameters
> >>>
> >>> The DVBv3 quality parameters are limited on several ways:
> >>>          - Doesn't provide any way to indicate the used measure;
> >>>          - Userspace need to guess how to calculate the measure;
> >>>          - Only a limited set of stats are supported;
> >>>          - Doesn't provide QoS measure for the OFDM TPS/TMCC
> >>>            carriers, used to detect the network parameters for
> >>>            DVB-T/ISDB-T;
> >>>          - Can't be called in a way to require them to be filled
> >>>            all at once (atomic reads from the hardware), with may
> >>>            cause troubles on interpreting them on userspace;
> >>>          - On some OFDM delivery systems, the carriers can be
> >>>            independently modulated, having different properties.
> >>>            Currently, there's no way to report per-layer stats;
> >>
> >>
> >> per layer stats is a mythical bird, nothing of that sort does exist. If
> >> some
> >> driver states that it is simply due to lack of knowledge at the coding
> >> side.
> >>
> >> ISDB-T uses hierarchial modulation, just like DVB-S2 or DVB-T2
> >
> >
> > Manu, you confused now two concept (which are aimed to resolve same real
> > life problem) - hierarchical coding and multiple transport stream. Both are
> > quite similar on lower level of radio channel, but differs on upper levels.
> >
> > Hierarchical is a little bit weird baby as it remuxes those lower lever
> > radio channels (called layers in case of ISDB-T) to one single mux!
> 
> That is not really correct. There is one single OFDM channel, the layers
> are processed via hierarchial separation. Stuffing exists, to maintain
> constant rate.
> 
> http://farm9.staticflickr.com/8077/8343296328_e1e375b519_b_d.jpg
> 
> When rate is constant within the same channel..
> (The only case what I can think parameters could be different with a
> constant rate,
> is that stuffing frames are unaccounted for. Most likely a bug ?)

What did you smoke? That picture has nothing to do with ISDB!

ISDB not only does hierarchical split. It also splits the OFDM carriers
into 3 layers, each layer with its own modulation, guard interval, inner
FEC, etc. Each of those layers behave as an independent channel,
providing different bit rates.


Cheers,
Mauro
--
To unsubscribe from this list: send the line "unsubscribe linux-media" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Manu Abraham Jan. 3, 2013, 7:32 p.m. UTC | #11
On Fri, Jan 4, 2013 at 12:57 AM, Mauro Carvalho Chehab
<mchehab@redhat.com> wrote:
> Em Fri, 4 Jan 2013 00:39:25 +0530
> Manu Abraham <abraham.manu@gmail.com> escreveu:
>
>> Hi Antti,
>>
>> On Thu, Jan 3, 2013 at 9:04 PM, Antti Palosaari <crope@iki.fi> wrote:
>> > On 01/01/2013 06:48 PM, Manu Abraham wrote:
>> >>
>> >> On Tue, Jan 1, 2013 at 8:30 PM, Mauro Carvalho Chehab
>> >> <mchehab@redhat.com> wrote:
>> >>
>> >>> [RFCv4] dvb: Add DVBv5 properties for quality parameters
>> >>>
>> >>> The DVBv3 quality parameters are limited on several ways:
>> >>>          - Doesn't provide any way to indicate the used measure;
>> >>>          - Userspace need to guess how to calculate the measure;
>> >>>          - Only a limited set of stats are supported;
>> >>>          - Doesn't provide QoS measure for the OFDM TPS/TMCC
>> >>>            carriers, used to detect the network parameters for
>> >>>            DVB-T/ISDB-T;
>> >>>          - Can't be called in a way to require them to be filled
>> >>>            all at once (atomic reads from the hardware), with may
>> >>>            cause troubles on interpreting them on userspace;
>> >>>          - On some OFDM delivery systems, the carriers can be
>> >>>            independently modulated, having different properties.
>> >>>            Currently, there's no way to report per-layer stats;
>> >>
>> >>
>> >> per layer stats is a mythical bird, nothing of that sort does exist. If
>> >> some
>> >> driver states that it is simply due to lack of knowledge at the coding
>> >> side.
>> >>
>> >> ISDB-T uses hierarchial modulation, just like DVB-S2 or DVB-T2
>> >
>> >
>> > Manu, you confused now two concept (which are aimed to resolve same real
>> > life problem) - hierarchical coding and multiple transport stream. Both are
>> > quite similar on lower level of radio channel, but differs on upper levels.
>> >
>> > Hierarchical is a little bit weird baby as it remuxes those lower lever
>> > radio channels (called layers in case of ISDB-T) to one single mux!
>>
>> That is not really correct. There is one single OFDM channel, the layers
>> are processed via hierarchial separation. Stuffing exists, to maintain
>> constant rate.
>>
>> http://farm9.staticflickr.com/8077/8343296328_e1e375b519_b_d.jpg
>>
>> When rate is constant within the same channel..
>> (The only case what I can think parameters could be different with a
>> constant rate,
>> is that stuffing frames are unaccounted for. Most likely a bug ?)
>
> What did you smoke? That picture has nothing to do with ISDB!
>

ARIB STD – B31
Version 1.6-E2
?17?
Fig. 3-2 shows the basic configuration of the channel coding.

It just shows, you understand crap.

Manu
--
To unsubscribe from this list: send the line "unsubscribe linux-media" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Mauro Carvalho Chehab Jan. 3, 2013, 7:53 p.m. UTC | #12
Em Fri, 4 Jan 2013 01:02:02 +0530
Manu Abraham <abraham.manu@gmail.com> escreveu:

> On Fri, Jan 4, 2013 at 12:57 AM, Mauro Carvalho Chehab
> <mchehab@redhat.com> wrote:
> > Em Fri, 4 Jan 2013 00:39:25 +0530
> > Manu Abraham <abraham.manu@gmail.com> escreveu:
> >
> >> Hi Antti,
> >>
> >> On Thu, Jan 3, 2013 at 9:04 PM, Antti Palosaari <crope@iki.fi> wrote:
> >> > On 01/01/2013 06:48 PM, Manu Abraham wrote:
> >> >>
> >> >> On Tue, Jan 1, 2013 at 8:30 PM, Mauro Carvalho Chehab
> >> >> <mchehab@redhat.com> wrote:
> >> >>
> >> >>> [RFCv4] dvb: Add DVBv5 properties for quality parameters
> >> >>>
> >> >>> The DVBv3 quality parameters are limited on several ways:
> >> >>>          - Doesn't provide any way to indicate the used measure;
> >> >>>          - Userspace need to guess how to calculate the measure;
> >> >>>          - Only a limited set of stats are supported;
> >> >>>          - Doesn't provide QoS measure for the OFDM TPS/TMCC
> >> >>>            carriers, used to detect the network parameters for
> >> >>>            DVB-T/ISDB-T;
> >> >>>          - Can't be called in a way to require them to be filled
> >> >>>            all at once (atomic reads from the hardware), with may
> >> >>>            cause troubles on interpreting them on userspace;
> >> >>>          - On some OFDM delivery systems, the carriers can be
> >> >>>            independently modulated, having different properties.
> >> >>>            Currently, there's no way to report per-layer stats;
> >> >>
> >> >>
> >> >> per layer stats is a mythical bird, nothing of that sort does exist. If
> >> >> some
> >> >> driver states that it is simply due to lack of knowledge at the coding
> >> >> side.
> >> >>
> >> >> ISDB-T uses hierarchial modulation, just like DVB-S2 or DVB-T2
> >> >
> >> >
> >> > Manu, you confused now two concept (which are aimed to resolve same real
> >> > life problem) - hierarchical coding and multiple transport stream. Both are
> >> > quite similar on lower level of radio channel, but differs on upper levels.
> >> >
> >> > Hierarchical is a little bit weird baby as it remuxes those lower lever
> >> > radio channels (called layers in case of ISDB-T) to one single mux!
> >>
> >> That is not really correct. There is one single OFDM channel, the layers
> >> are processed via hierarchial separation. Stuffing exists, to maintain
> >> constant rate.
> >>
> >> http://farm9.staticflickr.com/8077/8343296328_e1e375b519_b_d.jpg
> >>
> >> When rate is constant within the same channel..
> >> (The only case what I can think parameters could be different with a
> >> constant rate,
> >> is that stuffing frames are unaccounted for. Most likely a bug ?)
> >
> > What did you smoke? That picture has nothing to do with ISDB!
> >
> 
> ARIB STD – B31
> Version 1.6-E2
> ?17?
> Fig. 3-2 shows the basic configuration of the channel coding.
> 
> It just shows, you understand crap.

That is the picture you need to look, not the random one you picked.
It clearly shows there that, after the hierarchical coding done by
the "Division of TS into hierarchical levels", the TS packets are
split into 3 independent channels, each with its own convolutional
coding, carrier modulation, etc.

This picture shows how each program is split at the FDM sub-carriers:
	http://en.wikipedia.org/wiki/File:ISDB-T_CH_Seg_Prog_allocation.jpg.svg

There, LD programs are at segment 0 (S0). HD programs use 12 segments
and SD programs use 4 segments.

As each segment group has a different spectrum (as they're using FDM),
and are modulated with different encoding schemas (modulation type, FEC,
etc), they have different QoS measures.

Segment 0 (the one at the center of the spectrum) is less sensitive to
inter-channel interference. That's why it is used for LD programs.


Cheers,
Mauro
--
To unsubscribe from this list: send the line "unsubscribe linux-media" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Manu Abraham Jan. 3, 2013, 8:26 p.m. UTC | #13
On Thu, Jan 3, 2013 at 6:50 PM, Mauro Carvalho Chehab
<mchehab@redhat.com> wrote:
> Em Wed, 2 Jan 2013 00:38:50 +0530
> Manu Abraham <abraham.manu@gmail.com> escreveu:
>
>> On Tue, Jan 1, 2013 at 10:59 PM, Mauro Carvalho Chehab
>> <mchehab@redhat.com> wrote:
>> > Em Tue, 1 Jan 2013 22:18:49 +0530
>> > Manu Abraham <abraham.manu@gmail.com> escreveu:
>> >
>> >> On Tue, Jan 1, 2013 at 8:30 PM, Mauro Carvalho Chehab
>> >> <mchehab@redhat.com> wrote:
>> >>
>> >> > [RFCv4] dvb: Add DVBv5 properties for quality parameters
>> >> >
>> >> > The DVBv3 quality parameters are limited on several ways:
>> >> >         - Doesn't provide any way to indicate the used measure;
>> >> >         - Userspace need to guess how to calculate the measure;
>> >> >         - Only a limited set of stats are supported;
>> >> >         - Doesn't provide QoS measure for the OFDM TPS/TMCC
>> >> >           carriers, used to detect the network parameters for
>> >> >           DVB-T/ISDB-T;
>> >> >         - Can't be called in a way to require them to be filled
>> >> >           all at once (atomic reads from the hardware), with may
>> >> >           cause troubles on interpreting them on userspace;
>> >> >         - On some OFDM delivery systems, the carriers can be
>> >> >           independently modulated, having different properties.
>> >> >           Currently, there's no way to report per-layer stats;
>> >>
>> >> per layer stats is a mythical bird, nothing of that sort does exist.
>> >
>> > Had you ever read or tried to get stats from an ISDB-T demod? If you
>> > had, you would see that it only provides per-layer stats. Btw, this is
>> > a requirement to follow the ARIB and ABNT ISDB specs.
>>
>> I understand you keep writing junk for ages, but nevertheless:
>>
>> Do you have any idea what's a BBHEADER (DVB-S2) or
>> PLHEADER (DVB-T2) ? The headers do indicate what MODCOD
>> (aka Modulation/Coding Standard follows, whatever mode ACM,
>> VCM or CCM) follows. These MODCOD foolows a TDM approach
>> with a hierarchial modulation principle. This is exactly what ISDB
>> does too.
>
> No, I didn't check DVB-S2/T2 specs deeply enough to understand
> if they're doing the same thing as ISDB.
>
> Yet, ISDB-T doesn't use a TDM approach for hierarchical modulation.
> It uses a FDM (OFDM is a type of Frequency Division Multiplexing).

• ISDB?T uses a modulation method referred to as Band
Segmented OFDM Transmission with Time Interleave.

Definition: Time Division Multiplexing (TDM) is the time interleaving of
samples from several sources so that the information from these sources
can be transmitted serially over a single communication channel.

Manu
--
To unsubscribe from this list: send the line "unsubscribe linux-media" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Antti Palosaari Jan. 3, 2013, 8:39 p.m. UTC | #14
On 01/03/2013 09:53 PM, Mauro Carvalho Chehab wrote:
> Em Fri, 4 Jan 2013 01:02:02 +0530
> Manu Abraham <abraham.manu@gmail.com> escreveu:
>
>> On Fri, Jan 4, 2013 at 12:57 AM, Mauro Carvalho Chehab
>> <mchehab@redhat.com> wrote:
>>> Em Fri, 4 Jan 2013 00:39:25 +0530
>>> Manu Abraham <abraham.manu@gmail.com> escreveu:
>>>
>>>> Hi Antti,
>>>>
>>>> On Thu, Jan 3, 2013 at 9:04 PM, Antti Palosaari <crope@iki.fi> wrote:
>>>>> On 01/01/2013 06:48 PM, Manu Abraham wrote:
>>>>>>
>>>>>> On Tue, Jan 1, 2013 at 8:30 PM, Mauro Carvalho Chehab
>>>>>> <mchehab@redhat.com> wrote:
>>>>>>
>>>>>>> [RFCv4] dvb: Add DVBv5 properties for quality parameters
>>>>>>>
>>>>>>> The DVBv3 quality parameters are limited on several ways:
>>>>>>>           - Doesn't provide any way to indicate the used measure;
>>>>>>>           - Userspace need to guess how to calculate the measure;
>>>>>>>           - Only a limited set of stats are supported;
>>>>>>>           - Doesn't provide QoS measure for the OFDM TPS/TMCC
>>>>>>>             carriers, used to detect the network parameters for
>>>>>>>             DVB-T/ISDB-T;
>>>>>>>           - Can't be called in a way to require them to be filled
>>>>>>>             all at once (atomic reads from the hardware), with may
>>>>>>>             cause troubles on interpreting them on userspace;
>>>>>>>           - On some OFDM delivery systems, the carriers can be
>>>>>>>             independently modulated, having different properties.
>>>>>>>             Currently, there's no way to report per-layer stats;
>>>>>>
>>>>>>
>>>>>> per layer stats is a mythical bird, nothing of that sort does exist. If
>>>>>> some
>>>>>> driver states that it is simply due to lack of knowledge at the coding
>>>>>> side.
>>>>>>
>>>>>> ISDB-T uses hierarchial modulation, just like DVB-S2 or DVB-T2
>>>>>
>>>>>
>>>>> Manu, you confused now two concept (which are aimed to resolve same real
>>>>> life problem) - hierarchical coding and multiple transport stream. Both are
>>>>> quite similar on lower level of radio channel, but differs on upper levels.
>>>>>
>>>>> Hierarchical is a little bit weird baby as it remuxes those lower lever
>>>>> radio channels (called layers in case of ISDB-T) to one single mux!
>>>>
>>>> That is not really correct. There is one single OFDM channel, the layers
>>>> are processed via hierarchial separation. Stuffing exists, to maintain
>>>> constant rate.
>>>>
>>>> http://farm9.staticflickr.com/8077/8343296328_e1e375b519_b_d.jpg
>>>>
>>>> When rate is constant within the same channel..
>>>> (The only case what I can think parameters could be different with a
>>>> constant rate,
>>>> is that stuffing frames are unaccounted for. Most likely a bug ?)
>>>
>>> What did you smoke? That picture has nothing to do with ISDB!
>>>
>>
>> ARIB STD – B31
>> Version 1.6-E2
>> ?17?
>> Fig. 3-2 shows the basic configuration of the channel coding.
>>
>> It just shows, you understand crap.
>
> That is the picture you need to look, not the random one you picked.
> It clearly shows there that, after the hierarchical coding done by
> the "Division of TS into hierarchical levels", the TS packets are
> split into 3 independent channels, each with its own convolutional
> coding, carrier modulation, etc.
>
> This picture shows how each program is split at the FDM sub-carriers:
> 	http://en.wikipedia.org/wiki/File:ISDB-T_CH_Seg_Prog_allocation.jpg.svg
>
> There, LD programs are at segment 0 (S0). HD programs use 12 segments
> and SD programs use 4 segments.
>
> As each segment group has a different spectrum (as they're using FDM),
> and are modulated with different encoding schemas (modulation type, FEC,
> etc), they have different QoS measures.
>
> Segment 0 (the one at the center of the spectrum) is less sensitive to
> inter-channel interference. That's why it is used for LD programs.
>
>
> Cheers,
> Mauro
>


Manu, here is manual of the professional ISDB-T signal analyzer. Look 
especially BER measurement picture from "Slide 10".

I think you don't bother to say Anritsu MS8901A ISDB-T Digital Broadcast 
Signal Analyzer, which street price is $20,000, does not know how to 
measure ISDB-T statistics....

http://downloadfiles.anritsu.com/Files/en-US/Product-Introductions/Product-Introduction/MS8901A_EL1100.pdf


Antti
Manu Abraham Jan. 3, 2013, 8:54 p.m. UTC | #15
On Fri, Jan 4, 2013 at 2:09 AM, Antti Palosaari <crope@iki.fi> wrote:

>
> Manu, here is manual of the professional ISDB-T signal analyzer. Look
> especially BER measurement picture from "Slide 10".

Sure, it looks so. Just wondering what the TDM stuffing would do after
the hierarchial combiner.

Manu
--
To unsubscribe from this list: send the line "unsubscribe linux-media" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Antti Palosaari Jan. 3, 2013, 9:33 p.m. UTC | #16
On 01/03/2013 06:29 PM, Mauro Carvalho Chehab wrote:
> Em Thu, 3 Jan 2013 14:14:29 -0200
> Mauro Carvalho Chehab <mchehab@redhat.com> escreveu:
>
>> Em Thu, 03 Jan 2013 16:18:26 +0100
>> Klaus Schmidinger <Klaus.Schmidinger@tvdr.de> escreveu:
>>
>>> On 03.01.2013 14:20, Mauro Carvalho Chehab wrote:
>>>> Em Wed, 2 Jan 2013 00:38:50 +0530
>>>> Manu Abraham <abraham.manu@gmail.com> escreveu:
>>>>
>>>>> On Tue, Jan 1, 2013 at 10:59 PM, Mauro Carvalho Chehab
>>>>> <mchehab@redhat.com> wrote:
>>>>>> Em Tue, 1 Jan 2013 22:18:49 +0530
>>>>>> Manu Abraham <abraham.manu@gmail.com> escreveu:
>>>>>>
>>>>>>> On Tue, Jan 1, 2013 at 8:30 PM, Mauro Carvalho Chehab
>>>>>>> <mchehab@redhat.com> wrote:
>>>>>>>
>>>>>>>> [RFCv4] dvb: Add DVBv5 properties for quality parameters
>>>>>>>>
>>>>>>>> The DVBv3 quality parameters are limited on several ways:
>>>>>>>>           - Doesn't provide any way to indicate the used measure;
>>>>>>>>           - Userspace need to guess how to calculate the measure;
>>>>>>>>           - Only a limited set of stats are supported;
>>>>>>>>           - Doesn't provide QoS measure for the OFDM TPS/TMCC
>>>>>>>>             carriers, used to detect the network parameters for
>>>>>>>>             DVB-T/ISDB-T;
>>>>>>>>           - Can't be called in a way to require them to be filled
>>>>>>>>             all at once (atomic reads from the hardware), with may
>>>>>>>>             cause troubles on interpreting them on userspace;
>>>>>>>>           - On some OFDM delivery systems, the carriers can be
>>>>>>>>             independently modulated, having different properties.
>>>>>>>>             Currently, there's no way to report per-layer stats;
>>>>>>>
>>>>>>> per layer stats is a mythical bird, nothing of that sort does exist.
>>>>>>
>>>>>> Had you ever read or tried to get stats from an ISDB-T demod? If you
>>>>>> had, you would see that it only provides per-layer stats. Btw, this is
>>>>>> a requirement to follow the ARIB and ABNT ISDB specs.
>>>>>
>>>>> I understand you keep writing junk for ages, but nevertheless:
>>>>>
>>>>> Do you have any idea what's a BBHEADER (DVB-S2) or
>>>>> PLHEADER (DVB-T2) ? The headers do indicate what MODCOD
>>>>> (aka Modulation/Coding Standard follows, whatever mode ACM,
>>>>> VCM or CCM) follows. These MODCOD foolows a TDM approach
>>>>> with a hierarchial modulation principle. This is exactly what ISDB
>>>>> does too.
>>>>
>>>> No, I didn't check DVB-S2/T2 specs deeply enough to understand
>>>> if they're doing the same thing as ISDB.
>>>>
>>>> Yet, ISDB-T doesn't use a TDM approach for hierarchical modulation.
>>>> It uses a FDM (OFDM is a type of Frequency Division Multiplexing).
>>>>
>>>> So, if you're saying that DVB-S2 uses TDM, it is very different than
>>>> ISDB-T. As DVB-T2 uses an FDM type of modulation (OFDM), it would
>>>> be possible to segment the carriers there, just like ISDB, or to
>>>> use TDM hierarchical modulation techniques.
>>>>
>>>>>
>>>>> And for your info:
>>>>>
>>>>> " The TMCC control information is
>>>>> common to all TMCC carriers and
>>>>> error correction is performed by using
>>>>> difference-set cyclic code."
>>>>
>>>> Yes, TMCC carriers are equal and they are always modulated using DBPSK.
>>>> That is done to make it possible to receive the TMCC carriers even under
>>>> worse SNR conditions, where it may not be possible to decode the segment
>>>> groups.
>>>>
>>>> It seems that you completely missed the point though. On ISDB-T, the
>>>> carriers that belong to each group of segments (except for the control
>>>> carriers - carriers 1 to 107) uses a completely independent modulation.
>>>> Also, as they're spaced in frequency, the interference of each segment
>>>> is different. So, error indications are different on each segment.
>>>>
>>>> Btw, in any case, the datasheets of ISDB-T demods clearly shows that
>>>> the BER measures are per segment group (layer).
>>>>
>>>> For example, for the BER measures before Viterbi, those are the register
>>>> names for a certain demod:
>>>>
>>>> 	VBERSNUMA Bit count of BER measurement before Viterbi in A layer
>>>> 	VBERSNUMB Bit count of BER measurement before Viterbi in B layer
>>>> 	VBERSNUMC Bit count of BER measurement before Viterbi in C layer
>>>>
>>>> It has another set of registers for BER after Viterbi, and for PER after
>>>> Viterbi and RS, for bit count errors, etc.
>>>>
>>>> There's no way to get any type of "global" BER measure, simply because
>>>> ISDB-T demods don't provide.
>>>
>>> Maybe we should put all this theoretical discussion aside for the moment and
>>> think about what is *really* needed by real world applications. As with any
>>> receiver, VDR simply wants to have some measure of the signal's "strength"
>>> and "quality". These are just two values that should be delivered by each
>>> frontend/demux, using the *same* defined and mandatory range. I don't care
>>> what exactly that is, but it needs to be the same for all devices.
>>> What values a particular driver uses internally to come up with these
>>> is of no interest to VDR. The "signal strength" might just be what is
>>> currently returned through FE_READ_SIGNAL_STRENGTH (however, normalized to
>>> the same range in all drivers, which currently is not the case). The "signal
>>> quality" might use flags like FE_HAS_SIGNAL, FE_HAS_CARRIER, FE_HAS_VITERBI,
>>> FE_HAS_SYNC, as well as SNR, BER and UNC (if available) to form some
>>> value where 0 means no quality at all, and 0xFFFF means excellent quality.
>>> If a particular frontend/demux uses totally different concepts, it can
>>> just use whatever it deems reasonable to form the "strength" and "quality"
>>> values. The important thing here is just that all this needs to be hidden
>>> inside the driver, and the only interface to an application are ioctl()
>>> calls that return these two values.
>>>
>>> So I suggest that you define this minimal interface and allow applications
>>> to retrieve what they really need. Once this is done, feel free to implement
>>> whatever theoretical bells and whistles you fell like doing - that's all
>>> fine with me, as long as the really important stuff keeps working ;-)
>>
>> Klaus,
>>
>> On ISDB-T, it splits the TS into (up to) three independent physical channels
>> (called layers).
>>
>> Each channel has its own statistics, as they're completely independent:
>> they use different inner FEC's, use different modulations, etc.
>>
>> The ISDB demods don't provide a single value for the 3 layers. They
>> can't, as they're independent. So, signal-strengh and SNR measures are
>> also independent for each of those 3 layers.
>>
>> A typical ISDB transmission uses 13 segments of carriers, each segment
>> using a 4.28 kHz bandwidth, grouped into 3 layers. While it is up to
>> the broadcaster to decide how to group the segments, it is typically
>> arranged like that:
>>
>> 	layer A - 1 segment for LD programs - modulated using QPSK;
>> 	layer B - 3 segments for SD programs - modulated using 16QAM;
>> 	layer C - 9 segments for HD programs - modulated using 64QAM.
>>
>> The TDM TS packets from the Transport Stream are broken into those 3
>> layers, each being an independent transmission channel.
>>
>> So, all channel level QoS measure are per-layer (SNR, signal strength,
>> BER, MER, ...).
>>
>> While the demods I have datasheets here don't provide it, it would be
>> possible to provide error counts for a given program ID, by summing
>> the error count that applies to each PID.
>>
>> So, let's assume, for example, that the UCB count is:
>> 	layer A = 0
>> 	layer B = 12
>> 	layer C = 30
>>
>> an 1-seg LD program will have 0 uncorrected blocks;
>> an SD program will have 12 uncorrected error blocks;
>> a HD program will have 42 uncorrected error blocks.
>>
>> It shouldn't be that hard to take it into account on userspace, but
>> doing it at kernel level would be very painful, if possible, as
>> kernelspace would be required to know what PID's are being shown, in
>> order to estimate the error count measures for them. Also, it would
>> require a much more complex kernelspace-userspace interface.
>
> Two additional notes:
>
> 1) If you want to get further information, it is available on ARIB
> 	STD-B31 spec:
>
> 	http://www.arib.or.jp/english/html/overview/doc/6-STD-B31v1_6-E2.pdf
>
> There, table 3-2 shows the main characteristics of the modulation;
> how the 3 independent channels are handled and fig. 3.4
> shows a simplified diagram to give an idea on how the hierarchical TS
> packets are broken into the 3 layers
>
> 2) There are in the market some narrow-band decoders. Those tunes only
> 1 segment (440kHz), and are meant to be used on mobile devices that can
> receive only LD programs. Only for those devices, it is possible to
> offer a single set of statistics (SNR, strength, BER, UCB, etc),
> because it can decode just one layer. I have a few of them here,
> and we have 2 drivers for those 1-seg devices (s921 and siano).
> The full-seg drivers currently provide crappy information or don't
> provide any QoS stats at all due to the lack of a proper API.
>
> Regards,
> Mauro

What I propose is quite near what Klaus wants. Just only new simple ways 
to report current statistics with beforehand scale/values.

1) Signal Strength
* linear scale 0-0xffff

2) Quality SNR
* SNR in resolution 0.1dB

3) Quality BER
* ~like currently (no exact units)
* measured from inner coding

4) Quality UCB
* ~like currently (no exact units)
* measured from outer coding (naturally)
* counter is increased over lifetime
* tune resets counter?
* driver is responsible of polling statistic in background and report 
from cache


I would not like to define exact units for BER and USB as those are 
quite hard to implement and also non-sense. User would like just to see 
if there is some (random) numbers and if those numbers are rising or 
reducing when he changes antenna or adjusts gain. We are not making a 
professional signal analyzers - numbers does not need to be 100% correctly.

ISDB-T statistics are forced also to that simple API. Calculating 
average value for example. Statistic differences between layers are so 
minor that users does not even care to know.

And as there is some persons who surely like to do QoS API like need of 
$10k professional equipment, I propose to add more accurate reports as 
alternative BUT that minimalist API should be offered even professional 
API exits.


regards
Antti
Mauro Carvalho Chehab Jan. 3, 2013, 10:18 p.m. UTC | #17
Em Thu, 03 Jan 2013 23:33:49 +0200
Antti Palosaari <crope@iki.fi> escreveu:

> On 01/03/2013 06:29 PM, Mauro Carvalho Chehab wrote:
> > Em Thu, 3 Jan 2013 14:14:29 -0200
> > Mauro Carvalho Chehab <mchehab@redhat.com> escreveu:
> >
> >> Em Thu, 03 Jan 2013 16:18:26 +0100
> >> Klaus Schmidinger <Klaus.Schmidinger@tvdr.de> escreveu:
> >>
> >>> On 03.01.2013 14:20, Mauro Carvalho Chehab wrote:
> >>>> Em Wed, 2 Jan 2013 00:38:50 +0530
> >>>> Manu Abraham <abraham.manu@gmail.com> escreveu:
> >>>>
> >>>>> On Tue, Jan 1, 2013 at 10:59 PM, Mauro Carvalho Chehab
> >>>>> <mchehab@redhat.com> wrote:
> >>>>>> Em Tue, 1 Jan 2013 22:18:49 +0530
> >>>>>> Manu Abraham <abraham.manu@gmail.com> escreveu:
> >>>>>>
> >>>>>>> On Tue, Jan 1, 2013 at 8:30 PM, Mauro Carvalho Chehab
> >>>>>>> <mchehab@redhat.com> wrote:
> >>>>>>>
> >>>>>>>> [RFCv4] dvb: Add DVBv5 properties for quality parameters
> >>>>>>>>
> >>>>>>>> The DVBv3 quality parameters are limited on several ways:
> >>>>>>>>           - Doesn't provide any way to indicate the used measure;
> >>>>>>>>           - Userspace need to guess how to calculate the measure;
> >>>>>>>>           - Only a limited set of stats are supported;
> >>>>>>>>           - Doesn't provide QoS measure for the OFDM TPS/TMCC
> >>>>>>>>             carriers, used to detect the network parameters for
> >>>>>>>>             DVB-T/ISDB-T;
> >>>>>>>>           - Can't be called in a way to require them to be filled
> >>>>>>>>             all at once (atomic reads from the hardware), with may
> >>>>>>>>             cause troubles on interpreting them on userspace;
> >>>>>>>>           - On some OFDM delivery systems, the carriers can be
> >>>>>>>>             independently modulated, having different properties.
> >>>>>>>>             Currently, there's no way to report per-layer stats;
> >>>>>>>
> >>>>>>> per layer stats is a mythical bird, nothing of that sort does exist.
> >>>>>>
> >>>>>> Had you ever read or tried to get stats from an ISDB-T demod? If you
> >>>>>> had, you would see that it only provides per-layer stats. Btw, this is
> >>>>>> a requirement to follow the ARIB and ABNT ISDB specs.
> >>>>>
> >>>>> I understand you keep writing junk for ages, but nevertheless:
> >>>>>
> >>>>> Do you have any idea what's a BBHEADER (DVB-S2) or
> >>>>> PLHEADER (DVB-T2) ? The headers do indicate what MODCOD
> >>>>> (aka Modulation/Coding Standard follows, whatever mode ACM,
> >>>>> VCM or CCM) follows. These MODCOD foolows a TDM approach
> >>>>> with a hierarchial modulation principle. This is exactly what ISDB
> >>>>> does too.
> >>>>
> >>>> No, I didn't check DVB-S2/T2 specs deeply enough to understand
> >>>> if they're doing the same thing as ISDB.
> >>>>
> >>>> Yet, ISDB-T doesn't use a TDM approach for hierarchical modulation.
> >>>> It uses a FDM (OFDM is a type of Frequency Division Multiplexing).
> >>>>
> >>>> So, if you're saying that DVB-S2 uses TDM, it is very different than
> >>>> ISDB-T. As DVB-T2 uses an FDM type of modulation (OFDM), it would
> >>>> be possible to segment the carriers there, just like ISDB, or to
> >>>> use TDM hierarchical modulation techniques.
> >>>>
> >>>>>
> >>>>> And for your info:
> >>>>>
> >>>>> " The TMCC control information is
> >>>>> common to all TMCC carriers and
> >>>>> error correction is performed by using
> >>>>> difference-set cyclic code."
> >>>>
> >>>> Yes, TMCC carriers are equal and they are always modulated using DBPSK.
> >>>> That is done to make it possible to receive the TMCC carriers even under
> >>>> worse SNR conditions, where it may not be possible to decode the segment
> >>>> groups.
> >>>>
> >>>> It seems that you completely missed the point though. On ISDB-T, the
> >>>> carriers that belong to each group of segments (except for the control
> >>>> carriers - carriers 1 to 107) uses a completely independent modulation.
> >>>> Also, as they're spaced in frequency, the interference of each segment
> >>>> is different. So, error indications are different on each segment.
> >>>>
> >>>> Btw, in any case, the datasheets of ISDB-T demods clearly shows that
> >>>> the BER measures are per segment group (layer).
> >>>>
> >>>> For example, for the BER measures before Viterbi, those are the register
> >>>> names for a certain demod:
> >>>>
> >>>> 	VBERSNUMA Bit count of BER measurement before Viterbi in A layer
> >>>> 	VBERSNUMB Bit count of BER measurement before Viterbi in B layer
> >>>> 	VBERSNUMC Bit count of BER measurement before Viterbi in C layer
> >>>>
> >>>> It has another set of registers for BER after Viterbi, and for PER after
> >>>> Viterbi and RS, for bit count errors, etc.
> >>>>
> >>>> There's no way to get any type of "global" BER measure, simply because
> >>>> ISDB-T demods don't provide.
> >>>
> >>> Maybe we should put all this theoretical discussion aside for the moment and
> >>> think about what is *really* needed by real world applications. As with any
> >>> receiver, VDR simply wants to have some measure of the signal's "strength"
> >>> and "quality". These are just two values that should be delivered by each
> >>> frontend/demux, using the *same* defined and mandatory range. I don't care
> >>> what exactly that is, but it needs to be the same for all devices.
> >>> What values a particular driver uses internally to come up with these
> >>> is of no interest to VDR. The "signal strength" might just be what is
> >>> currently returned through FE_READ_SIGNAL_STRENGTH (however, normalized to
> >>> the same range in all drivers, which currently is not the case). The "signal
> >>> quality" might use flags like FE_HAS_SIGNAL, FE_HAS_CARRIER, FE_HAS_VITERBI,
> >>> FE_HAS_SYNC, as well as SNR, BER and UNC (if available) to form some
> >>> value where 0 means no quality at all, and 0xFFFF means excellent quality.
> >>> If a particular frontend/demux uses totally different concepts, it can
> >>> just use whatever it deems reasonable to form the "strength" and "quality"
> >>> values. The important thing here is just that all this needs to be hidden
> >>> inside the driver, and the only interface to an application are ioctl()
> >>> calls that return these two values.
> >>>
> >>> So I suggest that you define this minimal interface and allow applications
> >>> to retrieve what they really need. Once this is done, feel free to implement
> >>> whatever theoretical bells and whistles you fell like doing - that's all
> >>> fine with me, as long as the really important stuff keeps working ;-)
> >>
> >> Klaus,
> >>
> >> On ISDB-T, it splits the TS into (up to) three independent physical channels
> >> (called layers).
> >>
> >> Each channel has its own statistics, as they're completely independent:
> >> they use different inner FEC's, use different modulations, etc.
> >>
> >> The ISDB demods don't provide a single value for the 3 layers. They
> >> can't, as they're independent. So, signal-strengh and SNR measures are
> >> also independent for each of those 3 layers.
> >>
> >> A typical ISDB transmission uses 13 segments of carriers, each segment
> >> using a 4.28 kHz bandwidth, grouped into 3 layers. While it is up to
> >> the broadcaster to decide how to group the segments, it is typically
> >> arranged like that:
> >>
> >> 	layer A - 1 segment for LD programs - modulated using QPSK;
> >> 	layer B - 3 segments for SD programs - modulated using 16QAM;
> >> 	layer C - 9 segments for HD programs - modulated using 64QAM.
> >>
> >> The TDM TS packets from the Transport Stream are broken into those 3
> >> layers, each being an independent transmission channel.
> >>
> >> So, all channel level QoS measure are per-layer (SNR, signal strength,
> >> BER, MER, ...).
> >>
> >> While the demods I have datasheets here don't provide it, it would be
> >> possible to provide error counts for a given program ID, by summing
> >> the error count that applies to each PID.
> >>
> >> So, let's assume, for example, that the UCB count is:
> >> 	layer A = 0
> >> 	layer B = 12
> >> 	layer C = 30
> >>
> >> an 1-seg LD program will have 0 uncorrected blocks;
> >> an SD program will have 12 uncorrected error blocks;
> >> a HD program will have 42 uncorrected error blocks.
> >>
> >> It shouldn't be that hard to take it into account on userspace, but
> >> doing it at kernel level would be very painful, if possible, as
> >> kernelspace would be required to know what PID's are being shown, in
> >> order to estimate the error count measures for them. Also, it would
> >> require a much more complex kernelspace-userspace interface.
> >
> > Two additional notes:
> >
> > 1) If you want to get further information, it is available on ARIB
> > 	STD-B31 spec:
> >
> > 	http://www.arib.or.jp/english/html/overview/doc/6-STD-B31v1_6-E2.pdf
> >
> > There, table 3-2 shows the main characteristics of the modulation;
> > how the 3 independent channels are handled and fig. 3.4
> > shows a simplified diagram to give an idea on how the hierarchical TS
> > packets are broken into the 3 layers
> >
> > 2) There are in the market some narrow-band decoders. Those tunes only
> > 1 segment (440kHz), and are meant to be used on mobile devices that can
> > receive only LD programs. Only for those devices, it is possible to
> > offer a single set of statistics (SNR, strength, BER, UCB, etc),
> > because it can decode just one layer. I have a few of them here,
> > and we have 2 drivers for those 1-seg devices (s921 and siano).
> > The full-seg drivers currently provide crappy information or don't
> > provide any QoS stats at all due to the lack of a proper API.
> >
> > Regards,
> > Mauro
> 
> What I propose is quite near what Klaus wants. Just only new simple ways 
> to report current statistics with beforehand scale/values.
> 
> 1) Signal Strength
> * linear scale 0-0xffff
> 
> 2) Quality SNR
> * SNR in resolution 0.1dB
> 
> 3) Quality BER
> * ~like currently (no exact units)
> * measured from inner coding
> 
> 4) Quality UCB
> * ~like currently (no exact units)
> * measured from outer coding (naturally)
> * counter is increased over lifetime
> * tune resets counter?
> * driver is responsible of polling statistic in background and report 
> from cache

I still think that the better is to provide exact units where available.
Userspace can easily discard whatever scale it is, provided that they're
properly specified (including their typical range). Developers should only
implement the specific range when they're sure about that (e. g.
reverse-engineered drivers will be relative - even for SNR - devices based
on the datasheets can provide real values).

> I would not like to define exact units for BER and USB as those are 
> quite hard to implement and also non-sense. User would like just to see 
> if there is some (random) numbers and if those numbers are rising or 
> reducing when he changes antenna or adjusts gain. We are not making a 
> professional signal analyzers - numbers does not need to be 100% correctly.

No, but this API can be used by them or by STB's. So, it should have a way
to be used by professional applications.

> ISDB-T statistics are forced also to that simple API. Calculating 
> average value for example. Statistic differences between layers are so 
> minor that users does not even care to know.

There's no simple way to merge those values, especially for the error
counters, as it will depend on what program is being displayed. 

With regards to signal strength and SNR, Segment 0 information
(the central one) is probably the best shot.

What we can do is to estimate "global" value and put it at data[0] information
for GET_PROPERTIES. This way, simple applications can just use that info.

So, what we can do for ISDB-T "global" QoS measure is:

	- Strength and SNR: report the segment 0 value for the "global"
			    indicator;
	- BER, UCB: to sum up the error count of all active segments. This
	  is a worse case scenario, and the more likely one, as people tend
	  to watch to the HD program, when available (of course, if the
	  display hardware has enough resources to decode 1080p).

Keep reporting a per-layer stats. We do have enough space at the data
payload for that.

This way simple applications can just get the first value and don't care
if the standard is ISDB-T.

More sophisticated applications can get all data, and automatically switch
to a lower resolution stream if the QoS is not good enough for HD but
reliable enough for LD.

> 
> And as there is some persons who surely like to do QoS API like need of 
> $10k professional equipment, I propose to add more accurate reports as 
> alternative BUT that minimalist API should be offered even professional 
> API exits.

If you agree with this combined proposal, I can write a new patchset.

Regards,
Mauro
--
To unsubscribe from this list: send the line "unsubscribe linux-media" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
VDRU VDRU Jan. 4, 2013, 5:03 a.m. UTC | #18
On Thu, Jan 3, 2013 at 1:33 PM, Antti Palosaari <crope@iki.fi> wrote:
> I would not like to define exact units for BER and USB as those are quite
> hard to implement and also non-sense. User would like just to see if there
> is some (random) numbers and if those numbers are rising or reducing when he
> changes antenna or adjusts gain. We are not making a professional signal
> analyzers - numbers does not need to be 100% correctly.

Just a small comment here. Since this may finally be done, why not do
it the best way? In the end I think that's better and I don't see any
harm in having the capability to make a pro-grade signal analyzer.
After years of waiting, I don't think half-assing is a good idea.
--
To unsubscribe from this list: send the line "unsubscribe linux-media" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Manu Abraham Jan. 4, 2013, 5:30 a.m. UTC | #19
On Fri, Jan 4, 2013 at 10:33 AM, VDR User <user.vdr@gmail.com> wrote:
> On Thu, Jan 3, 2013 at 1:33 PM, Antti Palosaari <crope@iki.fi> wrote:
>> I would not like to define exact units for BER and USB as those are quite
>> hard to implement and also non-sense. User would like just to see if there
>> is some (random) numbers and if those numbers are rising or reducing when he
>> changes antenna or adjusts gain. We are not making a professional signal
>> analyzers - numbers does not need to be 100% correctly.
>
> Just a small comment here. Since this may finally be done, why not do
> it the best way? In the end I think that's better and I don't see any
> harm in having the capability to make a pro-grade signal analyzer.
> After years of waiting, I don't think half-assing is a good idea.

The problem is not in creating an API for such a thing. The problem arises
from the fact that all devices need to worked to comply to the API. It might
not factually possible to do that, since most drivers are reverse engineered
or written in a crude way.. In a lot many cases, there are not even the right
documents to do that. An API alone doesn't solve anything at all. Here we
are talking about making pro grade software based on home grade silicon,
for which most don't have proper documentation.

Manu
--
To unsubscribe from this list: send the line "unsubscribe linux-media" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Mauro Carvalho Chehab Jan. 6, 2013, 5:03 p.m. UTC | #20
Em Thu, 03 Jan 2013 23:33:49 +0200
Antti Palosaari <crope@iki.fi> escreveu:

> On 01/03/2013 06:29 PM, Mauro Carvalho Chehab wrote:
> > Em Thu, 3 Jan 2013 14:14:29 -0200
> > Mauro Carvalho Chehab <mchehab@redhat.com> escreveu:
> >
> >> Em Thu, 03 Jan 2013 16:18:26 +0100
> >> Klaus Schmidinger <Klaus.Schmidinger@tvdr.de> escreveu:
> >>
> >>> On 03.01.2013 14:20, Mauro Carvalho Chehab wrote:
> >>>> Em Wed, 2 Jan 2013 00:38:50 +0530
> >>>> Manu Abraham <abraham.manu@gmail.com> escreveu:
> >>>>
> >>>>> On Tue, Jan 1, 2013 at 10:59 PM, Mauro Carvalho Chehab
> >>>>> <mchehab@redhat.com> wrote:
> >>>>>> Em Tue, 1 Jan 2013 22:18:49 +0530
> >>>>>> Manu Abraham <abraham.manu@gmail.com> escreveu:
> >>>>>>
> >>>>>>> On Tue, Jan 1, 2013 at 8:30 PM, Mauro Carvalho Chehab
> >>>>>>> <mchehab@redhat.com> wrote:
> >>>>>>>
> >>>>>>>> [RFCv4] dvb: Add DVBv5 properties for quality parameters
> >>>>>>>>
> >>>>>>>> The DVBv3 quality parameters are limited on several ways:
> >>>>>>>>           - Doesn't provide any way to indicate the used measure;
> >>>>>>>>           - Userspace need to guess how to calculate the measure;
> >>>>>>>>           - Only a limited set of stats are supported;
> >>>>>>>>           - Doesn't provide QoS measure for the OFDM TPS/TMCC
> >>>>>>>>             carriers, used to detect the network parameters for
> >>>>>>>>             DVB-T/ISDB-T;
> >>>>>>>>           - Can't be called in a way to require them to be filled
> >>>>>>>>             all at once (atomic reads from the hardware), with may
> >>>>>>>>             cause troubles on interpreting them on userspace;
> >>>>>>>>           - On some OFDM delivery systems, the carriers can be
> >>>>>>>>             independently modulated, having different properties.
> >>>>>>>>             Currently, there's no way to report per-layer stats;
> >>>>>>>
> >>>>>>> per layer stats is a mythical bird, nothing of that sort does exist.
> >>>>>>
> >>>>>> Had you ever read or tried to get stats from an ISDB-T demod? If you
> >>>>>> had, you would see that it only provides per-layer stats. Btw, this is
> >>>>>> a requirement to follow the ARIB and ABNT ISDB specs.
> >>>>>
> >>>>> I understand you keep writing junk for ages, but nevertheless:
> >>>>>
> >>>>> Do you have any idea what's a BBHEADER (DVB-S2) or
> >>>>> PLHEADER (DVB-T2) ? The headers do indicate what MODCOD
> >>>>> (aka Modulation/Coding Standard follows, whatever mode ACM,
> >>>>> VCM or CCM) follows. These MODCOD foolows a TDM approach
> >>>>> with a hierarchial modulation principle. This is exactly what ISDB
> >>>>> does too.
> >>>>
> >>>> No, I didn't check DVB-S2/T2 specs deeply enough to understand
> >>>> if they're doing the same thing as ISDB.
> >>>>
> >>>> Yet, ISDB-T doesn't use a TDM approach for hierarchical modulation.
> >>>> It uses a FDM (OFDM is a type of Frequency Division Multiplexing).
> >>>>
> >>>> So, if you're saying that DVB-S2 uses TDM, it is very different than
> >>>> ISDB-T. As DVB-T2 uses an FDM type of modulation (OFDM), it would
> >>>> be possible to segment the carriers there, just like ISDB, or to
> >>>> use TDM hierarchical modulation techniques.
> >>>>
> >>>>>
> >>>>> And for your info:
> >>>>>
> >>>>> " The TMCC control information is
> >>>>> common to all TMCC carriers and
> >>>>> error correction is performed by using
> >>>>> difference-set cyclic code."
> >>>>
> >>>> Yes, TMCC carriers are equal and they are always modulated using DBPSK.
> >>>> That is done to make it possible to receive the TMCC carriers even under
> >>>> worse SNR conditions, where it may not be possible to decode the segment
> >>>> groups.
> >>>>
> >>>> It seems that you completely missed the point though. On ISDB-T, the
> >>>> carriers that belong to each group of segments (except for the control
> >>>> carriers - carriers 1 to 107) uses a completely independent modulation.
> >>>> Also, as they're spaced in frequency, the interference of each segment
> >>>> is different. So, error indications are different on each segment.
> >>>>
> >>>> Btw, in any case, the datasheets of ISDB-T demods clearly shows that
> >>>> the BER measures are per segment group (layer).
> >>>>
> >>>> For example, for the BER measures before Viterbi, those are the register
> >>>> names for a certain demod:
> >>>>
> >>>> 	VBERSNUMA Bit count of BER measurement before Viterbi in A layer
> >>>> 	VBERSNUMB Bit count of BER measurement before Viterbi in B layer
> >>>> 	VBERSNUMC Bit count of BER measurement before Viterbi in C layer
> >>>>
> >>>> It has another set of registers for BER after Viterbi, and for PER after
> >>>> Viterbi and RS, for bit count errors, etc.
> >>>>
> >>>> There's no way to get any type of "global" BER measure, simply because
> >>>> ISDB-T demods don't provide.
> >>>
> >>> Maybe we should put all this theoretical discussion aside for the moment and
> >>> think about what is *really* needed by real world applications. As with any
> >>> receiver, VDR simply wants to have some measure of the signal's "strength"
> >>> and "quality". These are just two values that should be delivered by each
> >>> frontend/demux, using the *same* defined and mandatory range. I don't care
> >>> what exactly that is, but it needs to be the same for all devices.
> >>> What values a particular driver uses internally to come up with these
> >>> is of no interest to VDR. The "signal strength" might just be what is
> >>> currently returned through FE_READ_SIGNAL_STRENGTH (however, normalized to
> >>> the same range in all drivers, which currently is not the case). The "signal
> >>> quality" might use flags like FE_HAS_SIGNAL, FE_HAS_CARRIER, FE_HAS_VITERBI,
> >>> FE_HAS_SYNC, as well as SNR, BER and UNC (if available) to form some
> >>> value where 0 means no quality at all, and 0xFFFF means excellent quality.
> >>> If a particular frontend/demux uses totally different concepts, it can
> >>> just use whatever it deems reasonable to form the "strength" and "quality"
> >>> values. The important thing here is just that all this needs to be hidden
> >>> inside the driver, and the only interface to an application are ioctl()
> >>> calls that return these two values.
> >>>
> >>> So I suggest that you define this minimal interface and allow applications
> >>> to retrieve what they really need. Once this is done, feel free to implement
> >>> whatever theoretical bells and whistles you fell like doing - that's all
> >>> fine with me, as long as the really important stuff keeps working ;-)
> >>
> >> Klaus,
> >>
> >> On ISDB-T, it splits the TS into (up to) three independent physical channels
> >> (called layers).
> >>
> >> Each channel has its own statistics, as they're completely independent:
> >> they use different inner FEC's, use different modulations, etc.
> >>
> >> The ISDB demods don't provide a single value for the 3 layers. They
> >> can't, as they're independent. So, signal-strengh and SNR measures are
> >> also independent for each of those 3 layers.
> >>
> >> A typical ISDB transmission uses 13 segments of carriers, each segment
> >> using a 4.28 kHz bandwidth, grouped into 3 layers. While it is up to
> >> the broadcaster to decide how to group the segments, it is typically
> >> arranged like that:
> >>
> >> 	layer A - 1 segment for LD programs - modulated using QPSK;
> >> 	layer B - 3 segments for SD programs - modulated using 16QAM;
> >> 	layer C - 9 segments for HD programs - modulated using 64QAM.
> >>
> >> The TDM TS packets from the Transport Stream are broken into those 3
> >> layers, each being an independent transmission channel.
> >>
> >> So, all channel level QoS measure are per-layer (SNR, signal strength,
> >> BER, MER, ...).
> >>
> >> While the demods I have datasheets here don't provide it, it would be
> >> possible to provide error counts for a given program ID, by summing
> >> the error count that applies to each PID.
> >>
> >> So, let's assume, for example, that the UCB count is:
> >> 	layer A = 0
> >> 	layer B = 12
> >> 	layer C = 30
> >>
> >> an 1-seg LD program will have 0 uncorrected blocks;
> >> an SD program will have 12 uncorrected error blocks;
> >> a HD program will have 42 uncorrected error blocks.
> >>
> >> It shouldn't be that hard to take it into account on userspace, but
> >> doing it at kernel level would be very painful, if possible, as
> >> kernelspace would be required to know what PID's are being shown, in
> >> order to estimate the error count measures for them. Also, it would
> >> require a much more complex kernelspace-userspace interface.
> >
> > Two additional notes:
> >
> > 1) If you want to get further information, it is available on ARIB
> > 	STD-B31 spec:
> >
> > 	http://www.arib.or.jp/english/html/overview/doc/6-STD-B31v1_6-E2.pdf
> >
> > There, table 3-2 shows the main characteristics of the modulation;
> > how the 3 independent channels are handled and fig. 3.4
> > shows a simplified diagram to give an idea on how the hierarchical TS
> > packets are broken into the 3 layers
> >
> > 2) There are in the market some narrow-band decoders. Those tunes only
> > 1 segment (440kHz), and are meant to be used on mobile devices that can
> > receive only LD programs. Only for those devices, it is possible to
> > offer a single set of statistics (SNR, strength, BER, UCB, etc),
> > because it can decode just one layer. I have a few of them here,
> > and we have 2 drivers for those 1-seg devices (s921 and siano).
> > The full-seg drivers currently provide crappy information or don't
> > provide any QoS stats at all due to the lack of a proper API.
> >
> > Regards,
> > Mauro
> 
> What I propose is quite near what Klaus wants. Just only new simple ways 
> to report current statistics with beforehand scale/values.

Ok, just added a new RFC. I tried to put there everything discussed on both
ML and on IRC:
	http://patchwork.linuxtv.org/patch/16145/

It is also available on my experimental tree, at branch "stats":
	git://linuxtv.org/mchehab/experimental.git stats

I didn't start coding yet. After we agree with that, I'll write a v7 with
both DVB core changes and one driver implementation.

Regards,
Mauro.

PS.: I'm enclosing the main documentation chapter of the specs, in order
to help with discussions, as it is better do comment on a plain-text
email, than to reply to an XML file. If you want to see it in HTML, just
pull it from my tree, run "make htmldocs" and see it on your favorite
browser, like:
	firefox file:///home/mchehab/stats/Documentation/DocBook/media_api/FE_GET_SET_PROPERTY.html


---


- Frontend Quality of Service/Statistics indicators


Except for DTV_QOS_ENUM, the values are returned via dtv_property.stat.

For most delivery systems, this will return a single value for each parameter.

It should be noticed, however, that new OFDM delivery systems like ISDB can use different modulation types for each group of carriers. On such standards, up to 8 groups of statistics can be provided, one for each carrier group (called "layer" on ISDB). In order to be consistent with other delivery systems, the first value at dtv_property.stat.dtv_stats array refers to a global indicator, if any. The other elements of the array represent each layer, starting from layer A(index 1), layer B (index 2) and so on

The number of filled elements are stored at dtv_property.stat.len.

Each element of the dtv_property.stat.dtv_stats array consists on two elements:

    value - Value of the measure

    scale - Scale for the value. It can be:

        FE_SCALE_NOT_AVAILABLE - If it is not possible to collect a given parameter (could be a transitory or permanent condition)

        FE_SCALE_DECIBEL - parameter is a signed value, measured in 0.1 dB

        FE_SCALE_RELATIVE - parameter is a unsigned value, where 0 means 0% and 65535 means 100%.

- DTV_QOS_ENUM

A frontend needs to advertise the statistics it provides. This property allows to enumerate all DTV QoS statistics that are supported by a given frontend.

dtv_property.len indicates the number of supported DTV QoS statistics.

dtv_property.data16 is an 16 bits array of the supported properties.

- DTV_QOS_TUNER_SIGNAL

Indicates the signal strength level at the analog part of the tuner.

- DTV_QOS_CNR

Indicates the signal to noise relation for the main carrier.

- DTV_QOS_BIT_ERROR_COUNT

Measures the number of bit errors since the last counter reset.

In order to get the bit error rate, it should be divided by DTV_QOS_BIT_ERROR_COUNT_TIME, if available. Otherwise, it should be divided by the time lapsed since the previous call for DTV_QOS_BIT_ERROR_COUNT.

- DTV_QOS_BIT_ERROR_COUNT_TIME

measures the time since the last DTV_QOS_BIT_ERROR_COUNT reset.

It might not be available on certain frontends, even when DTV_QOS_BIT_ERROR_COUNT is provided, due to the lack of frontend's documentation when the driver was developed.

- DTV_QOS_ERROR_BLOCK_COUNT

Measures the number of block errors since the last counter reset.

In order to get the bit error rate, it should be divided by DTV_QOS_ERROR_BLOCK_COUNT_TIME, if available. Otherwise, it should be divided by the time lapsed since the previous call for DTV_QOS_ERROR_BLOCK_COUNT.

- DTV_QOS_ERROR_BLOCK_COUNT_TIME

measures the time since the last DTV_QOS_ERROR_BLOCK_COUNT reset.

It might not be available on certain frontends, even when DTV_QOS_BIT_ERROR_BLOCK_COUNT is provided, due to the lack of frontend's documentation when the driver was developed.
Antti Palosaari Jan. 6, 2013, 5:43 p.m. UTC | #21
On 01/06/2013 07:03 PM, Mauro Carvalho Chehab wrote:
> Em Thu, 03 Jan 2013 23:33:49 +0200
> Antti Palosaari <crope@iki.fi> escreveu:
>
>> On 01/03/2013 06:29 PM, Mauro Carvalho Chehab wrote:
>>> Em Thu, 3 Jan 2013 14:14:29 -0200
>>> Mauro Carvalho Chehab <mchehab@redhat.com> escreveu:
>>>
>>>> Em Thu, 03 Jan 2013 16:18:26 +0100
>>>> Klaus Schmidinger <Klaus.Schmidinger@tvdr.de> escreveu:
>>>>
>>>>> On 03.01.2013 14:20, Mauro Carvalho Chehab wrote:
>>>>>> Em Wed, 2 Jan 2013 00:38:50 +0530
>>>>>> Manu Abraham <abraham.manu@gmail.com> escreveu:
>>>>>>
>>>>>>> On Tue, Jan 1, 2013 at 10:59 PM, Mauro Carvalho Chehab
>>>>>>> <mchehab@redhat.com> wrote:
>>>>>>>> Em Tue, 1 Jan 2013 22:18:49 +0530
>>>>>>>> Manu Abraham <abraham.manu@gmail.com> escreveu:
>>>>>>>>
>>>>>>>>> On Tue, Jan 1, 2013 at 8:30 PM, Mauro Carvalho Chehab
>>>>>>>>> <mchehab@redhat.com> wrote:
>>>>>>>>>
>>>>>>>>>> [RFCv4] dvb: Add DVBv5 properties for quality parameters
>>>>>>>>>>
>>>>>>>>>> The DVBv3 quality parameters are limited on several ways:
>>>>>>>>>>            - Doesn't provide any way to indicate the used measure;
>>>>>>>>>>            - Userspace need to guess how to calculate the measure;
>>>>>>>>>>            - Only a limited set of stats are supported;
>>>>>>>>>>            - Doesn't provide QoS measure for the OFDM TPS/TMCC
>>>>>>>>>>              carriers, used to detect the network parameters for
>>>>>>>>>>              DVB-T/ISDB-T;
>>>>>>>>>>            - Can't be called in a way to require them to be filled
>>>>>>>>>>              all at once (atomic reads from the hardware), with may
>>>>>>>>>>              cause troubles on interpreting them on userspace;
>>>>>>>>>>            - On some OFDM delivery systems, the carriers can be
>>>>>>>>>>              independently modulated, having different properties.
>>>>>>>>>>              Currently, there's no way to report per-layer stats;
>>>>>>>>>
>>>>>>>>> per layer stats is a mythical bird, nothing of that sort does exist.
>>>>>>>>
>>>>>>>> Had you ever read or tried to get stats from an ISDB-T demod? If you
>>>>>>>> had, you would see that it only provides per-layer stats. Btw, this is
>>>>>>>> a requirement to follow the ARIB and ABNT ISDB specs.
>>>>>>>
>>>>>>> I understand you keep writing junk for ages, but nevertheless:
>>>>>>>
>>>>>>> Do you have any idea what's a BBHEADER (DVB-S2) or
>>>>>>> PLHEADER (DVB-T2) ? The headers do indicate what MODCOD
>>>>>>> (aka Modulation/Coding Standard follows, whatever mode ACM,
>>>>>>> VCM or CCM) follows. These MODCOD foolows a TDM approach
>>>>>>> with a hierarchial modulation principle. This is exactly what ISDB
>>>>>>> does too.
>>>>>>
>>>>>> No, I didn't check DVB-S2/T2 specs deeply enough to understand
>>>>>> if they're doing the same thing as ISDB.
>>>>>>
>>>>>> Yet, ISDB-T doesn't use a TDM approach for hierarchical modulation.
>>>>>> It uses a FDM (OFDM is a type of Frequency Division Multiplexing).
>>>>>>
>>>>>> So, if you're saying that DVB-S2 uses TDM, it is very different than
>>>>>> ISDB-T. As DVB-T2 uses an FDM type of modulation (OFDM), it would
>>>>>> be possible to segment the carriers there, just like ISDB, or to
>>>>>> use TDM hierarchical modulation techniques.
>>>>>>
>>>>>>>
>>>>>>> And for your info:
>>>>>>>
>>>>>>> " The TMCC control information is
>>>>>>> common to all TMCC carriers and
>>>>>>> error correction is performed by using
>>>>>>> difference-set cyclic code."
>>>>>>
>>>>>> Yes, TMCC carriers are equal and they are always modulated using DBPSK.
>>>>>> That is done to make it possible to receive the TMCC carriers even under
>>>>>> worse SNR conditions, where it may not be possible to decode the segment
>>>>>> groups.
>>>>>>
>>>>>> It seems that you completely missed the point though. On ISDB-T, the
>>>>>> carriers that belong to each group of segments (except for the control
>>>>>> carriers - carriers 1 to 107) uses a completely independent modulation.
>>>>>> Also, as they're spaced in frequency, the interference of each segment
>>>>>> is different. So, error indications are different on each segment.
>>>>>>
>>>>>> Btw, in any case, the datasheets of ISDB-T demods clearly shows that
>>>>>> the BER measures are per segment group (layer).
>>>>>>
>>>>>> For example, for the BER measures before Viterbi, those are the register
>>>>>> names for a certain demod:
>>>>>>
>>>>>> 	VBERSNUMA Bit count of BER measurement before Viterbi in A layer
>>>>>> 	VBERSNUMB Bit count of BER measurement before Viterbi in B layer
>>>>>> 	VBERSNUMC Bit count of BER measurement before Viterbi in C layer
>>>>>>
>>>>>> It has another set of registers for BER after Viterbi, and for PER after
>>>>>> Viterbi and RS, for bit count errors, etc.
>>>>>>
>>>>>> There's no way to get any type of "global" BER measure, simply because
>>>>>> ISDB-T demods don't provide.
>>>>>
>>>>> Maybe we should put all this theoretical discussion aside for the moment and
>>>>> think about what is *really* needed by real world applications. As with any
>>>>> receiver, VDR simply wants to have some measure of the signal's "strength"
>>>>> and "quality". These are just two values that should be delivered by each
>>>>> frontend/demux, using the *same* defined and mandatory range. I don't care
>>>>> what exactly that is, but it needs to be the same for all devices.
>>>>> What values a particular driver uses internally to come up with these
>>>>> is of no interest to VDR. The "signal strength" might just be what is
>>>>> currently returned through FE_READ_SIGNAL_STRENGTH (however, normalized to
>>>>> the same range in all drivers, which currently is not the case). The "signal
>>>>> quality" might use flags like FE_HAS_SIGNAL, FE_HAS_CARRIER, FE_HAS_VITERBI,
>>>>> FE_HAS_SYNC, as well as SNR, BER and UNC (if available) to form some
>>>>> value where 0 means no quality at all, and 0xFFFF means excellent quality.
>>>>> If a particular frontend/demux uses totally different concepts, it can
>>>>> just use whatever it deems reasonable to form the "strength" and "quality"
>>>>> values. The important thing here is just that all this needs to be hidden
>>>>> inside the driver, and the only interface to an application are ioctl()
>>>>> calls that return these two values.
>>>>>
>>>>> So I suggest that you define this minimal interface and allow applications
>>>>> to retrieve what they really need. Once this is done, feel free to implement
>>>>> whatever theoretical bells and whistles you fell like doing - that's all
>>>>> fine with me, as long as the really important stuff keeps working ;-)
>>>>
>>>> Klaus,
>>>>
>>>> On ISDB-T, it splits the TS into (up to) three independent physical channels
>>>> (called layers).
>>>>
>>>> Each channel has its own statistics, as they're completely independent:
>>>> they use different inner FEC's, use different modulations, etc.
>>>>
>>>> The ISDB demods don't provide a single value for the 3 layers. They
>>>> can't, as they're independent. So, signal-strengh and SNR measures are
>>>> also independent for each of those 3 layers.
>>>>
>>>> A typical ISDB transmission uses 13 segments of carriers, each segment
>>>> using a 4.28 kHz bandwidth, grouped into 3 layers. While it is up to
>>>> the broadcaster to decide how to group the segments, it is typically
>>>> arranged like that:
>>>>
>>>> 	layer A - 1 segment for LD programs - modulated using QPSK;
>>>> 	layer B - 3 segments for SD programs - modulated using 16QAM;
>>>> 	layer C - 9 segments for HD programs - modulated using 64QAM.
>>>>
>>>> The TDM TS packets from the Transport Stream are broken into those 3
>>>> layers, each being an independent transmission channel.
>>>>
>>>> So, all channel level QoS measure are per-layer (SNR, signal strength,
>>>> BER, MER, ...).
>>>>
>>>> While the demods I have datasheets here don't provide it, it would be
>>>> possible to provide error counts for a given program ID, by summing
>>>> the error count that applies to each PID.
>>>>
>>>> So, let's assume, for example, that the UCB count is:
>>>> 	layer A = 0
>>>> 	layer B = 12
>>>> 	layer C = 30
>>>>
>>>> an 1-seg LD program will have 0 uncorrected blocks;
>>>> an SD program will have 12 uncorrected error blocks;
>>>> a HD program will have 42 uncorrected error blocks.
>>>>
>>>> It shouldn't be that hard to take it into account on userspace, but
>>>> doing it at kernel level would be very painful, if possible, as
>>>> kernelspace would be required to know what PID's are being shown, in
>>>> order to estimate the error count measures for them. Also, it would
>>>> require a much more complex kernelspace-userspace interface.
>>>
>>> Two additional notes:
>>>
>>> 1) If you want to get further information, it is available on ARIB
>>> 	STD-B31 spec:
>>>
>>> 	http://www.arib.or.jp/english/html/overview/doc/6-STD-B31v1_6-E2.pdf
>>>
>>> There, table 3-2 shows the main characteristics of the modulation;
>>> how the 3 independent channels are handled and fig. 3.4
>>> shows a simplified diagram to give an idea on how the hierarchical TS
>>> packets are broken into the 3 layers
>>>
>>> 2) There are in the market some narrow-band decoders. Those tunes only
>>> 1 segment (440kHz), and are meant to be used on mobile devices that can
>>> receive only LD programs. Only for those devices, it is possible to
>>> offer a single set of statistics (SNR, strength, BER, UCB, etc),
>>> because it can decode just one layer. I have a few of them here,
>>> and we have 2 drivers for those 1-seg devices (s921 and siano).
>>> The full-seg drivers currently provide crappy information or don't
>>> provide any QoS stats at all due to the lack of a proper API.
>>>
>>> Regards,
>>> Mauro
>>
>> What I propose is quite near what Klaus wants. Just only new simple ways
>> to report current statistics with beforehand scale/values.
>
> Ok, just added a new RFC. I tried to put there everything discussed on both
> ML and on IRC:
> 	http://patchwork.linuxtv.org/patch/16145/
>
> It is also available on my experimental tree, at branch "stats":
> 	git://linuxtv.org/mchehab/experimental.git stats
>
> I didn't start coding yet. After we agree with that, I'll write a v7 with
> both DVB core changes and one driver implementation.
>
> Regards,
> Mauro.
>
> PS.: I'm enclosing the main documentation chapter of the specs, in order
> to help with discussions, as it is better do comment on a plain-text
> email, than to reply to an XML file. If you want to see it in HTML, just
> pull it from my tree, run "make htmldocs" and see it on your favorite
> browser, like:
> 	firefox file:///home/mchehab/stats/Documentation/DocBook/media_api/FE_GET_SET_PROPERTY.html
>
>
> ---
>
>
> - Frontend Quality of Service/Statistics indicators
>
>
> Except for DTV_QOS_ENUM, the values are returned via dtv_property.stat.
>
> For most delivery systems, this will return a single value for each parameter.
>
> It should be noticed, however, that new OFDM delivery systems like ISDB can use different modulation types for each group of carriers. On such standards, up to 8 groups of statistics can be provided, one for each carrier group (called "layer" on ISDB). In order to be consistent with other delivery systems, the first value at dtv_property.stat.dtv_stats array refers to a global indicator, if any. The other elements of the array represent each layer, starting from layer A(index 1), layer B (index 2) and so on

Typo, it is up to 3 groups currently. However, I could guess DVB-T is 
also able to provide similar statistics but only max two layers (in 
hierarchical mode). I didn't check that from the specs yet, but I will do.


> The number of filled elements are stored at dtv_property.stat.len.
>
> Each element of the dtv_property.stat.dtv_stats array consists on two elements:
>
>      value - Value of the measure
>
>      scale - Scale for the value. It can be:
>
>          FE_SCALE_NOT_AVAILABLE - If it is not possible to collect a given parameter (could be a transitory or permanent condition)
>
>          FE_SCALE_DECIBEL - parameter is a signed value, measured in 0.1 dB
>
>          FE_SCALE_RELATIVE - parameter is a unsigned value, where 0 means 0% and 65535 means 100%.

I am not a big fan of that kind of unit complexity, but I can live with 
it. It is easy to be lazy and return some random register values without 
converting...


> - DTV_QOS_ENUM
>
> A frontend needs to advertise the statistics it provides. This property allows to enumerate all DTV QoS statistics that are supported by a given frontend.
>
> dtv_property.len indicates the number of supported DTV QoS statistics.
>
> dtv_property.data16 is an 16 bits array of the supported properties.
>
> - DTV_QOS_TUNER_SIGNAL
>
> Indicates the signal strength level at the analog part of the tuner.

how about just SIGNAL_STRENGTH

> - DTV_QOS_CNR
>
> Indicates the signal to noise relation for the main carrier.
>
> - DTV_QOS_BIT_ERROR_COUNT
>
> Measures the number of bit errors since the last counter reset.
>
> In order to get the bit error rate, it should be divided by DTV_QOS_BIT_ERROR_COUNT_TIME, if available. Otherwise, it should be divided by the time lapsed since the previous call for DTV_QOS_BIT_ERROR_COUNT.
>
> - DTV_QOS_BIT_ERROR_COUNT_TIME
>
> measures the time since the last DTV_QOS_BIT_ERROR_COUNT reset.
>
> It might not be available on certain frontends, even when DTV_QOS_BIT_ERROR_COUNT is provided, due to the lack of frontend's documentation when the driver was developed.

I dind't like at all. IMHO it is driver job to calculate BER. Reporting 
BER as [BER = error_bits / total_bits] at the time is quite standard manner.

How you thought application calculates total bit stream needed to 
calculate BER? I think it is very hard.

Also, I would like to document that BER is measured from the inner 
coding and it is pre BER rather than post BER (because it is nice to see 
signal errors just before correction, after the inner coding numbers are 
quite small or even 0 all the time).


> - DTV_QOS_ERROR_BLOCK_COUNT
>
> Measures the number of block errors since the last counter reset.
>
> In order to get the bit error rate, it should be divided by DTV_QOS_ERROR_BLOCK_COUNT_TIME, if available. Otherwise, it should be divided by the time lapsed since the previous call for DTV_QOS_ERROR_BLOCK_COUNT.
>
> - DTV_QOS_ERROR_BLOCK_COUNT_TIME
>
> measures the time since the last DTV_QOS_ERROR_BLOCK_COUNT reset.
>
> It might not be available on certain frontends, even when DTV_QOS_BIT_ERROR_BLOCK_COUNT is provided, due to the lack of frontend's documentation when the driver was developed.

I think uncorrected blocks are not usually reported as rate, instead 
just blocks found to be faulty after outer coding. This is counter which 
very rarely increases, if you have picture it should remain 0 or at 
least near zero, otherwise your picture is totally garbage.


I hate you have added some time counting logic here (BER & UCB) which 
even should be done inside driver and report user space.

regards
Antti
diff mbox

Patch

--- patchwork.orig/include/uapi/linux/dvb/frontend.h
+++ patchwork/include/uapi/linux/dvb/frontend.h
@@ -365,7 +365,14 @@  struct dvb_frontend_event {
 #define DTV_INTERLEAVING			60
 #define DTV_LNA					61
 
-#define DTV_MAX_COMMAND				DTV_LNA
+/* Quality parameters */
+#define DTV_ENUM_QUALITY	62	/* Enumerates supported QoS parameters */
+#define DTV_FE_SIGNAL		63	/* Signal strength at the demod */
+#define DTV_QUALITY_SNR		64	/* Signal/Noise ratio */
+#define DTV_ERROR_BER		65	/* Bit error rate since signal lock */
+#define DTV_ERROR_COUNT		66	/* Monotonic error count since signal lock at TMCC or TPS carrier */
+
+#define DTV_MAX_COMMAND		DTV_ERROR_COUNT
 
 typedef enum fe_pilot {
 	PILOT_ON,
@@ -452,12 +459,63 @@  struct dtv_cmds_h {
 	__u32	reserved:30;	/* Align */
 };
 
+/**
+ * Scale types for the quality parameters.
+ * @FE_SCALE_NOT_AVAILABLE: That QoS measure is not available. That
+ *			    could indicate a temporary or a permanent
+ *			    condition.
+ * @FE_SCALE_DECIBEL: The scale is measured in 0.1 dB steps, typically
+ *		  used on signal measures.
+ * @FE_SCALE_RELATIVE: The scale is a relative percentual measure,
+ *			ranging from 0 (0%) to 0xffff (100%).
+ */
+enum fecap_scale_params {
+	FE_SCALE_NOT_AVAILABLE,
+	FE_SCALE_DECIBEL,
+	FE_SCALE_RELATIVE
+};
+
+/**
+ * struct dtv_status - Used for reading a DTV status property
+ *
+ * @value:	value of the measure. Should range from 0 to 0xffff;
+ * @scale:	Filled with enum fecap_scale_params - the scale
+ *		in usage for that parameter
+ *
+ * For most delivery systems, this will return a single value for each
+ * parameter.
+ * It should be noticed, however, that new OFDM delivery systems like
+ * ISDB can use different modulation types for each group of carriers.
+ * On such standards, up to 8 groups of statistics can be provided, one
+ * for each carrier group (called "layer" on ISDB).
+ * In order to be consistent with other delivery systems, the first
+ * value refers to the entire set of carriers ("global").
+ * dtv_status:scale should use the value FE_SCALE_NOT_AVAILABLE when
+ * the value for the entire group of carriers or from one specific layer
+ * is not provided by the hardware.
+ * In other words, for ISDB, those values should be filled like:
+ *	stat.status[0] = global statistics;
+ *	stat.scale[0] = FE_SCALE_NOT_AVAILABLE (if not available);
+ *	stat.status[1] = layer A statistics;
+ *	stat.status[2] = layer B statistics;
+ *	stat.status[3] = layer C statistics.
+ * and stat.len should be filled with the latest filled status + 1.
+ */
+struct dtv_status {
+	__u16 value;
+	__u16 scale;
+} __attribute__ ((packed));
+
 struct dtv_property {
 	__u32 cmd;
 	__u32 reserved[3];
 	union {
 		__u32 data;
 		struct {
+			__u8 len;
+			struct dtv_status status[4];
+		} stat;
+		struct {
 			__u8 data[32];
 			__u32 len;
 			__u32 reserved1[3];