DVBStreamExplorer export to Web Services

The next release of DVBStreamExplorer Professional edition will introduce new features for exporting data to Web Services.

• Service WS export
• EIT WS export

Service WS export
This feature will export the same data as when exporting to CSV file. The difference being that it will export directly to a server that implements a WS that can be consumed by DVBStreamExplorer.
This could be used by users who are using DVBStreamExplorer to produce data for an on-line DTV service list.
For information on how to develop a WS that can be consumed by DVBStreamExplorer visist http://developer.dvbstreamexplorer.com
The WS export is invoked from export menu.

Endpoint URL of Web Service has to be entered. There is also an ID field. The meaning of this is defined by WS implementation. It is not used internally by DVBStreamExplorer. This could for instance be a unique DTV service list instance, so that multiple DTV service lists could be hosted.

EIT WS export
This feature will export EIT event data to a Web Service. DVBStreamExplorer could then be used to provide event data for an on-line TV guide.
For information on how to develop a WS that can be consumed by DVBStreamExplorer visist http://developer.dvbstreamexplorer.com
This feature is part of the DVB TS Monitor EIT page.

Before starting TS Monitor session, export to WS must be configured. Configuration of EIT WS export can be invoked from Options -> Preferences EIT export tab.

Endpoint URL of Web Service has to be entered. There is also an ID field. The meaning of this is defined by WS implementation. It is not used internally by DVBStreamExplorer.
EIT module will cache events in memory. When a sufficient number of events have been cached they will be exported to Web Service. This will happen either when 'Upload interval (minutes)' has passed or count of events has exceeded 'Event threshold', depending on which of these two events occurs first.
A maximum number of 'Max events per request' events will be sent in one request to Web Service. If number of cached events exceeds this value multiple requests to Web Service will be made.

Using Google Drive with DVBStreamExploerer exported CSV files

Service information files exported from DVBStreamExplorer in CSV format can be uploaded to Google Drive and converted to Google Spreadsheets. Service information can then be shared easily with other users and you can access information from everywhere including handheld and mobile devices.
You will need either a Google Account or a Google Apps Account.

Below is a DVBStreamExplorer scan of local DVB-T channels (Malta) exported to Google Docs spreadsheet.

Direct link to spreadsheet: https://docs.google.com/spreadsheet/ccc?key=0AlzKD8F--RuTdDJpUWlRYXF3WFpiT3pRdUxRWFgxUUE

Below is a DVBStreamExplorer scan of 'Astra 19.2°E' exported to Google Docs spreadsheet.



Direct link to spreadsheet: https://docs.google.com/spreadsheet/ccc?key=0AlzKD8F--RuTdHhqcVJNWmdTRDVoNVdhM1ZESWphLXc

Once service CSV file has been converted to spreadsheet one could apply Google Apps Script to add special features. This could for instance be extraction of certain information which would normally be difficult to spot when looking at a large spreadsheet. Or it could be used to spot service changes over time. For instance if 'Astra 19.2°E' was scanned one day and then scanned again a week after a script could generate a report of service changes. Such a report could include added and removed services, and list of services where one or more properties changed.

Follow +DVBStreamExplorer to keep up to date with DVBStreamExplorer  related news.

NorDig support in DVBStreamExplorer

The next release of DVBStreamExplorer will introduce decoding of NorDig logic_channel_descriptor and content_protection_descriptor private descriptors.
This might be of interest to users in Nordic region (Denmark, Finland, Iceland, Norway, Sweden) and Éire.
For logic_channel_descriptor, both Version 1 (descriptor_tag 0x83) and Version 2 (descriptor_tag 0x87) are decoded.
Below is a screenshot of logic_channel_descriptor (Version 1) decoding in DVB PSI/SI scanning.

And below is screenshot of decoding of same descriptor in DVB Transport Stream Monitor.

Extracting CA EMM/ECM sections with TS File Demux

In this post I will demonstrate how to use TS File Demux section  filter feature to extraxt CA entitlement management message (EMM) / entitlement control message (ECM) sections. I will use a local DVB-T MUX 770 MHz from GO as example.
Screenshot below show the MUX in DVBStreamExplorer with Boomerang TV service expanded.

I have outlined CA EMM and ECM streams which are those of interest for this post. It can be seen that CA provider is Norwegian Telekom (CA_ID 0x0b00) which would imply that the CA system is Conax. The EMM PID is carried on PID 0x0063 (99) and the ECM PID for Boomerang TV service is carried on PID 0x01f6 (502).
I used DVBStreamExplorer to capture a Transport Stream file that I will use as input in TS File Demux. But you can use any software capable of capturing Transport Stream file.

After launching TS File Demux the first thing to do is to open the Transport Stream file saved in previous step. TS File Demux will scan the file for program and PID information and you will end up with something similar to screenshot below.

You can now locate the PID streams of interest and add filters. You can can double-click on an entry containing a PID value in programs tree to populate filter controls. The application will try to find the most appropriate filter type from the context. You can change parameters before adding the filter by applying 'Add' button.
Below is screenshot after adding 0x0063 (EMM) and 0x01f6 (EMM) section filters.

Now apply 'Start' button and the selected streams will be saved to the folder specified. Two files were generated, sec_0x0063.bin and sec_0x01f6.bin. Their file names were generated from the filter type (section ) and from the PID values saved.
Below is screenshot of sec_0x01f6.bin shown in a hex editor (PSPad).

One ECM section is highlighted. It can be seen that table_id field is 0x80 and section_length field is 0x44 (68) bytes.
It is beyond the scope of this post trying to interpret the content of ECM section. I will leave this as an exercise for the reader.
Happy CA exploring!

How DVBStreamExplorer is keeping old PSI/SI section versions

In this post I will describe an often overlooked, but nice feature in DVBStreamExplorer Pro Edition TS Monitor feature. When scanning for SI/PSI table sections DVBStreamExplorer Pro Edition will like most other analysing software keep a copy of all unique sections. However when a new version of a given section arrives in stream, DVBStreamExplorer Pro Edition will keep the old section when inserting the new 'current' section in the tree. Below is an example of how an EIT section version 6 was first acquired and then later superseded by version 7, and finally by version 8. The section version currently applicable in TS is labelled (current) for easy identification. The section version_number field  is 5 bits long. So DVBStreamExplorer Pro Edition will keep up to 32 versions, including current version, of each unique section.

The curious user can then examine different section versions in the tree to find out exactly what changed. Below is EIT section version 7 expanded in SI/PSI tree.

And  below is EIT section version 8 expanded in SI/PSI tree.

It can be seen that the event ID of first event in loop was changed from 707 (section version 7) to 708 (section version 8).
This approach can also be applied for table sections like PAT, CAT, PMT, NIT, SDT, NIT etc. In this example I used EIT because the section version change can be observed after scanning for a relatively short period of time. For the other table section types one usually have to scan for a significantly longer time to catch a section version change.

Creating DVB-T MUX list in DVBStreamExplorer

If you are going to use DVBStreamExplorer with DVB-T capture hardware, one of the first thing you need to do is to create a DVB-T MUX list. If you are going to do a 'Multiple DVB-T MUX scanning' session, only entries in DVB-T MUX list can be scanned. When tuning manually it's also offers an convenient way select entry from list instead of having to type frequency manually.
The DVB-T MUX list can easily be created by the application performing a full scan of all standard frequencies. Non standard frequencies can be added to the list manually. Band scan can be invoked from scan button on 'Options  -> DVB-T MUX list' menu.
If you are using BDA hardware the very first thing you should do is to set the 'Tuning wait for lock delay (ms)' to an appropriate value, i.e. 1000 ms. Leaving it at default value 0 ms can cause problems with some cards. In those cases operations like 'Multiple DVB-T MUXscanning' would fail each tuning attempt and you will end up with no data found. The wait for lock value can be set in Device -> Settings menu. See screenshot below.

The video below demonstrates how to set  'Tuning wait for lock delay (ms)' value and how to start DVB-T MUX list band scan operation.



The video below shows the conclusion of DVB-T MUX list band scan operation. It also shows that DVB-T MUX list entries are now available for 'Multiple DVB-T MUX scanning' session.



Similar operations can be applied for DVB-C, ATSC and ISDB-T MUX lists. The case of DVB-S MUX list is more complex and will be documented separately in future post.

Analysing DVB MHP application using DSM-CC analyser

I was studying some DVB-T services with DVBStreamExplorer. I noticed that on 546 MHz MUX some services were carrying Multimedia Home Platform (MHP) applications. I decided to study MHP application further with DSM-CC analyser in DVBStreamExplorer. In this exercise I shall only be using DVB Transport Stream Monitor. From that I will acquire both PSI/SI and DSM-CC information.
Image below show stream details for one of the services in MUX, LA7. Besides the common video and audio streams the presence of MHP application can also be seen here. In the stream list references to Application Information Table (AIT) and DSM-CC object carousels can be found. For instance one AIT can be found on PID 0x1ba8 (7080). An DSM-CC Object Carousel can be found on PID 0x1ba9 (7081).

Below can be seen the content of an AIT. That contains various information about the application. That covers such information as application name, transport protocol, initial Java class of application and more.

Next step is to look into the DSM-CC carousel containing the application data. Image below shows that three DSM-CC carousel streams were found. Select the streams so that DVBStreamExplorer DSM-CC analyser will start collecting DSI, DII and DDB sections. These sections will be shown fully decoded on DSM-CC sections tab.

Once all DSI, DII and DDB sections for a carousel have been acquired the data carousels can be decoded. Image below shows raw data of carousel modules found. The object carousel will be decoded from these modules.

Image below shows the decoded object carousel. It shows part of the file directory structure of the application. it also shows the binary content of one of the Java class files that is part of the application.

With DSM-CC analyser in DVBStreamExplorer there is the option to save entire carousel on local file system. After doing so you can then launch your preferred Java decompiler and have closer look at the code behind MHP application. I will leave this as an exercise for the reader.