Giuseppe Guerrasio : Media

Rai Net have expanded use of Live Smooth Streaming for Sports Event: Roma 09 FINA Swimming world championships and Berlin 2009 Athletics World Championships

Giuseppe Guerrasio — Fri, 11 Sep 2009 08:03:00 GMT

How I introduced in my previous post , Rai Net the internet company of RAI, the Italian public broadcaster, during the last FIFA Confederations Cup South Africa 2009 delivered free, live Web broadcasting in high definition television quality (720p) of the Italian soccer team matches during the tournament using Live Smooth Streaming .

Smooth Streaming is an innovative technology based on IIS Media Services and Silverlight, which the scope to simplify and make a reality delivering video content at high quality level through Internet up to true HD for Live and VOD.

Smooth Streaming permit to solve one of the most significant limits in Internet video streaming connected to typical behaviors of the last mile for reproducing High Quality videos with high bitrates. When we think to the way that the content is delivered to the client, we have a lot of parameter that change every second during the client reproduction. In particular bandwidth and CPU power availability conditions change every second and causing a video glitch and buffering during the reproduction of the content, especially for the content at high level of bitrates, generating a poor user experience.

During the reproduction of the content Smooth Streaming dynamically detects current network and local PC conditions, and seamlessly switches the video quality of a media file that a Silverlight player receives. Consumers with high-bandwidth connections and newer hardware can experience HD quality streaming, and others with lower bandwidth speeds or older hardware receive an appropriate stream for their conditions, allowing the audience to enjoy an uninterrupted streaming experience with the highest quality possible.

In my previous post for Confederation Cup and for Smooth Streaming and in general and in the official web site , you can found more information about this innovative technology

After the success of this first test for Live Smooth Streaming, Rai Net have expanded the use of this technology for sports event on Rai Sport portal and delivered, up to true HD for Live and VOD after the Live event: Roma 09 FINA Swimming world championships and Berlin 2009 Athletics World Championships.

In the follow pictures you can see some screen shots from Rai Sport Web Site from Roma 09 FINA Swimming world championships were you can see the adaptive part of smooth streaming at work:

An from Berlin 2009 Athletics World Championships:

The architecture used for these events is practically the same used for the Confederation Cup and illustrated in the previous post, with the exception of the CDN because for this event RAI use two different CDN and balance the load on both at 50%. In the follow picture you can see the architecture used for these two events:

The original HDTV 1080i signal in input is encoded in 7 different bitrates streams composed by a contiguous 2 secs video chunks codified in VC-1, created by 2 encoders. Each encoder creates 3-4 streams and synch through the encoders are achieved with a timecode generator in input in the encoders with the source video. Encoders push the streams to an origin server farm in load balancing were the two CDN pull for distribute Live, archive streaming and XAP code and offers these to Silverlight client player published on RAI.IT and RAI.TV portal .

RAINET Test Live Smooth Streaming in HD for Italian soccer team matches during Confederation Cup

Giuseppe Guerrasio — Mon, 22 Jun 2009 17:42:00 GMT

How I have introduced in my previous post , Smooth Streaming is an innovative technology based on IIS Media Services and Silverlight, which the scope to simplify and make a reality delivering video content at high quality level through Internet up to true HD for Live and VOD. Using this technology RAI.IT, the internet company of RAI, the Italian broadcast host of FIFA Confederations Cup South Africa 2009, in cooperation with Microsoft Corp., Inlet Technologies and Level 3 Communications, offered exclusively for Italy (video content was geo-blocked to Italian IP) on www.raisport.rai.it and www.rai.tv free, live Web broadcasting in high definition television quality (720p) of the Italian soccer team matches during the tournament.

In the next picture an image of the player used during the streaming of the event:

RAI.IT use already Smooth Streaming for VOD on RAI TV portal (http://www.rai.tv ) in the High Quality area (not in HD but to 480p) and now starts to adopt this technology for Live sports event.

The original video source was acquired by specific encoders and transformed in a set of separated streams at different quality (one for different bitrates level) based on a contiguous sequence of 2 sec of video chunks, and an XML manifest with the metadata. These streams are published to the client through a set of servers based on IIS7 with IIS Media Services . In Live streaming all the streams are offered to the client via HTTP and archived automatically in a set of file in the storage of the origin servers. Archived files are republished automatically to the client offering DVR capability to the client in addition to Live Stream offer. For the Confederation Cup matches RAI use the new beta version of IIS Media Services that support Live Streaming.

Video streaming is delivered to the client trough a sequence of HTTP call with REST style and for this reason it is possible leverage all the classic Internet HTTP Cache Infrastructure to increase streaming scalability.

The client part of the Smooth Streaming based on Silverlight detects current network and local PC conditions, and seamlessly switches the video quality of a media adapting the experience at the actual conditions in the client permitting the best experience possible, eliminating buffer and glitch .

The streaming starts with the first REST call from the client to the source and the response from the server with a manifest that describe chunks and streams level offered with the streaming.

The client reads this information and in order to start the download of video chunks and initiate video reproduction requesting the chunk with a too low bitrates for first.

During the download of the first chunk, client code calculate the current bitrates and CPU conditions and in base at this information proceeds with requests of the others chunks, adapting continuously the bitrates of chunks requested.

As written before streaming works with a sequence of HTTP client requests with a REST style to a stateless server and this permit to insert the origin server behind a CDN with a classic HTTP Edge Caching, improving in a simple way the scalability of streaming. For the same reason all the intermediate http caching level (for example enterprise intermediate cache , etc ) can act automatically as a cache for our chunk and contributing to the scalability of our solution.

The client code can offer a lot of services leveraging the capability of the programming platform offered with Silverlight and can consume and presents a lot of contextual data and information with the video. The smooth streaming player element can send back to the server a lot of logging information can be captured using logging functionality of IIS Media Services. It is possible build to service that can permit start and stop of the event and services for control and limit bandwidth consumption from the client.

In the client of RAI event we have start and stop control, bandwidth control and list highlights in the match that can permit to navigate and replay in the archive to the most important point of the event quickly. In this picture you can see the highlights box in overlay on the video in the player.

highlights future leveragin the fact that all the streams are archived automatically in a set of file in the storage of the origin servers. Archived files are republished automatically to the client offering DVR capability to the client .

In the case of RAI Confederation Cup streaming the architecture in place for the event is represented in the next picture :

The original HDTV 1080i signal in input is encoded in 7 different bitrates streams composed by a contiguous 2 secs video chunks codified in VC-1, created by 2 encoders Spinnaker 7000 from Inlet. Each encoder creates 3-4 streams and synch through the encoders are achieved with a timecode generator in input in the encoders with the source video. Encoders push the streams to an origin server farm in load balancing were the Level3 Communication CDN pull for distribute Live, archive streaming and XAP code and offers these to Silverlight client player published on RAI.IT and RAI.TV portal .

Move Forward Internet video streaming: IIS7 Media Service and Smooth Streaming

Giuseppe Guerrasio — Fri, 15 May 2009 13:50:00 GMT

Delivering video content through Internet is one of the hottest technical topics. The increase of bandwidth and connectivity in combination with the numbers from the last big video event delivered like Olympics, demonstrate the potential of Internet channel for this type of content. In the 2008 Summer Olympics Games in Beijing, NBCOlympics.com attracted close to 6½ million users daily, who, on average, stayed more than 13 minutes per visit and spent more than 27 minutes when consuming video. In total, users watched more than 6 million hours of video content or 60 million video streams, which represent a 5-fold and 20-fold increase compared to the Torino and Athens games combined. Another big difference that the last Olympic on line demonstrate it is relative, not only to the number of users, but also to the viewing time. In the past, online video consumption was limited to “snippets” of video, now the prolonged viewing indicates that users enjoy the experience and find the image and sound quality acceptable.

Worldwide, billions of videos are viewed online every month. This type of trend are confirmed to a lot of service that came up on the market with success in the previous months like BSKYB channels offering or the new version of Netflix service and many others.

Setting up an online video service was a complex undertaking and fraught with difficulties, and the process of preparing, loading, and delivering multimedia content, combined with the serving of appropriate advertising, required a lot of manual work and did not really scale very well. In particular “delivering” video content at good or high level of quality through Internet connection is not a simple thing. A lot of details and technical issues are involved on this and build a solution for this and monetize the result is really complex.

In the last NAB conference Microsoft announce a first Release of new technology called Smooth Streaming based on IIS7 Media Services and Silverlight, which the scope to simplify and make a reality delivering video content at high quality level through Internet up to true HD for Live and VOD. You can see this web site from akamai an example of HD content delivered with smooth streaming and at this website you can found others technical information about this technology.

In the previous months before this announcement I had the opportunity to test and build a real production environment based on this technology, inside a new Web TV portal of RAI – Radio Televisione Italiana ( http://www.rai.tv ) . In the new version of this web site recently released there is a new area based on this smooth streaming technology with fiction, sport and junior videos. Content from RAI website have a good quality but is not HD content because the original produced video is not HD. Based on this experience I want write some post on this new fantastic technology that can permit to deliver uninterrupted video streaming to Silverlight clients at High Quality, via HTTP in VOD and delivers massively scalable Smooth Streaming for live events.

Before to go in deep in Smooth Streaming technology and in order to explain difference and advantage that come with this new streaming technique I want recap the typical way used to delivery video content through Internet. Before the smooth streaming you can use essentially two different way for delivering video content:

- Progressive Download

- Traditional Streaming (RTSP, streaming through HTTP, RTMP, etc)

Both approaches have pros and cons and smooth streaming try to combine the best of the previous techniques.

When you use Progressive Download to delivery video content you essentially leverage the capability of Web Server to delivery file.

The Web Server receive a request for a specific URL for the video content and the web server start to deliver the file to the client via classic HTTP.

Big advantages of this solution are:

- simplicity of the approach ;

- not required a specialized server;

- leveraging all HTTP infrastructure in terms of caching and distribution.

Problems connected to this solution are:

- Bandwidth controls;

- Security of content;

- Limited user tracking;

- Poor user experience (no seeking feature on classic web server);

- VOD video content only.

In general the most significant problem of this approach is about bandwidth consumption. In fact the web server delivers the video content to the cache of the client without any type of control or limitation leveraging all the bandwidth available.

You can improve efficiency of progressive download through system that can permit the control of bandwidth and offering control and seeking capabilities. IIS7 Media Services enabled this feature too called Intelligent bit rate throttling. This feature is available for the most used type of media content, and permit to reduce bandwidth consumption saves money on network costs by metering the download speed of multiple media file types as well as data. Intelligent Bit Rate Throttling accomplishes this by automatically detecting the encoded bit rates of 11 common media formats, such as Windows Media Video (WMV), Flash Video (FLV), and MPEG 4 (MP4), and then throttling the response to the client. Bit Rate Throttling and allows administrators to configure throttling rules for all file and MIME types.

With this technique you could obtain more efficiency for bandwidth consumption and simplicity in the delivery but remaining a lot of limitations. For example, in the case of the user put in pause the reproduction the download proceeds consuming bandwidth; in this way we can use of a single bitrates for reproduction or complex variable bitrates technique with poor efficiency , and remaining the problem illustrated before about the last mile and client reproduction, in particular for content with high quality and consequent high bitrates.

When you use classical streaming you use a specialized server to delivery to the client only the portion of the stream involved in the reproduction of the video in base of the command received from the client.

Big advantages of this solution are:

- Efficiency in control and delivery of content ;

- Simple way to track user activity;

- Good user experience (seeking, forward, etc);

- Good for VOD and Live .

Problems connected to this solution are:

- Required specialized services ;

- Not cache in HTTP CDN;

- Scalability required a specialized network .

In general the most significant problem of this approach is about scalability. In order have a large distribution of video content is necessary achieve a specialized infrastructure of server with a specific streaming services. Is not possible reuse the big cache infrastructure that we have in place on Internet today for HTTP . In order to delivery this type streaming, Windows Server offers a good platform through the Windows Media Services today available for Windows Server 2008.

When you use Smooth Streaming now available in beta with IIS7 Media Services , you have the capability to deliver media at variables bit rates to Silverlight clients over HTTP combining the advantage of streaming and progressive download and adding capability of adapt the bit rates at the real situation in the last mile in terms of bandwidth and CPU.

With the Smooth Streaming you deliver video content starting from a set of specific encoded file with different bit rates level. During the encoding phase, realizable today for VOD (with Expression Encoder 2 SP1 with adaptive streaming encoding profile) or for live using a specialized encoder from an encoders vendor, you obtain the source files to stream. You can choose a set of different bitrates and obtaining a set of N file one for every level of bit rates that you have selected in the profile and a xml manifest with the metadata for server and an xml file for client. You can choose at maximum of 10 different bit rates. The manifest files are used from the server and from the client to understand level of bitrates available for streaming and others information relative to the media content.

A typical Smooth Streaming media asset (presentation) consists of the following files:

MP4 files containing video/audio

- *.ismv Contains video and audio, or only video (one file ISMV file per encoded video bit rate )

- *.isma Contains only audio

Server manifest file

- *.ism -Describes the relationships between the media tracks, bit rates and files on disk . Used by IIS Media Services Server

Client manifest file

- *.ismc - first file delivered to the client. Describes the available streams to the client: bitrates level available, codecs used, video resolutions, markers, captions, etc.

For live streaming the same structure is created in the archive of event and client manifest is generated at runtime from encoding available. Origin Media Server can archive all live streaming and offer DVR capability to the client.

In every video\audio file (ismv , isma) you have a contiguous sequence of video chunks at fixed time length (default 2 sec) based on the ISO/IEC 14496-12 ISO Base Media File Format specification, better known as the MP4 file specification . There are actually two parts to the Smooth Streaming format: the wire format, and the disk file format. In Smooth Streaming, a video is recorded in full length to the disk as a single file (one file per encoded bit rate), but it's transferred to the client as a series of small file chunks. The wire format defines the structure of the chunks that are sent by IIS to the client, whereas the file format defines the structure of the contiguous file on disk, enabling better file management. The MP4 specification is perfect for this because allows MP4 to be internally organized as a series of fragments, which means that in Smooth Streaming the wire format is a direct subset of the file format. Every fragment is delivered to the client through HTTP and in this way can be leverage all the HTTP caching infrastructure of Internet.

For VOD and for Live the content are published through IIS Media Service using Windows Server 2008 and IIS7 with IIS Media services Extensions.

During the delivery of the content, the client, in order to reproduce the video, sends a set of specific requests to the server to download the video and audio chunks. In every request for a single chunk, the client specifies level of bit rates and timeframe required in base of the current condition of bandwidth and CPU and position in content reproduction. In this way smooth streaming is delivered with a set of progressive download of video and audio chunks.

The client send a set of HTTP GET request in REST style like this : http://smoothstreaming.com/title.ism/QualityLevels(1775000)/Fragments(video=840000000) that contains information about bit rates and timeframe requested for the chunk . This type of request is a typical HTTP request and is it possible leverage all the http infrastructure on Internet in order to cache this content to near as possible at the client and using a typical http edge offered from a CDN.

A specific part of code downloaded in the client contains the Heuristic Algorithms that drive the requests of chunks. The process start with the base request from the client at the client manifest file on the server in order to obtain the metadata that can guide the behavior of the client. This request have this format : http://smoothstreaming.com/title.ism/Manifest, IIS receive the request send manifest client in response. In the next picture you can see a typical client manifest response with the indication of the bit rates available for the content downloads.

The client manifest can contains more information about media metadata like : codecs, Text streams, Multi-language audio tracks, Alternate video and audio tracks (for example, multiple camera angles, director's commentary, etc.), Multiple hardware profiles (for example, a bit rate targeted at different playback devices) , Script commands, markers/chapters, captions

The process continue with the request of the first chunk video and audio, requested with the low bit rates available in the manifest with a REST style call described before . The IIS Media Service Module receive the request and use a server manifest file to determine the file to load in base of the request bitrates and provides to the right chunk of video. In the next picture you can see a sample of server manifest file with the indication of file for bitrates:

For the successive request based on download speed, the client can detect available bandwidth and requesting the right bit rates in base of the current situation of bandwidth and CPU. In this way the smooth streaming have the capability of adapt continuously creating an immersive User Experience. The process Work in a similar way for Live and VOD streaming and during the delivery of the video chunk through Internet, all the HTTP caching infrastructure of the network can be leverage in order to improve scalability in particular for Live Streaming. In the case of Live Smooth Streaming the Origin media server can archive the content produced from the encoder and offering DVR capability too during the Live Streaming.

There is a lot of advantage in this approach respect to traditional streaming technique. In traditional streaming, the client state is managed both by the client and the server. The server keeps a record of each client for things such as playback state, streaming position, selected bit rate (if multiple bit rates are supported), etc. While this gives the streaming server more control, it also adds overhead to the server. In terms of scalability another important limitation is about the fact that each client has to maintain the server affinity throughout the streaming session.

With the pull model in VOD or Live Smooth Streaming, the server is stateless, the client drive the process and has the responsibility for the state and this fact increase scalability and remove any single point of failure. Any client request can be satisfied by any server configured and the network topology can leverage a stateless infrastructure and can reroute the client requests to the best server. In this way we have a big improvement in:

- quality because all the decisions, resulting in a much faster response (for example bit rate switching) offering a superior UX :

- reproduction start fast because the client select the low bitrates and can download in parallel more chunks, potentially cached from the most near web server

- bitrates are continuously adapted at the real situation in the last mile in terms of bandwidth and CPU status

- requests can be served from different servers without affinity or persistent connection improving scalability and fault-tolerance

- scalability because you can leverage a simple, efficient and load balanced HTTP infrastructure to deliver video content

- cost because you can leverage an HTTP CDN with generic web server with low number of specialized server and leverage all the HTTP cache ( for example intermediate proxy )

Al the URL obfuscation technique can be used to protect the content and is announce the support of Playready content protection encryption in order to protect smooth streaming for Live and for VOD.

Smooth streaming enable a new video delivery mechanism that can permit to stream HD content in VOD and Live through Internet with a cool UX .

In next posts we can see in more details the three phase for create and publish content with this technology .

IIS7 e gli scenari video in progressive download

Giuseppe Guerrasio — Wed, 19 Mar 2008 16:00:00 GMT

Una delle tendenze più rilevanti nell'evoluzione dei contenuti nel mondo del Web è sicuramente rappresentata dall'enorme crescita dei contenuti video che oramai sta investendo siti di ogni dimensione. Si va dalla possibilità di ospitare materiale video generato dagli utenti, news, film , trailer.

Erogare contenuti video è estremamente costoso in termini di consumo di banda e gli scenari attraverso i quali viene generalmente affrontata questa tematica sono essenzialmente di tre tipi:

- Hosting dei contenuti video presso strutture terze specializzate nella distribuzione di questo tipo di contenuti attraverso CDN (Content Delivery Network) specifiche

- Infrastruttura di streaming installata nella server farm

- Download progressive , ovvero distribuzione del file attraverso un server web

La piattaforma Microsoft offre una soluzione per tutti e tre gli scenari :

- Silverlight Streaming che offre una CDN per la distribuzione di contenuti video e di applicazioni Silverlight in generale

- Windows Media Services per Windows Server 2008 che forniscono una piattaforma di streaming integrata in Windows Server con completo supporto per il DRM

- IIS 7 Media Pack che offre nuove funzionalità ad IIS7 per gestire gli scenari di download progressive , da poco disponibile nella versione finale.

Uno degli scenari tipici, in particolare per la distribuzione di video di piccole dimensioni è lo scenario di download progressive, dove il file viene posizionato semplicemente sul server web, mentre lo streaming viene utilizzato in scenari più complessi o per scenari in cui vengono trasmessi video in diretta. Nel download progressive, non abbiamo le ottimizzazioni sulla banda offerte dal server di streaming , il consumo di banda è elevatissimo in quanto il server web effettua il download del file verso il player client utilizzatore, sfruttando completamente la banda disponibile, indipendentemente dal bit rate con cui il video è codificato per la visualizzazione. Inoltre, nella maggioranza dei casi, l'utente guarda solo i primi secondi del video che nel frattempo va in download a banda completa praticamente inutilmente.

Nel nuovo Media Pack per IIS7 troviamo le funzionalità di BitRate Throttling che consentono di gestire gli scenari di download progressive, permettendo di ottimizzare il consumo di banda in modo configurabile sulla base del reale formato di codifica del file, ed a garantire un maggior numero di connessioni contemporanee possibili. Tra le caratteristiche fondamentali del nuovo supporto il media pack installa un modulo che permette di effettuare il download del file in base al bit rate di encoding. La risposta ed il funzionamento del modulo è completamente configurabile dal livello del server web, fino ad arrivare al livello del singolo file.

La lista dei formati supportati è la seguente:

.asf
.avi
.flv
.m4v
.mov
.mp3
.mp4
.rm
.rmvb
.wma
.wmv

E' possibile estendere il Media Pack per supportare ulteriori formati.

Per installare il MediaPack Bit Rate Throttling potete trovare qui il setup:

32 bit - http://www.iis.net/downloads/default.aspx?tabid=34&g=6&i=1762

64 bit - http://www.iis.net/downloads/default.aspx?tabid=34&g=6&i=1764

Per dimostrare la funzionalità del media pack e la semplicità di configurazione ed utilizzo, ho effettuato un piccolo test installandomi una macchina di prova. Sulla root del sito ho posizionato un file di esempio;

E l'ho pubblicato con la funzionalità di throttling disabilitata a livello del server web:

Di seguito potete vedere l'andamento del consumo di banda realizzato durante la riproduzione del video:

Il Media Pack per configuare il Bit Rate Throttling offre una interfaccia completamente integrata nell'ambiente di amministrazione di IIS7

Dall'interfaccia provvedo ad abilitare la funzionalità :

ed in particolare di seguito vediamo la configurazione impostata per i file di tipo .wmv:

Con la configurazione impostata , i primi 20 secondi necessari per l'avvio rapido del filmato verranno forniti a banda completa (FastStart) , successivamente il download avverrà al 100% (Throttle Rate) del bitrate di encoding del video che nel nostro esempio è di 300kps.

La funzionalità di Bit Rate Throttling può anche essere applicata ad altri tipi di contenuti , configurando la parte data dell'entry e stabilendo quindi il la banda assegnata per il download

Effettuado nuovamente il play del video l'andamendo del consumo di banda in questo caso avrà un andamento che è il seguente:

Contribuendo significativamente ad ottimizzare il consumo di banda , ed a garantire un maggior numero di connessioni contemporanee possibili.

Per uteriori informazioni vi suggerisco i seguenti walkthrough:

Tra le altre caratteristiche offerte da IIS7 all'interno del Media Pack vi segnalo anche la possibilità di costruire delle playlist server side anche per i contenuti in download progressive, funzionalità utile ad esempio per costruire playlist introducendo contenuti pubblicitaril. Informazioni e Walkthrough di utilizzo qui .