Introduction

This document is to complement the original Expression Encoder 4 SP1 GPU Encoding white paper and discuss the changes that SP2 introduced on this subject. The latest service pack includes a major revision of the CUDA H.264 encoder as well as the introduction of Intel Sandy Bridge QSV support, which both will be discussed in depth below.

Please note that an updated performance report for Cuda GPU Encoding as well as a new one for QSV GPU Encoding are available.

General GPU Encoding Recommendations

To Maximize Performance

As before, we recommend applying the proper GPU for video processing load. We provide below basic guidelines for optimizing your performance using a GPU to encode. Using proper GPU hardware and a conservative "GPU Stream" setting can reduce the chances of maxing out the GPU resources. Another way to reduce the GPU memory footprint is by lowering the "Number of Reference Frames". GPU-Z is a great freeware to monitor the load and memory usage.

To calculate the appropriate number of GPU streams setting for a specific scenario, we also created a simple performance tool. More information is available here.

Finally, since available GPU memory is more important than ever, we recommend closing any other applications to minimize GPU memory usage outside of Encoder.

To Maximize Quality

• Consider using 1-pass constrained VBR over CBR.
• Make the "Buffer Window" longer. Note that this will impact latency.
• Use accurate pre-processing settings (aka de-interlacing and resizing methods) for a given source. For example, using the wrong de-interlacing method can introduce weave artifacts, which are hard to encode and will significantly impact overall output quality.
• Consider removing any letterboxing introduced by the source and/or the encode settings. Those introduce hard high-contrast edges which are hard to encode and will also impact output quality.

Second Generation of CUDA GPU Encoding

In collaboration with our partners at Main Concept, a lot of work was done on the CUDA H.264 encoder to optimize the performance of the GPU, as well as improving output quality and adding new encode functionality like true CBR and HRD support. Those changes have significantly impacted its behavior, especially in terms of performance:

1. GPU memory requirements have doubled using with the same settings when compared to SP1.
2. GPU load has more than doubled too.
3. CPU load has been reduced when encoding with CUDA.

In other words, you will experience better quality and less CPU load, at the cost of higher GPU load and some overall performance loss.

Finally, because the CUDA encoder doesn't support 2-pass encodings, they have been disabled. Please note that while Encoder will revert to software encoding if a 2-pass encode is selected, the new 1-pass H.264 VBR encoding options are fully supported in both CUDA and QSV-based encoding.

Recommendations

Because of the new behavior, we now recommend using only higher-end CUDA GPUs for this feature. A rough suggestion would be to get CUDA hardware with a minimum of 200 CUDA cores per HD stream to be encoded via CUDA and at the least 1 GB of video memory (2 GB or more preferred). It means that for encoding a single HD stream on the GPU, at the least a GeForce GTX 550 Ti or Quadro 2000 GPU is recommended, while encoding 2 or more HD streams on the GPU would require at the least a GeForce GTX 570, Quadro 6000 or Tesla C2050 card. As in Encoder SP1, multiple video cards can be used to speed up Smooth Streaming encodes. Note that we also changed our default of number of GPU streams to 2, which can be tweaked to take full advantage of the hardware used.

Since the GPU is taking more of the load, matching the CPU may not be as important for single stream encodes. Because most current CUDA GPUs can't take more that 2-3 HD streams at a time, a powerful CPU is still required for HD Smooth Streaming encodes.

Finally, make sure to install the latest Nvidia drivers available from Nvidia's website here. Note that the GPU driver version is required to be higher than 267.91 for SP2.

New Intel Sandy Bridge QSV GPU Encoding Support

What is QSV?

Intel Quick Sync Video (QSV) is a GPU acceleration technology available on some Intel processors that can dramatically increase the speed of transcoding media files. It is available on most 2^nd generation i3, i5 and i7 processors (aka "Sandy Bridge"), as well as some newer Xeon CPUs, like the E3-1285. More information about QSV is available on Intel's website. Please note that the newly Sandy Bridge E CPU line (example: i7-3960X) does not have an embedded GPU, and thus does not support QSV.

QSV encoding support has been integrated in Encoder 4 Pro SP2, which can be used in a similar fashion as CUDA encoding to speed up MP4 and H.264-based Smooth Streaming encode operations.

Usage

QSV GPU encoding works the same way than CUDA encoding as explained in the Encoder SP1 GPU Encoding white paper. Because the two GPU-based H.264 encoders behave very differently, Expression Encoder 4 Pro SP2 provides a way to control the number of streams independently between CUDA and QSV, which are defaulting to 2 and 8 respectively (see below). As with CUDA, QSV encoding is turned on by default in the Encoder application and turned off in the SDK.

As for the SDK, the QSV device is enumerated in the H264EncodeDevices.Devices along with the CUDA devices. A new H264EncodeDevice property, DeviceType of type H264DeviceType, provides a way to differentiate between the two GPU types. H264EncodeDevices.MaxNumberGPUStream keeps controlling the number of streams to be encoded by CUDA, while H264EncodeDevices.MaxNumberIntelHDGraphicsStream controls the streams encoded by QSV.

Multi GPU Encoding

Both CUDA and QSV encoding can be used together, either on one multi-stream encode or multiple instances of encoder, which may provide even better performance results. While encoding multiple streams on a PC with both technologies, the streams will be allocated in a snake pattern starting with the Intel GPU (1-2-2-1, 1-2-3-3-2-1) from the highest resolution stream down.

It's worth mentioning that, when used with a discrete GPU, Intel QSV requires the Intel GPU to be enabled as default video adapter in the BIOS and needs to be active and connected to a monitor. Note that LucidLogix Virtu technology, bundled with some Sandy Bridge PCs, enables virtualization of the discrete GPU through the Intel GPU, enabling the usage of both QSV and Cuda and using only one output. More information is available on LucidLogix' website.

Limitations

Like the CUDA implementation, the QSV supports only a subset of encode settings. For instance, those encoding features are not supported:

1. Reference B-frames
2. Adaptive B-frames
3. Multi pass encoding (2-pass encodes will use software encoding)

QSV encoding is not supported on XP.

An active Intel HD Graphics GPU is required to be accessible. This means that:

1. The PC's motherboard needs to support the Intel Accelerator GPU (most 1155-H67 and 1155-Z68 motherboards do, but all 1155-P67 don't).
2. The GPU needs to be connected to a screen and be activated.
3. Accessing the PC via Remote Access, VM, or via a service isn't supported.
4. If the PC is locked or accessed via Remote Desktop, the next encode operation will fail to use the GPU since it will not be available.

As with CUDA, QSV encoding is an approximation of what the software-only based encoding can provide. Output quality differences, especially at low bitrates should be expected. Because of this, we recommend using the software H.264 encoder if highest quality results are a priority over higher performance.

Recommendations

The integrated Intel Accelerator 3000 GPU, found in high-end mobile CPUs and in the K line of 2^nd generation i5 and i7 based desktops as well as on some Xeon and i3 CPUs, is highly recommended. See the Intel website for more details.

Because memory is shared, we highly recommend having a minimum of 4 GB for single stream encoding and 8 GB or more for Smooth Streaming and/or parallel encoding. Running out of system memory will greatly affect performance.

It is highly recommended to install the very latest drivers directly from the Intel website to ensure best stability and reliability.

While the QSV encoding can take a large load of concurrent streams, it is worth noting that a blend of hardware and software encoding will likely result in better performance in most Smooth Streaming cases.

Known Issues

A few issues were found in the recent sets of Intel drivers currently available. While all of them have been investigated and resolved, some of the fixes didn't make it into the latest drivers, but will be available in the next driver version (ETA early 2012). Here is the short list:

1. When using either Intel's driver 8.15.10.2361 or 8.15.10.2509, about 5% of H.264 Smooth Streaming encodes using QSV will hang at the start of the encode or may contain a corrupted stream. This issue was fixed in the driver 8.15.10.2559.
2. When using Intel's driver 8.15.10.2361 and encoding to QVBR in H.264 Main or High profile using QSV, output will be corrupted. The issue was fixed in the driver 8.15.10.2509.
3. Windows 8 Beta Users: Intel's current Windows 8 drivers do not yet support QSV, but will be in the future.

Other GPU Acceleration Technologies

OpenCL and DirectCompute technologies were evaluated but deemed not ready for integration in Expression Encoder 4 Pro SP2. We are planning to continue monitoring them and investigating potential integration in the future.

A new service pack is now available for Expression Encoder 4, which includes over 300 bug fixes as well as these new features:

General Encoding

Support for Intel QSV GPU encoding (Pro only) – GPU-accelerated encoding has added support for the Intel HD Graphics GPU included in Sandy Bridge processors, in addition to the improved Nvidia CUDA support. You can use both of these devices when encoding for increased speed and performance. More information available here. QSV and Cuda performance reports are also available.

Support for 1 and 2-pass VBR for offline H.264 encodes (Pro only) – You can now select 1-pass VBR for MP4 encodes as well as both 1-pass and 2-pass VBR for H.264-based Smooth Streaming encodes.

Support for more AAC audio options (Pro only) – You can now choose multiple-channel audio along with more bitrate and sample rate options when you use the AAC codec.

Support for SRS audio encoding – You can now choose to enable SRS to convert an audio stream from multiple audio channels to two SRS 5.1 surround sound channels.

Live

Manage publishing points from within Expression Encoder (Pro only) – In a Live Broadcasting Project, you can now create, restart or update publishing points on an IIS Smooth Streaming server. You can also choose to automatically update a broadcast’s EventID.

Live crop – You can now crop your live or file sources in a Live Broadcasting project.

Support for Windows Media Streaming MBR in a Live broadcast – You can now add multiple bitrate streams to your WM Streaming broadcast in a Live Broadcasting Project.

Support for Live H.264 VBR 1-pass (Pro only) – You can now select 1-pass VBR encoding for H.264-based Live Smooth Streaming encodes.

Other

No Screen Capture limit – The 10-minute Screen Capture limit has been removed from Expression Encoder.

Follow mouse cursor – You can now enable the screen capture area to follow your mouse cursor in Screen Capture.

Template improvements – Offline playback of DRM content, SRS playback support, improved TTML support, audio volume on startup, auto-repeat, etc.

New SDK samples – Following your requests and suggestions, we added 5 C# and 6 VB.NET samples, including a thumbnail generator, a performance tool and a Live console application.

Let us know what you think! Thanks.

Once in a while on our forum, somebody asks why the CPU usage isn't at 100% when offline encoding. I recently took the time to provide a thorough answer, which I think would be beneficial posted here.

So first, let's explore what's happening when encoding a media file using Expression Encoder. The encoding process involves 5 phases:

1. Reading the source(s) from storage.
2. Demuxing and decoding the source(s) into uncompressed frames.
3. Pre-processing the frames (deinterlacing, resizing, cropping, etc).
4. Encoding and muxing the frames.
5. Writing the stream to storage.

If any one of those operations is slower than the others, all the encoding pipeline will suffer in terms of performance because it will bottleneck the flow of frames into it.

#1 and #5 are usually the bottleneck when source files are either not local (aka slow Network share) or are very high-bandwidth coupled with a slow storage. Possible solutions: making sure the source files are copied locally on your fastest storage available and outputting to a different local storage are good solutions to reduce the bottleneck in those two areas.

#2 usually becomes a bottleneck because of the type of sources and the codecs used to decode them. Obviously, using complex HD sources will use significantly more resources than simple lower-resolution sources. Some 3rd party codecs are extremely slow, running only on one core. Trying to test other codecs available on the PC by disabling some of the codecs listed in the "Tools -> Options -> Compatibility" dialog may help reducing the bottleneck. It's also worth noting that because many codecs are single-thread, having faster cores can help enormeously here. This is the main reason why a 3.4MHz 4-core is faster than a 2.6MHz 8-core PC in many cases.

#3 can become the bottleneck when unneeded cropping/rescaling is applied, when "SuperSampling" resizing and/or "Auto Pixel Adaptive" deinterlacing are applied. The last two are our defaults, which were the best choice for "high quality" encodes at the time we shipped v4 RTM over 18 months ago and we unfortunately can't change the defaults until our next major version since it would be an SDK breaking change. For better balance between performance and quality, we highly recommend using "Bicubic" resizing and "Auto selective blend" deinterlacing. Both of them will perform 2-3x times faster than the defaults and will reduce the chances of the pre-processing phase being the bottleneck. Of course, if at all possible, removing the need for a resize (aka using the same frame size and pixel aspect ratio as the source) will also reduce the load on that phase while keeping better output quality.

This leaves phase #4, where most of the CPU cycles should be used if there are no bottlenecks elsewhere in the pipeline. In the best of worlds, this phase should be your bottleneck, which would very likely max out CPU usage on an 8-core PC for a single-stream encode. Depending on the encode settings, 100% may still not be achieved, but it should be pretty high. Here are few ideas to make this phase faster:

1. Make sure to use the defaults for number of threads and slices used to encode (where applicable).
2. If speed is most important, consider applying our "Fastest" Quality preset, which will significantly speed up encoding process at the cost of some output quality.
3. Use proper encode settings. For example, trying to encode 480p content into 300kbps would be stressing the encode phase and could slow it down to a crawl. We recommend about 1.5+ Mbps for 480p and about 3+ Mbps for 720p. Consider reducing the frame size and/or frame rate if you are trying to achieve significantly lower output bitrates than this.
4. If GPU encoding is enabled on a slow GPU when encoding to H.264, it could make the GPU the bottleneck. Solution: disable GPU encoding or get a better GPU.
5. On larger scale PCs (16 cores and above), single one-stream encodes can't maximize the CPU usage. Parallel encoding is an option to maximize the PC resources. If using a NUMA enabled system, process affinity is highly recommended to ensure memory is not shared across CPUs between the different encoding processes, which can reduce performance.

In the case where nothing seems to help maximizing CPU usage, one could also consider running 2-4 jobs in parallel in separate encoding processes. Using the UI, simply start 2-4 Encoder instances and run the encodes in parallel, bearing that enough system memory is available. This suggestion would certainly not help with a storage bottleneck situation (aka phase #1 and/or #5), but should greatly help speed up the process of encoding multiple jobs.

Finally, if encoding to H.264, using GPU encoding may drastically help speed up the encoding phase. Cuda is supported in Encoder 4 Pro SP1, and Intel QSV (Sandy Bridge) in our next SP release coming soon. When used with the proper hardware, either (or even both) options can cut down encoding time by more than half, and in some cases significantly more.

Hopefully, this gives you some insights on how to isolate and resolve encode performance issues.

When importing certain media files like VOB, TS and M2TS files, Expression Encoder needs to parse through the file to ensure the duration and video frame timestamps are accurate by creating its own corrected table of content in memory. Without this analysis, the encoded output files could have the wrong duration (either trimmed or having extra black frames at the end) and audio video sync could not be guaranteed, especially if the file was trimmed.

This analysis operation, called indexing, is unfortunately time consuming since it requires to read and demux the whole media file and typically requires a few minutes per hour of content, mostly depending on storage speed. Because of this, it is recommended to first copy this type of media file locally before importing it to ensure maximum efficiency.

Note that while this operation is automatically executed upon import within the Encoder application, it is not executed by default when importing a media file via the SDK and needs to be explicitly called before the MediaItem constructor via the CalculateDuration member of AudioVideoFile like this:

                     AudioVideoFile file = new AudioVideoFile(filename);

                     file.CalculateDuration(null);

                     MediaItem item = new MediaItem(file);

It is worth pointing that if the CalculateDuration method isn't called, there will still be indexing done in-line during the encoding which will fix the timestamps, but it is possible that the duration of the output file will be wrong. For this reason, we highly recommend using the code above if source files that require indexing are used.

 Enterprise video is increasingly being recognized by industry experts, such as Gartner, as an important strategic technology.  Accordingly, the Expression Encoder and IIS Media Services Team attended the new Business Video Expo, part of ITEXPO East 2011.  The event included about 180 exhibitors and attracted nearly 8,000 attendees, 80% of whom were executives.

Microsoft exhibited with Accordent Technologies, a partner that has developed an automated software-based video creation, management, publishing, and delivery platform leveraging Microsoft infrastructure in the enterprise, including SharePoint, Lync, Windows Server, IIS, Windows Media Services, Silverlight, and Expression Encoder.

The team participated in conference panel discussions on topics ranging from mobile delivery to YouTube in the enterprise (videos of those panels coming soon), and IIS Media Services senior product manager Chris Knowlton was recorded doing both an in-booth Expression Encoder / IIS / iPad demo and an on-site interview with Steve Vonder Haar. 

Additionally, both Microsoft and Accordent were recipients of Best in Show awards - Expression Encoder was awarded Most Innovative Product and Accordent received Best Enterprise Solution.  Congrats to the team! 

Microsoft and Accordent recently participated in a webinar to talk about how the companies are addressing the challenges of deploying video in the enterprise.  To view the archived content, please login here.   

In the service pack for Expression Encoder 4, we added the ability to specify an EventID for live smooth streaming broadcasts.  This feature is extremely useful to prevent down-level cache pollution (where you can here/see stale cached fragments when reusing publishing points).   You can use any string that contains URL safe characters or clicking the Generate button will put the current date/time as a string.

Another benefit of the EventID field is that it will allow you to restart your publishing point from a Stopped state.  When the Encoder stops broadcasting to an IIS Smooth Streaming publish point, we send an empty 'mfra' box which informs the publish point to switch to a Stopped state.  This allows for DVR playback to be available to the users immediately after the broadcast ends.  The Encoder does not allow you to connect to a publishing point that is in use and to check this, we check to see if there is a manifest associated with the pub point.  A manifest will exist if the publishing point is in a Started or Stopped state so by default we block restarting the publish point.  To work around this, if you specify a different EventID for each broadcast (including the first one, null doesn't count)  to the publishing point that contains the tag:   <meta name="restartOnEncoderReconnect" content="true" /> :  We will ignore the manifest check and restart the publishing point.  Here are the steps:

1. Install Expression Encoder 4 SP1
2. Install IIS Media 4.0 on your server
3. Create a Live Smooth Streaming publishing point on the server.
4. Using Explorer, navigate to the location of the publishing point.
5. Open the ISML file in a text editor and add the following between the <head> tags:  <meta name="restartOnEncoderReconnect" content="true" />
6. Now set up your broadcast in the Encoder and point to the publishing point.
7. Before clicking Start, specify an EventID on the Output panel.
8. Now if you want to reuse the same publishing point for another broadcast,  just specify a different EventID and click Start

Today we have released a service pack for Expression Encoder 4 that includes some new features and a bunch of bug fixes.  Here is a list of the new features we have added:

• GPU encoding - If you have an NVIDIA CUDA-accelerated GPU board in your system, you can now take advantage of the additional processors to speed up encoding time or squeeze more live streams into your broadcasts.  Check out Eric Juteau's white paper for all the details.
• Screen capture as a live source - Now you can select the screen as a device in Live projects to stream or archive.  
• Live templates - Before if you wanted to use the Expression Encoder templates, you needed to hand edit the template to point to your live source.  Now you can generate the template directly from the Live project and we will hook up all the links.  The workflow is slightly different than on-demand templates.  Since these templates usually need to be live before you start broadcasting, you will need to generate the files manually beforehand.  Once you have selected your output type, the Generate button will enable on the Templates panel where you can manually create the files or upload them to a WebDAV enabled server.  If the panel doesn't appear after upgrade, you can find it on the Windows menu.
• Live Push/Pull - Now you can broadcast and publish to a Windows Media Services publish point at the same time.
• Improved De-interlacing Options - We have added a new de-interlacing option called 'Selective Blend' which provides a higher quality picture without the high CPU requirements of Pixel Adaptive.  We have also exposed this and the other de-interlacing options in live.
• Import script commands - Both in live (no timing) and on-demand, you can now import multiple captioning formats (ISMT, DFXP, SRT, XML, SAMI, SUB, TXT) to the Script Command panel.  This allows you to pre-propagate your captions in live from a script or do some basic editing of your scripts before encoding.  If you do not have a caption file for you encode or broadcast, you can now easily create a 'return-delimited' TXT file with your captions to import in either project type as well.
• DRM Content Key - If you PlayReady license server supports Content Key encryption, we now allow you to specify this key instead of a Key Seed to enable individual keys per item.
• HE-AAC - We have added both Version 1 and Version 2 of the HE-AAC codec to allow encoding for supported devices.
• New Windows Phone 7 presets - Now that the WIndows Phone 7 is out (they rock!), we have created presets to enable direct playback (WMV, MP4) or creating video to use in a Smooth Streaming application using the Silverlight Media Framework phone template.

Check it out and let us know what you think!  Thanks

Jamie

Hi everyone, we just posted a publish plugin to upload your output directory to the new SharePoint 2010 Asset libraries.  Check it out:

http://gallery.expression.microsoft.com/en-us/EESharePointPlugin?SRC=Home

jamie

Thanks to everyone that voted for us in the Streaming Media Readers' Choice Awards!  We are a finalist in the Encoders under $1000 category.

http://bit.ly/bhTepe

September 2010

Authors:  Jamie Lang and Sam Wookey

 INTRODUCTION

Expression Encoder 2 was the first version of Encoder to support encoding content into the IIS Smooth Streaming format. In Expression Encoder 4 Pro, we have taken this support one step further with the introduction of Live Smooth Streaming. Now, you can stream audio and video from your webcam or capture card directly to an Internet Information Services (IIS) server with the latest IIS Media Services installed. Live Smooth Streaming content is delivered over existing HTTP networks thus reducing the custom infrastructure needed to broadcast a live event. CDNs and content providers no longer have to worry about how much capacity they can dedicate on their much more resource constrained streaming networks. They can take advantage of all the resources of their HTTP network and do not have to worry about maxing out network capacity and shutting out users from live events. For a user viewing a Live Smooth Streaming broadcast, they no longer need to worry about watching a video feed tailored to the lowest common denominator of consumer hardware. A Live Smooth Streaming broadcast contains different video quality levels all the way up to true HD quality (720p+), allowing users on cells phones and the fastest PC’s to consume video from the same stream at a quality level appropriate for their platform.

Live IIS Smooth Streaming is only available in the Pro version of Expression Encoder.  You can purchase it individually or as part of Expression Studio 4 Ultimate.

In the rest of this article, we are going to walk through the steps needed to get up and broadcasting your first Live Smooth Streaming event. At the end, we have included a few links where you can find more information about Live Smooth Streaming and ways that you can take advantage of it for you particular application. 

 SETTING UP IIS MEDIA SERVICES

The first step in using Live Smooth Streaming in Expression Encoder 4 Pro is to download and install IIS Media Services for Internet Information Server. IIS Media Services provides the server side component which distributes Live Smooth Streaming content to clients (or other edge servers).  IIS Media Services can be installed using the Web Platform Installer (WPI) or from the IIS download site. The following prerequisites must be fulfilled prior to installing IIS Media Services:

• You must use IIS 7 running on one of the following operating systems:
  • Windows Server 2008
  • Windows Server 2008 R2
  • Windows Vista with SP1
  • Windows 7
• To manage IIS Live Smooth Streaming using the IIS Manager user interface, the IIS Management Console for IIS must be installed.

The IIS site contains an excellent tutorial on installing and configuring the IIS Media Services. This page will take you step by step on installing Live Smooth Streaming support on the server, creating your first publishing point, and testing your installation using sample content.

Live Smooth Streaming in Expression Encoder 4 only supports Push mode on a publishing point. This means that Encoder will push content to the IIS server (as opposed to Pull mode, where the IIS server will pull video from a support encoder). It is important to make sure that all publishing points you create that you wish to use with Expression Encoder 4 have the Push option selected under the Live source type combo box, as in the picture on the below.

If you are running IIS Media Services 4.0 Beta, you can also set up the publish point to transmux the stream into a format compatible with Apple mobile devices along with a .m3u8 format manifest.   This will require that the Expression Encoder is streaming using H.264 Baseline (iPhone) or H.264 Main (iPad) profiles.  When setting up the publish point, there will be a new tab called Mobile Devices.  Checking the Enable output to Apple mobile digital devices will enable this feature.  Then you can set up a webpage using a HTML 5 video tag to point to the new .m3u8 manifest:

        <video width="640"
               height="480"
               src="smoothApple.isml/manifest(format=m3u8-aapl)"
               poster="MyPosterFrame.png"
               autoplay="true"
               controls="true" >Live</video>

For more information, see the Apple HTTP Live Streaming article on the IIS site.

CONFIGURING EXPRESSION ENCODER

Now that you have installed the required server side components, we can begin using Expression Encoder 4 to start broadcasting some content. The first step after installing Expression Encoder 4 is to create a new Live Broadcasting Project from the New Project window.

Next, we need to choose the sources that we are going to use to broadcast.  Expression Encoder 4 can broadcast from a range of different sources, from video and audio capture cards, still pictures or audio and video files.  All of the live encoding functionality that was supported in past versions of Encoder, such as switching between different sources, looping file sources, jumping from one file source to the next, and configuring all of your source properties, are supported for Live Smooth Streaming broadcasting.   Also, make sure you remember to Cue the source you want to begin broadcasting with. The Start button stays disabled until you Cue one source to start with.

After you have added all of the sources that you wish to broadcast from, it is time to configure your encoding settings to use Live Smooth Streaming. To make this easier, we have created many built-in presets to get you started. Expression Encoder 4 supports creating Live Smooth Streaming content using either the VC1 or h264 video codecs, giving you the flexibility to distribute your content on a whole range of different clients and devices. Silverlight clients support the playback of both types of content, but if you wish to broadcast to clients such as an iPhone, you will need to use one of the h264 presets.

The built-in presets will get you started with some basic settings to get you broadcasting quickly. If you wish to configure these settings further, you can find all of the options you want under the Video and Audio tabs. Each stream supports different settings for bit rate and size, and using the various tabs, you can add and remove streams, as well as configure the settings for each stream. See the Tuning Your Broadcast section below for suggestions on optimal settings for your encoding machine.

You can also change the Advanced Video Encoding Settings for the overall broadcast by expanding the chevron under the Video settings expander. However, unless you know what each setting does, it’s best to leave these as the defaults from one of the presets for Live Smooth Streaming, as some of these settings can have a very detrimental effect on your broadcast. Some settings, such as Closed GOP, cannot be changed, as they will create an invalid Live Smooth Streaming broadcast.

Now that you have established that you are creating a Live Smooth Streaming broadcast, you can set the location to push the broadcast. On the Output tab, click on the Streaming checkbox to enable pushing a broadcast to an IIS Live Smooth Streaming enabled server. By default, the Publishing Point radio button is enabled. Broadcast mode is disabled when a Live Smooth Streaming output format is selected.

Type the location of your Live Smooth Streaming publishing point into the Location combo box (Figure 4). The publishing point URL looks like http://<server name>/<virtual directory>/<name>.isml. The <name> for the publishing point comes from the file name you chose when creating the publishing point on the server.

 Live Smooth Streaming publishing points behave differently than Windows Media Services publishing points, so if you are familiar with WMS publishing, there are a couple caveats to keep in mind. Live Smooth Streaming publishing points aren’t created automatically like WMS ones, and they don’t start automatically either when you start streaming to them. They must be created manually on the server and they must be started before broadcasting.   You can set the ‘Start publishing point upon first client request’ checkbox on the Advanced tab of the Edit Publish Point dialog to have it start when the broadcast starts.

Also, a Live Smooth Streaming publishing point must be reset on the server if you wish to broadcast to it multiple times. Each time you start and stop encoding, you must shutdown and restart the publishing point on the server before you can connect to it again.Otherwise, data from the last run will still be on the server and you will encounter errors while broadcasting the next time. This is the most likely source of errors you will receive from the server when using Live Smooth Streaming in Expression Encoder, and your first course of action should always be to shutdown and restart the publishing point on the server before trying to broadcast again.

After you have chosen the publishing point in the Location combo box, click Connect to test your connection to the server. If you have enabled authentication on the server, you will be prompted for your access credentials. If your credentials are accepted and your publishing point is set up properly, then the Connect button will turn green and the status will say Connected. If you encounter any errors while connecting, or your publishing point is invalid, the Connect button will turn red and an error message will show in the status area.

Now that you have chosen your sources, selected a Live Smooth Streaming output format, and chosen your publishing point, you are ready to begin your broadcast. Click the Start button to begin encoding.

There are a few errors that you may encounter while broadcasting.  During broadcasting, if Expression Encoder detects that the number of dropped frames exceeds a minimum threshold, it will show an warning informing you that the playback experience may not be optimal. Broadcasting will not be stopped for this warning but the output may not be optimal.

Figure 5:  Dropped frames warning

Errors  can occur as well if the source device or file cannot send information fast enough to send a full chunk to the server.  This usually happens on the lower bit rate streams where the bit rate cannot be written fast enough for the key frame interval which will cause the streams to be out of sync.   If this happens, encoding will fail and you will need to increase the bit rate of the lower streams to make sure there is enough data in these chunks.

Figure 6:  Bit rate too low error

A third type of error can occur when the source content changes frame rates.  This can happen with some live devices like web cams that reduce the frame rate in low light conditions or it can happen in variable frame rate video files.  When these types of inputs are used as sources, the chunk lengths of the video stream can vary greatly and cause sync issues with the audio stream during playback for long encodes.  To prevent these types of errors make sure the live devices or files can maintain a constant frame rate during broadcasting.

SETTING UP YOUR SILVERLIGHT CLIENT

The third piece in the Live Smooth Streaming ecosystem is the client used to playback a Live Smooth Streaming broadcast. Any number of different clients can be used to view a Live Smooth Streaming broadcast. The Silverlight Media Framework contains a default Silverlight client that you can use to test your streams. If you followed the instructions on the Live Smooth Streaming for IIS 7 tutorial to create and test a publish point, then you already have installed a sample player under %SystemDrive%\inetpub\wwwroot\LiveSmooth (unless you installed the Silverlight Media Framework to a different directory). You can use the default player in this development kit to test out your streams. It includes a graph which displays information such as the current bit rate that is playing, the current bit rate that is downloading, and the current frame rate that is playing. If you wish to use this template to view the streams you are publishing from Expression Encoder 4, then you need to update the mediaSource attribute in the default.html page to point to the publishing point that Encoder is pushing to. Note that the mediaSource attribute is the fully qualified URL that you entered into the Expression Encoder 4 Location combo box, plus the the added tag “/Manifest” at the end. So for example, if in Expression Encoder 4 you entered http://server_name/LiveSmooth/LiveSmoothStream.isml as your publishing point, then the mediaSource attribute in the default.html would be http://server_name/LiveSmooth/LiveSmoothStream.isml/Manifest.

For a more complete client experience, you can also use one of the live media player templates that are included in Expression Encoder 4 SP1. In SP1, we modified a set of our templates to allow the controls to appear based on the publishing output you select to give user full control over the playback. In the Live Broadcasting project, you should see the new Templates panel next to the Output panel on the right hand side. If you do not see this panel, select it from the Windows menu. Since live broadcasting is different than generating templates in on-demand project, you will need to pre-generate your template to the location where your users can see it. To set up a live template for broadcasting:

1.  Import your live/files sources and select your output format from the Encode panel.

2.  Select the output format you want to publish to from the Output panel.  If you select a publishing point, enter the URL to push the broadcast.

3.  On the Templates panel, select the template you want to use and then click the Generate button.  This will pull up the Generate Template dialog.

4.  On this dialog, you can generate multiple templates for different types of broadcasts for each output type selected.

5.  Select the output type you want to generate a template for and select the location you want to publish to and click Generate.

The template files will be generated in this location and the <MediaSource> will be hooked up to the output type URL.  Once you are finished generating the templates you need, click Finish and you are now ready to broadcast directly to the Silverlight template.

TUNING YOUR BROADCAST

Live Smooth Streaming encoding is a very computationally-expensive process, and requires a fast machine to get good results. For example, you cannot encode 12 720p streams on a dual core laptop.  More memory, faster CPU’s, and more cores will all enable you to encode more streams at higher qualities.  To encode at 720p quality levels, you are also going to need to make sure that these cores are fast. Having 8 cores running at 2.0GHz is good configuration, but having 4 cores at 3.0+GHz is better, especially if you want to encode at 720p. Often, it will take experimenting ­with encoding settings to find out what a particular machine can handle. Pay attention to the number of dropped frames while encoding, as Expression Encoder will drop frames when it detects that it cannot achieve the current encoding settings. Dropping frames is usually a sign that your machine is not fast enough to encode using your current settings. Reducing the number of streams, lowering the bit rate of individual streams, and reducing the output resolution of each stream will all result in better performance.

The Expression Encoder team has created a tool that can allow you to test your hardware's ability to broadcast live IIS Smooth Streaming.  It allows you to select a file type and an Expression Encoder live preset to test against.  It also has additional options to specify the failure tolerance or you can let the tool can generate the optimal streams during the test.  Then Expression Encoder will iteratively run the file though a different IIS Smooth Streaming broadcast scenarios and will complete when the output meets the failure tolerance requirements.  Once finished, you can save the generated preset so that it can be used with the Encoder.  It is recommended that you use different file types to determine a more accurate prediction on your hardware’s capabilities since decoding different codecs can affect broadcasting. 

You can download this tool from the Expression Gallery website.

PROTECTING YOUR OUTPUT

In Expression Encoder 4, you can also encrypt your output IIS Smooth Streaming broadcast using Microsoft PlayReady technology.  You will need to have a PlayReady license server or provider set up to obtain a KeySeed which will allow the server to decode and send a license to the client during playback.  On the Output panel, check the ‘Digital Rights Management’ checkbox to enable the feature.

Figure 8:  Digital Rights Management

In this section, you will need to enter:

• License URL -  the URL to the server that will provide the license.  This URL should point to http://[server name]/playready/rightsmanger.asmx.  Click the Validate button to verify the URL is correct.
• Key ID:  This field is a unique GUID to identify the broadcast.  Clicking Generate will create a new GUID automatically.
• Key Seed:  This string should be the base64 encoded string provided by the license server to be used to encrypt the streams.  The Key Seed is used to encrypt the content and has to match the server’s Key Seed so the server can provide the correct license to decrypt the content.

Silverlight handles the licenses request and downloads during playback on the client.  You can use any of the Expression Encoder templates to playback encrypted content.

ADDITIONAL INFORMATION

Here are some sites where you can find more information about Expression Encoder 4 and Live Smooth Streaming:

The Expression Encoder Team Blog is a great place to stay up to date with the latest information about encoder releases and tips and tricks on using Encoder.  If you have any questions about using Expression Encoder 4, the Expression Encoder forum is a great place to start. Here, you can connect directly with the Encoder team and other users to ask questions, request features, and get help with any problems you might be having.

The IIS Media website has a wealth of information about Live Smooth Streaming. Here, you will find instructions on setting up your server, sample content, comparisons against competing formats, and case studies about other companies using Live Smooth Streaming technology.  

If you are interested in developing your own media player to consume Live Smooth Streaming content, then the Silverlight Media Framework is the first place you should look.

There is also a white paper from the IIS team on how to set up live IIS Smooth Streaming in Expression Encoder to stream to Apple Devices.

This week we announced a plan to license Expression Encoder Pro for redistribution.  Due to popular demand, partners will now be able to directly license Encoder Pro from Microsoft and distribute it as an add-on or integrated solution. Prior to this announcement, Expression Encoder Pro has been available through retail and volume licensing, but it has not been available for redistribution.  Direct licensing is expected to be available later this year.  For additional information on Expression Encoder or details on the direct volume licensing program, please contact the Encoder Licensing Team, 

We have updated our build on the download center to resolve a couple of issues related to the screen capture application.  Some machines with Expresion Encoder 4 Pro were incorrectly limiting the screen capture time limit and we had an issue with XESC files being played back in the Windows Media Player on 64-bit Vista and Windows Server 2008.  

In order to upgrade, simply grab the newest version of the free Encoder (available here) and install over the current one. The version number will go from 4.0.1639.0 to 4.0.1651.0. Note that there is no need to uninstall the older version, setup will upgrade it for you. Please also take note that those international versions already have the fixes mentioned above, so there is no need to upgrade them.

We have just release the Expression Encoder Screen Capture codec as a separate download package.  This will to allow users to playback screen capture output files (XESC) on machines without having to install Expression Encoder on the machine.  Installing the package will install and register the codec so the files can be played back in the Windows Media Player.  You can find the download package here.

Earlier this month we announced the availability of Express Studio 4. At that time we made the free version of Expression Encoder 4 available on our website.

Today we are pleased to announce that Expression Encoder 4 Pro is now available for purchase from the Microsoft Store. If you purchased a retail version of either Expression Encoder 3 with IIS Smooth Streaming or the full-featured Expression Studio SKU, you can upgrade to Expression Encoder 4 Pro for FREE!. Find out how here.

Expression Encoder 4 Pro contains the full feature set of the product and gives you the following additional functionality not available in the free version:

• H.264/AAC Encoding
• Additional decoders for input types (MPEG2, MPEG/TS Splitter, Dolby Digital (AC3), MP4 and H.264/AAC)
• Live IIS Smooth Streaming Support
• Unlimited Screen Capture
• Digital Rights Management (PlayReady) Integration

More information about the difference between Expression Encoder 4 and Expression Encoder 4 Pro can be found here.

Additional Information/Links

Main Expression Encoder 4 Product Page
Expression Encoder 4 Pro Demonstration
Expression Encoder Forum
How to Use Live Smooth Streaming Article

Fact 2: Encoding video, especially when using HD sources and/or encoding to SmoothStreaming, requires loads of memory. In fact, some encoding operations requires so much memory that our encoding pipeline has to reduce the number of concurrent encoding streams because it's starving memory-wise.

Fact 3: Memory is much cheaper, with some <$1000 PCs having 12GB of RAM. Concurrently, the 64-bit Windows OS market share has never been higher and users are successfully running their 32-bit applications via Wow64.

Until Expression Encoder is available in a 64-bit flavor, that 2GB limitation will be a serious bottleneck affecting performance on custom applications using the Encoder SDK.

Is there a way to mitigate that 2GB memory limit on a 32bit application? In a word: yes.

For a Native application, you can use the "/LARGEADDRESSAWARE" linker flag, which will let a 32-bit application running on a 64-bit OS allocate up to 2GB of extra user-mode memory, effectively doubling the usable memory and therefore removing some of that bottleneck. And while that flag never got ported to the managed linker, you can also achieve the same result by doing this post-link operation:

EditBin.exe /LARGEADDRESSAWARE C:\...\bin\Debug\<YourApplication>.exe

Unfortunately, it’s a little more complicated to take advantage of this extra memory on a 32-bit based OS, as it does involve adding a boot flag ("/3GB") to reduce the kernel memory allocation in favor of user-mode memory, which will provide up to an extra 1GB to applications. But it comes at a price and should be avoided on XP since it may affect system stability because of some GPU memory mapping. A lot more information on this subect is available on those 2 pages:

http://msdn.microsoft.com/en-us/library/ms791558.aspx

http://support.microsoft.com/kb/291988

It's worth noting that the “/LARGEADDRESSAWARE” flag will simply be ignored on a 32-bit OS if the “/3GB” boot flag isn’t set, so there are no drawbacks in modifying your app with EditBin as explained above.

Please also note that we already did this operation on the Encoder GUI (EncoderUI.exe) for our users’ convenience, but that step will be required if you want your custom application using the Encoder SDK to be able to use more than 2GB of memory.

Here is a good blog entry with more details on this topic: http://bilbroblog.com/wow64/hidden-secrets-of-w0w64-ndash-large-address-space/

A QFE has been released for Expression Encoder v3. The download can be found here:

http://www.microsoft.com/downloads/details.aspx?displaylang=en&FamilyID=3fa77346-6838-4963-9561-ad6ea7b62fbb

 The QFE contains a significant number of fixes and enhancements including:

·         The Audio panel now appears when the user selects two or more media files in the Media Content panel.

·         You can now read or write to MP4 files of 4 GB or greater size.

·         Audio recordings created using certain USB microphones no longer contain distortion.

·         You can successfully encode to MP4 CBR when the bitrate is set to 4.

·         Substantial drops in playback bit rates no longer occur during CPU spikes while playing smooth streaming content.

·         Smooth Streaming content no longer stutters after 10 or more minutes of full-screen playback in some scenarios.

·         Playback of Smooth Streaming content encoded with the VC1 Smooth Streaming 720P VBR preset has been improved.

·         You no longer receive an error when repeatedly editing a Silverlight template in Expression Blend 3 using the Edit the template in Expression Blend command.

·         You no longer receive an error when playing back Smooth Streaming content from a server that is different from the one where the Silverlight template resides.

·         Performance has been improved in the Black Glass template.

·         In Live Encoding mode, you can now publish to a publishing point that requires authentication.

·         Expression Encoder now includes support for AVC1 files.

·         You can now compile the PowerShell code sample from the Expression Encoder SDK when working in Windows 7.

·         Running Expression Encoder as a SYSTEM level service no longer causes an exception.

·         The Silverlight player no longer crashes when you load a file containing unsupported East Asian characters. You will receive an error message, however, if the player does not support the characters.

·         You will now be able to select Adaptive GOP and Closed GOP in the VC-1 Settings options when you have chosen VC-1 Main as your video codec.

·         Expression Encoder will now use its native MP4 codec to decode MP4 files, instead of using QuickTime.

·         Fixes have been made to better support importing and editing Smooth Streaming files.

·         The Expression Encoder Timeline will now display the proper length of imported AVCHD files.

·         Playlists and chapters now correctly display when performing offline playback.

·         Imported WTV files now play back correctly.

·         Seeking has been improved during playback of Smooth Streaming files.

·         If you choose the VC-1 Simple profile for encoding, the bitrate in the Video category will now correctly switch to one is valid for the Simple profile.

Summary:

· Microsoft Silverlight Streaming by Windows Live Beta is being discontinued and will eventually be taken down.

· A new Windows® Azure™-based hosting and delivery service will be launched by the end of 2009, though this is not a direct replacement for Silverlight Streaming and will have costs associated with its use.

· You can access, retrieve and if desired remove your content that is hosted on Silverlight Streaming by following the instructions below.

· Key dates and information about the availability of the new Windows Azure functionality, and when the existing Silverlight Streaming beta service will be taken down, will be communicated in the coming weeks.

Background:

Since May 2007, we have been offering a free beta service to host Silverlight applications and videos. This service is being discontinued to make way for a new Windows Azure-based service. Once this new service is launched, the current Silverlight Streaming beta service will eventually be taken down.

Don’t panic! All your current content is safe and you will receive sufficient notice for you to make an informed decision on where to host your Silverlight content and applications. However, in the interim, we would like you to be aware of the following:

· Effective immediately, no new sign-ups are permitted for the Silverlight Streaming service. In addition, the Silverlight Streaming publishing plug-in for Expression Encoder will no longer be available for download.

· The new Windows Azure functionality will not be a direct replacement for the Silverlight Streaming service and will be a paid subscription service.

Steps you can take today:

· You can access all of your content that is hosted on Silverlight Streaming and also view the file system to retrieve your content locally. For complete instructions on how to do this, please refer to the Step by step instructions for retrieving Silverlight Streaming content section below.

· Stay tuned for more information. Key dates about the new Windows Azure-based functionality, and when the existing service will be taken down will be communicated in the coming weeks.

· You can learn more about the Windows Azure Platform here.

If you have additional questions, please contact us at slsfbk@microsoft.com.

Step by step instructions for retrieving Silverlight Streaming content:

You can directly access the filesystem of the Silverlight Streaming hosting service. This will be the easiest way to retrieve content directly to your computer. In order to do this, you need to know the Account ID and Account Key used to access the filesystem. To get this information, do the following:

· Open a web browser and connect to http://silverlight.live.com

· Login with your Live ID.

· Click on Manage Account.

· You will see your Account ID and Key under the Account Information section at the bottom of the page.

You can connect to the SLS filesystem in a number of ways:

1. Map a network drive

Map to the following network drive, replacing <Account ID> with the numeric Account ID you found by following the steps above.

https://silverlight.services.live.com/<AccountID>/

2. Open as a web folder

· Windows XP, Windows Vista, Windows Server 200 and Windows Server 2003 users must install the latest Software Update for Web Folders.

· Internet Explorer users: open the web folder by connecting to the following URL, replacing <Account ID> with the numeric Account ID you found by following the steps above:

https://silverlight.services.live.com/<AccountID>/

· Other web browsers: copy and paste the following path into the “Run…” box on the Start menu:

\\silverlight.services.live.com@SSL\DavWWWRoot\<AccountID>\

· When prompted for credentials, supply your Silverlight Streaming account information that you found by following the steps in the previous section.

Note for Vista users:

For improved performance, disable thumbnails as follows:

1. Click on the Start button, then choose All Programs followed by Accessories.

2. Run Windows Explorer.

3. In Windows Explorer, click on the Organize button on the menu bar.

4. In the pull-down menu, click on Folder and Search Options.

5. In the Folder Options dialog window, click on the View tab.

6. Under the Files and Folders tree heading, ensure that Always show icons, never thumbnails is checked.

3. Mac OS X users

In the Finder, choose Go à Connect To Server. In the Server Address field, paste the following address, replacing <Account ID> with the numeric Account ID you found by following the steps above.

https://silverlight.services.live.com/<AccountID>/

In their recent QuickTime update, Apple added a new DLL dependency to their SDK which unfortunately breaks our QuickTime integration. Until this situation is resolved in a more permanent fashion, here is a workaround to get QuickTime to work again in all versions of Expression Encoder:

• From the Start menu, right-click on "Computer" and select "Properties".
• Select "Advanced system settings"
• In the "System Properties" dialog, click on "Environment Variables..." in the Advanced tab.
• In the "System variables" list, find the "Path" variable and double click on it.
• Add this string at the end of "Variable value" to add the missing directory to your path:

;C:\Program Files (x86)\QuickTime\QTSystem\         (For a 64-bit environment)

or

;C:\Program Files\QuickTime\QTSystem\                 (For a 32-bit environment)

• Click OK to close all the dialogs and restart Windows. QuickTime will be working again in Expression Encoder.

We have just made available a new option that allows users to purchase the full version of Expression Encoder as a standalone product.  To recap what this means, let’s start with the free version of Expression Encoder.  This is available from here:

http://www.microsoft.com/downloads/details.aspx?FamilyID=b6c8015b-e5de-46c0-98cd-1be12eef89a8&displaylang=en

and contains the full feature set of the product with the following limitations:

• Screen capture up to 10 minutes
• No Smooth Streaming output
• The following CODECs are not present: AC3 Decoder, MPEG2/TS Decoder, AVC-HD Decoder, H.264 Encoder, AAC-LC Encoder/Encoder

(Note.. the installer is currently miss-labeled as Encoder_Trial_en.exe but is, in fact fully featured bar the above list and also not time-bombed as in previous versions.)

If you try and access these features you’ll get the following message:

As of today, the purchase link now points to a new Microsoft Store page where you can get an Expression Encoder 3 license for $49:

Expression Encoder 3 is still available as part of Expression Studio 3, Expression Web 3 as well as through the Expression Professional Subscription and MSDN programs.

To allow the encoded files to playback seamlessly in most environments, including when streamed on a wireless network, the XBox 360 preset uses a bandwidth of 4Mbps while preserving the video frame size of the original file. This is fine for standard definition content however will produce mediocre output for 720p sources and will be even worse for 1080p content. Because of that, I would advise tweaking the video bitrate to your specific needs. So, if network speed isn't an issue, I would personally recommend changing the video bitrate to something like 6-7Mbps for 720p content or 8-10Mbps for 1080p content, which is still within the Xbox360 supported limits for VC-1 decoding.

Finally, it's important to note that the Advanced VC-1 video profile is not supported by the Xbox360 (nor the PS3 through Media Connect), so it's probably best to leave it as "VC-1 Main", like the XBox 360 Preset does.

When Encoder 3 is installed we now install the API help file, intellisense files and the sample projects, so you’re ready to go with the SDK as soon as Encoder is installed. You will find a SDK link on the start menu that will take you to the folder from where you can find the samples (in C# and Visual Basic) and the help CHM file.

Live Encoding Support

I know a bunch of you have been waiting for this for a while now and you’ll be glad to see that V3 has full support for the Live mode of the application through the object model.

e.g.

// Create a new LiveJob to begin broadcasting this file.
using (LiveJob job = new LiveJob())
{
    // Create a new file source from the file name we were passed in
   LiveFileSource fileSource = job.AddFileSource(@"C:\myvideo.wmv");

    // Set this source to Loop when finished
   fileSource.PlaybackMode = FileSourcePlaybackMode.Loop;

    // Make this source the active one
   job.ActivateSource(fileSource);
…

From the LiveJob object you can get a list of video and audio capture devices installed on the system and add them as sources as well. See the Microsoft.Expression.Encoder.Live namespace for further details and the new Live SDK sample that is installed.

Redesigned profile support with full support for H.264 customization.

In V3 we now have separate profile classes for each profile type. This way if you’re dealing with the Main H.264 video profile for instance you’ll just see the properties that pertain to that profile on the object. (Note that you need the full version of Encoder for H.264 support). Along with this you now specify the profiles within the OutputFormat property on the MediaItem.

Bitrate is specified by using one of the new bitrate classes, ConstantBitrate, VariableConstrainedBitrate, VariableUnconstrainedBitrate and VariableQualityBitrate. Again this is so that only the properties that make sense are exposed on the class. For example when using VariableConstrainedBitrate there is a PeakBitrate property but that property isn’t exposed on any of the other bitrate classes as it only applies to the variable constrained bitrate type.

e.g.

MainVC1VideoProfile videoProfile = new MainVC1VideoProfile();
videoProfile.Bitrate = new ConstantBitrate(350);
videoProfile.Complexity = VideoComplexity.Fastest;
videoProfile.Size = new System.Drawing.Size(640, 480);

MediaItem item = new MediaItem(@"C:\myvideo.wmv");
item.OutputFormat = new WindowsMediaOutputFormat()
{
    VideoProfile = videoProfile
};

Smooth Streaming

The Encoder SDK supports encoding to the smooth streaming output format (when using the full version of Expression Encoder). The video profile class now has a streams property which is used to specify the details of each stream that you wish to encode. Smooth Streaming can be used when encoding to constant bitrate for VC-1 and H.264 and to variable constrained bitrate when using VC-1. Additionally when using VC-1 and variable constrained bitrate you can specify that the sizes of the streams should be automatically determined by the encoder based upon the content and the bitrate. In this case you specify the maximum width and height to use for each stream.

e.g.

AdvancedVC1VideoProfile videoProfile = new AdvancedVC1VideoProfile();

// When you create a VideoProfile you'll get one stream by default.
// In this example remove that one as we’re going to explicity 
// add the three streams below.
videoProfile.Streams.RemoveAt(0);
videoProfile.Streams.Add(
    new VariableConstrainedBitrate(1450, 1600),
    new System.Drawing.Size(800, 600));
videoProfile.Streams.Add(
    new VariableConstrainedBitrate(1050, 1600),
    new System.Drawing.Size(640, 480));
videoProfile.Streams.Add(
    new VariableConstrainedBitrate(600, 1600),
    new System.Drawing.Size(400, 300));

// Use smooth streaming with automatically sized streams.
videoProfile.SmoothStreaming = true;
videoProfile.Streams.AutoSize = true;

Encoding Multiple Sources Together

In V2 you could essentially combine up to three sources by using the leader, main video and trailer. In V3 you can move past this limit and combine multiple sources into one by using the new Source class.

e.g.

MediaItem item = new MediaItem("mymovie1.wmv");
item.Sources.Add(new Source("mymovie2.wmv"));
item.Sources.Add(new Source("mymovie3.avi"));
item.Sources.Add(new Source("mymovie4.wmv"));

File Information

There are new classes to help analyze an existing media file to determine information about its video streams and audio streams (in case the file has more than one).

e.g.

AudioVideoFile source = new AudioVideoFile("myvideo.wmv");
Console.WriteLine(source.VideoStreams[0].VideoSize);
Console.WriteLine(source.VideoStreams[0].AspectRatio);
Console.WriteLine(source.VideoStreams[0].Duration);
Console.WriteLine(source.AudioStreams[0].Channels);
Console.WriteLine(source.AudioStreams[0].SampleSize);

If the source file does have more than one audio stream you can now specify which stream is used when encoding by setting the AudioStreamIndex property on the Source object.

e.g.

item.Sources[0].AudioStreamIndex = 2;

Presets

For all the standard presets that are installed with the application we have specified pre-defined static instances for each one to make it easier to use in case you don’t want to set all the properties of the profiles individually.

e.g.

mediaItem.ApplyPreset(Presets.VC1HighSpeedBroadbandVBR);

If you look at the Presets Members in the SDK help file you’ll see all the options along with descriptions.

So there you have a quick introduction to some of the new changes and improvements to the Expression Encoder object model in version 3. You can check out the What’s New section in the SDK help file for further details and feel free to give us your feedback and ask questions either by adding a comment here or over in the Expression Encoder Forum

We are very pleased to announce that both Expression Studio 3 trial and Expression Encoder 3  are now available to download:

Encoder: here

Studio: here

In this release we have taken on board your feedback and simplified the product line. There are now two versions: Expression Encoder and Expression Encoder with IIS Smooth Streaming. We have retired the Express version and, instead, have made all the features in Expression Encoder available for free with no more 30 day limit.

So what do you get in Expression Encoder with IIS Smooth Streaming?

· CODECs (AC3 Decode, MPEG2/TS Decode, AVC-HD Decode, H.264 Encode, AAC-LC Encode/Decode)

· Unlimited Screen Capture (Expression Encoder can capture up to 10 minutes)

· IIS Smooth Streaming

Other than that, all functionality in the Expression Encoder is freely available with no time limit (Full VC-1 encode, Live encoding, Overlays, Screen Capture up to 10 minutes).

If you wish to obtain the fully licensed version of Expression Encoder with IIS Smooth Streaming, it is available in both of the Expression Studio 3 and Expression Web 3 SKUs as well as in the Expression Professional Subscription which can be ordered from the Microsoft Store or your favorite online retailer.

As previously mentioned, there will be a freely redistributable version of Expression Encoder 3 available in the future, that can be shipped as part of 3^rd party applications.

We on the Expression Encoder team are proud to announce that Expression Encoder 3 is complete and will shortly be available!  Since releasing V2 SP1 back in November of 2008 we have worked hard to improve core encoding capabilities,  continued to enhance new features such as Smooth Streaming, H.264 and editing as well as broaden the applicability of the product by adding new features such as Screen Capture.

We have also focused heavily on our Silverlight output upgrading the MediaPlayer control to Silverlight 3 and the template skins that are based on it.  We’ll drill into the details of what is new over the course of this post and look forward to hearing your feedback.

So how do you get Encoder 3? We’ll keep you in suspense a little while longer and direct you to the http://team.silverlight.net blog for the official announcement of Expression Studio 3.

Encoder Enhancements

We’ll start our tour of what’s new with the enhancements to the core encoding engine of the product.

Improved H.264.  First up, we’ve improved the H.264 encoder that was part of V2 SP1 by adding support for custom resolutions (V2 SP1 was limited to two portable device-specific resolutions), single pass CBR, VBR Constrained, VBR unconstrained and Quality-based VBR rate control methods, Main Profile support, B-Frame support and advanced settings which include the ability to choose between the two major Entropy modes of Context-adaptive variable-length coding (CAVLC) and Context-adaptive binary arithmetic coding (CAVAC).

VBR Smooth Streaming.  Microsoft is investing heavily in Smooth Streaming across its product lines, which in conjunction with Silverlight, enables everyone to watch great quality video on the web at up to full HD 1080p. One of the results, that we are very excited to be shipping, is a brand new VC-1 encoder that has been optimized specifically for the multi-bitrate, multi-resolution nature of IIS Smooth Streaming.  Some of the features of the new technology include:

• A new rate control that uses an analysis pass to distribute rate across GOPs
• Support for variable GOP duration (the original CBR form of encoding requires fixed GOPs)
• Dynamic optimization of coded resolution based on scene complexity and bitrate.  Additionally, directional energy measurements are used to change resolution both horizontally and vertically
• An optimized polyphase filter for resizing resulting in better speed and quality that has been optimized for the SSE4 instruction set.   
• Significant performance optimizations such as sharing motion information across bitrates and a faster analysis pass
• Ability to cancel an encode if QPs start to spike, and restart with lower resolution

Source CODECs.  In Encoder 3, we’ve broadened our support for source CODECs.  We have improved the performance of our H.264 decoder as well as adding support for AVCHD format.  We’ve also added some enhancements to our QuickTime stack to enable compatibility with ProRes which will help with post production workflows.  We are also pleased to be shipping Dolby Digital decoding in the box.  Finally, we’ve added the ability to import Smooth Streaming files so that you can review an encode outside Silverlight.  In this case, you are able to open the .ism file and the largest stream will be rendered.  This final feature is also available outside of Expression Encoder because we register the decoder on the system.  This means that once the full version of Expression Encoder has been installed, other applications on the system that can make use of DirectShow source filters will be able to open .ismv files.

Audio Enhancements.  We heard your feedback and added support for multi-channel audio output:

We have also added the ability to select between multiple audio streams, where sources have them, as well as support for the Voice codec and VBR audio encoding. Last but not least, we dynamically determine what audio encoding capabilities are available from the host OS.

Performance.  Performance is something we are never done with.  In Encoder 3 we’ve made great progress, particularly on multi-core, multi-processor scenarios.  First off, our core encoders have been rev’d to include support for newer Intel SSE instructions as well as support for up to 8 physical cores.  Secondly, we’ve improved CPU utilization on multi-core systems by parallelizing preprocessing tasks.  To illustrate the improvements, here are some before and after comparison charts based on analysis of Encoder 2 and Encoder 3 performance:

  
Core 2 Duo Laptop; Vertical axis time in seconds – smaller is better; V2 in blue, V3 in red; results aggregated from 6 different source files

 
Dual proc 8 Core Xeon; Vertical axis time in seconds – smaller is better; V2 in blue, V3 in red; results aggregated from 6 different source files

We are working on more performance comparisons including i7 Nehalem systems which we’ll publish in due course.

Live Encoding.  We spent a bunch of time in this release moving the Live portion of our encoder over to using the new encoding pipeline first introduced in Encoder 2.  This results in some immediate benefits such as better performance and de-interlacing support as well as some potential future benefits.  We’ve also added support for WMA Pro audio in Live, a new object model for Live (more details on this in the API section) and support for a broader range of capture devices.

Expression Screen Capture

In recent years, screen-casting has become an extremely popular way of sharing information you are looking at on your system with other users; encompassing eLearning, product support, user-generated content and many other domains. Included with Expression Encoder 3 is a new application called Expression Encoder Screen Capture available in the start menu or from within the main Expression Encoder UI.

You simply decide if you want to record webcam, and mic as well as the screen and system sound, pick a screen region with the easy to use Window picker (or by free-form selection):

.. and start capturing.  One of the benefits of our new implementation is that we capture to a light weight intermediate CODEC, developed by Microsoft Research, rather than attempting to capture and encode directly to VC-1 or H.264.  This leaves more of the CPU available for the application you are capturing.  After capture, it is easy to import the sources into Expression Encoder for final encoding and Silverlight templating if required.

UI Enhancements

Improved profile palette.  As Expression Encoder has evolved between versions, the old Encoding settings palette had became increasingly unwieldy.. so for Encoder 3 we rebuilt it from the ground up. 

The most common encoding settings are now easier to find and set.  The advanced encoding settings are hidden by default so you only see the options you need.  And the ability to address different output CODECs is now explicitly designed in.

Inside audio and video sections, there are many enhancements that make day to day tasks easier such as changing the aspect ratio or configuring multiple streams for Smooth Streaming.

Preset Palette.  Encoder 3 now has a more extensive library of encoding presets including an expanded set that target devices such as Xbox, iPod, Zune and the forthcoming Zune HD, online video services such as Facebook and YouTube and an expanded and improved set that target IIS Smooth Streaming.  As well as the expanded library, we have also added a new searchable preset palette where both system and user presets are stored as well as the ability to easily apply presets to all items via right-click.

Dockable palettes.  In common with the rest of the Expression 3 suite, Encoder 3 now uses the common palette docking framework which enables you to lay out your interface however you choose by simply dragging palettes and either popping them out into floating windows or grouping them into docked locations.  The Hide Palettes function has also been enhanced allowing the hidden palettes to pop open on hover.

Summary Palette. Summary information has been moved into a separate palette which can be displayed via the Window menu.

Cog Menus.  In order to streamline palette-specific functions, we’ve introduced cog menus:

Cog menus are present on all palettes.  We’ve also got rid of the small white dot menus that were somewhat undiscoverable :-).

Editing Enhancements.   In Encoder 2, we had the concept of adding a leader and trailer to a main file.  In Encoder 3, we have removed leader and trailer and replaced it with an extended Clips palette which enables multiple items to be stitched together.  This enables scenarios such as multiple leaders / trailers, Ad insertion and simple editing.  

It is also possible to set the stretch mode per clip and choose to apply crop which was not previously possible with leader and trailer.  You can append still frames (and set their duration) as well as XAML animations. Note that all of the enhanced editing capabilities are also available through the object model.

Silverlight 3 MediaPlayers

The Silverlight mediaplayers that have shipped in the box with Expression Encoder since V1 make it really easy to publish videos that target Silverlight.  In Encoder 3, we have updated all of the included templates to be based upon Silverlight 3 which means they are offered in the easy to use and deploy XAP format and all share the same common MediaPlayer control at their core.  We have also exposed the core MediaPlayer control in the Blend 3 toolbox which makes it easy for designers to embed and customize a full featured MediaPlayer in their projects that supports captions, chapters, smooth streaming etc.

SL3 specific features.  We have added a number of Silverlight 3 specific features to all of the new players including

• Cached composition:  Think of cached composition as hardware acceleration for the final frame presentation step.  Enabling cached composition (on by default) is as simple as a checkbox and results in much lower CPU utilization which helps general playback performance particularly on lower end systems such as NetBooks.  It also means that setting Stretch Mode Uniform incurs no performance penalty.
  
• Offline support : Offline also known as Out Of Browser (OOB) is a new feature of Silverlight 3 that enables users to install Silverlight apps locally on their Mac or Windows machine.  In Encoder 3, if you choose to enable offline support when you publish a Silverlight video, the user will have the option of downloading and installing not only the media player code but also the video and audio files.
  
  This feature also works in conjunction with IIS Smooth Streaming so you get the benefit of adaptive streaming when online and a downloaded, high quality version to watch when away from a network.
  
  To show this in action, I created the following screencast.  Note that you can actually try the offline feature on the screencast it by clicking on the cog menu and selecting Offline :-)
  
  
  
  
• We’ve made some changes to the Adaptive Streaming support that is baked into our MediaPlayer control.  Firstly, we’ve improved the bandwidth heuristics in a number of ways such as using the new CPU load metrics available from the Silverlight 3 runtime and constraining the playback resolution based on the size of the video window (to prevent bandwidth wasted by downloading e.g. 720p when the window size is only e.g. 320x240).
  
  We have also moved the heuristics into a separate XAP so that the adaptive streaming module can be loaded only when necessary thus helping to reduce the core download size.  Finally, we have changed the way we distribute the Media Steam Source (MSS) that implements the client portion of IIS Smooth Streaming.  We have moved to a model where we ship object code for this component, not source.  Source code is shipped for all other MediaPlayer functionality and is usable, redistributable and modifiable under the MS-PL.

New Player Skins.  We have worked with ace design firm Archetype to create some great new player skins featuring smoothly animating controls and new Silverlight 3 features such as perspective transform.

Other Silverlight Improvements:

• XAP Size reduction: we’ve taken steps to reduce the size of the MediaPlayer control.. the core player is now 109k down from 162k in Encoder 2 SP1.
  
• Popout: We’ve added a Popout feature to the templates that allows a video to be popped out of a webpage as a webpage dialog.. you can then drag to another monitor and go fullscreen to keep watching while you work.   
  
• Scriptability: The core mediaplayer control is now fully scriptable from Javascript.
  
• Improved Install Experience: Expression Encoder now automatically generates a Content Gate install experience for users that don’t have Silverlight Installed:
  
  
     
• Improved accessibility on Silverlight Media Player: We’ve added tooltips to buttons on the MediaPlayer and done some additional testing with screen readers.
  
• Edit in Expression Web:  We offer the ability to edit the output of a job in Expression Web.
    
• Preview in Browser:  You can now open a Smooth Streaming job previously encoded and saved and start up the Smooth Streaming preview server and launch the default browser without having to re-encode the job.
  
  
  
• MediaPlayer control available in Blend Asset Palette:  When Expression Encoder is installed on the same system as Blend, the Expression MediaPlayer control becomes available in the Blend asset palette.  This enables designers and developers to easily use and customize our base player and take advantage of all the built-in functionality such as chapter points, closed captions, play lists smooth streaming, Offline support etc.
  
• Transcoding in Expression Web:  Expression Web ships with Expression Encoder and utilizes the API to offer simplified transcoding functionality integrated right in.

API Enhancements

Expression Encoder has had a .NET API since Version 2.  In Version 3, we are making some changes:

Object Model for Live Encoding. Live encoding now has an object model so you can create applications that capture and stream from live capture sources as shown here:

   1: using (LiveJob job = new LiveJob())

   2:            {

   3:                // Create a new file source from the file name we were passed in

   4:                LiveFileSource fileSource = job.AddFileSource(fileToEncode);

   5:  

   6:                fileSource.PlaybackMode = FileSourcePlaybackMode.Loop;

   7:  

   8:                //Alternatively, enumerate a capture device (not shown here)

   9:  

  10:                job.ActivateSource(fileSource);

  11:  

  12:                WindowsMediaBroadcastOutputFormat outputFormat = new WindowsMediaBroadcastOutputFormat();

  13:  

  14:                // Let's broadcast on the local machine on port 8080

  15:                outputFormat.BroadcastPort = 8080;

  16:  

  17:                // Set the output format on the job

  18:                job.OutputFormat = outputFormat;

  19:  

  20:                // Start encoding

  21:                Console.Out.Write("Press 'x' to stop encoding...");

  22:                job.StartEncoding();

  23:  

  24:                while (Console.ReadKey(true).Key != ConsoleKey.X)

  25:                {

  26:                    // We are waiting for the 'x' key

  27:                }

  28:  

  29:                // Stop our encoding

  30:                Console.Out.WriteLine("Encoding stopped.");

  31:                job.StopEncoding();

  32:            }

Refactored Encoding Profile OM.  In addition to refactoring the Encoding profile palette, we have also refactored the underlying object model for configuring profiles making it simpler to use.  Instead of a monolithic Profile class, settings have been broken into a class hierarchy allowing the app to more easily and logically support multiple output CODECs such as H.264 and VC-1.

SDK in the Box.  The SDK is now shipped along with the application.  It has also been updated to include more samples, including a Powershell cmdlet sample.

Redistributable version of Expression Encoder.  We will also be offering a no fee redistributable version of Expression Encoder for developers who wish to ship applications using the SDK.  Stay tuned for more information on this that we’ll share here on the blog in due course. 

General

Windows 7 Superbar Progress: View at a glance how far through the encode has progressed:

First pass:

Second pass:

Removal of PS 2.0 requirement: We had feedback from users of older graphics cards that did not support Pixel Shader 2.0 that they were unhappy that they could not preview video.  So in Encoder 3, we still take advantage of PS 2.0 if present, but do not require it for preview.

Support for LargeAddressAware flag:  Some of the new features, in particular the new VBR Smooth Streaming Encoder, can require a lot of memory in certain scenarios.  In Encoder 3, we set the /LARGEADDRESSAWARE bit in the application’s PE header which enables 3GB of addressable memory on x86 systems (requires BCDEDIT /Set IncreaseUserVa 3072) and 4GB on x64 systems.  For more information on this and how to configure an x86 system to take advantage of this feature, see http://msdn.microsoft.com/en-us/library/aa906217.aspx and http://blogs.technet.com/brad_rutkowski/archive/2006/10/03/Hey-where-did-_2F00_3GB-go-in-Longhorn-and-Vista_3F00_.aspx

Removal of command-line encoder:  One of the tough trade-offs we had to make for this release was what to do with the command-line.  In Encoder 2 SP1 the command-line encoder was functionally behind what was possible in the UI and the Object Model.  We also observed a lot of community support for specialized command-line alternatives such as WMCmd.vbs by Alex Zambelli.  So, for Encoder 3 we took the difficult decision of removing the command-line encoder from the product.  Instead we directed our energies into making the .NET Object Model full fidelity in order to give application developers the maximum API surface area possible, including support for Live encoding.  To mitigate the decision, we are shipping working sample code in the box in the form of a Powershell cmdlet wrapper around the object model which provides basic Powershell-based command-line encoding supporting batch operations via the Powershell object pipeline:

Additionally, we are keen to work with the community to get an equivalent to WMCmd.vbs built and published on Codeplex to satisfy users who really want to automate encoder using cmd.exe rather than from Powershell or C#.  We hope you agree that this was the right trade-off to make.  If you have any opinions, positive or negative, please do send us your feedback.

Summary

We look forward to making the release available for download in the coming weeks and talking more about its capabilities.  Stay tuned and, as ever, your feedback is welcome.
The Expression Encoder Team.

Back in October, we announced a new Microsoft video delivery technology called IIS Smooth Streaming.  In a nutshell, this approach combines both client (based on Silverlight) and server (based on IIS) components that, together, deliver scalable media experiences up to 720p+ over the internet using the plain old HTTP protocol.

We have had a preview of the technology running over at http://www.smoothhd.com courtesy of our CDN partner, Akamai.  We also added support for encoding to, and previewing, the new format in back in Expression Encoder 2 SP1.

Today we are announcing beta availability of the IIS Smooth Streaming component completing the picture and enable users to test the technology end to end..  You can download the extension over on the IIS Site.

There will also be a number of talks on all aspects of Smooth Streaming at MIX (including one I’m doing on encoding).. keep an eye on https://content.visitmix.com/2009/sessions/?categories=Media for more info. 

For more information and background on the technology check out the following links :

John Bocharov

I’ve seen the future and the future is… Smooth!

Smooth Streaming Questions- We’ve Got Answers!

Alex Zambelli

Smooth Streaming Architecture

The Birth of Smooth Streaming

A Brief History of Multi-Bitrate Streaming

SmoothMBRCalc

James Clarke (me :-) )

Inside Expression Encoder’s MediaPlayer control part 3: Smooth Streaming playback heuristics (coming soon)

Inside Expression Encoder’s MediaPlayer control part 2: Debugging IIS Smooth Streaming client code

Inside Expression Encoder’s MediaPlayer control part 1