16.7ms in the life

Peter Rosser — Wed, 21 Mar 2007 07:35:00 GMT

So, the last several months have been fairly busy, what with Vista shipping (woohoo!), vacations, and plannng the next rev. I worked long hours alongside my fellow eHomers, we shipped, and I took off a couple of months to find myself in Mexico and decompress. I've been back for a while now, and been bugged enough about my blog neglect--so here I am posting again.

The timbre of my postings will probably shift quite a bit now that I am in a new position on a new team. As of February, I have left the Windows TV team and joined the Media Center Core, a move that shifts me from the south side of building 50 to the north side, across the Great Divide of the central hallway. That is, when my office move happens to place me somewhat closer to my new team. It's always a little melancholy moving teams, but I am excited at the new opportunities I have to make a positive impact on the product.

The Media Center Core Team owns the Media Center application, non-TV experiences: Movies Guide, music, photos & video library, extensibility, online spotlight, and a bunch of other stuff on the side. It's always difficult to talk about what we do without talking about what we're doing, if you get my meaning.

One thing I can talk about is that we are growing, and with growing comes new jobs! We have several positions open for experienced developers posted on the Microsoft job site (here's one: http://members.microsoft.com/careers/search/details.aspx?JobID=9D2139A0-0D36-414E-8C66-B3C8CFD35086 ). If you think you have what it takes to help us ship the coolest feature in Windows to millions of users, send me your resume! Whether your passion is TV, music, photos, or you're just a scary-smart engineer, we are hiring across the board, for Test, Development, and Program Management.

Oh, if anyone has anything in particular they'd like me to talk about, shoot me an email (peter.rosser@microsoft.com) or add a comment.

Thoughts on software complexity

Peter Rosser — Sat, 03 Jun 2006 07:06:00 GMT

If you've ever tried to create a software application of any decent size, you have probably realized that what works with small tools more often than not does not scale well to larger sizes. There are whole books written about how to scale up and out, new languages and technologies invented to help manage it, and millions of dollars in educational budgets spent every year to achieve that elusive goal.

And yet, the landscape is still rife with examples of poorly designed software.

A significant part of the problem is not that engineers are not smart enough--it's that we outsmart ourselves. The conventional wisdom is that humans can store 7 ± 2 primitives (e.g. numbers) on the top stack of short-term memory. The number skews higher for strongly associative items, and vice versa. We can store fewer abstract items, often only 2 or 3 at a time. The issue is not one of comprehension, it's one of focused retension. When the number of data items in a class exceeds our capacity to retain them at once, the brain tends to move the perceived "less important" fields/properties into peripheral storage. This, I think, leads to not fully comprehending the interactions in the class. We engineers like to think of ourselves as intelligent, and that we can understand very complex ideas. This may be true, but it misses the point; the point of software engineering is to create reliable, performant, maintainable software. It is not to create complex applications.

Well, there's the problem, isn't it? In order to create the really cool applications, you must have complexity. It's simply unavoidable, especially given the advanced competition for consumer dollars. You can't very well create a next generation RPG that has realistic physics, with a "real" economy, kickass graphics and brutally smart AI without complexity.

Or can you?

Actually, I believe the answer is that you can, at least the bad kind of complexity that causes the pain. The key is differentiating between behavioral complexity and system complexity. Behavioral complexity is the aforementioned features like smart AI, or realistic physics. Those sorts of things are monstrously difficult to get right, but that's not what holds back the very large application development projects. The major cause for schedule slips, regressions, out-and-out bugs, and general angst is system complexity.

Before I go on, I'd like to define what I mean (yes, I do get to hijack words like this) by system complexity:

System complexity a property of a system that is directly proportional to the difficulty one has in comprehending the system at the level and detail necessary to make changes to the system without introducing instability or functional regressions.

In other words, the higher the system complexity, the harder it is to upgrade, maintain, and develop the system. Whether the system is a single class, an object library, an inheritance heirarchy, or an entire application, someone "schooled in the art" of software engineering should be able to understand it, given a reasonable time to examine the system. This ability should scale through the system. When one examines a particular object, it should be apparent without examining the private implementation details how to use it. From top to bottom, from bottom to top; the entire system should be accessible to the non-super-geniuses out there. What goes for good UI design also goes for good C++, C# and COM design: your target audience should be able to use it without extensive systems training.

System Complexity is what makes the Mythical Man Month true. It's what keeps that guy who wrote the accounting system from scratch in 1986 employed. It's what causes games, productivity applications, and even (some might say especially) large operating system projects to slip their schedules.

So what can you do about it? The first step to recovery is admitting you have a problem. Pedantic, but true. Getting back to the point I tried to make earlier about engineers--we like to think we are smart. Egos loom large in the software industry, and often with good reason. One of the smartest developers I have come across worked on a critical piece of the code for several years, and guarded his territory jealously. Make no mistake, this guy was brilliant--and he knew it. To make a long story short, when he left, the code he left behind was an engineering marvel [he says wryly] that no one else understood. Oh, we know what it does, but there's a crucial difference in understanding what something does, and grokking fully how it does it. Tracing execution paths through this code is difficult, and 1200-line methods named "Run" are common.

The author understood it all, or said he did, but my opinion is that he was fooling himself, too. Once you get a piece of the system working, after all, you don't need to touch that piece again unless the design changes, a bug is found, or something like that. Over time, though, people started to notice that even small fixes in this particular area would touch a dozen or more files, and churn lots of code. That's a symptom of high system complexity; he would go in to make the fix, then discover that 100 variables influenced the behavior, or the implementation in one area was strongly dependent on the implementation in another, or most often, both. But since the author had traveled this area so much, he was able to turn around the fixes in an acceptable timeframe, so he "got away" with it.

It's no surprise that when the author decided to move on to other things, as employees often do, that we had a tough decision to make. Do we maintain this code that's working? Do we refactor? In the end it came down to pragmatism and schedule: the problem was recognized, and we chose to refactor the system in measured stages, minimizing the risk to the product and getting back to a good, maintainable state.

That's all I have time for today... my build is done, so back to work I go. Windows Vista waits for no man!

Where's Peter?

Peter Rosser — Fri, 02 Jun 2006 02:01:00 GMT

... working, working, working.

That, in short, is why there have been no updates in so long. Between my work schedule, family and renovation project I have not had much time to myself. I have neglected my blog in favor of meatspace; sorry!

When things settle down a bit, I'll blog the drama a bit (what I can). ^_^

Media Center Technical Discussion 1: Tuners and TuningSpaces

Peter Rosser — Tue, 04 Apr 2006 06:08:00 GMT

The Media Center TV experience driven by a flexible, multi-standard engine that works with any compatible tuner (including a good number of analog tuner cards and all BDA digital tuner cards) to create portable recordings viewable on any Windows XP machine. To support “trick play” on Live TV (Pause, FFWD, etc.), the video engine is separated into two main segments: capture and playback. The capture engine, controlled through the Media Center Receiver Service (ehrecvr.exe), is responsible for creating the actual recordings, including controlling tuners, creating a media stream, and optionally encrypting the stream before writing it to disk. The playback engine, controlled through the Media Center Shell (ehshell.exe), processes media streams created by the capture engine and renders them for the user. Everything else, as far as TV goes, is built on top of or leverages these two components.

The capture engine uses the Microsoft Unified Tuning Model, with TuningSpaces and TuneRequests, as a layer on top of the physical tuner, allowing Media Center to treat tuners agnostically. To actually tune to a channel, Media Center first creates (or looks up) a tune request that corresponds to that channel, then submits it to the capture engine, which uses it to tell the tuner what to do.

A TuningSpace corresponds to code written specifically to handle a type of video broadcast, such as ATSC, NTSC, or DVB-T. Some properties are present in all TuningSpaces, like its ID (e.g. {8A674B4D-1F63-11D3-B64C-00C04F79498E} for NTSC Analog Cable), and some are specific to an implementation. The implementation-specific properties--like NetworkType or DefaultLocator for digital standards--let the capture engine configure the underlying BDA driver, and provide a way to tweak default values. A TuningSpace can be considered a "template" for how Media Center will try to talk to the tuner card, depending on how you have it configured--mapping roughly to the old TV setting of "Antenna" or "Cable".

All by itself, a TuningSpace is not enough to actually tune to a broadcast stream. For that, Media Center creates a TuneRequest (by using the ITuningSpace object). A TuneRequest can be thought of as "everything you need to know about tuning to a channel"; what it contains varies from TuningSpace to TuningSpace. For an NTSC tune request, you could expect it to contain a channel number and a reference to the TuningSpace; an ATSC tune request would contain a physical channel (the "major" channel) and a minor channel; a DVB-T tune request would contain the broadcast frequency, ONID, TSID, and SID. All tune requests contain a reference back to the proper TuningSpace.

Media Center stores information about which TuningSpace(s) are configured on the system on a per-tuner basis in the Windows registry under the HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\Media Center\Service\Video\Tuners key. If you open this key, you'll see something like this:

The GUIDs correspond to "tuning categories", which are digital ({71985F48-1CA1-11D3-9CC8-00C04F7971E0}) and analog ({A799A800-A46D-11D0-A18C-00A02401DCD4}), each containing subkeys that correspond to individual tuner devices. Note that there can be more than one tuner device per physical tuner card or USB device--Media Center treats multi-tuner cards as separate devices. I will use my Hauppauge WinTV PVR PCI II device's entry as an example:

As you can see above, the tuner's auto-generated GUID is {99F4E181-4B58-4258-8547-2CE73F3C0386}. This key is created by the ehrecvr service automatically when it detects the tuner: for a PCI card, this happens at service startup; for a USB device, this happens either at service startup or when you plug in the device. If you delete the key and restart the ehrecvr service--and the card is still installed--a key will be created with a new GUID. If a USB device is detached and reattached to a different port it will also get a new key.

Examining the properties, there are a couple of interesting values:

ConfiguredTuningspace is a REG_MULTI_SZ that contains a list of the TuningSpace IDs for which this card has been configured in Media Center.
SupportedTuningspace contains a list of the TuningSpace IDs with which the tuner's driver reports compatibility. These lists can differ if the tuner supports more than one type of tuning, like some of the newer "hybrid" tuners--a card that supports both NTSC and PAL is an example of such a card.
DevInstance, DevName, and DevPath are obtained from the tuner driver, and give Media Center "pointers" to the physical device. If you open up Device Manager and look at the advanced properties for the matching device, you will find that the values map 1:1 to entries here.
PrefEncoder either points to another hardware ID (in the case of a tuner with hardware MPEG2 encoding) or a software encoder (e.g. ATI AIW cards). This is populated after we successfully build a capture graph for the device, and points to the MPEG2 encoder that is included in the graph at that time.

It's important to note that ehrecvr will create a key for every tuner device detected, whether or not it is configured in Media Center or not. Whether it is available for tuning in Media Center is just a matter of whether the registry keys says it is. For example, you can easily get more than 2 tuners working for scheduling, recording and Live TV viewing by editing the registry entries for the tuners to enable them. The reason that you cannot do this through the UI is simply because there is not a good graphical way to resolve recording/viewing conflicts amongst more than 2 tuners for a given TuningSpace. Instead of being able to choose which of the 4 tuners to "steal", you can only choose from 2 of them. However, with 4 tuners you're much less likely to actually need conflict resolution, so IMHO it's acceptable.

To actually get the n-tuner scenario working properly, every tuner must be in the same TuningSpace, and change channels the same way. So you cannot mix STB with non-STB, Satellite with Cable, etc. This is due to limitations in the Guide and PVR engine, not in the capture or playback engines. STB tuners are somewhat of a pain, so I recommend using Method 2 (below) if you are going to go with n-tuners on STBs. Also, you must have an STB for each and every tuner... not a fun scenario if you have to pay for each one.

This week's Media Center tweak is getting N tuners working in Media Center. There are various versions floating around out there on the internet, most of which include extra, unnecessary steps, likely because the authors figured out this stuff without the benefit of internal knowledge of the codebase, so they included everything they did to get it all working. The version of this tweak posted on The Green Button works just like Method 2 below, but requires a reboot between each configuration run.

If you have NTSC + ATSC, you can use this method for the ATSC tuners, the NTSC tuners, or both! The only limitations are your hard drive's bandwidth, your system bus bandwidth, CPU time, and the number of tuners you can attach to the machine. For NTSC that means a practical limit of about 10 tuners, if they are all recording non-DRM content. For High Definition ATSC and DVB it's a little more intensive, but since there's no encoding going on (digital broadcasts are already encoded!), it actually balances out a little, with a top limit of around 7-8 tuners. I have personally run 3 ATSC + 6 NTSC on my home system (through dual-tuner USB devices like the Adaptec AVC-3610), and it hummed right along (1 GB DDR Dual-Channel DDR 3200 RAM, Pentium 4 2.8 GHz (HT), 2x 7200 RPM 500GB Hitachi SATA drives in RAID-0 with the system/boot drive on a 40 GB PATA drive). That's a bit more than I need, so I backed off to only 3 NTSC (2 USB + 1 PCI) and 2 ATSC (both PCI).

Step-by-step, here's how to get n-tuner going. Both methods assume you've already installed the tuners and have the drivers installed.

N Tuners in MCE, Method 1 (only registry editing):

Stop the Media Center Receiver Service (net stop ehrecvr)
Open Registry Editor (regedit.exe) and open this key:
HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\Media Center\Service\Video\Tuners\{GUID}
where {GUID} is either {71985F48-1CA1-11D3-9CC8-00C04F7971E0} for Digital (ATSC or DVB-T) or {A799A800-A46D-11D0-A18C-00A02401DCD4} for Analog (NTSC or PAL).
Under the {GUID} key, there will be one key per tuner device.
Select the first tuner's key.
Select the UserSettings key.
Set these DWORD values as follows (case sensitive, all values in hexadecimal), creating them if they do not exist:
EnabledForMCE = 0xffffffff
UseSTB = 0
Create a REG_SZ value named UserDefName and set its value to "Tuner X", where X is the tuner number (1 for the first tuner, 2 for the 2nd, etc.)
If this is the FIRST tuner:
RecordOrder = 0
RecordPrefered = 0xffffffff
WatchOrder = number of tuners - 1, e.g. for 4 tuners set this to 3
WatchPrefered = 0

If this is the LAST tuner:
RecordOrder = number of tuners - 1, e.g. for 4 tuners set this to 3
RecordPrefered = 0
WatchOrder = 0
WatchPrefered = 0xffffffff

If this is a tuner somewhere in the middle:
RecordOrder = whatever you put for the last tuner + 1
RecordPrefered = 0
WatchOrder = whatever you put for the last tuner - 1
WatchPrefered = 0

The idea here is that you increment the RecordOrder by 1 for each tuner, and decrement the WatchOrder by 1. RecordOrder starts at 0 on the first tuner and goes up to #Tuners - 1; WatchOrder starts at #Tuners - 1 and ends at 0 on the last tuner. RecordPrefered is true (0xffffffff) on the first tuner, false (0x00000000) everywhere else; WatchPrefered is true on the last tuner, false everywhere else.
Repeat steps 7 - 10 for each tuner, adjusting the values of RecordOrder, RecordPrefered, WatchOrder and WatchPrefered as appropriate.
Close Registry Editor.
Start Media Center. Voila!

Yes, "Prefered" is misspelled. We caught that after we shipped, and it's a little late to change now...

N Tuners in MCE Method 2 (a little registry stuff, no manual editing though):

Start the Media Center shell (Green Button)
Go to Settings-->TV-->Set Up TV Signal
Choose the 2 tuners from the list of tuners that you will configure this time (first time through, just the first 2)
Close the Media Center shell.
Stop the Media Center Receiver service (net stop ehrecvr or stop the service from Control Panel-->Administrative Tools-->Services)
Open Registry Editor (regedit.exe) and open this key:
HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\Media Center\Service\Video\Tuners\{GUID}
where {GUID} is either {71985F48-1CA1-11D3-9CC8-00C04F7971E0} for Digital (ATSC or DVB-T) or {A799A800-A46D-11D0-A18C-00A02401DCD4} for Analog (NTSC or PAL).
Under the {GUID} key, there will be one key per tuner device.
Find the 2 keys that were just configured by Media Center. You can tell which ones they are by looking for the "EnabledForMCE" value under the UserSettings key. If it is set to 0xffffffff, it was configured--otherwise it will be 0x00000000 (0).
Export the 2 keys to .reg files on your hard disk. (right-click on the {GUID} key and choose Export [selected branch only]). Make a note of where you save those files... you're going to need them! Name each one differently (e.g. Tuner1.reg, Tuner2.reg).
Repeat steps 1 - 8 for each set of 2 tuners (if you have 3, then just then 3rd, etc.), selecting different tuners each time.
Repeat steps 4 and 5.
Double-click on each of the .reg files you created in step 9 to import them into the registry.
Close Registry Editor.
Start Media Center. Voila!

Next week we'll talk about the PVR engine and recording management.

This posting is provided AS-IS with no warranties. The views expressed above do not represent those of Microsoft Corporation.

Media Center Technical Discussion Series

Peter Rosser — Mon, 03 Apr 2006 21:52:00 GMT

One of the more popular subjects for users who email me is how to get Media Center to do something that cannot be done (or is hard to do) from the UI. Since I have not been able to talk about my current work much (Windows Vista and unannounced other stuff) I am kicking off a roughly weekly technical discussion on tweaking Media Center.

Planned discussions:

Tuners and Tuning Spaces
PVR and Recording
Unattended/Scripted Configuration
Video Playback
Writing Your Own Media Center Application

Want to know how to tweak something in Media Center that's not listed above? Let me know and I'll see if I can work it in.

VoodooPC's 8TB Media Center PC

Peter Rosser — Fri, 03 Mar 2006 01:51:00 GMT

From Engadget:

When we heard that VoodooPC was planning a media PC with 16 drive bays and a maximum capacity of 8 terabytes... [Read full posting on Engadget]

Sounds pretty cool--maybe I'll get one here to "test" with ^_^.

Tripod sings about romantic love

Peter Rosser — Sat, 25 Feb 2006 03:01:00 GMT

I saw this the other day... definitely worth a viewing.

WARNING: If you have a girlfriend/wife/S.O. and you game a lot, you might want to put on headphones or suffer facial contortions and rolling eyes.

http://video.google.com/videoplay?docid=1329362959167995041&q=tripod

HAHAHA

Using exported DLL functions

Peter Rosser — Thu, 23 Feb 2006 01:11:00 GMT

Now and then it's necessary to use "private" or "internal-only" functions in DLLs that you did not write, don't have the source for, and/or cannot get public interfaces for. I recently had to do this again for a utility application, so I thought it would be nice to document the process a little more clearly for others.

A starting assumption is that the reader is familiar with GetProcAddress() in its "main" form, which is when you know the full signature of the function you need to call. For example, if you have a copy of foo.dll that exports Bar(LPWSTR, DWORD*):

// library function is: Bar(LPWSTR lpParam, DWORD *pdwParam)
typedef void (__stdcall *BAR_PROC)(LPWSTR, DWORD*);

HANDLE hFoo = LoadLibrary("foo.dll");
BAR_PROC proc = (BAR_PROC)GetProcAddress(hFoo, "Bar");

// call the library function
DWORD dwParam = 0x4;
(proc)(L"string", &dwParam);

But what if you need to access a function that is not included in the named exports of the library? 90% of the time the functions will be exported with a name, but sometimes not, and for various reasons. The most common reason is for "security through obscurity", because using the function is supposed to be something that you, the user, "should never have to do". Unfortunately, there are scenarios when you need to do just that, for reasons the original publisher never considered, such as if you need to tweak the way something behaves in your mission-critical server process, and the company that made the original custom component has gone out of business.

Enter the "ordinal" function access method. If you look at the MSDN docs, they even allude to using a function's ordinal with GetProcAddress, but it's a bit of a mystery how to a) get the ordinal, and b) how to use it even if you get it. Once you know the ordinal and prototype of a function, it's simple to use it--getting the function prototypes is the fun part.

Say you got your hands on a .DEF file that lists the function ordinals, or have used link.exe and through sufficiently wizardly use of a hex editor have examined the [NONAME] functions and determined what their prototypes are. Let's pretend that the Bar function above was actually a [NONAME] function, and you got a listing like this from link.exe /dump /exports foo.dll:

Microsoft (R) COFF/PE Dumper Version 7.10.4035 
Copyright (C) Microsoft Corporation. All rights reserved. 


Dump of file c:\myprojects\foo.dll 

File Type: DLL

  Section contains the following exports for foo.dll

    00000000 characteristics 
    41107692 time date stamp Tue Aug 03 22:39:30 2005 
        0.00 version 
           1 ordinal base 
           6 number of functions 
           4 number of names 

    ordinal hint RVA      name 
          3    0 00006948 Connect 
          4    1 00007798 Close 
          5    2 00012000 GetFoo 
          6    3 00014D12 SetFoo 
          1      0000F3A0 [NONAME] 
          2      0000D20A [NONAME]

  Summary

        6000 .data
        2000 .reloc
        9000 .rsrc
       18000 .text

The depends.exe tool provides similar data in a GUI fashion. You can examine the binary at offsets F3A0 and D20A to determine what the function signature is (well, what the base pointer sig is), and match them up to the expected signature of Bar(LPWSTR, DWORD*) manually to get the proper ordinal. Or more easily, if you know that one of the methods is the droid you are looking for, you can just code up a quick test app and poke at the method as if it were the one you wanted. If it works without throwing an exception... ;-)

Getting back to the main point, to load the exported-by-ordinal function, there's a handy macro pre-defined in the Visual Studio (v6 or later, IIRC) build environment that can package up that ordinal into the "expected" LPSTR for GetProcAddress. In my example, assuming Bar was actually at ordinal 1, you would do this:

// library function is: Bar(LPWSTR lpParam, DWORD *pdwParam)
typedef void (__stdcall *BAR_PROC)(LPWSTR, DWORD*);

HANDLE hFoo = LoadLibrary("foo.dll");
BAR_PROC proc = (BAR_PROC)GetProcAddress(hFoo, MAKEINTRESOURCEA(1));

// call the library function
DWORD dwParam = 0x4;
(proc)(L"string", &dwParam);

The only difference is that you use the MAKEINTRESOURCEA macro to convert that ordinal into something that GetProcAddress can understand, rather than the string name of the function.

If you really want to get tricky, you don't actually need GetProcAddress at all to do this magic. If you can guarantee that the DLL won't change underneath you, or you don't mind breaking if it does, or you have smart scanning code that can "find" the offsets again, you can plug the RVA directly into the FARPROC variable. It's all just numbers under the hood, after all. Using the example output of link.exe from above, you can do this:

// library function is: Bar(LPWSTR lpParam, DWORD *pdwParam)
typedef void (__stdcall *BAR_PROC)(LPWSTR, DWORD*);
#define PROC_OFFSET 0x0000F3A0

HANDLE hFoo = LoadLibrary("foo.dll");
BAR_PROC proc = (BAR_PROC)(hFoo + PROC_OFFSET);

// call the library function
DWORD dwParam = 0x4;
(proc)(L"string", &dwParam);

Why does this work? LoadLibrary(dll) loads the specified library into the process' address space, and returns a HANDLE value. This HANDLE is the starting address of the library, and since the RVA values for a DLL are offsets from the starting address, you can perform simple addition to the base to get a function's entry point. Pretty cool, huh? All GetProcAddress does is read the DLL's export table into a struct and performs the math for you, returning the result. I'm a big fan of not reinventing the wheel, so I use GetProcAddress when there's an exports entry for what I need.

What do you do if the function you need is not exported at all? Nine times out of ten it's much easier to just get it exported, but if that's not possible, well, that's a topic for another day.

Captioning Support Sucks

Peter Rosser — Sun, 19 Feb 2006 12:42:00 GMT

Looking back at the bugs we've fixed over the last several months, we certainly have had a good share of captioning issues, especially with regards to the size of the codebase compared to the "video" code. Surprisingly, we don't hear from users a ton when captions don't work properly, but maybe that's because a minority of the users actually use them in the first place. I'm not usually so negative, but I'm just annoyed at captioning today. The following is a poorly thought-out rant on the subject. ......(read more)

DVB Guide issues - "cannot schedule recoding because guide data cannot be accessed"

Peter Rosser — Thu, 16 Feb 2006 10:27:00 GMT

We have been getting reports about Guide problems from DVB users across Europe and the world, hopefully in time for the next Rollup release (external date not yet announced, but should be late Q1 to early Q2). The problem typically manifests itself when scheduling many shows to record, but can occur anytime. Without a code fix, you cannot work around the issue cleanly/easily, but closing the shell (Media Center UI) and restarting the "Media Center Scheduler Service" (ehsched) will reset the state of the machine.

Edit: This is the "cannot schedule recording because guide data can't be accessed" issue.

We root-caused the problem to an unusual memory management problem in how we deal with the Guide data. For performance reasons, we temporarily increase certain buffer sizes during a Guide download, and decrease them again later. There is a timing issue with garbage collection events in some situations, though, that can cause the decrease to not take effect "in time", thus keeping the buffer sizes large. This does not actually reduce the amount of real memory used, but it does exhaust the virtual address space eventually. You can observe the behavior using SysInternals' Process Explorer by adding the "Virtual Size" column (under the Process Performance tab) and observing its value for ehrec.exe.

With better knowledge of the cause, though, comes a better workaround than "gun ehshell.exe and restart ehsched when it happens". Until the fix comes out, the best thing to do to prevent this problem from occurring is probably to set up a Scheduled Task to do those things for you at a predictable time. To wit:

Download recycle_mce.cmd from here. (script to recycle ehshell.exe and TV services). Unpack the script somewhere on your drive.
Change your Guide download time to a time that you are not likely to have a need to either watch TV on your MCE or record a show. 4am is pretty safe for me, but to each his own. I wrote a tool that lets you edit this time easily, you can download it from my site.
Create a Scheduled Task to run the script daily about 30 minutes after the time you configured the Guide to download. If you have a modem, you may need to give it more time. If you have a pretty fast computer and broadband, 30 minutes is very generous. Guide to creating scheduled tasks.

It's not a perfect solution, since a) it presumes you have a time during the day you will not make recordings or watch TV, and b) it cannot prevent the problem 100%, especially if you are a "heavy" user. If you run into the problem, though, running recycle_mce.cmd will at least reset your state.

The real fix is coming with the next rollup, but since there are users out there feeling pain now, I thought I'd try to help a little.

The programs, scripts, and instructions in this posting are provided AS-IS, with no warranties, and are not sanctioned nor released by Microsoft.

A Brief History of ATSC in Media Center

Peter Rosser — Fri, 03 Feb 2006 11:09:00 GMT

In the beginning there was NTSC in Freestyle (Media Center 2002), and it was good. Media Center supported the #1 market and proved that computers could do a great job recording television, pausing Live TV, and organizing recordings. With support for one tuner under our belt and an enthusiastic user base, we moved forward with Harmony (Media Center 2004) to a more mature, flexible and polished product that supported dual NTSC or PAL tuners, opening up the European, Asian and Australian marketplaces. A few intrepid souls figured out how to hack together ATSC support into Harmony using little-known interfaces and scarce hardware like the VBox 110.

Between the release of Harmony and Symphony (Media Center 2005), ATSC demand increased by leaps and bounds, though still smaller than the NTSC base by far. To this day, only a very small percentage of Media Center users actually use ATSC, although it's a matter of some discussion whether that number is small because the number of HDTVs is small, or because Media Center's support is... less than optimal. In any case, by the release of Symphony the user community was clamoring for ATSC support without hacks, and was disappointed when Symphony shipped without it. An update quickly followed, though, and Symphony Rollup 1 with HDTV Support was released as a free update through Windows Update and the Microsoft Download Center. The changes in Emerald (Rollup 2) and Diamond (Windows Vista) did/will not not substantively change the architecture of ATSC, so what we have today will more or less be the way it works for another couple of years.

••• design decisions

When we (speaking in generalities here, since I was not a member of the team at the time) first investigated how to include ATSC into the product, there were several major hurdles standing in the way. The first was a consequence of our decision to use the VMR (Video Mixing Renderer) instead of the deprecated Overlay Mixer for rendering video. Overlay rendering differs from mixing in the way that video is prepared for display. In overlay mode, the video is literally painted over the top of whatever else is being rendered. In mixing mode, "surfaces" with video or other graphics are blended together, either in software or hardware-accelerated, prior to rendering on the monitor. The advantage of mixing mode is greatly increased flexibility in what you can display at the cost of performance. It takes a lot more horsepower to process all those pixels, especially with alpha (transparency) and 3D effects. 99% of the video hardware on the market today uses a form of overlay mode, which drastically reduces the hardware requirements (e.g. no 3D hardware required), but hamstrings them when viewed side-by-side with the sort of effects you can get with mixing. The semi-transparent menus on top of video that you see in Media Center that give it such a cool look are just not possible with overlay mode. All that alpha blending and those 3D effects come at a hefty cost, though. Media Center has to work many times harder to get the video to the screen, even when no special effects are on-screen at the time. That, in a nutshell, is why Media Center requires not just an OK video card to shine. And a faster CPU. As a result of the increased power needed to render video, and with the hardware available to consumers in 2000-2002, it wasn't really feasible to offer HD with the first couple of releases.

Even if the video could keep up (which it could on higher-end systems, even in 2002), there was another difficulty: Guide data. The problem is that when you get multiple lineups (the term we use for the programming information associated with the set of channels available on a given medium in a geographical area), merging them into a single lineup and making it easy for the user is not an easy solution. The decision was made to punt it until demand was increased, and to stick with a simpler homogenous lineup, rather than the more advanced heterogeneous lineup that would be required to support both NTSC and ATSC on a single system. The market of people that have more than one TV provider is, after all, not that large, and we wanted to get it right for the most people we could.

••• support for ATSC gathers steam

A vocal minority of the user community, along with a large majority of the eHome TV product team, were not pleased with the lack of support. We got the funding to put ATSC in the product, as long as we could fit it into the schedule and without major risks to the architecture of Media Center's TV and PVR functionality. With compressed schedules the norm, and the development and testing staff already deeply into a bad work/life balance, many options were considered for getting ATSC out to users. What was finally implemented is a good first stab at doing it, with a few facets that are, um, less than what we'd like them to be. Since we did not have ATSC-specific listings available through our Guide data provider for many, many areas, we looked instead at reusing the data available from existing analog lineups via "mapping". Since ATSC channels are in a completely different format (like 13-1 or 9-5), and to avoid putting a new button on the remote (requiring a redesign), ATSC channels are renumbered in the UI. The scheduling/PVR conflicts were just ignored, leaving it up to the user to say "only record this channel", omitting support like "record this in HD if you can, but get it in SD if not". The end result was a pseudo-heterogeneous lineup support that we have today. You do get the additional ATSC lineup merged into your Guide, but the analog source is still required--you cannot have ATSC without NTSC today.

••• the current state of things

During First Run (the initial Media Center setup wizard), or TV Signal Setup, we detect whether you are in a region that supports ATSC, and then detect if you have a compatible ATSC card installed (or for Emerald, up to 2 ATSC cards). After configuring your SD tuner, ATSC is configured as an adjunct to the "primary" analog tuners. If you choose an analog, sans-STB (set-top box) cable lineup for your analog tuners, then that data is what's available to ATSC to map to its channels. The same goes for satellite or OTA (off-the-air/broadcast) lineups. Only with digital cable lineups do you usually get the full Guide data to assign to your ATSC channels.

To assign listings to the ATSC channels in the lineup, we run a heuristic match against the available channel metadata, and try to get a "best-fit" match for existing channels to the new ATSC channels. For example, if there is a channel "KOMO-DT" in the primary lineup, it can be matched to the known ATSC channel KOMO-DT on 4-1. In some areas, the match can't be made automatically, but you can go to the Guide settings (Settings-->TV-->Guide-->Edit Digital TV Listings) and manually associate the listings of an existing channel to the ATSC channel. Basically what you are doing is saying "this channel that exists on my "primary" lineup has identical programming to that ATSC channel". It works best with digital cable lineups because they carry all of the local OTA ATSC channels on their networks (due to Federal must-carry laws). To get the minor channels (where the "minor channel" > 1, e.g. KCTS-DT2 on 9-2 in Seattle), you have to add them to your lineup manually using Add Missing Channels, and then map them to listings using Edit Digital TV Listings.

The end result of all of this is a unified lineup that can be presented to the user in a single Guide grid. ATSC channels are assigned numbers like 1041 for channel 4-1, and slotted in where they fit best. If your primary lineup already has a channel 1041, we look for another block of numbers to put the ATSC channels. A side-effect of this is that if your primary lineup is changed--like if Comcast decides to add a channel 1000 to their lineup--the ATSC channel numbers may get reassigned to another empty block. Personally, I'd like to implement a better solution, but as I've said before, we Americans are unnaturally attached to our channel numbers. Most would just give you a blank look if you suggested that we don't even really need channel numbers at all. Not when you have a good enough search/filter mechanism, anyhow.

••• the days to come

For the Diamond release, we have announced support for OCUR (OpenCable Unidirectional Receiver) tuners, which will allow native tuning of digital cable signals, with CableCard support (for premium channels like HBO). This includes HDTV signals, which should be good news to most users. More people pay for TV service now than don't, and if you own a Media Center, you are about 20 times more likely to have cable or satellite service than not. Roughly. OCUR support probably means (definitely, for Diamond) that we won't be supporting heterogeneous lineups anytime soon, which is a shame, but hard to make a business case against. Why spend time and treasure on something that most people won't use, and could possibly destabilize the entire product? That's a little hyperbolic, but the onus is on the people who want the feature to justify it.

We announced a deal with DirecTV where we will directly support their newest STB in Media Center, just like--or very similar to--the OCUR does with digital cable. The field is finally opening up and we're getting to a point where we can offer our users very attractive options for "traditional" TV services. The European market, or more precisely, the non-U.S./S.Korea/Japan market, has already standardized (for the most part) on a fairly sensible broadcasting standard--DVB--which we are improving our support for with every release.

Non-traditional video services are also booming in popularity, with Online Spotlight partners offering more movies and other multimedia content for free (in many cases) or on an ala-carte basis, directly competing with PPV services offered by traditional TV providers. Apple has changed course on its "we'll never get into the PVR business" policy with iTunes video (and their toe-dip product Front Row, which is a pretty spiffy little QuickTime automation script), and The Market is scrambling all over itself to get on the bandwagon. Internet TV is coming, just as soon as we (as in "we the market", not necessarily "we at Microsoft") figure out what Internet TV is.

DISH Network & ATSC channels

Peter Rosser — Wed, 01 Feb 2006 12:14:00 GMT

I have been working on a fix for the ATSC channel number assignment issue currently impacting Rollup 2 users. The scenario is that if you have any channels in the 9000s (like DISH Network) and you have ATSC channels, they will get assigned numbers in the 10000s. That's a problem if you want to change directly to the channel by typing in its channel number, due to UI restrictions.

Rather than fix the UI, we chose the safer/less impact path of changing how we figure out the channel numbers for the ATSC lineup. The exact algorithm probably won't be made public, so I can't get too specific, but basically it will work like this:

If the highest non-ATSC channel is less than 9000, use the next block of 1000 numbers after that channel for ATSC. ATSC channels will get numbers in the format: offset + major*10 + minor (unless there is a minor>9, in which case they get assigned sequential numbers). So if the highest non-ATSC is 5001, ATSC channel 13.2 will be 6132.
If there is a non-ATSC channel in the 9000s, search for a free block of numbers within the entire range of channels, preferring larger blocks, and blocks that fall on even multiples of their size (i.e. blockStart % blockSize = 0). Try fairly hard to keep the major + minor data present as much as possible in the ultimate channel number.
If no blocks of sufficient size are found, assign channel numbers to the ATSC lineup in a first-available way, slotting them into empty numbers as necessary.

The three methods above are tried in order, so if 1 succeeds, 2 and 3 are not used, etc.. Number 3 is very unlikely, absent some boneheaded move by a cable or satellite provider to evenly space their channels throughout the first 10000 numbers. The most likely is number 1, except for DISH Network users, who get channels in the 9000s right now.

My fondest wish is that we could just ditch channel numbers entirely and be done with it... but we Americans are too attached to them. For now.

Fix coming for ATSC minor channels guide bug

Peter Rosser — Mon, 30 Jan 2006 21:36:00 GMT

If you use ATSC with Media Center, and have mapped minor channels to listings, you probably noticed that the listings from the .1 channel are duplicated across the other minor channels (if the callsigns are similar enough). This results in a display like this in the guide:

This is more than a little inconvenient, since it also means that Media Center cannot schedule recordings accurately on the affected channels.

Coming in the next rollup QFE for Media Center (late Q1/early Q2), we have a fix for the problem. Unfortunately, the fix required a code change and cannot be worked around with a registry tweak.

The way it should look is like this:

Fix for ATSC after uninstalling Rollup 2

Peter Rosser — Tue, 24 Jan 2006 22:24:00 GMT

In some cases you may find that ATSC (HDTV) no longer works after uninstalling Media Center Rollup 2 from your computer. The issue is caused by a schema change in the Tuners registry key. Today I identified a workaround to get your ATSC back up and running without reinstalling Symphony or Rollup 2.

To restore your ATSC functionality in Symphony:

Stop all Media Center services and processes

Close Media Center
Start-->Run-->services.msc (or Control Panel-->Administrative Tools-->Services)
Stop "Media Center Receiver Service"
Stop "Media Center Scheduler Service"

Open Registry Editor (Start-->Run-->regedit.exe)
Delete the key at:
HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\Media Center\Service\Video\Tuners
Start Media Center

This has not been rigorously tested, and has not been "officially" released to the Microsoft Knowledge Base yet. I am fairly confident of the fix, but this is posted AS-IS, with no warranties.

DVB Radio "No TV Signal" Workaround

Peter Rosser — Mon, 23 Jan 2006 07:50:00 GMT

I am currently investigating a fix for two DVB Radio issues identified a workaround for those affected. If you are getting a "No TV Signal" blue overlay when tuning to a DVB Radio station, particularly a low-bitrate station like BBC World Service, or you are seeing 15+ second channel change times for changing between DVB Radio and DVB Video channels, this workaround may apply to you.

This involves editing your registry, so if you are not comfortable doing that, please wait for the binary fix to be released.

The key is:

HKEY_LOCAL_MACHINE\Software\Microsoft\Windows\CurrentVersion\Media Center\Service\Video\Tuners\DVR

Set NearUnderflowCheckMillis=4000 (0x00000fa0).

If you are not experiencing problems with DVB Radio, do not change this value. If you are in the U.S., for example, it can cause instability with ATSC tuning. Once the binary fix is released, you should return this value to its original value of 2000 (decimal).

This is a non-supported, non-official workaround and is provided AS-IS, with no warranty.