A couple of weeks ago, I had a chance to sit in on a webinar delivered by Flexera Software and chat about Windows 7 Application Compatibility. The recording of the webinar is now online and available here:

http://mktg.flexerasoftware.com/mk/get/BPWin7CompatApp_wbnr?mc=eml-flw

A few weeks ago, I spoke about how our information on vendor data was slowly making it through the channel, beginning with the compatibility center (complete), moving on to the downloadable Application Compatibility List for IT Pros (complete), and finally getting to ACT. I suggested, “The target date for this integration is mid-December 2009.” To call December 22 “mid-December” is already stretching it, and yet it’s still not showing up in ACT. What gives?

Well, let’s talk a little bit about the data and how it has to move through the system. In the end, it all comes from the same database. The compat center, the downloadable list, the data that feeds the upgrade advisor – all from the same source. What is taking so long to turn on the spigot for ACT?

Well, the big challenge is data transformation. First, the App ID has to be translated. The 3 live services all use a new algorithm for determining an application ID. ACT hopes to transition to use this new algorithm in the next version, but it’s one of those “running in place” changes where, after spending a lot of time and money, the end result looks identical to the user, so it’s difficult to justify rushing the transition (particularly since invalidating an existing inventory full of old App IDs is a terrible idea, so the application will need to support both old and new, and be able to differentiate between them). That translation needs to be automated so it can run in perpetuity, and that process has to be build. Then, the web service that feeds ACT is the old version (v1) while the others are using a much newer version (v4), so the old web service interface has to be modified to leverage the new database feeding it. So, there are two feats of engineering that need to be implemented, tested, and proven at scale.

We’re still hoping to have this done the first week of January, but vacations and winter colds are making this more challenging. Rest assured, this data is coming and we are terrible sorry for any inconvenience the delay is causing – we really do want the data to be accessible and convenient and are not neglecting the value this has to customers.

Earlier today, I received this request from a customer:

This (http://www.microsoft.com/downloads/details.aspx?FamilyID=07fb0bd8-f390-458d-a629-6f0258ac7cdf) is a .NET 1.1 app and I have installed .NET 1.1 SP1 on Windows 7 32-bit and all .NET 1.1 SP1 security patches but it doesn't run. Gives me this error: An internal error has occurred: Object reference not set to an instance of an object" and then it quits.

Always being up for a challenge, I decided to have a look and see what wasn’t working. Reproducing the error is pretty straightforward. Just run the app – it’ll gladly crash for you. But the question is, why? And can you fix it?

If you hit this with a debugger, you’ll find the following stack when you hit an AV:

0:000> !dumpstack -ee
Current frame: (MethodDesc 0x2098d28 +0x1c SystemTimeZone.ConvertToTimeZoneInfoStructure)
ChildEBP RetAddr  Caller,Callee
0018f4f4 0206d4ab (MethodDesc 0x2098d08 +0x43 SystemTimeZone.ToLocalTime)
0018f55c 0206cfc0 (MethodDesc 0x2098cd8 +0x38 SystemTimeZone.Convert)
0018f574 0206a484 (MethodDesc 0x2e5680 +0x15c frmMain.ShowCurrentTimes_Click)
0018f5d8 02067686 (MethodDesc 0x2e56d0 +0x446 frmMain.frmMain_Load)
0018f61c 02060365 (MethodDesc 0x2e5620 +0x2ed frmMain.Main)

Now, you could take a stab at this with SOS and start pulling out objects, but I didn’t believe that this would be the easiest way to solve the problem. Instead, I decided to reverse engineer the code and have a look. If you take a peek at the structure you’re trying to create from the SystemTimeZone class, you’ll find where the instance of the time zone came from – the code was rampaging through the registry looking for it. Here’s the method in question:

public TimeZoneList()
{
    m_TZHash = new Hashtable();
    m_TZList = new ArrayList();
    RegistryKey key = Registry.LocalMachine.OpenSubKey(@"SOFTWARE\Microsoft\Windows NT\CurrentVersion\Time Zones\");
    string[] subKeyNames = key.GetSubKeyNames();
    Type t = typeof(TZIStructure);
    int num2 = Marshal.SizeOf(t);
    int num3 = subKeyNames.Length - 1;
    for (int i = 0; i <= num3; i++)
    {
        RegistryKey key2 = key.OpenSubKey(subKeyNames[i]);
        SystemTimeZone zone = new SystemTimeZone();
        SystemTimeZone zone2 = zone;
        zone2.DaylightName = StringType.FromObject(key2.GetValue("Dlt"));
        zone2.DisplayName = StringType.FromObject(key2.GetValue("Display"));
        zone2.Index = IntegerType.FromObject(key2.GetValue("Index"));
        zone2.MapId = StringType.FromObject(key2.GetValue("MapID"));
        zone2.StandardName = StringType.FromObject(key2.GetValue("Std"));
        zone2.Name = key2.Name.Substring(key2.Name.LastIndexOf(@"\") + 1);
        zone2 = null;
        byte[] buffer = (byte[]) key2.GetValue("TZI");
        key2.Close();
        if ((buffer != null) && (buffer.Length >= num2))
        {
            GCHandle handle = GCHandle.Alloc(buffer, GCHandleType.Pinned);
            IntPtr ptr = handle.AddrOfPinnedObject();
            zone.TZI = (TZIStructure) Marshal.PtrToStructure(ptr, t);
            handle.Free();
            if (!m_TZHash.Contains(zone.Index))
            {
                m_TZHash.Add(zone.Index, zone);
                m_TZList.Add(zone);
            }
        }
    }
    key.Close();
}

If you spend enough time with the !do command, you can eventually stumble upon the answer, but the easier approach is, again, to stay out of the debugger and leverage your favorite search engine. Here’s a KB article that points out the problem:

http://support.microsoft.com/kb/935369

Relevant information: “The Index registry value does not exist in Windows Vista and in Windows Server 2008.”

Aha. See the source code above? It wants that value, and it’s not there. Root cause found.

Now, on to solutions.

First of all, I wonder if you need a solution at all. If you click on the clock, and then click on Change date and time settings… you will get to the Date and Time control panel. Click on the Additional Clocks tab, and you can add 2 additional clocks for 2 new time zones. If you need more, you can always use Gadgets to fill the rest of your needs. So, for most people, this should provide a built-in solution to the problem – you don’t need the app any more because the OS provides 2 solutions already.

But, if you really really love that particular app, you could always copy that piece of the registry from Windows XP onto your Windows 7 computer, drop it in a new place inside of the registry, and then use the VirtualRegistry shim to redirect that app’s checks of the timezone portion of the registry to the copy you brought over.

Special thanks to my friend Gov for pointing me to the KB article, and for coming up with the VirtualRegistry workaround idea.

Are your business-critical applications compatible with Windows 7? Join Microsoft's "App Compat Guy" Chris Jackson to learn best practices for how IT teams are resolving Windows 7 application compatibility issues. Hear about his experiences with early Windows 7 adopters, including the most common application compatibility issues and their root causes. Tips in this webinar will help your IT team minimize risk and make a smoother move to Windows 7.

Webinar Details
Thursday, December 17
12pm EST (9am PST, 5pm GMT)

One curious observation I have made while doing app compat work over the past few years: The ownership of the project is assigned to the packaging team more so than any other team. Perhaps that’s because it’s the only centralized group dealing with every application, or perhaps it’s because installation failures on the new OS are the first things you notice – the rationale varies. But regardless, the team responsible for packaging definitely is an extremely interested party, if not the outright owner of the project.

As a result, when I sat down for dinner with several execs from Flexera Software (makers of InstallShield and AdminStudio, among other things), they expressed this as an area of great concern for their customers. So, in between our discussion of which tapas was the tastiest (I assure you, there was some extremely tasty tapas), we were brainstorming how we can better work together to help people who are interested in packaging be more successful when an app compat project comes along.

One idea? Let’s work on a webinar together. Flexera has a pretty successful webinar series, so let’s just partner up on one of those. So, on Thursday, December 17, 2009 at 11:00 am Central, we’ll be doing that. It will be relatively quick and will be presented at an overview level, so if you’re looking for a debugging deep dive this probably isn’t your best bet, but one thing it will emphatically not be is marketing speak. If this perspective is useful to you, come and join us. Feel free to ask the hardest, nastiest questions you can think of!

http://mktg.flexerasoftware.com/mk/get/BPWin7CompatApp_wbnr

In the world of app compat, sometimes you get lucky and have the app itself clearly not working, and you debug it. (Well, I guess if you’re really going to get lucky, you’ll have the app just work.) Things get murkier when an app has a bad interaction with another app, but not stand-alone (in which case you could miss the interaction until you’ve deployed to end users, since labs generally start with a clean slate). This is where the finger pointing begins – the great “blame game” where each support organization points their finger at the other one.

And, of course, the most fun is when you have an app compat problem where it makes it look like the operating system just plain doesn’t work. When you see a message like, “The network path was not found” or “File or network path no longer exists”, do you think app compat problem? No, you think you were sold a busted up old operating system. Sometimes you’re right. (We have bugs too.) Sometimes it’s your network. And, yes, sometimes it’s an app compat problem.

Such is the case for a number of support incidents we’ve seen recently with Windows Server. We debug each one, and what we found was a deadlock in srtsp.sys / srtsp64.sys. I’ve seen deadlocks all over the place – they’re one of those equal-opportunity-style bugs, because multi-threaded programming is insanely hard to do perfectly. It just so happened that these deadlocks would rather unluckily take place on the kernel mode threads servicing SMB negotiation requests.

The vendor, of course, wants happy customers, so they released a patch. If you are running older versions of Symantec Endpoint Protection 11 and Symantec Antivirus 10.2 you should get the solution from Symantec. Symantec confirms that this is a known issue and there are updates to resolve the problem. This is one of those insanely useful patches to get (as opposed to patches that fix problems that you may never see in your lifetime). See: http://service1.symantec.com/SUPPORT/ent-security.nsf/docid/2007102613484948

My friend Robert put together a post on this over in the Windows Server Division Weblog.

I’m kind of disappointed that there isn’t a disturbing amount of debugger information in the post, given who wrote it (though you’ll see there is enough to communicate the message). Robert is the kind of guy where it only takes a few minutes of watching him work to realize just how profoundly dumb you are. I had one app I was debugging once, and I was just plain stuck. (It happens.) So, I asked him for his thoughts. He dumped hex to the screen, pointed to a bit of it, said, “there’s the problem, try this fix.” And he was right. (Of course, I later realized that I was just as bad when I told my friend Gov a joke in hex, and he got it.)

(Wow, I’m doing a lot more posting today than normal. Things kind of piled up while I was on leave.)

There is a ton of activity going on in the world regarding Windows 7 deployments, and one of the hottest topics we’re finding is x64. People are very interested in moving to the 64-bit version of the operating system, but one obvious question is, will my applications be broken?

The next version of the Application Compatibility Toolkit, ACT 6.0, is scheduled to have far more robust x64 support. (It’s scheduled to support it at all, which is already a bonus.) But what about in the interim?

First of all, there is the Setup Analysis Tool. This will detect 16-bit binaries (probably a problem for 64-bit Windows, unless setup is detecting the bitness of the OS and dropping file appropriately) and driver installation (possibly a problem for 64-bit Windows, unless setup is detecting the bitness of the OS and dropping files appropriately). But this requires a fair amount of work to get configured and running, and it also means that you have to have the setup files. Depending on how managed your environment is, that may be a problem.

If you’re the sort that doesn’t mind undocumented secrets, I’ll let you in on one: ACT already collects the bitness of the files it finds in an inventory scan, it just doesn’t surface that in the UI today. So, how can you view this information? Why, hack the database of course. My SQL skills are so rusty these days I’m beyond embarrassed of what I crufted together, but here is a query I hacked up to display all of the files that are neither 32-bit (user mode 32-bit is probably OK) nor 64-bit (definitely OK) in your ACT database:

USE ACT55
GO

SELECT DISTINCT Applications.appName, Static_App_Properties.fileName, fileModuleType

FROM Static_App_Properties
INNER JOIN Application_Instance_Files
ON Static_App_Properties.identity_hash = Application_Instance_Files.filePropertyID
INNER JOIN Applications
ON Application_Instance_Files.appID = Applications.identity_hash

WHERE fileModuleType<>'32BIT' AND fileModuleType<>'64BIT' AND propertyType='File'

ORDER BY appName
GO

The schema is, as always, undocumented and subject to change, but I have found this helpful in a few scenarios to quickly zero in on apps to specifically test on 64-bit when targeting a mixed environment. I suspect you’ll have to do less hacking with ACT 6.0.

This one has been on my to-do list for a while, so it’s far from breaking news. I use these posts as a resource to gauge how we’re doing with our presentation content. Typically these posts don’t generate a lot of feedback, but it is always welcome.

At TechReady 9, I presented the following sessions:

CLI323
The MCS Application Compatibility Strategy Engagement: Predicting the Unpredictable

CLI313
Application Compatibility Remediation: The Dark Magic of Fixing Broken Applications

CLICT300-R1
Application Compatibility - Tools discussion (Part One – AppDNA and ACT)

CLICT300-R2
Application Compatibility - Tools discussion (Part Two – ACT and ChangeBase)

CLI207
What EVERYONE should know about Application Compatibility (with Aaron Margosis)

CLI401
Not for the Faint of Heart: Hard Core App Compat Debugging (with Gov Maharaj)

SOLNCT202
How to Navigate the Microsoft Services Application Compatibility Offerings and Deploy Windows 7 at Your Customers

The rating criteria were:

Q1 - The speaker was knowledgeable about the subject matter.

Q2 - The speaker's presentation skills helped me better understand the material.

Q3 - The content was effective in achieving the stated learning objectives.

Q4 - The demonstration(s) were effective.

Q5 - This session builds skills improving how I sell, market, and/or provide services to our customers and partners.

Q6 - The knowledge/skills I gained are relevant to my role.

Q7 - This readiness solution was worth the investment of my time.

Q8 - I will recommend this readiness solution to others.

Q9 - Overall, I was satisfied with this Session/Chalk Talk.

Q10 - Content Level: Please rank the level of technical information actually delivered.

Q11 - Please provide additional constructive comments, suggestions, feedback.

QAvg - Average for all questions combined

Here are the rankings. I kind of vacillate between displaying rankings as raw numbers or as raw percentages. In this case, I have enough of a distribution that the raw numbers can be misleading, so I’m using percentages:

Analysis

People are interested in both the technology and the business approach. My two top sessions? A technology session, and a strategy session that didn’t include a single demo (which is incredibly odd for me). This was one of my areas targeted for improvement, and it seems as if we’re finally getting that story put together.

Hard core debugging is still a polarizing session. This session still generates one of two pieces of feedback. Either someone thinks it’s the best thing ever, or they think it’s impossibly obtuse and not useful. There is nothing in between. I threw in a demo this time of “debugging without a debugger” which has traditionally been really powerful with the folks who actually do the work, but for a general audience, it fell flat. They wanted to see hex, apparently. :-)

Don’t be depressing. Our “what everyone wants to know” session was still in the top half, but boy, it isn’t at all what I like to see in my scores. We raced through topic after topic, painting a horribly depressing picture in an effort to be “real.” But it’s not this incredibly depressing thing – you actually can do it. We didn’t leave any hope, only despair. It turns out people don’t really like that much. Do-over.

Don’t leave your technical guys behind. I did a series of 2 sessions – one with AppDNA and the other with ChangeBase. AppDNA sent their CEO and their CTO. It’s lit up in green. ChangeBase’s technical director didn’t make it, leaving only their director of sales. They are awash in a sea of red and orange as a result. Technical audiences like hearing it straight form the geeks’ mouths.

We need to keep our remediation sessions current. I did a minor revision of this content for TechReady 9, and have a major overhaul in store for the 2010 conference season. It’s still getting consistently high scores, which tells me to keep it current and continue to include it.

今日は。Guten Tag. Bonjour. ¡Hola! 안녕하세요. 你好. Ciao. Здравствуйте. Olá.

I’ve been on leave for a while, so my apologies for being quiet lately. I’ve been sitting on this news for a while, though I think these links might come in handy if you are touching a global audience. We have localized the documentation for ACT into 10 different languages:

日本語 (Japanese)

Deutsch (German)

Français (French)

Español (Spanish)

한국어 (Korean)

汉语 (Chinese - Mainland)

汉语 - 台灣 (Chinese – Taiwan)

Italiano (Italian)

русский язык (Russian)

Português (Portuguese – Brazil)

If there is one thing I have noticed in my travels, it’s that language is a hard barrier to cross. This is a small but important step. As group after group stretches their brains to understand my English (or my horrible Japanese), we’re taking a few more steps to meet in the middle. I’ve also been doing some work to build more labs – things with fewer words that let the products and experiences do the teaching so they are shorter and less expensive to translate, but still deliver the core knowledge. Because, no matter where you are in this little world of ours, we’re all in this together.