If you have a rusty old bucket and it is overflowing because you put too much water in it, then you should certainly think about buying a nice new plastic one at some point. But the new one will still overflow if you put too much water in it.
I was investigating an issue today and needed to create a kernel dump on demand in my repro machine, a copy of Windows Server 2003 SP2 hosted in Hyper-V. I successfully blue-screened it (!):
and left it to get on with generating the dump and rebooting. Because I wasn’t watching it I never saw it actually get to 100%.
However when I opened the dump I got a whole load of messages I didn’t like the look of:
Kernel Complete Dump File: Full address space is available
************************************************************
WARNING: Dump file has been truncated. Data may be missing.
**************************************************************************
THIS DUMP FILE IS PARTIALLY CORRUPT.
KdDebuggerDataBlock is not present or unreadable.
**************************************************************************
*********************************************************************
Unable to read PsLoadedModuleList
**************************************************************************
THIS DUMP FILE IS PARTIALLY CORRUPT.
KdDebuggerDataBlock is not present or unreadable.
**************************************************************************
Unable to read selector for PCR for processor 0
GetContextState failed, 0x80070026
GetContextState failed, 0x80070026
Unable to get current machine context, Win32 error 0n38
GetContextState failed, 0x80070026
Unable to get current machine context, Win32 error 0n38
GetContextState failed, 0x80070026
and so it went on, and on, and on, page after page of scary looking error messages.
So I tried again a few times but always the same result.
Then I got to thinking about why the dump might not be getting fully written and I thought “pagefile”.
The pagefile settings in the pre-configured virtual image I was using looked something like this:
And sure enough if you try to set such settings you get a warning like this:
So I set it back to “system managed size”:
and rebooted then crashed the system again and this time the dump was fine with no corruption reports.
Now I had always thought (and thought it had been my experience) that if your pagefile was too small then you simply didn’t get the dump file created when the crash occurred. I had not realised that you might end up with a “partially corrupt” dump file.
HTH
Doug
Once you’ve installed Visual Studio 2010 Beta 1 and read the readme (you do read readmes, now, don’t you?) you find that all the help for this beta release is online. No offline documentation is included. That’s understandable and all very well but what if you want to browse it while you are not online, on a plane or whatever? Also, it makes targeted searching a little difficult as well, for example if you just want to browse and search within the Visual Web Developer and ASP.NET documentation.
Fortunately the MSDN Publishing System Content Service and a cunning utility/sample called PackageThis come to your rescue. This tool talks directly to the MSDNPSCS and finds out what is available. It then lets you select those parts of it that you are interested in and will then package them into a Compiled Help Module (CHM) file:
Note that if you want to select all topics within a node you have to actually right click and select “Select This Node and All Children”. Simply checking the check box next to the node only selects that node.
After selecting the content you want, select Export to CHM from the File menu:
Wait a few minutes and there you have it! Your own fully searchable, fully functioning offline help file:
As one reviewer on the CodePlex site comments, it is not perfect but it is perfectly functional in my opinion and very handy.
HTH
Doug
I don’t think I’ve ever come across anything written by John Robbins that is not worth reading. But in a recent post he talks about a subject that rings particularly true for me – the importance of debug symbols. Symbols are like the Rosetta stone of debugging – they are the key to interpreting debugger output and relating it back to the source code of the application or component. Without it, much of the debugger output is meaningless. Not meaningfree. Just meaningless. It is not impossible to debug without symbols. It is just much harder. Maybe 10 times harder, maybe 100 times harder. And harder means more time and more time means more money.
I have lost count of the number of situations where I have assisted customers with application in production that are dying on their feet due to some crash or memory leak and the conversation goes something like this:
Me: You application is failing/leaking memory from modulename!RandomSoundingFunctionName+0xVERYBIGHEXOFFSET
Customer: My development team doesn’t recognise that function
Me: That’s because it is simply the nearest exported function name that the debugger has found, plus an offset
Customer: Can we get the actual function name?
Me: Unfortunately not without matching symbols
Customer: My development team have just rebuilt the component with symbols and I’ve uploaded the PDB to you
Me: I can’t use that because it doesn’t match the binary that was on the server when the dump file was generated
Customer: My development team says they haven’t changed the source code much since the original deployment
Me: Unfortunately it won’t work. Even with no code changes a PDB can change from one compilation to the next, depending on the compiler. They will need to recompile and then you will have to redeploy, reproduce the issue, gather more dump files and upload them
Customer: That will take a week of dev, a week of test and a week of change control approval
Me: Ok, well, let’s see what we can do in the meantime
So my recommendation is – read John’s post and heed his wisdom.
HTH
Doug
Do you live in the UK near Reading?
Do you manage, maintain or develop a web site and are you working to ensure it is fully compatible with Internet Explorer 8?
There are a few places available on a couple of FREE hands-on lab events to give assistance with Internet Explorer 8 compatibility. The events are on 22nd and 23rd April. These events are at the main Microsoft UK campus, in Reading. If you are interested in attending, please send email to IE8UKLABS@live.co.uk with your…
Full name
Job Title
Company name
Date you wish to attend (22nd or 23rd)
Learning objectives – briefly state what you would like to get from attending the lab
If the events prove popular (which they already appear to be) then there may be some additional ones scheduled.
HTH
Doug
A new started kit “for planning, architecting, and implementing Web 2.0 applications and services using the Microsoft Platform” has been released:
Kobe: Web 2.0 Service Development Resource Kit
There is also a new “Startup Zone” which is “The guide to working with Microsoft for software startups and their investors.”
HTH
Doug
We’ve now published the KB article for a recent hotfix that I was involved in.
I’m not going to write much about it here as a lot of my words have already gone into the KB article:
FIX: You receive a Stop 0x0000007e error message on a blue screen when the AppPoolCredentials attribute is set to true and you use a domain account as the application pool identity in IIS 7.0
The main points are however:
- This problem does not occur “out of the box” – a number of factors have to come into play before you hit this issue.
- Kernel mode authentication (new in Windows Server 2008) has to be enabled.
- appPoolCredentials has to be set to ”true”. A reason for doing this is if your application pool is set to run under a domain account and you are configuring your site to use Kerberos authentication
- we’ve only seen this issue so far in relation to Microsoft Office SharePoint Server 2007, but it is not a problem with that product. it is just that the way that product sometimes requires IIS to be configured leads it to fall victim to the bug in HTTP.SYS
HTH
Doug
[Updated 8th April 2009. I don’t generally like to change posts after they have been published but I wrote this post while some of the KB articles were still being authored. For a number of reasons there have been some changes to which articles document what issue, so I’m updating the below table to reflect the current situation as accurately as possible]
[Updated 27th April 2009. Correcting some build numbers and adding rollup KB969612]
There have been a few ASP.NET issues lately that have had to be resolved by the production of new hotfixes.
These fixes are recent and are not included in either the Microsoft .NET Framework 3.5 SP1 nor the subsequent .NET Framework Family Update.
When we release a number of fixes at around the same time, it can get a little bit confusing as to which fix package you actually need to get hold of and install in order to resolve the problem you are having. In due course this should be clearly documented in the KB article.
The below tables will hopefully help clarify things a bit. The first column has the link to the KB article that documents the issue. The description column is the high level description of the problem. You then have to decide which operating system you need the fix for. The fix packaging technology is different for different families of operating system. Look in the column appropriate to what you need and look down to the correct row. Either click on the link you find there or, if you see a ‘V’ continue looking down until you find a link or information about how to get the fix.
An important point is the build number. Microsoft hotfixes are always cumulative. So if a fix is included with a particular build of a particular set of files then any later build (higher build number) of the same files will include all previous fixes that affect those files.
Note: at the time of writing this blog post, not all the KB articles describing the problems have been published. So the KB article link may not yet work when you read this. The fixes are available however.
The fix packages in the first table update this set of files:
aspnet_wp.exe
system.web.dll
webengine.dll
I think of these as the “core” ASP.NET files.
The file version number is 2.0.50727.xxxx where xxxx is the build number.
| KB describing the problem | Description | Fix package for Windows 2000/XP/2003 | Fix package for Windows Vista SP1 / Windows Server 2008 | Build number |
KB961847
| FIX: Error message when you view a page in design mode after you recompile a .NET Framework 3.5 Service Pack 1-based project: "Error creating control – 'ControlInstanceName [text]' could not be set on property 'PropertyName'" | V | V | 4002 |
| KB961864 | FIX: Web clients may have several problems when you access an ASP.NET 2.0 AJAX-enabled Web site | V | V | 4002 |
| KB961884 | FIX: You experience very slow performance on an initial request to an ASP.NET Web site after you change the App_Code folder, the bin folder, or the Global.asax file | V | V | 4005 |
| KB962351 | FIX: ASP.NET Dynamic menus do not render correctly in IE8 in non-compatibility view | V | V | 4007 |
| KB961902 | FIX: Error message when you compile an ASP.NET project that contains many application resource files: "A first chance exception of type 'System.ArgumentException' occurred in mscorlib.dll" | V | V | 4011 |
| KB967535 | FIX: A hotfix is available to fix two problems in ASP.NET on IIS 7.0 for Windows Vista and Windows Server 2008 | n/a | KB967535 | 4013 |
| KB969612 | Description of the hotfix rollup package for ASP.NET 2.0: April 2009 | KB969612 | | 4028 |
The following fix affects this file:
System.Web.Extensions.dll
| KB describing the problem | Description | Fix package for Windows 2000/XP/2003 | Fix package for Windows Vista SP1 / Windows Server 2008 | Build number |
| KB957661 | FIX: Exception error message when you use a virtual path provider to serve a Web page that has a ScriptManager control: "Directory '<file directory>' does not exist. Failed to start monitoring file changes." | KB957661 | KB957661 | 4030 |
HTH
Doug
A bit of a random link this, but I stumbled upon this list of .NET sample applications hosted on the MSDN Academic Alliance site:
http://docs.msdnaa.net/ark_new3.0/cd3/content/Type_Sample%20Applications.htm
Doug
Ever seen this error in a WinDBG/CDB debug session?
Failed to load data access DLL, 0x80004005
Verify that 1) you have a recent build of the debugger (6.2.14 or newer)
2) the file mscordacwks.dll that matches your version of mscorwks.dll is
in the version directory
3) or, if you are debugging a dump file, verify that the file
mscordacwks_<arch>_<arch>_<version>.dll is on your symbol path.
4) you are debugging on the same architecture as the dump file.
For example, an IA64 dump file must be debugged on an IA64
machine.
You can also run the debugger command .cordll to control the debugger's
load of mscordacwks.dll. .cordll -ve -u -l will do a verbose reload.
If that succeeds, the SOS command should work on retry.
If you are debugging a minidump, you need to make sure that your executable
path is pointing to mscorwks.dll as well.
This error message is something that often faces people trying to debug dumps of .NET 2.0 applications using WinDBG/CDB using the SOS debugger extension. I’ve been having more than my fair share of issues with it lately and I thought it justified a bit of explanation.
What is mscordacwks.dll?
The Common Language Runtime (CLR) is the core engine of the Microsoft .NET Framework that executes managed code. In simple terms it does this by taking the intermediate language and metadata in a managed assembly, JIT compiling the code on demand, building in memory representations of the types the assembly defines and uses and ensures the resulting code is safe, secure and verifiable and gets executed when it is meant to. This engine is itself implemented in native code. When we want to debug a .NET application using a native debugger like CDB or WinDBG (which we currently do a lot of if we want to debug it using post-mortem memory dump files) we have to use a “bridge” between the native debugger and the managed world because the native debugger does not inherently understand managed code. It is a native debugger.
To provide this bridge, the CLR helpfully ships with a debugger extension – SOS.DLL. This understands the internals of the CLR and so allows us to do things like outputting managed calls stacks, dumping the managed heap etc.
But from time to time, these internal data structures and details of the CLR change and so it is useful to abstract the interface to the CLR that this debugger extension needs from the actual internal implementation of the CLR that makes .NET applications work. Enter mscordacwks.dll. This provides the Data Access Component (DAC) that allows the SOS.DLL debugger extension to interpret the in memory data structures that maintain the state of a .NET application.
If you look in your framework folder you should always see a matching set of these 3 DLLs:
If you work with 64-bit you should also see a matching DLL set in the Framework64 folder.
What does this error message mean?
It means that the SOS.DLL debugger extension has not been able to find the matching mscordacwks.dll that it needs to be able to debug the dump file you are trying to debug.
How do you know I am debugging a dump file?
Because if you were debugging a live application the debugger extension would automatically find and load the mscordacwks.dll from the framework directory.
When am I likely to get this error message and when will I not get it?
If you are debugging a dump file from an application that was using a different build (e.g. different installed service pack or hotfix) of the CLR to the one installed on your local system, or if the .NET Framework was installed in a different location to where it is installed on your system and if the correct mscordacwks.dll is not discoverable by the debugger by some other means.
What “other means”?
Because having the matching mscordacwks.dll is so important for SOS.DLL to work correctly, SOS has a number of tricks up its sleeve to find it. In particular, provided the correct indexing to the symbol server has occurred the debugger will load it from there. The debugger will also look for it in your debuggers directory provided it has been renamed in a special way (see below).
So how do I fix it?
Most of the time, if you have your symbol path set up correctly (which you will need to anyway to make any headway at all with debugging anything, let alone managed applications) then the debugger should be able to get the correct mscordacwks.dll from the symbol server automatically:
!sym noisy
.symfix c:\mylocalsymcache
.cordll -ve -u -l
What if that doesn’t work?
The simplest thing is to ask the person that gave you the dump file to look at to give you a copy of the mscordacwks.dll. Once you have it, check its file properties for the version number. It should be something like 2.0.50727.xxxx. Then rename it to
mscordacwks_AAA_AAA_2.0.50727.xxxx.dll
where xxxx is the appropriate bit of the version number and AAA is either x86 or AMD64 depending on whether you are dealing with a 32-bit or a 64-bit application dump. (The AMD64 is a legacy thing before we referred to x64). Then put this renamed copy into your debuggers directory (the one where WinDBG is installed). Then, as per the error message, tell the debugger to try again:
Although we try to ensure that every build of the CLR that is released (as a service pack, a hotfix or whatever) has its mscordacwks.dll indexed on the public symbol server, unfortunately it sometimes does not happen. But since it always ships as part of the CLR you always have the option of getting it from the machine the dump came from.
I tried the verbose logging option and it seems to be confused about whether it wants x86 or x64. Now what?
So you ran .cordll –ve –u –l as instructed and got a message something like this:
CLR DLL status: ERROR: Unable to load DLL mscordacwks_AMD64_x86_2.0.50727.3053.dll, Win32 error 0n87
What this means is that you most likely took a dump of a 32-bit process (running under WoW64) on a 64-bit system using a 64-bit debugger and you are now trying to analyse the dump using a 64-bit debugger. That's why the message references AMD64 and then x86. This is not going to work. Because the SOS.DLL extension actually makes use of the framework while debugging the bitnesses need to match. I strongly recommend always generating the dump using a debugger of the same bitness as the process (so x86 debugger for WoW64 processes even though the system is an x64 system) and analysing the dump with the same bitness of debugger that generated it. And that means of course you cannot debug a 64-bit dump on a 32-bit system. It also means you have to have the framework installed to debug a managed application dump.
Now it’s telling me it sufferred an “init failure”?
You might see this:
0:018> .cordll -ve -u -l
CLRDLL: ERROR: DLL C:\Windows\Microsoft.NET\Framework\v2.0.50727\mscordacwks.dll init failure, Win32 error 0n87
CLR DLL status: ERROR: DLL C:\Windows\Microsoft.NET\Framework\v2.0.50727\mscordacwks.dll init failure, Win32 error 0n87
This also points to a bitness mix up. I've seen this when using a 32-bit debugger to analyse a dump of a WoW64 process generated with a 64-bit debugger.
Isn’t this problem as old as the hills?
I am certainly not the first person to blog about this and won’t be the last either but I thought it was worth me attempting to explain some of these mysteries since it still continues to get people confused. Here are some other posts about this:
Failed to load data access DLL, 0x80004005 – hm
"Failed to start stack walk: 80004005", "Following frames may be wrong" and other errors you may see in windbg
Production Debugging for Hung ASP.Net 2 applications – a crash course
Loading CLR DAC dll from a different path
HTH
Doug
Internet Explorer 8.0 is currently available as “RC1” or “Release Candidate 1” (which means it is pretty near final version, much more so than, say, a beta version).
This version of IE is a major new version and has lots to offer developers but also has a lot of implications for web site owners and developers due to some of the bug changes it includes.
My colleague Rainer Maurer has pulled together an excellent collection of links to IE8 resources which I am posting here on his behalf:
What's New in Internet Explorer 8
http://msdn.microsoft.com/en-us/library/cc288472(VS.85).aspx
Internet Explorer 8
http://www.microsoft.com/ie/ie8
Internet Explorer Developer Center
http://msdn2.microsoft.com/en-us/ie/default.aspx
Internet Explorer 8 Readiness Toolkit
http://www.microsoft.com/windows/products/winfamily/ie/ie8/readiness/default.htm
IE Blog
http://blogs.msdn.com/ie/
IE8 on Channel 9
http://channel9.msdn.com/tags/IE8/
IE8 Videos
http://www.microsoft.com/windows/internet-explorer/beta/videos.aspx
http://www.youtube.com/profile_videos?user=ie8videos
IE8 Demos
http://www.ie8demos.com
IE8 Add-ons Gallery
http://ieaddons.com/en/
Description of the Compatibility View list in Windows Internet Explorer 8
http://support.microsoft.com/kb/960321/
Some Web sites may not be displayed correctly or work correctly in Windows Internet Explorer 8
http://support.microsoft.com/kb/956197
HTH
Doug
Windows Management Instrumentation (WMI) among other things exposes to WMI clients performance counters that are present on a system. This is useful because it makes performance counter data accessible to script based clients. It also offers a better way for remote clients to access performance counter data because (among other things) the “traditional” method of accessing performance counters relies on the remote registry service and can run into a variety of strange issues.
The standard performance monitoring tool PerfMon offers the option of accessing performance counter data via the WMI interface which is useful if you need to check that the counters you are interested in are being published correctly via that route. To do this just start PerfMon with the /wmi switch.
However the exposing of .NET performance counters via WMI is not without issues. Unfortunately in .NET 2.0 SP1 (which also ships as part of .NET 3.5) we introduced a bug whereby some ASP.NET counters were sometimes not available via WMI and a hotfix was produced:
FIX: No values are returned when you query some ASP.NET-related objects after you install the .NET Framework 3.5 or the .NET Framework 2.0 Service Pack 1
This fix is also included in .NET 3.5 SP1 (which includes .NET 2.0 SP2).
Recently, I assisted my colleague Drew with a problem one of our customers had run into when installing the slipstreamed version of .NET 3.5 SP1 (that is to say, starting from a machine without a previous version of .NET present and installing 3.5 SP1 in one go). After installing this, the same symptom of missing counters when using a WMI client was present.
After much work by Drew and the customer, including having a repro environment set up in our support lab, the problem appeared to be narrowed down to a very particular “corner case” issue. What we found was that if you installed .NET 3.5 and then the fix – no problem. If you installed .NET 3.5 and then SP1 – no problem. But if you installed .NET 3.5 with SP1 as a slipstream install the counters were missing. And all this ONLY seemed to occur if all the following factors were in the mix -
OS: Windows Server 2003
Edition: R2
Bitness: x64
Hardware: dual core
And what’s more it seemed to have to be physical hardware to reproduce the issue; a virtualized environment wouldn’t do it!
If you are wondering what the “R2” edition of Windows Server 2003 is, it is a “second” release of Windows Server 2003 that added some new features, included service pack 1 out of the box and various other things. It is also one of the few (if not only) version of Windows that ships with .NET 2.0 included. Perhaps that is a factor at play in this problem.
Anyway, Drew and I were a bit stuck with this one so we had to get some help from one of the developers on the ASP.NET team to log onto the machine and take a look.
After some poking around he was able to resolve the situation on the lab machine with the following:
Winmgmt /resyncperf
Net stop winmgmt /y
Net start winmgmt
Net start ccmexec (this was a dependent service of winmgmt that was stopped as a result of stopping winmgmt)
This effectively told the WMI service to “look again” at what performance counters were available on the system and expose them via WMI. This “resynching” should have happened automatically but for some reason did not.
So far this is the only occurrence of this particular problem we have seen so I hope you never run into it but just in case, I hope this is useful.
If you are trying to follow these steps on your system be aware that you may not have a ccmexec service.
HTH
Doug
