Windows CE Networking Team WebLog : Author: John Spaith

Kerberos cannot resolve netapi32.dll in CE5.0 after QFE updates

cenet — Fri, 16 May 2008 19:37:00 GMT

If you install the recent CE 5.0 QFE's AND try to build Kerberos.dll, you will run into a bug that unfortunately was introduced in this QFE. Check out customer thread alerting us to it here.

Short summary is that customer is seeing:

"Warning: Unable to do imports from kerberos.dll to NETAPI32.dll - will late bind"

Root cause:

To fix a problem with how Kerberos resolves domain names (basically turning REDMOND domain into a DC controller mydomain-controllername.redmond.microsoft.com), Kerberos was changed to call into the official NetApi32 DsGetDcName instead of using a private interface. This meant that kerberos has a dependency on Netapi32.dll being in the image which previously it did not. Customers never think about dependency logic (or hardly ever) because Microsoft runs a large suite of tests to make sure that if component A relies on component B being in the image it explicitly sets it (Kerberos needing Netapi32 in this case) and we fix these issues before releasing the product.

In this case, because it was a QFE we don't usually add dependencies like this, so we didn't catch the dependency prior to QFE release.

The customer workaround is straightforward. Basically set SYSGEN_NETAPI32=1 (I'm not sure what exactly this translates to in the IDE catalog, probably something like "Domain Discovery"). You only need to do this if you include Kerberos component, which most devices do not. As near as I can tell, the only way you get kerberos is of course if you manually include it, or if you bring in the voipPhone local authentication plugin (SYSGEN_VOIPPHONE_LAP).

We apologize for the inconvenience. We're going to be releasing the same QFE effectively in CE6.0 and we will fixup the dependencies prior to releasing that QFE.

John

Old Man Spaith is out of ideas

cenet — Mon, 10 Mar 2008 01:08:00 GMT

I just checked out the blog and realized it's been ~1.5 months since the last post, which is lame. I have a dilemna since I've run out of tricks/advice that I think are useful, for now at least. I can either write nothing or do something random, like put up a picture of my dog, Daisy. It never would've occurred to me to do this, even in desperation for posts, on what I think of as a work-blog until I checked out the main feed at blogs.msdn.com. I was amazed at the number of personal, totally random, stuff up there.

My compromise is that I'll just link to pictures of Daisy rather than wasting bandwidth for people here for work. So check out Daisy Spaith on my Toastmasters blog, www.mySpaith.com. (I guess given MS investment in Facebook I should've named SpaithBook, but I put it up before that and mySpaith is better anyway.)

I wish I could say we had made CE networking technologies so easy and awesome for our developers that we could turn the CENet blog into the "dogs of the CE Networking team" site. We know we're not there yet. Please keep the questions coming in the newsgroups and we'll do our best to help. Thanks for your continued support through the years. I'll try to come up with better stuff so we can avoid the self-indulgent posts like this I promise.

-- Old Man Spaith (just turned 31, it's down hill from here)

Problems building svsutil.hxx in Windows Mobile SDK

cenet — Tue, 22 Jan 2008 19:42:00 GMT

If you try to build svsutil.hxx on certain versions of the Windows Mobile SDK, you may run into build errors. I know it's on WM5 SDK at a minimum, I don't know about WM6 SDK.

The error you'll see will complain about the methods int operator==(int iBit) and int operator==(SVSBitField &bf) in the class SVSBitField in particular. What happened is that the compiler got a bit more picky at some point but the SDK headers weren't updated to reflect this in the given SDK version. We have fixed this in latest & greatest svsutil.hxx.

If you run into this, what you'll need to do is change the initial
for (int i = 0 ; i < m_iWLength - 1 ; ++i) {

So that the 'int i' is outside the for loop, namely:

int i;
for (i = 0 ; i < m_iWLength - 1 ; ++i) {

Our apologies to anyone who hits this.

John

The History of the DCOM Remoting Addon Pack

cenet — Wed, 09 Jan 2008 00:54:00 GMT

I recently wrote about the recent DCOM Remoting Addon pack being available.

Since I'm running out of things to write about, I'll explain the geeky details of how the DCOM addon pack came to be. This post is for the very bored only or those who are intrigued by the Windows CE Build Process. There are no profound insights on COM or any other subject, for that matter.

First we had to figure out what to do with DCOM remoting. We had to either re-port Windows Vista COM or else remove DCOM remoting from the product. Even though CE is componentized and we could have warned that this component was insecure & should be avoided almost always, NT4 vintage DCOM was *so* insecure that this wasn't enough. After all, how often do you click through the obnoxious warnings that software pops up?

This was a no win situation. Even if we wanted to spend the effort to re-port COM, the amount of time needed to do the work and then get it stable (think about how key COM is to the system) didn't allow us to go down this path. At the same time we knew we had some OEMs who desperately needed DCOM remoting.

So the Group Program Manager, who makes a lot more money than I do for good reason, helped push through the idea of an addon pack. DCOM remoting would be available only in CE6 if you downloaded a separate product. The addon pack made our security people happy. Since you have to go to a website to get the bits, you can't accidentally click on some buttons and end up with it. It also lets us warn you point blank on the download site what you're getting into in a way less likely to be missed than warning popup box #73.

From the website:

<
"the component contains known MSRC class critical bulletins. Installing the supplemental pack ... is discouraged, and Microsoft assumes no liability if this pack is deployed."
>

How does the DCOM addon pack work? Ironically for all the time I spent in meetings discussing various aspects of it from biz/legal/security, the actual dev work was straightforward. On NT4, DCOM has the concept of loadable transport DLL's to actually do the heavy lifting of talking to the transport layer. rpcltscm.dll & rpcltscm.dll.

The way PB builds MS doesn't ship these DLL's but instead rpcltscm.lib and rpcltscm.lib, and under the covers OEMs are building the DLL themselves. We have config files that allow us to specify which files (in this case in public\dcom\oak\lib\<CPU>\...) are shipped out to customers. So I changed the logic in them so that rpcltscm.lib & rpcltscm.lib were no longer shipped, even though they were still building. I also changed a few configuration files to add some comments should some geek poke around the build and try to add the DLLs manually. Check out public\dcom\oak\lib\(dcom.reg & dcom.bib) if you want to see Spaith's warnings.

There was the extra step here of making sure that DCOM didn't have any direct calls to Winsock or other networking layer technology to do an end-round the transport DLL layer. It doesn't. I was impressed with the cleanliness of the design. I anticipated whacking out winsock calls thrown all over the place and had given myself a lot more time on the schedule than I actually needed. Of course a lot of other work that I underestimated ended up filling the free time I had!

So the addon pack is basically just an MSI installer, various EULA's (the most important piece from a biz perspective), and the libraries.

At the time of shipping this I joked about trying to get a ship-it award, which would be riduculous given that I hardly spent a few weeks on this. (Other ship-its have taken me +1 year of work). The Group PM said in all seriousness he'd try to make it happen, but I think it's best to let it drop.

John

DCOM Remoting on Windows CE 6.0

cenet — Fri, 04 Jan 2008 23:53:00 GMT

As I wrote here, in Windows CE 6.0 we removed DCOM remoting - that is the ability to create and host COM objects across a network. We did this due to security. Our DCOM is built on NT4 vintage code and there was no way we could patch up our implementation to meet security requirements. The desktop faced a similar problem post-NT4 and also came to the conclusion they couldn't do a patch up, so they had to redo lots of their DCOM.

Note this is all embedded CE only - Windows Mobile doesn't even have cross-proc COM objects, much less remoting.

I'm happy to announce that an add on kit to CE6 is available here that will allow OEMs to get DCOM remoting in their image.

As the docs call out but I will emphasize here, this is ONLY for completely closed networks. I'm thinking of a factory floor as the main example (and they're the main ones who want this). This is not safe for the Internet of course or anywhere that untrusted machines may get on a network due to security worries.

While we can't make any official statements about future products on the blog, I'll say unofficially that we are most likely not going to be doing another remoting addon pack for Windows CE 7.0. We wanted to provide some sort of stop gag solution as Web Services for Devices for instance (available in CE 6.0 R2) or other technologies become available.

UTF-8 in ASP pages on Windows CE

cenet — Fri, 14 Dec 2007 22:53:00 GMT

My "stuff to blog about" well has pretty much run dry, as anyone who follows this blog knows. Fortunately there's always a customer running into something that I've never thought about something to inspire me. Bad for the poor customer hitting the issue, but good for the rest of us.

A customer wasn't able to make ASP pages from CE Web Server work when the pages were UTF-8. The same page had worked fine as ANSI. The error they got was when setting <% Response.Expires = -1 %> in their page.

-----------------START

The HTTP headers are already written to the client. HTTP header

modifications must be made before writing page content

ASP scripting compilation error: '80020009'

Description:

In file: /Test/default.asp

On line: 1

-----------------END

ASP aficionados, can you guess what was happening?

What happens is UTF-8 puts special characters in the first bytes of a file indicating to an editor that it is UTF-8. When our ASP interpreter parses out a page, it sees those UTF-8 characters and (unfortunately) doesn't know about UTF-8 and assumes its ASP body that needs to be sent out. Once ASP body has been sent, you're not allowed to send HTTP headers anymore, which is why the Expires=-1 wasn't working.

I had the customer remove the special bytes and also look into the Response.Charset() to make sure the browser knew the encoding of the file (always a good idea with non-ANSI anyway).

GPSID: Problem & workaround on recent WM6 release

cenet — Thu, 06 Dec 2007 19:14:00 GMT

Background
A number of customers on a recent WM6 versions have run into a problem where GPSGetPosition returns ERROR_INVALID_PARAMETER always, even on apps that worked fine in the past. The newsgroup thread is here.

The device I've reproed and tested on was an HTC P3300, but based on newsgroup thread I think other devices may have been affected also.

I have determined the cause and a workaround for the problem. It'll take a while to get a real fix, but we're working on that too.

Underlying Cause
In a future Windows Mobile, we're exposing extensions to the GPS_POSITION and GPS_DEVICE structures. GPSID is actually aware of the extended structure already - in WM6 AKU0.4 & above, also known as 18120+ builds. The reason being that GPS chips can do some "magic" with this stuff even before we actually expose the structures to app developers. (It's a very long story, if you're not a GPS OEM you don't care.) Because the structure is larger, it means the sizeof() checks on it have changed but sizeof(GPS_POSITION) that you're passing in has not changed because you're using the same GPS_POSITION as always. The MS GPSID allows the old struct size for legacy.

What complicates things is that OEMs are allowed to replace GPSID on their device, because they may want to tweak it in order to work with their GPS chips. I don't have the source, but I'm almost positive what's happening is that an OEM GPSID is checking only against the extended GPS_POSITION size and hence is failing when it gets the original GPS_POSITION.

Workaround
Your application can lie about how big a structure it is passing in to worakround this. So if your application gets ERROR_INVALID_PARAMETER, it can pass in the sizes of the new struct. Here's code below. I tested this with an HTC 3300 and was successfully able to get my lat/long using this. GPSGetDeviceState has similar issue so I'm including code for it, too.

// Retrieves GPS location data and prints it to the screen.
void GetLocationAndPrint(HANDLE hGPS) {
    BYTE gpsPositionRaw[376];
    GPS_POSITION *pGpsLocation = (GPS_POSITION*)gpsPositionRaw;

    pGpsLocation->dwVersion = GPS_VERSION_1;
    pGpsLocation->dwSize    = sizeof(gpsPositionRaw);

    DWORD dw = GPSGetPosition(hGPS,pGpsLocation,1000,0);
    if (dw != ERROR_SUCCESS) {
        wprintf(L"GPSGetPosition() fails, GLE = 0x%08x\r\n",dw);
        DebugBreak();
        return;
    }

PrintGPSPosition(pGpsLocation);
}

void GetDeviceStateAndPrint(void) {
BYTE gpsDeviceRaw[332];

    GPS_DEVICE *pGpsDevice = (GPS_DEVICE *)gpsDeviceRaw;
    pGpsDevice->dwVersion = GPS_VERSION_1;
    pGpsDevice->dwSize = sizeof(gpsDeviceRaw);

    DWORD dw = GPSGetDeviceState(pGpsDevice);
    if (dw != ERROR_SUCCESS) {
        wprintf(L"GPSGetDeviceState() fails, GLE = 0x%08x\r\n",dw);
        DebugBreak();
        return;
    }
    PrintGPSDeviceState(pGpsDevice);
}

Going Forward
This is clearly a stop-gag solution. There's no way that we expect our customers to recompile existing apps for a new WM release, especially considering how hard it is to get applications signed/tested/deployed. I have alerted the OEM to the problem and am coordinating with them on rolling out a fix. I'll provide an update when I can.

Once again I apologize for the pain that this has caused and the delay.

John

How to drop a (Microsoft Generated) debug DLL into a Windows CE device?

cenet — Tue, 10 Apr 2007 02:21:00 GMT

Sometimes on the newsgroups I tell an OEM that's having problems with some component to "Drop DEBUG DLL foo into the release directory and turn on full DEBUGOUT and get me the output." Most Windows CE components have pretty good debug-time logging so a lot of times a customer can see the exact cause of the problem spit out in DEBUGOUT.

But how do you get this DEBUG DLL and its output? RETAIL DLLs don't have this, of course. The easiest way is if you can build a fully DEBUG version of the OS and boot it while connected to Platform Builder, then you're pretty much done. Just turn on appropriate debug zones for the component you need. This is recommended since it's one less step.

However there are scenario where you not have enough RAM/ROM to take a fully DEBUG OS. In this case you want to just drop on the DEBUG dll that you're interested in but have everything else be RETAIL. To do this:

1) Build the complete platform for both DEBUG and RETAIL OS.

2) Open RETAIL project in PB, then open a build window. In CE 6.0 this is Tools->Open Release directory in Build Window, menu and wording may be slightly different in different versions.

3) This will put you into the release directory for the RETAIL build. The release directory is just the dir where all your intermediate DLLs/PDBs/etc get dumped prior to creating a ROM image. Create a new directory here, ORG, and copy ModuleYouWantDebugOutFor.* to ORG.

4) If you "cd ..", the DEBUG directory will also be present. (I'm assuming you haven't customized releasedir position in anyway). copy ..\<debugDir>\ModuleYouWantDebugOutFor.* to RETAIL releasedirectory.

5) Now in PB, run Build->Make Run-Time Image. (Again, wording may change slightly across releases.) This automatically puts the DLL you want DEBUG into the ROM image.

6) Before you ship this ROM image to the world you need to remove the DEBUG bits from it. Ideally you have a formal build process of generating DLLs from a "clean" source, not developer's machine, but in case you don't be aware of this issue.

Note that not all CE Components have DEBUG messages, but most do.

Happy debugging!

[Author: John Spaith]

time.h for WinCE at OpenNetCF

cenet — Thu, 22 Mar 2007 20:28:00 GMT

Windows CE does not support the time.h structures, as I blog about here. Fortunately Chris Tacke at OpenNETCF has created a free library to do this, which is available here, in order to make porting easier to CE. (If you're not porting code you should be using the Win32 time functions, which WinCE does support.)

Whenever I point folks at non-MS software I always have the disclaimer that MS doesn't officially support it, buyer beware, etc etc... I'm embarrassed to have to put that in for this case because I know Chris is a really good guy who's done a lot for the CE development community. Thanks Chris for your work here!

[Author: John Spaith]

MEDC 2007 and Location Talk

cenet — Mon, 26 Feb 2007 21:16:00 GMT

MEDC is the Mobile and Embedded DevCon. The US version will by May 1-3 this year in Las Vegas. I thought about trying to sell it some in this blog, but the official site does a much better job. I went to one of these back in '04 (it had a different name then). I got a lot out of it getting to interact with the folks actually using our software and I hope our customers got a lot out of it too.

I will be going to this MEDC to give a presentation on getting your location at the platform layer on Windows CE. The sessions are listed here, mine is called "Beyond Maps with Windows® Embedded CE Platform."

My talk is pretty simple:
* Problems with apps accessing location in the past, with GPS as the case study. Understanding the history actually is helpful.
* Introduction to GPSID
* Introduction to Location Framework

It won't be a marketing pitch about how cool location is going to be, but instead a pretty deep technical dive. I'll have demos of actually writing apps to use GPSID and Location Framework during the talk to show how easily it is (hopefully) and prove to the world I haven't forgotten how to program quite yet :).

In addition to location, I have pretty deep technical knowledge of services and middleware on CE and I'm the dev lead for VOIP (which doesn't mean I have deep technical knowledge of it though :(). I'll be hanging around the conference and would love to talk to as many folks face-to-face to answer your questions and understand any problems you have with CE. I don't even know what day I'm speaking so my schedule is up in the air, but I'll announce on the blog once I know as well as times I'll definitely be hanging around the Q&A booths they have setup.

I've got a chance to work with and learn from a lot of you folks over the blog and newsgroups. I really am looking forward to getting to be under the same roof with you finally!

[Author: John Spaith]

Important WinCE KB article regarding Daylight Savings Time

cenet — Fri, 12 Jan 2007 20:20:00 GMT

Because I've been leading off a bunch of my more recent posts on obscure subjects with "Well, probably only 5 people care about this..." I'm afraid only 5 people are reading the blog now. I hope not because this post is important. If you're a Platform Builder customer you need to manually change some registry keys in order to get daylight savings dates to occur at the right time in the US as a result of the new change over dates. You also need to have all the QFE's installed, though that hopefully goes without saying anyway.

There's been a KB article up about this for some time - http://support.microsoft.com:80/kb/923027 - but I'm not sure if even 5 people read those. In particular check out "Replacement registry key information" for the version of CE OS you're running.

This is critical if you're relying on the CE daylight savings time auto-rollover feature, since it reads the time to roll over from the registry. Any well behaved application should be reading these dates from the registry also, so anything else that relied on this would be affected by reg changes also.

MS is looking into rolling the .reg change into a QFE so it's picked up automatically, it wasn't back when the KB article was written because then we were following the old daylight savings dates and would have broken any device's rollover for the Oct 2006 instance.

[Author: John Spaith]

How do I check a username/password validity on a local device on a WinCE device?

cenet — Thu, 11 Jan 2007 20:42:00 GMT

Suppose that you get a username and a password and you want to see whether it is legitimate. The telnet server as an example of this – you give it a username & password that it needs to check before letting you party on CE. It’s possible to do the check by some calls into the SSPI layer, but these are convoluted. WinCE has a helper library, authhlp, in order to do the work for you. Note that authhlp is not officially supported or doc’d in MSDN. It is used by quite a few of our network services so the code is very well tested, but it’s not something we can officially support. Consider this a blog for the desperate who doesn’t want to mess with SSPI.

The include header is authhlp.h, and you link to authhlp.lib. For general embedded it lives in <WinceRoot>\public\<YourProject>\cesysgen\sdk\lib\<CPUPATH>\authhlp.lib. It is a static library, so there are no extra SYSGEN’s that you need to be able to use it. Note however that you do need to include the underlying security SYSGENs, in particular SYSGEN_AUTH_NTLM.

Using it is straightforward. You can see sample code of it in the telnet server in <WinceRoot>\public\servers\sdk\samples\telnetd. You need a one time AuthHelpInitialize(), AuthHelpUnload() at the time your service/app is shutting down. To check username/password, just call AuthHelpValidateUserA/AuthHelpValidateUserW. The code is reasonably self-documenting, including a bunch of information at the top of the file. There is also an optional and primitive “ACL” system that Authhelp can support, where it can check not only for a username/password being legitimate but whether a given user is in the allowed or denied list or even if they’re in the NTLM security group.

How do you actually tie into getting the username & passwords? To use a domain controller to validate against, set HKEY_LOCAL_MACHINE\Comm\Redir\DefaultDomain to the name of the domain. The domain should be the actual domain name, not a particular domain controller (so WINGROUP not wingroup-dc2.microsoft.com). If this registry key is not set, then the security mechanism will use a local CE database of user names and passwords. This is configured with NTLMSetUserInfo<http://msdn2.microsoft.com/fr-fr/library/ms926215.aspx> and friends.

I don't know if this would work at all on Windows Mobile, in particular I don't think we ship the libs & headers in its SDK, and I've only ever tested this with Platform Builder generated images. Even for you PB guys, remember this is technically unsupported (which is why this is in a blog & not MSDN proper :)).

[Author: John Spaith]

Avoid TLS calls in services/device drivers while processing IPC calls on WinCE

cenet — Fri, 05 Jan 2007 20:08:00 GMT

This post is another of the pretty low-level, under the hood about interprocess communication on WinCE. I considered not even posting it, but it meets my geek threshold (5 people in the world will need it sooner or later, but probably not more than 5) so I'm going ahead.

One of the main functions of Services.exe on Windows CE is that it provides glue to tie services to applications running on the same device. There's a bunch of low-level kernel magic that has to happen to make Interprocess Communication (IPC) work and services.exe hides that from you. An example of this is that an application call to DeviceIoControl() gets turned into the specified service's xxx_IOControl() handler.

When writing these xxx_ handlers (like xxx_Open, xxx_Init, xxx_IOControl -- see here) you must not make any calls that require TLS (thread local storage). The reason is that the kernel does not support TLS when doing interprocess calls.

The main user of TLS you'll run into is COM. On PSL threads you can call pre-existing COM objects (provided you know they're clear of TLS calls), but you can't call COM API's like CoCreateInstance and friends. Fortunately most services avoid COM in the first place, but should you absolutely need to do COM from within a xxx_IOControl() you'll need to spin up a worker thread to do the work, and block until it's completed. We're aware this is inconvenient and are looking into adding cross-process TLS support in a future version of WinCE (it's not in CE 6.0).

Also note that a lot of DLLs in their DllMain() tend to do TLS just by virtue of doing TlsAlloc, since if they'll do TLS at any point they'll want the slot reserved. This means that loading or unloading DLLs from within an xxx_* function can cause problems. Services.exe in fact spins up a worker thread to load your service when processing an ActivateService() API call (rather than using the same thread that the ActivateService() came in on) in case your service is using TLS in its DllMain. This worker thread is also used to call your xxx_Init(). Similarly, services.exe spins up a worker thread to unload your service and call its xxx_DeInit().

Everything I say here (save one point) applies to device drivers as well. I just am biased toward services since that's where most of my experience is so that's why I use it as my examples. The only difference is that device.exe doesn't spin up worker threads for DLL Load/unload/xxx_Init/xxx_DeInit, since for whatever reason service DLLs are much more likely to bring in stuff that wants to use TLS than device drivers.

As far as what happens with TLS messing up, it depends. On a fully DEBUG build you'll get a DEBUGCHK if you try to do TLS across a PSL call. In any event, it's up to the caller to deal with the error gracefully. This is a good reminder always to handle error conditions even on functions that "shouldn't" fail like the TLS fcns.

[Author: John Spaith]

GPSID source code: the ultimate answer

cenet — Wed, 03 Jan 2007 19:26:00 GMT

Recently I've been asked some questions about how GPSID handles certain weird scenarios that fall outside the scope of the docs. I'm happy to answer these of course - it is my job after all :). The really hard questions have come from GPS device driver writers, but this post may help app developers also. Another and often faster you can get the answers about GPSID is to look at the GPSID source code. It's shared in the public tree so Platform Builder customers or OEMs with the src access in WM can look and modify, if needed.

The root directory is \public\COMMON\oak\drivers\GPSID. Under here is the .\client directory, which is the link-time library for gpsapi.dll. This isn't very interesting (it's just IPC magic between the app calling GPSID and GPSID.dll, either in device.exe or servicesd.exe depending on OS version). You shouldn't ever mess with this code.

The more interesting piece is in the .\server directory, which compiles gpsid.dll itself. This is the DLL that runs in device.exe/servicesd.exe and does the interaction with the GPS device as well as taking IPC calls from apps. It's only about 6000 lines of code and (I flatter myself :)) it's written well enough that no one has ever had problems following it that I know of.

For the record, here's a brief overview of the files in this directory and what they do:

GPSAPI.cpp -> "Receives" the IPC calls to GPSOpenDevice, checks params, and calls into rest of GPSID.

GPSCore.cpp -> This is the "main" set of routines for GPSID. Most interesting fcn is GPSWorkerThread(), which is spun up to interact with the GPS device.

GPSDriver.cpp -> Provides abstraction between rest of GPSID and the underlying GPS driver, be it a standard NMEA device, the Poll Driver (CE6.0 and above), or the simple text file reader.

GPSHandle.cpp -> Manages handles from API/caller process layer

GPSIntrf.cpp -> xxx_Init/xxx_IOControl etc interface for tying GPSID into device.exe/servicesd.exe

GPSMultiplexer.cpp -> Implements the logic for the virtual COM port/multiplexer in GPSID.

GPSParse.cpp -> Parse out the NMEA string and (CE6.0 and above) when using the poll driver, potentially generate a NMEA string.

GPSString.cpp -> Helper utilities that parse out individual fields in a NMEA string

It's interesting that the way the alphabetical list ended up working out, you don't actually get to the string parsing until the very bottom of the list. Ah, abstraction!

The best way to follow the code is to look at what the API's do (GPSOpenDeviceIntrnl & GPSGetLocationIntrnl especially) and to trace through GPSWorkerThread.

[Author: John Spaith]

Hardcore pointer marshalling samples for Windows CE 6

cenet — Wed, 03 Jan 2007 02:43:00 GMT

This is one of those blogs where I hesitated to post it because it may be going too hard core into the guts of CE. But from dealing with our customers who use our shared source and some of the really hard core questions I get, I'm hoping there are at least 5 super-geeks who can benefit from this.

Suppose you're writing a device driver or a service that has to do all sorts of really gross pointer marshalling across process. For instance, consider MSMQ. Apps get at MSMQ by calling API's like MQOpenQueue in msmqrt.dll. This function is called from user space, but just takes the paramaters the user specified, calls DeviceIoControl(hHandleToMSMQ,...), which calls the MSMQ service to do the real heavy lifting.

In the old kernel with the flat address space this was easy (relatively). In the new VM model it gets trickier. Sue Loh has a good blog about this at http://blogs.msdn.com/ce_base/archive/2006/11/09/Memory-marshalling-in-Windows-CE.aspx.

Sue Loh has another good blog about the raw pointer marshal API for CE 6 at http://blogs.msdn.com/ce_base/archive/2006/11/22/marshalling-helper-apis.aspx.

This API was designed so that both kernel level drivers and user mode drivers (e.g. udevice.exe & servicesd.exe) could use the same set of API's. To make marshaling even easier, we have a set of templates in the %_WINCEROOT%\public\common\oak\inc\psl_marshaler.hxx. This header is extensively commented along with samples showing how to make it work. The upshot is that if you have some user-mode API that takes say 5 paramaters (some ptrs that need to be marshalled across process, others regular old DWORDs) then by using the PSL Marshaller templates you can save yourself a lot of the tedium as far as manually packing and unpacking these datatypes during the DeviceIoControl() call.

Besides the psl_marshaler code itself, some samples may help if you have to do this.

GPSID: GPS Intermediate Driver. This is the easiest sample (by far) I know of that uses psl_marshaler stuff in actual shipping code.

%_WINCEROOT%\public\COMMON\oak\drivers\GPSID\client - GPSAPI.dll src, link time library. All it does is just call to the psl_marshall helpers
%_WINCEROOT%\public\COMMON\oak\drivers\GPSID\service - GPSID.dll, actual service that implements guts of this.

You can look on http://msdn.microsoft.com/library/en-us/mobilesdk5/html/mob5grfGPSIntermediateDriverFunctions.asp?frame=true to get a sense that there's just not that much going on with this API. Actually that's intentional to keep implementation easier :). The harder ones are code & API definitions we port from desktop Windows. Here they have MIDL to do all the really heavy lifting for them, unlike CE where it's still psuedo-manual. CE actually does have interprocess COM and we could rely on it & MIDL to do our IPC, but it adds 500KB to our ROM, isn't on Windows Mobile devices, and isn't as efficient in terms of CPU/memory as using the raw access routines & templates.

MSMQ: Microsoft Message Queuing. Note you'll need to be a Platform Builder customer and install the shared source kit in order to get this code.
We're past the point of the mythical 5 people who actually need this much information. I'm hoping you guys can figure out how to do the interprocess calls based on docs, blog entries, and the GPSID sample. Now I'm basically targetting 1 person who actually has to marshall pointers inside structures inside arrays of VARIANTs, and to do it asyncronously at that. My first recommendation is that if you have any control over your API, then try to keep it as simple as possible. No embedded pointers or arrays, no asyncronous writing back of data.

However, if you're in this situation you're where I was at when getting MSMQ up on the new kernel. In this case I'll point you at the MSMQ code in hopes that it may provide you some inspiration as to what you're trying to do.

%_WINCEROOT%\private\servers\msmq\sc\ce\client -> msmqrt.dll, runtime library apps link to to call into MSMQ.
%_WINCEROOT%\private\servers\msmq\sc\ce\driver\msmqdev.cxx - MSMQD.dll. Especially MMQ_IOControl().

[Author: John Spaith]