I've been working recently with some customers who have had to make some tweaks to our build system and they had a heck of a time of it. We have very good reference material on MSDN, but I don't know of anyplace that ties it all together. So using my favorite component, the web server, I'm going to through how we turn C code into DLL's into ROM images.
If you're adding a new, custom component to CE you probably don't have to go through all this. You should just build your EXE or DLL directly and burn it into the ROM directly. However I hope this insight of how we do this internally at MS is useful. Hopefully this will at least convince you we're not crazy for making things the way they are :). Of course there's weird cases that don't exactly follow these rules, but for now we're just going to look at a straightforward component.
Also when I say "you" I'm assuming you are an OEM who has Platform Builder and creates your own platforms.
10000 FOOT OVERVIEW
* STEP 1 BUILD: Turns C code into libraries. Microsoft does this stage and gives pre-built libraries to customers in Platform Builder. The remaining stages you perform yourself (whether or not you realize it :)).
* STEP 2 SYSGEN: Turns libraries into DLL's, based on which libraries are required for componentization.
* STEP 3 BUILDREL: Copy all the DLL's from stage 2 into a central location.
* STEP 4: MAKEIMG: Munge all the DLL's, registry settings, etc... into a ROM image.
It's up to you to get the ROM image onto your device. That's outside scope of this article.
As you read this, if you have PB it may be helpful to have a command line window open as I walk through this. To do this, go to Build OS->Open Release Directory. (The exact wording/menu location of this may change from version to version.) Even if you totally use the IDE and never cmd line to build, the principles are the same since the IDE is just a wrapper.
Also note I'm being lazy on directory names below - when I say \private\servers\http to be technically correct I should say %_WINCEROOT%\private\servers\http.
***** STEP 1 BUILD: C CODE -> LIBRARIES *****
The Web Server (HTTPD) source code is checked into \private\servers\http. It's part of the shared source package if you want to see it. The first thing we need to do is turn all the C code (C++ in HTTPD's case) into intermediate libraries.
Microsoft runs a tool named cebuild to do this. It will first run 'build' (http://msdn.microsoft.com/library/default.asp?url=/library/en-us/wcepb40/html/_wcepb_windows_ce_build_environment_tool.asp) on \private\winceos, then on public\winceos, then it will run sysgen (see step 2). Then it runs build on \private\dcom, then \public\dcom, then sysgen -p dcom, etc... Eventually it gets to \private\servers, the tree we care about.
In \private\servers, there is a file named dirs. This lists the directories that build should recurse into and also the order they should be visited. These may have other dirs files. Node directories that contain source to build contain a file named SOURCES. This is effectively a dumbed down makefile, listing C files to build and various other rules. Build.exe is smart enough to figure out various dependencies, rather than requiring it be done explicitly via nmake files.
\private\servers\http contains a bunch of sub directories, each one of which builds a separate library. For instance .\core contains basic server stuff like logging, HTTP request+response handling, and services.exe interface. .\auth contains HTTP authentication work, and .\webdav contains the webDav piece. They respectively build httpd.lib, httpauth.lib, and httpdav.lib. These files are placed into \public\servers\oak\lib\<CPU>\<debug|retail>\.
These libs are then shipped to you, the customer. We do not ship the web server as a DLL itself. You build this during STEP 2. Note that even if you don't build a device with the web server, you will still get all these library bits for all processors and for both debug+retail. That's why PB takes so much space on your hard drive. After all, we're not psychic and can't predict whether you want to build a web server or not before we send the bits to you.
There's no difference whether a LIB was built from \private\servers or \public\servers as far as the build is concerned. This is just a shared source/licensing thing with regards to where the source code is placed.
***** STEP 2 SYSGEN: LIBRARIES -> DLLS *****
If you want all the details of the sysgen tool, check out http://msdn.microsoft.com/library/en-us/wcepb40/html/_wcepb_Sysgen_Tool.asp?frame=true.
Why don't we ship the Web Server DLL? Wouldn't it be easier and faster to NOT have customer link libs to the DLL? The issue is that CE is a componentized operating system. Some DLL's have dozens of different permutations they can be built with, depending on the scenario. MSXML is a good example - i.e. SAX, XMLHTTP, XSLT are all independently configurable. It's impractical for MS to build all combinations of such a DLL. That means we give you the libs and a process to turn these into DLL's on your own.
The Web Server is also a componentized. To keep things simple, for now let's look at the standard non-componentized DLL - the telnet server. Telnet built telnetd.lib in the same dir as the web server lib was placed in in STEP 1.
If you want the telnet server, you need 'SYSGEN_TELNETD=1' set in your command window. (The PB IDE does this under the covers if you drag Telnet Server into the catalog.) In the file \public\cebase\oak\misc\servers.bat (which is called when sysgen is run), there are some lines that look like this:
if "%SYSGEN_TELNETD%"=="1" set SYSGEN_CMD=1
if "%SYSGEN_TELNETD%"=="1" set SYSGEN_STDIO=1
if "%SYSGEN_TELNETD%"=="1" set SYSGEN_SERVICES=1
if "%SYSGEN_TELNETD%"=="1" set SYSGEN_FSPASSWORD=1
These lines indicate that if telnet was included, then to bring in CMD.exe, STDIO, services.exe, and the filesystem password stuff as well. This saves you from having to figure out all its dependencies.
After that line a little ways, we have this:
if "%SYSGEN_TELNETD%"=="1" set SERVERS_MODULES=%SERVERS_MODULES% telnetd
(Digression: This is CE 5.0 only. In CE 4.0-4.2 we had a similar SYSGEN_XXX idea but the files were in either \public\iabase\oak\misc\cesysgen.bat or \public\hlbase\oak\misc\cesysgen.bat. CE 3.0 was different but we have too much info already)
This means that as we go through the sysgen phase, this SERVERS_MODULES has a list of all the various servers modules (anything built in \public\servers or \private\servers). Now we need to take some action based on this list. In \public\servers\cesysgen\makefile we have standard makefile rules for building the component. The lines for telnet are as follows.
telnetd::
@set SOURCELIBS=$(SG_INPUT_LIB)\$@.lib
telnetd::
@set TARGETNAME=$@
@set RELEASETYPE=OAK
@set TARGETLIBS=$(SG_INPUT_LIB)\$@.lib $(SG_OUTPUT_SDKLIB)\ws2.lib $(SG_OUTPUT_SDKLIB)\coredll.lib $(SG_OUTPUT_SDKLIB)\ceosutil.lib $(SG_OUTPUT_SDKLIB)\authhlp.lib
@set DLLENTRY=_DllEntryCRTStartup
$(MAKE) /NOLOGO $@.dll
The magic here is that we set a bunch of environment variables (SOURCESLIBS tells which libraries are "core", TARGETLIBS are what we link to, etc...) and then a call to $(MAKE) executable. This exe looks at the variables and then spits out telnetd.dll in \public\cebase\cesysgen\oak\target\{CPU}\{debug|retail}. (Again this dir is CE 5.0 only, though on prev version the concept is the same.)
How about the Web Server? In this case, there are 2 separate SYSGEN variables for it. SYSGEN_HTTPD brings in the core functionality except WebDAV, which requires SYSGEN_HTTPD_WEBDAV=1 to be set. In either case, set SERVERS_MODULES=%SERVERS_MODULES% httpd is specified. In order to help make figure out whether it needs to build just core HTTPD or Core+WebDAV, we introduce a new variable in public\cebase\oak\misc\servers.bat named HTTPD_COMPONENTS. This has a bunch of HTTPD libraries (httpauth httpfilt ...) always and may include httpdav if DAV is included.
When it comes time to sysgen the web server, we tie all this together via this rule in the makefile:
httpd::
@set TARGETLIBS=
@set SOURCELIBS=$(SG_INPUT_LIB)\$@.lib $(SG_INPUT_LIB)\httpparse.lib
httpd::$(HTTPD_COMPONENTS)
@set TARGETNAME=$@
@set RELEASETYPE=SDK
@set TARGETLIBS=%%TARGETLIBS%% $(SG_INPUT_LIB)\$@.res $(SG_OUTPUT_SDKLIB)\coredll.lib $(SG_OUTPUT_SDKLIB)\ws2.lib $(SG_OUTPUT_SDKLIB)\ceosutil.lib $(SG_OUTPUT_SDKLIB)\authhlp.lib $(SG_INPUT_LIB)\httpstubs.lib
@set DLLENTRY=_DllEntryCRTStartup
@echo TARGETLIBS=%TARGETLIBS%
@echo SOURCELIBS=%SOURCELIBS%
$(MAKE) /NOLOGO $@.dll
httpauth httpasp httpextn httpfilt httpisapi:
@set SOURCELIBS=%%SOURCELIBS%% $(SG_INPUT_LIB)\$@.lib
httpdav:
@set SOURCELIBS=%%SOURCELIBS%% $(SG_INPUT_LIB)\$@.lib
@set TARGETLIBS=$(SG_OUTPUT_SDKLIB)\ole32.lib $(SG_OUTPUT_SDKLIB)\oleaut32.lib $(SG_OUTPUT_SDKLIB)\uuid.lib
What this does is that "httpd::$(HTTPD_COMPONENTS)" indicates that HTTPD has dependencies on the various libraries (i.e. httpddav) and that these rules should be executed first. So httpauth rules adds httpauth.lib to the SOURCESLIBS variable list. httpdav not only adds httpdav.lib to SOURCESLIBS, but it also has target libs include all the COM libs.
(UPDATE (2/20/07): My description of SOURCESLIBS and TARGETLIBS is correct in how they are used in practice to do stubs, but the actual implementation details I describe are wrong. In the comments of this blog below I have the nitty gritty from the architect, not included it here since there's so much info already. Note I got by here for 8 years not knowing the build magic myself so this probably isn't a critical distinction.)
(Digression: Since HTTPD always requires httpauth httpextn httpfilt and httpisapi, I should've just added them in the "@set SOURCELIBS=$(SG_INPUT_LIB)\$@.lib $(SG_INPUT_LIB)\httpparse.lib" line. But I was young and wanted a super-componentized web server (i.e. no authentication option)! This can't be built in reality because SYSGEN_HTTPD always sets httpauth as a target that it depends on and hence the lib is always included in the final httpd.dll. I guess you could hack it out, but that would be foolish. Always use web server auth!)
This is also the stage where the componentization magic takes place. The Core Web server logic calls a function in its parser called HandleWebDav() which (surprise surprise) handles WebDav requests if the right HTTP verb is sent by a client. However, what happens if the WebDAV library isn't linked to httpd.dll? How do we still link, and even if we can somehow link how do we handle it at runtime?
The answer is in "@set TARGETLIBS=... $(SG_INPUT_LIB)\httpstubs.lib" This library has a dummy stub function named HandleWebDav() which is just a few lines long and returns an error indicating WebDAV isn't supported. This lets us link and it means we could theoretically implement custom error handling of this case at runtime if we wanted to.
That brings up a new question -- how do we avoid multiple definitions of HandleWebDav() if it's defined in both httpdav.lib and httpstubs.lib? The answer is that if the linker sees HandleWebDav() in both a library in the SOURCELIBS list (httpdav.lib) AND TARGETLIBS list (httpstubs.lib), then it will favor the SOURCELIBS one and disregard the TARGETLIBS assuming that this is a stub.
Besides generating DLL's, sysgen also generates .reg and .bib files that specify the initial CE registry and the DLL's and other files that need to be in the ROM. \public\servers\oak\files\Servers.reg contains all possible registry settings for stuff from the servers project. It's just like how we give you all the libs for CE, knowing you'll never need all of them on the same device. Since your image won't include everything, sysgen will make a copy of servers.reg in \public\CEBase\cesysgen\oak\files\servers.reg and will strip out unused component's registry (or .bib) settings while at it. It only includes stuff in "; @CESYSGEN IF" for the modules that were built - ultimately determined based on the SYSGEN_XXX environment variables.
***** STEP 3 BUILDREL: DLL'S -> _FLATRELEASEDIR
It gets easier from here, I promise. The sysgen process generates files all over your release tree (DLL's, .reg/.bib, etc...), which I won't get into. The BuildRel tree simply copies them from these dirs into a single directory, specified by the %_FLATRELEASEDIR% cmd environment variable. BuildRel is documented at http://msdn.microsoft.com/library/en-us/wcepb40/html/_wcepb_Build_Release_Directory_Tool.asp?frame=true.
***** STEP 4 MAKEIMG: _FLATRELEASEDIR -> ROM IMAGE *****
Whew! We have all our DLL's, .reg, .bib, and other random files together in one place. Now we just need to run makeimg. Makeimg is documented at http://msdn.microsoft.com/library/en-us/wcepb40/html/_wcepb_Make_Binary_Image_Tool.asp?frame=true.
Makeimg looks at the .bib files to see what DLL's and other files to include in the ROM and .reg to figure out the initial registry. There are also some environment variables that allow further tweaking, though these are outside of the scope of this entry.
And that's it - enjoy!
[Author: John Spaith]
