Windows Security Logging and Other Esoterica : Tools

Mapping pre-Vista Security Event IDs to Security Event IDs in Vista+

Eric Fitzgerald — Thu, 11 Jun 2009 02:09:00 GMT

I've written twice (here and here) about the relationship between the "old" event IDs (5xx-6xx) in WS03 and earlier versions of Windows, and between the "new" security event IDs (4xxx-5xxx) in Vista and beyond.

In short, EventID(WS03) + 4096 = EventID(WS08) for almost all security events in WS03.

The exceptions are the logon events. The logon success events (540, 528) were collapsed into a single event 4624 (=528 + 4096). The logon failure events (529-537, 539) were collapsed into a single event 4625 (=529+4096).

Other than that, there are cases where old events were deprecated (IPsec IIRC), and there are cases where new events were added (DS Change). These are all new instrumentation and there is no “mapping” possible- e.g. the new DS Change audit events are complementary to the old DS Access events; they record something different than the old events so you can’t say that the old event xxx = the new event yyy because they aren’t equivalent. The old event means one thing and the new event means another thing; they represent different points of instrumentation in the OS, not just formatting changes in the event representation in the log.

Of course I explained earlier why we renumbered the events, and (in the same place) why the difference is "+4096" instead of something more human-friendly like "+1000". The bottom line is that the event schema is different, so by changing the event IDs (and not re-using any), we force existing automation to be updated rather than just misinterpreting events when the automation doesn't know the version of Windows that produced the event. We realized it would be painful but it is nowhere near as painful as if every event consumer had to be aware of, and have special casing for, pre-Vista events and post-Vista events with the same IDs but different schema.

So if you happen to know the pre-Vista security events, then you can quickly translate your existing knowledge to Vista by adding 4000, adding 100, and subtracting 4. You can do this in your head.

However if you're trying to implement some automation, you should avoid trying to make a chart with "<Vista" and ">=Vista" columns of event ID numbers, because this will likely result in mis-parsing one set of events, and because you'll find it frustrating that there is not a 1:1 mapping (and in some cases no mapping at all).

Eric

WEvtUtil Scripting

Eric Fitzgerald — Wed, 16 Jul 2008 19:13:00 GMT

If you haven't used wevtutil.exe to script event log tasks in Windows Vista or Windows Server 2008, you're missing out. The new tool makes getting events out of the log pretty easy, but the main thing is that it doesn't suffer from any of the drawbacks around getting field delimiting correct.

The tool's command to query events from a log is "qe", and takes a log name as a parameter.

If you want to specify a query expression, then you can use XPath with the /q switch. The easiest way to do this is to use Event Viewer to build a filter for just the events that you want, and then copy just the XPath expression out of the XML tab of the filter dialog in Event Viewer. Be careful to copy only the filter expression and not the XML that surrounds it.

Finally, the default output format of wevtutil is XML. However it dumps each event as XML, but does not include a root element- in other words it's not well-formed XML by default. To include a root element you need to include the /e switch and a root element name.

I put this all together in a batch file, with an example XPath filter that just gathers interactive logon events (event ID=4624, logon type=2). You can save this as a .cmd file and run it as an administrator on Vista or WS08 and it will pull up a list of your interactive logons in Internet Explorer (or your default XML handler application if you've changed the registration). It has to run as admin because it accesses the security event log.

If you're really good (better than me, which is not hard) you could write an XSL style sheet and put this into a report format.

Good luck!

@echo off

set outputfile=%temp%\interactive-logon-events.xml

if "%1" NEQ "" set outputfile=%1

REM The next command is all one line and has no carriage returns

REM The only spaces in the XPath are around the AND keywords

wevtutil qe Security /q:"*[System[Provider[@Name='Microsoft-Windows-Security-Auditing'] and Task=12544 and (EventID=4624)] and EventData[Data[@Name='LogonType']='2']]" /e:Events > %outputfile%

start %outputfile%

set outputfile=

Windows Server 2008 Security Events Posted

Eric Fitzgerald — Thu, 17 Apr 2008 09:09:00 GMT

Fadi, Ned and Brian of the auditing team have documented all the auditing events by audit policy category and subcategory for your reference.

Check it out in the Knowledge Base.

Even better, they documented all the events in spreadsheet format, and that's propagating to the Microsoft Download Center. I'll publish the link when it's online.

2008-04-17 UPDATE: Brian just sent me the link: here is the spreadsheet.

Shameless Self-Promotion

Eric Fitzgerald — Wed, 05 Mar 2008 23:47:00 GMT

There's one topic that I know is on everyone's mind- no, not American Idol- it's "What's new in Auditing in Windows Server 2008?"

Well, funny that you brought that up. My friend Jesper Johanssen just wrote a new book, the Windows Server 2008 Security Resource Kit, and he invited me to write a chapter about auditing for it, which I did. So you, dear reader, are getting information straight from the horse's mouth, so to speak.

Anyway I think the book hits store shelves on March the 10th. A number of distinguished individuals contributed to the book: Susan Bradley, Darren Canavor, Kurt Dillard, Roger Grimes, Brian Komar, Alun Jones and others.

I'd also like to send out special props to my auditing posse: Raghu (who was the primary developer for auditing for Vista & WS08) and Ned (who is the resident guru for auditing in Microsoft Customer Support Services), both of whom made significant contributions. Raghu introduces the new "special group logon tracking" feature, and Ned contributed a spreadsheet mapping all the events (360-ish) to the policy category and subcategory and giving other key information about each event; this is included on the CD bundled with the book, along with an XML file defining the schema for all the events and event messages. Ned's also working on getting a version of the spreadsheet available for download from the Microsoft download site.

In other news, the Windows Server 2008 Security Guide is also out, and yes, yours truly contributed in small part to the auditing guidance in there too, although I seem to have been overlooked in the credits (in all fairness my work delta from the Vista Security Guide was really small so maybe it did not meet their "credits bar").

Anyway, download the security guide and buy a copy of the book. Buy more than one copy of the book, and give copies to your friends and loved ones. Nothing says "Happy Anniversary, Honey" quite like a book or white paper about computer security. OK, so maybe I should stick to computer security and stay away from relationship advice. Flowers work well in my experience.

ACS Event Transformation Demystified

Eric Fitzgerald — Thu, 28 Feb 2008 04:43:00 GMT

I've decided to start dumping my knowledge of ACS for posterity's sake. My first installment is here, and it's an excerpt from an external email I put together which describes how event transformation works on ACS.

Transformation is performed on the agent (using instructions provided at connect time by the collector) and on the collector. Transformation instructions are all stored on the collector in a file called EventSchema.xml which is in the AdtServer directory (%windir%\system32\security\adtserver). This file is pointed to in the collector’s registry and is read during startup of the collector service; failure to successfully read and parse this file at startup is a fatal error for the collector (the debug log will complain about parsing).

The collector reads EventSchema.xml and builds in-memory binary tables of event transformation instructions and event string types by OS version/event log/event source.

The collector (as explained elsewhere) also reads AcsConfig.xml to get its persistent state and configuration for all known agents, to know what logs/sources to collect for each agent/agent group, etc. This is all read into in-memory state for each agent.

At connect time, the agent sends version information- what the OS and agent version and service pack are, etc. The collector first looks in its in-memory agent state to see what configuration applies to the agent. Then it looks in its transformation tables and extracts the appropriate version-specific transformation instructions for the events that the collector is configured to collect from that agent. Then it packages these instructions and sends them to the agent.

The agent starts reading events, transforming them according to its instructions from the collector, and sending the transformed events to the collector. The collector finishes the transformation, services real-time subscriptions and loads the events into the database as appropriate.

If the agent encounters an event that is it configured to send (by log/source) but does not have transformation instructions for, then it simply builds a copy the event string for string and sends the copy of the event to the collector as an “unschematized” event. The collector will handle this event without problems but will not extract non-header user fields (no primary/client/target user fields) and will not add string type information.

I’ll take Windows Server 2003 (build 3790), Event Log: Security, Event Source: Security, Event ID: 644 as an example.

Here’s the WS03 schema for 644 (excerpt from %systemroot%\system32\security\adtserver\EventSchema.xml in the path “Schema\Log[@Name=’Security’\Source[@Name=’Security’]\Version[@MinBuild=’3790’]\Event[@SourceId=’644’]”).

</Event>

The instructions are all applied in order. “Call” instructions are executed agent-side; “Param” instructions are executed server-side.

These instructions can be translated as:

· Take string 1 from the original event and make it string 1 in the new event. It is of type “typeUserDn”.

· Take string 3 from the original event and make it string 2 in the new event. It is of type “typeComputerName”. Note that we are doing reordering here by appending original string #3 before original string #2. Nifty, eh?

· Take string 2 from the original event and make it string 3 in the new event. It is of type “typeTargetSid”.

· Take string 4 from the original event and make it string 4 in the new event. It is of type “typeClientUser”.

· Take string 5 from the original event and make it string 5 in the new event. It is of type “typeClientDomain”.

· Take string 6 from the original event and make it string 6 in the new event. It is of type “typeClientLogonId”.

· Take string 4 from the original event and treat is as a user name, and take string 5 from the original event and treat it as a domain name, look up the associated SID and make it string 7 in the new event. The new string is of type “typeClientSid”.

· Take string 3 from the new event, treat it as a SID, look up the user/domain name associated with it and append the user name as string 8 to the new event and the domain name as string 9 to the new event. String 8 is of type “typeTargetUser” and String 9 is of type “typeTargetDomain”.

See the reordering? Now here is an instance of the event with the original event data. If you’re not familiar with the XML, it’s the XML output of Crimson, the new eventlog service introduced in Vista/WS08, but this is a WS03 [pre-Crimson] machine; we're looking at a saved event log (evt) file.

<Channel>C:\Users\ericf\AppData\Local\Temp\SERVER34_SecEvts.evt</Channel>

<Computer>SERVER34</Computer>

</System>

<Data>user09</Data> // String 1 – user name

<Data>SERVER34</Data> // String 2 – looks like a machine name, confirmed by string 4

<Data>%{S-1-5-21-5998314728-109421381-169156293-611111}</Data> // String 3 – definitely a SID

<Data>SERVER34$</Data> // String 4 – definitely an account name (machine account)

<Data>CONTOSO</Data> // String 5 – looks like a domain name

<Data>(0x0,0x3E7)</Data> // String 6 – definitely a logon ID

<Data>-</Data> // String 7 – empty null string at the end of the event (ignored by ACS)

</EventData>

</Event>

When the event arrives at the collector, type information is applied, and then the user fields (typePrimary*, typeClient*, typeTarget*) are extracted from the string data section and the strings that are left are re-numbered starting at 1 (no reordering occurs).

Here’s a chart of what the event looks like at the various points in the system. The changes at each step are shown in red.

Original Event in Event Log		Client-Side Transformation at Agent		Server-Side Normalization (WMI/SQL output)
Field	Content Description (implicit)	Field	Content Description (implicit)	Field	Content Description (explicit)
		Client User		Client User	typeClientUser
		Client Domain		Client Domain	typeClientDomain
		Client Sid		Client Sid	typeClientSid
		Client Login Id		Client Login Id	typeClientLogonId
		Target User		Target User	typeTargetUser
		Target Domain		Target Domain	typeTargetDomain
		Target Sid		Target Sid	typeTargetSid
String01	typeUserDn	String01	typeUserDn	String01	typeUserDn
String02	typeTargetSid	String02	typeComputerName	String02	typeComputerName
String03	typeComputerName	String03	typeTargetSid	String03
String04	typeClientUser	String04	typeClientUser	String04
String05	typeClientDomain	String05	typeClientDomain	String05
String06	typeClientLogonId	String06	typeClientLogonId	String06
String07		String07	typeClientSid	String07
String08		String08	typeTargetUser	String08
String09		String09	typeTargetDomain	String09

To finish off a description of transformation, there are 7 transformation functions, each of which can optionally take 2 integers as parameters. Note that there is no “destination event” field specifier; all references are only to the original event. That’s because when constructing the destination event, any data added to the event is always appended- it is constructed from beginning to end- so the implicit destination field is “at the end of the event as it is now”.

Function	Parameter 1	Parameter 2	Description
AppendString	Reference to a string parameter in the source event in the event log	Unused	Appends the referenced string to the event which will be sent to the collector
AppendStringFromTable	Reference to a constant string in the statically defined <Strings> table (1-based) in the relevant Source\Version element in EventSchema.xml	Unused	Appends the referenced constant string to the event which will be sent to the collector
AppendProcessNameFromPid	Reference to a string parameter in the source event in the event log (source string is expected to be a numeric process ID)	Unused	Looks up the process image path name for the referenced PID and appends it to the event which will be sent to the collector
AppendTimeFromDatetime	Unused	Unused	Not Implemented/No Action
AppendSidFromNames	Reference to a string parameter in the source event in the event log (source string is expected to be a user name)	Reference to a string parameter in the source event in the event log (source string is expected to be a domain name)	Looks up the SID for the account represented by the specified user and domain names, and appends the SID to the event which will be sent to the collector
AppendNamesFromSid	Reference to a string parameter in the source event in the event log (source string is expected to be a security ID)	Unused	Looks up the user name and domain name for the account represented by the specified SID, and appends the user name and the domain name as separate strings to the event which will be sent to the collector
AppendNumber	Unused	Unused	Not Implemented/No Action

Out of range params cause the transformation instruction to be ignored and skipped. Non-integer params or other XML formatting/malformation problem (including non-UTF8 formatting) cause an EventSchema.xml parsing error at collector startup which in turn causes collector startup failure.

So that’s ACS transformation in a nutshell. I hope this helps you guys understand ACS functionality a little better.

Shortly I will finish my write-up on AcsConfig.xml but that is a simple file and not too hard to figure out if you are into experimentation.

Here are some cool things that you can try with the event schema file if you are adventurous:

1. Drop fields. We have modified eventschema.xml successfully to cause it not to collect certain fields (e.g. logon GUIDs) of certain events:

// try deleting a line here

// or, to preserve ordering of subsequent strings

// try replacing “AppendString” with “AppendStringFromTable (param1=1)”

2. Add an event source. Some caveats are:

· You must have a unique, well-formed GUID for the new source

· You have to get events of the new source into the log (try “AuthzReportSecurityEvent” from MSDN)

· You have to modify AcsConfig.xml to tell the agent(s) to collect the new source

NB I have used the C/C++ comment syntax throughout this post but note that ACS does not support either C/C++ nor XML style comments in the XML config files it uses

ACS Tidbits

Eric Fitzgerald — Sat, 02 Feb 2008 04:04:00 GMT

Well there has been a lot happening on my old project, ACS (Audit Collection Services, a feature of SystemCenter Operations Manager 2007).

Two more of our partners, Enterprise Certified and NetPro, have released compliance solutions on top of ACS.

Another of our partners with ACS-based compliance solutions, SecureVantage, has started a new blog where ACS is a frequent topic.

Anyway I'm pleased to see that ACS is becoming a successful platform and I'm happy to answer ACS questions! To you ISV's out there, Joseph and I welcome your questions as well (if we aren't already talking to you). Let us know who you are so we can stay in touch with you!

List of Windows Server 2003 Events

Eric Fitzgerald — Fri, 12 Oct 2007 21:45:00 GMT

So a long time ago, back in my days of providing technical support for Windows NT 4.0, I published "Security Event Descriptions". This article was the "schema" so to speak, for the Windows NT 4.0 security event log events.

Technically Windows events are not schematized until Windows Vista; or put another way the schema is implicit based on the instrumentation in the code- since the event is raised by some function in the code, the "schema" could be interpreted as the parameter order in the call to that function.

Anyway security monitoring types love that article, but I hate it. It's just better than nothing. It doesn't state which events map to which audit policy categories. It does tell you whether the event is a succss or failure event but it doesn't alert you to the cases where the same event is used for success and failure (e.g. event 560).

When Windows 2000 came around and we added two new audit policy categories (DS Access and Account Logon [which was a huge naming blunder]), I wrote an article for the Windows 2000 security events. However it was so large I broke it into two articles.

I didn't write an article for Windows Server 2003. At first I didn't think it was necessary because we propagated all the WS03 events to the Technet Events & Errors Message Center web site. I wrote custom content for the top 30 or so events by volume of searches

(On a side note, did you ever wonder what happens when you click the "More Information" link at the bottom of the Event Viewer event description? We send the event source, event ID, OS version and so forth to the Technet E&E site and display the content that is returned. We count the number of hits for each OS Version/Source/Event ID combination and then our writing teams pester the component owners to populate that content.)

Anyway, I was making excu^h^h er, explaining why I didn't write the KB articles for Windows Server 2003 security events. So I thought the E&E message center would be all that anyone needed. It didn't strike me as that important that you had to have seen the event (or at least know it exists) before you could use the site. However since then I have received a large number of requests for the event definitions, mainly from people who were creating security event management solutions.

So here's what I have for you, courtesy of Ned, one of the audit log posse here at Microsoft. If you want a complete list of WS03 security events, then I suggest you look at chapter 4 of the Windows Server 2003 Security Guide. This documents the event IDs of all the security events on Windows Server 2003. Plus, it groups them by policy category, in case you ever wanted to know what you are in for if you enable one of the categories for audit. If you want the layout of the event (what data is in the description field, and in what order) then just look for that specific event on the Technet E&E site or click the link in the bottom of the event description in Event Viewer.

I've already described how the Vista and Windows Server 2008 (and subsequent releases) event systems are self-documenting, so I won't go into that further here.

One last tip: If you own Microsoft System Center Operations Manager 2007, then you can search for a file called EventSchema.xml on the media. It is an XML document that describes one possible normalization all the security events from Windows 2000 forward, and the semantic content of the normalized events.

2007-10-31 UPDATE: There is also an event-id-to-audit-policy-category map here.

Help! Someone has deleted events from my Windows event log!

Eric Fitzgerald — Sat, 11 Aug 2007 01:59:00 GMT

From time to time I hear this, and it usually turns out not to be the case.

I'll begin with a little background.

First, The eventlog service does not have (and never did have) any public or private API to delete individual events- there is a log clear API but nothing else. The eventlog team thought about implementing selective delete for Vista (there were some internal groups asking for it) but a lot of us security types yelled at them and nothing came of it. Logs are logs, not databases- if you want selective delete, export the events you want to a database and have at it.

Second, there is no getting around the 10 Immutable Laws of Security, particularly law #6 (a computer is only as secure as the administrator is trustworthy) and law #2 (if a bad guy can alter the operating system on your computer, it's not your computer anymore).

What this means is that, no matter if we implemented the most advanced, coolest, 1337est event-signing/real-time-exporting/writing-to-optical-media-and-a-line-printer-too event system, IT DOESN'T MATTER IF THE ATTACKER IS THE ADMINISTRATOR- all we do is reduce the window of time that that person has to do his dirty work. Now I will admit there is value in those features, but if such an evil person were to use his powers of debugging to open the services.exe process (where eventlog lives) and inject a thread which alters the eventlog data structures in memory in real time, prior to commit to disk, then none of that stuff would help us. As a matter of fact such tools exist, they are not theoretical. I haven't seen the Vista versions yet but there's no technical reason why such a tool could not be built for Vista.

However, the cases I've seen of apparent gaps in event logs have a much more mundane explanation: the "Retain X days" event retention policy. This is an evil setting; if people truly understood it they wouldn't use it. Prepare to truly understand it.

Imagine you have a finite space S to store resources of type R. You get a constant incoming stream of R's, and put each of them into S. Now imagine that S is full and a new R arrives. You have two choices:

1.       Throw the new R away.
2.       Remove one or more of the old R's from S (enough so that the new R can fit into S) and put the new R into S.

When selecting which old R's to discard, the generally accepted best practice is that you should throw away the oldest R first- in other words freshness is a priority. Of course if you wanted to optimize for space you could just pick the smallest old R equal to or larger than the new R, but that would cause ordering problems if you wanted to maintain sequential access. You could even pick one or more old R's at random but that would be too arbitrary for most structured purposes like logging.

Now imagine that you had an additional constraint: you can throw away old R's, but only if they're more than X days old.

Now your choices are:

1.       Throw the new R away, or
2.       Throw away one or more of the old R's, if and only if there are enough R's that are older than X.

If there are no R's older than X, you may not discard any old R's and since you have a fixed size buffer S and there is no room for the new R, you MUST choose option 1- throw away the new R. You have no other choices.

This is the situation with event log. "Retain X days" actually CAUSES event loss (as does "Overwrite as needed"). However, Overwrite as needed causes predictable event loss (oldest events gone). Retain X days causes unpredictable event loss (if the log is full and there are no events older than X days, then NEW events are thrown away until there are some events older than X days).

Detecting gaps in your log

Can we detect if someone has deleted events out of your log?

At the end of the day, the event log is an ordered list of data structures (called event records), with each having a pointer to the next. There is also a unique, monotonically increasing sequence number associated with each event record.

A clever attacker will have disabled the instrumentation that causes the event to be raised; eventlog will never have been involved so there will be no gap (but no event).

A less clever or less resourceful attacker who deletes an event from the log, probably will not go fix up the sequence numbers for the rest of the log (in fact the attacker might not even fix up the pointers, causing the eventlog service to crash or hang).

We can use this priciple to our advantage. By examining each event and looking at its sequence number, we can look for gaps in the stream and, although we can never be sure that there was no tampering, we can often tell when there was tampering.

Here's a VBScript that demonstrates this concept. I don't provide VBScript support; you're on your own on this one.

'EventLogGapDetector.vbs

' (c) 2007 Microsoft Corporation, All Rights Reserved

'

strComputer = "."

Set objWMIService = GetObject("winmgmts:\\" & strComputer & "\root\CIMV2")

Set colItems = objWMIService.ExecQuery( _

    "SELECT * FROM Win32_NTLogEvent ",,48)

iPrev = 0

first = true

gapdetected = false

newgapdetected = false

currlogfile = ""

oldlogfile = ""

For Each objItem in colItems

    iCurrent = CInt(objItem.RecordNumber)

    currlogfile = objItem.Logfile

    if ((iCurrent <> (iPrev-1)) and (not (first)) and currlogfile=oldlogfile) then newgapdetected = true

    if (newgapdetected) then Wscript.Echo "Gap detected, log file = " & currlogfile & ", last record = " & iPrev & ", current record = " & objItem.RecordNumber

    if (newgapdetected) then gapdetected = true

   iPrev = CInt(objItem.RecordNumber)

    first = false

    newgapdetected = false

    oldlogfile = currlogfile

Next

if not (gapdetected) then Wscript.Echo "No gaps detected."

Eric

The information provided in this post is provided "AS-IS" with no warranty, and confers no rights.

Documentation on the Windows Vista and Windows Server 2008 Security Events

Eric Fitzgerald — Wed, 01 Aug 2007 00:36:00 GMT

I'm hearing lots of complaints that we don't have KB articles on these yet. Doriansoft has a blog post complaining that the "add 4096" rule doesn't work because we collapsed the logon events into a single success event and failure event (from 2 success events [528, 540] and 10 failure events [529-537, 539]).

Well, In Vista and beyond the event log is self-documenting. From an elevated command prompt (one with admin privileges), type the following:

wevtutil gp Microsoft-Windows-Security-Auditing /ge /gm:true

This example dumps only the 360 or so unique security event messages (publisher=Microsoft-Windows-Security-Auditing); other publishers can be enumerated with the ep switch of wevtutil.

Event messages can be formatted as XML using the /f switch, see the command-line help.

As a side note, this is, in slightly different format, the same information we publish in the KB, and a KB article is in the works.

Why did we renumber the events? As explained in my earlier post, we changed the internal detail of each event so much (to improve understandability, readability, consistency, etc.) that we would have broken essentially all existing automation anyway. By renumbering the events we made the automation break in as obvious a way as possible, and also made it as clear as possible that THESE ARE DIFFERENT EVENTS.

The "add 4096" rule is not meant to imply that the events are the same, but rather allows you to find the new equivalent event, if you have knowledge of the old event. Simply renumbering your automation will not make it work. It's a mental aid for you, the Windows security professional.

[2007-10-12 Update: changed tags]

Where do I get my information on Windows auditing?

Eric Fitzgerald — Wed, 07 Feb 2007 01:12:00 GMT

You might want to know where I go to get my information on audit events and so forth.

Mostly I go to the source code or one of our developers. For continuity-of-employment reasons I won't be posting a link to that here ;-) We have some old specs and some new specs but sometimes the code doesn't function quite like the spec says it should so I usually go to the code instead of the spec.

However you can download the Windows Platform SDK for free, and it includes all the header files which define all the Windows error codes and so forth.

I also search my email with Outlook 2007's Instant Search feature (yes that was a blatant plug). A lot of times I find that I answered a question or had a discussion about something a long time ago. This has become less useful lately as the legal guys are making us delete all our old email. I am trying to capture some of this kind of content in this blog since hopefully it will outlive my email :-) Maybe I will write a book.

I use the Technet Events & Errors web site. This is the site where Event Viewer goes for content when you click the link in the bottom of an event. Not all events are populated (actually only a small percentage have hand-written content).

I use Windows Live Search or sometimes that other search engine to search the internet for content, but only rarely. Sometimes I go to Randy Smith's web site, Ultimate Window Security.

Anyway these are all my primary sources.

2007-02-08 Updated Platform SDK download link per PSDK group advice

What is up with Audit Collection Services?

Eric Fitzgerald — Wed, 09 Nov 2005 23:37:00 GMT

A lot of you have been asking me to write about Audit Collection Services (ACS, which some of you might know as MACS).

For those of you unfamiliar with ACS, it's a client-server application to collect, normalize and store large volumes of security event log data from large numbers of machines, and to make the normalized data easily available for near-real-time analysis via WMI or after-the-fact analysis via SQL queries. (A single ACS collector was designed to collect in near-real-time from up to 20,000 concurrent machines, and handle volumes in excess of 2,000 events per second sustained). We use 8 ACS collectors in production here at Microsoft, storing a quarter billion events per day, and often collecting and processing many times that number of events.

The project was started in 2001 in the Windows Core Security group here at Microsoft. We finished what we intended to build last year, but during the time it took us to build it there were a number of external changes which affected the project- changes in Windows management and organization, and the rise of web services.

It took us a while to sort out what to do with ACS in light of these changes. In the end we decided that it fit better with our Operations Manager product (MOM) than with Windows where we originally developed it. My team is working with them to include the ACS code in the next version of MOM, and to keep all of our ACS scenarios intact while gaining the advantages that MOM provides such as data warehousing and reporting.

We are also making a change to the ACS protocol to allow convergence of our different event collection technologies in the future. The protocol is web-services based but is not textual XML over HTTP. We'll retain the tight, stingy bandwidth use that you've come to expect from ACS, but all of our technologies will interoperate in the future.

So now the FAQ:
Q1: How can I get ACS?
A1: You can't. Please don't ask. The beta program is not accepting new testers at this time although we will continue to work with our existing testers.

Q2: When can I get ACS?
A2: When the next version of MOM ships, but I don't know the date. ACS integration will be for the beta 2 release in the spring.

Q3: How much will it cost?
A3: Licensing terms haven't been set yet.

Please understand that I get a LOT of questions about ACS- sometimes a dozen or more per day- and that I don't have time to answer individual questions about the project at this time. I'm working closely with the MOM team and I promise that we'll publish new information when the time is right. ACS is not dead :-)

Best regards,
Eric

Managed Code Developers: You no longer have an excuse!

Eric Fitzgerald — Fri, 30 Sep 2005 20:51:00 GMT

One of my former teammates, Mark, designed and built a set of managed classes for generating audit from .NET applications (for example, consider a web service). His work is published in the latest issue of MSDN magazine.

A lot of people aren't aware of this, but in Windows Server 2003 we added an API set that allowed application developers to generate security audit.

Of course, we've always had support for generating your own object access audit, but now we have a generic audit facility that's usable for other kinds of events.

For non-developers, when you're talking to application vendors about their products, just as you would demand integration with Windows' authentication for single sign on, you should consider demanding integration with Windows' audit facility for single point of monitoring.

Eric

Yay! A fix for EventQuery

Eric Fitzgerald — Wed, 28 Sep 2005 03:17:00 GMT

Those of us "in the know" :-) use eventquery.vbs to export events to a delimited file, and then use Excel to analyze the log- autofiltering rocks. Unfortunately if you have a large log, this doesn't work!

Well, I finally used MSN Search to see if there was a KB article on this, and I found this post to the microsoft.public.windows.server.general newsgroup. Remember that you might have to disable Windows File Protection.

Yay! Now I just need to make myself a to-do to get that fixed.

2005/09/28 UPDATE: I have requested that this issue be fixed in EventQuery.vbs in XP SP3. Also, Jean-Baptiste pointed out that EventQuery.vbs is over-localized. I've also requested a change for this in XP SP3 but changing it at this point might do as much harm as good. This might not be an issue in Vista as the event log engine and tools have been re-written from scratch.

2006/02/01 UPDATE: Brandon has done some scripting with EventQuery and figured out how to run daily reports. Here's a short script he wrote that reports errors in your application & system logs (I've made only minor changes). Just paste this into notepad and save it as "DailyErrorReport.cmd".

2006/08/23 UPDATE: Fixed a typo that Paul pointed out. Line 8 (set LongYear...) should only have one equal sign, not two as it originally had.

@echo off

REM Get today's date
set today=
set month=%date:~4,2%
set datenum=%date:~7,2%
set year=%date:~12,2%
set longyear=%date:~10,4%

REM Start query at midnight
set today=%month%/%datenum%/%year%,12:00:00AM

REM This could be scheduled to run daily

REM Execute query and store results in a daily log file
eventquery /l system /l application /v /fo table /fi "Type eq Error" /fi "Datetime gt %today%" >> %temp%\errors_%longyear%%month%%datenum%.log