Welcome to MSDN Blogs Sign in | Join | Help

I'm sorry I haven't written anything in a while.

For those who don't know me, I have transitioned to work on NTFS proper.  It's fun, technical work, but it's also maintenance, something which just seems less exciting for a public audience.  As a result, I don't have quite so much to talk about.

However, I feel the need to write today.  I've just read an article on Linux-watch - available here - which, among other points, mentions "...Vista is based on a single user PC operating system."  To be fair, many people over the years have raised this argument, and I feel I should articulate a response.

Many of us will remember Windows 3.1 (and its predecessors) which were inherently single-user.  Many of us will remember Windows 95 and 98, which had some level of support for multiple users.  Some of us will also remember that "other" version of Windows, Windows NT.

Windows NT is multi-user, and always has been.  If a user attempts to open or create a file, NTFS is told as part of the same operation information about that user and what rights they have.  There's no concept of the "current user" or anything similar.  We can even process the open or create on a thread outside of the process that requested it - so the "user of the running process" is also somewhat meaningless at this level.

Historically, NT supported only a single user to be logged in interactively with the system.  Background processes could always run in other user contexts.  As of NT 4 Terminal Server Edition, this was replaced by multiple interactive users (on different sessions).  As of Windows 2000, the "runas" tool allowed the same desktop to host applications in different user contexts.  As of Vista, UAC allows the same desktop to host applications in the same user context with different rights.  However, NTFS always supported the idea that an operation could be requested by any user, and has not had to change to accommodate these developments.

While the "Windows" brand has referred to single-user operating systems, Vista is not derived from one; nor was XP.

Many of the more technical readers out there will point out specific technologies which were required to support Windows 3.1/95/98 compatibility, where multi-user support was added later.  A good example is DDE, where applications on the same session can communicate using broadcast window messages.  This facility was added in Windows 3.0, and did not have multi-user scenarios in mind.  However, even here, a Terminal Server machine can have multiple simultaneous sessions, with different user contexts, and still maintain compatibility with DDE.  While opinions may vary on the DDE design, the fact that this works is an achievement for compatibility; in itself, it does not reduce the multi-user capabilities of the underlying operating system.

Linux supports per-process user contexts.  So does NT.  Linux supports multiple concurrent X sessions.  NT supports multiple concurrent windows sessions.  Linux has support for permissions - and now Access Control Lists (ACLs) - in its filesystems.  NT has supported ACLs from 3.1.  Linux supports X applications on the same desktop running in different user contexts.  NT supports applications on the same desktop running in different user contexts.

The transition from Windows 98/ME to Windows 2000/XP was only a small jump for users, but it was a very large jump for technology.  If many users didn't realize that a simple upgrade moved them from a single-user to a multi-user operating system, that's an amazing accomplishment in terms of compatibility.  However, having made the switch, the "Windows" of today cannot really be claimed to be single-user.

 

One of the things on my list at the moment is to implement a field for how many milliseconds have passed since something happened.  It sounds so simple, but from experience, dealing with time always makes me shudder.  Consider the properties of electronic time:
  1. Time is finite.  It doesn't matter what field size you choose, eventually, it will run out.  Sometimes you can wrap around to zero without pain; other times, that breaks things.
  2. Time is bidirectional.  Some of this comes from network time resynchronisation, and occasionally, hardware limitations.  Time can go backwards.  If you want to know how long it's been since an event, and poll twice in succession, the second can come before the first.  It also follows that time from any given event may be negative.
  3. Time is unstable.  If you're using the current clock at all in calculations, and it changes, it could change by years at a time.  To avoid this, you either can't use the current clock, or have to tolerate some really odd calculations.
  4. If an error occurs, time will remain in error forever.  Consider computers with flat CMOS batteries and odd times in the past.  Many time based calculations will follow the error paths, because the time being compared against won't happen for decades.  This can be anything from a file created in the future, an audit log entry for something that hasn't yet happened, unzipping files and creating files that won't be created into the future, an encyption certificate that won't be issued for years, or dependency-based build process.  It gets worse if any new data is created using that bogus date: consider how your audit log looks when it contains 2005 data followed by 1980 data followed by 2005 data again...
I'm sure time has other odd properties too.  When Einstein announced his belief that time was not absolute, the reaction was mixed; in electronic terms, time is not only not absolute, but tends towards being random.  Software can only compensate so much.

Today I read a newsgroup post by an academic at my former university, who I know reasonably well, and who I used to work for.  In that post, he talks about things that Microsoft has done to him, and how he views Microsoft today; the fact that the subject he is lecturing and I used to tutor is a Linux subject should be a good indication of those views :).

The funny thing is, I read through it, and can't argue with it.  Although I disagree with his interpretation of what's happening at the moment (discontinuation of UNIX/Linux support for new sales of Sybari products), his previous experience dealing with Microsoft is all too familiar.  Trouble getting solid documentation, not adhering to standards properly, and of course, the legal challenge brought by the US government.

I remember my first Windows SDK shipped with a coupon for the DDK; that coupon had a space where you had to write down why you wanted a copy.  This may have been just for statistics, but the idea of justifying why you need low-level product documentation seems a little strange.

On Tuesday Steve Ballmer was speaking to a bunch of us introducing the Engineering Excellence/Trustworthy Computing Forum.  To summarise one of his points, Microsoft can't deliver trustworthy computing to customers who don't trust us; computers today will store and communicate the most intimate details of our lives, our financial information, and data related to our identity.  When Microsoft talks about security, everyone needs to be able to believe that we're doing it with customer interests in mind - if customers don't believe we're serious about security, they'll switch to another product, not regularly apply patches, and not follow procedures designed to minimise risk.  To deliver Trustworthy computing, Microsoft needs to adhere to the strictest business practice guidelines to foster trust and respect.

These two things came together in my head.  Sure, when I bought my first SDK it retailed for around $1,000.  Today, that documentation is freely available on the web, as are the tools.  Compilers and linkers used to cost that much too.  Now, they're a free download.  So are debuggers, and debugging symbols. The DDK is available for the cost of shipping.  If documentation for a particular Windows API or topic isn't public, then the other teams at Microsoft aren't allowed to use it either.  Required by settlement, enforced at Microsoft.  That helps provide great feedback for platform features, so that everyone else can benefit too.

Today Microsoft leverages standards in nearly all of our products.  The recent announcement of Office moving to an XML-based file format should serve as an example, although there are many others.  NFS support is another free download.  Standards deliver the best experience to our customers, allow them to leverage existing tools, and deploy new Microsoft solutions.  That provides great opportunities for Microsoft into new markets, and enables a smooth transition for our customers.  It's win-win.

A lot of things have changed since the Antitrust case.  It's visible looking around campus culture; culture is not an easy thing to change with such a large group of people.  But here I am talking about these things, with a link so that anybody can contact me, ask me questions, ask for a feature, or anything.  I read all of the posts here, and all of the comments I get.  I follow them up.  I've dug into source code to find answers for people multiple times - not because they pay me, but because Microsoft now understands the benefit of participating in the community to achieve great results.

Maybe we have the government to thank, in a twisted kind of way.  Or maybe the open source community has shown the value of software as a community activity.  Or maybe Steve's right - for Microsoft to grow into new markets, customers have to trust us more than ever, which demands ever more openness, honesty, and accountability.

Great software is the beginning, but a great user experience is what ultimately matters.
I confess: I'm breaking the first rule of blogging and talking about something I know nothing - literally, nothing - about.

When I first started this blog, lots of people were asking about Transactional Registry support.  I guess somebody in MS agreed, because that project has been under development for some time now.  It integrates with Transactional NTFS via KTM - the Kernel Transaction Manager - so that a single transaction can encompass both file and registry operations, and commit or rollback those changes together.

This is primarily designed to provide atomicity to application installs and updates on Longhorn; as always, creative uses are welcome and encouraged.

Update: Oh, I wanted to mention this.  What this means is NTFS and Registry calls both understand the concept of Transactions.  Who knows what will tomorrow? It's a good idea to remain in a transaction for as short a time as possible; switch to a transaction, do your work, and switch out again.  You never know, somebody might want to read your setup log.  And you don't want to have your newly-transacted-UI in 2020 rollback on you if the install fails...right?

If you have some really cutting questions you'd like to ask about this, shoot.  Bear in mind I have no idea how to answer them, so if they're really good, I'll chase up an answer for you.  Happy coding :)

At the moment, I'm finding it really hard to concentrate on work, so I thought I'd share my thoughts with you.

Readers of this blog will recall that I have an iMac, that I've done a fair bit of Apple development prior to moving to MS, and those who know me often find amusement in my insistance of wearing Apple tshirts to work.

While this may be thought of as the "bombshell that wasn't" - rumors have been circulating literally for months - the implications of this are pretty intense.  A few minutes ago something snapped in my mind; I'd been wondering all weekend, "Does this mean Apple now see themselves as a software company? Is Microsoft facing a competitor in the platform space?" Well, the answer hit me like this: even if Apple wanted to prevent people running OS X on PCs, it would likely be almost impossible.  Sure, there might be some hardware differences, some odd devices, maybe even a completely new BIOS.  But how hard is it - really - to emulate those things and run at near-native speed?  How hard is it for hardware vendors to be compatible? Or OEMs?

Apple must be aware of this.  The only question is whether they intend to be behind it - making money from it - or make it difficult and lose that revenue to software pirates.  Either way, it will happen; and faced with this choice, the first seems likely.  When these rumors started, they started by claiming that a major OEM had approached Apple and asked for this.

Perhaps this is all a good thing.  I mean, I'm not afraid of healthy competition or anything - that helps everyone make better products, and customers are the beneficiaries.  But I can't recall a time since OS/2 when the desktop client was really subject to intense competition.  How will MS adapt, culturally, to being concerned with what's over its shoulder?  Competition means that customer-focus has to become the primary focus; dropping the ball on what the market wants on any minor point can cause massive harm.

Perhaps that's what I'm getting at here: to all the MS people who read this, don't shrug off the threat that Apple poses.  They really do have some pretty cool stuff down there.  Lowering the transition cost to their OS (by eliminating hardware cost) will attract people.  Application vendors are attracted to platforms with users.  This is critical mass thinking.  I can't recall a time in the last decade where it has been so important for MS to focus on the needs of customers, and the needs of developers, to provide real platform leadership.

Bottom line: the standard just went up a notch. Time to step up.

Firstly, apologies for not updating this more frequently.  I haven't wanted to post about things until they're bug-free, just to be certain I'm not talking about something that won't work for you.

So far, I've shown transactional writers.  In these cases, modifications to files and directories aren't visible outside of a transaction until that transaction commits.  There is also a second class of transactions, transactional readers, that isolate changes made outside of a particular transaction from being visible.  This form of isolation is performed at the handle level: within a transaction, a handle opened for read will typically be isolated for the duration of that handle.

To illustrate, let's start with some code:
#include <windows.h>
#include <ktmw32.h>
#include <stdio.h>
#include <stdlib.h>
#include <tchar.h>
#include <strsafe.h>

LPTSTR szKeywords[10]={_T("cat"), _T("dog"), _T("rabbit"), _T("horse"), _T("cow"), _T("chicken"),
_T("bird"), _T("mouse"), _T("donkey"), _T("camel")};

#define NUM_WORDS 10

LPTSTR szFileName=_T("testfile");

void WriteRandomWord(HANDLE hFile)
{
BOOL bRet;
DWORD dwWord, dwWrote;
dwWord = rand()%NUM_WORDS;


SetFilePointer(hFile, 0, NULL, FILE_BEGIN);
bRet = WriteFile(hFile, szKeywords[dwWord], (_tcslen(szKeywords[dwWord])+1) * sizeof(TCHAR),
&dwWrote, NULL);
SetEndOfFile(hFile);

if (bRet)
_tprintf(_T("Wrote \"%s\" to file\n"), szKeywords[dwWord]);
}

void DisplayFileContents(HANDLE hFile, LPTSTR szDesc)
{
TCHAR szReadIn[20];
DWORD dwLen;
DWORD dwRead;
BOOL bRet;

SetFilePointer(hFile, 0, NULL, FILE_BEGIN);
dwLen = GetFileSize(hFile, NULL);
bRet = ReadFile(hFile, szReadIn, dwLen, &dwRead,
NULL);

if (bRet)
_tprintf(_T("Read \"%s\" from file (%s)\n"), szReadIn, szDesc);
}

int __cdecl _tmain(int argc, TCHAR ** argv)
{
HANDLE hWriterFile;
HANDLE hReaderFile;
HANDLE hNonTransReaderFile;
HANDLE hWriterTrans;
HANDLE hReaderTrans;
DWORD dwLoop;

srand(GetTickCount());


hReaderTrans = CreateTransaction(NULL, NULL, 0, 0, 0, 0, NULL);
if (hReaderTrans == NULL) {
_tprintf(_T("Could not create transaction\n"));
return EXIT_FAILURE;
}


hWriterFile = CreateFile(szFileName,
GENERIC_READ|GENERIC_WRITE,
FILE_SHARE_READ|FILE_SHARE_WRITE|FILE_SHARE_DELETE,
NULL,
CREATE_NEW,
FILE_ATTRIBUTE_NORMAL,
NULL);
if (hWriterFile == INVALID_HANDLE_VALUE) {
_tprintf(_T("Could not create test file\n"));
return EXIT_FAILURE;
}

WriteRandomWord(hWriterFile);

CloseHandle(hWriterFile);

hNonTransReaderFile = CreateFile(szFileName,
GENERIC_READ,
FILE_SHARE_READ|FILE_SHARE_WRITE|FILE_SHARE_DELETE,
NULL,
OPEN_EXISTING,
FILE_ATTRIBUTE_NORMAL,
NULL);

SetCurrentTransaction(hReaderTrans);

hReaderFile = CreateFile(szFileName,
GENERIC_READ,
FILE_SHARE_READ|FILE_SHARE_WRITE|FILE_SHARE_DELETE,
NULL,
OPEN_EXISTING,
FILE_ATTRIBUTE_NORMAL,
NULL);

for (dwLoop=0;dwLoop<10;dwLoop++) {
hWriterTrans = CreateTransaction(NULL, NULL, 0, 0, 0, 0, NULL);
if (hWriterTrans == NULL) {
_tprintf(_T("Could not create transaction\n"));
return EXIT_FAILURE;
}

SetCurrentTransaction(hWriterTrans);

hWriterFile = CreateFile(szFileName,
GENERIC_READ|GENERIC_WRITE,
FILE_SHARE_READ|FILE_SHARE_WRITE|FILE_SHARE_DELETE,
NULL,
OPEN_EXISTING,
FILE_ATTRIBUTE_NORMAL,
NULL);

WriteRandomWord(hWriterFile);
CloseHandle(hWriterFile);
CommitTransaction(hWriterTrans);
CloseHandle(hWriterTrans);
SetCurrentTransaction(NULL);
DisplayFileContents(hReaderFile, _T("transacted"));
DisplayFileContents(hNonTransReaderFile, _T("non-transacted"));
}
CloseHandle(hReaderFile);
CloseHandle(hNonTransReaderFile);
RollbackTransaction(hReaderTrans);
CloseHandle(hReaderTrans);
SetCurrentTransaction(NULL);
DeleteFile(szFileName);

return EXIT_SUCCESS;
}

Take note of a few things in this code: firstly, it has no error checking.  That's due to space constraints, not the absence of failure conditions.  I'm not advocating zero error checking in code.

More to the point, note that the reader handle is opened once, and closed once.  It remains open the entire time.  Also note that the writing transaction commits, to make its changes visible to other users.  To make the illustration clearer, I've added a traditional, non-transacted reader handle into the mix.  Both readers, and sometimes a writer, are open at the same time.

Here is some sample output from this application:
Wrote "cat" to file
Wrote "rabbit" to file
Read "cat" from file (transacted)
Read "rabbit" from file (non-transacted)
Wrote "mouse" to file
Read "cat" from file (transacted)
Read "mouse" from file (non-transacted)
Wrote "cat" to file
Read "cat" from file (transacted)
Read "cat" from file (non-transacted)
Wrote "cat" to file
Read "cat" from file (transacted)
Read "cat" from file (non-transacted)
Wrote "bird" to file
Read "cat" from file (transacted)
Read "bird" from file (non-transacted)
Wrote "bird" to file
Read "cat" from file (transacted)
Read "bird" from file (non-transacted)
Wrote "donkey" to file
Read "cat" from file (transacted)
Read "donkey" from file (non-transacted)
Wrote "cat" to file
Read "cat" from file (transacted)
Read "cat" from file (non-transacted)
Wrote "dog" to file
Read "cat" from file (transacted)
Read "dog" from file (non-transacted)
Wrote "chicken" to file
Read "cat" from file (transacted)
Read "chicken" from file (non-transacted)

While the view from the non-transacted reader handle changes with each modification, the transacted reader handle remains frozen at that point in time.  To get an updated view, a transacted reader simply needs to close the existing handle and open the file again.

Recently, The NT Insider published an article entitled Don't be Afraid to Commit -- The Transactional Filesystem (TxFS) in Windows. The article deserves mention for a number of reasons, mainly because it's a really good conceptual explanation of what we're working on.

There are some small nits that should probably also be explained.  If you see the term TxFS, or TxF, this term refers to the same feature: Transactional NTFS.  In this blog, I'm trying to use the latter term, just to be clear: this support is integrated into NTFS.  Transactions aren't supported on FAT32, and nor do you need a new filesystem to support this functionality.  In fact, we've done a lot of work to make sure the on-disk format doesn't change at all: when you boot into Longhorn, the necessary structures will be created; you don't need to reformat.

Seeing as I'm linking to this article from a Microsoft blog, I should also make one small correction: we changed our mind on namespace isolation.  Prior to commit, changes made to the namespace from within a transaction are only visible inside the transaction, and upon commit, those changes are visible to both non-transacted applications, and other transactions.  That is, if you delete a file transactionally and commit, other uncommitted transactions will see the delete; prior to commit, only the deleting transaction will see the change.

The article is very correct that Transactional NTFS has a huge impact on filter drivers.  If you're writing a filesystem filter driver, and we haven't heard from you at plugfests or similar events, we'd like to hear from you now.  In many cases the impact for filters is huge, and it'd be great to have filters ready for these changes in Longhorn; this will help users migrate to Longhorn, and help your customers from being inconvenienced in the process.
Some time ago my manager told me that I'd be one of three people maintaining the Windows builds used in our group, and this week is when I'm supposed to do that.

For those non-Microsofties, or at least, people outside the Windows group, Windows is a darned big codebase.  Building it takes time, so builds are scheduled overnight on some amazingly expensive hardware; and being so large, even if checkins are individually tested for buildability, build breaks still happen if two changes aren't completely compatible.  The probability in each case is low, but with enough activity eventually some happen anyway.

After a weekend of build breaks, I'm here at 5.30 am to fix build breaks, so that everyone will have a clean build by the time they get here.  Unfortunately for me, today it decides to 'just work.'  Still, it was fun to run here in the middle of the road due to lack of traffic...
One of the things I've had to get used to at Microsoft is operating with cutting-edge code.  By that I mean, somebody just changed it yesterday, new things are going in, plus fixes, and on such a large codebase, problems do happen.

A couple of days ago, somebody asked whether Transactional NTFS would be able to copy a System32 directory.  I tested it, and confirmed that it could.  But how far would it go? I've never really looked at that, so I thought it would be a fun exercise to write some more code to check.  Here, for your reference, is that very code:


#include <windows.h>
#include <ktmw32.h>
#include <stdio.h>
#include <stdlib.h>
#include <tchar.h>
#include <strsafe.h>

int __cdecl _tmain(int argc, TCHAR ** argv)
{
HANDLE hTrans;
HANDLE hFile;
DWORD dwFileNum;
TCHAR szFileName[MAX_PATH];

// Create the transaction.
hTrans = CreateTransaction(NULL, NULL, 0, 0, 0, 0, NULL);

if (hTrans == INVALID_HANDLE_VALUE) {
_tprintf(_T("Could not create transaction.\n"));
return EXIT_FAILURE;
}

// Currently, we're still operating as a normal, non-transacted
// thread. To use the transaction we've just created, call
// SetCurrentTransaction.
if (!SetCurrentTransaction(hTrans)) {
_tprintf(_T("Could not change transactions.\n"));
return EXIT_FAILURE;
}

dwFileNum = 1;
while (TRUE) {
StringCchPrintf(szFileName, sizeof(szFileName)/sizeof(szFileName[0]), _T("%i"), dwFileNum);
hFile = CreateFile(szFileName,
GENERIC_READ|GENERIC_WRITE,
FILE_SHARE_READ|FILE_SHARE_WRITE|FILE_SHARE_DELETE,
NULL,
CREATE_NEW,
FILE_ATTRIBUTE_NORMAL,
NULL);

if (hFile == INVALID_HANDLE_VALUE) {
_tprintf(_T("Failed to create file number %i, GLE %i\n"), dwFileNum, GetLastError());

if (!RollbackTransaction(hTrans)) {
_tprintf(_T("Failed to rollback transaction\n"));
}

CloseHandle(hTrans);

if (!SetCurrentTransaction(NULL)) {
_tprintf(_T("Could not switch back to non-transactional state.\n"));
}

return EXIT_SUCCESS;
}


dwFileNum++;

if (dwFileNum % 10000 == 0) {
_tprintf(_T("%i\n"), dwFileNum);
}
}

return EXIT_SUCCESS;
}
I'd hoped this code would give me some idea of how many files I could create in an uncommitted Transaction. All Transacted applications should be aware that Transactions can be failed by the system in exceptional circumstances; this means that applications should check for this possibility. Most importantly, all memory-mapped I/O should catch exceptions, because if a Transaction is rolled back and exceptions aren't handled, that app will just crash, seemingly randomly, in this eventuality.  The application will still be associated with a Transaction, but since the Transaction is no longer active, all the Transacted calls made by that application are expected to fail, at least, in theory.  Because you can't fail a memory access, all it can do is raise an exception.

Working with pre-beta code makes things always a little uncertain.  Sometimes things just don't go according to plan.  So while I do know that I can create more than 110,000 files in a single directory within a transaction, I'll probably spend tomorrow morning investigating what happened after that.  Bear in mind that that 110,000 figure is not a final figure, and you shouldn't rely on it except to seed your random number generator, or as some kind of rough benchmark about where we are right now.  I can't make any promises about tomorrow :)

Update: well, as I was writing this post, my app actually did finish.  So the final answer is 117,341, and the failed error was ERROR_TRANSACTION_NOT_ACTIVE.  There are still quirks, and I'll still end up working on it tomorrow :)
Yesterday I demonstrated how a Transaction can be inherited by processes.  This means that a process never designed for Transactions can run in a Transacted mode.  Clearly, this has profound implications.  Some applications may exhibit odd or unexpected behavior if run in this way.  It isn't really possible to enumerate every possible case where odd things will happen; the best approach is to use caution.  Let's consider a few cases.

Suppose a non-transactionally aware application recorded a simple log file.  When run from within a transaction, those log records become isolated from other processes, and can be completely rolled back, possibly defeating the purpose of the log.  Applications that record log accesses should be aware of this possibility, and may choose to ensure that they switch out of a Transacted state before performing logging.

Isolating applications that use temporary files into Transactions has the side-effect that it becomes virtually impossible to recover data in event of failure.  Suppose you ran Microsoft Word in a transaction; Word will create a temporary file of your document, which is isolated from public view.  If Word were to fail, the temporary file remains inaccessible and Word can't be invoked to recover that lost data in event of a crash.  Depending on how Transactions are managed in this case, the Transaction may even be rolled back, purging the temporary file forever, and losing data in the process.

The most interesting case, in my opinion, involves launching an application with incomplete Transaction support from within a Transaction.  The test application in my previous post is a good example: what happens if test itself is run from within a Transaction? It will blindly create a second, completely independent transaction, do some work, and commit that Transaction.  Now the application which called the test application wants to rollback its Transaction - which it does, successfully - but all of the changes made by the test application remain committed to disk.

Inheritable Transactions are a powerful user tool, but are also another case that application developers should be aware of.  Well-behaved applications should firstly check if they are being executed within a Transaction and make appropriate ajustments.  What an application should do next depends in part upon what the application was intended to do in the first place.  Maybe the application should simply exit (with an appropriate error message, of course.)  Maybe the semantics of an application could be changed - instead of a Service Pack installer maintaining its own Transaction, it could instead rely on its inherited Transaction and not duplicate Transactional logic.  A well behaved application should never silently ignore the existence of an inherited Transaction; doing this only leads to confusing, potentially inconsistent results.

The test application does ensure that the child process has completed before commiting the Transaction.  This code is particularly important.  If the test application commits the Transaction while a child process is still using it, filesystem calls made by the child will begin to fail.  This includes both reads and writes: all filesystem calls made by the child are made in the context of an invalid Transaction.  The child may not be designed to handle this case elegantly.  When inheriting Transactions, an application should make sure that no process is still running within that Transaction before committing or rolling back.

So what happens if the process being executed is intending to remain executing permanently? Bluntly: don't do this.  That process will eventually exhaust all the resources for recording Transactional changes, and will invariably fail eventually, possibly failing other processes as well.  If a server is to run with Transaction support, it should be designed to support Transactions natively.

In each case of using Transactions, the first question to ask yourself is always: what does this mean? What should I expect?  The implications of Transactions may easily break existing applications, so use caution doing this.  However, Transactional inheritance when used correctly enables you to leverage this new technology without heavy reinvestment in new applications.
A strange thing happened to me yesterday.  I went to install an OS X update to my iMac.  The update failed midway through, and this left my machine in an unbootable state.  The truth is that it could have been Windows, or a Windows product, that did the same thing, and it really reminded me why what we're doing is so important.

So far, you've only heard vague talk about Transactional NTFS.  Today I'd like to show you Transactional NTFS in action.  If you're in Microsoft, you can start using this today; if you're outside of Microsoft, I encourage you to take advantage of the upcoming Longhorn beta programs and try it out.

#include <windows.h>
#include <ktmw32.h>
#include <stdio.h>
#include <stdlib.h>
#include <tchar.h>

int __cdecl _tmain(int argc, TCHAR ** argv)
{
HANDLE hTrans;
STARTUPINFO siStartupInfo;
PROCESS_INFORMATION piProcessInformation;
SECURITY_ATTRIBUTES saSecurityAttributes;

if (argc<2) {
_tprintf(_T("Usage: %s <appname>\n"), argv[0]);
return EXIT_FAILURE;
}

// Clear out the structure
memset (&saSecurityAttributes, 0, sizeof(saSecurityAttributes));

// Make the transaction inheritable, so child processes can use it.
saSecurityAttributes.bInheritHandle = TRUE;

// Create the transaction.
hTrans = CreateTransaction(&saSecurityAttributes, NULL, 0, 0, 0, 0, NULL);

if (hTrans == INVALID_HANDLE_VALUE) {
_tprintf(_T("Could not create transaction.\n"));
return EXIT_FAILURE;
}

// Currently, we're still operating as a normal, non-transacted
// thread. To use the transaction we've just created, call
// SetCurrentTransaction.
if (!SetCurrentTransaction(hTrans)) {
_tprintf(_T("Could not change transactions.\n"));
return EXIT_FAILURE;
}

memset(&siStartupInfo, 0, sizeof(siStartupInfo));
siStartupInfo.cb = sizeof(siStartupInfo);

// Start a child process, inheriting the transaction. Anything
// you do from this process will now be transactional! Neat, huh?
if (!CreateProcess(NULL,
argv[1],
NULL,
NULL,
TRUE, // Inherit transaction handle!
NORMAL_PRIORITY_CLASS,
NULL,
NULL,
&siStartupInfo,
&piProcessInformation)) {
_tprintf(_T("Could not run %s, GetLastError returns %i.\n"), argv[1],
GetLastError());
return EXIT_FAILURE;
}

// Wait for the child process to finish. This is really
// important, because we don't want to commit the transaction
// until the child process isn't using it anymore.
WaitForSingleObject(piProcessInformation.hProcess, INFINITE);

CloseHandle(piProcessInformation.hThread);
CloseHandle(piProcessInformation.hProcess);

// All of a sudden, changes the child process made to the
// filesystem become atomically visible to everybody else :)
if (!CommitTransaction(hTrans)) {
_tprintf(_T("Could not commit the transaction.\n"));
return EXIT_FAILURE;
}

// We're still running transactionally, but that transaction
// has just been committed. Any other file-based operations
// would fail now. So let's switch back to the normal,
// boring, non-transacted state.
if (!SetCurrentTransaction(NULL)) {
_tprintf(_T("Could not switch back to non-transactional state.\n"));
return EXIT_FAILURE;
}

// A transaction handle still needs to be closed, like everything
// else.
CloseHandle(hTrans);

return EXIT_SUCCESS;
}

What does it do? It allows you to transact any existing application you like. So I'd like to show now what happens when I use this program on Longhorn, and run 'cmd.exe'. The output on the left is using transactions; the output on the right is not.
N:\temp>dir
Volume in drive N is New Volume
Volume Serial Number is EE91-9028

Directory of N:\temp

04/25/2005 12:57 PM <DIR> .
04/25/2005 12:57 PM <DIR> ..
04/22/2005 02:01 PM 4,608 test.exe
1 File(s) 4,608 bytes
2 Dir(s) 3,081,523,200 bytes free

N:\temp>test cmd
Microsoft Windows [Version 6.0.5061]
(C) Copyright 1985-2005 Microsoft Corp.

N:\temp>echo foobar>file

N:\temp>dir
Volume in drive N is New Volume
Volume Serial Number is EE91-9028

Directory of N:\temp

04/25/2005 12:58 PM <DIR> .
04/25/2005 12:58 PM <DIR> ..
04/25/2005 12:58 PM 8 file
04/22/2005 02:01 PM 4,608 test.exe
2 File(s) 4,616 bytes
2 Dir(s) 3,081,523,200 bytes free

N:\temp>exit

N:\temp>dir
Volume in drive N is New Volume
Volume Serial Number is EE91-9028

Directory of N:\temp

04/25/2005 12:58 PM <DIR> .
04/25/2005 12:58 PM <DIR> ..
04/25/2005 12:58 PM 8 file
04/22/2005 02:01 PM 4,608 test.exe
2 File(s) 4,616 bytes
2 Dir(s) 3,081,523,200 bytes free

N:\temp>
N:\temp>dir
Volume in drive N is New Volume
Volume Serial Number is EE91-9028

Directory of N:\temp

04/25/2005 12:57 PM <DIR> .
04/25/2005 12:57 PM <DIR> ..
04/22/2005 02:01 PM 4,608 test.exe
1 File(s) 4,608 bytes
2 Dir(s) 3,081,523,200 bytes free







N:\temp>dir
Volume in drive N is New Volume
Volume Serial Number is EE91-9028

Directory of N:\temp

04/25/2005 12:58 PM <DIR> .
04/25/2005 12:58 PM <DIR> ..
04/22/2005 02:01 PM 4,608 test.exe
1 File(s) 4,608 bytes
2 Dir(s) 3,081,523,200 bytes free




N:\temp>dir
Volume in drive N is New Volume
Volume Serial Number is EE91-9028

Directory of N:\temp

04/25/2005 12:58 PM <DIR> .
04/25/2005 12:58 PM <DIR> ..
04/25/2005 12:58 PM 8 file
04/22/2005 02:01 PM 4,608 test.exe
2 File(s) 4,616 bytes
2 Dir(s) 3,081,523,200 bytes free

N:\temp>
This question seemed so good I thought I'd answer it in a whole new blog post :)

As always, it'd be great to see software developers impress me and find new uses for this technology that I haven't foreseen.  I can't tell developers how to write a great application; but I can pick a great application when I see one, and I'm looking to you to impress me.

Transactional NTFS can't guarantee that changes you make to a set files will always be successful, but it can guarantee that on failure the result will always be consistent.  Consider an application install: if you can't copy a particular file, and want to undo all of the files you've already installed, what happens if your undo fails? Maybe a user has your readme opened, and locked, in their editor; maybe they've already moved your start menu entries; maybe they've deleted or moved files you carefully put there.  With Transactional NTFS, all of your changes can be removed - this is guaranteed.

A better example would be a patch or service pack.  If it succeeds, great, if it doesn't, that's okay, but if it half-succeeds, that's terrible.  Transactional NTFS will protect your application from inconsistency.

Imagine updating a website: you make some changes, but fail to update a particular target.  This leaves your site inconsistent, with broken links.  Transactional NTFS will protect you from this possibility; and better still, will ensure all your updates become visible at the same time, reducing the risk of inconsistent results.

When saving a document, some software will attempt to delete the old file, then create a new file in its place.  What happens if something goes wrong? Both copies can be lost.  Transactional NTFS eliminates this possibility.

Suppose on a network, you want to copy a large amount of data from one machine to another: Transactional NTFS can ensure this succeeds or fails atomically, preventing inconsistent results between servers.

These are my thoughts, as an developer.  How you can best use Transactional NTFS depends on your application.  You may have different ideas; I'm interested to hear about them, and I'm sure other developers are too, so feel free to share.

I started here at Microsoft a little over three months ago.  It's been an amazing transition from self-confessed "GNU hippy" to now, and has been more than a mild culture shock.  The good news is that Seattle has far better weather than it's reputed to have.

At work, I've been coming up to speed with a new technology shipping in Longhorn, and have finally been given permission to talk about it with you.  This blog will introduce you to Transactional NTFS, why we're so excited about this technology, and how you can use it in your own applications.

At it's heart, this isn't a really difficult thing to do.  At minimum, an existing application can be made to use transactions by adding only two additional API calls at the beginning, and three additional API calls at the end.  Best of all, the results of these calls can be inherited: five calls in one application can add transactions to another application.  You don't even need source code access!

Of course, the implications of using transactions can be profound, so I'll be explaining some of the things you should be aware of along the way.  This is particularly true when using transactions in a way that modifies the behavior of an application in ways its original author may not have envisioned.

I'll also be trying to answer questions along the way.  The last three months has been full-time learning for me, due to the inherent complexity in parallelising what we've all grown up thinking was a single, shared resource: our filesystem.

 
Page view tracker