<?xml version="1.0" encoding="UTF-8" ?>
<?xml-stylesheet type="text/xsl" href="http://blogs.msdn.com/utility/FeedStylesheets/rss.xsl" media="screen"?><rss version="2.0" xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:slash="http://purl.org/rss/1.0/modules/slash/" xmlns:wfw="http://wellformedweb.org/CommentAPI/"><channel><title>Engineering Windows 7 : Storage</title><link>http://blogs.msdn.com/e7/archive/tags/Storage/default.aspx</link><description>Tags: Storage</description><dc:language>en-US</dc:language><generator>CommunityServer 2.1 SP1 (Build: 61025.2)</generator><item><title>Support and Q&amp;A for Solid-State Drives</title><link>http://blogs.msdn.com/e7/archive/2009/05/05/support-and-q-a-for-solid-state-drives-and.aspx</link><pubDate>Tue, 05 May 2009 10:00:00 GMT</pubDate><guid isPermaLink="false">91d46819-8472-40ad-a661-2c78acb4018c:9567698</guid><dc:creator>e7blog</dc:creator><slash:comments>69</slash:comments><comments>http://blogs.msdn.com/e7/comments/9567698.aspx</comments><wfw:commentRss>http://blogs.msdn.com/e7/commentrss.aspx?PostID=9567698</wfw:commentRss><description>&lt;P&gt;&lt;I&gt;There’s a lot of excitement around the potential for the widespread adoption of solid-state drives (SSD) for primary storage, particularly on laptops and also among many folks in the server world.&amp;nbsp; As with any new technology, as it is introduced we often need to revisit the assumptions baked into the overall system (OS, device support, applications) as a result of the performance characteristics of the technologies in use.&amp;nbsp; This post looks at the way we have tuned Windows 7 to the current generation of SSDs.&amp;nbsp; This is a rapidly moving area and we expect that there will continue to be ways we will tune Windows and we also expect the technology to continue to evolve, perhaps introducing new tradeoffs or challenging other underlying assumptions.&amp;nbsp; Michael Fortin authored this post with help from many folks across the storage and fundamentals teams.&amp;nbsp; --Steven&lt;/I&gt;&lt;/P&gt;
&lt;P&gt;Many of today’s Solid State Drives (SSDs) offer the promise of improved performance, more consistent responsiveness, increased battery life, superior ruggedness, quicker startup times, and noise and vibration reductions. With prices dropping precipitously, most analysts expect more and more PCs to be sold with SSDs in place of traditional rotating hard disk drives (HDDs).&lt;/P&gt;
&lt;P&gt;In Windows 7, we’ve focused a number of our engineering efforts with SSD operating characteristics in mind. As a result, Windows 7’s default behavior is to operate efficiently on SSDs without requiring any customer intervention. Before delving into how Windows 7’s behavior is automatically tuned to work efficiently on SSDs, a brief overview of SSD operating characteristics is warranted.&lt;/P&gt;
&lt;H3&gt;&lt;B&gt;Random Reads: A very good story for SSDs&lt;/B&gt;&lt;/H3&gt;
&lt;P&gt;SSDs tend to be very fast for random reads. Most SSDs thoroughly trounce traditionally HDDs because the mechanical work required to position a rotating disk head isn’t required. As a result, the better SSDs can perform 4 KB random reads almost 100 times faster than the typical HDD (about 1/10&lt;SUP&gt;th&lt;/SUP&gt; of a millisecond per read vs. roughly 10 milliseconds).&lt;/P&gt;
&lt;H3&gt;&lt;B&gt;Sequential Reads and Writes: Also Good&lt;/B&gt;&lt;/H3&gt;
&lt;P&gt;Sequential read and write operations range between quite good to superb. Because flash chips can be configured in parallel and data spread across the chips, today’s better SSDs can read sequentially at rates greater than 200 MB/s, which is close to double the rate many 7200 RPM drives can deliver. For sequential writes, we see some devices greatly exceeding the rates of typical HDDs, and most SSDs doing fairly well in comparison. In today’s market, there are still considerable differences in sequential write rates between SSDs. Some greatly outperform the typical HDD, others lag by a bit, and a few are poor in comparison.&lt;/P&gt;
&lt;H3&gt;&lt;B&gt;Random Writes &amp;amp; Flushes: Your mileage will vary greatly&lt;/B&gt;&lt;/H3&gt;
&lt;P&gt;The differences in sequential write rates are interesting to note, but for most users they won’t make for as notable a difference in overall performance as random writes.&lt;/P&gt;
&lt;P&gt;What’s a long time for a random write? Well, an average HDD can typically move 4 KB random writes to its spinning media in 7 to 15 milliseconds, which has proven to be largely unacceptable. As a result, most HDDs come with 4, 8 or more megabytes of internal memory and attempt to cache small random writes rather than wait the full 7 to 15 milliseconds. When they do cache a write, they return success to the OS even though the bytes haven’t been moved to the spinning media. We typically see these cached writes completing in a few hundred &lt;B&gt;&lt;I&gt;micro&lt;/I&gt;&lt;/B&gt;seconds (so 10X, 20X or faster than actually writing to spinning media). In looking at millions of disk writes from thousands of telemetry traces, we observe 92% of 4 KB or smaller IOs taking less than 1 millisecond, 80% taking less than 600 microseconds, and an impressive 48% taking less than 200 microseconds. Caching works!&lt;/P&gt;
&lt;P&gt;On occasion, we’ll see HDDs struggle with bursts of random writes and flushes. Drives that cache too much for too long and then get caught with too much of a backlog of work to complete when a flush comes along, have proven to be problematic. These flushes and surrounding IOs can have considerably lengthened response times. We’ve seen some devices take a half second to a full second to complete individual IOs and take 10’s of seconds to return to a more consistently responsive state. For the user, this can be awful to endure as responsiveness drops to painful levels. Think of it, the response time for a single I/O can range from 200 &lt;U&gt;micro&lt;/U&gt;seconds up to a whopping 1,000,000 &lt;U&gt;micro&lt;/U&gt;seconds (1 second).&lt;/P&gt;
&lt;P&gt;When presented with realistic workloads, we see the worst of the SSDs producing very long IO times as well, as much as one half to one full second to complete individual random write and flush requests. This is abysmal for many workloads and can make the entire system feel choppy, unresponsive and sluggish.&lt;/P&gt;
&lt;H3&gt;&lt;B&gt;Random Writes &amp;amp; Flushes: Why is this so hard?&lt;/B&gt;&lt;/H3&gt;
&lt;P&gt;For many, the notion that a purely electronic SSD can have more trouble with random writes than a traditional HDD seems hard to comprehend at first. After all, SSDs don’t need to seek and position a disk head above a track on a rotating disk, so why would random writes present such a daunting a challenge?&lt;/P&gt;
&lt;P&gt;The answer to this takes quite a bit of explaining, Anand’s &lt;A href="http://www.anandtech.com/storage/showdoc.aspx?i=3531&amp;amp;p=1" mce_href="http://www.anandtech.com/storage/showdoc.aspx?i=3531&amp;amp;p=1"&gt;article&lt;/A&gt; admirably covers many of the details. We highly encourage motivated folks to take the time to read it as well as this fine &lt;A href="http://www.usenix.org/event/usenix08/tech/full_papers/agrawal/agrawal_html/index.html" mce_href="http://www.usenix.org/event/usenix08/tech/full_papers/agrawal/agrawal_html/index.html"&gt;USENIX paper&lt;/A&gt;. In an attempt to avoid covering too much of the same material, we’ll just make a handful of points.&lt;/P&gt;
&lt;UL&gt;
&lt;LI&gt;&lt;EM&gt;Most SSDs are comprised of flash cells (either &lt;/EM&gt;&lt;A href="http://en.wikipedia.org/wiki/SLC_flash" mce_href="http://en.wikipedia.org/wiki/SLC_flash"&gt;&lt;EM&gt;SLC&lt;/EM&gt;&lt;/A&gt;&lt;EM&gt; or &lt;/EM&gt;&lt;A href="http://en.wikipedia.org/wiki/MLC_flash" mce_href="http://en.wikipedia.org/wiki/MLC_flash"&gt;&lt;EM&gt;MLC&lt;/EM&gt;&lt;/A&gt;&lt;EM&gt;).&lt;/EM&gt; It is possible to build SSDs out of DRAM. These can be extremely fast, but also very costly and power hungry. Since these are relatively rare, we’ll focus our discussion on the much more popular NAND flash based SSDs. Future SSDs may take advantage of other nonvolatile memory technologies than flash.&lt;/LI&gt;
&lt;LI&gt;&lt;EM&gt;A flash cell is really a trap, a trap for electrons and electrons don’t like to be trapped. &lt;/EM&gt;Consider this, if placing 100 electrons in a flash cell constitutes a bit value of 0, and fewer means the value is 1, then the controller logic may have to consider 80 to 120 as the acceptable range for a bit value of 0. A range is necessary because some electrons may escape the trap, others may fall into the trap when attempting to fill nearby cells, etc… As a result, some very sophisticated error correction logic is needed to insure data integrity. &lt;/LI&gt;
&lt;LI&gt;&lt;EM&gt;Flash chips tend to be organized in complex arrangements, such as blocks, dies, planes and packages. &lt;/EM&gt;The size, arrangement, parallelism, wear, interconnects and transfer speed characteristics of which can and do vary greatly.&lt;/LI&gt;
&lt;LI&gt;&lt;EM&gt;Flash cells need to be erased before they can be written.&lt;/EM&gt; You simply can’t trust that a flash cell has no residual electrons in it before use, so cells need to be erased before filling with electrons. Erasing is done on a large scale. You don’t erase a cell; rather you erase a large block of cells (like 128 KB worth). Erase times are typically long -- a millisecond or more.&lt;/LI&gt;
&lt;LI&gt;&lt;EM&gt;Flash wears out. &lt;/EM&gt;At some point, a flash cell simply stops working as a trap for electrons. If frequently updated data (e.g., a file system log file) was always stored in the same cells, those cells would wear out more quickly than cells containing read-mostly data. Wear leveling logic is employed by flash controller firmware to spread out writes across a device’s full set of cells. If done properly, most devices will last years under normal desktop/laptop workloads.&lt;/LI&gt;
&lt;LI&gt;&lt;EM&gt;It takes some pretty clever device physicists and some solid engineering to trap electrons at high speed, to do so without errors, and to keep the devices from wearing out unevenly.&lt;/EM&gt; Not all SSD manufacturers are as far along as others in figuring out how to do this well.&lt;/LI&gt;&lt;/UL&gt;
&lt;H3&gt;&lt;A title=Trim name=Trim&gt;&lt;/A&gt;&lt;B&gt;&lt;FONT color=#000000&gt;Performance &lt;/FONT&gt;&lt;/B&gt;&lt;B&gt;Degradation Over Time, Wear, and Trim &lt;/B&gt;&lt;/H3&gt;
&lt;P&gt;As mentioned above, flash blocks and cells need to be erased before new bytes can be written to them. As a result, newly purchased devices (with all flash blocks pre-erased) can perform notably better at purchase time than after considerable use. While we’ve observed this performance degradation ourselves, we do not consider this to be a show stopper. In fact, except via benchmarking measurements, we don’t expect users to notice the drop during normal use. &lt;/P&gt;
&lt;P&gt;Of course, device manufactures and Microsoft want to maintain superior performance characteristics as best we can. One can easily imagine the better SSD manufacturers attempting to overcome the aging issues by pre-erasing blocks so the performance penalty is largely unrealized during normal use, or by maintaining a large enough spare area to store short bursts of writes. SSD drives designed for the enterprise may have as high as 50% of their space reserved in order to provide lengthy periods of high sustained write performance.&lt;/P&gt;
&lt;P&gt;In addition to the above, Microsoft and SSD manufacturers are adopting the Trim operation. In Windows 7, if an SSD reports it supports the Trim attribute of the ATA protocol’s Data Set Management command, the NTFS file system will request the ATA driver to issue the new operation to the device when files are deleted and it is safe to erase the SSD pages backing the files. With this information, an SSD can plan to erase the relevant blocks opportunistically (and lazily) in the hope that subsequent writes will not require a blocking erase operation since erased pages are available for reuse.&lt;/P&gt;
&lt;P&gt;As an added benefit, the Trim operation can help SSDs reduce wear by eliminating the need for many merge operations to occur. As an example, consider a single 128 KB SSD block that contained a 128 KB file. If the file is deleted and a Trim operation is requested, then the SSD can avoid having to mix bytes from the SSD block with any other bytes that are subsequently written to that block. This reduces wear.&lt;/P&gt;
&lt;P&gt;Windows 7 requests the Trim operation for more than just file delete operations. The Trim operation is fully integrated with partition- and volume-level commands like Format and Delete, with file system commands relating to truncate and compression, and with the System Restore (aka Volume Snapshot) feature.&lt;/P&gt;
&lt;H3&gt;&lt;B&gt;Windows 7 Optimizations and Default Behavior Summary&lt;/B&gt;&lt;/H3&gt;
&lt;P&gt;As noted above, all of today’s SSDs have considerable work to do when presented with disk writes and disk flushes. Windows 7 tends to perform well on today’s SSDs, in part, because we made many engineering changes to reduce the frequency of writes and flushes. This benefits traditional HDDs as well, but is particularly helpful on today’s SSDs.&lt;/P&gt;
&lt;P&gt;Windows 7 will disable disk defragmentation on SSD system drives. Because SSDs perform extremely well on random read operations, defragmenting files isn’t helpful enough to warrant the added disk writing defragmentation produces. The FAQ section below has some additional details.&lt;/P&gt;
&lt;P&gt;Be default, Windows 7 will disable Superfetch, ReadyBoost, as well as boot and application launch prefetching on SSDs with good random read, random write and flush performance. These technologies were all designed to improve performance on traditional HDDs, where random read performance could easily be a major bottleneck. See the FAQ section for more details.&lt;/P&gt;
&lt;P&gt;Since SSDs tend to perform at their best when the operating system’s partitions are created with the SSD’s alignment needs in mind, all of the partition-creating tools in Windows 7 place newly created partitions with the appropriate alignment.&lt;/P&gt;
&lt;H3&gt;&lt;B&gt;Frequently Asked Questions&lt;/B&gt;&lt;/H3&gt;
&lt;P&gt;Before addressing some frequently asked questions, we’d like to remind everyone that we believe the future of SSDs in mobile and desktop PCs (as well as enterprise servers) looks very bright to us. SSDs can deliver on the promise of improved performance, more consistent responsiveness, increased battery life, superior ruggedness, quicker startup times, and noise and vibration reductions. With prices steadily dropping and quality on the rise, we expect more and more PCs to be sold with SSDs in place of traditional rotating HDDs. With that in mind, we focused an appropriate amount of our engineering efforts towards insuring Windows 7 users have great experiences on SSDs. &lt;/P&gt;
&lt;P&gt;&lt;B&gt;Will Windows 7 support Trim?&lt;/B&gt;&lt;/P&gt;
&lt;P&gt;Yes. See the above section for details.&lt;/P&gt;
&lt;P&gt;&lt;B&gt;Will disk defragmentation be disabled by default on SSDs?&lt;/B&gt;&lt;/P&gt;
&lt;P&gt;Yes. The automatic scheduling of defragmentation will exclude partitions on devices that declare themselves as SSDs. Additionally, if the system disk has random read performance characteristics above the threshold of 8 MB/sec, then it too will be excluded. The threshold was determined by internal analysis. &lt;/P&gt;
&lt;P&gt;The random read threshold test was added to the final product to address the fact that few SSDs on the market today properly identify themselves as SSDs. 8 MB/sec is a relatively conservative rate. While none of our tested HDDs could approach 8 MB/sec, all of our tested SSDs exceeded that threshold. SSD performance ranged between 11 MB/sec and 130 MB/sec. Of the 182 HDDs tested, only 6 configurations managed to exceed 2 MB/sec on our random read test. The other 176 ranged between 0.8 MB/sec and 1.6 MB/sec.&lt;/P&gt;
&lt;P&gt;&lt;B&gt;Will Superfetch be disabled on SSDs?&lt;/B&gt;&lt;/P&gt;
&lt;P&gt;Yes, for most systems with SSDs. &lt;/P&gt;
&lt;P&gt;If the system disk is an SSD, and the SSD performs adequately on random reads and doesn’t have glaring performance issues with random writes or flushes, then Superfetch, boot prefetching, application launch prefetching, ReadyBoost and ReadDrive will all be disabled.&lt;/P&gt;
&lt;P&gt;Initially, we had configured all of these features to be off on all SSDs, but we encountered sizable performance regressions on some systems. In root causing those regressions, we found that some first generation SSDs had severe enough random write and flush problems that ultimately lead to disk reads being blocked for long periods of time. With Superfetch and other prefetching re-enabled, performance on key scenarios was markedly improved.&lt;/P&gt;
&lt;P&gt;&lt;B&gt;Is NTFS Compression of Files and Directories recommended on SSDs?&lt;/B&gt;&lt;/P&gt;
&lt;P&gt;Compressing files help save space, but the effort of compressing and decompressing requires extra CPU cycles and therefore power on mobile systems. That said, for infrequently modified directories and files, compression is a fine way to conserve valuable SSD space and can be a good tradeoff if space is truly a premium.&lt;/P&gt;
&lt;P&gt;We do not, however, recommend compressing files or directories that will be written to with great frequency. Your Documents directory and files are likely to be fine, but temporary internet directories or mail folder directories aren’t such a good idea because they get large number of file writes in bursts.&lt;/P&gt;
&lt;P&gt;&lt;B&gt;Does the Windows Search Indexer operate differently on SSDs?&lt;/B&gt;&lt;/P&gt;
&lt;P&gt;No.&lt;/P&gt;
&lt;P&gt;&lt;B&gt;Is Bitlocker’s encryption process optimized to work on SSDs?&lt;/B&gt;&lt;/P&gt;
&lt;P&gt;Yes, on NTFS. When Bitlocker is first configured on a partition, the entire partition is read, encrypted and written back out. As this is done, the NTFS file system will issue Trim commands to help the SSD optimize its behavior.&lt;/P&gt;
&lt;P&gt;We do encourage users concerned about their data privacy and protection to enable Bitlocker on their drives, including SSDs.&lt;/P&gt;
&lt;P&gt;&lt;B&gt;Does Media Center do anything special when configured on SSDs?&lt;/B&gt;&lt;/P&gt;
&lt;P&gt;No. While SSDs do have advantages over traditional HDDs, SSDs are more costly per GB than their HDD counterparts. For most users, a HDD optimized for media recording is a better choice, as media recording and playback workloads are largely sequential in nature.&lt;/P&gt;
&lt;P&gt;&lt;B&gt;Does Write Caching make sense on SSDs and does Windows 7 do anything special if an SSD supports write caching?&lt;/B&gt;&lt;/P&gt;
&lt;P&gt;Some SSD manufacturers including RAM in their devices for more than just their control logic; they are mimicking the behavior of traditional disks by caching writes, and possibly reads. For devices that do cache writes in volatile memory, Windows 7 expects flush commands and write-ordering to be preserved to at least the same degree as traditional rotating disks. Additionally, Windows 7 expects user settings that disable write caching to be honored by write caching SSDs just as they are on traditional disks.&lt;/P&gt;
&lt;P&gt;&lt;B&gt;Do RAID configurations make sense with SSDs?&lt;/B&gt;&lt;/P&gt;
&lt;P&gt;Yes. The reliability and performance benefits one can obtain via HDD RAID configurations can be had with SSD RAID configurations.&lt;/P&gt;
&lt;P&gt;&lt;B&gt;Should the pagefile be placed on SSDs? &lt;/B&gt;&lt;/P&gt;
&lt;P&gt;Yes. Most pagefile operations are small random reads or larger sequential writes, both of which are types of operations that SSDs handle well.&lt;/P&gt;
&lt;P&gt;In looking at telemetry data from thousands of traces and focusing on pagefile reads and writes, we find that&lt;/P&gt;
&lt;UL&gt;
&lt;LI&gt;Pagefile.sys reads outnumber pagefile.sys writes by about 40 to 1, &lt;/LI&gt;
&lt;LI&gt;Pagefile.sys read sizes are typically quite small, with 67% less than or equal to 4 KB, and 88% less than 16 KB.&lt;/LI&gt;
&lt;LI&gt;Pagefile.sys writes are relatively large, with 62% greater than or equal to 128 KB and 45% being exactly 1 MB in size.&lt;/LI&gt;&lt;/UL&gt;
&lt;P&gt;In fact, given typical pagefile reference patterns and the favorable performance characteristics SSDs have on those patterns, there are few files better than the pagefile to place on an SSD.&lt;/P&gt;
&lt;P&gt;&lt;B&gt;Are there any concerns regarding the Hibernate file and SSDs?&lt;/B&gt;&lt;/P&gt;
&lt;P&gt;No, hiberfile.sys is written to and read from sequentially and in large chunks, and thus can be placed on either HDDs or SSDs.&lt;/P&gt;
&lt;P&gt;&lt;B&gt;What Windows Experience Index changes were made to address SSD performance characteristics?&lt;/B&gt;&lt;/P&gt;
&lt;P&gt;In Windows 7, there are new random read, random write and flush assessments. Better SSDs can score above 6.5 all the way to 7.9. To be included in that range, an SSD has to have outstanding random read rates and be resilient to flush and random write workloads.&lt;/P&gt;
&lt;P&gt;In the Beta timeframe of Windows 7, there was a capping of scores at 1.9, 2.9 or the like if a disk (SSD or HDD) didn’t perform adequately when confronted with our random write and flush assessments. Feedback on this was pretty consistent, with most feeling the level of capping to be excessive. As a result, we now simply restrict SSDs with performance issues from joining the newly added 6.0+ and 7.0+ ranges. SSDs that are not solid performers across all assessments effectively get scored in a manner similar to what they would have been in Windows Vista, gaining no Win7 boost for great random read performance.&lt;/P&gt;&lt;img src="http://blogs.msdn.com/aggbug.aspx?PostID=9567698" width="1" height="1"&gt;</description><category domain="http://blogs.msdn.com/e7/archive/tags/Storage/default.aspx">Storage</category></item><item><title>Disk Defragmentation – Background and Engineering the Windows 7 Improvements</title><link>http://blogs.msdn.com/e7/archive/2009/01/25/disk-defragmentation-background-and-engineering-the-windows-7-improvements.aspx</link><pubDate>Sun, 25 Jan 2009 11:00:00 GMT</pubDate><guid isPermaLink="false">91d46819-8472-40ad-a661-2c78acb4018c:9375342</guid><dc:creator>e7blog</dc:creator><slash:comments>50</slash:comments><comments>http://blogs.msdn.com/e7/comments/9375342.aspx</comments><wfw:commentRss>http://blogs.msdn.com/e7/commentrss.aspx?PostID=9375342</wfw:commentRss><description>&lt;P&gt;&lt;I&gt;One of the features that you’ve been pretty clear about (I’ve received over 100 emails on this topic!) is the desire to improve the disk defrag utility in Windows 7. We did. And from blogs we saw a few of you noticed, which is great. This is not as straight forward as it may appear. We know there’s a lot of history in defrag and how “back in the day” it was a very significant performance issue and also a big mystery to most people. So many folks came to know that if your machine is slow you had to go through the top-secret defrag process. In Windows Vista we decided to just put the process on autopilot with the intent that you’d never have to worry about it. In practice this turns out to be true, at least to the limits of automatically running a process (that is if you turn your machine off every night then it will never run). We received a lot of feedback from knowledgeable folks wanting more information on defrag status, especially during execution, as well as more flexibility in terms of the overall management of the process. This post will detail the changes we made based on that feedback. In reading the mail and comments we received, we also thought it would be valuable to go into a little bit more detail about the process, the perceptions and reality of performance gains, as well as the specific improvements. This post is by Rajeev Nagar and Matt Garson, both are Program Managers on our File System feature team. --Steven&lt;/I&gt;&lt;/P&gt;
&lt;P&gt;In this blog, we focus on disk defragmentation in Windows 7. Before we discuss the changes introduced in Windows 7, let’s chat a bit about what fragmentation is, and its applicability.&lt;/P&gt;
&lt;P&gt;Within the storage and memory hierarchy comprising the hardware pipeline between the hard disk and CPU, hard disks are relatively slower and have relatively higher latency. Read/write times from and to a hard disk are measured in milliseconds (typically, 2-5 ms) – which sounds quite fast until compared to a 2GHz CPU that can compute data in less than 10 nanoseconds (on average), once the data is in the L1 memory cache of the processor.&lt;/P&gt;
&lt;P&gt;This performance gap has only been increasing over the past 2 decades – the figures below are noteworthy.&lt;/P&gt;
&lt;P&gt;&lt;A href="http://blogs.msdn.com/blogfiles/e7/WindowsLiveWriter/DiskDefragmentationBackgroundandEngineer_CA52/image_2.png" mce_href="http://blogs.msdn.com/blogfiles/e7/WindowsLiveWriter/DiskDefragmentationBackgroundandEngineer_CA52/image_2.png"&gt;&lt;IMG style="BORDER-RIGHT-WIDTH: 0px; DISPLAY: block; FLOAT: none; BORDER-TOP-WIDTH: 0px; BORDER-BOTTOM-WIDTH: 0px; MARGIN-LEFT: auto; BORDER-LEFT-WIDTH: 0px; MARGIN-RIGHT: auto" title="Graph of Historical Trends of CPU and IOPS Performance" border=0 alt="Graph of Historical Trends of CPU and IOPS Performance" src="http://blogs.msdn.com/blogfiles/e7/WindowsLiveWriter/DiskDefragmentationBackgroundandEngineer_CA52/image_thumb.png" width=579 height=396 mce_src="http://blogs.msdn.com/blogfiles/e7/WindowsLiveWriter/DiskDefragmentationBackgroundandEngineer_CA52/image_thumb.png"&gt;&lt;/A&gt; &lt;/P&gt;
&lt;P&gt;&lt;A href="http://blogs.msdn.com/blogfiles/e7/WindowsLiveWriter/DiskDefragmentationBackgroundandEngineer_CA52/image_4.png" mce_href="http://blogs.msdn.com/blogfiles/e7/WindowsLiveWriter/DiskDefragmentationBackgroundandEngineer_CA52/image_4.png"&gt;&lt;IMG style="BORDER-RIGHT-WIDTH: 0px; DISPLAY: block; FLOAT: none; BORDER-TOP-WIDTH: 0px; BORDER-BOTTOM-WIDTH: 0px; MARGIN-LEFT: auto; BORDER-LEFT-WIDTH: 0px; MARGIN-RIGHT: auto" title="Chart of Performance Improvements of Various Technologies" border=0 alt="Chart of Performance Improvements of Various Technologies" src="http://blogs.msdn.com/blogfiles/e7/WindowsLiveWriter/DiskDefragmentationBackgroundandEngineer_CA52/image_thumb_1.png" width=504 height=408 mce_src="http://blogs.msdn.com/blogfiles/e7/WindowsLiveWriter/DiskDefragmentationBackgroundandEngineer_CA52/image_thumb_1.png"&gt;&lt;/A&gt; &lt;/P&gt;
&lt;P&gt;In short, the figures illustrate that while disk capacities are increasing, their ability to transfer data or write new data is not increasing at an equivalent rate – so disks contain &lt;B&gt;&lt;U&gt;more&lt;/U&gt;&lt;/B&gt; data that takes &lt;B&gt;&lt;U&gt;longer&lt;/U&gt;&lt;/B&gt; to read or write. Consequently, fast CPUs are relatively idle, waiting for data to do work on. &lt;/P&gt;
&lt;P&gt;Significant research in Computer Science has focused on improving overall system I/O performance, which has lead to two principles that the operating system tries to follow:&lt;/P&gt;
&lt;OL&gt;
&lt;LI&gt;Perform less I/O, i.e. try and minimize the number of times a disk read or write request is issued. &lt;/LI&gt;
&lt;LI&gt;When I/O is issued, transfer data in relatively large chunks, i.e. read or write in bulk. &lt;/LI&gt;&lt;/OL&gt;
&lt;P&gt;Both rules have reasonably simply understood rationale:&lt;/P&gt;
&lt;OL&gt;
&lt;LI&gt;Each time an I/O is issued by the CPU, multiple software and hardware components have to do work to satisfy the request. This contributes toward increased latency, i.e., the amount of time until the request is satisfied. This latency is often directly experienced by users when reading data and leads to increased user frustration if expectations are not met. &lt;/LI&gt;
&lt;LI&gt;Movement of mechanical parts contributes substantially to incurred latency. For hard disks, the “&lt;I&gt;rotational time&lt;/I&gt;” (time taken for the disk platter to rotate in order to get the right portion of the disk positioned under the disk head) and the “&lt;I&gt;seek time&lt;/I&gt;” (time taken by the head to move so that it is positioned to be able to read/write the targeted track) are the two major culprits. By reading or writing in large chunks, the incurred costs are amortized over the larger amount of data that is transferred – in other words, the “per unit” data transfer costs decrease. &lt;/LI&gt;&lt;/OL&gt;
&lt;P&gt;File systems such as NTFS work quite hard to try and satisfy the above rules. As an example, consider the case when I listen to the song “Hotel California” by the Eagles (one of my all time favorite bands). When I first save the 5MB file to my NTFS volume, the file system will try and find enough contiguous free space to be able to place the 5MB of data “together” on the disk. Since logically related data (e.g. contents of the same file or directory) is more likely to be read or written around the same time. For example, I would typically play the entire song “Hotel California” and not just a portion of it. During the 3 minutes that the song is playing, the computer would be fetching portions of this “related content” (i.e. sub-portions of the file) from the disk until the entire file is consumed. By making sure the data is placed together, the system can issue read requests in larger chunks (often pre-reading data in anticipation that it will soon be used) which, in turn, will minimize mechanical movement of hard disk drive components and also ensure fewer issued I/Os.&lt;/P&gt;
&lt;P&gt;Given that the file system tries to place data contiguously, when does fragmentation occur? Modifications to stored data (e.g. adding, changing, or deleting content) cause changes in the on-disk data layout and can result in fragmentation. For example, file deletion naturally causes space de-allocation and resultant “holes” in the allocated space map – a condition we will refer to as “fragmentation of available free space”. Over time, contiguous free space becomes harder to find leading to fragmentation of newly stored content. Obviously, deletion is not the only cause of fragmentation – as mentioned above, other file operations such as modifying content in place or appending data to an existing file can eventually lead to the same condition.&lt;/P&gt;
&lt;P&gt;So how does defragmentation help? In essence, defragmentation helps by moving data around so that it is once again placed more optimally on the hard disk, providing the following benefits:&lt;/P&gt;
&lt;OL&gt;
&lt;LI&gt;Any logically related content that was fragmented can be placed adjacently &lt;/LI&gt;
&lt;LI&gt;Free space can be coalesced so that new content written to the disk can be done so efficiently &lt;/LI&gt;&lt;/OL&gt;
&lt;P&gt;The following diagram will help illustrate what we’re discussing. The first illustration represents an ideal state of a disk – there are 3 files, A, B, and C, and all are stored in contiguous locations; there is no fragmentation. The second illustration represents a fragmented disk – a portion of data associated with File A is now located in a non-contiguous location (due to growth of the file). The third illustration shows how data on the disk would look like once the disk was defragmented.&lt;/P&gt;
&lt;P&gt;&lt;A href="http://blogs.msdn.com/blogfiles/e7/WindowsLiveWriter/DiskDefragmentationBackgroundandEngineer_CA52/image_6.png" mce_href="http://blogs.msdn.com/blogfiles/e7/WindowsLiveWriter/DiskDefragmentationBackgroundandEngineer_CA52/image_6.png"&gt;&lt;IMG style="BORDER-RIGHT-WIDTH: 0px; DISPLAY: block; FLOAT: none; BORDER-TOP-WIDTH: 0px; BORDER-BOTTOM-WIDTH: 0px; MARGIN-LEFT: auto; BORDER-LEFT-WIDTH: 0px; MARGIN-RIGHT: auto" title="Example of disk blocks being defragmented." border=0 alt="Example of disk blocks being defragmented." src="http://blogs.msdn.com/blogfiles/e7/WindowsLiveWriter/DiskDefragmentationBackgroundandEngineer_CA52/image_thumb_2.png" width=368 height=368 mce_src="http://blogs.msdn.com/blogfiles/e7/WindowsLiveWriter/DiskDefragmentationBackgroundandEngineer_CA52/image_thumb_2.png"&gt;&lt;/A&gt; &lt;/P&gt;
&lt;P&gt;Nearly all modern file systems support defragmentation – the differences generally are in the defragmentation mechanism, whether, as in Windows, it’s a separate, schedulable task or, whether the mechanism is more implicitly managed and internal to the file system. The design decisions simply reflect the particular design goals of the system and the necessary tradeoffs. Furthermore, it’s unlikely that a general-purpose file system could be designed such that fragmentation never occurred.&lt;/P&gt;
&lt;P&gt;Over the years, defragmentation has been given a lot of emphasis because, historically, fragmentation was a problem that could have more significant impact. In the early days of personal computing, when disk capacities were measured in megabytes, disks got full faster and fragmentation occurred more often. Further, memory caches were significantly limited and system responsiveness was increasingly predicated on disk I/O performance. This got to a point that some users ran their defrag tool weekly or even more often! Today, very large disk drives are available cheaply and % disk utilization for the average consumer is likely to be lower causing relatively less fragmentation. Further, computers can utilize more RAM cheaply (often, enough to be able to cache the data set actively in use). That together, with improvements in file system allocation strategies as well as caching and pre-fetching algorithms, further helps improve overall responsiveness. Therefore, while the performance gap between the CPU and disks continues to grow and fragmentation does occur, combined hardware and software advances in other areas allow Windows to mitigate fragmentation impact and deliver better responsiveness.&lt;/P&gt;
&lt;P&gt;So, how would we evaluate fragmentation given today’s software and hardware? A first question might be: &lt;I&gt;how often does fragmentation actually occur and to what extent&lt;/I&gt;? After all, 500GB of data with 1% fragmentation is significantly different than 500GB with 50% fragmentation. Secondly, &lt;I&gt;what is the actual performance penalty of fragmentation, given today’s hardware and software&lt;/I&gt;? Quite a few of you likely remember various products introduced over the past two decades offering various performance enhancements (e.g. RAM defragmentation, disk compression, etc.), many of which have since become obsolete due to hardware and software advances.&lt;/P&gt;
&lt;P&gt;The incidence and extent of fragmentation in average home computers varies quite a bit depending on available disk capacity, disk consumption, and usage patterns. In other words, &lt;U&gt;there is no general answer&lt;/U&gt;. The actual performance impact of fragmentation is the more interesting question but even more complex to accurately quantify. A meaningful evaluation of the performance penalty of fragmentation would require the following:&lt;/P&gt;
&lt;UL&gt;
&lt;LI&gt;Availability of a system that has been “aged” to create fragmentation in a typical or representative manner. But, as noted above, there is no single, representative behavior. For example, the frequency and extent of fragmentation on a computer used primarily for web browsing will be different than a computer used as a file server. &lt;/LI&gt;
&lt;LI&gt;Selection of meaningful disk-bound metrics e.g. boot and first-time application launch post boot. &lt;/LI&gt;
&lt;LI&gt;Repeated measurements that can be statistically relevant &lt;/LI&gt;&lt;/UL&gt;
&lt;P&gt;Let’s walk through an example that helps illustrate the complexity in directly correlating extent of fragmentation with user-visible performance.&lt;/P&gt;
&lt;P&gt;In Windows XP, any file that is split into more than one piece is considered fragmented. Not so in Windows Vista if the fragments are large enough – the defragmentation algorithm was changed (from Windows XP) to ignore pieces of a file that are larger than 64MB. As a result, defrag in XP and defrag in Vista will report different amounts of fragmentation on a volume. So, which one is correct? Well, before the question can be answered we must understand why defrag in Vista was changed. In Vista, we analyzed the impact of defragmentation and determined that the most significant performance gains from defrag are when pieces of files are combined into &lt;I&gt;sufficiently large chunks&lt;/I&gt; such that the impact of disk-seek latency is not significant relative to the latency associated with sequentially reading the file. This means that &lt;U&gt;there is a point after which combining fragmented pieces of files has no discernible benefit&lt;/U&gt;. In fact, there are actually negative consequences of doing so. For example, for defrag to combine fragments that are 64MB or larger requires significant amounts of disk I/O, which is against the principle of minimizing I/O that we discussed earlier (since it decreases total available disk bandwidth for user initiated I/O), and puts more pressure on the system to find large, contiguous blocks of free space. Here is a scenario where a certainly amount of fragmentation of data is just fine – doing nothing to decrease this fragmentation turns out to be the right answer! &lt;/P&gt;
&lt;P&gt;Note that a concept that is relatively simple to understand, such as the amount of fragmentation and its impact, is in reality much more complex, and its real impact requires comprehensive evaluation of the entire system to accurately address. The different design decisions across Windows XP and Vista reflect this evaluation of the typical hardware &amp;amp; software environment used by customers. Ultimately, when thinking about defragmentation, it is important to realize that there are many additional factors contributing towards system responsiveness that must be considered beyond a simple count of existing fragments. &lt;/P&gt;
&lt;P&gt;The defragmentation engine and experience in Windows 7 has been revamped based on continuous and holistic analysis of impact on system responsiveness:&lt;/P&gt;
&lt;P mce_keep="true"&gt;In Windows Vista, we had removed all of the UI that would provide detailed defragmentation status. We received feedback that you didn’t like this decision, so we listened, evaluated the various tradeoffs, and have built a new GUI for defrag! As a result, in Windows 7, you can monitor status more easily and intuitively. Further, defragmentation can be safely terminated any time during the process and on all volumes very simply (if required). The two screenshots below illustrate the ease-of-monitoring: &lt;/P&gt;
&lt;P&gt;&lt;A href="http://blogs.msdn.com/blogfiles/e7/WindowsLiveWriter/DiskDefragmentationBackgroundandEngineer_CA52/image_8.png" mce_href="http://blogs.msdn.com/blogfiles/e7/WindowsLiveWriter/DiskDefragmentationBackgroundandEngineer_CA52/image_8.png"&gt;&lt;IMG style="BORDER-RIGHT-WIDTH: 0px; DISPLAY: block; FLOAT: none; BORDER-TOP-WIDTH: 0px; BORDER-BOTTOM-WIDTH: 0px; MARGIN-LEFT: auto; BORDER-LEFT-WIDTH: 0px; MARGIN-RIGHT: auto" title="New Windows 7 Defrag User Interface" border=0 alt="New Windows 7 Defrag User Interface" src="http://blogs.msdn.com/blogfiles/e7/WindowsLiveWriter/DiskDefragmentationBackgroundandEngineer_CA52/image_thumb_3.png" width=653 height=521 mce_src="http://blogs.msdn.com/blogfiles/e7/WindowsLiveWriter/DiskDefragmentationBackgroundandEngineer_CA52/image_thumb_3.png"&gt;&lt;/A&gt; &lt;/P&gt;
&lt;P&gt;&lt;A href="http://blogs.msdn.com/blogfiles/e7/WindowsLiveWriter/DiskDefragmentationBackgroundandEngineer_CA52/image_10.png" mce_href="http://blogs.msdn.com/blogfiles/e7/WindowsLiveWriter/DiskDefragmentationBackgroundandEngineer_CA52/image_10.png"&gt;&lt;IMG style="BORDER-RIGHT-WIDTH: 0px; DISPLAY: block; FLOAT: none; BORDER-TOP-WIDTH: 0px; BORDER-BOTTOM-WIDTH: 0px; MARGIN-LEFT: auto; BORDER-LEFT-WIDTH: 0px; MARGIN-RIGHT: auto" title="New Windows 8 Defrag User Interface" border=0 alt="New Windows 8 Defrag User Interface" src="http://blogs.msdn.com/blogfiles/e7/WindowsLiveWriter/DiskDefragmentationBackgroundandEngineer_CA52/image_thumb_4.png" width=654 height=522 mce_src="http://blogs.msdn.com/blogfiles/e7/WindowsLiveWriter/DiskDefragmentationBackgroundandEngineer_CA52/image_thumb_4.png"&gt;&lt;/A&gt; &lt;/P&gt;
&lt;P mce_keep="true"&gt;&amp;nbsp;&lt;/P&gt;
&lt;P&gt;In Windows XP, defragmentation had to be a user-initiated (manual) activity i.e. it could not be scheduled. Windows Vista added the capability to schedule defragmentation – however, only one volume could be defragmented at any given time. Windows 7 removes this restriction – multiple volumes can now be defragmented in parallel with no more waiting for one volume to be defragmented before initiating the same operation on some other volume! The screen shot below shows how defragmentation can be concurrently scheduled on multiple volumes: &lt;/P&gt;
&lt;P&gt;&lt;A href="http://blogs.msdn.com/blogfiles/e7/WindowsLiveWriter/DiskDefragmentationBackgroundandEngineer_CA52/image_12.png" mce_href="http://blogs.msdn.com/blogfiles/e7/WindowsLiveWriter/DiskDefragmentationBackgroundandEngineer_CA52/image_12.png"&gt;&lt;IMG style="BORDER-BOTTOM: 0px; BORDER-LEFT: 0px; DISPLAY: block; FLOAT: none; MARGIN-LEFT: auto; BORDER-TOP: 0px; MARGIN-RIGHT: auto; BORDER-RIGHT: 0px" title="Windows 7 Defrag Schedule" border=0 alt="Windows 7 Defrag Schedule" src="http://blogs.msdn.com/blogfiles/e7/WindowsLiveWriter/DiskDefragmentationBackgroundandEngineer_CA52/image_thumb_5.png" width=450 height=340 mce_src="http://blogs.msdn.com/blogfiles/e7/WindowsLiveWriter/DiskDefragmentationBackgroundandEngineer_CA52/image_thumb_5.png"&gt;&lt;/A&gt; &lt;/P&gt;
&lt;P&gt;&lt;A href="http://blogs.msdn.com/blogfiles/e7/WindowsLiveWriter/DiskDefragmentationBackgroundandEngineer_CA52/image_14.png" mce_href="http://blogs.msdn.com/blogfiles/e7/WindowsLiveWriter/DiskDefragmentationBackgroundandEngineer_CA52/image_14.png"&gt;&lt;IMG style="BORDER-BOTTOM: 0px; BORDER-LEFT: 0px; DISPLAY: block; FLOAT: none; MARGIN-LEFT: auto; BORDER-TOP: 0px; MARGIN-RIGHT: auto; BORDER-RIGHT: 0px" title="Windows 7 Defrag Disk Selection" border=0 alt="Windows 7 Defrag Disk Selection" src="http://blogs.msdn.com/blogfiles/e7/WindowsLiveWriter/DiskDefragmentationBackgroundandEngineer_CA52/image_thumb_6.png" width=461 height=392 mce_src="http://blogs.msdn.com/blogfiles/e7/WindowsLiveWriter/DiskDefragmentationBackgroundandEngineer_CA52/image_thumb_6.png"&gt;&lt;/A&gt; &lt;/P&gt;
&lt;P&gt;Among the other changes under the hood in Windows 7 are the following:&lt;/P&gt;
&lt;UL&gt;
&lt;LI&gt;Defragmentation in Windows 7 is more comprehensive – many files that could not be re-located in Windows Vista or earlier versions can now be optimally re-placed. In particular, a lot of work was done to make various NTFS metadata files movable. This ability to relocate NTFS metadata files also benefits volume shrink, since it enables the system to pack all files and file system metadata more closely and free up space “at the end” which can be reclaimed if required. &lt;/LI&gt;
&lt;LI&gt;If solid-state media is detected, Windows disables defragmentation on that disk. The physical nature of solid-state media is such that defragmentation is not needed and in fact, could decrease overall media lifetime in certain cases.&lt;/LI&gt;
&lt;LI&gt;By default, defragmentation is disabled on Windows Server 2008 R2 (the Windows 7 server release). Given the variability of server workloads, defragmentation should be enabled and scheduled only by an administrator who understands those workloads.&lt;/LI&gt;&lt;/UL&gt;
&lt;P&gt;Best practices for using defragmentation in Windows 7 are simple – you do not need to do anything! Defragmentation is scheduled to automatically run periodically and in the background with minimal impact to foreground activity. This ensures that data on your hard disk drives is efficiently placed so the system can provide optimal responsiveness and I can continue to enjoy glitch free listening to the Eagles :-).&lt;/P&gt;
&lt;P&gt;Rajeev and Matt &lt;/P&gt;&lt;img src="http://blogs.msdn.com/aggbug.aspx?PostID=9375342" width="1" height="1"&gt;</description><category domain="http://blogs.msdn.com/e7/archive/tags/Storage/default.aspx">Storage</category></item></channel></rss>