Antimail : Storage

Script recipe: How to increase the maximum number of quotas in the FSRM Quota Report (W2K3 R2)

AdiOltean — Sat, 19 Sep 2009 01:45:00 GMT

One of the customer requests we had on Windows Server 2003 R2 was the ability to increase the maximum number of quotas in the FSRM Quota report. The current limit is 1000 quotas maximum. How do I set this limit to 20,000 for example?

Here is how - assuming that you have Powershell 1.0 installed, just run this script:

$gm=New-Object -com Srmsvc.SrmGlobalStoreManager

$s=[xml]$gm.GetStoreData("Settings", "ReportSettings")

$s.Save("\fsrm-reports-backup.xml")

$s.root.MaxQuotas="20000"

$gm.SetStoreData("Settings", "ReportSettings",$s.get_InnerXml())

One note, however. The script above uses an COM object that is internally used by FSRM to store its internal configuration. Use this COM object as your own risk as this is not a published API.

FCI – how to use classification (video)

AdiOltean — Thu, 28 May 2009 23:23:31 GMT

Just found a new video demonstrating the new classification feature in Windows Server 2008 R2. Enjoy!

Special thanks to dawho1 who posted the video.

Script of the week: how to expire files with Classification

AdiOltean — Fri, 15 May 2009 21:06:00 GMT

It is an eye opening experience to sit at our FCI booth and see customer after customer telling us their biggest problem with managing file servers today: lots of old data sitting on their file servers. When I tell them how that our classification feature solves this problem, it something that always brings a sincere smile on their face.

Creating a file expiration policy is super easy:

Open FSRM management console
Go to the "File Management Tasks" view
Create a file management task to expire your files. You need to specify the following settings:
- The source directory of files to be expired
- The target directory (containing expired files)
- A condition for expiration (such as files created ten years ago (or files that were not modified in the last year)
- A schedule (say, weekly)

Here is an example of the Action tab:

That's it. One simple task to solve the "old files lying around" problem.

Now, a note to be added: the effect of this command will be the to move these expired files into the target location (while trying to keep the original path). One effect though is that the original files will "disappear" from the original location, which in rare cases it might cause confusion to the end users. If you are concerned about that problem, the solution is easy: as part of the file management task you can run your own custom "move" command which leaves in the original path a "stub" (a text file) explaining where the files have gone. Or, you can replace the original files with a symbolic link (or some other form of link) pointing to the target location.

To do this, you need to do a few things:

In the File Management Task dialog, in the Action tab, change the task type from "File expiration" to "Custom". Several more options appear (such as the path to the custom script, the account the script will run under, etc)
Add the [Source File path] macro to the script arguments
Run the command as "local system" (so it will be able to perform the move operation).

Here is an example of how the task will look like:

The move_file_and_leave_link.cmd (located in c:\windows\system32) file is simple:

if exist "c:\protected\%~pnx1" @echo Target file already exists! & goto :EOF
md "c:\HSM\%~p1"
move %1 "c:\protected\%~p1"
mklink %1 "c:\protected\%~pnx1"

The last command (mklink) has the role of creating a symbolic link from the source to the target. That’s it!

A simple way to access Shadow Copies in Vista

AdiOltean — Thu, 28 Feb 2008 21:37:18 GMT

In the past, I presented various ways to browse these mysterious device objects called "shadow copies". Shadow copies are static images in time (snapshots) of your volume contents, at some point in the past. These shadow copies are volumes on their own, with a file system namespace accessible through the regular Win32 APIs such as FindFirstFile/FindNextFile. For example the existing sample code in MSDN for these APIs that enumerates files on a real volume will work just fine on a shadow copy volume. In fact, that's how all backup applications are accessing shadow copy content today.

So, if these devices are real volumes, how can we view them in Explorer? It turns out that you can't view them by default - this is simply because these are volumes without an associated drive letter or root mount point. However, in XP or Windows Server (and Vista), you can still access these shadow copies by assigning them a drive letter using utilities like DOSDEV, or by doing tricks with the FOR command, etc.

Now, if you have Vista, it is much simpler to access shadow copy devices directly from Explorer. The trick is to use a new feature called Symbolic Links: to access the contents of a shadow copy as a "directory", simply create a symbolic link to the device. Vista also includes a convenient command-line tool called MKLINK.EXE to create symbolic links, which makes this operation very easy.

Here is an example of accessing the contents of a shadow copy device. The first step is to enumerate shadow copies on the machine, using the VSSADMIN LIST SHADOW command. This will give us the devices and also a creation timestamp.

C:\Windows\system32>vssadmin list shadows |more
vssadmin 1.1 - Volume Shadow Copy Service administrative command-line tool
(C) Copyright 2001-2005 Microsoft Corp.

Contents of shadow copy set ID: {c72c8036-d563-43c8-b351-1994dfad580a}
   Contained 1 shadow copies at creation time: 2/23/2008 9:59:04 AM
      Shadow Copy ID: {f3727808-bea6-4b59-bef7-6849ee721709}
         Original Volume: (C:)\\?\Volume{3e83355f-7c0e-11dc-b416-806e6f6e6963}\
         Shadow Copy Volume: \\?\GLOBALROOT\Device\HarddiskVolumeShadowCopy4
         Originating Machine: Adi-Game-PC
         Service Machine: Adi-Game-PC
         Provider: 'Microsoft Software Shadow Copy provider 1.0'
         Type: ClientAccessibleWriters
         Attributes: Persistent, Client-accessible, No auto release, Differential, Auto recovered

Contents of shadow copy set ID: {0bf23f77-8461-4869-b391-da4d213940a5}
   Contained 1 shadow copies at creation time: 2/24/2008 4:00:24 AM
      Shadow Copy ID: {87d59b22-9e84-4d0d-81ca-2b565d6f7e55}
         Original Volume: (C:)\\?\Volume{3e83355f-7c0e-11dc-b416-806e6f6e6963}\
         Shadow Copy Volume: \\?\GLOBALROOT\Device\HarddiskVolumeShadowCopy5
         Originating Machine: Adi-Game-PC
         Service Machine: Adi-Game-PC
         Provider: 'Microsoft Software Shadow Copy provider 1.0'
         Type: ClientAccessibleWriters
         Attributes: Persistent, Client-accessible, No auto release, Differential, Auto recovered

C:\Windows\system32>mklink /d c:\shadowcopy \\?\GLOBALROOT\Device\HarddiskVolumeShadowCopy5\
symbolic link created for c:\shadowcopy <<===>> \\?\GLOBALROOT\Device\HarddiskVolumeShadowCopy5\

C:\Windows\system32>dir c:\shadowcopy
Volume in drive C has no label.
Volume Serial Number is 4A02-860C

Directory of c:\shadowcopy

12/14/2007 01:46 AM    <DIR>          Asi
01/15/2008 12:56 AM    <DIR>          bin
12/13/2007 11:59 PM    <DIR>          debuggers
12/13/2007 11:55 PM        17,644,031 dir.log
01/14/2008 11:41 PM    <DIR>          Downloads
01/01/2008 05:50 PM    <DIR>          dumps
12/30/2007 11:43 PM    <DIR>          garbage
01/08/2008 11:13 PM    <DIR>          Garmin
10/15/2007 09:03 PM    <DIR>          Intel
12/30/2007 11:59 PM    <DIR>          Program Files
01/27/2008 01:32 AM    <DIR>          Program Files (x86)
01/15/2008 12:17 AM    <DIR>          test
01/30/2008 06:52 AM    <DIR>          Users
12/14/2007 01:55 AM    <DIR>          WinDDK
02/13/2008 05:23 AM    <DIR>          Windows
02/21/2008 10:43 PM    <DIR>          Work
               1 File(s)     17,644,031 bytes
              15 Dir(s) 147,657,666,560 bytes free

That's it. Now I have a persistent link called c:\shadowcopy which points to the contents of the shadow copy device - which is the image of my C:\ drive at 4:00 AM (this is when my latest system restore point was created).

A new notes,though:

1) Make sure you use the "/D" option in MKLINK so you create a directory-based, not a file-based symbolic link

2) Make sure you append a backslash to the shadow copy device in the MKLINK command (marked in red above)

If this made you interested about shadow copies - note that you can create, enumerate and delete shadow copies programatically using either VB scripts that use the WMI API for shadow copy administration, or by using the VSS API (documented publicly on MSDN). Sample code is available in the Platform SDK as well.

Mommy, Why is There a Server in the House?

AdiOltean — Wed, 09 Jan 2008 09:58:39 GMT

I love this - if you have kids, and if you are a geek, you'll understand.

http://gizmodo.com/photogallery/microserveces08/1000446145

Flash-based harddisks - the new laptop feature to look for

AdiOltean — Thu, 27 Dec 2007 10:33:04 GMT

A review in ComputerWorld features the new Samsung's 64 GB Flash drive:

Samsung rates the drive with a read speed of 100MB/sec and write speed of 80 MB/sec, compared to 59MB/sec and 60MB/sec (respectively) for a traditional 2.5" hard drive. We ran several benchmarks. In our HD Tune performance tests, the drive turned in an average transfer speed of 28.6 MB/sec with .3ms average access time. Burst speed was 25.5MB/sec with 4.5% CPU utilization. HD Tach tests using both 8MB and 32MB blocks for testing measured the burst speed at 30.8MB/sec, the average sequential read speed at 28.0MB/sec. HD Tach won't measure write speeds on primary drives, so we turned to Fresh Diagnose, which reported an average write speed of 11.65 MB/sec.

With such a spectacular performance, it's easy to see why Flash drives will become the next top-of-the-line feature in any high-end laptop, maybe more important than a high-speed CPU.

Is my harddisk (almost) dead?

AdiOltean — Tue, 13 Nov 2007 21:03:30 GMT

I just started to play with HD Tune, just to get a tool to look at my S.M.A.R.T. data. Not that I trust SMART a lot, but I wanted to see what's there.

So that's how I discovered my main drive has a bunch of reallocated sectors. Ouch!

What is a reallocated sector? Whenever a sector becomes "bad" for some reason, modern harddisks are "remapping" this sector to some other location of the disk. That's why you almost never see "bad clusters" on a modern harddisk. See for example the result of the CHKDSK command on the same drive - it says that everything is fine:

C:\Windows\system32>chkdsk c:
The type of the file system is NTFS.

WARNING! F parameter not specified.
Running CHKDSK in read-only mode.

CHKDSK is verifying files (stage 1 of 3)...
118464 file records processed.
File verification completed.
129 large file records processed.
0 bad file records processed.
2 EA records processed.
75 reparse records processed.
CHKDSK is verifying indexes (stage 2 of 3)...
460369 index entries processed.
Index verification completed.
5 unindexed files processed.
CHKDSK is verifying security descriptors (stage 3 of 3)...
118464 security descriptors processed.
Security descriptor verification completed.
16840 data files processed.
CHKDSK is verifying Usn Journal...
35955328 USN bytes processed.
Usn Journal verification completed.
Windows has checked the file system and found no problems.

244196351 KB total disk space.
104969600 KB in 101394 files.
     50680 KB in 16841 indexes.
         0 KB in bad sectors.
    227651 KB in use by the system.
     65536 KB occupied by the log file.
138948420 KB available on disk.

      4096 bytes in each allocation unit.
61049087 total allocation units on disk.
34737105 allocation units available on disk.

There are two problems with reallocated sectors. First, each harddisk comes with a limited "pool" of empty sectors that can be used as reallocated sectors. When you run out of those, then the automatic protection goes away, so you will start seeing more and more bad sectors at the OS level.

Second (and most importantly) there is a performance problem with reallocated sectors. Due to the fact that some sectors are remapped to another area on the disk, sequential I/O on those sectors is getting randomized (becomes random I/O) with very different performance characteristics. How big is the performance impact? Pretty big. Let's say that you have a no reallocated sectors in a 40 MB interval. If you want to read 1 MB in this interval, you will read it at a standard sequential I/O rate, say about 50 MB/s on a regular SATA disk. So reading will take 1/50 = 20 ms.

Now, let's pretend we have a reallocated sector in the middle of the 1 MB that we want to read. The reading time will include those 20 ms above plus the two additional seeks. Since a seek is usually around 8-9 ms (or even more) for a standard SATA harddisk, you get around 40 ms for reading the 1 MB which is twice as long. So, from a bandwidth perspective, you are reading that 1 MB at 25 MB per second, which half of the actual speed. So what might see is a sudden decrease in sequential I/O bandwidth whenever there is a reallocated sector around.

The interesting thing is that you can "spot" the approximate location of reallocated sectors by doing a sequence of sequential reads over the entire harddisk (from the beggining to end) and see where you have sudden drops in I/O performance. Fortunately, the same tool mentioned above - HD Tune - has a benchmark mode which allows precisely this.

Here is the picture of my backup drive (a Hitachi 250 GB drive):

You can see that the transfer rate decreases smoothly from 60 MB/s (near to the outer region of the disk) to 30 MB/s (near to the center of the disk). There are no sudden drops. In contrast, here is a disk with a lot of reallocated sectors:

You can see a lot of drops along the blue sequential read path, with a variability of about 50% in some cases.

Note - if you are doing these sequential I/O tests, make sure that your harddisk is not in use by any other applications or the page file - this will cause additional seeks which would "pollute" the original graph.

For that reason, I haven't run yet the benchmark on my main drive since I know that paging will distort the results - I'll try this later ...

What causes Harddisk failures?

AdiOltean — Tue, 13 Nov 2007 10:48:20 GMT

A fascinating article in ACM Queue examines the various ways in which a harddisk can fail.

Here is an example - the fault tree for reading failures.

Toshiba doubles hard-disk density

AdiOltean — Sun, 09 Sep 2007 08:09:52 GMT

Just when we thought that there is no room more room on those little bits on a harddisk, Toshiba doubles their density again: http://www.newswire.ca/en/releases/archive/September2007/06/c5819.html

From their press release:

Toshiba Corporation today announced a prototype hard disk drive (HDD) that uses Discrete Track Recording (DTR) technology to boost capacity to a record-breaking 120 gigabytes (GB) on a single 1.8-inch platter. The drive is the first in the world to apply DTR, a breakthrough technology that boost the areal density of a perpendicular magnetic recording (PMR) by a full 50 percent. Toshiba plans to start mass production of HDDs integrating DTR technology in 2009.

Windows Home Server - links

AdiOltean — Fri, 16 Mar 2007 01:43:27 GMT

A friend of mine didn't know about Windows Home Server - so I sent him these links:

- Main site: http://stopdigitalamnesia.com

- Interview with Charlie Kindel http://channel9.msdn.com/ShowPost.aspx?PostID=270965

- Download: https://connect.microsoft.com/site/sitehome.aspx?wa=wsignin1.0&siteid=38

- Articol on Wikipedia: http://en.wikipedia.org/wiki/Windows_Home_Server

- Blog: http://blogs.technet.com/homeserver

- Press release: http://www.microsoft.com/Presspass/press/2007/jan07/01-08WindowsHomeServerPR.mspx

Demo: New virtualization technologies in Windows Server (Longhorn)

AdiOltean — Fri, 02 Mar 2007 23:32:16 GMT

A nice demo that shows you several things:

- Windows Server virtualization (a separate partition running a striped-down version of Windows to manage all other virtual machines)

- Running an 8-proc virtual machine

- 64-bit and 32-bit machines running concurrently

- Managing virtual machines with System Center (nice UI)

- System Center Operations Manager: monitoring the VMs, provisioning new hardware in a scriptable manner.

Video: Longhorn - Windows Server Virtualization

Previous Versions in Vista - on Channel 9!

AdiOltean — Tue, 27 Feb 2007 05:29:48 GMT

Hot and fresh from Channel 9: a session on Shadow Copies in Vista, and its applications: Previous Versions and the brand-new System Restore.

Check it out.

The next generation of storage management is here!

AdiOltean — Thu, 07 Dec 2006 08:45:00 GMT

Storage management is complex, we all know it. For many people, even the simplest opeartions in a SAN look like black magic. It's no wonder why - if you want to, say, create a brand new 5 TB file share using your brand new your SAN box: you need to create a new LUN in your SAN, mounting it on a machine, initialize the disk, create partitions, assigning drive letters, create the file shares and assign proper permissions, etc.

The new Windows product launched today solves this problem. WUDSS offers simple, easy-to use wizards, for the complex operations above, plus of course, the famous Finish button. It's that simple!

And, yes, as you could guess, the main theme is simplifying and integrating all storage management scenarios (at least for file servers) in a single management UI:

- A new out-of-the-box experience (OOBE) that allows you to "initialize" your brand new shiny storage box with a few mouse clicks.

- Combined file, print and block-level storage services in a single management interface

- Easy remote management through a single HTTP URL

- New cluster configuration wizard that vastly simplifies provisioning and creation of shared disk resources.

The product will ship on Dell's new PowerVault NX 1950 hardware initially, and other OEMs will incorporate it in other NAS boxes.

P.S. WUDSS stands for Windows Unified Data Storage Server 2003 - a long name, in our established Microsoft tradition :-)

Datacenter in a box

AdiOltean — Wed, 18 Oct 2006 08:41:00 GMT

This is an interesting concept. This datacenter looks like a simple cargo container that can be deployed anywhere.

To deploy their cluster, all you need is:

1) A 500 kW electricity source

2) Good network connections.

3) A hose with cooling water

4) A truck to move the container to the desired location

5) And, of course a $500,000 cashier check to SUNW.

: http://www.nytimes.com/2006/10/17/technology/17sun.html?ex=1318737600&en=14da430369cdde55&ei=5088&partner=rssnyt&emc=rss

MENLO PARK, Calif., Oct. 16 — Sun Microsystems has developed a novel “data center in a box” in an effort to transform the fundamental economics of corporate computing.

The expandable computer system, called Project Blackbox, is based on a standard 20-foot shipping container and can be deployed virtually anywhere there is electricity, chilled water and an Internet connection.

Sun will introduce a prototype at its headquarters here on Tuesday. The system is planned for commercial availability in the second half of 2007, with prices beginning around $500,000.

[...]

On Monday, he gave a reporter a tour of the prototype system, which sits in a container case adjacent to a Sun office building here, connected to two large fire hoses for water cooling and 500 kilowatts of redundant power.

Painted black with a lime green Sun logo, the system can consist of up to seven tightly packed racks of 35 server computers based on either Sun’s Niagara Sparc processor or an Opteron chip from Advanced Micro Devices. The system includes sensors to detect tampering or movement and features a large red button to shut it down in an emergency. Once plugged in, it requires just five minutes to be ready to run applications.

The new must-have gadget: Flash laptops

AdiOltean — Fri, 13 Oct 2006 23:31:49 GMT

The entire world is now slowly shifting to Flash-based storage in mobile devices. The latest trend is to get rid of harddisks in your laptop - and you will have longer battery life, faster shutdown/reboot, better shock-resistance. Not to mention that flash memory takes less space than a harddisk.

Fujitsu follows this trend with their NAND-convertible Lifebook laptops:

Fujitsu is estimating that it will receive roughly 20,000 orders for its new NAND-convertible Lifebook laptops through March. If 20 percent of those orders specify a flash drive, as Yamamoto predicts, that figure will amount to 4,000 computers with flash.

Replacing the standard hard disk drive on Fujitsu's new laptops with a flash drive would make the laptops lighter and more shock-resistant, extend battery life by 30 minutes and cut in half the time required to start up Windows. But replacing a 20GB hard drive with a 16GB worth of flash memory would cost an extra $670 (80,000 yen). That figure would be $1,335 to switch to a 32GB flash drive.

"The price is high, but we judged there is adequate demand from companies for NAND PCs, which greatly reduce the risk of losing data," Yamamoto said. "It may be a slow start, but we want to be ready when laptops will be as handy as mobile phones."

[source: CNet]