Monad

Please point your RSS readers to http://blogs.msdn.com/PowerShell/

Monad Team — Tue, 25 Apr 2006 20:27:00 GMT

Along with our new name comes a new team blog: http://blogs.msdn.com/PowerShell/.

Please subscribe to that blog instead -- as this blog will now go dark.

Lee

Windows PowerShell (Monad) Has Arrived

Monad Team — Tue, 25 Apr 2006 19:48:00 GMT

I just got out of Bob Muglia's keynote speech where he formally announced Windows PowerShell (previously known as Monad). The key things he announced are:

Monad (MSH) has a new name: Windows PowerShell
Windows PowerShell will RTW (Web download) in Q4 2006
RC1 of Windows PowerShell is available today
Exchange 2007 and MOM will be built upon Windows PowerShell
The next wave of products will be leverage the same architecture - Admin GUI layered on Windows PowerShell

Superstar Vivek Sharma demonstrated the Exchange 2007 Admin experience. First he showed their new MMC 3.0 managed code snap-in - it totally rocks! (you'll be amazed by what you can do with MMC 3.0). He showed an example of adding a distribution list using the GUI. At the end of this wizard, it showed the results of the operation and the Command equivalent. This highlighted the fact that the Exchange GUI is layered on top of PowerShell meaning that there is parity between the GUI and the CLI and that the GUI helps teach the CLI. He then opened a PowerShell window and typed the Cmdlet name using tab-completion and hit enter, this then prompted him for all the necessary information to complete the task showing you PowerShell helps guide the user to successful complete the task. He then showned a number of features of the Exchange cmdlets highlighting the ability to safely manage a large number of objects in a single command line. He concluded by showing a command sequence which leveraged a Exchange 2007 cmdlet piped to a Monad Cmdlet piped to a Community script to generate a graphical chart report of Exchange data. It was a very cool demo and was very well received.

Here are the ramifications of this announcement:
Windows PowerShell is real and you can count on it shipping. Most of you already knew that but some people just won't talk to you until you have a ship date and ship vehicle. We have those now. Everyone can start making concrete plans for how and when you'll support, leverage and/or use Windows PowerShell.

Windows PowerShell is ready for the big leagues. Exchange 2007 is a major league, high volume, high stress, production application and it is using Windows PowerShell as the core for its entire admin experience. We have worked closely with that team to ensure that Windows PowerShell V1.0 has the features and quality necessary to make that a safe bet. Exchange has been hardcore about the functional and operation requirements of their customers. This has driven PowerShell and resulted in an unusually robust and functional V1.0 release. You too can expose your admin experience via Windows PowerShell and deliver your customer's an awesome admin experience. If you are an admin, you should start asking your vendors when they'll support PowerShell.

You can use RC1 of Windows PowerShell today. RC1 is in great shape. We changed the name of the exe to PowerShell.exe and the extension is now .PS1 . We also did a big consistency pass on the cmdlets and their parameters make a number of changes along with those required to support the new name. We release that this means that you'll have to relearn some commands and perhaps rework some scripts - apologizes for that but in the long turn, these changes really improve the consistency of the surface - a virtue that will pay back year and year. There are also a number of bug fixes and a few DCRs. The one you'll all go goo-goo over is Tab Completion. You can now Tab-Complete command names, parameter names, filenames, and variable properties/methods. Tab-Completion is now a user-definable function so if you don't like our implementation, you can replace it with one of your choosing. Pick up RC1 today. Here are the links:
- X86 Windows PowerShell latest build: http://go.microsoft.com/fwlink/?linkid=64772&clcid=0x409
- X64 Windows PowerShell latest build: http://go.microsoft.com/fwlink/?linkid=64773&clcid=0x409
- Windows PowerShell documentation: http://go.microsoft.com/fwlink/?linkid=64774&clcid=0x409

Side Note: The names of many of the APIs have changed to reflect the new name (basically you'll have to search and replace MSH with PS). We're sorry about the inconvenience of this, we always expected the official name to be the Microsoft Shell or Management Shell so we felt confident that MSH was safe to use. When Marketing saw what the technology actually did and the incredibly positive reaction that customers and partners, they decided that we warranted a "Marquee" name (I'm not making this up). Marquee names are given to those features that are going to be emphasized during the Marketing push. So here is the net of all that: the downside is that you've got to change your API calls, the upside is that you are leveraging a technology that Microsoft Marketing plans to feature widely.

We have a new blog site to befit our new name: http://blogs.msdn.com/powershell/default.aspx

This is a pretty big day for the community.

Enjoy!
jps

Exposing the Power of .NET in a Admin-friendly way

Monad Team — Mon, 24 Apr 2006 16:33:00 GMT

One of our primary goals for Monad was to: "Expose the power of .NET in an Admin-friendly way".

The challenges of systems administration are large and growing at the same time organizations are under ever more pressure to reduce costs and do more with less. That was true 10 years ago, it is true now, it will be true 10 years from now.

One of the key strategies for dealing with this is leverage the power of communities by participating in user/news groups where you can give and get help and share scripts for solving problems. As such, we wanted an environment that provided the widest possible dynamic range so that a large and diverse community of users could come together and help one another. Monad is designed to meet the needs of admins, beginning scripters, advanced scripters and programmers. The other benefit of this approach is that it allows people to start off as admins and migrate to whatever level of programming they want to. I fully expect Monad to be the vector that some Admins will use to make a career change into software engineering. I also expect things to go in the other direction as well as is the case in Unix environments where the line between programmers and admins can get quite fuzzy.

Here is how you should think about this issue: Most programs/scripts perform 3 activities:

Get Objects
Process Objects
Generate Output

For each of these areas, Monad provides 3 modes of access along a spectrum of Programmer-Oriented to Admin-Oriented.

Raw .NET access
Enhanced .NET access
Command access

Monad provides a sort of Rubic's cube model where you can choose whatever access mode you want for whatever activity you want. You can use Raw .NET access to get objects, enhanced .NET access to process those objects and then Command access to generate output. I won't go through all the combinations but you see what I'm getting at? You can do everything at one mode or mix and match to your hearts content.

Raw .Net Access
Monad allows you to write programs that get/process/output raw .net objects just like any other .NET language. Here is an example written in C# and then in Monad:
C#
Process myProcess = new Process();
try
{
    string myDocumentsPath =
      Environment.GetFolderPath(Environment.SpecialFolder.Personal);

    myProcess.StartInfo.FileName = myDocumentsPath + "\\MyFile.doc";
    myProcess.StartInfo.Verb = "Print";
    myProcess.StartInfo.CreateNoWindow = true;
    myProcess.Start();
}
catch (ComponentModel.Win32Exception e)
{
    Console.WriteLine(e.Message + ". Check the path.");
}

Monad
[System.Diagnostics.Process]$myProcess = new-object System.Diagnostics.Process;
trap [ComponentModel.Win32Exception] {
[Console]::WriteLine("Error: " + $_.Error);
}
$myDocumentsPath = [Environment]::GetFolderPath("Personal");
$myProcess.StartInfo.FileName = $myDocumentsPath + "\\MyFile.Doc";
$myProcess.StartInfo.Verb = "Print";
$myProcess.StartInfo.CreateNoWindow = $true;
$myProcess.Start()

That is a small code fragment but you'll see the same thing if you use a larger program, you can pretty much code C# or VB.NET using Monad (in the same way you can code Fortan using C and C using C++). That is great but if you just wanted to raw .NET access, you are probably better off just using C# or VB.Net. Monad provides raw .NET access to establish a rockbed of function that ensures that you'll always be able to get the job done. If you are coding at a higher level and the function you need is not available, you don't have to throw away your work and start over, you can always access the raw .NET layer and do anything.

Anyone that has done scripting for very long knows exactly what I mean - there are always times when you need to turn the dial to 11 and when you can't - the situation turns gruesome quickly.

Enhanced .NET access
Monad provides gobs of features that enhance and simplify the access to .NET. I'm sure that entire books will be dedicated to just this one topic so I can't hope to do much here beyond giving a few examples.

In the area of enhancing getting objects, Monad has a number of accelerators for supporting object types widely used in scripting like [xml], [array], [regex], [scriptblock], [hashtable] (in addition to the obvious ones like [int], [string], [double], etc). In addition to syntactic support for creating these types of objects, Monad has a number of built in conversion rules which simplify getting the object you need when you need it.

Monad's synthetic type system also makes it easier to get objects than raw .NET access. This is used to normalize object naming (e.g. Monad provides the NAME property alias for process which maps to PROCESSNAME and ServiceController which maps to SERVICENAME). This is also used to increase the granularity/usefulness of raw .NET objects. For instance on the Process object, we add the properties DESCRIPTION, FILEVERSION, PRODUCT, and PRODUCTVERSION. All of these properties are available from the process object but to get them you have to navigate through the MainModule.FileVersionInfo path. i.e.
$p.PRODUCTVERSION == $p.MainModule.FileVersionInfo.PRODUCTVERSION
Another way of saying this is that .NET objects are often designed for the needs of Programmers not Admins and Monad has capabilities to tweak those objects so they meet the needs of Admins.

Monad enhances .NET objects to make it easier to process those objects as well. Take the following simple expression which tells you how long until Christmas (My daughter's favorite question). Let's make it a little more precise since what she really cares about is when she can open presents and we've established a firm rule that the kids have to stay in bed until at least 7am on Christmas (established after the 4am wake up incident). Here is the code I run whenever she asks that question:

MSH> [DateTime]"12/25/2006 7:00" - [DateTime]::now

Days              : 245
Hours             : 11
Minutes           : 4
Seconds           : 11
Milliseconds      : 359
Ticks             : 212078513593750
TotalDays         : 245.461242585359
TotalHours        : 5891.06982204861
TotalMinutes      : 353464.189322917
TotalSeconds      : 21207851.359375
TotalMilliseconds : 21207851359.375

This is possible because many .NET types can manipulate strings to create types. This is an example of where leveraging .NET makes it dramatically easier for Admins to do hard things. In this case, DateTime takes that string and makes the following data available:

MSH> [DateTime]"12/25/2006 7:00" |fl *

DateTime    : Monday, December 25, 2006 7:00:00 AM
Date        : 12/25/2006 12:00:00 AM
Day         : 25
DayOfWeek   : Monday
DayOfYear   : 359
Hour        : 7
Kind        : Unspecified
Millisecond : 0
Minute      : 0
Month       : 12
Second      : 0
Ticks       : 633026268000000000
TimeOfDay   : 07:00:00
Year        : 2006

Many .NET types can do equally powerful things, saving users TONS of effort by parsing strings and making the intersesting elements available as properties and/or exposing powerful methods. The problem is that some .NET types do this by taking a string constructor, some do it by having a static PARSE() method on the class, others do it by having a CONVERTER for Strings. Admins don't want to deal with that so Monad takes care of this mechanical housekeeping for the user. We will look for the appropriate constructor, parse method, or converter necessary to implement the user intentions. For instance, [DateTime]"12/25/2006" uses the Parse() method whereas [System.Diagnostics.EventLog]"Application" uses a string constructor.

I mentioned that the .NET type then exposes a set of great methods for the user. I have relatives in the UK so if we want to ship presents to those kids, they need to get in the mail early. So let's just say that things need to be shipped 35 days in advanced - when do they need to get in the mail? I have no clue but luckily .NET is smarter than I am so I can do the following:

MSH> ([datetime]"12/25/2006").AddDays(-35)

Saturday, November 20, 2006 12:00:00 AM

So that is great if you knew that the .NET object had a AddDays method and that you could provide it a negative number to do subtraction. On the other hand, enhanced .NET access is event smarter than raw .NET access so you can just do the following:
MSH> [datetime]"12/25/2006" - "35.0:00"

Monday, November 20, 2006 12:00:00 AM

Monad takes advantage of the fact that the .NET DateTime type support the Op_Subtraction() method. Monad determines that DateTimes can subtract [System.TimeSpan] types so it automatically converts the string "35.0.00" into this format and calls that method. The result of all this is that admins get to think and type this:
   [datetime]"12/25/2006" - "35.0:00"
Instead of this:
   [DateTime] $date    = [DateTime]::Parse("12/25/2006 7:00")
   [TimeSpan] $t       = [TimeSpan]::Parse("35.0:00")
   [DateTime] $newdate = [DateTime]::Op_Subtraction($date, $t)
   [Console]::WriteLine($newdate.ToString())

When it comes to outputing objects, Monad allows you to enhance the .NET objects by defining overrides for their ToString() method, grouping the objects properties into named propertysets, and to define which of the objects properties are the default display properties.

Command Access
Command access is the most admin-friendly way to interact with the system. Commands exist because someone thought about the needs of the Administrator and coded a good user experience on top of the .NET objects. Let me give you some examples of the what that means (this is not a complete list):

Consistent naming of the commands and its parameters
Support for wildcards when specifying objects
Support for -Whatif -Confirm for actions that have side effects
Produce great errors with all the info necessary to recover from the error
Distinguish between terminating and non-terminating errors
Emit information on output, verbose, debug, and error streams
Issue progress records if an operation takes more than a second or two
Design the a set of commands and their outputs so that they can easily pipeline together (object properties with names and types that match command parameters)
Specify validation metadata on parameters so that the Monad engine
Provide meta data to drive the formating of your output objects
Provide great help with detailed descriptions and lots of examples

This sounds like a lot and while there is a lot to get right when it comes to providing the right user experience, the amount of code to do this is very small because this is the design center for Monad's SDK. Most cmdlets are extremely small making them one of the best bargains in the industry. In fact, that is the way we define what a Monad Cmdlet is: the very thin layer on top of .NET necessary to provide a great user experience.

Getting objects through the command access is pretty straight forward. It is typically done with a Get-XXX command. These commands often provide a rich set of functions for getting the right objects. Many support rich wildcarding (e.g. Get-Process [a-t]*[g-v] ) and provide excellent error messages and error handling if and when something goes wrong. The other way to get objects is to expose a datastore as a system drive and then Monad provides common set of commands to naviagate the datastore and get objects. (e.g. CD xxx; get-item; get-itemproperty, get-childitem, get-content, etc).

Manipulating objects through commands is done through a rich set of object-based utility commands like: Where, Group, Measure, Tee, Sort, and Compare.

Lastly, there are a wide range commands for outputing results including: Export-CSV, Export-CliXml, Out-File, Out-Printer, Out-Default, Out-String, Out-Host, Format-List, Format-Table, Format-Wide, Format-custom, Write-Object, Write-Progress, Write-Error, Write-Log, Write-Debug, Write-Warning, Write-Host.

So to sum up, users write scripts/programs that get objects, process objects and generate results. Monad provides 3 modes of doing each of these steps: raw .NET access, enhanced .NET access and Command access. Users can mix and match modes to their hearts content. This architecture:

Allows users to pick the programming style that matches their thinking style.
Allows the Monad engine to eliminate the programming housekeeping hassles so you can focus on problem you want to solve not HOW you are going to solve it.
Ensures full and complete functional coverage so that you aren't stuck if a function you want isn't available at the abstraction you want it.
Allows admins, basic scripters, advanced scripters and programmers to form a large common community to share information and scripts.
Allows individuals to migrate to whatever level of programming they are interested in or is required by the problem at hand.

Man - I just love this stuff!

Enjoy!
Jeffrey Snover
Monad Architect

Comparative Examples in MSH and KSH

Monad Team — Sat, 22 Apr 2006 02:36:00 GMT

Most shells (such as Windows CMD.EXE and the UNIX shells SH, KSH, CSH, and BASH) operate by executing a command or utility in a new process, and presenting the results (or errors) to the user as text. Text-based processing is the way in which system interaction is done with these shells. Over the years, a large number of text processing utilities—such as sed, AWK, and PERL—have evolved to support this interaction. The heritage of this operational process is very rich.

MSH is very different from these traditional shells. First, this shell does not use text as the basis for interaction with the system, but uses an object model based on the .NET platform. Second, the list of built-in commands is much larger; this is done to ensure that the interaction with the object model is accomplished with the highest regard to integrity with respect to interacting with the system. Third, the shell provides consistency with regard to interacting with built-in commands through the use of a single parser, rather than relying on each command to create its own parser for parameters.

This document constrasts the Korn Shell (KSH) and MSH by providing a number of examples of each.

Example 1

To stop all processes that begin with the letter "p" on a UNIX system, an administrator would have to type the following shell command line:

$ ps -e | grep " p" | awk '{ print $1 }' | xargs kill

The ps command retrieves list of processes and directs (|) the grep command to determine which process begins with "p"; which in turn directs (|) the awk command to select the 1st column (which is in the process id) and then passes (|) those to the xargs command which then executes the kill command for each process. The fragility of this command line may not be evident, but if the ps command behaves differently on different systems, where the "-e" flag may not be present, or if the processed command is not in column 1 of the output, this command-line procedure will fail.

MSH:

MSH> get-process p* | stop-process

Thus, the command line may now be expressed as "get the processes whose name starts with "p" and stop them". The get-process Cmdlet takes an argument that matches the process name; the objects returned by get-process are passed directly to the stop-process Cmdlet that acts on those objects by stopping them.

The second, more convoluted example, which stops processes that use more than 10 MB of memory becomes quite simple.

Example 2

Another, even more complex example, such as "find the processes that use more than 10 MB of memory and kill them" can lead to an equally failed outcome:

$ ps -el | awk '{ if ( $6 > (1024*10)) { print $3 } }' | grep -v PID | xargs kill

The success of this command line relies on the user knowing that the ps -el command will return the size of the process in kilobytes (kb) in column 6 and that the PID of the process is in column 3. It is still required that the first row is removed.

Comparing Example 1 using a standard shell to Example 1a using MSH, we can see that the commands act against objects rather than against text.

MSH:

MSH> get-process | where { $_.VS -gt 10M } | stop-process

There is no issue about determining the column that contains the size of the process, or which column contains the ProcessID. The memory size may be referred to logically, by its name. The where Cmdlet can inspect the incoming object directly and refer to its properties. The comparison of the value for that property is direct and understandable.

Example 3

For example, if you wanted to calculate the number of bytes in the files in a directory, you would iterate over the files, getting the length and adding to a variable, and then print the variable:

$ tot=0; for file in $( ls )

> do

> set -- $( ls -log $file )

> echo $3

> (( tot = $tot + $3 ))

> done; echo $tot

This example uses the set shell command that creates numbered variables for each white space separated element in the line rather than the awk command as in the examples above. If the awk command were used, it would be possible to reduce the steps to the following:

$ ls –l | awk ‘{ tot += $5; print tot; }’ | tail -1

This reduces the complexity, but requires specific knowledge of a new language, the language that is associated with the awk command.

The MSH loop is similar; each file in the directory is needed, but it is far simpler as the information about the file is already retrieved:

MSH:

MSH> get-childitem | measure-object -Property length

The measure-object Cmdlet interacts with objects and if it is provided with a property from the object, it will sum the values of that property. Because the property length represents the length of the file, the measure-object Cmdlet is able to act directly on the object by referring to the property name rather than “knowing” that the length of the file is in column 3 or column 5.

Example 4

Many objects provided by the system are not static, but dynamic. This means that after an object is acquired, it is not necessary to acquire the object at a later time. The data in the object is updated as the conditions of the system change. Also, changes to these objects are reflected immediately in the system.

As an example, suppose one wanted to collect the amount of processor time that a process used over time. In the traditional UNIX model, the ps command would need to be run iteratively and the appropriate column in the output would need to be found and then the subtraction would need to be done. With a shell that is able to access the process object of the system, it is possible to acquire the process object once, and since this object is continually updated by the system; all that is necessary is to refer to the property. The following examples illustrate the differences, where the memory size of an application is checked in ten second intervals and the differences are output:

$ while [ true ]

msize1=$( ps -el | grep application | grep -v grep | awk '{ print $6 }' )

sleep 10

msize2=$( ps -el | grep application | grep -v grep | awk '{ print $6 }' )

expr $msize2 - $msize1

msize1=$msize2

done

MSH:

MSH> $app = get-process application

MSH> while ( 1 ) {

>> $msize1 = $app.VS

>> start-sleep 10

>> $app.VS - $msize1

>> }

Example 5

It is even more difficult to determine whether a specific process is no longer running. In this case, the UNIX user must collect the list of processes and compare them to another list.

$ processToWatch=$( ps -e | grep application | awk '{ print $1 }'

$ while [ true ]

> do

> sleep 10

> processToCheck=$(ps -e | grep application | awk '{ print $1 }' )

> if [ -z "$processToCheck" -or "$processToWatch" != "$processToCheck" ]

> then

> echo "Process application is not running"

> return

> fi

> done

MSH:

MSH> $processToWatch = get-process application

MSH> $processToWatch.WaitForExit()

As is seen in this example, the MSH user need only collect the object and then subsequently refer to that object.

Example 6

For example, suppose the user wanted to determine which processes were compiled as PreRelease code, such as when applications have been compiled in such a way to mark them as "PreRelease".

$ ???

This information is not kept in the standard UNIX executable. To determine this information, one would need to have a set of specialized utilities to add this information to the binary and then another set of utilities to collect this information. These utilities do not exist; it is not possible to accomplish this task.

MSH:

MSH> get-process | where {$_.mainmodule.FileVersioninfo.isPreRelease}

Handles NPM(K) PM(K) WS(K) VS(M) CPU(s) Id ProcessName

------- ------ ----- ----- ----- ------ -- -----------

643 88 1024 1544 15 14.06 1700 AdtAgent

453 15 25280 7268 199 91.70 3952 devenv

In this example, a cascade of properties is done. The appropriate property from the process object (MainModule) is inspected, the property "FileVersionInfo" is referenced (a property of MainModule) and the value of the property "IsPreRelease" is used to filter the results. If IsPreRelease is true, the objects that are output by the get-process Cmdlet are output.

Example 7

Each object may or may not provide methods; MSH provides commands to aid the discovery of methods that are available for a specific object via the get-member Cmdlet. For example, the string object has a large number of methods:

MSH:

MSH> get-member -input "String" -membertype method | format-table "$_"

"$_"

---------

System.Object Clone()

Int32 Compare(System.String, System.String, Boolean, System.Globalization.Cul..

Int32 CompareOrdinal(System.String, Int32, System.String, Int32, Int32)

Int32 CompareTo(System.String)

System.String Concat(System.String, System.String)

Boolean Contains(System.String)

System.String Copy(System.String)

Void CopyTo(Int32, Char[], Int32, Int32)

Boolean EndsWith(System.String, Boolean, System.Globalization.CultureInfo)

Boolean Equals(System.String, System.StringComparison)

System.String Format(System.String, System.Object, System.Object)

Char get_Chars(Int32)

Int32 get_Length()

System.CharEnumerator GetEnumerator()

Int32 GetHashCode()

System.Type GetType()

System.TypeCode GetTypeCode()

Int32 IndexOf(System.String, Int32)

Int32 IndexOfAny(Char[], Int32, Int32)

System.String Insert(Int32, System.String)

System.String Intern(System.String)

System.String IsInterned(System.String)

Boolean IsNormalized()

Boolean IsNullOrEmpty(System.String)

System.String Join(System.String, System.String[])

Int32 LastIndexOf(Char, Int32)

Int32 LastIndexOfAny(Char[], Int32)

System.String Normalize(System.Text.NormalizationForm)

Boolean op_Equality(System.String, System.String)

Boolean op_Inequality(System.String, System.String)

System.String PadLeft(Int32, Char)

System.String PadRight(Int32, Char)

System.String Remove(Int32)

System.String Replace(System.String, System.String)

System.String[] Split(System.String[], System.StringSplitOptions)

Boolean StartsWith(System.String)

System.String Substring(Int32)

Char[] ToCharArray(Int32, Int32)

System.String ToLower()

System.String ToLowerInvariant()

System.String ToString(System.IFormatProvider)

System.String ToUpper(System.Globalization.CultureInfo)

System.String ToUpperInvariant()

System.String Trim()

System.String TrimEnd(Char[])

System.String TrimStart(Char[])

As can be seen, 46 different methods are available to the string object all of which are available to the MSH user. Unfortunately, the semantics of these methods is not visible from the shell, but a number of .NET object help is available online.

Example 8

The availability of these methods creates an explosion of possibilities. For example, if I wanted to change the case of a string from lower to upper I would do the following: (first ksh and then MSH).

$ echo "this is a string" | tr [:lower:] [:upper:]

$ echo "this is a string" | tr '[a-z]' '[A-Z]'

MSH:

MSH> "this is a string".ToUpper()

The UNIX example relies on the tr cmdlet.

Example 9

For example, suppose the string "ABC" was to be inserted after the first character in the word "string" to have the result "sABCtring". Here are the following, first with ksh, then with MSH:

$ echo "string" | sed "s|$.$\(.*)|\1ABC\2|"

MSH:

MSH> "string".Insert(1,"ABC")

Both examples require specific knowledge; however, using the "insert" method is more intuitive than using the capture buffers available in sed. Moreover, the domain specific knowledge of the "sed" language required may be somewhat more advanced than is required to use the Insert method.

Jim Truher and Jeffrey Snover

Why did you do that? $VAR/ {} / Weak Intellisense

Monad Team — Mon, 17 Apr 2006 04:46:00 GMT

I strongly encourage people to let us know where we could be doing better and to let us know if we are getting into the weeds. I believe that being open to such bad news is core to the the virtuous cycle of self improvement so such feedback provides opportunities to reflect and make changes when appropriate. Other times we'll just agree to disagree. Occassionally I'll take some of that feedback and use it to explain what or why we do things.

Recently Raul from Switzerland took some time to document some of the things he thought we should be doing better. I thought I would discuss some of these here so that everyone could understand how we think about things. I asked Raul what 3 things he would change. He replied:

- OK, first get rid of the $-sign to denominate variables. We have come much further than that in the meantime. Low level programming languages like Visual Dialog Script (www.dialogscript.com), Kix, Pearl, or PHP need these parsing aids. We should not go back to that.

- Next, get rid of the horrific {$._} construct, not only because the curly braces are available only together with ALT-GR on most non-US keyboards. Furthermore, it is impossible to read (for non-programmers). Programmers are accustomed to such constructs and worse ones. But if you want to gain heart and mind of the administrators, that is not the way.

- Provide some kind of assistance when typing the dot after an object, simlar to Intellisense. If you want administrators NOT to read tons and tons of books, well, the only way is to provide online assistance, like Intellisense. Your presentation started gearing towards administrators but very shortly thereafter, there were hardcore programmers speaking to other hardcore programmers, in the most pure tech-talk. I assume, that you would have lost your non-hardcore programmers audience after 20-30 minutes.

The $ in front of variables comes from our design center of being an interactive command oriented shell. Interactive shells are NOT the same thing as Scripting languages with command interpretors. Interactive shells are case studies in engineering tradeoffs (another way of saying that is that they tend to be a little quirky). There is a tension between the needs of a great interactive experience and a great scripting/programming experience. Time and time again during the course of the project, people would try to make us "choose" between one or the other. We refused.

We believe that if you work hard enough at it, you can solve the puzzle and provide a great solution for both. We were absolutely subborn on this point. Time and time again, we were able to succeed with this approach if we travailed long enough. There has only been one issue where we could not optimize for both goals and had make a choice (using ">" to mean file redirection instead of "-gt").

Anyway, that is the reason why we use "$"s in front of variables. Bruce Payette is writting a book about Monad and goes into significantly more detail about the specifics of our decisions and why we made them (your going to love it).

"{}" vs BEGIN/END IF/FI WHILE/WEND DO/OD etc etc. This was resolved pretty quickly for us based upon 2 factors.

We looked at the all the mechanisms shells/scripts use and thought that most of the shell and script ones were .... well .... ummm .... inconsistent and not the strongest features of these environments.
We wanted to provide a smooth glide path between GUI, CLI, and C#.

As such, use of {} was an easy choice for us. Now let me say a couple more words about #2. Monad is NOT C#Script. It is a distinct environment focused on distinct user scenarios that are quite different than C#. The language and environment reflect that. When we designed the language we looked at a number of different environments (SH, PERL, RUBY, VMS/DCL, AS400/CL, and C#) and took inspiration from elements of many of these. We also wanted Monad to have the widest dynamic range possible. It should be easy to start using, easy to start scripting, easy to do sophisticated scripting and you should be able to use Monad to do most everything you want to do. That said, we know that at some point Monad will not meet your needs and you'll need to do systems programming (typically for perf reasons). When this occurs, we want to minimize your switching costs. This is why we use choose to use C# for many basic elements that do not conflict with our interactive scenarios. {} is just one of many. We think that this has worked out pretty well. Monad is a fantastic environment to explore .NET and while most of Monad is written using C#, a number of our devs do prototyping work in Monad itself and then translate it into C#.

Intellisense. I totally concur. The current system is relies too much on .NET objects and our tab completion is too weak. It is going to take a while before you can do everything using cmdlets (instead of .NET objects) however in the next drop (Next week), you'll see that tab completion gets much better. We'll provide tab completion for filenames, command names, command parameters, object properties and methods. Tab completion is actually a callout to a user-definable function so that the community can get as sophisticated as they'd like to. This feature has lagged because we have strongly focused on the command line experience and invested there instead of spending time on things like IDEs. We are relying on the community and third parties to fullfill this need in the V1 timeframe. That said, user feedback was strong enough on this point that we knew that we had do improve things for V1. We hope you like what you get next week.

As always, please let me know if this is not the case or if you think that there are things we should be doing better.

Raul - thanks for taking the time to organize and send me your feedback. I hope this blog provides some more context around why we made the decisions that we did.

Enjoy!

Jeffrey Snover
Monad Architect

Sunday morning drive with my hair on fire: Types/Community/Synthetic types/Democracy/Cool XML tricks

Monad Team — Sun, 16 Apr 2006 20:55:00 GMT

Exploring types is a pain!

Monad provides Get-Member which makes it pretty nice to explore an OBJECT but if you want to explore that object's type, you have to use the capabilities of the System.RuntimeType class.

Let me make that distinction a little clearer. Imagine that I've got some xml:
$x=[XML]"<a><b>TEST</b><a>"

I can explore this object using get-member (using the GM alias)

MSH> $x |gm
TypeName: System.Xml.XmlDocument

Name                        MemberType            Definition
----                        ----------            ----------
ToString                    CodeMethod            static System.St...
add_NodeChanged             Method                System.Void add_...
add_NodeChanging            Method                System.Void add_...
...

But now imagine that I want to exlore System.XML.XMLDocument ([XML]) itself? Here is what you get:

MSH> [xml]

IsPublic IsSerial Name                                     BaseType
-------- -------- ----                                     --------
True     False    XmlDocument                              System....

MSH> [xml] |fl *

Module                     : System.Xml.dll
Assembly                   : System.Xml, Version=2.0.0.0, Culture=neu
                             tral, PublicKeyToken=b77a5c561934e089
TypeHandle                 : System.RuntimeTypeHandle
DeclaringMethod            :
BaseType                   : System.Xml.XmlNode
UnderlyingSystemType       : System.Xml.XmlDocument
FullName                   : System.Xml.XmlDocument
AssemblyQualifiedName      : System.Xml.XmlDocument, System.Xml, Vers
                             ion=2.0.0.0, Culture=neutral, PublicKeyT
                             oken=b77a5c561934e089
Namespace                  : System.Xml
GUID                       : 3d63bf4b-edbe-36cb-b7b2-df87f38db07d
GenericParameterAttributes :
IsGenericTypeDefinition    : False
IsGenericParameter         : False
GenericParameterPosition   :
IsGenericType              : False
ContainsGenericParameters : False
StructLayoutAttribute      : System.Runtime.InteropServices.StructLay
                             outAttribute
Name                       : XmlDocument
MemberType                 : TypeInfo
DeclaringType              :
ReflectedType              :
MetadataToken              : 33554636
TypeInitializer            : Void .cctor()
IsNested                   : False
Attributes                 : AutoLayout, AnsiClass, Class, Public, Be
                             foreFieldInit
IsVisible                  : True
IsNotPublic                : False
IsPublic                   : True
IsNestedPublic             : False
IsNestedPrivate            : False
IsNestedFamily             : False
IsNestedAssembly           : False
IsNestedFamANDAssem        : False
IsNestedFamORAssem         : False
IsAutoLayout               : True
IsLayoutSequential         : False
IsExplicitLayout           : False
IsClass                    : True
IsInterface                : False
IsValueType                : False
IsAbstract                 : False
IsSealed                   : False
IsEnum                     : False
IsSpecialName              : False
IsImport                   : False
IsSerializable             : False
IsAnsiClass                : True
IsUnicodeClass             : False
IsAutoClass                : False
IsArray                    : False
IsByRef                    : False
IsPointer                  : False
IsPrimitive                : False
IsCOMObject                : False
HasElementType             : False
IsContextful               : False
IsMarshalByRef             : False

So that has LOTS of great information but the things that I usually want are the types Constructors, Properties, Methods, Interfaces etc. That should be easy right? The type has the method GetConstructors(). Lets see what that gives you.

MSH> [xml].GetConstructors()

Name                      : .ctor
MemberType                : Constructor
DeclaringType             : System.Xml.XmlDocument
ReflectedType             : System.Xml.XmlDocument
MetadataToken             : 100666404
Module                    : System.Xml.dll
MethodHandle              : System.RuntimeMethodHandle
Attributes                : PrivateScope, Public, HideBySig, SpecialN
                            ame, RTSpecialName
CallingConvention         : Standard, HasThis
IsGenericMethodDefinition : False
ContainsGenericParameters : False
IsGenericMethod           : False
IsPublic                  : True
IsPrivate                 : False
IsFamily                  : False
IsAssembly                : False
IsFamilyAndAssembly       : False
IsFamilyOrAssembly        : False
IsStatic                  : False
IsFinal                   : False
IsVirtual                 : False
IsHideBySig               : True
IsAbstract                : False
IsSpecialName             : True
IsConstructor             : True

Name                      : .ctor
MemberType                : Constructor
DeclaringType             : System.Xml.XmlDocument
ReflectedType             : System.Xml.XmlDocument
MetadataToken             : 100666405
Module                    : System.Xml.dll
MethodHandle              : System.RuntimeMethodHandle
Attributes                : PrivateScope, Public, HideBySig, SpecialN
                            ame, RTSpecialName
CallingConvention         : Standard, HasThis
IsGenericMethodDefinition : False
ContainsGenericParameters : False
IsGenericMethod           : False
IsPublic                  : True
IsPrivate                 : False
IsFamily                  : False
IsAssembly                : False
IsFamilyAndAssembly       : False
IsFamilyOrAssembly        : False
IsStatic                  : False
IsFinal                   : False
IsVirtual                 : False
IsHideBySig               : True
IsAbstract                : False
IsSpecialName             : True
IsConstructor             : True

OK - well that is cool and there may be times when I'd like those details (that hasn't happened up to this point in my life but I'm still reasonably young so I'm not ruling it out). What I want to know is what is the signature of the constructor? Well if you are smart enough to have an office near Bruce Payette, he may hear you ranting and come in to calm you down and inform you that the signatures area available when you do a ToString() so the way to get that is:

MSH> [xml].GetConstructors() |foreach {"$_"}
Void .ctor()
Void .ctor(System.Xml.XmlNameTable)

Fine. Now let's shift the discussion and talk about the power of community and the democratization of types.

Look, its easy to sit back and throw rocks saying - "that is a complex experience, those idiots should have forseen that and made it simple for us". That rock is completely fair. Now, if you go the next step and actually file a bug report, you might get this fixed in a couple of years.

We designed MONAD with this in mind. My customers don't have a couple of years to wait for the system to be fine tuned for their specific needs - my customers tend to run around with their hair on fire. They need solutions ASAP. One of the quickest ways to get a solution is to ask another smart person. In my case, I was lucky enough to have Bruce just down the hall but with the internet, Bruce is virtually just down the hall from everyone. (As a side note - I've just started to review a WONDERFUL book that Bruce is writing on Monad so in the future, you'll all be $30 away from having Bruce be in your very own home.)

"Yea - the internet is good and communities help get answers fast" - so what?. Well that's a pretty big deal but now let's turn our attention to the issue of what form community answers take and the differences in power these different forms take. For instance, if someone gives you some expository text describing the solution, that is useful but not nearly as useful as when someone gives you a piece of script that provides a solution.

Now imagine that this goes on year after year, developer by developer - Where is all that code? How do people discover it? How do you maintain it? If you've pasted the code into your script, then there is a good chance that you've pasted it lots of time in lots of scripts and you've got a maintence mess on your hands. Also - how does someone (say a new admin) find this? If you factored it out into it's own script then you still have a discoverability problem even though you've addressed the maintenance problem.

Let's step back and ask the question - why are we doing any of this anyway? The answer is that the developers that created System.RunTimeType did not implement that class in a way that meets our needs and they can't react quickly enough for us to wait. But in the end, the best of all solutions is for this function to be implemented as part of the type. That way when I ask the type what it can do, it will tell me directly. I don't have to go looking around in lots of scripts looking for whether there are additional functions layered on top.

Monad addresses this problem with a synthetic type system which allows the democratization of types. Sure the developers of System.RunTimeType have an important role to play but they are just one voice in this this choir. Lots of people should be able to extend the type with their own functions. Below is a contents of a file MyTypes.mshxml (attached as well). This file defines a number of new properties (Constructors, Methods, Properties, Interfaces, Events) for this type. Type types are implemented using Monad so they are called ScriptProperites and in each case, the call an underlying api (using the $THIS variable which is set to the object being extended), sorting the elements by name and piping the results into a foreach loop which casts the data to a string for easy reading.

<Types>
    <Type>
        <Name>System.RuntimeType</Name>
        <Members>
            <ScriptProperty>
                <Name>Constructors</Name>
                <GetScriptBlock>$this.GetConstructors() | foreach {"$_"}</GetScriptBlock>
            </ScriptProperty>
            <ScriptProperty>
                <Name>Methods</Name>
                <GetScriptBlock>$this.GetMethods() |sort name | foreach {"$_"}</GetScriptBlock>
            </ScriptProperty>
            <ScriptProperty>
                <Name>Properties</Name>
                <GetScriptBlock>$this.GetProperties() |sort name | foreach {"$_"}</GetScriptBlock>
            </ScriptProperty>
            <ScriptProperty>
                <Name>Interfaces</Name>
                <GetScriptBlock>$this.GetInterfaces() |sort name | foreach {"$_"}</GetScriptBlock>
            </ScriptProperty>
            <ScriptProperty>
                <Name>Events</Name>
                <GetScriptBlock>$this.GetEvents() |sort name | foreach {"$_"}</GetScriptBlock>
            </ScriptProperty>
        </Members>
    </Type>
</Types>

The command Update-TypeData MyTypes.mshxml imports this information and updates the typedata for System.RuntimeType (I have a number of such statements in my profile file). Now when I inspect types, this information is also available:

MSH> [xml] |fl *

Constructors               : {Void .ctor(), Void .ctor(System.Xml.Xml
                             NameTable)}
Methods                    : {Void add_NodeChanged(System.Xml.XmlNode
                             ChangedEventHandler), Void add_NodeChang
                             ing(System.Xml.XmlNodeChangedEventHandle
...
Properties                 : {System.Xml.XmlAttributeCollection Attri
                             butes, System.String BaseURI, System.Xml
...
Interfaces                 : {System.ICloneable, System.Collections.I
                             Enumerable, System.Xml.XPath.IXPathNavig
                             able}
Events                     : {System.Xml.XmlNodeChangedEventHandler N
                             odeChanged, System.Xml.XmlNodeChangedEve
...
Module                     : System.Xml.dll
Assembly                   : System.Xml, Version=2.0.0.0, Culture=neu
                             tral, PublicKeyToken=b77a5c561934e089
...
MSH> [xml].constructors
Void .ctor()
Void .ctor(System.Xml.XmlNameTable)

MSH> [xml].interfaces
System.ICloneable
System.Collections.IEnumerable
System.Xml.XPath.IXPathNavigable

MSH> [xml].events
System.Xml.XmlNodeChangedEventHandler NodeChanged
System.Xml.XmlNodeChangedEventHandler NodeChanging
System.Xml.XmlNodeChangedEventHandler NodeInserted
System.Xml.XmlNodeChangedEventHandler NodeInserting
System.Xml.XmlNodeChangedEventHandler NodeRemoved
System.Xml.XmlNodeChangedEventHandler NodeRemoving

See how easy and powerful that is? Again, we want to build a system which is self-discoverable. That is why are are hard-core about naming (consistency on naming makes it easy for you to guess what to do and be right). It is also why we provide interactive exploration tools like Get-Member and the interactive shell. By using a synthetic type system, we allow new extensions to be added in a way that makes it easy for you to discover and utilize them.

Cool but wait a minute. Did you notice that the XML datatype supports the NodeChanged Event. What's up with that? Let's check it out. Rember that I defined $x to be an xml fragment up above, let's use to explore using the Start-NewScope function I provided yesterday:

MSH> $x.Add_NodeChanged({Start-NewScope "NODE CHANGED> "})
MSH> $x.a.b = "New Value"
Starting New Scope
NODE CHANGED> $this | fl *

a : a

NODE CHANGED> $_ | fl *

Action : Change
Node : #text
OldParent : b
NewParent : b
OldValue : TEST
NewValue : New Value

NODE CHANGED> exit
MSH> $x=[xml]"<a><b>TEST</b></a>"

MSH> $x.Add_NodeChanged({Write-Host $("XMLChange old {0} new {1}" -f $
_.OldValue, $_.NewValue)})

MSH> $x.a.b="NEW-VALUE"
XMLChange old TEST new NEW-VALUE

Is that cool or what? I would have not known about that unless I was exploring the type. 3 cheers for the democratization of types! Now just to be clear, democracies can be messy things (that is why I like the analogy). With lots of people updating the types, you can get name collisions, differing quality levels, etc. There is no doubt that there are downsides to this model and that if you can get the original development teams to modify that type to meet your needs - it is a better model. But for those of us that don't have either the bandwidth to engage the original development teams, the ability to convince the teams that our scenarios are going to be needed by 80+% of the people, or the luxury of waiting a few years for the process to work itself out - there is now a mechanism for us with our hair on fire to stick our heads in a bucket of water. Sure, doing this might leave us the smell of burnt hair but I'd rather have that than to have my hair still on fire. :-)

Enjoy!

Jeffrey Snover
Monad Architect

How does Match-String with with Pipelines of objects?

Monad Team — Sun, 16 Apr 2006 01:28:00 GMT

In our newsgroup (Microsoft.Public.Windows.Server.Scripting) , Vasu asked about how match-string works in pipelines:

Here is what I observe:
1. MSH C:\> get-alias
..truncated..
Alias ri remove-item
Alias rni rename-item
..truncated..

2. MSH C:\> get-alias | match-string ri
ri
MSH C:\>

The question is why did the string "ri" get emitted instead of the alias record.

So what is going on here?

Match-String works on Streams of STRINGS. When you pass it a string of OBJECTS, the MONAD engine converts those objects to strings before passing them to Match-String. Thus the following 2 commands are functionaly equivalent:

get-alias | match-string "ri"
get-alias | foreach {"$_"} |match-string "ri"

Converting an object to a string is accomplished by call that object's ToString() method (vs using MONAD's formating subsystem). Notice the difference - the first uses Monad's formating and the second is the object's ToString()

MSH> (gal ri)

CommandType     Name                          Definition
-----------     ----                          ----------
Alias           ri                            remove-item

MSH> (gal ri).ToString()
ri
MSH>

If you wanted Match-String to work on the Monad formatted output, you'll need to get that as a string. Here is the thing to grok about our outputing. When your command sequence emits a stream of strings, we emit it without processing. If instead, your command sequence emits a stream of objects, then we redirect those objects to the command Out-Default. Out-Default looks at the type of the object and the registered formating metadata to see if there is a default view for that object type. A view defines a FORMATTER and the metadata for that command. Most objects get vectored to either Format-Table or Format-List (though they could go to Format-Wide or Format-Custom). THESE FORMATTERS DO NOT EMIT STRINGS! You can see this for yourself by the following:

MSH> gps |ft |group {$_.GetType().name} |ft Count,Name -auto

Count Name
----- ----
    1 FormatStartData
    1 GroupStartData
   53 FormatEntryData
    1 GroupEndData
    1 FormatEndData

These formating records are then vectored to an OUT-xxx command to be rendered into the appropriate data for a particular output device. By default, they go to Out-Host but you can pipe this to Out-File, Out-Printer or Out-String. (NOTE: these OUT-xxx commands are pretty clever, if you pipe formating objects to them, they'll render them. If you pipe raw object to them, they'll first call the appropriate formatter and then render them.)

So to make match-string work against the string version of Monad's output, you'd do this:

MSH> get-alias | out-string -stream | match-string "ri"
Alias           clv                       clear-variable
Alias           gdr                       get-drive
Alias           gv                        get-variable
Alias           ndr                       new-drive
Alias           nv                        new-variable
Alias           rdr                       remove-drive
Alias           ri                        remove-item
Alias           rv                        remove-variable
...

Notice a couple things 1) you had to say "Out-String -STREAM" (if you didn't say -STREAM you would have gotten a single string) 2) now match-string matches any line that has an "ri" in it not just the ones where the NAME has an "ri" in it.

Think about the last statement 'just the ones where the NAME has an "ri" in it'. One of the things we designed Monad to do was to give you an experience where you think it, you type it, you get it. So let's say it, "I want to GET the ALIASes WHERE the NAME matches the string RI". Now lets type it:

MSH> GET-ALIAS |WHERE {$_.NAME -match "ri"}

CommandType     Name                       Definition
-----------     ----                       ----------
Alias           ri                         remove-item
Alias           write                      write-object

It take a while to stop thinking in terms of streams of text and start thinking in terms of streams of objects. The benefit is that once you do this, you begin to ask questions in a way that makes it really easy to answer.

Enjoy!

Jeffrey Snover
Monad Architect

Is it safe to use ALIASES in scripts?

Monad Team — Sat, 15 Apr 2006 23:49:00 GMT

In our newsgroup (Microsoft.Public.Windows.Server.Scripting) , Mark Ayers asked the question:
> Shouldn't best practice for scripts be full command name?

The answer is YES, NO, and MAYBE.

YES - Full names provide the most readable experience for scripts. This is very important. People often throw the rock at Perl saying that it is a "write-only language" meaning that after you write a script, you can't read it or understand what it does. We believe that Monad scripts will be widely shared and will be used in environments where it is really important to know exactly what is going on. As such, script readability has always been a strong focus for us. This is one of the key reasons why we support both a pithy and verbose invocation model (pithy for high-speed interactive use, verbose for scripting).

NO - Not all aliases are the same. There are a special class of aliases that are READONLY and ALLSCOPE. You can find these with the following command:

gal | where {""+$_.options -match "Readonly" } | ft name, definition, options -auto

Take a look at how regular and predictable these are:

MSH> gal g* | where {""+$_.options -match "Readonly" } |
ft name, definition, options -auto

Name Definition               Options
---- ----------               -------
gal   get-alias     ReadOnly, AllScope
gc    get-content   ReadOnly, AllScope
gci   get-childitem ReadOnly, AllScope
gcm   get-command   ReadOnly, AllScope
gdr   get-drive     ReadOnly, AllScope
ghy   get-history   ReadOnly, AllScope
gi    get-item      ReadOnly, AllScope
gl    get-location ReadOnly, AllScope
gm    get-member    ReadOnly, AllScope
gp    get-property ReadOnly, AllScope
gps   get-process   ReadOnly, AllScope
group group-object ReadOnly, AllScope
gsv   get-service   ReadOnly, AllScope
gu    get-unique    ReadOnly, AllScope
gv    get-variable ReadOnly, AllScope

MSH> gal ?v | where {""+$_.options -match "Readonly" } |
ft name, definition, options -auto

Name Definition                 Options
---- ----------                 -------
gv   get-variable    ReadOnly, AllScope
nv   new-variable    ReadOnly, AllScope
rv   remove-variable ReadOnly, AllScope
sv   set-variable    ReadOnly, AllScope

MSH> gal n* | where {""+$_.options -match "Readonly" } |
ft name, definition, options -auto

Name Definition              Options
---- ----------              -------
nal new-alias    ReadOnly, AllScope
ndr new-drive    ReadOnly, AllScope
ni   new-item     ReadOnly, AllScope
nv   new-variable ReadOnly, AllScope

MSH> gal *al | where {""+$_.options -match "Readonly" } | ft name, definition, options -auto

Name Definition                  Options
---- ----------                  -------
epal export-alias     ReadOnly, AllScope
gal get-alias        ReadOnly, AllScope
ilal initialize-alias ReadOnly, AllScope
ipal import-alias     ReadOnly, AllScope
nal new-alias        ReadOnly, AllScope
sal set-alias        ReadOnly, AllScope

In previous drops, these aliases where CONSTANT which meant that users could not delete or remove them. They exist because they are very predictable and pithy short hands for the long names. We made them constant so that you could feel safe using them in any script and that you could share that script with other people knowing that you were guaranteed that those aliases would exist everywhere and mean the same thing so your scripts would not break. The community provided strong feedback that MSFT should not ship any CONSTANT aliases so we made them READONLY and ALLSCOPE. This means that you can remove them (using the -FORCE flag) and create your own definitions but it is strongly discouraged.

So in the end, you are pretty safe using the aliases that are READONLY and CONSTANT. You might argue that this is less safe then using the full command name. This would not be correct. Token resolution works by first expanding aliases then binding to a function, then a cmdlet, lastly as an external executable. When you type "Get-Process" - it could have been aliased to any other command or there could be a function with this name. So in the end, the READONLY ALLSCOPE aliases are just about as safe as the full command names.

MAYBE. The other thing you can do in your script is to set up the aliases you'll use yourself. Aliases are scoped so this is generally a safe operation and will not contaminate the users environment. Let me demonstrate using a one of my favorite functions that I define in my profile file, Start-NewScope and its alias sans

function Start-NewScope
{
param($Prompt = $null) Write-Host "Starting New Scope"
if ($Prompt -ne $null)
{ if ($Prompt -is [ScriptBlock])
     {
       $null = New-Item function:Prompt -Value $Prompt -force
     }else
     {   function Prompt {"$Prompt"}
     }
}
$host.EnterNestedPrompt()
}
sal sans Start-NewScope

BTW - every time you enter a nested prompt, $NESTEDPROMPTLEVEL is incremented so I include this information in my prompt:

function prompt
{
"[{0}:{1}]MSH> " -f $PID, $NestedPromptLevel
}

So watch that as I create and exit new scopes. I'll create a new scope, redefine some aliases and then exit the scope and show how those aliases are cleaned up and the old aliases are now in scope.

[3312:0]MSH> gal write |ft name,definition -auto

Name Definition
---- ----------
write write-object

[3312:0]MSH> write test
test

# NOW WE START A NEW SCOPE AND REDEFINE WRITE
[3312:0]MSH> sans
Starting New Scope
[3312:1]MSH> sal write write-error
[3312:1]MSH> write test
write test : test
[3312:1]MSH> gal write |ft name,definition -auto

Name Definition
---- ----------
write write-error

# NOW WE EXIT THE SCOPE WHICH REMOVES THIS ALIAS
# WHICH NOW REVEALS THE PREVIOUS DEFINITION
[3312:1]MSH> exit
[3312:0]MSH> write test
test
[3312:0]MSH> gal write |ft name,definition -auto

Name Definition
---- ----------
write write-object

Enjoy!

Jeffrey Snover
Monad Architect

Verb-Noun vs Noun-Verb

Monad Team — Wed, 12 Apr 2006 16:41:00 GMT

Kurt asked the question "why not Noun-Verb vs Verb-Noun" in his post: http://blogs.msdn.com/monad/archive/2006/02/16/533522.aspx#574708

There are lots of answers to this some better than others. Let me rattle off a couple of them:

VMS DCL/AS400. Issac Newton once said that if he could see farther, it was because he was standing on the shoulders of giants. So it is with Monad. People like to point out the similarities between Monad and various UNIX shells/utilities and scripting languages. Absolutely true - many of us have deep UNIX backgrounds and have a lot of respect for that body of work. That said, Monad is influenced as much by VMS/DCL and AS400/CL as it is UNIX. These are wonderful environments built with the clear vision that they would be used by people running production shops where operations and management really mattered. These environments are strict about naming guidelines, common syntax, environmental setup, operational characteristics, etc etc.

Both of these environments are hard core about naming and leverage a Verb-Noun pattern. Apollo's Domain/OS and AIX also used Verb-Noun models as well. BTW - you can see the influences of the VMS & AS400 focus on production qualities in: having a single common command line parser, -VERBOSE, -CONFIRM, -ERRORACTION (actually our entire near-manical focus on error handling comes from a focus on production readiness), and others.
Programmer vs Admin focus. Kurt points out that a Noun-Verb schema would mirror the OO programming world. That is another reason we are Verb-Noun. There are 2 aspects to this:

Admins are admins with needs similar but distinctly different from those of programmers. We already have world-class languages and tools for programmers, what we need improvement on is Admin languages and tools. I did not want teams thinking that they could just expose their .NET objects and think that they were meeting the needs of admins. These teams need to think about the needs of admins and provide consistent, high level, task-oriented, set-oriented operations with a strong focus on error handling and operational control. You can easily do that with a thin layer on top of .NET objects but you can't get that by just exposing .NET objects. So being Verb-Noun helps drive home this point.
Take a look at the inconsistency of Method names on .net objects. It is Kill() for a process object but Stop() on a serviceController object. Putting the noun first puts people in the mindset that their domain is the high order bit and they pick methodnames optimized for their domain. Putting the Verb first makes it clear that the User Experience is the high order bit and the domain specifics are the low order bit. We want an very high level of verb reuse to provide a consistent, predictable user-experience.

Jeffrey P. Snover
Monad Architect

Monitor the Event Log

Monad Team — Mon, 10 Apr 2006 21:15:00 GMT

Administrators often want to monitor the event logs and look for specific error conditions. The most capable way to do this, of course, is to use a dedicated monitoring application such as Microsoft Operations Manager, or get down-and-dirty with the Win32 API NotifyChangeEventLog. However, Monad can be used for simple applications.

Suppose you are looking for event ID 4226 in the System log. The trivial way to do this is "get-eventlog System | where {$_.EventID -eq 4226}", but this has two problems:

This will report the same errors every time you run it, no matter how old they get, until they are cleared from the log.
Event logs can contain literally millions of events. So, a simple script like this might work on your test server, but could get completely bogged down on your production servers. If you are building software to be distributed to customers whose log configuration you do not control, this is almost certainly unacceptable.

For these reasons, you may want to be a little more clever about searching the event log. Here is a script which searches logs incrementally, keeping track each time of the log index it reached last time and only reporting new hits. The log position is stored in a content object, by default file "<home>\System_Position.txt". Because this content object is only accessed using get-content and set-content, it doesn't have to be a file; it could be an item from any provider implementing IContentCmdletProvider! The output objects are the same as get-eventlog, so no additional formatting directives are needed.

There are five script parameters:

-LogName is the script you want to read, by default "System".
-Filter is the filter which identifies the events you want. This should be expressed as a script block which gets an EventLogEntry as "$_" and yields a boolean value. By default this is "{$true}" which means output all entries.
-PositionFile is the path to the content object which keeps track of the last checked position, by default "<home>\<LogName>_position.txt".
-Force: If specified, continue with the operation even if the current content object does not exist or is invalid. All "[switch]" parameters are false by default; you can just specify "-Force" to make it true, or "-Force:$var" to set it to a shell variable value.
-Restart: If specified, create a new position file and start from the beginning of the log. You will need to use -Restart the first time to create the position file. This is a "[switch]" parameter like -Force.

One line of this script is of particular interest:

# This line actually generates the output
$entry | where $Filter

If the scriptblock in $Filter evaluates to true, then the entry will be written to the script's output stream. The scriptblock will evaluate in the same scope as the script as a whole; to achieve better isolation between the user-provided scriptblock and the main script, "where {&$Filter}" will create a new scope every time.

Also note the use of the write-debug, write-warning, and write-verbose cmdlets in the script. Script users can set $DebugPreference, $VerbosePreference, and/or $WarningPreference to see this text when desired.

A little domain-specific information about the EventLog is required to fully understand this script. First, we check EventLogEntry.Index to determine whether an entry has already been seen, rather than EventLogEntry.TimeGenerated or the like. Remember that the system clock can be modified at any time (e.g. if the user determines that it is running fast), whereas the Index will reliably increment until the log is cleared. Second, we also remember the TimeGenerated of the newest log entry in the position file. This enables us to detect when the log has been cleared, which is important since new entries will have Index starting from 1 after the reset.

Further features could probably be added to this script, for example, a way to initialize a position file to the current position so that you don't ever have to read the entire log.

Thanks to Bruce Payette and Jim Truher for their help with the script.

Enjoy!

Jon Newman [MSFT]

# Eventlog monitoring script 2006/03/28 JonN
#
# Event logs can contain hundreds of thousands of items, so
# "get-eventlog System | where {<condition>}" can take excessively long
# on production systems. Plus, the same failures will be reported
# over and over.
# This script will remember your last Index position in the log,
# and only report events which occurred since then.
# It also remembers the TimeGenerated of the oldest log entry,
# so that it can detect when the log has been cleared.
# Example:
# eventlog.msh1 -LogName System -Filter {$_.EventID -eq 4226} -PositionFile "c:\logs\syslogpos.txt"

param (
    [string]$LogName = "System",
    [ScriptBlock]$Filter = {$true},
    [string]$PositionFile = $home + "\" + $LogName + "_position.txt",
    [switch]$Force,
    [switch]$Restart
    )

write-debug "`$LogName = $LogName"
write-debug "`$Filter = '$Filter'"
write-debug "`$PositionFile = '$PositionFile'"
write-debug "`$Force = $Force"
write-debug "`$Restart = $Restart"

[int]$lastCheckedIndex = -1
[DateTime]$lastCheckedTime = [DateTime]::MaxValue

if (!$Restart)
{
    $filecontent = get-content $PositionFile -ea SilentlyContinue
    if ($null -eq $filecontent)
    {
        if (!$Force)
        {
     throw "Position file '$PositionFile' does not exist or could not be opened. Use -Force or -Restart to create a new position file."
        }
        write-verbose "Position file '$PositionFile' does not exist or could not be opened. Rebuilding position file because -Force was specified."
    }
    else
    {
        write-debug "Position file contains '$filecontent'"
        trap {
            if ($Force)
            {
                write-warning "Content of position file '$PositionFile' is invalid. Rebuilding position file because -Force was specified."
                $lastCheckedIndex = -1
                $lastCheckedTime = [DateTime]::MaxValue
                continue
            }
     throw "Content of position file '$PositionFile' is invalid. Terminating operation. Use -Force or -Restart to rebuild the position file."
        }
        # These lines will throw if the cast fails
        $lastCheckedIndex = $filecontent[0]
        $lastCheckedTime = $filecontent[1]
    }
}

write-debug "`$lastCheckedIndex = $lastCheckedIndex"
write-debug "`$lastCheckedTime = $lastCheckedTime"

# I don't simply call get-eventlog $LogName because I don't want to
# build an array with all the hundreds of thousands of event log entries.
# Instead, I make sure the EventLogEntryCollection is not unrolled.
$log = get-eventlog -List | where {$_.Log -ieq $LogName}
if ($null -eq $log)
{
throw "Log not found: $LogName"
}

$entries = $log.Entries
if (0 -eq $entries.Count)
{
write-debug "Log empty: $LogName"
return
}

$oldestEntryTime = $entries[0].TimeGenerated
if ($oldestEntryTime -gt $lastCheckedTime)
{
write-verbose "The log appears to have been cleared since it was last checked: $LogName."
$lastCheckedIndex = -1
}

# The index can diverge from the number of entries
# when the log reaches its maximum size and is configured
# to "Overwrite entries as needed"
$newestEntryIndex = $entries[$entries.Count - 1].Index
$newestEntryTime = $entries[$entries.Count - 1].TimeGenerated

write-debug "Newest entry in log has index $newestEntryIndex"
write-debug "Newest entry in log has TimeGenerated $newestEntryTime"

# $entries.Count could be more than the maximum range "50000 .. 0"
foreach ($i in ($entries.Count - 1) .. 0)
{
    $entry = $entries[$i]
    if ($entry.Index -le $lastCheckedIndex) {
        break
    }
    # This line actually generates the output
    $entry | where $Filter
}

write-debug "Writing index $newestEntryIndex to position file '$PositionFile'"
write-debug "Writing time $newestEntryTime to position file '$PositionFile'"
$newestEntryIndex,$newestEntryTime | set-content $PositionFile

Base64 Encode/Decode a string

Monad Team — Mon, 27 Mar 2006 19:21:00 GMT

Today I came across http://scripts.readify.net/ . This sight is focused on Monad and MSH and is starting a collection of scripts at: http://scripts.readify.net/Scripts.aspx . You should visit their site and let them know what type of scripts would be useful to you.

I particularly liked their entry on how to Base64 encode a file. This is something that I need occassionally and I can never remember how to do it. I was about to include it as-is into my profile and decided that there was a better way to do this.

<IMPORTANT POINT>
Whenever you are adding some functions, you should make a conscious decision about whether those functions are best exposed as a "function" or as a "type extension".
</IMPORTANT POINT>

Both of these mechanisms are great and have their purpose but my observation is that people are not using type-extensions as much as they should. One of the huge benefits of type extensions is discoverability. Let me show you how to do this function as a type-extension and highlight the benefits of this approach.

When doing a type extension, you first need to decide what TYPE you are going to extend. In this case, I decided that I would extend SYSTEM.STRING as that was the most general purpose type for this function. I then encoded the following in a file called My.Types.Mshxml

</Types>
    <Type>
        <Name>System.String</Name>
        <Members>
            <ScriptProperty>
                <Name>ToBase64String</Name>
                <GetScriptBlock>
                 [System.Convert]::ToBase64String([System.Text.Encoding]::UNICODE.GetBytes($this))
  </GetScriptBlock>
            </ScriptProperty>
            <ScriptProperty>
                <Name>FromBase64String</Name>
                <GetScriptBlock>
                 [System.Text.Encoding]::UNICODE.GetString([System.Convert]::FromBase64String($this))
  </GetScriptBlock>
            </ScriptProperty>
        </Members>
    </Type>
</Types>

In my profile I load this file via the command:

Update-TypeData c:\msh\My.Types.mshxml

Once I do that, these properties are available to any STRING and can be discovered via Get-Member:

MSH> $x="Hello World"
MSH> $x |Get-Member

TypeName: System.String

Name             MemberType            Definition
----             ----------            ----------
Clone            Method                System.Object Clone()
CompareTo        Method                System.Int32 CompareTo(Object value), System.Int32 CompareTo(String strB)
Contains         Method                System.Boolean Contains(String value)
CopyTo           Method                System.Void CopyTo(Int32 sourceIndex, Char[] destination, Int32 destinationIndex, Int32 count)
EndsWith         Method                System.Boolean EndsWith(String value), System.Boolean EndsWith(String value, StringComparison comparisonType), System.Boolean EndsWith(String value, Boolean ignoreCase, CultureInfo culture)
...

Trim             Method                System.String Trim(Params Char[] trimChars), System.String Trim()
TrimEnd          Method                System.String TrimEnd(Params Char[] trimChars)
TrimStart        Method                System.String TrimStart(Params Char[] trimChars)
Chars            ParameterizedProperty System.Char Chars(Int32 index) {get;}
Length           Property              System.Int32 Length {get;}
FromBase64String ScriptProperty        System.Object FromBase64String {get=[System.Text.Encoding]::UNICODE.GetString([System.Convert]::FromBase64String($this));}
ToBase64String   ScriptProperty        System.Object ToBase64String {get=[System.Convert]::ToBase64String([System.Text.Encoding]::UNICODE.GetBytes($this));}

MSH> $x
Hello World
MSH> $x.ToBase64String
SABlAGwAbABvACAAVwBvAHIAbABkAA==
MSH> $x.ToBase64String.FromBase64String
Hello World
MSH>

Enjoy!

Jeffrey Snover
Monad Architect

GetObject()

Monad Team — Sun, 26 Mar 2006 18:02:00 GMT

Monad provides a way to create new com objects with new-object

$ie = New-Object -ComObject InternetExplorer.Application
$ie.Navigate2(http://blogs.msdn.com/monad)
$ie.Visible=1

Great but what about if you want to bind to an existing object? Where is the equivalent of GetObject()?

This is one of those good new/bad news stories. First the bad news. Monad does not provide a Cmdlet that exposes this function. Alas, to ship is to choose and we are not going to get to in for V1. Now for the good news. Monad provides great .NET support and .NET exposes this function.

MSH> $w=[System.Runtime.InteropServices.Marshal]::GetActiveObject("Word.Application")

MSH> $w.documents |ft name,path -auto
Name                                              Path
----                                              ----
RE: Question regarding Word scripting with MSH... RE:
cmdlets as classes.doc                            \\imself-dfs-07\dfsroot\dfsmydocs\js...Event Schema.doc                                  \\imself-dfs-07\dfsroot\dfsmydocs\js...

Let me be quick to point out a problem that exists in your beta code but is fixed in the upcoming release. If you just typed $w.Documents - it is going to appear to have hung and you'll either have to wait a LONG time or hit CTRL-BREAK to terminate the session. The work around is very simple, pipe the output to Get-Member and use that to find out what properties you want and then ask for those specific properties.

Anyway, this is an example of how Monad's support for a wide range of technologies (text processing, COM, WMI, ADSI, ADO, XML, .NET, etc) really pays off. As a rule, we are trying to expose the system with Cmdlet and namespace semantics. It is going to take a while before the entirety of the system is exposed this way but by providing direct access to these other technologies, it enables the community to provide this support. For example the function:

Function Get-Object ($ProgID) {
[System.Runtime.InteropServices.Marshal]::GetActiveObject($ProgID)
}

and the following formating directives:

<Configuration>
    <ViewDefinitions>
        <View>
            <Name>CustomView</Name>
            <ViewSelectedBy>
                <TypeName>System.__ComObject#{0002096b-0000-0000-c000-000000000046}</TypeName>
            </ViewSelectedBy>
            <TableControl>
                <TableHeaders>
                    <TableColumnHeader>
                       <Label>Name</Label>
                       <Width>20</Width>
                     </TableColumnHeader>
                     <TableColumnHeader>
                       <Label>FullName</Label>
                     </TableColumnHeader>

            </TableHeaders>
                <TableRowEntries>
                    <TableRowEntry>
                        <TableColumnItems>
                     <TableColumnItem>
                       <PropertyName>Name</PropertyName>
                     </TableColumnItem>
                      <TableColumnItem>
                       <PropertyName>FullName</PropertyName>
                     </TableColumnItem>
                        </TableColumnItems>
                    </TableRowEntry>
                 </TableRowEntries>
            </TableControl>
        </View>
    </ViewDefinitions>
</Configuration>

allows us to do the following:

MSH> $w = Get-Object -ProgId Word.Application
MSH> $w.documents
Name                                              Fullname
----                                        --------
RE: Question regarding Word scripting with MSH... RE:
cmdlets as classes.doc                            \\imself-dfs-07\dfsroot\dfsmydocs\js...Event Schema.doc                                  \\imself-dfs-07\dfsroot\dfsmydocs\js...

Extend the system and Enjoy!

Jeffrey P. Snover
Monad Architect

Finding out what determines which properties are displayed

Monad Team — Wed, 22 Mar 2006 21:10:00 GMT

MoW, a Monad newsgroup regular, asked why only the Caption, Name and PeakUsage properties were displayed from the command “get-WMIObject Win32_PageFileUsage” while the output object had a lot more properties. I think it is worth it to explain in details how to find out which properties are displayed.
First, I would look at which view defined in one of the *.format.msh1xml files is used. The Trace-Command Cmdlet can help here:
Trace-Command –option All –Name FormatViewBinding { get-WMIObject Win32_PageFileUsage | out-host} –MshHost

The trace output is in yellow. It reads at the end: “DEBUG: FormatViewBindin Information: 0 : No applicable view has been found.” Now, I know that no view is used. The next place to look is the types.msh1xml file. In this file, each type description can have a node called DefaultDisplayPropertySet where the default display properties are defined. Since the class in question is Win32_PageFileUsage, we can search for the string in notepad. Here is the section for the class.

    <Type>
        <Name>System.Management.ManagementObject#root\cimv2\Win32_PageFileUsage</Name>
        <Members>
            <PropertySet>
                <Name>MshStatus</Name>
                <ReferencedProperties>
                    <Name>Status</Name>
                    <Name>Name</Name>
                    <Name>CurrentUsage</Name>
                </ReferencedProperties>
            </PropertySet>
            <MemberSet>
                <Name>MshStandardMembers</Name>
                <Members>
                    <PropertySet>
                        <Name>DefaultDisplayPropertySet</Name>
                        <ReferencedProperties>
                            <Name>Caption</Name>
                            <Name>Name</Name>
                            <Name>PeakUsage</Name>
                        </ReferencedProperties>
                    </PropertySet>
                </Members>
            </MemberSet>
        </Members>
    </Type>

Under the DefaultDisplayPropertySet node, there are Caption, Name, and PeakUsage. The question is answered.

- Kevin[MSFT]

Dynamically loading assemblies in mshsnapins.

Monad Team — Thu, 16 Mar 2006 02:43:00 GMT

In Monad, mshsnapins can be installed in a different directory from the Monad executables. At runtime, the Monad engine will load the mshsnapin assembly and its referenced assemblies based on application base information provided during mshsnapin installation.

If an mshsnapin itself dynamically loads an assembly through an assembly name or a relative path, however, CLR may not be able to locate the assembly. This is because the CLR runtime knows nothing about the mshsnapin installation directory and thus will not search there during dynamic assembly resolution.

The following is a common scenario for this: Assume that an mshsnapin named "mysnapin" is installed in the c:\mysnapin directory and the monad executable is installed in c:\monad\ directory. Also assume that cmdlet "my-command" in "mysnapin" will dynamically load another assembly like this, (assume mylibrary.dll sits in the same directory as "mysnapin" assembly)

Assembly.LoadFrom("mylibrary.dll");

If you run "my-command" in msh.exe, you will get an error indicating that mylibrary.dll can't be found. The reason for this is that CLR runtime only searchs through the GAC and the currently executable directory (which is installation directory for msh.exe) for assembly resolution.

If you want to dynamically load an assembly in your mshsnapin, always provide an absolute path unless the assembly itself is in GAC.

-George[MSFT]

Working with Sort-Object Cmdlet

Monad Team — Tue, 14 Mar 2006 22:57:00 GMT

In this blog I will try to explain different features of sort-object(Sort) cmdlet. For the purpose of this blog, I assume the following objects exist:

MSH C:\temp\monad> $a,$b,$c,$d

Score Name

----- ----

100 John

90 Henry

90 Tom

80 David

From the definition Sort looks like:

MSH C:\temp\monad> (get-command sort-object).Definition

sort-object [[-Property] Object[]] [-Descending] [-Unique] [-InputObject MshObj

ect] [-Culture String] [-CaseSensitive] [-Verbose] [-Debug] [-ErrorAction Actio

nPreference] [-ErrorVariable String] [-OutVariable String] [-OutBuffer Int32]

The basic usage looks like:

Sort works on a collection of input objects possibly coming over from a pipeline. Sort will collect all the input objects from the pipeline and then sorts the objects based on other parameters like Property, Descending, Unique etc.

Example: To sort objects based on Score

MSH C:\temp\monad> $a,$b,$c,$d | sort score

Score Name

----- ----

80 David

90 Tom

90 Henry

100 John

Example: To sort objects based on Name

MSH C:\temp\monad> $a,$b,$c,$d | sort Name

Score Name

----- ----

80 David

90 Henry

100 John

90 Tom

Example: To sort objects in descending order based on Score

MSH C:\temp\monad> $a,$b,$c,$d | sort Score -Descending

Score Name

----- ----

100 John

90 Tom

90 Henry

80 David

Ok, this is all simple and you might yourself have already explored all this. Let us get into some tricky stuff. How to sort on Score (descending order) and Name (ascending order)

Example: sort on Score descending and then Name ascending

MSH C:\temp\monad> $a,$b,$c,$d | sort @{expression="Score";Descending=$true},@{e

xpression="Name";Ascending=$true}

Score Name

----- ----

100 John

90 Henry

90 Tom

80 David

Notice the use of @{expression=…}. If you notice the definition of sort-object, the parameter Property takes in a object[]. Property parameter can take entries of form <string> or hashtable (notice @). If the input is a hashtable, then Sort cmdlet specifically looks for “expression”,”ascending”,”descending” keys. Ascending and Descending can take only Booleans ( i.e., $true or $false ). Using expression you can do many things.

For example, lets say Henry is very punctual and very dedicated and you wanted to give some additional marks (lets say 2 marks) to encourage him.

Example:

MSH C:\temp\monad> $a,$b,$c,$d | sort @{expression={if ($_.Name -eq "Henry") { $

_.Score += 2 }}},Score -Descending

Score Name

----- ----

100 John

92 Henry

90 Tom

80 David

Notice the use of Script for expression. In the above example, I changed the original object itself by using $_ and then sorted on Score

MSH C:\temp\monad> $b

Score Name

----- ----

92 Henry

Using Script block you can do many things like computing Grades and then sorting on Grade.

MSH C:\temp\monad> $a,$b,$c,$d | sort Grade,@{expression={if ($_.Score -gt 91) {

$Grade="A" } else {$Grade="B" }; add-member noteproperty "Grade" $Grade -InputOb

ject $_}} -Descending

Score Name Grade

----- ---- -----

90 Tom B

80 David B

100 John A

92 Henry A

Sort in its simplest form is very useful. Using expressions effectively makes it more powerful.

One last thing, a number of you has asked how to sort a hashtable using Sort-Object cmdlet.

MSH C:\temp\monad> $hash=@{"Tom"=90;"Henry"=90;"David"=80;"John"=100}

MSH C:\temp\monad> $hash | sort value -descending

Name Value

---- -----

Henry 90

Tom 90

David 80

John 100

This will not work as there is only one object on the pipeline and sort-object does not have anything to sort. However use GetEnumerator() on HashTable to write all the hash entries to the pipeline and then use sort.

MSH C:\temp\monad> $hash.GetEnumerator() | sort value -descending

Name Value

---- -----

John 100

Tom 90

Henry 90

David 80

-Krishna[MSFT]