Media And Microcode

New WPK Video Online: Multitouch Fingerpaint in 30 Lines of PowerShell Script

2009-10-30T20:32:21Z

The 2nd video about WPK just hit Channel9. The first video was a video walk thru of How to Write User Interfaces with WPK, and this video walks thru How to Write a Multitouch FingerPaint with WPK.

Enjoy,

James Brundage [MSFT]

Microcode: PowerShell Scripting Trick: Fun With Parameter Binding: The Fake Parameter Set Trick

2009-04-10T10:00:04Z

I’ve been hearing a lot of questions recently about what things you can do with advanced functions. People seem to be aching for good examples of using PowerShell V2 functions, and, since I’ve discovered a few handy tricks for advanced functions, I thought I’d start to share them there in a series called “Fun with Parameter Binding.”

Here’s volume one: The Fake Parameter Set Trick

I found myself wanting to make an advanced function that took something strongly typed (in this case, the CommandInfo object, which is the base object for what you see when you run Get-Command) from the pipeline and returned items that matched a certain characteristic. If I didn’t pipeline in any items, I wanted to get all of the commands and check for the same characteristic.

What I really wanted was the function to act two ways, one parameterized, one unparameterized (like how you can run Get-Process or Get-Process PowerShell*)

I know of one way to approach this, using the $psBoundParameters variable (see this enlightening post by Marcel Ortiz Soto), but I recently happened upon another approach that I find kind of novel. I call it the fake parameter set trick.

Here’s an example:

[CmdletBinding(DefaultParameterSetName="All")]            
                
    param(            
    [Parameter(Mandatory=$true,            
        ParameterSetName="Command",            
        ValueFromPipeline=$true)]            
    [Management.Automation.CommandInfo]            
    $command            
    )            
                
    process {            
        $psCmdlet.ParameterSetName               
    }

The CmdletBinding attribute at the top of the function is one of the markers that the function is an advanced function, and it specifies the DefaultParameterSetName. If no default parameter set name is specified, it would go ahead and default to whichever ParameterSetName it found on a parameter. If you didn’t every mention a ParameterSetName, then the parameter set name would implicitly be __AllParameterSets.

In my case, I’ve set the DefaultParameterSetName to All. But there aren’t any parameters at all in All. In fact, for All intents and purposes, All is nothing. All is a fake parameter set.

This lets me mark the $command parameter as mandatory and from pipeline, and lets me easily figure out if the parameter is there or not. This is ideal for building good Get- advanced functions, because Get- cmdlets and functions tend to either be able to retrieve all of the data or retrieve a filtered subset of the data.

Hope this Helps,

James Brundage

Microcode: A Quick Trick to turn regular XML into Xaml

2009-01-22T13:07:36Z

MSDN has tons of examples about XAML. Nearly everything in WPF has an example in XAML, but not all of these XAML examples actually reference the namespace that is required for this Xaml to be loadable by WPF. It's still valid XML, just not valid XAML.

Here's a quick example of what I mean. I just copied and pasted this chunk from the MSDN topic on LinearGradientBrush:

<Rectangle Width="200" Height="100">
<Rectangle.Fill>
    <LinearGradientBrush StartPoint="0,0" EndPoint="1,1">
      <GradientStop Color="Yellow" Offset="0.0" />
      <GradientStop Color="Red" Offset="0.25" />
      <GradientStop Color="Blue" Offset="0.75" />
      <GradientStop Color="LimeGreen" Offset="1.0" />
    </LinearGradientBrush>
</Rectangle.Fill>
</Rectangle>

Now if I try to simply load that up as XAML, XamlReader barfs

$x =[xml]'
<Rectangle Width="200" Height="100">
<Rectangle.Fill>
    <LinearGradientBrush StartPoint="0,0" EndPoint="1,1">
      <GradientStop Color="Yellow" Offset="0.0" />
      <GradientStop Color="Red" Offset="0.25" />
      <GradientStop Color="Blue" Offset="0.75" />
      <GradientStop Color="LimeGreen" Offset="1.0" />
    </LinearGradientBrush>
</Rectangle.Fill>
</Rectangle>
'
[Windows.Markup.XamlReader]::Parse($x.OuterXml)

Exception calling "Parse" with "1" argument(s): "'' XML namespace prefix does not map to a namespace URI, so cannot resolve property 'Width'. Line '1' Position '79'."
At line:14 char:35
+ [Windows.Markup.XamlReader]::Parse <<<< ($x.OuterXml)
+ CategoryInfo : NotSpecified: (:) [], MethodInvocationException
+ FullyQualifiedErrorId : DotNetMethodException

Obviously, that's not ideal. Luckily, with CTP3, there's a cmdlet called Select-Xml, which will allow me to query a document with xPath, regardless of namespaces.

This means that I can check to see if the fist element has a reference to the WPF namespace, and add it if it's not there.

$t = $x |
    Select-Xml //* |
    Select-Object -First 1 |
    Foreach-Object {
        if (-not $_.Node.xmlns) {
            $_.Node.SetAttribute("xmlns", 'http://schemas.microsoft.com/winfx/2006/xaml/presentation')
        }
        $_.Node.OuterXml
    }

I'll stop and explain this pipeline a little. $x is the XML, I pipe it into Select-Xml, with the xPath parameter of //*. //* roughly translates to any node. I pick out the first node (with Select-Object -first 1) and then I check to see if the Node contains an xmlns attribute. If it doesn't, I add it. Either way, I emit the first node's OuterXml, which basically means I emit the entire XML chunk as text. That's what $t is when I am done.

Then I use this chunk to display it:

$w = New-Object Windows.Window
$w.Content = [Windows.Markup.XamlReader]::Parse($t)
$w.SizeToContent = "WidthAndHeight"
$w.ShowDialog()

Now I have the MSDN sample working, regardless of if it had an xmlns attribute in the first place.

Hope this Helps,

James Brundage [MSFT]

Media: The Moment Obama Got Inaugurated and PhotoSynth Hit Mainstream

2009-01-20T23:12:34Z

Photosynth is a really cool product out of live labs. It's sort of an uber photo collage application. It takes photos taken from different people, at different resolutions and different times, and collects them into a 3d view of a location that you can navigate around.

CNN.com worked with Photosynth to show the world the Moment of Obama's Inauguration. It's a pretty cool way to view history.

Check it out:

http://www.cnn.com/SPECIALS/2009/44.president/inauguration/themoment/

Hope this Helps,

James Brundage [MSFT]

PowerScripting Podcast Online

2009-01-20T00:44:35Z

My interview with the PowerScripting Podcast is now online. Check it out:

http://powerscripting.wordpress.com/2009/01/18/episode-55-jeff-hicks-in-for-hal-and-an-interview-with-james-brundage/

Hope this helps,

James Brundage [MSFT]

Please Join Me for a Power Scripting Podcast Tonight @ 9PM EST (6PM PST)

2009-01-15T21:30:11Z

I’m going to be doing a PowerScripting Podcast tonight @ 9PM EST. There’s a live chat room where you can ask questions and pick the brains of a tester / avid scripter on the PowerShell team.

It should be fun. I hope to see your questions there.

http://powerscripting.wordpress.com/2009/01/14/tomorrow-on-powerscripting-live-james-brundage/

Hope this Helps,

James Brundage [MSFT]

Media: The Future of Newspapers is Pay For Play Journalism?

2008-12-28T03:45:35Z

I love my news. For whatever reason, I find catching up on the world to be one of the most relaxing and rewarding things you can do in your spare time. Even though I like to read the news, I rarely read a physical newspaper or magazine or watch a news program . My cell phone and web browser are my gateways to the news world. I make an exception for especially good local weeklies (Seattle's The Stranger) and very insightful magazines with a broad range of news ( like The Economist ). But times are tough for newspapers and magazines alike. Local small-market newspapers are dropping like flies, and even big outfits like The New York Times are struggling. Digitization has been a painfully slow paradigm shift with major publications.

This is primarily because of a loss in advertising revenue on many fronts. It's harder to convince a businessman to buy a 6" ad for hundreds of dollars upfront with limited tracking and return when they can buy a search keyword for a fraction of that. It's also very hard to convince someone to put a classified ad in at a dollar a word when Craigslist will do it for free. So it's no suprise that there have been any number of propositions about "the future of newspapers." Most are still advertising focused, and the general mind set in publishing still seems to ignore what happens when the Long Tail effect hits news and ignoring the peril of wire services. By this I mean that if finding the news I am interested in is no longer restricted by where I am, then I'll end up reading publications from wherever news comes that interests me, and it becomes harder for any publication to really gain a mental foothold. Further more, if every time I pick up a newspaper, 80% of the content comes from a news wire service, then that newspaper is no different from any other newspaper I'd run across, and I'm no more or less likely to pick it up in the future.

This is a very tricky situation, that presents no great way out. But today, I ran across a way that is so worrisomely bad that I've decided to talk about it here.

Ironically, I ran this very bad answer to the problems newspapers are facing on Good, through a link on MSN title "When Newspapers Are Gone". This is an article about a web site called Spot.Us, where a few community minded individuals pay a reporter to write a news story.

In my personal opinion, this is a very, very bad idea.

Why?

First and foremost, it's "pay for play" journalism. If this is the future of news, then the future of news will mean that people who cannot afford to be covered will not be. What's that mean? Well, unless you get a lot of donations from a lot of people, no coverage of any danger zone, so most likely no international coverage whatsoever. If this sort of news becomes popular, than suffering becomes a "tree falls in a forest" problem. If 1,000 people are killed in an earthquake due to shoddy infrastructure, and no one can afford to report on it, do their deaths mean anything?

The second problem is what I call NIMBY journalism (NIMBY is Not In My Backyard). An individual can provide up to 20% of the amount required to cover a story, which means that a small group of individuals can fund a story of their own liking. Don't like that eyesore building going up across the street? Pool 5 of your neighbors and have someone you've never met run a news story about the decline and fall of your neighborhoods look. Concerned about increasing homelessness in your area? Fund an article pushing for a homeless shelter far away.

The third problem is funding transparency. If only a few people can fund a story on Spot.Us, which pitches the story to other major outlets, then it's possible to get stories written that back up your political beliefs. While Spot.Us does have "fact-checking editors" (a freelance journalist can turn down the first two if they choose), there's little on who watches the watchdogs, less on how retractions work, and nothing that I can see that would prevent individuals with different political slants from funding a story. As the story is picked up by other outlets, both patent lies and innuendo can seep into the news cycle with a significantly greater pedigree than information recycled from blogs. While you can see the individuals that sponsored the article on Spot.Us, you do not have any information about their affiliations with companies, and it's unlikely that whoever would pick up a story from Spot.Us would list the individuals who paid for the article.

The fourth problem is that pitch-based news doesn't come up with the news no one wants to hear. A simple and sarcastic example of this was my time (12 years) writing film reviews. During this time, I saw a lot of movies I suspected would be awful, not because I particularly wanted to, but because someone has to and my editor gave me marching orders to cover the movies. Some of them were awful, some of them were unexpectedly great. When journalists do their own pitching of stories entirely, the editorial process is greatly degraded, because people simply do not go out to cover many things, like violent crime (or exceedingly bad movies), without being told to do so. Sometimes the best stories start from the most mundane pieces of information.

The fifth problem is the transparency in the expenses of a story. Salaried employees of a newspaper can sometimes expense costs of the story to a newspaper, but there is a lot of transparency in the process. Receipts must be kept. Purchases must be justified. As much as I wish journalists were paid better for their work, I do not believe that they should be their own accountants or set all of their own rates.

A quick look of the story pitches on Spot.Us gives a lot of foundation for these concerns. Right now, I see a pitch about problems with state adopted children ($500 required), a pitch about a retirement community residents who had to resign from a community board because they formed their own online paper ($25 required), a pitch about why the VOA takes too long to process claims ($1000), a pitch to cover the poor in San Francisco ($500), a pitch about the digital future of Bay Area newspapers ($1000), a pitch about the electricity sources in the Bay Area ($1500), a pitch about if Oakland can survive the next quake ($450), and a pitch about how the recession is hitting bay area hookers ($500). Each of these pitches raises a lot of questions about what the money's being spent on (especially the last one) and why the rates are so wildly different. I'd go out on a limb and guess that the community board pitch ($25) is someone who wants to give more legitimacy to their community association dispute. That there's even a pitch for a digitization of news story on an online news site raises obvious conflict of interest questions. Several of the stories seem wildly overvalued, and others incredibly under expensed (how can someone assess is Oakland is ready for the next quake for $450?). Magazines with great readership (hundreds of thousands or even millions) will generally not pay more than a few thousand dollars a story to a freelancer. Why is a story that can be researched from a laptop while sipping a latte worth $1000?

I apologize to veer off my normal course of technical information and highlighting interesting information that I find online, but the idea of journalism as funded by the angry mob makes me very worried. Journalism should help the mob become incensed about what they do not know, not become the voice of the angry mob to recruit more members. Journalists should calm the mob when it is irrationally angry, and not owe their paycheck to the mob. Journalists should show be a check and balance to the world, instead of balancing their checkbooks by what their neighborhood says is right.

Hope this Helps,

James Brundage [MSFT]

Microcode: PowerShell + XNA: New-SpriteFont

2008-12-24T13:07:46Z

As I said in a previous post, I've started to explore the wild world of XNA. XNA just released version 3, which allows you to make games for Windows, Xbox, and Zune. The SDK is free, and you can download it here.

The last post introduced Get-Font, a quick function that lets me see what fonts are on the system. I wrote this function to save me the potential future headache of making a typo in my SpriteFont file, and in this post I'll show you New-SpriteFont, which uses Get-Font and emits a Sprite Font file with a number of parameters for any font on my system.

New-SpriteFont was fairly straightforward. I simply check for any font that is named what I say, and throw an error if one is not found. Then I copied a sprite font file from the sample Platformer project into my function, and added a parameter for each part of the XML I wanted to parameterize in a script. Since font names cannot contain characters that would trouble XML (and I've already checked to make sure the font exists), I don't need to worry about escaping my XML for the font name. Almost everything else was an integer, except for style. The only interesting curve ball that the spritefont XML threw me was that it used a lowercase true. True will read as "True", and since XML is case-sensitive (even though PowerShell is not), I coerced the switch parameter into a string and then turned the string into a lowercase string.

Then I tacked on some inline help, and I had a New-SpriteFont that did the trick. I'm using Write-CommandBlogPost to write up New-SpriteFont for public consumption. Enjoy!

Synopsis:

Creates a New XNA SpriteFont

Syntax:

New-SpriteFont [[-font] [<Object>]] [[-size] [<Int32>]] [[-spacing] [<Int32>]] [[-style] [<Object>]] [<CommonParameters>]

Detailed Description:

Generates an XNA SpriteFont file. Allows you to specify the font name, size, spacing, style, and if kerning is used or not

Examples:

    -------------------------- EXAMPLE 1 --------------------------





# Create a simple New sprite font
New-SpriteFont

    -------------------------- EXAMPLE 2 --------------------------





# Create a sprite font using the font Garamond and a fontsize of 12
New-SpriteFont "Garamond" 12

Command Parameters:

Name Description

font The font name of the XNA sprite font to use (default Lucida Console)

size The font size of the XNA Sprite Font (default 14)

spacing The spacing between the fonts (default 0)

useKerning If set, the XNA sprite font will use kerning

style The style of the font (e.g. Regular, Bold, Italic)

Name	Description
font	The font name of the XNA sprite font to use (default Lucida Console)
size	The font size of the XNA Sprite Font (default 14)
spacing	The spacing between the fonts (default 0)
useKerning	If set, the XNA sprite font will use kerning
style	The style of the font (e.g. Regular, Bold, Italic)

Here's New-SpriteFont:

function New-SpriteFont {
           
    <#
    .Synopsis
        Creates a New XNA SpriteFont
    .Description
        Generates an XNA SpriteFont file.  Allows you to specify the font name, size, spacing, style, and if kerning is used or not
    .Parameter font
        The font name of the XNA sprite font to use (default Lucida Console)
    .Parameter size
        The font size of the XNA Sprite Font (default 14)
    .Parameter spacing
        The spacing between the fonts (default 0)
    .Parameter useKerning
        If set, the XNA sprite font will use kerning
    .Parameter style
        The style of the font (e.g. Regular, Bold, Italic)
    .Link
        Get-Font
    .Example
        # Create a simple New sprite font
        New-SpriteFont
    .Example
        # Create a sprite font using the font Garamond and a fontsize of 12
        New-SpriteFont "Garamond" 12
    #>
param($font = "Lucida Console",
    [int]$size = 14,
    [int]$spacing = 0,
    [switch]$useKerning,
    $style = "Regular")
if (-not (Get-Command Get-Font -ErrorAction SilentlyContinue )) { throw $error[0] }
    $checkedFont = Get-Font $font
    if (-not $checkedFont) {
        throw "The font name $font is not valid. Pick a new one a-hole."
    }
@"
<?xml version="1.0" encoding="utf-8"?>
<!--
This file contains an xml description of a font, and will be read by the XNA
Framework Content Pipeline. Follow the comments to customize the appearance
of the font in your game, and to change the characters which are available to draw
with.
-->
<XnaContent xmlns:Graphics="Microsoft.Xna.Framework.Content.Pipeline.Graphics">
  <Asset Type="Graphics:FontDescription">

    <!--
    Modify this string to change the font that will be imported.
    -->
    <FontName>$font</FontName>

    <!--
    Size is a float value, measured in points. Modify this value to change
    the size of the font.
    -->
    <Size>$size</Size>

    <!--
    Spacing is a float value, measured in pixels. Modify this value to change
    the amount of spacing in between characters.
    -->
    <Spacing>$spacing</Spacing>

    <!--
    UseKerning controls the layout of the font. If this value is true, kerning information
    will be used when placing characters.
    -->
    <UseKerning>$($useKerning.ToString().ToLower())</UseKerning>

    <!--
    Style controls the style of the font. Valid entries are "Regular", "Bold", "Italic",
    and "Bold, Italic", and are case sensitive.
    -->
    <Style>$style</Style>

    <!--
    If you uncomment this line, the default character will be substituted if you draw
    or measure text that contains characters which were not included in the font.
    -->
    <!-- <DefaultCharacter>*</DefaultCharacter> -->

    <!--
    CharacterRegions control what letters are available in the font. Every
    character from Start to End will be built and made available for drawing. The
    default range is from 32, (ASCII space), to 126, ('~'), covering the basic Latin
    character set. The characters are ordered according to the Unicode standard.
    See the documentation for more information.
    -->
    <CharacterRegions>
      <CharacterRegion>
        <Start>&#32;</Start>
        <End>&#126;</End>
      </CharacterRegion>
    </CharacterRegions>
  </Asset>
</XnaContent>
"@    

}

Hope this Helps,

James Brundage [MSFT]

Automatically generated with Write-CommandBlogPost

Microcode: PowerShell Scripting Tricks: Get-Font

2008-12-24T12:33:57Z

I've started to explore the wild world of XNA. XNA just released version 3, which allows you to make games for Windows, Xbox, and Zune. The SDK is free, and you can download it here. It comes with a nifty little platformer game (think a 3-screen Super Mario Brothers), which helps a lot with the process of learning XNA.

XNA involves a lot of little resource files, including one called a SpriteFont. The format for a SpriteFont file is really simple XML, so I decided to make a quick PowerShell function that would create a sprite font file. I could be very lazy, and simply accept whatever font name they gave me on faith, but I'm sure I would be very frustrated the first time I typed New-SpriteFont Arrial, and spent the next 15 minutes of my life wondering why my project wasn't compiling when all that was wrong was an extra 'r'. To save myself this future pain, I've made Get-Font, so New-SpriteFont can check what fonts exist before creating the file.

Making Get-Font wasn't all that hard. I simply used my Get-Type function to find all of the types named fonts (Get-Type | Where-Object { .Name -eq "Fonts" } | Select FullName) and then explored static member properties. I wrote Get-Font in the Windows PowerShell Integrated Scripting Editor, so I added a check to load up the right assembly (which I also found out by using Get-Type) in case I was trying to use Get-Font from the PowerShell console. All told, Get-Font took about 5 minutes to write, and ended up being a quick PowerShell lesson for my girlfriend. I added inline help, and then I was done. Here's Get-Font. I'll show New-SpriteFont in a separate post.

Get-Font

Synopsis:

Gets the fonts currently loaded on the system

Syntax:

Get-Font [[-font] [<Object>]] [<CommonParameters>]

Detailed Description:

Uses the type System.Windows.Media.Fonts static property SystemFontFamilies,
to retrieve all of the fonts loaded by the system. If the Fonts type is not found,
the PresentationCore assembly will be automatically loaded

Examples:

    -------------------------- EXAMPLE 1 --------------------------





# Get All Fonts
Get-Font

    -------------------------- EXAMPLE 2 --------------------------





# Get All Lucida Fonts
Get-Font *Lucida*

Command Parameters:

Name Description

font A wildcard to search for font names

Name	Description
font	A wildcard to search for font names

Here's Get-Font:

function Get-Font {
           
    <#
    .Synopsis
        Gets the fonts currently loaded on the system
    .Description
        Uses the type System.Windows.Media.Fonts static property SystemFontFamilies,
        to retrieve all of the fonts loaded by the system.  If the Fonts type is not found,
        the PresentationCore assembly will be automatically loaded
    .Parameter font
        A wildcard to search for font names
    .Example
        # Get All Fonts
        Get-Font
    .Example
        # Get All Lucida Fonts
        Get-Font *Lucida*
    #>
    param($font = "*")
if (-not ("Windows.Media.Fonts" -as [Type])) {
        Add-Type -AssemblyName "PresentationCore"
    }       

    [Windows.Media.Fonts]::SystemFontFamilies |
        Where-Object { $_.Source -like "$font" } 

}

Hope this Helps,

James Brundage [MSFT]

Automatically generated with Write-CommandBlogPost

Microcode: Cool Posts: Visual Studio Tips & Tricks

2008-12-18T01:57:11Z

Benko just posted a Visual Studio Tips & Tricks listing. It's missing on of my favorites, which a coworker on the PowerShell team, Ibrahim Abdul Rahim, showed me:

In a couple of circumstances, you'll see a red squiggle (like a syntax error) at the end of a line. This squiggle occurs when you're renaming something, or when you're referencing a class that exist in a namespace that hasn't been used. If you press ALT + SHIFT + F10, you'll get a handy menu that allows you to rename all of the references or add a "using" to that namespace. Since this is the first option, and both are really handy, you can fly using this keyboard combo in visual studio by using ALT + SHIFT + F10 + ENTER (to select the first item and add the using or rename other mentions of a property).

Hope this helps,

James Brundage [MSFT]

Media: xBox 360: 70 XNA Games in 24 Days

2008-12-13T14:26:50Z

It’s the weekend. I love my video games, and I really love my xBox 360. I’ve had one since just after they first came out, and I’ve slowly but surely accumulated about 70 physical games for the 360 made by major producers. I remember seeing the total xBox 360 games climb past one hundred, and I remember losing count. But until now, it’s been rare to see more than a few games come out each month.

But now there’s XNA.

XNA is a way for Joe Schmoe to write (and now profit) off of writing video games. Using XNA, you can write video games in C#. It’s been possible to write XNA games for a couple of years, but it was only November 19th that XNA games went out on xBox Live and were up for sale.

It’s been 24 days since then, and there are 70 community games on Xbox Live. They range from 200-800 points (~$3-~$10) and a large chunk of the cash goes to the one or two people that develop the games. They all have free trial versions.

So support poor game developers. Fire up your xbox 360 and check some XNA games out.

Enjoy,

James Brundage [MSFT]

Microcode: PowerShell Scripting Tricks: Scripting the Web (Part 3) (Resolve-Link, Get-WebPageLink)

2008-12-12T12:59:00Z

The first post in this series was learning to crawl. I introduced Get-Web, which allows you to use System.Net.Webclient to download web sites in a variety of ways. The next post was learning to walk. I showed us Get-MarkupTag, which helps coerce parts of the web into XML. Now we can start to really have some fun with the data and run wild.

Pulling out semi-structured data is one thing, but it’s important to be able to pull out more complex information as well. One interesting case is pulling out all of the links from a webpage. This task breaks down into four smaller tasks:

Downloading the page (done with Get-Web)
Getting the <a> tags in a meaningful way (done with Get-MarkupTag)
Extracting out the href attribute
Determining if the link is relative or absolute

To determine if the link is relative or absolute, I made a Resolve-Link function. It takes a base url (e.g http://www.foo.com/blah/blah.asp) and a link found on it, and returns the real item it resolves to. It optionally returns a property bag with the type of link and the resolved link.

Here’s Resolve-Link:

function Resolve-Link([Uri]$uri, [string]$link, [switch]$returnLinkType) { #.Synopsis # Resolves a relative or absolute link to an absolute url #.Description # Takes a uri and a link to a page and returns the absolute url, or # optionally returns a property bag with the link type # (absolute, relative, or host relative) and the link #.Parameter uri # The uri the link is located on #.Parameter link # The original link text #.Parameter returnLinkType # The return link type #.Example # Resolve-Link http:/www.microsoft.com/ /technet/scriptcenter if ($link.StartsWith("/")) { # Relative to Host site if ($returnLinkType) { return New-Object Object | Add-Member NoteProperty Type "Host Relative" -PassThru | Add-Member NoteProperty Link ([uri]"$($uri.Scheme)://$($uri.DnsSafehost)$($link)") -PassThru } return "$($uri.Scheme)://$($uri.DnsSafehost)$($link)" } else { if ($link.StartsWith("$($uri.Scheme)://")) { # Absolute Link if ($returnLinkType) { return New-Object Object | Add-Member NoteProperty Type "Absolute" -PassThru | Add-Member NoteProperty Link ([uri]$link) -PassThru } return $link } else { # Relative link $realLink = $uri.AbsoluteUri.Substring(0, $uri.AbsoluteUri.LastIndexOf("/")) + "/$link" if ($returnLinkType) { return New-Object Object | Add-Member NoteProperty Type "Relative" -PassThru | Add-Member NoteProperty Link ([uri]$realLink) -PassThru } return $realLink } } }

Once Resolve-Link was written, making Get-WebPageLink is an incredible snap. It’s below, and it actually takes only 3 lines to do the real work and 11 lines to explain the work and give examples.

function Get-WebPageLink($url) { #.Synopsis # Returns all of the links within a webpage #.Description # Resolves all <A> references and returns a property bag with # the text contained in the link, the page the link came from, # and the type of link returned (absolute, host relative, or relative) #.Parameter urltp # The page to get links from #.Example # Get-WebPageLink http://blogs.msdn.com/ Get-MarkupTag a (Get-Web $url) | Foreach-Object { Resolve-Link $url $_.Xml.Href -returnLinkType | Add-Member NoteProperty Text $_.Xml."#text" -PassThru } }

Go ahead and give Get-WebpageLink a whirl:

Get-WebpageLink http://blogs.msdn.com

Ready for some real fun? Remember way back when I did a post about getting RSS feeds in PowerShell with Microsoft.FeedsManager (Get-Feed). If you have that script handy, go ahead and check out this one liner that will refresh every RSS item you’ve got and extract out all of the links from it.

Get-Feed -recurse -articles | Foreach-Object { Get-WebPageLink $_.Link }

That particular command line can take a while, depending on how many blogs you subscribe to, but it gives you a brand new view on blogs (as a simmering stew of scripts, rather than just text to be read and comprehended).

There’s more fun to come in unlocking the web, but these two scripts should get you started in extracting a little more into the wild world of the web.

Hope this Helps,

James Brundage [MSFT]

Microcode: PowerShell Scripting Tricks: Scripting The Web (Part 2) (Get-MarkupTag)

2008-12-08T11:17:00Z

The first post about scripting the was a lot of waxing philosophical but little about how to extract data and give it form. There are several approaches, with various difficulties. I could build a full HTML parser and walk though object models, or I could use the object model of IE. Since I personally like to minimize dependencies, I’ve chosen to use System.Net.Webclient to download the webpage as text rather than use Internet Explorer and get it through a proper object model. This means I will either have to write an HTML parser, or I’ll have to nudge HTML into something more useful to avoid writing a full parser.

HTML is preciously close to XML, and XML is something that PowerShell supports quite well, so e IE sI decided to try to nudge HTML into XML. I wrote a function, Get-MarkupTag, which will extract out the text for a markup tag (e.g. <a>) and attempt to coerce it into XML.

It’s not a perfect approach, because there several things that are legal in HTML but not in XML. Here’s an inventory of every curveball I’ve hit so far.

Unclosed tags like <IMG>:

Some tags in HTML, like <IMG> are unmatched. I’ll first have to identify all of the unmatched tags and coerce them into property closed xml (e.g. ensure that <IMG is <IMG />). For things like <BR> or <HR> this is easy, because attributes are rare, but for IMG, getting this right is critically important, because it’s how you download images.

HTML Escape Sequences end up as XML unrecognized entities:

There’s a lot of ways to embed special characters within HTML that get recognized by the parser as XML entities. The most common example is   (the explicit space), but foreign currencies often show up in this format as well. I found a complete list online that I could extract out all of the sequences from, and I embedded a hashtable within to hold each item. Then I walked through the hashtable and replaced the escape sequences with their real value. However, since the entity is legal in any since the entity is legal in any case (e.g. &nsbp; is the same as &NBSP; to an HTML parser), I had to write a quick case insensitive replace.

Unquoted Attributes:

Most browsers will accept HTML attributes without quotes, e.g. <a href=www.microsoft.com>, but XML can handle this. Once I’ve got figured out the text in each tag, I need to check each attribute within that tag to ensure that the attributes are quoted.

Nested tags:

One of the more interesting parts of parsing HTML was nested tags. In order to match nested tags, what I did was use two regular expressions to identify all of the matches. While PowerShell has a –match operator, –match doesn’t work for multiline strings. So what I did was use New-Object to create the Regular Expression for extracting out a tag and then stored the result in a variable.

If the number of start tags was equal to the number of end tags, I assumed that the tags are balanced. If they’re odd, I’ll only deal with the start tags instead. If I need to determine which start tag matches which end tag, I need to use a data structure called a stack. Luckily, .NET comes with one (System.Collections.Stack). A stack in code is just like a stack in real life. You can put something on the top of the stack (Push), see what’s on top of the stack (Peek), and pull off the top of of the stack (Pop).

If I put start tags and end tags in one list, and sort them by the order that they occurred (in PowerShell, this is $list1 + $list2 | Sort-Object PropertyName), then I can simply walk through the list, pushing the start tags onto the stack and popping off end tags as I encounter them.

I don’t believe that I’ve taken care of all of the possible pains that exist (for instance, I know that georgraphic coordinates on wikipedia do not coerce into XML at this point. However, it’s a good start and it can yield some quite interesting results.

Since Get-MarkupTag involves a lot of escape sequences, which lends to very annoying blog formatting, I’m going to attach the script instead of embedding it. You can download it here.

If you look at the code in Get-MarkupTag, close to the end, you’ll see a trap statement. This means that there’s any error coercing the chunk of HTML into XML will be swallowed into the verbose log. If you want to examine why you can’t use the XML, simply set $verbosePreference to “continue” and you will be able to see the errors produced trying to convert the tag into XML. This is an example of a technique I call error redirection.

One of the kind of cool things you can do with it is extract the individual rows from ConvertTo-HTML:

$text = Get-ChildItem | Select Name, LastWriteTime | ConvertTo-HTML | Out-String Get-MarkupTag "tr" $text

It's also possible to extract out all of the pages www.microsoft.com links to:

$microsoft= (New-Object Net.Webclient).DownloadString("http://www.microsoft.com/") Get-MarkupTag "a" $microsoft | % { $_.Xml.Href }

What this fairly slightly quick and dirty approach to extracting out HTML gives you is a fairly simply way to start giving the data form. With XML you get the syntactic sugar PowerShell pours on XML, and you also get the power of XPath, and from that, you can start turning the HTML into something more interesting.

In the next post, I’ll look at some of the things you can do with this new toy.

Try it out on a few sites.

Hope this helps,

James Brundage [MSFT]

Microcode: PowerShell Scripting Tricks: Scripting The Web (Part 1) (Get-Web)

2008-12-01T06:13:06Z

Several of the last posts have tackled how to take the wild world of data and start to turn it into PowerShell objects, so that it’s easier to make heads or tails out of it. Once all of that data is in a form that PowerShell can use more effectively in the object pipeline, you can use PowerShell to slice and dice that data in wonderfully effective ways.

Data mining is a strange art that I find scripting languages normally are a little better at than compiled languages. I find that PowerShell can be incredibly useful for data miners for two important reasons. First, it can pull upon an unprecedented array of technologies (COM, WMI, .NET, SQL, Regular Expressions, Web Service, Command Line) in order to get structured data. Second, and more importantly, since you can easily create objects on the fly in PowerShell, and since PowerShell has wonderful string processing, you can often use PowerShell to extract structure data out of unstructured data.

Being able to pull data out of the mist and give it form is a very valuable skill, because people do not think in structured data. While it might be useful to the computing world to have most information in structured data, most people disseminating information don’t think or record their thoughts with rigorous structure. However, many people do record their thoughts in semi-rigorous structure, like the sentences and paragraphs you’re reading now.

The fact of the matter is that tons of data floats on the web requiring a very little bit of work and a small amount of art to extract it. This is because the web that people record their thoughts in is largely in HTML, and so, it is possible to learn a few ways to pull the data from the little structure that exists.

The first piece of the toolkit to extract out data from the Web is a function I’ve called Get-Web. Get-Web will simply download web pages, and it wraps part of the System.Net.Webclient object.

Using WebClient has pros and cons. The biggest pro is that it relies on .NET, rather than on a particular browser, which means that you can use it without IE. The biggest con is that a lot of web pages do checks on the browser in order to change how they display.

Get-Web is below. As with Get-HashtableAsObject, I’m using comments to declare some nifty inline help.

function Get-Web($url, 
    [switch]$self,
    $credential, 
    $toFile,
    [switch]$bytes)
{
    #.Synopsis
    #    Downloads a file from the web
    #.Description
    #    Uses System.Net.Webclient (not the browser) to download data
    #    from the web.
    #.Parameter self
    #    Uses the default credentials when downloading that page (for downloading intranet pages)
    #.Parameter credential
    #    The credentials to use to download the web data
    #.Parameter url
    #    The page to download (e.g. www.msn.com)    
    #.Parameter toFile
    #    The file to save the web data to
    #.Parameter bytes
    #    Download the data as bytes   
    #.Example
    #    # Downloads www.live.com and outputs it as a string
    #    Get-Web http://www.live.com/
    #.Example
    #    # Downloads www.live.com and saves it to a file
    #    Get-Web http://wwww.msn.com/ -toFile www.msn.com.html
    $webclient = New-Object Net.Webclient
    if ($credential) {
        $webClient.Credential = $credential
    }
    if ($self) {
        $webClient.UseDefaultCredentials = $true
    }
    if ($toFile) {
        if (-not "$toFile".Contains(":")) {
            $toFile = Join-Path $pwd $toFile
        }
        $webClient.DownloadFile($url, $toFile)
    } else {
        if ($bytes) {
            $webClient.DownloadData($url)
        } else {
            $webClient.DownloadString($url)
        }
    }
}

To walk through a few examples of Get-Web, simply point it to any webpage.

	Get-Web http://en.wikipedia.org/

To save a page to disk

	Get-Web http://www.msn.com/ -toFile www.msn.com.html

Just downloading the data is only the first step. All being able to download web data gives you is a way to get the mist into a bottle, but it doesn’t help you give it form. The next piece will cover how to pull the data out of the web and into PowerShell.

Hope this helps,

James Brundage [MSFT]

Microcode: PowerShell Scripting Tricks: More Joy of Hashtables (with Get-HashtableAsObject)

2008-11-30T05:10:27Z

I recently did two different "in depth" posts. One on The Joy Of Hashtables and another on Select-Object versus Add-Member. At the end of Select-Object vs Add-Member, I hinted at more convenient ways to write objects that used Script Properties. Script Properties are particularly useful because, unlike Note Properties, they are evaluated when they are requested. This means that PowerShell scripts that use script properties will often be faster than those using note properties. Sadly, they're kind of clunky to add. A script property will normally look like:

New-Object Object | Add-Member ScriptProperty Foo { Get-Random } -passThru

This gets pretty ugly, pretty fast.

Luckily, the post on hashtables gives us a good way out of this problem. Hashtables look fairly clean in PowerShell scripts, and, with a little bit of work, you can easily turn a hashtable into an object. Here's the Get-HashtableAsObject function.

In case you're curious, the comments inside of the function aren't just there for fun. They're an example of script-embedded help (which will not be out until CTP3). Once script-embedded Help is there, you can write help content for a function with just what's in the function below, and you'll be able to Get-Help on your function just like any other command.

function Get-HashtableAsObject([Hashtable]$hashtable) { #.Synopsis # Turns a Hashtable into a PowerShell object #.Description # Creates a new object from a hashtable. #.Example # # Creates a new object with a property foo and the value bar # Get-HashtableAsObject @{"Foo"="Bar"} #.Example # # Creates a new object with a property Random and a value # # that is generated each time the property is retreived # Get-HashtableAsObject @{"Random" = { Get-Random }} #.Example # # Creates a new object from a hashtable with nested hashtables # Get-HashtableAsObject @{"Foo" = @{"Random" = {Get-Random}}} process { $outputObject = New-Object Object if ($hashtable -and $hashtable.Count) { $hashtable.GetEnumerator() | Foreach-Object { if ($_.Value -is [ScriptBlock]) { $outputObject = $outputObject | Add-Member ScriptProperty $_.Key $_.Value -passThru } else { if ($_.Value -is [Hashtable]) { $outputObject = $outputObject | Add-Member NoteProperty $_.Key (Get-HashtableAsObject $_.Value) -passThru } else { $outputObject = $outputObject | Add-Member NoteProperty $_.Key $_.Value -passThru } } } } $outputObject } }

Let's check out a few practical examples of Get-HashtableAsObject:

Check out how we can use Get-HashtableAsObject to create a random card

$card = Get-HashtableAsObject @{ Card = {2..9 + "Jack", "Queen", "King", "Ace" | Get-Random} Suit = {"Clubs", "Hearts", "Diamonds", "Spades" | Get-Random} } $card $card $card

Notice that each time I output $card, the Card and Suit change. This is because the script property is being evaluated each time it is read. Essentially, it's shuffling every card. Cool, right?

The last post introduced you to PowerShell's ability to add hashtables together to merge them. Check out how we can use this to combine an object that generates user information and an object that generates system information.

$userInfo = @{ LocalUsers = {Get-WmiObject "Win32_UserAccount WHERE LocalAccount='True'"} LoggedOnUsers = {Get-WmiObject "Win32_LoggedOnUser" } } $systemInfo = @{ Memory = {Get-WmiObject Win32_PhysicalMemory} Drives = {Get-WmiObject Win32_LogicalDisk} } $liveUserInfo = Get-HashtableAsObject $userInfo $liveSystemInfo = Get-HashtableAsObject $systemInfo $allInfo = Get-HashtableAsObject ($userInfo + $systemInfo) $liveUserInfo.LocalUsers $allInfo.Drives

This trick is particularly convenient for quickly joining together chunks of logic from different scripts. For instance, if I wanted to add a property the $userInfo object that kicked off all users who had been logged on for over a period of time, I'd simply add another hashtable to the user info to accomplish this task. For those much more in tune with object-oriented programming, what this essentially gives me is the ability to inherit from an objects defined this way in PowerShell, without the joy of overloading. Faking overloading is considerably more complicated, so we'll save that one for another day, but I hope that the tricks contained within this post help you write more robust scripts.

Hope this helps,

James Brundage [MSFT]