Microsoft Compute Cluster now has a PowerShell Provider and Cmdlets

Microsoft Compute Cluster now has a PowerShell Provider and Cmdlets

  • Comments 3

Today the Compute Cluster group announced the release of their Tool Pack HERE. This includes a monitoring tool, a ping/pong diagnostic tool and (drum roll please……) a PowerShell Snap-in which provides a provider and a set of commands. I really like what they have done. I've include the text of their about file so that you can see for yourself.

Here is a test - take a look at the file below but don't read any of the text – just look at the command examples.

See if you can figure out what it is doing. Now go back and read it again with the text and ask yourself whether the text really added a lot more to your understanding.

Let me be very quick to state that I'm NOT making the case against documentation. Heavens no! Great documentation is super important. What I'm trying to point out is the self-documenting nature of PowerShell scripts. This is why we are so hard core about the naming guidelines.

At some point, the mess is going to hit the fan and you are going to be under lots of pressure to figure out what a script is doing and modify it to meet some new requirement. We've all been there before, trying to figure out what the heck a script is doing. Now imagine that same scenario but the script is written in PowerShell. We are working our butts off to make sure that that is a good experience for you.

BTW (I'm going to keep reminding you of this) you all play a super important role in making this come true. When you see anyone produce Cmdlets or Scripts which don't conform to the naming guidelines – you should scream! Chaos is the natural state of things and everyone is going to argue that their world is "special" and they need their own terms. If you accept that, then we'll end up with yet another incoherent mess. What those people should do is to use the standard terms and then define aliases from their "special" terms to the standard terms.

Anyway – I love what they have done with PowerShell. It's another example of a team delivering a great experience using PowerShell.

This train keeps gathering steam! SNOOPY DANCE!

Jeffrey Snover [MSFT]
Windows Management Partner Architect
Visit the Windows PowerShell Team blog at: http://blogs.msdn.com/PowerShell
Visit the Windows PowerShell ScriptCenter at: http://www.microsoft.com/technet/scriptcenter/hubs/msh.mspx

 

 

TOPIC CCPPSH

SHORT DESCRIPTION
Microsoft Compute Cluster Pack Snap-in for Windows PowerShell


LONG DESCRIPTION

The Compute Cluster Pack (CCP) snap-in lets you view and manage Compute Cluster
Pack features in Windows PowerShell. The snap-in adds Compute Cluster Pack
cmdlets and a CCP Windows PowerShell provider that lets you view and navigate
cluster data in Windows PowerShell.


INSTALL THE SNAPIN

To add the Compute Cluster Pack (CCP) snap-in to Windows PowerShell, at the
Windows PowerShell prompt, type:

Add-PSSnapin CCPSnapIn

To display CCP data correctly, load the CCP type file and formatting files, at
the Windows PowerShell prompt, type:

Update-TypeData -PrependPath "ccp.ps1xml"
Update-FormatData -PrependPath "ccp.format.ps1xml"

These commands affect only the current Windows PowerShell console. To add the
files to all Windows PowerShell consoles, add these commands to your Windows
PowerShell profile.

1. To determine whether you have a Windows PowerShell profile, at the Windows
PowerShell prompt, type:

test-path $profile

"True" means that you have a profile. "False" means that you do not. If
"True", skip to step 3.

2. To create a Windows PowerShell profile, at the Windows PowerShell prompt,
type:

new-item -type file -path $profile -force

The $profile variable is a built-in variable that stores the path to your
Windows PowerShell profile.

3. To open the profile in Notepad, type:

notepad $profile

4. Copy and paste the commands into your profile. The new profile is
effective when you restart Windows PowerShell.



NEW FEATURES

The CCP snap-in adds two features to Windows PowerShell:

-- CCPProvider, a Windows PowerShell provider that lets you browse and navigate
cluster data. (For information about Windows PowerShell providers, type
"get-help about_provider".)

-- A set of CCP cmdlets for Windows PowerShell, including Get-Node, Get-Job,
Get-Task, New-Job, and New-Task.


NAVIGATING CCP DATA IN WINDOWS POWERSHELL

You can use the features added by the CCP snap-in to manage CCP cluster in
Windows PowerShell.

The following command creates a new Windows PowerShell drive called CCP: and
mounts a cluster onto the Windows PowerShell CCP: drive. The cluster is
identified by its head node, which is stored in the value of the
CCP_SCHEDULER environment variable.

New-PSDrive -name CCP -provider CCPProvider -root $Env:CCP_SCHEDULER

After the drive is created, you can switch to this drive and access the data
as though it were located in the drive.

For example, the following command changes to the new CCP: drive. It uses the
"cd" alias for the Set-Location cmdlet, and the "dir" alias for the
Get-Childitem cmdlet.

PS c:\> cd ccp:
PS ccp:\> dir nodes

Name Status Proc Idle
---- ------ ---- ----
HPC-HN-01 Pend... 2 2
HPC-CN-01 Ready 2 2
...
HPC-CN-08 Ready 1 1

PS ccp:\> dir jobs

ID SubmitBy Name Status Priority
-- -------- ---- ------ --------
96 CCP\USER1 Job 1 Failed Normal



EXAMPLES: USING CCP CMDLETS IN WINDOWS POWERSHELL

The CCP cmdlets lets you manage clusters in Windows PowerShell. All of the CCP
cmdlets have a Scheduler parameter that specifies the head node that the command
affects.

If the Scheduler parameter is not included in the command, the CCP cmdlets use
the value of the CCP_SCHEDULER environment variable ($ENV:CCP_SCHEDULER)as the
default head node. You can use the Set-Content cmdlet to change the value of
CCP_SCHEDULER.

For help with any cmdlet, at the Windows PowerShell prompt, type:

get-help <cmdlet-name>"

For example:

get-help get-node



Get-Node: Gets all of the nodes under the head node.

This command uses the Get-Node cmdlet to list all of the nodes under the head
node.

PS ccp:\jobs> get-node

Name Status Proc Idle
---- ------ ---- ----
HPC-DEMO-CN-11 Pend... 2 2
HPC-DEMO-CN-12 Pend... 2 2
HPC-DEMO-CN-13 Pend... 2 2
HPC-DEMO-CN-14 Pend... 2 2
HPC-DEMO-CN-15 Pend... 2 2
HPC-DEMO-HN-01 Pend... 2 2


Enable-Node: Approves any pending nodes under the head node. This changes the
status of the node from Pending to Paused. Use the Resume-Node cmdlet to make
them ready.

The first command in this example uses the Get-Node cmdlet to get all of the
nodes under the head node. Then, it uses the pipeline operator (|) to send the
results of the Get-Node command to the Enable-Node cmdlet, which approves the
pending nodes. As a result, the nodes that were in a Pending status are now in a
Paused status.

The second command uses the Get-Node cmdlet to display the updated status of the
nodes.

PS ccp:\jobs> get-node | enable-node
PS ccp:\jobs> get-node

Name Status Proc Idle
---- ------ ---- ----
HPC-DEMO-CN-11 Paused 2 2
HPC-DEMO-CN-12 Paused 2 2
HPC-DEMO-CN-13 Paused 2 2
HPC-DEMO-CN-14 Paused 2 2
HPC-DEMO-CN-15 Paused 2 2
HPC-DEMO-HN-01 Paused 2 2


Resume-Node: Resumes any paused nodes under the head node. This changes their
status from Paused to Ready. Because newly-enabled nodes are paused, you need to
use this cmdlet to change their status to Ready.

The first command in this example uses the Get-Node cmdlet to get all of the
nodes under the head node. Then, it uses the pipeline operator (|) to send the
results of the Get-Node command to the Resume-Node cmdlet, which enables any
paused nodes. As a result, the nodes that were in a Paused status are now in a
Ready status.

The second command uses Get-Node to display the updated status.


PS ccp:\jobs> get-node | resume-node
PS ccp:\jobs> get-node

Name Status Proc Idle
---- ------ ---- ----
HPC-DEMO-CN-11 Ready 2 2
HPC-DEMO-CN-12 Ready 2 2
HPC-DEMO-CN-13 Ready 2 2
HPC-DEMO-CN-14 Ready 2 2
HPC-DEMO-CN-15 Ready 2 2
HPC-DEMO-HN-01 Ready 2 2

You can save a collection of nodes in a variable and use array notation to view
and manage the node objects. (For more information, type "get-help
about_shell_variable" and "get-help about_array".)

This command uses the assignment operator (=) to save the results of the
Get-Node command in the $a variable.

PS ccp:\> $a = get-node

This command displays the first node (element 0) in the $a variable.

PS ccp:\> $a[0]

Name Status Proc Idle
---- ------ ---- ----
HPC-DEMO-CN-11 Ready 2 2

You can use the Windows PowerShell filtering cmdlets, such as Select-Object,
Where-Object, and Sort-Object, to manage CCP data.

The first command uses the Select-Object to assign only the first two nodes in
the collection to the $a variable. Because the pipeline operator (|) has
precedence over the assignment operator (=), the "get-node | select-object
first 2" part of the command is completed first and the results are assigned
to the $a variable.

The second command displays the two nodes that are stored in $a.

PS ccp:\> $a = get-node | select-object -first 2
PS ccp:\> $a

Name Status Proc Idle
---- ------ ---- ----
HPC-DEMO-CN-11 Ready 2 2
HPC-DEMO-CN-12 Ready 2 2


Suspend-Node: Suspends ready nodes. This changes their status from Ready to
Paused.

The following command uses the Select-Object cmdlet and its First parameter to
select only the first three nodes returned by Get-Node. The second pipeline
operator in the command sends the objects representing the three nodes to
Suspend-Node, which pauses the nodes.

The second Get-Node command confirms that Suspend-Node affected only the first
three nodes.

PS ccp:\> get-node | select-object -first 3 | Suspend-Node
PS ccp:\> get-node

Name Status Proc Idle
---- ------ ---- ----
HPC-DEMO-CN-11 Paused 2 2
HPC-DEMO-CN-12 Paused 2 2



Get-Job: Gets all of the jobs on the head node or any nodes under the head node.

This command gets all of the jobs on the default head node, that is, the value
of the CCP_SCHEDULER parameter.

PS ccp:\> get-job

ID SubmitBy Name Status Priority
-- -------- ---- ------ --------
96 CCP\USER1 Job 1 Failed Normal


Restart-Job: Restarts jobs and returns them to the queue.

This command gets all of the jobs under the head node. It uses the pipeline
operator (|) to send the jobs to the Restart-Job cmdlet, which restarts any jobs
that not started. As a result, a job that had failed is now requeued.

Unlike the Start-Job cmdlet, the Restart-Job cmdlets starts only jobs that were
started, but failed, were cancelled, or were suspended.


PS ccp:\> get-job | restart-job
PS ccp:\> get-job

ID SubmitBy Name Status Priority
-- -------- ---- ------ --------
96 CCP\USER1 Job 1 Queued Normal


Stop-Job: Stops and cancels jobs.

This command gets all of the jobs under the head node. It uses the pipeline
operator (|) to send the jobs to the Stop-Job cmdlet, which stops the jobs that
it receives. As a result, a job that had been queued is now cancelled.

The second command uses Get-Job to display the cancelled jobs.

PS ccp:\> get-job | stop-job PS ccp:\> get-job

ID SubmitBy Name Status Priority
-- -------- ---- ------ --------
96 CCP\USER1 Job 1 Cancelled Normal


Get-Task: Gets the tasks in one or more jobs.

This command gets the tasks in a job and displays them in order by task ID.

This command uses the Get-Job cmdlet to get the jobs under the head node. It
uses array notation to select the second job ([1] in a zero-based array). The
pipeline operator (|) sends the job to the Get-Task cmdlet, which gets all of
the tasks in the job and sends them to the Sort-Object cmdlet, which sorts them
in order by task ID.

By default, tasks are displayed in the order that they are discovered.

PS ccp:\> (get-job)[1] | get-task | sort-object -property id

ID Status Name CommandLine
-- ------ ---- -----------
1 Finished CCPDemo 10K Tasks Group 00 echo 1
2 Finished CCPDemo 10K Tasks Group 00 echo 2
...
9998 Queued CCPDemo 10K Tasks Group 09 echo 998
9999 Queued CCPDemo 10K Tasks Group 09 echo 999
1... Queued CCPDemo 10K Tasks Group 09 echo 1000



This command gets the tasks in a selected job and uses the Where-Object cmdlet
to select only the tasks with IDs greater than 500 and less than 530.

The Where-Object command uses the "$_." notation, which refers to the current
object in the pipeline. The pipeline operator sends objects down the pipeline
one at a time. The "$_." notation indicates that the operation should be
performed on each object as it arrives from the pipeline. When all objects have
been processed, the results are sent down the pipeline to the next cmdlet.

PS ccp:\> (get-job)[1] | get-task | where-object { ($_.id -gt 500) ¨Cand
($_.id - lt 530)}

ID Status Name CommandLine
-- ------ ---- -----------
521 Queued CCPDemo 10K Tasks Group 00 echo 521
522 Queued CCPDemo 10K Tasks Group 00 echo 522
...
529 Queued CCPDemo 10K Tasks Group 00 echo 529
501 Finished CCPDemo 10K Tasks Group 00 echo 501
502 Finished CCPDemo 10K Tasks Group 00 echo 502
...
520 Finished CCPDemo 10K Tasks Group 00 echo 520


Stop-Task: Stops one or more tasks.

The first command in this series stops any queued tasks in a selected job that
have IDs between 500 and 530 (exclusive). It uses the Get-Job cmdlet to get the
jobs in the second job in the system (job [1] in a zero-based array) and the
Get-Task cmdlet to get the tasks in that job. It uses the Where-Object cmdlet to
select tasks with a status of "queued" and a task ID between 500 and 530. It
sends the selected tasks to the Stop-Task cmdlet, which stops them.

The second command displays the tasks with task IDs between 500 and 530. The
results show that the affected tasks are cancelled.

PS ccp:\> (get-job )[1] | get-task | where-object { ($_.id -gt 500) -and
($_.id - lt 530) -and ($_.status -eq "queued")} | stop-task

PS ccp:\> (get-job)[1] | get-task | where-object { ($_.id -gt 500) -and
($_.id - lt 530)}

ID Status Name CommandLine
-- ------ ---- -----------
501 Finished CCPDemo 10K Tasks Group 00 echo 501
502 Finished CCPDemo 10K Tasks Group 00 echo 502
...
520 Finished CCPDemo 10K Tasks Group 00 echo 520
521 Cancelled CCPDemo 10K Tasks Group 00 echo 521
...
528 Cancelled CCPDemo 10K Tasks Group 00 echo 528
529 Cancelled CCPDemo 10K Tasks Group 00 echo 529


Restart-Task: Restarts (or starts) tasks.

The first command in this series starts the tasks in a selected job that have
IDs between 500 and 530 (exclusive) and a status of "Cancelled". It uses the
Get-Job cmdlet to get the jobs in the second job in the system (job [1] in a
zero-based array) and the Get-Task cmdlet to get the tasks in that job. It uses
the Where-Object cmdlet to select tasks with a status of "cancelled" and a task
ID between 500 and 530. It sends the selected tasks to the Restart-Task cmdlet,
which starts them.

The second command displays the tasks with task IDs between 500 and 530. The
results show that the affected tasks are requeued.


PS ccp:\> (get-job)[1] | get-task | where-object { ($_.id -gt 500) -and
($_.id - lt 530) -and ($_.status -eq "cancelled")} | Restart-Task
PS ccp:\> (get-job)[1] | get-task | where-object { ($_.id -gt 500) -and
($_.id - lt 530)}

ID Status Name CommandLine
-- ------ ---- -----------
501 Finished CCPDemo 10K Tasks Group 00 echo 501
...
519 Finished CCPDemo 10K Tasks Group 00 echo 519
520 Finished CCPDemo 10K Tasks Group 00 echo 520
521 Queued CCPDemo 10K Tasks Group 00 echo 521
522 Queued CCPDemo 10K Tasks Group 00 echo 522
...
529 Queued CCPDemo 10K Tasks Group 00 echo 529


New-Job: Creates a new job.

The first command in this series creates a new job and stores it in the $j
variable.

The second command names the job "CCPDemo". It uses the <object>.<property>
format to access the Name property of the job object in the $j variable. It uses
the assignment operator (=) to assign the "CCPDemo" value to the Name property
of the job object in the $j variable.

The third command uses the Format-List cmdlet to display the properties of
newly-created job object as a list.

PS ccp:\> $j = new-job
PS ccp:\> $j.Name = "CCPDemo"
PS ccp:\> $j | format-list

AllocatedNodes :
AskedNodes :
CreateTime : 12/31/9999 11:59:59 PM
EndTime : 12/31/9999 11:59:59 PM
ExtendedJobTerms : {}
Id : 0
IsBackfill : False
IsExclusive : True
MaximumNumberOfProcessors : 1
MinimumNumberOfProcessors : 1
ErrorMessage :
Name : CCPDemo
SubmittedBy :
Priority :
Normal Project :
RunUntilCanceled : False
Runtime : Infinite
StartTime : 12/31/9999 11:59:59 PM
SoftwareLicense :
Status : NotSubmitted
SubmitTime : 12/31/9999 11:59:59 PM
Tasks : {}
User :
TaskCount : 0
Timestamp :


This command limits the CCPDemo job to four processors. It uses the
<object>.<property> format to access the MaximumNumberOfProcessors property of
the job object in the $j variable, and an assignment statement to assign a value
of "4" to the property.

PS ccp:\> $j.MaximumNumberOfProcessors = 4


New Task: Creates a new task in a job.

The first command in this series uses the New-Task cmdlet to creates a new task.
Then, it stores the new task in the $t1 variable.

The second command uses the <object>.<property> format to access the Name
property of the new task object in the $t variable. Then, it assigns the "t1"
name to the task.

The third command uses the object>.<property> format to access the CommandLine
property of the task object in the $t1 variable. Then, it assigns the "t1cmd"
value to the CommandLine property.


PS ccp:\> $t1 = new-task
PS ccp:\> $t1.Name = "t1"
PS ccp:\> $t1.CommandLine = "t1cmd"


This series of commands is much like the previous one. It creates a new task and
stores it in the $t2 variable; it assigns the name, "t2" to the task and the
value "t2cmd" to the CommandLine property.


PS ccp:\> $t2 = new-task
PS ccp:\> $t2.Name = "t2"
PS ccp:\> $t2.CommandLine = "t2cmd"

Add-Task: Adds a task to a job. The task and the job must exist before they are
associated.

The first command uses the Add-Task cmdlet to add the tasks stored in the $t1
and $t2 variables to the job stored in the $j variable. It uses the Job
parameter to specify the job.

The second command uses the Format-List cmdlet to display the properties of job
object in a list. It shows that the maximum number of processors was changed to
4, and that the new tasks are added.


PS ccp:\> $t,$t2 | add-task -job $j
PS ccp:\> $j | format-list

AllocatedNodes :
AskedNodes :
CreateTime : 12/31/9999 11:59:59 PM
EndTime : 12/31/9999 11:59:59 PM
ExtendedJobTerms : {}
Id : 0
IsBackfill : False
IsExclusive : True
MaximumNumberOfProcessors : 4
MinimumNumberOfProcessors : 1
ErrorMessage :
Name : CCPDemo
SubmittedBy :
Priority : Normal
Project :
RunUntilCanceled : False
Runtime : Infinite
StartTime : 12/31/9999 11:59:59 PM
SoftwareLicense :
Status : NotSubmitted
SubmitTime : 12/31/9999 11:59:59 PM
Tasks : {t1, t2name}
User :
TaskCount : 2
Timestamp :


Start-Job: Submits the job for execution.

The first command starts the job stored in the $j variable by piping the
variable to the cmdlet. After it is started, the status is "queued".

The second command gets the jobs under the head node. In this case, the job
stored in the $j variable is the only one.


PS ccp:\> $j | start-job
PS ccp:\> get-job

ID SubmitBy Name Status Priority
-- -------- ---- ------ --------
...
6 DOMAIN1\User1 CCPDemo Queued Normal


This command uses the Get-Task cmdlet to display the tasks in the newly started
job. It shows that the tasks in the job are queued when the job is started.


PS ccp:\> get-job)[5] | get-task
ID Status Name CommandLine
-- ------ ---- -----------
1 Queued t1 t2cmd
2 Queued t2 t2cmd


Using Variables and Arrays with CCP Data

This command saves the result of a Get-Node command in the $a variable. As a
result, $a contains objects representing all of the nodes under the head node.


PS ccp:\> $a = get-node
PS ccp:\> $a

Name Status Proc Idle
---- ------ ---- ----
HPC-DEMO-CN-11 Ready 2 2
HPC-DEMO-CN-12 Ready 2 2
HPC-DEMO-CN-13 Ready 2 2
HPC-DEMO-CN-14 Ready 2 2
HPC-DEMO-CN-15 Ready 2 2
HPC-DEMO-HN-01 Ready 2 2


The Get-Node command actually gets an array of objects, each of which represents
one node. You can use standard array notation to access the items in the array.

This command uses array notation to access the first node object in the $a
variable.

PS ccp:\> $a[0]

Name Status Proc Idle
---- ------ ---- ----
HPC-DEMO-CN-11 Ready 2 2


You can use the cmdlets that come with Windows PowerShell to manipulate the CCP
data.

This command uses the Select-Object cmdlet and its First parameter to select
only the first two nodes returned by Get-Node. The results are stored in the $a
variable.

The parentheses in this command have no effect. They are added to show that the
Get-Node and Select-Object commands are completed before the result is stored in
the $a variable. (The pipeline operator takes precedence over the assignment
operator.)


PS ccp:\> $a = (get-node | select-object -first 2)
PS ccp:\> $a

Name Status Proc Idle
---- ------ ---- ----
HPC-DEMO-CN-11 Ready 2 2
HPC-DEMO-CN-12 Ready 2 2


This command uses the Suspend-Node cmdlet, which pauses ready nodes.

The command uses the Select-Object cmdlet and its First parameter to select only
the first three nodes returned by Get-Node. The second pipeline operator in the
command sends the objects representing the three nodes to Suspend-Node, which
pauses the nodes.

The second Get-Node command confirms that Suspend-Node affected only the first
three nodes.

PS ccp:\> get-node | select-object -first 3 | Suspend-Node # node pause
PS ccp:\> get-node

Name Status Proc Idle
---- ------ ---- ----
HPC-DEMO-CN-11 Paused 2 2
HPC-DEMO-CN-12 Paused 2 2
HPC-DEMO-CN-13 Paused 2 2
HPC-DEMO-CN-14 Ready 2 2
HPC-DEMO-CN-15 Ready 2 2
HPC-DEMO-HN-01 Ready 2 2


This command uses the Where-Object cmdlet to select only the paused nodes from
the array that Get-Node gets. The paused nodes are sent to the Resume-Node
cmdlet, which changes their status from "paused" to "ready."

The pipeline operator sends objects down the pipeline one at a time. The Where-
Object command uses the "$_." symbol, which refers to the current object sent
down the pipeline. The operation that it specifies is performed on each object
as it arrives. When all objects have been processed, the results are sent to the
next cmdlet.


PS ccp:\> get-node | where-object {$_.status -eq "paused"} | Resume-Node
PS ccp:\> get-node

Name Status Proc Idle
---- ------ ---- ----
HPC-DEMO-CN-11 Ready 2 2
HPC-DEMO-CN-12 Ready 2 2
HPC-DEMO-CN-13 Ready 2 2
HPC-DEMO-CN-14 Ready 2 2
HPC-DEMO-CN-15 Ready 2 2
HPC-DEMO-HN-01 Ready 2 2


SEE ALSO
For information about individual Cmdlets, use the help alias (for the
get-help Cmdlet) followed by the Cmdlet name. For example, to view information
about the get-node Cmdlet, Enter the following command:

help get-node

To get a full list of Cmdlets for CCP, enter the following command:

get-command -pssnapin CCPSnapIn

To change the value of the environment set CCP_SCHEDULER, use this command:

set-content env:CCP_SCHEDULER your_scheduler


Leave a Comment
  • Please add 7 and 5 and type the answer here:
  • Post
Page 1 of 1 (3 items)