PowerShell For Fun

PowerShell Script: Amazon WII Watch

2007-11-30T14:22:00.000-05:00

I just wanted a WII for Christmas gift and didn't want to spend $500 on ebay.

A Small powershell script will do!

# Begin of script
# =============================================================
# This code is for test purposes only. Use it at your own risk.
# =============================================================
param([string] $query= "B0009VXBAQ", [string] $button = "btn-atc._V46858960_.gif")

#Submit query sequence
"================================================================="
"ASIN: " + $query
"Button: " + $button
""
"Submit query ..."
$uri="http://www.amazon.com/gp/product/" + $query
$AmazonClient = new-object System.Net.WebClient

#checking for results
While ($true)
{
$pagecontent = $AmazonClient.DownloadString($uri);
"================================================================="
# Get Availability
if ($pagecontent -match $button)
{
write-host "In Stock`a`a`a" -foregroundcolor Red
}
else
{
write-host "out of Stock" -foregroundcolor yellow
}
[datetime]::Now.tostring()
"Wait..."
Start-sleep 60
}

#End of script

=================================================================
out of Stock
2007-11-30 13:07:52
Wait...
=================================================================
In Stock

2007-11-30 13:08:58
Wait...
=================================================================
out of Stock
2007-11-30 13:10:01
Wait...
=================================================================
In Stock

2007-11-30 13:11:05
Wait...

Tags: msh monad PowerShell

New Project Home at CodePlex.com

2007-02-28T14:07:00.000-05:00

Private workspaces @ gotdotnet.com will be phased out on 3/20/2007. I moved new project home to CodePlex.com @ http://www.codeplex.com/powershellremoting. You can get news, report bugs and request feature there.

We are working on a beta release on X64 and VISTA. We will come up a road map for future PowerShell Remoting development soon. If you have any suggestions, please reply here or discussion section @ new Powershell Remoting project home.

Tags: msh monad PowerShell

PowerShell Remoting Go OpenSource!

2007-01-25T14:47:00.000-05:00

You can now download full source code of PowerShell Remoting from my workspace.

Let's welcome our new developers: Romangeeko and JaviRuso. We are still looking for developers. If you are intersted, please Email me (shenzhonghao AT gmail DOT com).

Enjoy.

Tags: msh monad PowerShell

Looking For Developers

2006-12-12T14:31:00.000-05:00

I am very busy with my daytime job and my new born daughter. So I don't have enough time for PowerShell Remoting Project. But I would like to continue the development of this project in a community (probably open source).

I have been contact some friends also intersted in PowerShell Remoting.There are also a couple of user requesting for source code of PowerShell Remoting. If you are interested in this project and would like to contribute your time to it, please Email me (shenzhonghao AT gmail DOT com). It would be great if you could include some details about your previous development experience. Any other suggestion about this project is also welcome.

Tags: msh monad PowerShell

PowerShell Remoting version 0.2.9.0 For V1.0

2006-11-15T14:35:00.000-05:00

PowerShell Remoting version 0.2.9.0
1. Rebuild For PowerShell V1.0 RTW.
2. Fix "All User" profile not load at server problem.

Download here (381k)

Some user reported that PowerShellRemoting.ClientPSHost.dll is not registered properly in x64 platform. So I also uploaded a x64 build of PowerShell Remoting. Warning: This package have not been tested on x64 platform, use it at your own risk.

Please uninstall old version of Powershell Remoting before uninstall PowerShell RC2. After install PowerShell v1.0, you can install new version of Powershell Remoting.

Tags: msh monad PowerShell

PowerShell Remoting version 0.2.8.0 For RC2 Drop

2006-09-27T12:42:00.000-04:00

Windows PowerShell Team finally released PowerShell RC2. If you are running windows server 2003, you can enjoy new active directory provider. If you are a windows XP (non-English) user like me, I suggest you download Windows PowerShell 1.0 RC2 Language-Neutral Package for Windows XP (KB925228)

Some important changes not in release note.

Big window, different color

> $host.ui.rawui

ForegroundColor       : DarkYellow
BackgroundColor       : DarkMagenta
CursorPosition        : 0,4
WindowPosition        : 0,0
CursorSize            : 25
BufferSize            : 120,3000
WindowSize            : 120,44
MaxWindowSize         : 120,44
MaxPhysicalWindowSize : 128,44
KeyAvailable          : False
WindowTitle           : Windows PowerShell

If you like color setting of old version

>$host.ui.rawui. ForegroundColor = "gray"
>$host.ui.rawui. BackgroundColor = "black"
>clear

Path change
1. Home
C:\WINDOWS\system32\windowspowershell

2. User Profile folder
<My Documents>\WindowsPowerShell

3. Assembly change
System.Management.Automation now goes to GAC

PowerShell Remoting version 0.2.8.0
1. Rebuild for PowerShell RC2 version

Download from here. (381k)

Please uninstall old version of Powershell Remoting before uninstall PowerShell RC1. After install PowerShell RC2, you can install new version of Powershell Remoting.

Tags: msh monad PowerShell

Do You Know There is a $Host Variable

2006-08-06T23:19:00.000-04:00

PowerShell has a build-in $host variable which expose a System. Management. Automation. Internal. Host. InternalHost object. It is an internal object so you can't access it from assembly but monad team made it directly accessible from PowerShell script. $host is interesting and powerful. Just an example how powerful it is: my PowerShell Remoting "steal" the $host.UI for user interface remoting.

$Host is an implementation of System. Management. Automation. Host. PSHost abstract class. PSHost defined some important properties and methods which will be used for all cmdlet. For more detailed information about PSHost, you can consult Monad SDK online.

Let's see what we can do with $host
1. Some read-only properties provide information about current host.
These could be useful when you have to distinguish among multiple hosts (For example, my PowerShell Remoting has a remote host. Remote host have different Name, InstanceId, Version, etc).

CurrentCulture
CurrentUICulture
InstanceId
Name
PrivateData
Version

2. Two methods to support legacy application (like ping.exe).
Usually you do not need to call them directly. Monad engine will call them automatically if you invoke legacy application.

NotifyBeginApplication()
NotifyEndApplication()

3. Two methods about nested prompt.
I already discussed how to use those methods previously.

EnterNestedPrompt()
ExitNestedPrompt()

4. One method to force PowerShell exit.
This could be useful if some sever error occurs and you want to quit PowerShell.exe from your script. You can also pass error code to environment. For example:

>$host.SetShouldExit(10000)

5. $host.UI
This is an implementation of PSHostUserInterface object which expose some import user interface API.

5.1 public override Dictionary<string, PSObject> Prompt(string caption, string message, Collection<FieldDescription> descriptions)
This API is called when cmdlet required field(s) is null or empty. It is capable of get user input ([string]) and cnvert them to almost any .Net type in current appdomain. But the task of constructing Collection<FieldDescription> from script can be daunting.

5.2 public override int PromptForChoice(string caption, string message, Collection<ChoiceDescription> choices, int defaultChoice)
This is standard PowerShell text-base menu system. Don't re-invent wheels! If you want to provide a menu for user input, look nowhere and this method got you all covered. Mow has a nice example of how to use this methods.

5.3 Some more input API to get user input

ReadLine: Read a line from standard input, similar to read-host cmdlet
PromptForCredential: Get PSCredential object from User input, similar to get-credential cmdlet.
ReadLineAsSecureString: read from standard input and convert to SecureString, similar to Read-host -AsSecureString

5.4 Some output API to direct output to different channel and with color support.

Write: Write to standard output without /n/r
WriteLine: Write a line to standard output
WriteProgress: Wrtie to Progress panel, similar to Write-progress cmdlet
WriteDebugLine: Write to Debug channel, similar to Write-debug cmdlet
WriteErrorLine: Write to Standard Error, similar to Write-error cmdlet
WriteVerboseLine: Write to Verbose channel, similar to Write-verbose cmdlet
WriteWarningLine: Wrtie to Warning channel, similar to Write-warning cmdlet

5.5 $Host.UI.RawUI
This is an implementation of PSHostRawUserInterface abstract class. This is a new concept in PowerShell. In the prompt post, we use this object to do a lot of tricks.

5.5.1 Some properties to control Console window size, window title, cursor, buffer size, background/foreground color

BackgroundColor
BufferSize
CursorPosition
CursorSize
ForegroundColor
KeyAvailable
MaxPhysicalWindowSize
MaxWindowSize
WindowPosition
WindowSize
WindowTitle

5.5.2 Some console buffer API. Lee has a great burn-console script to demonstrate these methods.

FlushInputBuffer
GetBufferContents
LengthInBufferCells
NewBufferCellArray
ScrollBufferContents
SetBufferContents

5.5.3 public override KeyInfo ReadKey(ReadKeyOptions options)
It is a powerful API to read keyboard input. Check out the retuned KeyInfo object, with this method, one can theoretically write a script act like a simple screen editor.
Here is the ReadKeyOption enum:

[Flags]
public enum ReadKeyOptions
{
      AllowCtrlC = 1,
      IncludeKeyDown = 4,
      IncludeKeyUp = 8,
      NoEcho = 2
}

For example

> $host.ui.RawUI.ReadKey(12)
a
     VirtualKeyCode           Character     ControlKeyState             KeyDown
     --------------           ---------     ---------------             -------
                 65                   a              262176                True

Just to remind you, my PowerShell Remoting remote host implemented all Host, UI and RawUI. So feel free to explore those interesting API on you remote host.

Have Fun

Tags: msh monad PowerShell

Runspace Remoting

2006-07-18T23:11:00.000-04:00

Scott Hanselman is one of the early users and supporters of my PowerShell Remoting. He did an interesting post about Runspace Remoting. Briefly, he used export-clixml to serialize objects at server and used import-clixml to reconstruct PSObject at client. I have tried similar mechanism and failed. Actually Karl Prosser and I have been talking about using export-clixml/import-clixml a while ago . Great job, Scott!

Despite of my failure on serialization/de-serialization of PSObject, there is other reasons made me choose user interface remoting:

Monad was designed to capable of user interface remoting. Monad engine will manage to display and format (out-default) remote objects. Monad engine can even truncate objects collection for you if there are too many objects in the pipline.
Not all tasks require local objects exactly matching remote objects. For most remote scripting tasks, user interface remoting is good enough. User interface remoting deliver similar user experience as local shell. SSHD already proved itself successful story by similar mechanism.
User can manipulate "Real" remote objects via user interface remoting (No "Heisenberg Uncertainly Principle of PowerShell").
Serialization/de-serialization method has its own limitations.
- It can use a lot of resources and band-width. Consider the situation when multiple runspace ouput huge collection of objects at the same time, even with proper threading management server could still be overloaded.
- What if you just need one simple property of an object with hundreds or thounds of properties, you would have to serialize all public properties of that object and transfer it to client.
- It can loose some property associate with original objects.
"Clustering" multiple servers is possible with user interface remoting. I am trying to "fork" single client input to multiple servers by simply add current client component to server component. (There are still some problems with this model)

PS: To Monad Team, a public helper API to serialize PSObject to string will be really helpful.

Have Fun

Tags: msh monad PowerShell

How to: Using PowerShell through SSH

2006-07-13T23:51:00.000-04:00

It is possible to use PowerShell through SSH. Read the story from The Hive Archive.

But there are certain limitations:
1. You have to install cygwin, sshd and ssh client like putty (Well I can certainly live with that. Actually they are always on my hard drive).
2. No prompt
3. some Raw UI might not work

I am really looking forward to the new version of PowerShell with buildin Remoting function. According to Jeffrey Snover : Our plans for remoting are to leverage WS-MGMT the remoting protocol recently standardized through the DMTF.

Have Fun

Tags: msh monad PowerShell

Why There Is an Out-Default Cmdlet

2006-07-12T23:37:00.000-04:00

PowerShell have a couple of output control cmdlets:

> gcm out-*

CommandType     Name                            Definition
-----------     ----                            ----------
Cmdlet          Out-Default                     Out-Default [-InputObject <P...
Cmdlet          Out-File                        Out-File [-FilePath] <String...
Cmdlet          Out-Host                        Out-Host [-Paging] [-InputOb...
Cmdlet          Out-Null                        Out-Null [-InputObject <PSOb...
Cmdlet          Out-Printer                     Out-Printer [[-Name] <String...
Cmdlet          Out-String                      Out-String [-Stream] [-Width...

Out-Default cmdlet is one of the most mysterious one.

> help out-default

NAME
    Out-Default

SYNOPSIS
    The default controller of output.

DETAILED DESCRIPTION
    The standard treatment at the end of a pipeline is to send all objects to o
    ut-default. Out-default then sends them all to format-default. It takes th
    e objects that return and sends them to the default destination. For this
    reason, it is functionally equivalent to out-host but is not called from th
e console.

The help message does not help at all. Luckily enough, Jeffrey Snover has a blog entry talking about this cmdlet. In summary, It will "figure out how to format and output the object stream" and send them to host via Out-host cmdlet.

Here is my two cents:
1. Out-Default cmdlet is NOT for interactive console User.
Every interactive command from console will have Out-Default appended automatically by Monad engine. Add Out-Default in the middle of pipline could even cause unexpected output.

> gps | format-table
> gps | out-default | format-table # format-table will not work.

2. Out-Default cmdlet is for Monad hosting application.
To have a full-blown Monad hosting application, you have to create your own "Host" (System.Management.Automation.Host.PSHost) which implement interface to process output (If you don't have your PSHost output interface implemented, your will get an exception when Out-default finally called out-host.). Every interactive command from user input should have Out-Default appended. Then the output of user command were assessed by out-default, formatted by format-* cmdlet, and eventually sent to user interface by out-host cmdlet.

using System.Management.Automation;
using System.Management.Automation.Host;
using System.Management.Automation.Runspaces;

namespace HostingExample
{
   class TestHost
   {
       static void Main(string[] args)
       {
           string command = "gps";
           PSHost myhost = new MyHost();
           Runspace myRunspace = RunspaceFactory.CreateRunspace(myhost);
           myRunspace.Open();
           Pipeline pipeline1 = myRunspace.CreatePipeline(command, true);
           pipeline1.Commands[0].MergeMyResults(PipelineResultTypes.Error, PipelineResultTypes.Output);
           pipeline1.Commands.Add("out-default");
           Pipeline1.Invoke();
       }
   }

   Class MyHost : PSHost
   {
            // MyHost class should be derived from PSHost abstract class.
   }
}

Although you don't have to (and probably should not) type "Out-Default" at the end of your command, it is always there working for you. Given this reason, Out-Default cmdlet is registered and loaded as default cmdlets.

You see it is there, you do not use it, but it is definitely important.

Tags: msh monad PowerShell

Naive and Generic Object Collections in Powershell

2006-07-02T23:17:00.000-04:00

I believed that you must have dealt with collections in Powershell Scriting. PowerShell has a nice way to support naive object array (object[]):

>$a = 1,2,3 #comma were interpreted as object[]
>$a.gettype().AssemblyQualifiedName
System.Object[], mscorlib, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089

>$a= 1..10 # .. were interpreted as continuous filled object[]
>$a.gettype().AssemblyQualifiedName
System.Object[], mscorlib, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089

For naive object ArrayList/stack/Queue, you have to use new-object cmdlet

> $a = new-object System.Collections.Stack
> $a.GetType().AssemblyQualifiedName
System.Collections.Stack, mscorlib, Version=2.0.0.0, Culture=neutral, PublicKey
Token=b77a5c561934e089
> $a.Push($([System.Net.IPAddress] "192.168.0.1"))

Most of time, you are fine with naive object model because PowerShell will do the type conversion for you.

>$a= 1..10
>$a |gm
   TypeName: System.Int32

Name        MemberType Definition
----        ---------- ----------
CompareTo   Method     System.Int32 CompareTo(Int32 value), System.Int32 Com...
Equals      Method     System.Boolean Equals(Object obj), System.Boolean Equ...
GetHashCode Method     System.Int32 GetHashCode()
GetType     Method     System.Type GetType()
GetTypeCode Method     System.TypeCode GetTypeCode()
ToString    Method     System.String ToString(), System.String ToString(IFor...

But sometimes it is not good enough. You probably want to use generic object model.

For generic array (like System.Net.IPAddress[]), it requires a conversion:

>$a = ($([System.Net.IPAddress] "192.168.0.1"),$([System.Net.IPAddress] "127.0.0.1") )
>$a.gettype().AssemblyQualifiedName
System.Object[], mscorlib, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089
>$b= [System.Net.IPAddress[]] $a
> $b.GetType().AssemblyQualifiedName
System.Net.IPAddress[], System, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089

For generic collection (like System.Collections.ObjectModel.Collection<System.Net.IPAddress>), it becomes a little nasty (it requires Assembly Qualified Name) :

>$a = New-Object System.Collections.ObjectModel.Collection"`1[[System.Net.IPAddress, System, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089]]"
> $a.GetType().AssemblyQualifiedName
System.Collections.ObjectModel.Collection`1[[System.Net.IPAddress, System, Vers
ion=2.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089]], mscorlib, Vers
ion=2.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089
> $a.Add($([System.Net.IPAddress] "192.168.0.1"))

What about generic Stack?

> $a = New-Object System.Collections.Generic.Stack"`1[[System.Net.IPAddres
s, System, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089]]"
> $a.GetType().AssemblyQualifiedName
System.Collections.Generic.Stack`1[[System.Net.IPAddress, System, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089]], System, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089
> $a.Push($([System.Net.IPAddress] "192.168.0.1"))
> $a.pop()
IPAddressToString : 192.168.0.1
Address           : 16820416
AddressFamily     : InterNetwork
ScopeId           :
IsIPv6Multicast   : False
IsIPv6LinkLocal   : False
IsIPv6SiteLocal   : False

Have Fun

Tags: msh monad PowerShell

PowerShell Remoting version 0.2.7.1

2006-06-29T23:18:00.000-04:00

Download here

Uninstall old version of PowerShell Remoting before install newer version.

New GUI client manager: Save/delete server information & connection option, generate connection script automatically, launch powershell remoting client directly.

Add option to reject connection if user found invalid remote X509 certificate.

Bug fix: Null login credential (press Cancel in CredUI window) cause server crash and client throw Null reference exception.

Bug fix: other minor bugs

Check out the help.chm in release package for details.

Have Fun

Tags: msh monad PowerShell

Enter Nested Prompt function

2006-06-22T23:27:00.000-04:00

In windows PowerShell, $Host object provides direct access to some important APIs. One of the interesting functions is $Host.EnterNestedPrompt(). Frankly speaking, it did not appeal to me at all when I first tried it. But I discovered this beauty after I implemented my own PSHost in PowerShell Remoting,

What did it do?

Suspend & Save current execution context state.
Create a new nested execution context.(Nested pipline, new prompt execution helper and new script excution helper)
Push the new nested execution context to stack. (PowerShell use a static stack. In my PowerShell Remoting, it is more complicate. Every client has its own host; every host has its own stack for nested prompt.)
Increase $NESTEDPROMPTLEVEL

That is to say:

Your current pipeline is suspended. All variables and errors state were saved.
In the meantime, you still have access to monad engine. You can have new prompt, run new script or other interactive command.
When you are done with the new scope, you can call invoke $Host.ExitNestedPrompt() or "exit" (which will call $Host.ExitNestedPrompt()) to return to previous pipeline.

What can I do with it?

First, Let's see how "Suspend" works when using -confirm option or "Set-PSDebug -Step"

> kill 2804 -Confirm

Confirm
Are you sure you want to perform this action?
Performing operation "Stop-Process" on Target "Notepad (2804)".
[Y] Yes [A] Yes to All [N] No [L] No to All [S] Suspend [?] Help
(default is "Y"):

> Set-PSDebug -Step
> D:\ps1\special.ps1

Continue with this operation?
1+ D:\ps1\special.ps1
[Y] Yes [A] Yes to All [N] No [L] No to All [S] Suspend [?] Help
(default is "Y"):

Secondly, $Host.EnterNestedPrompt() can be used as break point when debugging your script.

> 1..10 | %{if($_ -eq 5) {$host.EnterNestedPrompt()};$_}
1
2
3
4
>>> $_
5
>>> exit
5
6
7
8
9
10

Be careful

Nested prompt and original prompt share same variables provider. So when you change a variable in nested prompt, the change will remain effective in original prompt.
Comannd will be written in history when it is completed. So a suspended command will not be found in history.
The maximum depth of nested prompt level ($NESTEDPROMPTLEVEL) is 128. So always check $NESTEDPROMPTLEVEL for current nested prompt level. You can write it into your prompt function.
Do not try to make a nested function using $Host.EnterNestedPrompt(). But you can always do this.

function factorial
{
    param ([int]$n = 1)
    if ($n -le 0)
    {
        "Invalid parameter."
         return
    }
    if ($n -eq 1)
    {
        return 1
    }
    return $n * (factorial ($n-1))
}

Reference: Start-NewScope

Tags: msh monad PowerShell

What Access Rights Do You Have?

2006-06-12T11:30:00.000-04:00

A PowerShell script to check User/Group access rights on FileSystem and Registry.

Yesterday, I was reading post on Mark's Sysinternals Blog: The Power in Power Users. It says "a user that belongs to the Power Users group can easily elevate themselves to fully-privileged administrators". He used a tool called AccessChk to find out what access rights does "Power User" group have.

If you read my serise posts on access control list in monad. you will find out that we can get/set access rule in PowerShell directly without using extra tools. So I wrote a PowerShell script (Check-AccessRights.ps1) to get similar function as AccessChk. It is pretty dirty, but dose the job. You may want to try

D:\ps1\Check-AccessRights.ps1 .\
# Check your access rights on current path (could be filesystem or registry)

D:\ps1\Check-AccessRights.ps1 HKLM:\sytem\CurrentControlSet\Services $true "Power Users"
# HKLM:\sytem\CurrentControlSet\Services is Path to check
# $true is to get child object ACL recursively
# "Power Users" is user/group

Be prepared if you use -Recurse option ($true), you may want to redirect results to a file.

#################################################################
#
# File: Check-AccessRights.ps1
# Author: Tony (http://MSHForFun.blogspot.com/)
# Parameters:
#    $Path: PowerShell Path (for example, c:\ or HKLM:\)
#    $Recurse: Check child object recursively
#    $Account: User name / Group (for example, "domain\alice",
#               "Power Users")
#
################################################################
param([string] $Path = {throw "Please specify a path"}, [bool] $Recurse = $false, [string] $Account)
if (-not (test-path $Path))
{
    $Path + " not exists!"
    return
}
if ([string]::ISNullOrEmpty($Account))
{
    $SID = ([System.Security.Principal.WindowsIdentity]::GetCurrent()).Owner
}
else
{
    $SID = (new-object System.Security.Principal.NTAccount($Account)).Translate([System.Security.Principal.SecurityIdentifier])
    if ($SID -eq $null) {return}
}
"Account: " + $SID.ToString()
"Path: " + $Path
$AccessRules = (get-acl $Path).Access
if ($AccessRules -eq $null)
{
    "Can't get access rules!"
    return
}
$AccessRules|foreach-object {
    $CurrentSID = $_.IdentityReference.Translate([System.Security.Principal.SecurityIdentifier])
    if ($CurrentSID.ToString() -eq $SID.ToString())
    {
        $_
    }
}
if ($Recurse)
{
    Get-ChildItem $Path -Recurse| foreach {
        "========================================"
        "Child Path: " + $_.ToString()
        $AccessRules = (get-acl $Path).Access
        if ($AccessRules -eq $null)
        {
            "Can't get access rules!"
            return
        }
        $AccessRules|foreach-object {
            $CurrentSID = $_.IdentityReference.Translate([System.Security.Principal.SecurityIdentifier])
            if ($CurrentSID.ToString() -eq $SID.ToString())
            {
              $_
            }
        }
    }
}

Have Fun

Tags: msh monad PowerShell

PowerShellIDE: the Good, the Bad and the Ugly

2006-06-07T23:45:00.000-04:00

PowerShellIDE? Yes, an IDE for PowerShell! Oh, boy! That's a whole lot fun. If you like PowerShell, you should try this out.

The good:
It is advertised (it is free at this moment) as "The World's First PowerShell Editor" (what is "PowerShell Analyzer" then?) and wrapped in a dazzling GUI (you should really checkout its Mac style chooser).

You got intellisense functionality, a debugger, variable explorer and properties window.

You can set up a breakpoint in your script and checkout pipline in a GUI environment.

It is the first GUI Monad hosting application I have seen which can handle legacy console application (like ping.exe or netshe.exe)

The bad:
1. Not yet implemented.
You'd better forgot about following API:
Read-host -AsSecureString (ReadLineAsSecureString)
PromptForCredential
SetBufferContents
Getbuffercontents

2. It is not the same!
Don't expect same user experience with
Write-progress
$host.UI.ReadLine() (Oops, Popup window)
$host.UI.RawUI.ReadKey()

The ugly:
PowerShellIDE has to be run under Administrator account. This is killing me. Come on, men! Why it has to read prefetch?

PowerShellIDE.E:3300 OPEN C:\WINDOWS\Prefetch\POWERSHELLIDE.EXE-3628D30E.pf ACCESS DENIED

Some other thoughts:
I came across someone's post (I forgot where) talking about PowerShell: "This thing is for Programmer." Now, we have an IDE for PowerShell. It again prove "This thing is for Programmer." Remember the famous joke about Microsoft Visual Studio:" The IDE is so good and you were having so much fun with it. Eventually you forgot what you were doing." This is exactly what happened here:

Despite of intellisense, the entire "breakpoint, pipline reader and properties" thing can be done in PowerShell.exe command line directly. Just add "$host.EnterNestedPrompt()" in my script, I can get current pipline status, explore properties as I wish. But in the windows world, everyone indulged themselves to the IDE thing. People always try to find an IDE desperately even there is no need for that. I bet those guys from "slash dot" will laugh at us again.

Tags: msh monad PowerShell

PowerShell Remoting version 0.2.6.0

2006-06-06T00:25:00.000-04:00

Download here

Uninstall old version of PowerShell Remoting before install newer version.

Write Server Information & Error to EventLog (Source: PowerShellRemoting). Log file will no longer be used by newer version.

SSL support using SSLStream.

Warning: SSL is for advanced user.

Get X509 certificate for server. For testing purpose, you can use makecert.exe (included in .NET framework SDK)
Install X509 certificate at StoreLocation: LocalMachine, StoreName: My (personal).
Grand "NetworkService" account read access to X509 certificate associated private key file.
Set [HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\PowerShellRemoting\Parameters\ X509] to thumbprint of X509 Certificate.

There is pretty good walkthrough at John Howard's blog.

X509 Option

DebugHost Option (Default 0: false)

Bug fix: Write-Progress throw null object Exception.

Check out the help.chm in release package for details.

Have Fun

Tags: msh monad PowerShell

Download Gene Sequences Using NCBI eFetch Tools

2006-05-30T23:56:00.000-04:00

Recently, I was working on a bioinformatics research project which needed to download hundreds of gene mRNA sequences. I have all the gene IDs in one text file. So a simple PowerShell Script could solve my problem.

I have a old post talking about NCBI Entrez eUtils tools. Today, I will use the eFetch tool (also included in eUtils). The script is simple and self-explaining.

# ===========================================================================
#
# Author:      Tony (http://MSHForFun.blogspot.com)
# File:        Efetch.ps1
# Description: Download gene sequences using NCBI eUtils.eFetch tool
# Reference: http://eutils.ncbi.nlm.nih.gov/entrez/query/static/eutils_example.pl
# Reference: http://eutils.ncbi.nlm.nih.gov/entrez/query/static/efetch_help.html
# Reference: http://eutils.ncbi.nlm.nih.gov/entrez/query/static/efetchseq_help.html
#
# ===========================================================================
param
(
[string] $Path=$(throw "Please Specify a file")
)
$BaseURL = "http://www.ncbi.nlm.nih.gov/entrez/eutils/efetch.fcgi?db=nucleotide&id="
$Option= "&rettype=fasta&retmode=text"
$WebClient = new-object System.Net.WebClient
$SavePath = $Path + ".result"
if (test-path $savePath)
{
del $SavePath
}
foreach ( $id in (get-content $path))
{
# Construct eFetch URL
$URL=$BaseURL + $id + $Option
Write-Progress -Activity "Download Sequences" -Status "Submit gene $Id"
# Submit and download data
$Data = $WebClient.DownloadString($URL)
# Parse Data
if ($Data.Length -gt 1)
{
    Write-Progress -Activity "Download Sequences" -Status "$id OK"
    # Write to Console
    $data
    # Wrtie To file
    $data >> $SavePath
}
else
{
    Write-Progress -Activity "Download Sequences" -Status "$Id is not found!"
    "$Id is not found!`n`r"
    "$Id is not found!`n`r" >> $SavePath
}
# Try not to overload NCBI Server
start-sleep 1
}
# Clear Progress pane
Write-Progress -Activity "Download Sequences" -Status "Done" -completed

You need a text file (genes.txt) to test this script:

0
NM_008176
NM_009140
NM_009141
NM_011333
NM_013654
NM_016960
NM_009142
NM_008491
NM_031168
NM_009883
NM_007679
NM_010030
NM_009971
NM_010809
NM_008607
NM_030612
NM_011198
NM_007987

If you are a biologist, you can see what kind of genes I am intersted in. The first "0" is just to cause an "Not Found" Error. You can run this script like following:

.\efetch.ps1 genes.txt

Your results is printed to screen as well as "genes.txt.result" file.

Have Fun

Tags: msh monad PowerShell

PowerShell Remoting, Lock Down

2006-05-18T21:24:00.000-04:00

Any remote shell application is dangerous because you open a door to outside world. So does PowerShell Remoting. Previous version of PowerShell Remoting totally depended on NegotiateStream. You have no control of login process or you can not limit user access. So anyone with a valid local or domain account can login remotely into your computer. As more and more people began to download and install PowerShell Remoting on their computer, I decided to add client access control policy to PowerShell Remoting.

Firstly, you should limit maximum number of clients which can connect to server simultaneously. This is done by modify registry:

[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\PowerShellRemoting\Parameters]
"MaxClient"=dword:0000000a

Default value is 10. Given 256MB memory is standard configuration for desktop PC, this number is more than enough if you decided to login to your desktop from home network. You can definitely increase this number if you have more clients but remember each runspace will allocate quite a few of memory. So do some experiment and calculation then you can find a reasonable number.

Secondly, you should limit maxium number of clients which can connect to server simultaneously from Same IP address. This is done by modify registry:

[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\PowerShellRemoting\Parameters]
"MaxClientPerIP"=dword:00000002

Default value is 2. You can also change that but you don't want all your available slots of connection were occupied by clients from same computer.

Thirdly, you may want to limit which user can login from which IP address. This is done by modify
%Program Files%\PowerShell Remoting\user.xml
Default installation only allow user in Administrators group login from localhost. So you have to have your own user.xml before you can login from another IP. If user.xml file parse error occorred when start service, default policy will be loaded. If you can't login, check log file or EventLog for details of parse error.
User.xml file contain information about your user access control list. It has SEVEN tags:

<NetworkACL>: container of one or more access control entry (<NetworkACE>)
<NetworkACE>: access control entry container of <Account>, <Access> and <IPRange>
<Account>: Valid Group or User
<Access>: Allow / Deny
<IPRange>: container of <IP> and <Subnet>
<IP>: IPAddress. 0.0.0.0 for any IP, 127.0.0.1 for localhost, 192.168.0.2 etc. IPv6 string should be fine, but I have not been able to test it.
<Subnet>: IPv4 subnet. 192.168.0.0/255.255.255.0 or CIDR like format 192.168.0.0/24.

Within <NetworkACE> and </NetworkACE> tags, order matters.

<Account>
<Access>
<IPRange>

Within <IPRange> and </IPRange> tags, <IP> and <Subnet> can be used in any order.
As you can probably already figured it out : After user provide credential and got a token (WindowsIdentity), PowerShell Remoting will try to match User/Group and IP range in records of access control list. If no record was found, access is denied. If a records match User/Group and IP range were found, PowerShell Remoting will assess access in following order:

User Deny > User Allow > Group Deny > Group Allow

For example: if client were a privileged user (Administrators group) from 192.168.0.2, but there were only one record which wanted to deny access of Users group from subnet 192.168.0.0/255.255.255.0. Then access from this client will be denied because any user belongs to Administrators group also belongs to Users group and 192.168.0.2 belongs to subnet 192.168.0.0/255.255.255.0.

After you logged in and get a PowerShell prompt, you can check $UserACL for serialized access control list.

Right now, subnet parsing and match could be buggy. So tell me if you found it did not act as expected.

These changes could make it more difficult to start using PowerShell Remoting. But I believed that you will like them later on. Remember, if you make any changes to previous setting, you will have to restart service to make them take effect.

Stop-Service PowerShellRemoting
Start-Service PowerShellRemoting

There is also some exciting new feature:

PowerShell Remoting Client becomes a PSSnapin now. So installation become usier. Just run install.ps1 in client folder and invoke Start-RemoteHost, you are on you way to your remote shell.
Ctrl+C Ctrl+Break support.

dir c:\windows\system32

$RemotingClient.CanHandleCancelKey = $false

You will find out that information written in log file has been dramatically reduced because I am going to swith to EventLog in later verion.

I can actually run PowerShell Remoting Client with in PowerShell Analyzer. But its ReadLine popup window really bothers me.

Tags: msh monad PowerShell

PowerShell Remoting version 0.2.5.1

2006-05-18T17:05:00.000-04:00

Version 0.2.5.1

Uninstall old version of PowerShell Remoting before install newer version.

Support Ctrl+C and Ctrl+Break to cancel current pipeline (UDP datagram, So not 100% reliable)

Support Maximum Client per IP option (default 2)

Customized client filter policy (%Program Files%\PowerShell Remoting\user.xml, see sample files for format information)

Important

only allow user in Administrators group login from localhost

$UserACL variable for current client filter policy

Client is installed as PSSnapin

New install/uninstall script for both server and client, new start-remotehost.ps1 script

CanThrowException and CanHandleCancelKey property of client (For developer)

Change Log file path to %Documents and Settings%\NetworkService\Local Settings\Application Data\PowerShellRemoting.log

Change Service Name to "PowerShellRemoting"

Service related Exception is also Logged to Eventlog (EventLog: Application, source: PowershellRemoting)

Other Bugs fixed

Have Fun

Tags: msh monad PowerShell

Potential Security Problem of PSSnapin Installation and Execution

2006-05-13T22:47:00.000-04:00

Windows PowerShell is powerful (hehe, sounds like a bad salesman). To make it even better, user can extent it by PSSnapin. PSSnapin can contain PSProviders, Cmdlets and other class library. But there is a catch. PSSnapin is arbitrary code which could be bad designed, bugs loaded and somewhat out of your control. If you are not careful when install and execute, PSSnapin can cause serious security problems. Let's look following example:

Supposed you have a PSSnapin:

using System;
using System.ComponentModel;
using System.Management.Automation;
namespace TestSnapin
{
    [RunInstaller(true)]
    public class MySnapin : PSSnapIn
    {
         public override string Name
         {
            get { return "Test"; }
         }
        public override string Vendor
        {
             get { return "http://MSHForFun.blogspot.com/"; }
        }
        public override string Description
        {
            get { return "Test"; }
        }
    }

   public class Class2
   {
        public string Who
        {
              get { return "Good Guy!"; }
        }
   }
}

You built it by yourself or simply downloaded it from internet. You (An Administrator) copied it to a folder and ran installutil.exe to install it. Then you tried to use it in PowerShell:

>get-pssnapin -reg

Name : Test
PSVersion : 1.0
Description : Test

> add-pssnapin test
> $a = new-object TestSnapin.Class2
> $a.Who
Good Guy!

Looks perfect, right? But

The folder where you save your PSSnapin is writeable by any user, so anyone can change your PSSnapin.
Your PSSnapin assembly is not signed, so PowerShell will load it without checking its integrity. See the registry key about your PSSnapin

Windows Registry Editor Version 5.00

[HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\PowerShell\1\PowerShellSnapIns\Test]
"PowerShellVersion"="1.0"
"Vendor"="http://MSHForFun.blogspot.com/"
"Description"="Test"
"Version"="1.0.0.0"
"ApplicationBase"="D:\\ps1"
"AssemblyName"="Snapin1, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null"
"ModuleName"="D:\\ps1\\snapin1.dll"

Suppose there were a malicious user wrote a PSSnapin like yours PSSnapin except Class2:

public class Class2
{
    public string Who
    {
        get { return "Bad Guy!"; }
    }
}

He just overwrote your PSSnapin with his DLL. What happened next?

> add-pssnapin test
> $a = new-object TestSnapin.Class2
> $a.Who
Bad Guy!

That is to say user can run any code he wanted. In a worst condition, when you (An Administrator) tried to use this PSSnapin, you were actually tricked into running malicious code as Administrator!

So Lessons:

Keep an eye on your PSSnapin. Never put them in a directory which is writeable by unprivileged user

Edit: 2006-05-16 08:36

Always use a Signed PSSnapin. If a signed PSSnapin were altered, PowerShell will not load it.

> add-pssnapin test
Add-PSSnapin : Cannot load PSSnapIn test. Encountered following error: PSSnapin
module D:\ps1\snapin1.dll doesn't have required PSSnapin strong name Snapin1,
Version=1.0.0.0, Culture=neutral, PublicKeyToken=3daf9f8a713aaa33.
At line:1 char:13
+ add-pssnapin <<<< test

Tags: msh monad PowerShell

New Shell, New Script Language, Same Old Problem

2006-05-11T16:19:00.000-04:00

I was reading post on "The Old New Thing" titled "A new scripting language doesn't solve everything" yesterday. I was just going to write something about this. Lee was one step ahead of me and did a response post titled "Nothing solves everything - PowerShell and other technologies".

Monad (well, I mean PowerShell) is cool. We have new shell and new script language. But it is still going to be a very long time before we can do everything using cmdlet because not only we need time to write new cmdlets, but also we are limited by what .NET Framwork can do. Most important, monad is NOT supposed to "solve everything". Instead monad is supposed to work cooperatively with "OLD" techniques. So we have to deal with backwards compatibility issue. For example, other scripting language, unmanaged code and legacy program (console application). I will talk about support for console application in monad hosting application today.

For a local interactive user, powershell.exe provides nice support for "old" console application inherited from cmd.exe. Common user don't even feel much difference between cmd.exe and powershell.exe if they just invoke

ping localhost

For a programmer, there is a difference lying underneath the surface. Monad using PSHostUserinterface and PSHostRawUserinterface to get user input and display results if user invokes cmdlets. To support legacy console application, powershell.exe actually create a new process (outside of pipeline) and redirects stdin/stdout/stderr of legacy console application to its own console.

PowerShell.exe works fine for two reasons:
1. PowerShell.exe itself is a console application.
2. PowerShell.exe does NOT care whether the legacy console application runs in a different process or not. (cmdlet runs in-process)

It becomes a nightmare if you are going to write a hosting application. Considering what will happen when user invokes console application inside your hosting application like "netsh.exe" which requires user input/output and you did not redirect input and output.
1. Legacy console application start a new process which is out of your control
2. If your hosting application is not a console application (like PowerShell Analyzer), legacy console application start a new Console window which is out of your control. "You can loose output or hang waiting for input that never comes." -- William Stacey [MVP]
3. For a remoting host application (like my PowerShell Remoting), it is even worse because the new console window is on another computer, there is no way for remote user move their cursor to the console window and provide input.

It is NOT a trivial work to solve this problem. My "PowerShell Remoting" does not solve this issue (A NotSupportException will be thrown if user invoked legacy console application). Karl Prosser's PowerShell Analyzer does not solve this problem. If you follow the disscussion in newsgroup: Redirect msh.exe in/out you will find more people are struggling with this issue.

So here comes the question: Should this issue been taken care of by internal host or external hosting application?

I think the answer is both YES and NO.
1. NO: The Design of Monad internal host is to focus on process scripting language, work with objects and piplines. External host is supposed to take care of application logic and user interface.
2. YES: legacy console application is a PROBLEM for ALL monad hosting application. So monad should provide more support for it.

For me, a perfect solution would be that all legacy console application use PSHostUserinterface and PSHostRawUserinterface for ALL output/input (There are protential big problem behind this). Maybe improvement for PSHostUserinterface and PSHostRawUserinterface's definition is needed. NotifyBeginApplication() and NotifyEndApplication() is not good enough.

Have Fun

Tags: msh monad PowerShell

PowerShell Remoting version 0.1.1.7

2006-05-09T17:29:00.000-04:00

What's New

Version 0.1.1.7
1. Recompiled for new Windows PowerShell RC1 (Refresh version)
2. Using "Thread Pooling" method to schedule multiple Host threads. So Sever can potentially
accept more connections. (Old version uses one thread per connection method).
3. Using a separate thread actively reclaim resources from broken connection and dead host.
4. Clean exit when stop service: Disconnect all clients and dispose all running hosts.
5. Gracefully disconnect client when server reach maximum client capacity.
6. Fix: Nested Prompt stack error when multiple clients connected.
7. Fix: Server unable to exit when connection closed unexpectedly by client.
8. Fix: "SetShouldExit" method re-throw "SocketException" Error

Tags: msh monad PowerShell

PowerShell Remoting version 0.1.0.317

2006-05-05T17:06:00.000-04:00

Version 0.1.0.317
1. NestedPrompt (suspend host)
2. Multiple line input mode
3. Use local UI to get login credential
4. Save RawUI state on start, and Reset RawUI state on exit.
5. $CurrentUser Variable (WindowsIdentity Object represent current
login user)
6. Load user profile in following order
1) "\Documents and settings\All users\PsConfiguration\profile.ps1"
2) "\Documents and settings\All users\PsConfiguration\PowerShellRemoting_profile.ps1"
3) "My Documents\PsConfiguration\profile.ps1"
4) "My Documents\PsConfiguration\PowerShellRemoting_profile.ps1"

Tags: msh monad PowerShell

Perfect Prompt for Windows PowerShell

2006-05-01T01:00:00.000-04:00

What Can Tweaking Your Prompt Do For You?

Windows PowerShell have a default prompt in one color (usually gray) that tells you your current working directory. This is OK, but you can do much more with the prompt.
1. All sorts of information can be displayed (machine name, host name, user name, time & date ...)
2. The prompt can use colors
3. You can also manipulate the windows title to dispaly more information.
4. Other RawUI operation is allowed (move cursor)

Why Bother?

Beyond looking cool, it's often useful to keep track of system information.
1. Get your current working directory (how many files in current directory)
2. Current system time, how many process is running
2. If you use my su.msh, you would like to know you current windows identity
3. If you use my powershell remoting, different color helps distingish between local host and remote host.
4. Colorizing your prompt is the ability to quickly spot the prompt when you use scroll console.

First step: Profile and prompt function

Windows PowerShell use Profile to customize user environment. For more detail, about profile
The Story Behind the Naming and Location of PowerShell Profiles:
http://www.leeholmes.com/blog/TheStoryBehindTheNamingAndLocationOfPowerShellProfiles.aspx

1. You can have different prompt for different shell (Maybe you use makeshell.exe generated you own shell)
2. If you have different prompt function defined in multiple profile, the last one excuted take effect
Remember the excution order is

"All users" profile is loaded from "<Installation Directory>\profile.ps1"
"All users," host-specific profile is loaded from "<Installation Directory>\Microsoft.PowerShell_profile.ps1"
Current user profile is loaded from "<My Documents>\WindowsPowerShell\profile.ps1"
Current User, host-specific profile is loaded from "<My Documents>\WindowsPowerShell\Microsoft.PowerShell_profile.ps1"

Edit 09-29-2006

Windows PowerShell RC2 changed profile location to

<My documents>\WindowsPowerShell
Now you can customize you prompt in you profile (say <my document>\psconfigurtion\profile.ps1)
this is typical prompt function looks like:

function prompt
{
"PS " + $(get-location) + "> "
}

You can do whatever you want in prompt function. But remember
1) Always return a [string], otherwise Windows PowerShell will use default "PS> " prompt.
2) Try to limit your prompt in one (short) line
3) Host will evaluate prompt frequently, so don't do crazy stuff to slow down your work.

Colorized prompt

function prompt
{
Write-Host ("PS " + $(get-location) +">") -nonewline -foregroundcolor Magenta
return " "
}

Random color

function prompt
{
    $random = new-object random
    $color=[System.ConsoleColor]$random.next(1,16)
    Write-Host ("PS " + $(get-location) +">") -nonewline -foregroundcolor $color
    return " "
}

Cursor Movement
Display current time at the end of prompt line (this will mess up you console buffer)

function prompt
{
    $oldposition = $host.ui.rawui.CursorPosition
    $Endline = $oldposition
    $Endline.X+=60
    $host.ui.rawui.CursorPosition = $Endline
    Write-Host $(get-date).Tostring("yyyy-MM-dd HH:mm:ss")
    $host.ui.rawui.CursorPosition = $oldposition
    Write-Host ("PS " + $(get-location) +">") -nonewline -foregroundcolor Magenta
    return " "
}

Use Window Title
Show current user, host, current line number

$global:CurrentUser = [System.Security.Principal.WindowsIdentity]::GetCurrent()
function prompt
{
    $host.ui.rawui.WindowTitle = $CurrentUser.Name + " " + $Host.Name + " " + $Host.Version + " Line: " + $host.UI.RawUI.CursorPosition.Y
    Write-Host ("PS " + $(get-location) +">") -nonewline -foregroundcolor Magenta
    return " "
}

Make some noise
if your command take very long time to run, beep when it is done.

function prompt
{
Write-Host ("PS " + $(get-location) +">") -nonewline -foregroundcolor Magenta
return "`a "
}

More information
Count number of files(or items) in current path and number of process running

function prompt
{
    $host.ui.rawui.WindowTitle = "Files: " + (get-childitem).count + " Process: " + (get-process).count
    Write-Host ("PS " + $(get-location) +">") -nonewline -foregroundcolor Magenta
    return " "
}

Something for readers:
1. laptop battery is low ! (Edit 2006-07-21: Already done by Musings of a PC)
2. You got new mail !
3. LAN cable disconnected !

Have Fun

Reference:
http://www.tldp.org/HOWTO/Bash-Prompt-HOWTO/

Tags: msh monad powershell

PowerShell, Change ... is Good ... but Painful

2006-04-26T11:36:00.000-04:00

"Monad" has evolved into "Windows PowerShell”. Don’t tell me your feeling about this name. That’s one of the problems working on a pre-release product. Just to make you fell better, it means we are really close to final release.

There is a bunch of improvement for console user:

1. Tab completion gets better now.

Did anyone know where is the API mentioned in release note: We added support for parameters and variables tab completion. This was done by having the host call a PowerShell function TabExpansion that takes two parameters – the line being entered and the last token on that line.

2. Some cmdlet Parameters alias changed

For example: *-Process Changed -ProcessName to -Name alias -ProcessName, add alias ProcessId to –Id

This makes me feel much better when working with *-process cmdlet.

Here comes the pain:

1. Change your old script file extention: *.msh - > *.ps1

2. Check into individual script (especially you profile.msh) see if you use any old cmdlet names: for example, combine-path -> join-path

3. Change you %my documents%\msh\profile.msh to %my documents%\PSconfiguration\profile.ps1

If you previously wrote some cmdlets or hosting application, you are going to deal with some mess here:

1. Be careful about name changes:

CmdletAttribute
VerbNounCommand

MshHostRawUserInterface
PSHostRawUserInterface

MshCredential
PSCredential

MshCredentialTypes
PSCredentialTypes

MshCredentialUIOptions
PSCredentialUIOptions

MshInvalidCastException
PSInvalidCastException

Some hidden changes:

FieldDescription.AssemblyFullName
FieldDescription.ParameterAssemblyFullName

FieldDescription.TypeFullName
FieldDescription.ParameterTypeFullName

FieldDescription.TypeName
FieldDescription.ParameterTypeName

BufferCell.Type
BufferCell.BufferCellType

If you are using my “Monad Remoting”

Uninstall “Monad Remoting” before upgrade to “PowerShell”
You’ve been warned!

To use “PowerShell Remoting”, you have to change you client script.
To build your own client, be careful about name change.

Tags: msh monad

Build Your Own Client for Monad Remoting

2006-04-24T20:52:00.000-04:00

New version of Monad Remoting with a client class library suitable for building you own client. Available at MSH For Fun Workspace at gotdotnet.

Uninstall your old Server before upgrade!

My DELL laptop was down again. This time, it was hard drive. I lost two weeks' work. (Ouch! If anyone want to donate a hard drive to me for backup, please contact me. Hehe. ) But I am not giving up. After one month's hard work, I was able to present you the new version of Monad Remoting.

We have a better server: more compatible with ConsoleHost and many bug fixed. But most dramatic changes were made at client side. Now you can build your own cleint application using client class library :

1. To build your own client, you need to initialize an instance of ClientMshHost (Add reference MonadRemoting.ClientMshHost.dll, namespace MonadRemoting), which served as proxy between Monad Remoting server user interface (remote UI) and Local userinterface (Local UI). Although with similar public properties, this "Host" is not derived from MshHost thus can not be used to open a runspace.

2. ClientMshHost has only two public method Start() (to connect server and run) and SetShouldExit(int) (to force client to quit) . Let ClientMshHost do all the hard work: connect to server, authenticate & encrypt, open runspace and run script. Whenever a user interface API was invoked at Server, parameters were sent to ClientMshHost. ClientMshHost will invoke same user interface API at Local UI: display information or get user input, return data back to server. ClientMshHost tries not to throw any exceptions, instead it use Local UI to display all server error and local error.

3. The constructor of ClientMshHost requires a IPEndPoint (to connect Remote UI) and a MshHostUserinterface (to connect LocalUI).

public ClientMshHost(IPEndPoint host, MshHostUserInterface UI){}

MshHostRawUserinterface within MshHostUserinterface is NOT mandatory but is supported by ClientMshHost. Minimum requirement is to implement a MshHostUserinterface which overridden

public abstract string ReadLine();
public abstract SecureString ReadLineAsSecureString();
public abstract void Write(string value);
public abstract void Write(ConsoleColor foregroundColor, ConsoleColor backgroundColor, string value);
public abstract void WriteDebugLine(string message);
public abstract void WriteErrorLine(string value);
public virtual void WriteLine();
public abstract void WriteLine(string value);
public virtual void WriteLine(ConsoleColor foregroundColor, ConsoleColor backgroundColor, string value);
public abstract void WriteVerboseLine(string message);
public abstract void WriteWarningLine(string message);

Note: To simplify server and client implementation, following API in local UI were ignored (instead normal Read/Write API will be invoked):

public abstract Dictionary<string, MshObject> Prompt(string caption, string message, Collection<FieldDescription> descriptions);
public abstract int PromptForChoice(string caption, string message, Collection<ChoiceDescription> choices, int defaultChoice);
public abstract MshCredential PromptForCredential(string caption, string message, string userName, string targetName);
public abstract MshCredential PromptForCredential(string caption, string message, string userName, string targetName, MshCredentialTypes allowedCredentialTypes, MshCredentialUIOptions options);

There are great advantages of this design:
1. There is no need to implement my own console client. Don't invent wheels! Msh.exe already exposed a great MshHostUserinterface (ConsoleHostUserInterface) as $host.UI variable. So just run a simple script, we are able to "steal" ConsoleHostUserInterface for remoting (Or I should say,unleash the power of $host.UI). I already add following script into my profile.msh.

function start-remotehost {
Param ([string] $IPAddress = "127.0.0.1", [int] $Port = 8080)

# Make sure you change path to where you saved MonadRemoting.ClientMshHost.dll
[void][System.Reflection.Assembly]::LoadFile("D:\msh\MonadRemoting.ClientMshHost.dll")

$RemoteIP = [System.Net.IPAddress]::Parse($IPAddress)
$RemoteEP = new-object System.Net.IPEndPoint ($RemoteIP, $Port)
$RemotingClient = new-object
MonadRemoting.ClientMshHost($RemoteEP,$host.ui)
$RemotingClient.Start()
$RemotingClient = $null
}

2. We enjoy all the benefits of ConsoleHostUserInterface and whenever Monad was upgraded, our Monad Remoting local UI was automatically upgraded. Actually, because it use 100% naive msh.exe UI, it is difficult to find out you are in local shell or remote shell. You can always invoke

$host

If you saw ConsoleHost, you were in local shell

Name             : ConsoleHost
Version          : 1.0.7487.0
InstanceId       : ae801745-7671-4a46-9501-6f120bae8a81
UI               : System.Management.Automation.Internal.Host.InternalHostUserI
                   nterface
...

If you saw ServerMshHost, you were in remote shell.

Name             : ServerMshHost
Version          : 0.1.0.283
InstanceId       : 84574e63-2b39-4438-b8cc-bb13bd7af932
UI               : System.Management.Automation.Internal.Host.InternalHostUserI
                   nterface
...

3. You will never leave you msh.exe console window and start another program for remote access. You can call start-remoteHost to gain remote access within msh.exe. When you are done, type

exit

and return to your local shell. You want connect to Remote host again? Just call start-remoteHost again. Actually, I am now using this client as a in-process su command. I was thinking of building this client as a MshSnapin someday.

4. The ClientMshHost dose not care what kind of outter application it is in. It works for console application like msh.exe, also works for Winform base GUI application like Karl Prosser's Msh Analyzer. Hopefully, soon I can use Msh analyzer as my Monad remoting client (Its Rich UI is really cool)

Have Fun

[Edit: Monad has now been renamed to Windows PowerShell. This script or discussion may require slight adjustments before it applies directly to newer builds.]

Tags: msh monad PowerShell

MshObject, the Magic Therein

2006-04-09T01:03:00.000-04:00

From Arul Kumaravel's WebLog: "Monad is the only scripting platform that provides access to .Net, WMI, COM in a well-integrated manner."

How? It is MshObject which does all the tricks. MshObject serve as an adapter to .Net, WMI and COM objects. For more details, read MSH Object Concepts.

Here are members of System.Management.Automation.MshObject:

public class MshObject : IFormattable
{
      // Methods
      static MshObject();
      public MshObject();
      public MshObject(object obj);
      public static MshObject AsMshObject(object obj);
      public virtual MshObject Copy();
      public override string ToString();
      public string ToString(string format, IFormatProvider formatProvider);

      // Properties
      public object BaseObject { get; }
      public object ImmediateBaseObject { get; }
      public MshMemberInfoCollection Members { get; }
      public MshMemberInfoCollection Methods { get; }
      public MshMemberInfoCollection Properties { get; }
      public Collection<string> TypeNames { get; }
}

MshObject is hidden under the surface of Msh. You might not notice its exsitance unless you write a cmdlet which deal with MshObject directly. I prevously working on Serialization of MshObject in my Monad Remoting project. I wrote a small cmdlet to track Mshobject within MSH.exe

using System;
using System.Collections.Generic;
using System.Text;
using System.Management.Automation;

namespace MshCx
{
    [Cmdlet("get", "mshobject")]
    public class GetMshobject: Cmdlet
    {

        private MshObject[] inputObjects = null;
        [Parameter(Mandatory = false,ValueFromPipeline = true)]
        public MshObject[] InputObjects
        {
            get { return inputObjects; }
            set { inputObjects = value; }
        }

        protected override void ProcessRecord()
        {
            if (inputObjects != null)
            {
                StringBuilder sb = new StringBuilder();
                foreach (MshObject Item in inputObjects)
                {
                    sb.AppendLine();
                    sb.AppendLine("MshObject: ");
                    sb.AppendLine("\t" + Item.ToString());
                    sb.AppendLine("BaseObject: ");
                    sb.AppendLine("\t"+ Item.BaseObject.ToString() + " (" + Item.BaseObject.GetType().FullName + ")");
                    sb.AppendLine("ImmediateBaseObject: ");
                    sb.AppendLine("\t" + Item.ImmediateBaseObject.ToString() + " (" + Item.ImmediateBaseObject.GetType().FullName + ")");
                    sb.AppendLine("TypeNames: ");
                    foreach (string Type in Item.TypeNames)
                    {
                        sb.AppendLine("\t"+ Type);
                    }
                    sb.AppendLine("=============================================================");
                }
                WriteObject(sb.ToString());
            }
        }
    }
}

It is really intersting to see how export-clixml and import-clixml works:

> $file= (get-childitem)[0]
> $file

    Directory: Microsoft.Management.Automation.Core\FileSystem::D:\msh

Mode                LastWriteTime     Length Name
----                -------------     ------ ----
-a---          2006-4-3     22:27        350 1.txt

> $file | get-member

   TypeName: System.IO.FileInfo

Name                      MemberType     Definition
----                      ----------     ----------
AppendText                Method         System.IO.StreamWriter AppendText()
CopyTo                    Method         System.IO.FileInfo CopyTo(String de...
Create                    Method         System.IO.FileStream Create()
CreateObjRef              Method         System.Runtime.Remoting.ObjRef Crea...
CreateText                Method         System.IO.StreamWriter CreateText()
Decrypt                   Method         System.Void Decrypt()
Delete                    Method         System.Void Delete()
Encrypt                   Method         System.Void Encrypt()
Equals                    Method         System.Boolean Equals(Object obj)
get_Attributes            Method         System.IO.FileAttributes get_Attrib...
get_CreationTime          Method         System.DateTime get_CreationTime()
get_CreationTimeUtc       Method         System.DateTime get_CreationTimeUtc()
get_Directory             Method         System.IO.DirectoryInfo get_Directo...
get_DirectoryName         Method         System.String get_DirectoryName()
get_Exists                Method         System.Boolean get_Exists()
get_Extension             Method         System.String get_Extension()
get_FullName              Method         System.String get_FullName()
get_IsReadOnly            Method         System.Boolean get_IsReadOnly()
get_LastAccessTime        Method         System.DateTime get_LastAccessTime()
get_LastAccessTimeUtc     Method         System.DateTime get_LastAccessTimeU...
get_LastWriteTime         Method         System.DateTime get_LastWriteTime()
get_LastWriteTimeUtc      Method         System.DateTime get_LastWriteTimeUtc()
get_Length                Method         System.Int64 get_Length()
get_Name                  Method         System.String get_Name()
GetAccessControl          Method         System.Security.AccessControl.FileS...
GetHashCode               Method         System.Int32 GetHashCode()
GetLifetimeService        Method         System.Object GetLifetimeService()
GetObjectData             Method         System.Void GetObjectData(Serializa...
GetType                   Method         System.Type GetType()
InitializeLifetimeService Method         System.Object InitializeLifetimeSer...
MoveTo                    Method         System.Void MoveTo(String destFileN...
Open                      Method         System.IO.FileStream Open(FileMode ...
OpenRead                  Method         System.IO.FileStream OpenRead()
OpenText                  Method         System.IO.StreamReader OpenText()
OpenWrite                 Method         System.IO.FileStream OpenWrite()
Refresh                   Method         System.Void Refresh()
Replace                   Method         System.IO.FileInfo Replace(String d...
set_Attributes            Method         System.Void set_Attributes(FileAttr...
set_CreationTime          Method         System.Void set_CreationTime(DateTi...
set_CreationTimeUtc       Method         System.Void set_CreationTimeUtc(Dat...
set_IsReadOnly            Method         System.Void set_IsReadOnly(Boolean ...
set_LastAccessTime        Method         System.Void set_LastAccessTime(Date...
set_LastAccessTimeUtc     Method         System.Void set_LastAccessTimeUtc(D...
set_LastWriteTime         Method         System.Void set_LastWriteTime(DateT...
set_LastWriteTimeUtc      Method         System.Void set_LastWriteTimeUtc(Da...
SetAccessControl          Method         System.Void SetAccessControl(FileSe...
ToString                  Method         System.String ToString()
MshChildName              NoteProperty   System.String MshChildName=1.txt
MshDrive                  NoteProperty   System.Management.Automation.DriveI...
MshIsContainer            NoteProperty   System.Boolean MshIsContainer=False
MshParentPath             NoteProperty   System.String MshParentPath=Microso...
MshPath                   NoteProperty   System.String MshPath=Microsoft.Man...
MshProvider               NoteProperty   System.Management.Automation.Provid...
Attributes                Property       System.IO.FileAttributes Attributes...
CreationTime              Property       System.DateTime CreationTime {get;s...
CreationTimeUtc           Property       System.DateTime CreationTimeUtc {ge...
Directory                 Property       System.IO.DirectoryInfo Directory {...
DirectoryName             Property       System.String DirectoryName {get;}
Exists                    Property       System.Boolean Exists {get;}
Extension                 Property       System.String Extension {get;}
FullName                  Property       System.String FullName {get;}
IsReadOnly                Property       System.Boolean IsReadOnly {get;set;}
LastAccessTime            Property       System.DateTime LastAccessTime {get...
LastAccessTimeUtc         Property       System.DateTime LastAccessTimeUtc {...
LastWriteTime             Property       System.DateTime LastWriteTime {get;...
LastWriteTimeUtc          Property       System.DateTime LastWriteTimeUtc {g...
Length                    Property       System.Int64 Length {get;}
Name                      Property       System.String Name {get;}
Mode                      ScriptProperty System.Object Mode {get=$catr = "";
> $file | get-mshobject

MshObject:
        1.txt
BaseObject:
        1.txt (System.IO.FileInfo)
ImmediateBaseObject:
        1.txt (System.IO.FileInfo)
TypeNames:
        System.IO.FileInfo
        System.IO.FileSystemInfo
        System.MarshalByRefObject
        System.Object
=============================================================
> $file |export-clixml abc.xml

> $file = import-clixml abc.xml
> $file

    Directory: Microsoft.Management.Automation.Core\FileSystem::D:\msh

Mode                LastWriteTime     Length Name
----                -------------     ------ ----
               2006-4-3     22:27        350 1.txt

> $file | get-member

   TypeName: Deserialized.System.IO.FileInfo

Name              MemberType   Definition
----              ----------   ----------
MshChildName      NoteProperty System.String MshChildName=1.txt
MshDrive          NoteProperty System.Management.Automation.MshObject MshDri...
MshIsContainer    NoteProperty System.Boolean MshIsContainer=False
MshParentPath     NoteProperty System.String MshParentPath=Microsoft.Managem...
MshPath           NoteProperty System.String MshPath=Microsoft.Management.Au...
MshProvider       NoteProperty System.Management.Automation.MshObject MshPro...
Attributes        Property     System.String {get;set;}
CreationTime      Property     System.DateTime {get;set;}
CreationTimeUtc   Property     System.DateTime {get;set;}
Directory         Property     System.String {get;set;}
DirectoryName     Property     System.String {get;set;}
Exists            Property     System.Boolean {get;set;}
Extension         Property     System.String {get;set;}
FullName          Property     System.String {get;set;}
IsReadOnly        Property     System.Boolean {get;set;}
LastAccessTime    Property     System.DateTime {get;set;}
LastAccessTimeUtc Property     System.DateTime {get;set;}
LastWriteTime     Property     System.DateTime {get;set;}
LastWriteTimeUtc Property     System.DateTime {get;set;}
Length            Property     System.Int64 {get;set;}
Name              Property     System.String {get;set;}

> $file | get-mshobject

MshObject:
        @{MshPath=Microsoft.Management.Automation.Core\FileSystem::D:\msh\1.txt;
MshPa
rentPath=Microsoft.Management.Automation.Core\FileSystem::D:\msh; MshChildName=
1.txt; MshDrive=; MshProvider=; MshIsContainer=False; Name=1.txt; Length=350; D
irectoryName=D:\msh; Directory=D:\msh; IsReadOnly=False; Exists=True; FullName=
D:\msh\1.txt; Extension=.txt; CreationTime=2006-4-3 22:27:16; CreationTimeUtc=2
006-4-4 2:27:16; LastAccessTime=2006-4-3 22:27:16; LastAccessTimeUtc=2006-4-4 2
:27:16; LastWriteTime=2006-4-3 22:27:16; LastWriteTimeUtc=2006-4-4 2:27:16; Att
ributes=Archive}
BaseObject:
         (System.Management.Automation.MshCustomObject)
ImmediateBaseObject:
         (System.Management.Automation.MshCustomObject)
TypeNames:
        Deserialized.System.IO.FileInfo
        Deserialized.System.IO.FileSystemInfo
        Deserialized.System.MarshalByRefObject
        Deserialized.System.Object
=============================================================

If you look carefully, you will notice:
1. export-clixml write MshObject.Properties into a xml file and import-clixml reconstruct a MshObject.
2. Although they looks the same, something changed after serializtion and deserializtion:
Before: A FileInfo object is wrapped within MshObject
After: A MshCustomObject is wrapped within MshObject. All Methods are lost.
3. Deserialized MshCustomObject is a Hash table.

Have Fun.
[Edit: Monad has now been renamed to Windows PowerShell. This script or discussion may require slight adjustments before it applies directly to newer builds.]

Tags: msh monad PowerShell

PowerShell Remoting - Beta and future

2006-03-31T14:14:00.000-05:00

Release of "PowerShell Remoting", 0.2.7.1 Beta version.
Version 0.2.7.1

New GUI client manager: Save/delete server information & connection option, generate connection script automatically, launch powershell remoting client directly.
Add option to reject connection if user found invalid remote X509 certificate.
Bug fix: Null login credential (press Cancel in CredUI window) cause server crash
Bug fix: other minor bugs

Version 0.2.6.0

Write Server Information & Error to EventLog (Source: PowerShellRemoting). Log file will no longer be used by newer version.

SSL support using SSLStream.

Warning: SSL is for advanced user.

Get X509 certificate for server. For testing purpose, you can use makecert.exe (included in .NET framework SDK)
Install X509 certificate at StoreLocation: LocalMachine, StoreName: My (personal).
Grand "NetworkService" account read access to X509 certificate associated private key file.
Set [HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\PowerShellRemoting\Parameters\ X509] to thumbprint of X509 Certificate.

There is pretty good walkthrough at John Howard's blog.

X509 Option

DebugHost Option (Default 0: false)

Bug fix: Write-Progress throw null object Exception.

Check out the help.chm in release package for details.

Version 0.2.5.1
1. Support Ctrl+C and Ctrl+Break to cancel current pipeline (UDP datagram, So not 100% reliable)
2. Support Maximum Client per IP option (default 2)
3. Customized client filter policy (%Program Files%\PowerShell Remoting\user.xml, see sample files for format information)
Important: default installation only allow user in Administrators group login from localhost. So you have to have your own user.xml before you can login from another IP. If user.xml file parse error occorred when start service, default policy will be loaded. Check log file or EventLog for details of parse error.
4. $UserACL variable for current client filter policy
5. Client is installed as PSSnapin
6. New install/uninstall script for both server and client, new start-remotehost.ps1 script
7. CanThrowException and CanHandleCancelKey property of client (For developer)
8. Change Log file path to %Documents and Settings%\NetworkService\Local Settings\Application Data\PowerShellRemoting.log
9. Change Service Name to "PowerShellRemoting"
10. Service related Exception is also Logged to Eventlog (EventLog: Application, source: PowershellRemoting)
11. Other Bugs fixed

Version 0.1.1.7
1. Recompiled for new Windows PowerShell RC1 (Refresh version)
2. Using "Thread Pooling" method to schedule multiple Host threads. So Sever can potentially accept more connections. (Old version uses one thread per connection method).
3. Using a separate thread actively reclaim resources from broken connection and dead host.
4. Clean exit when stop service: Disconnect all clients and dispose all running hosts.
5. Gracefully disconnect client when server reach maximum client capacity.
6. Fix: Nested Prompt stack error when multiple clients connected.
7. Fix: Server unable to exit when connection closed unexpectedly by client.
8. Fix: "SetShouldExit" method re-throw "SocketException" Error

Version 0.1.0.318
1. Fix "All User" profile Path
1) "\Documents and settings\All users\Documents\PsConfiguration\profile.ps1"
2) "\Documents and settings\All users\Documents\PsConfiguration\PowerShellRemoting_profile.ps1"

Version 0.1.0.317
1. NestedPrompt (suspend host)
2. Multiple line input mode
3. Use local UI to get login credential
4. Save RawUI state on start, and Reset RawUI state on exit.
5. $CurrentUser Variable (WindowsIdentity Object represent current
login user)
6. Load user profile in following order
1) "\Documents and settings\All users\PsConfiguration\profile.ps1"
2) "\Documents and settings\All users\PsConfiguration\PowerShellRemoting_profile.ps1"
3) "My Documents\PsConfiguration\profile.ps1"
4) "My Documents\PsConfiguration\PowerShellRemoting_profile.ps1"

Version 0.1.0.311
1. Solve Server cause CPU overload problem
2. Solve Port or MaxClient change not take effect problem
3. Except for PSHostUserInterface.Prompt, all other UI/RawUI
API were connect to local UI/RawUI API
4. improved Client exception report

Version 0.1.0.301
1. Recompile for new Windows PowerShell RC1
2. Client class libary to build your own client
3. Console client use naive PowerShell.exe user interface
4. Support for all RawUI API
5. Bug fix for server

Version 0.1.0.128
1. Multiple connection (Maximum 10 connection)
2. New log file format
3. Hide some common exception on Client
4. Some bug fix

This software is distributed under BSD license. See license.txt for details.

I. What is "PowerShell Remoting"

PowerShell Remoting is a light-weighted server-client application which
provides secure remote access to Windows PowerShelll (codename Monad).
It contains two components:

Server: A Windows Service (Service Name "PowerShell Remoting") running
under NetworkService account. It will listen at certain TCP port, accept
income connection, authenticate client and provide PowerShell hosting
environment.

Client: A Class libary. It severs as a proxy between sever user interface
and client user interface. It will connect to server, provide client
credential for authentication, get user input and display results.

This software is distributed under BSD license. See license.txt for
details.

II. System requirement.
1. Microsoft .NET framework v2.0.50727 (Server & Client)
2. PowerShell RC1 or Higher (Server only)
3. Windows XP/2003 (For server); Windows 2000/XP/2003 (For client)
4. 256MB Memory, 1M hard drive space.

III. Features
1. Client authentication & protection
1) All communication between server and client is encrypted and signed.
Depend on server environment, authentication method could be NTLM or
Kerberos.

2) Multipul connection from client. (default Port 8080, Maximum 10 connection)
Can be changed by modify registry.

2. Support PSHostUserInterface & PSHostRawUserInterface
1) PSHostUserInterface will be passed to constructor of client
2) PSHostRawUserInterface is not mandatory but is supported

Note : Not supported at this version
1) Tab completion
2) start-transcript

Note. Known problems
1) Outputs of legacy program are not redirected to client

IV Installation and usage
# Server
1. Please use provided install.ps1 (install service) and uninstall.ps1
(uninstall service) to install/uninstall server component.

2. Default installation, server will listen to all available IP
(IPAddress.Any) listen at port 8080 and maximum client number is 10.
Those can be changed by modify registry
[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\PowerShell Remoting
\Parameters]
"Port"=dword:00001f90
"MaxClient"=dword:0000000a

3. Log file was "Hard Coded" to
C:\Documents and Settings\NetworkService\PowerShellRemoting.log

4. Default installed as Manual start service.

# Client
1. Client application do not need to be installed, just copy it to client
PC and execute "start-remotehost.ps1". (Some modification is needed)

2. argument:
start-remotehost
Where is IP address which server is listening
(For example: 127.0.0.1- for localhost; 192.168.0.2, et, al.)
Where is port number which server is listening
(default 8080)

If you have any other suggestions, please drop a few lines in comment.

Have Fun

Tags: msh monad PowerShell

Monad Remoting - Windows Service

2006-03-27T01:05:00.000-05:00

Just upload a new version of "Monad Remoting". Now it is a windows service with a bunch of new features and recommanded for upgrading.

New feature:
1. Running as windows service under NetworkService account (Service Name "Monad Remoting").

2. Receiving customized NetworkCredential for logon. (Yes, I mean really logon) But only accept one connection at a time.

3. Impersonation as remote NetworkCredential. You can either login as an unprivileged user or an Administrator. Try my faverate id.msh to find out your current identity.

4. Log file was "Hard Coded" to
C:\Documents and Settings\NetworkService\MonadRemoting.log

5. Automatic install service (install.msh) and uninstall service (uninstall.msh). You 'd better install service using my install.msh because it creates required registry key.

6. Default installed as Manual start service. You can change it to Automatic in MMC service snapin. You have to start service before first connection.

Start-service "Monad Remoting"

7. Default listen at port 8080, can be changed by modify registry
[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\Monad Remoting\Port]

8: Default listen to all available IP (IPAddress.Any)

Have Fun

Tags: msh monad

Monad Remoting - Now Can Read SecureString From Remote Client

2006-03-22T22:18:00.000-05:00

Implementation of ReadLineAsSecureString() in MshHostUserInterface.

Just upload a new version of "Monad Remoting". It can read SecureString from remote client console now.

SecureString will not be kept secret if you store its content in a normal string first. That is to say, we have to read it from console and store them directly as SecureString. But before sent them to remote server, we have to decrypt it to byte[ ]. NegotiateStream will encrypt (decrypted) byte[ ] before write to NetworkStream. At server side, we have to get the byte[ ] and restore it to a SecureString.

Although I tried to minimize exposure and clear my footprint behind me, there are still potential security problems. So use it at your own risk. I post the code here, in case someone want to take a close look at those steps I mentioned here.

                do
                {
                    keyinfo = Console.ReadKey(true);
                    if ((keyinfo.Modifiers & ConsoleModifiers.Alt) != 0 || (keyinfo.Modifiers & ConsoleModifiers.Control) != 0) continue;
                    if (keyinfo.Key == ConsoleKey.Enter)
                    {
                        Console.WriteLine();
                        break;
                    }
                    if (password.Length == 512)
                    {
                        Console.Write("\r\nRead 512 (Maxium) Characters!");
                        break;
                    }
                    if (keyinfo.Key == ConsoleKey.Backspace)
                    {
                        password.RemoveAt(password.Length - 1);
                        Console.Write('\b');
                        Console.Write(' ');
                        Console.Write('\b');
                        continue;
                    }
                    password.AppendChar(keyinfo.KeyChar);
                    Console.Write('*');
                }
                while (keyinfo.Key != ConsoleKey.Enter);
                if (password.Length > 0)
                {
                    Plantext = GetByteArrayFromSecurString(password);
                    authStream.Write(Plantext, 0, Plantext.Length);
                    Array.Clear(Plantext, 0, Plantext.Length);
                }

At this point, you can now enjoy the get-credential, new-securestring cmdlets.

I really wish we could have a in-process su command, because the trick of

[System.Diagnostics.Process]::Start()

will not work for remote client. Well, we have to expect that at next version of monad.

Have Fun.

Tags: msh monad

Script to Resize Remote Client Console Window

2006-03-22T14:05:00.000-05:00

There is an intersting msh script from Web ChangeLog blog to demonstrate RawUI functionality of my Monad Remoting tools.

MSH: Remote scripting host

Interstingly, if you run this script in msh.exe, you will probobly get some exception. But it works fine for my Monad Remoting tools.

Exception setting "WindowSize": "Window cannot be wider than the screen buffer.
Parameter name: value.Width Actual value was 85."
At D:\msh\script\resizewin.msh:7 char:18

Actually, This is a bug of my tools. I simply applied all RawUI operation directly to client console and but did not check screen buffer.

Have Fun!

Tags: msh monad

Monad Remoting (updated version)

2006-03-20T22:12:00.000-05:00

Just upload a new version of Monad Remoting for testing. Download at gotdotnet.

Feature:

# Automatic authentication
1) Client credential is automaticly provided to server for authentication.
2) All communication between server and client is entrypted and signed.

# Basic MshHostUserInterface.
1) Load User Profile at server when connected (new)
2) Evaluation for Prompt function (new)
3) Read and write from console
4) Colorized output
5) Prompt for undefined parameters (new)
6) Prompt for choice (new)

# Not supported at this version
1) Read Securestring
2) ReadKey

# known problems
1) Outputs of legacy program are not redirected to client

Have Fun.

Tags: msh monad

Securely Extend Msh Host User Interface over Network

2006-03-17T19:16:00.000-05:00

Added at 20th Mar, 22:20
There is a followup post here.

Securely extend Msh Host User Interface over network using NegotiateStream.

If you follow my previous post, you will find out that I changed my strategy. There is no need to sent Mshobjects to remote client. We just need to implement a customized MshHostUserInterface which get input and write result to remote client. Monad is designed to be extended in this way, so it only took me a couple of days to get a usable solution. You can download testing binary at gotdotnet.

Server

Client

The underline idea is to implement all APIs in MshHostUserInterface and MshHostRawUserInterface. It was much easier than I originally thought. But I still had some problems with

public override Dictionary Prompt(string caption, string message, Collection descriptions)   { }

public override int PromptForChoice(string caption, string message, Collection choices, int defaultChoice) { }

public override MshCredential PromptForCredential(string caption, string message, string userName, string targetName) { }

public override MshCredential PromptForCredential(string caption, string message, string userName, string targetName, MshCredentialTypes allowedCredentialTypes, MshCredentialUIOptions options)   { }

public override SecureString ReadLineAsSecureString()   { }

in MshHostUserInterface and

public override KeyInfo ReadKey(ReadKeyOptions options) { }

public override void ScrollBufferContents(Rectangle source, Coordinates destination, Rectangle clip, BufferCell fill) { }

public override void SetBufferContents(Coordinates origin, BufferCell[,] contents) { }

public override void SetBufferContents(Rectangle rectangle, BufferCell fill) { }

in MshHostRawUserInterface.

That is to say: all sercurestring-related, credential-related, scroll console-related cmdlet are not working. But it is really fun to run some test on remote machine. Try out yourself by downloading the testing binary at gotdotnet.

Monad Rocks!

Have Fun!

Tags: msh monad

Separation of Monad Engine and User Interface

2006-03-14T15:24:00.000-05:00

Research on MshHost & MshHostUserInterface abstract class.

The msh.exe is actully a console program hosting monad engine. In another word: monad engine does not worry about user interface, it is host environment's job. My Remote MSH server is not INTERACTIVE (For example: remote user will not be prompted to input credentials when invoking "get-credential" at server) because I use DefaultHost which does not Implement MshHostUserInterface.

OK, OK, it is to fast, let's start from Create a Runspace:
System.Management.Automation.Runspaces.RunspaceFactory.CreateRunspace()

public static Runspace CreateRunspace()
{
      Runspace runspace1;
      using (IDisposable disposable1 = RunspaceFactory._tracer.TraceMethod())
      {
            runspace1 = RunspaceFactory.CreateRunspace(new DefaultHost(Thread.CurrentThread.CurrentCulture, Thread.CurrentThread.CurrentUICulture));
      }
      return runspace1;
}

So it acually create a Runspace using DefaultHost
Microsoft.Management.Automation.Internal.DefaultHost is a implement of MshHost abstract class.

internal class DefaultHost : MshHost
{
      // Methods
      static DefaultHost();
      internal DefaultHost(CultureInfo currentCulture, CultureInfo currentUICulture);
      public override void EnterNestedPrompt();
      public override void ExitNestedPrompt();
      public override void NotifyBeginApplication();
      public override void NotifyEndApplication();
      public override void SetShouldExit(int exitCode);

      // Properties
      public override CultureInfo CurrentCulture { get; }
      public override CultureInfo CurrentUICulture { get; }
      public override Guid InstanceId { get; }
      public override string Name { get; }
      public override MshHostUserInterface UI { get; }
      public override Version Version { get; }

      // Fields
      private CultureInfo currentCulture;
      private CultureInfo currentUICulture;
      private Guid id;
      [TraceSource("DefaultHost", "DefaultHost subclass of S.M.A.MshHost Tracer")]
      private static MshTraceSource tracer;
      private Version ver;
}

Let's take a look at System.Management.Automation.Host.MshHost

public abstract class MshHost
{
      // Methods
      protected MshHost();
      public abstract void EnterNestedPrompt();
      public abstract void ExitNestedPrompt();
      public abstract void NotifyBeginApplication();
      public abstract void NotifyEndApplication();
      public abstract void SetShouldExit(int exitCode);

      // Properties
      public abstract CultureInfo CurrentCulture { get; }
      public abstract CultureInfo CurrentUICulture { get; }
      public abstract Guid InstanceId { get; }
      public abstract string Name { get; }
      public virtual MshObject PrivateData { get; }
      public abstract MshHostUserInterface UI { get; }
      public abstract Version Version { get; }
}

MshHostUserInterface is the object which actully deals with user interface:

public abstract class MshHostUserInterface
{
      // Methods
      protected MshHostUserInterface();
      public abstract Dictionary<string, MshObject> Prompt(string caption, string message, Collection<FieldDescription> descriptions);
      public abstract int PromptForChoice(string caption, string message, Collection<ChoiceDescription> choices, int defaultChoice);
      public abstract MshCredential PromptForCredential(string caption, string message, string userName, string targetName);
      public abstract MshCredential PromptForCredential(string caption, string message, string userName, string targetName, MshCredentialTypes allowedCredentialTypes, MshCredentialUIOptions options);
      public abstract string ReadLine();
      public abstract SecureString ReadLineAsSecureString();
      public abstract void Write(string value);
      public abstract void Write(ConsoleColor foregroundColor, ConsoleColor backgroundColor, string value);
      public abstract void WriteDebugLine(string message);
      public abstract void WriteErrorLine(string value);
      public virtual void WriteLine();
      public abstract void WriteLine(string value);
      public virtual void WriteLine(ConsoleColor foregroundColor, ConsoleColor backgroundColor, string value);
      public abstract void WriteProgress(long sourceId, ProgressRecord record);
      public abstract void WriteVerboseLine(string message);
      public abstract void WriteWarningLine(string message);

      // Properties
      public abstract MshHostRawUserInterface RawUI { get; }
}

But when we check the public property UI in DefaultHost:

public override MshHostUserInterface UI
{
      get
      {
            MshHostUserInterface interface1;
            using (IDisposable disposable1 = DefaultHost.tracer.TraceProperty())
            {
                  interface1 = null;
            }
            return interface1;
      }
}

So, it is a null object.

If we want a REAL interactive remote shell, we have to implement our own MshHost and MshHostUserInterface (and maybe MshHostRawUserInterface). MshHostUserInterface will be responsible to write results to and get input from remote client. I bet this is the way Microsoft monad team will be using to provide remote access (maybe in version 2 of msh).

For me, it is too much work (and I would not compete with monad team). I really wish someone can show a example of implementation of MshHost and MshHostUserInterface. Maybe monad team would like to share their source code of msh.exe

Hehe, You know I am joking, right?

Added on Mar 15th 10:50am
I just find out that Microsoft Command Shell (MSH) Programmer's Guide are online now! It has exact information I needed.

Have Fun!

Tags: msh monad

Securely Extend Monad Pipline over Network (2)

2006-03-13T00:42:00.000-05:00

I have uploaded a testing binary package. You can download it from gotdotnet. If you have any suggestions or bug reports, please drop a few lines as comment.

Usage:

1. Start MonadServiceHost.exe at Host Windows box with Monad installed (beta 3.1 for my testing):

MonadServiceHost.exe 8888

Where "8888" is the port number you want server to listen. (Remember to allow accecc to local port 8888 in you firewall setting).

2. Start MSH.exe at client Windows box

MSH.exe

3. Load MonadServiceClient.dll from within client MSH.exe

[System.Reflection.Assembly]::LoadFile("D:\MonadServiceClient.dll")

where "D:\MonadServiceClient.dll" full path to MonadServiceClient.dll

4. Initialize a client object

$client = new-object MonadServiceClient.ClientLib (8888,"192.168.0.2")

where "192.168.0.2" is the IP Address of Server, "8888" is port to connect .

5. Check out Method you can use

$client | get-member

6. Connect client to remote server

$client.Connect()

7. Using client to communicate with server

# Ping Server
$client.Ping()

# Invoke remote command
$client.Invoke("get-process")
$client.Invoke("set-location HKLM:")

# run remote script
$client.Invoke("D:\msh\id.msh")

# Redirect output to a file if return a lot of objects.

7. Disconnect client

$client.Close()

8. If something went wrong and you lost connection , just reconnect client

$client.Connect()

Have Fun!

Tags: msh monad

Securely Extend Monad Pipline over Network (1)

2006-03-09T16:31:00.000-05:00

Prototype (or a toy) of a light-weighted server, which host monad and interact with remote client via secured TCP channel.

There are a lot of questions been asked in newsgroup about using Monad securely from remote computer. Ideally, we should have something like sshd in linux. But unlike traditional text based shell, Monad deal with .Net object using pipeline. Sshd did not have buildin functionality to work with .Net object. So it make sense to host monad in a ASP.NET application and send object over soap protocal.

But what if you don't want a CPU-intensive program or don't want to rely on IIS web server? Maybe you just want to access your desktop via a laptop on the same home network. I still remember in the "old time" I can use NetCat.exe binding a shell and gain remote access instantly. "Simple is beautiful", isn't it? That's why I write this tiny "toy" (File size: Server 24kb, client 20kb).

Design:
1. Server is a console program which hosts Monad Runspace, authenticates client using NegotiateStream and impersonates client using remote credential.
2. Client is class libary which sents out msh command to server.
3. Server gets command input, invoke monad pipline and get output objects.
4. Server serializes object and send them to client over secure tcp channel.
5. Client de-serializes object and emit them to client side interactive MSH for reuse.

Some thoughts:
1. Simple text-base control commands were used to control server and client. For example: Server sent a "SendObject" command to client; Client sent back an "OK" command and prepared to receive object; Server then sent serialized objects, Client de-serialized objects and sent back an "OK" command to finish this sequence.
2. Serialization and de-serialization of MshObject is the most difficult part. I still did not make it fully work. Serialization of every member in MshObject is not needed because some fields are session-dependent. We just need most important informations about the object so that we can operated those objects at server side.
3. Instead of creating a new instance of monad runspace for each invoke, remote session state should be maintained by server for each client. So client can use variable generated by previous command.

At this moment
1. Authentication and network are working fine.
2. Server can only accept one client at a time.(I will swith to asynchronous method later)
2. Client maintain connection status and can be controled by msh script.
2. It is barely usable as a interactive remote shell. for example, start remote service, kill remote process, run remote script (this is really cool).
3. Can NOT sent real object between server and client. Only important object information were transfered to client.

I will post binary and source code when it became more mature. See a screenshot of server console and client output below:

Host:

Client output:

>[System.Reflection.Assembly]::LoadFile("D:\msh\MonadServiceClient.dll")

GAC    Version        Location
---    -------        --------
False v2.0.50727     D:\msh\MonadServiceClient.dll

> $client = new-object MonadServiceClient.ClientLib (8080,"127.0.0.1")
> $client.Connect()

IsConnected             : True
IsAuthenticated         : True
IsMutuallyAuthenticated : False
IsEncrypted             : True
IsSigned                : True
RemoteIdentity          : System.Security.Principal.GenericIdentity
Welcome                 : MSHForFun.MonadHostService Ready!
ConnectStartAt          : 2006-3-9 15:33:54
ConnectEndAt            : 0001-1-1 0:00:00

> $client.Invoke("set-location d:\msh")
OK: Null object in pipline

> $client.Invoke("get-location")
D:\msh
Drive:System.Management.Automation.DriveInfo Drive {get;}
Provider:System.Management.Automation.ProviderInfo Provider {get;}
ProviderPath:System.String ProviderPath {get;}
Path:System.String Path {get;}

> $client.Invoke("(get-process explorer).Id")
1644

> $client.ping()
Success

> $client.close()

IsConnected             : False
IsAuthenticated         : False
IsMutuallyAuthenticated : False
IsEncrypted             : False
IsSigned                : False
RemoteIdentity          :
Welcome                 :
ConnectStartAt          : 2006-3-9 15:38:46
ConnectEndAt            : 2006-3-9 15:41:43

Have Fun!

Tags: msh monad

Switch to BSD License

2006-03-09T11:04:00.000-05:00

License is a sensitivie issue. Any comments on certain licence will cause a "war" in newsgroup. I am not a lawyer and I do not want to get into trouble either. I wrote all those codes just for fun and I do want to get more feed back for my project. So I take Lee Holmes' advice.

My MSHForFun.Security mshsnapin are now under BSD license.

Have Fun!

Tags: msh monad

Release of MSHForFun.Security MshSnapin v0.1

2006-03-01T23:10:00.000-05:00

Finally, I finished my MSHForFun.Security MshSnapin v0.1. Source code and binary files can be downloaded from GotDotNet under BSD license (Sorry, you need an Internet explorer to visit gotdotnet.com. Firefox seemed to not willing to take .Net passport). Contain four cmdlets at this moment:

Name          : add-processowner
Definition    : add-processowner [-ProcessArray] Process[] [-Verbose] [-Debug]
                [-ErrorAction ActionPreference] [-ErrorVariable String] [-OutVa
                riable String] [-OutBuffer Int32] [-WhatIf] [-Confirm]
Wrapper of OpenProcessToken Win32 API. It takes Process (output of get-process cmdlet) as input and adds process owner information (System.Security.Pricipal.WindowsIdentity) as NoteProperty.

Name          : Get-Privilege
Definition    : Get-Privilege [[-Identity] WindowsIdentity] [-Verbose] [-Debug]
                 [-ErrorAction ActionPreference] [-ErrorVariable String] [-OutV
                ariable String] [-OutBuffer Int32] [-WhatIf] [-Confirm]
Wrapper of GetTokenInformation Win32 API. It takes System.Security.Pricipal.WindowsIdentity object as input and returns MSHForFun.Security.TokenPrivilegeCollection object. If used without parameter, it returns current process (msh.exe) token privileges.

Name          : get-windowsidentity
Definition    : get-windowsidentity [[-Credential] MshCredential] [-Verbose] [-
                Debug] [-ErrorAction ActionPreference] [-ErrorVariable String]
                [-OutVariable String] [-OutBuffer Int32] [-WhatIf] [-Confirm]
Wapper of LogonUser() Win32API. It takes MshCredential object as input and returns System.Security.Pricipal.WindowsIdentity object. If used without parameter, it returns System.Security.Pricipal.WindowsIdentity object represented current user.

Name          : start-process
Definition    : start-process [-FileName] String [-Credential MshCredential] [-
                Arguments String] [-LoadUserProfile] [-UseShellExecute] [-Worki
                ngDirectory String] [-CreateNoWindow] [-ErrorDialog] [-ErrorDia
                logParentHandle IntPtr] [-RedirectStandardError] [-RedirectStan
                dardInput] [-RedirectStandardOutput] [-StandardErrorEncoding En
                coding] [-StandardOutputEncoding Encoding] [-Verb String] [-Win
                dowStyle ProcessWindowStyle] [-Verbose] [-Debug] [-ErrorAction
                ActionPreference] [-ErrorVariable String] [-OutVariable String]
                 [-OutBuffer Int32] [-WhatIf] [-Confirm]
Wrapper of System.Diagnostics.Process.Start() Methods. It takes MshCredential object as input and start a process using that credential. (Runas in MSH).

What's new?

Wrap token handle into SafeTokenHandle, which is derived from Microsoft.Win32. SafeHandles. SafeHandleZeroOrMinusOneIsInvalid (try to prevent handle leakage and provides protection for handle recycling security attacks)
Get-Privilege cmdlet
Make sure all Exceptions were properly handled and using WriteWarning instead of WriteError, if we known errors will happen frequently (Add-Processowner)
Rewrite C# code, correct errors and clean comments.

Something intersting about Get-Privilege cmdlet:

System.Security.Pricipal.WindowsIdentity contains all important information inside a token except token privileges, so I took the challenge to write a Get-Privilege cmdlet.
Mashalling memory block (struct TOKEN_PRIVILEGE) using C# is killing me. I will not do it again.
MSHForFun.Security.TokenPrivilege contian 2 public property: Name and Status
MSHForFun.Security.TokenPrivilegeCollection is derived from System.Collection.CollectionBase.
Get-Privilege cmdlet is used for dump token privilege only.

Some examples of using these cmdlets:

>get-privilege (get-windowsidentity (get-credential))

Cmdlet get-credential at command pipeline position 1
Supply values for the following parameters:
Credential
User: Administrator
Password for user Administrator: *******

Name                                    Status
----                                    ------
SeChangeNotifyPrivilege                 Enabled | Enabled By Default
SeSecurityPrivilege                     Enabled | Enabled By Default
SeBackupPrivilege                       Enabled | Enabled By Default
SeRestorePrivilege                      Enabled | Enabled By Default
SeSystemtimePrivilege                   Enabled | Enabled By Default
SeShutdownPrivilege                     Enabled | Enabled By Default
SeRemoteShutdownPrivilege               Enabled | Enabled By Default
SeTakeOwnershipPrivilege                Enabled | Enabled By Default
SeDebugPrivilege                        Enabled | Enabled By Default
SeSystemEnvironmentPrivilege            Enabled | Enabled By Default
SeSystemProfilePrivilege                Enabled | Enabled By Default
SeProfileSingleProcessPrivilege         Enabled | Enabled By Default
SeIncreaseBasePriorityPrivilege         Enabled | Enabled By Default
SeLoadDriverPrivilege                   Enabled | Enabled By Default
SeCreatePagefilePrivilege               Enabled | Enabled By Default
SeIncreaseQuotaPrivilege                Enabled | Enabled By Default
SeUndockPrivilege                       Enabled | Enabled By Default
SeManageVolumePrivilege                 Enabled | Enabled By Default
SeCreateGlobalPrivilege                 Enabled | Enabled By Default
SeImpersonatePrivilege                  Enabled | Enabled By Default

> get-privilege (get-process -Id 3768 | add-processowner).Processowner

Name                                    Status
----                                    ------
SeChangeNotifyPrivilege                 Enabled | Enabled By Default
SeSecurityPrivilege                     Removed
SeBackupPrivilege                       Removed
SeRestorePrivilege                      Removed
SeSystemtimePrivilege                   Removed
SeShutdownPrivilege                     Removed
SeRemoteShutdownPrivilege               Removed
SeTakeOwnershipPrivilege                Removed
SeDebugPrivilege                        Enabled
SeSystemEnvironmentPrivilege            Removed
SeSystemProfilePrivilege                Removed
SeProfileSingleProcessPrivilege         Removed
SeIncreaseBasePriorityPrivilege         Removed
SeLoadDriverPrivilege                   Removed
SeCreatePagefilePrivilege               Removed
SeIncreaseQuotaPrivilege                Removed
SeUndockPrivilege                       Removed
SeManageVolumePrivilege                 Removed
SeCreateGlobalPrivilege                 Enabled | Enabled By Default
SeImpersonatePrivilege                  Enabled | Enabled By Default

>get-process | add-processowner

Handles NPM(K)    PM(K)      WS(K) VS(M)   CPU(s)     Id ProcessName
------- ------    -----      ----- -----   ------     -- -----------
     71       3     1548       5304    38     0.80    864 acrotray
WARNING: Failed to openy process token (Access Denied): alg
    105       5     1224       3604    35            2832 alg
....
(clipped)

>Start-process -Credential (get-credential) -FileName control.exe -Arguments timedate.cpl

Cmdlet get-credential at command pipeline position 1
Supply values for the following parameters:
Credential
User: Administrator
Password for user Administrator: *******

Handles NPM(K)    PM(K)      WS(K) VS(M)   CPU(s)     Id ProcessName
------- ------    -----      ----- -----   ------     -- -----------
      0       0       64         60     1     0.00   2220 control

PS. some Non-MSH stuff:
Check out the "Brrreeeport" game on the Scobleizer blog.
Check out results on
1. Technorati
2. MSN Search
3. Google
4. Yahoo

Have fun!

[Edit: Monad has now been renamed to Windows PowerShell. This script or discussion may require slight adjustments before it applies directly to newer builds.]

Tags: msh monad PowerShell

Groups Command in MSH: Get-WindowsIdentity cmdlet

2006-02-16T15:22:00.000-05:00

(Added on Mar 6th) There is a follow up post on this topic here.

In Linux Bash shell, there is a handy groups command, which takes a username as input and return user associated groups. To get similar functionality, I wrote the Get-WindowsIdentity cmdlet, which take a MshCredential as input, call LogonUser Win32 API and return a System.Security. Principal.WindowsIdentity object. The bad news, you got to have a username and password. Example:

>get-WindowsIdentity (get-credential)

AuthenticationType : MICROSOFT_AUTHENTICATION_PACKAGE_V1_0
ImpersonationLevel : None
IsAuthenticated    : True
IsGuest            : False
IsSystem           : False
IsAnonymous        : False
Name               : Domain\Administrator
Owner              : S-1-5-21-XXXXXXXXX-XXXXXXX-XXXX-XX
User               : S-1-5-21-XXXXXXXXX-XXXXXXX-XXXX-XX
Groups             : {S-1-5-21-XXXXXXXXX-XXXXXXX-XXXX-XX, , , , , , , }
Token              : XXXX

This cmdlet is designed for WindowsXP SP2. In Windows 2000, user required to have SeTcbPrivilege,a.k.a Act as part of the operating system, to call LogonUser. The reference (at the end of the post), provide a workaround to not call LongonUser: Use NegotiateStream. But it did not work for me.

If you are using windows server 2003 and want to check a domain account, don't use this cmdlet. Just do something like this:

>$id = new-object System.Security.Principal.WindowsIdentity("Domain\User")

This cmdlet could be useful if you are trying
1. To find out a specific user (you known his/her password) belongs to which groups

2. To check a user's password without login as that user or using runas/su.msh. If you input wrong password, you will get an error:

get Get-WindowsIdentity : Win32 API (LogonUser) Error: 1326

C# code:

using System;
using System.Security.Principal;
using System.Management.Automation;
using System.Net;
using System.Runtime.InteropServices;

namespace MSHForFun.Security
{
    public class LogonUserHelper
    {
        [DllImport("advapi32.dll", SetLastError = true)]
        static extern bool LogonUser(
          string principal,
          string authority,
          string password,
          LogonTypes logonType,
          LogonProviders logonProvider,
          out IntPtr token);
        [DllImport("kernel32.dll", SetLastError = true)]
        static extern bool CloseHandle(IntPtr handle);
        enum LogonTypes : uint
        {
            Interactive = 2,
            Network,
            Batch,
            Service,
            NetworkCleartext = 8,
            NewCredentials
        }
        enum LogonProviders : uint
        {
            Default = 0, // default for platform (use this!)
            WinNT35,     // sends smoke signals to authority
            WinNT40,     // uses NTLM
            WinNT50      // negotiates Kerb or NTLM
        }
        public static WindowsIdentity LogonUser(MshCredential UserCre)
        {
            IntPtr token = IntPtr.Zero;
            NetworkCredential Cre = UserCre.GetNetworkCredential();
            bool result = LogonUser(
                Cre.UserName, "", Cre.Password,
                LogonTypes.Network,
                LogonProviders.Default,
                out token);
            WindowsIdentity id;
            if (result)
            {
                id = new WindowsIdentity(token);
                CloseHandle(token);
            }
            else
            {
                throw new Exception("Win32 API (LogonUser) Error: " + Marshal.GetLastWin32Error().ToString());
            }
            return id;
        }

    }

    /// <summary>
    /// Get User WindowsIdentity
    /// </summary>
    [Cmdlet("Get", "WindowsIdentity", SupportsShouldProcess = true)]
    public class GetWindowsIdentityCmd : Cmdlet
    {
        #region Parameters
        private MshCredential userCre = null;
        /// <summary>Process Arrary to work with</summary>
        [Parameter(Mandatory = true, Position = 0)]
        public MshCredential UserCredential
        {
            get { return userCre; }
            set { userCre = value; }
        }
        #endregion

        /// <summary>
        /// Call LogonUser Return a WindowsIdentity Object
        /// </summary>
        protected override void ProcessRecord()
        {
            if (ShouldProcess("Get WindowsIdentity of User: " + userCre.UserName))
            {
                WriteObject(LogonUserHelper.LogonUser(userCre));
            }
        }
    }
}

Reference: http://pluralsight.com/wiki/default.aspx/Keith.GuideBook/HowToGetATokenForAUser.html

Have fun!

[Edit: Monad has now been renamed to Windows PowerShell. This script or discussion may require slight adjustments before it applies directly to newer builds.]

Tags: msh monad PowerShell

Add-ProcessOwner cmdlet (update)

2006-02-16T00:43:00.000-05:00

(Added on Mar 6th) There is a follow up post on this topic here.

Rewrite Add-ProcessOwner cmdlet and build into a MshSnapin. Most intersting thing here: Wrap an object into MshObject, add a MshNoteProperty.

MshObject Constructor:

System.Management.Automation.MshObject..ctor(Object)

MshNoteProperty Constructor:

System.Management.Automation.MshNoteProperty..ctor(String, Object)

Example to use this cmdlet:

>$ps=get-process | add-processowner
>$ps | get-member

TypeName: System.Diagnostics.Process

>$ps[0].ProcessOwner | get-member

System.Security.Principal.WindowsIdentity
# Some processOwner could be null due to security policy

//Add-ProcessOwner.cs
using System;
using System.Security.Principal;
using System.Diagnostics;
using System.Management.Automation;
using System.Runtime.InteropServices;

namespace MSHForFun.Security
{
    /// <summary>
    /// Add Process Owner infomation as NoteProperty
    /// </summary>
    [Cmdlet("add", "processowner", SupportsShouldProcess = true)]
    public class AddProcessOwnerCmd : Cmdlet
    {
        #region Parameters
        private Process[] allProcess = null;
        /// <summary>
        /// Arrary of Process to query
        /// </summary>
        [Parameter(Mandatory = true, Position = 0, ValueFromPipeline = true)]
        public Process[] ProcessArray
        {
            get { return allProcess; }
            set { allProcess = value; }
        }
        #endregion

        private SafeTokenHandle processToken = new SafeTokenHandle(IntPtr.Zero);
        private MshObject psex;
        private MshNoteProperty info;
        private WindowsIdentity id;

        /// <summary>
        /// Get WindowsIdentity for each process
        /// </summary>
        protected override void ProcessRecord()
        {
            foreach (Process ps in allProcess)
            {
                if (ShouldProcess(ps.ProcessName))
                {
                    psex = new MshObject(ps);
                    try
                    {
                        if (Win32Helper.OpenProcessToken(ps.Handle, TokenAccessLevels.Query, ref processToken))
                        {
                            if (processToken.IsInvalid)
                            {
                                WriteWarning("Invalid process token: " + ps.ProcessName);
                                info = new MshNoteProperty("ProcessOwner", null);
                                psex.Properties.Add(info);
                            }
                            else
                            {
                                id = processToken.GetWindowsIdenty();
                                WriteVerbose("Successfully get process owner information: " + ps.ProcessName);
                                info = new MshNoteProperty("ProcessOwner", id);
                                psex.Properties.Add(info);
                            }
                        }
                        else
                        {
                            WriteWarning("Unable to query process token (privilege not held): " + ps.ProcessName);
                            info = new MshNoteProperty("ProcessOwner", null);
                            psex.Properties.Add(info);
                        }
                    }
                    catch
                    {
                        WriteWarning("Unable to query process token (privilege not held): " + ps.ProcessName);
                        info = new MshNoteProperty("ProcessOwner", null);
                        psex.Properties.Add(info);
                    }
                    WriteObject(psex);
                }
            }
        }
    }
}

For MshSnapin

//MshSnapin.cs
using System;
using System.Collections.Generic;
using System.Text;
using System.Management.Automation;
using System.Configuration.Install;
using System.ComponentModel;

namespace MSHForFun.Security
{
    /// <summary> Defines the properties of MonadSecurity snapin</summary>
    [RunInstaller(true)]
    public class MonadSecurity : MshSnapIn
    {
        /// <summary>Creates an instance of MonadSecurity Snapin class.</summary>
        public MonadSecurity() : base()
        {
        }
        ///<summary>The snapin name which is used for registration</summary>
        public override string Name
        {
            get
            {
                return "MSHForFun.Security";
            }
        }
        /// <summary>Gets vendor of the snapin.</summary>
        public override string Vendor
        {
            get
            {
                return "http://mshforfun.blogspot.com/";
            }
        }
        /// <summary>Gets description of the snapin. </summary>
        public override string Description
        {
            get
            {
                return "Cmdlets for windows security related jobs";
            }
        }
    }
}

[Edit: Monad has now been renamed to Windows PowerShell. This script or discussion may require slight adjustments before it applies directly to newer builds.]

Tags: msh monad PowerShell

Peek into process token: Add-ProcessOwner cmdlet

2006-02-14T01:20:00.000-05:00

Take a look at process tab in windws task manager. You will find all process followed by a user name who start the process. Although Monad’s buildin cmdlet get-process also return information of processes running on current windows box, due to the limitation of System.Diagnostics.Process object, results does not contain process owner information.

In order to get process token, I have to call Win32API – OpenProcessToken, which return a System.IntPtr type token. This token can be used to construct a System.Security. Principal.WindowsIdentity object. (It is first time I use P/Invoke. Please tell me if I did anything wrong.)

Using add-member cmdlet, I add this WindowsIdentity object to System.Diagnostics.Process object as NoteProperty. To use it:

>$ps= Get-process –ProcessName exporler | Add-ProcessOwner.msh
>$ps | get-member
…
WaitForExit       Method        System.Boolean WaitForExit(Int...
WaitForInputIdle      Method       System.Boolean WaitForInputIdl...
__NounName         NoteProperty   System.String __NounName=Process
ProcessOwner      NoteProperty   System.Security.Principal.Wind...
…
>$ps.ProcessOwner

AuthenticationType : NTLM
ImpersonationLevel : None
IsAuthenticated    :
IsGuest            : False
IsSystem           : False
IsAnonymous        : False
Name               : Domain\tony
Owner              : S-1-5-21-854245398-XXXXXXXXXX-XXXXXX-XXX
User               : S-1-5-21-854245398--XXXXXXXXXX-XXXXXX-XXX
Groups             : {S-1-5-21-854245398--XXXXXXXXXX, , , , , }
Token              : XXXX

It is pretty sad that due to local security policy, I can not get owner of processes runing as System or Network service (which are those most intersting process). I might have to mount myself as a system service first than call OpenProcessToken. Well, that is to complicated for me.

using System;
using System.Runtime.InteropServices;
using System.Security.Principal;
using System.Collections;
using System.Diagnostics;

namespace Monad.Security
{
    public class OpenProcessTokenHelper
    {
        [Flags]
        enum TOKEN_ACCESS : uint
        {
            TOKEN_ASSIGN_PRIMARY = 0x0001,
            TOKEN_DUPLICATE = 0x0002,
            TOKEN_IMPERSONATE = 0x0004,
            TOKEN_QUERY = 0x0008,
            TOKEN_QUERY_SOURCE = 0x0010,
            TOKEN_ADJUST_PRIVILEGES = 0x0020,
            TOKEN_ADJUST_GROUPS = 0x0040,
            TOKEN_ADJUST_DEFAULT = 0x0080,
            TOKEN_ADJUST_SESSIONID = 0x0100,
            TOKEN_READ = 0x00020000 | TOKEN_QUERY,
            TOKEN_WRITE = 0x00020000 | TOKEN_ADJUST_PRIVILEGES | TOKEN_ADJUST_GROUPS | TOKEN_ADJUST_DEFAULT,
            TOKEN_EXECUTE = 0x00020000,
        };

        [DllImport("Advapi32.dll", SetLastError = true)]
        extern static int OpenProcessToken(IntPtr processHandle, TOKEN_ACCESS desiredAccess, out IntPtr tokenHandle);

        [DllImport("kernel32.dll", SetLastError = true)]
        extern static bool CloseHandle(IntPtr handle);

        public static WindowsIdentity GetProcessesIdentities(Process process)
        {
            try
            {
                IntPtr token = IntPtr.Zero;
                if (OpenProcessToken(process.Handle, TOKEN_ACCESS.TOKEN_QUERY, out token) == 0)
                {
                    throw new ApplicationException("Can't open process token for: " + process.ProcessName);
                }
                WindowsIdentity id = new WindowsIdentity(token);
                CloseHandle(token);
                return (id);
            }
            catch (Exception ex)
            {
                throw new ApplicationException("Open process token error", ex);
            }

        }
    }
}

#################################################
## Add-ProcessOwner.msh
## Add System.Security.Principal.WindowsIdentity object
## to System.Diagnostics.Process object as NoteProperty
## using P/invoke OpenProcessToken Win32API
##
## from http://mshforfun.blogspot.com/
################################################
begin
{
    [void][Reflection.Assembly]::LoadFile("D:\msh\OpenProcessTokenHelper.dll")
}
process
{
    if ($_)
    {
       # Only working with process object
       if ($_ -isnot [System.Diagnostics.Process]) {continue}
       else
       {
           # Get WindowsIdentityObject
        $id=[Monad.Security.OpenProcessTokenHelper]::GetProcessesIdentities($_)
           add-member -InputObject $_ -Type NoteProperty -Name "ProcessOwner" -Value $id -ErrorAction "SilentlyContinue"
       }
     }
}

Reference: http://www.sellsbrothers.com/askthewonk/secure/default.aspx?content=howcanigetthesecurityprin.htm

Tags: msh monad

Page load Activities increased!

2006-02-13T08:29:00.000-05:00

My laptop was down for two days. When it came back, I was surprised to find out that visits to my blog increased dramatically. All visitors were referred from monad team blog and Lee Holmes' blog. Thanks you, guys!

Although only small changes were made, their scripts delivered whole new user experience or better security compared to my (ugly) script. As you may have noticed, my daytime job is research scientist on immunology. I set up this blog for biology oriented readers and never thought IT professionals would be major readers. So please forgive my amateurish and feel free to comment on any errors or bad ideas in my posts.

PS,I need to mention is that Blastn is program for DNA sequence analysis. So input should match [ATGCN], otherwise you will get an error.

Have Fun.

Tags: msh monad

Tired of Typing Administrator’s Password Repeatedly?

2006-02-08T00:40:00.000-05:00

An example of how to use new- SecureString, export-SecureString and import-SecureString.

If you read my previous blog entry and were convinced to logon as non-privilege user, you must have discovered the beauty and power of runas.exe or our su.msh.

But being the only user of that laptop, isn't it annoying to type Administrator's password again and again just for some routing jobs? Why can't we just incorporate the Administrator's Password in our msh script?
Yes, we can. We are not dumb enough to put plain text password in our script. We have to store our password in an encrypted form. Monad beta3 contain three cmdlets to operate SecureString:

Name       : export-SecureString
Definition : export-SecureString [-SecureString] SecureString [[-SecureKey] Sec
             ureString] [-Verbose] [-Debug] [-ErrorAction ActionPreference] [-E
             rrorVariable String] [-OutVariable String] [-OutBuffer Int32]
             export-SecureString [-SecureString] SecureString [-Key Byte[]] [-V
             erbose] [-Debug] [-ErrorAction ActionPreference] [-ErrorVariable S
             tring] [-OutVariable String] [-OutBuffer Int32]

Name       : import-SecureString
Definition : import-SecureString [-String] String [[-SecureKey] SecureString] [
             -Verbose] [-Debug] [-ErrorAction ActionPreference] [-ErrorVariable
              String] [-OutVariable String] [-OutBuffer Int32]
             import-SecureString [-String] String [-Key Byte[]] [-Verbose] [-De
             bug] [-ErrorAction ActionPreference] [-ErrorVariable String] [-Out
             Variable String] [-OutBuffer Int32]

Name       : new-SecureString
Definition : new-SecureString [-Verbose] [-Debug] [-ErrorAction ActionPreferenc
             e] [-ErrorVariable String] [-OutVariable String] [-OutBuffer Int32]

Using export-SecureString, we can export a securestring (Administrator's Password) to a safe, persistable format (encrypted Administrator's Password). When needed, we can use import-SecureString to decrypt the output of export-securestring and restore it to a securestring.

> export-SecureString (new-SecureString)|out-file Admin.pass
Enter secret: ******

> get-content Admin.pass
01000000d08c9ddf0115d1118c7a00c04fc297eb0100000091700afb9271384c8d9d0f5fc5bbcad
d0000000002000000000003660000a8000000100000003ab5cb17d04880206425514877334d5600
…
(Clipped)

Now we can write a script like following:

##########################################
# Defrag.msh
# Start Disk Defragmenter as Administrator
# From http://mshforfun.blogspot.com/
# This script is provided AS IS
# Use it at your own risk
##########################################
$Password=import-SecureString (get-content D:\msh\Admin.pass)
$Windir=(get-childitem Env:windir).Value
$StartInfo = new-object System.Diagnostics.ProcessStartInfo
$StartInfo.UserName = "Admin"
$StartInfo.Password = $Password
$StartInfo.FileName = $Windir + "\system32\mmc.exe"
$StartInfo.Arguments = $Windir + "\system32\dfrg.msc"
$StartInfo.WorkingDirectory = $Windir + "\system32"
$StartInfo.LoadUserProfile = $true
$StartInfo.UseShellExecute = $false
[System.Diagnostics.Process]::Start($StartInfo)

Tada! No more prompts for password!

Something needed to mention:
1. Although I don't know the default key which export-SecureString used to encrypt password, I am pretty sure this key is account-specific and domain-specific. That is to say, another user on the same computer can not use import-SecureString to decrypt export-SecureString output. Even a user with same account name can't decrypt it if he/she is on another computer (domain). If an unauthorized user tried to import-SecureString, he/she will get an error:

import-SecureString : Key not valid for use in specified state.

But that does not rule out bruce attack. So make sure to apply restricted access rule to you password file. It should be only readable by yourself and Administrators group.

2. Export-SecureString can take a key (a SecureString or byte[]) to encrypt SecureString. Import-SecureString will require the same key to decrypt. (Then we would have to type another password - the encrypt key) This method will output an encrypted password which can be used by another user. But this feature is yet not available in version beta 3. You will get an error:

> $key=new-securestring
Enter secret: *****
> export-securestring (new-securestring) -SecureKey $key
Enter secret: ******
export-SecureString : Cannot process argument because the value of argument "key" is invalid.

Added on 14th, Feb 2006
This is not a bug:
From Lee Holmes--When you use an encryption key on export-secureString, it must be of a length supported by the encryption algorithm. That is 128,192, or 256 bytes -- which is 8, 12, or 16 unicode characters if you type them. I'll file a doc bug to make this more clear.
Added on 14th, Feb 2006
3. Sign your script, and apply the AllSigned execution policy to prevent the script was tampered by malicious user.

4. DO NOT try to grant access to another user using this method. As you can see, if a non-privilege user got the content of Admin.pass (if Admin.pass properly encrypted for this specific user), he can do anything as an Administrator!

There are more in depth discuss about Cryptograpy in MSH on MoW's Blog: here and here

Have Fun!

[Edit: Monad has now been renamed to Windows PowerShell. This script or discussion may require slight adjustments before it applies directly to newer builds.]

Tags: msh monad PowerShell

Give Monad a Voice: out-voice cmdlet

2006-02-04T00:56:00.000-05:00

This script will get objects from pipline and output as voice using Text To Speech (TTS) engine. If set $SaveToWav to $true, it will output to a serial of WAV files.

WindowsXP shipped with both speech recognition and Text To Speech Engine. TTS engine (sapi.dll) takes text as input and output that text as spoken audio. Although WinFX will contain the fully managed API for speech, at this moment, I have to use (ugly) com interop. To make things easier, I generate a managed API libaray (SpeechLib.dll) using tlbimp.exe, a tool in .NET 2.0 SDK.

set-alias tlbimp "C:\Program Files\Microsoft.NET\SDK\v2.0\bin\tlbimp.exe"
tlbimp "C:\Program Files\Common Files\Microsoft Shared\Speech\sapi.dll" `
/out:SpeechLib.dll /namespace:SpeechLib

You will see some warnings talking about type error. Just ignore them because we will not use those problemetic class today. Then we can load the libray, initialize a SpeechLib.SpVoiceClass object and call Speak method.

[Reflection.Assembly]::LoadFile("D:\msh\SpeechLib.dll")
$SpVoice = new-object SpeechLib.SpVoiceClass
$SpVoice.Speak("Hello", [SpeechLib.SpeechVoiceSpeakFlags]::SVSFDefault)

After using this cmdlet for a while, I found out that call Speak method asynchronously is a good idea. While listen to your voice output (which could be lengthy and boring), you can still get a prompt and working with MSH.

$SpVoice.Speak("Hello", [SpeechLib.SpeechVoiceSpeakFlags]::SVSFlagsAsync)

To get a .wav filename, I used System.Windows.Forms.SaveFileDialog object

$SaveFileD = new-object System.Windows.Forms.SaveFileDialog
$SaveFileD.Filter = "All files (*.*)|*.*|wav files (*.wav)|*.wav";
$SaveFileD.Title = "Save to a wave file";
$SaveFileD.FilterIndex = 2;
$SaveFileD.RestoreDirectory = true;
if ($SaveFileD.ShowDialog() -eq [System.Windows.Forms.DialogResult]::OK)
{
$FileName = $SaveFileD.FileName
}

To save to wav file, I use SpeechLib.SpFileStreamClass

$WaveFile = new-object SpeechLib.SpFileStreamClass
$WaveFile.Open($FileNameS, [SpeechLib.SpeechStreamFileMode]::SSFMCreateForWrite, $false)
$SpVoice.AudioOutputStream = $WaveFile
$SpVoice.Speak($StringToSay, [SpeechLib.SpeechVoiceSpeakFlags]::SVSFlagsAsync)
$SpVoice.WaitUntilDone([System.Threading.Timeout]::Infinite)
$WaveFile.Close()

I use $_ to get object from pipline and use out-string cmdlet to convert them to System.String object .

$StringToSay = $_ | out-string

If you want to try it, do something like

dir| select-object Name -first 4 | .\out-voice.msh
get-eventlog Application -Newest 4 | .\out-voice.msh $true

Make sure the speaker on your computer is poweron and enabled. Make sure you use select to format your results first. Otherwise you won't know what your computer are talking about. There is no guarantee that it would be fun. As I said before it could be lengthy and boring.

###############################################################
#
# out-voice.msh
# This script takes objects from pipline and output as voice using TTS engine.
# If set $SaveToWav to $true, output to WAV file(s).
#
# Usage: "Hello", "Welcome to monad world" | out-voice
#        "Hello", "Welcome to monad world" | out-voice $true
#
# from Tony http://mshforfun.blogspot.com/
#
################################################################

param ( [bool] $SaveToWav = $false )
begin
{
    [void][Reflection.Assembly]::LoadFile("D:\msh\SpeechLib.dll")
    $SpVoice = new-object SpeechLib.SpVoiceClass
    $count = 0
    if ($SaveToWav)
    {
        [void] [Reflection.Assembly]::Load("System.Windows.Forms, Version=2.0.0.0, `
    Culture=neutral, PublicKeyToken=b77a5c561934e089")
        $SaveFileD = new-object System.Windows.Forms.SaveFileDialog
        $SaveFileD.Filter = "All files (*.*)|*.*|wav files (*.wav)|*.wav";
        $SaveFileD.Title = "Save to a wave file";
        $SaveFileD.FilterIndex = 2;
        $SaveFileD.RestoreDirectory = true;
        if ($SaveFileD.ShowDialog() -eq [System.Windows.Forms.DialogResult]::OK)
        {
        $FileName = $SaveFileD.FileName
        }
       else
      {
        "Must Specify a wav file"
        break
      }
   }
}
process
{
    if ($_)
    {
       $count++
       $StringToSay = $_ | out-string
       "$count: $StringToSay"
       if ($SaveToWav)
       {
            $FileNameS = $FileName.Replace(".wav", $count.Tostring()+".wav")
            $WaveFile = new-object SpeechLib.SpFileStreamClass
            $WaveFile.Open($FileNameS, `
[SpeechLib.SpeechStreamFileMode]::SSFMCreateForWrite, $false)
            $SpVoice.AudioOutputStream = $WaveFile
            [void]$SpVoice.Speak($StringToSay, `
[SpeechLib.SpeechVoiceSpeakFlags]::SVSFlagsAsync)
            [void]$SpVoice.WaitUntilDone([System.Threading.Timeout]::Infinite)
            $WaveFile.Close()
       }
       else
       {
            [void]$SpVoice.Speak($StringToSay, `
[SpeechLib.SpeechVoiceSpeakFlags]::SVSFlagsAsync)
       }
    }
    else
    {
       "null object"
    }
}
end
{
"Number of objects successfully output: $count"
}
##############################################

Reference: Code4Fun, C# Corner

Have Fun!

[Edit: Monad has now been renamed to Windows PowerShell. This script or discussion may require slight adjustments before it applies directly to newer builds.]

Tags: msh monad PowerShell

Survival as a Non-Administrator: A Monad view

2006-02-02T18:52:00.000-05:00

DON'T ROOT AROUND!

Remember the security rule NO.1 in unix world? It is also 100% rule NO.1 in windows world. Do you get some worms, viruses, spywares or adware in you windows box? You should blame yourself first if you logged in as an Adminstrator EVERYTIME EVERYDAY. It is you who let those malware bypass all windows security check and write themselves directly in system folder and registry. Do you really need Administrator privilege to read Emails or surf the internet? Or do you really need Administrator privilege to compile your code? If you want to write secure code, login as a limited user and find out how you code works. Don't rely on testing afterwards.

Reality check:

Although Windows Vista is supposed to provide better user experence for limited user account, frankly speaking, using windows 2000 / xp / 2003 as limited user account is not easy and intuitive as Mac OS X or Linux . Default user in windows xp is a Administrator!
Being an Administrator for so long, you have indulged yourself by that power. After "eating your own dog food", it isn't really that fun to be a non-privilege user. You found yourself trapped in a "jungle" and can't breathe.

But calm down man! Don't point to that logout button so soon! There are plenty of ways to regain your power without logout. You can control your computer as Administrator and return back to your non-Administrator environment after you are done.

Since this is a blog about MSH, I am going to talk about some solutions in MSH.

1. -Credential Parameter
It is good to see that (although still in beta stage) monad designers kept "running at least privilege" in mind. For example, they plan to implemetation -Credential parameter for file/registry provider. There are some buildin cmdlet using -Credential parameter. For example those Item cmdlet:

Name       : clear-item
Definition : clear-item [-Path] String[] [-Force] [-Filter String] [-Include St
             ring[]] [-Exclude String[]] [-Credential MshCredential] [-Verbose]
              [-Debug] [-ErrorAction ActionPreference] [-ErrorVariable String]
             [-OutVariable String] [-OutBuffer Int32] [-WhatIf] [-Confirm]

Name       : copy-item
Definition : copy-item [-Path] String[] [[-Destination] String] [-Container] [-
             Force] [-Filter String] [-Include String[]] [-Exclude String[]] [-
             Recurse] [-PassThru] [-Credential MshCredential] [-Verbose] [-Debu
             g] [-ErrorAction ActionPreference] [-ErrorVariable String] [-OutVa
             riable String] [-OutBuffer Int32] [-WhatIf] [-Confirm]

Name       : get-item
Definition : get-item [-Path] String[] [-Filter String] [-Include String[]] [-E
             xclude String[]] [-Force] [-Credential MshCredential] [-Verbose] [
             -Debug] [-ErrorAction ActionPreference] [-ErrorVariable String] [-
             OutVariable String] [-OutBuffer Int32]

Name       : invoke-item
Definition : invoke-item [-Path] String[] [-Filter String] [-Include String[]]
             [-Exclude String[]] [-Credential MshCredential] [-Verbose] [-Debug
             ] [-ErrorAction ActionPreference] [-ErrorVariable String] [-OutVar
             iable String] [-OutBuffer Int32] [-WhatIf] [-Confirm]

Name       : move-item
Definition : move-item [-Path] String[] [[-Destination] String] [-Force] [-Filt
             er String] [-Include String[]] [-Exclude String[]] [-PassThru] [-C
             redential MshCredential] [-Verbose] [-Debug] [-ErrorAction ActionP
             reference] [-ErrorVariable String] [-OutVariable String] [-OutBuff
             er Int32] [-WhatIf] [-Confirm]

Name       : new-item
Definition : new-item [-Path] String[] [-Type String] [-Value Object] [-Force]
             [-Credential MshCredential] [-Verbose] [-Debug] [-ErrorAction Acti
             onPreference] [-ErrorVariable String] [-OutVariable String] [-OutB
             uffer Int32] [-WhatIf] [-Confirm]

Name       : remove-item
Definition : remove-item [-Path] String[] [-Filter String] [-Include String[]]
             [-Exclude String[]] [-Recurse] [-Force] [-Credential MshCredential
             ] [-Verbose] [-Debug] [-ErrorAction ActionPreference] [-ErrorVaria
             ble String] [-OutVariable String] [-OutBuffer Int32] [-WhatIf] [-C
             onfirm]

Name       : rename-item
Definition : rename-item [-Path] String [-Name] String [-Force] [-PassThru] [-C
             redential MshCredential] [-Verbose] [-Debug] [-ErrorAction ActionP
             reference] [-ErrorVariable String] [-OutVariable String] [-OutBuff
             er Int32] [-WhatIf] [-Confirm]

Name       : set-item
Definition : set-item [-Path] String[] [[-Value] Object] [-Force] [-PassThru] [
             -Filter String] [-Include String[]] [-Exclude String[]] [-Credenti
             al MshCredential] [-Verbose] [-Debug] [-ErrorAction ActionPreferen
             ce] [-ErrorVariable String] [-OutVariable String] [-OutBuffer Int3
             2] [-WhatIf] [-Confirm]

Unfortunately, if you try to use them now, you will get an error.

invoke-item : Dynamic parameters for the Cmdlet cannot be retrieved. Invocation of MakePath on the 'FileSystem' provider failed for path ''. Cannot call method. The provider does not support the use of credentials.

According to Abhishek Agrawal [MSFT] : "FileSystem and Registry provider do not supprt Credential parameter for V1. We plan to address this in V2. For now you will need to start an elevated MSH window using runas.exe (or Process.Start api) and carry out the required operations there. "

So you have to wait new version of monad which can utilize the -credential parameter for FileSystem and Registry provider.

2. System.Diagnostics.Process.Start() method
.NET framework 2.0 provide three new overloads of System.Diagnostics.Process.Start() method which (might use CreateProcessAsUser Win32 API to) start a process as different user:

public static Process Start ( ProcessStartInfo startInfo)
public static Process Start ( string fileName, string userName, SecureString password, string domain)
public static Process Start (string fileName, string arguments, string userName, SecureString password, string domain)

They take string type username and SecureString type password. (ProcessStartInfo.Password is a SecureString) SecureString is a new creature in .NET 2.0. How we can get an instance of SecureString? Using get-credential cmdlet we can easily and securely package username and password and then pass them to System.Diagnostics.Process.Start() method. See my previous blog entry for details.

Armed with this tool, we can do almost anything in MSH command line. For example, set file system access control rule, kill a process, modify registry or start a service.

Remember you need to type:

exit

to stop this msh.exe process. You don't want to leave a shell with Administrator privilege to someone else, do you?

3. Have to use GUI?
Well, I have to admit that sometimes command line is just not good enough. For example, you want to disable wireless network connection. There is no way to use runas.exe or do shift-click tricks. Even if you started msh.exe as an Administrator, invoking ncpa.cpl will not give you "Network Connections" in control panel. The only choice is to use other command line tools like: devcon.exe (from microsoft) or network.exe (from Wingnut Software) If you know how to do this in MSH script, please tell me, please!

ps: I found this funny story. The author had same problem as mine.

But there is always a work around: With help of monad, we can easily get a GUI shell(explorer.exe) as Administrator.

1) using su.msh to get a msh prompt as Adminstrator
2) run following commands

stop-process -ProcessName explorer
explorer.exe

Welcome to Administrator's Explorer! Go to control panel and disable wireless network connection as you wish. But remember:

a) Explorer.exe will reload itself quickly after been killed, so you better write those command in msh script.

b) After you finish your work, you need to return to your original explorer environment. You can't kill the explorer.exe process within task manager because you are a limited user (remember?). Instead, you can

stop-process -ProcessName explorer

in the msh as Adminstrator
What? you have already closed that window? Run su.msh again!
then you either wait explorer.exe to reload itself or start explorer (Do this as a non-Administrator!) by

explorer.exe

Added on 26th, Feb 2006
You can change registry to start exporler.exe in seperate process (so there is no need to kill your original explorer.exe process):
First, run su.msh
Then modify registry:

set-property HKCU:\Software\Microsoft\Windows\CurrentVersion\Explorer\Advanced -property SeparateProcess -value 1 -force

Without reboot, we can now start new explorer.exe windows as Administrator:

explorer.exe

You can even use System.Diagnostics.Process.Start() method to start a explorer.exe process from your msh runing as limited user.
Added on 26th, Feb 2006

Have Fun!

[Edit: Monad has now been renamed to Windows PowerShell. This script or discussion may require slight adjustments before it applies directly to newer builds.]

Tags: msh monad PowerShell

Dreaming of SU in MSH?

2006-01-27T01:16:00.000-05:00

(Added on Mar 6th) There is a follow up post on this topic here.

Shame on myself! I should do more research before posting a blog entry. MoW already had a similar script a couple of months ago.

Want to run a MSH script as another windows user? You got it now! I am not talking about Runas.exe. It is a MSH script to start new msh.exe process with a differnt windows identity. Just like su.exe in Linux.

###########################################
# File Name: su.msh
# Launch a new msh.exe with someone else's identity
# from tony http://mshforfun.blogspot.com/
###########################################
$SuAccount = get-credential
$StartInfo = new-object System.Diagnostics.ProcessStartInfo
$StartInfo.FileName = "msh.exe"
$StartInfo.UserName = $SuAccount.UserName
$StartInfo.Password = $SuAccount.Password
$StartInfo.LoadUserProfile = $true
$StartInfo.UseShellExecute = $false
$StartInfo.WorkingDirectory = (get-location).Path
[System.Diagnostics.Process]::Start($StartInfo)

Added on 23rd Feb 2006, 08:40
Jeffrey Snover Suggested to add the following line:

$StartInfo.Arguments="-noexit -command `$Host.UI.RawUI.WindowTitle=\`"Microsoft Command Shell ($($SuAccount.UserName)) \`""

Added on 23rd Feb 2006, 08:40

Added on 27th Jan 2006, 15:46
In Monad beta3 version, the default behavior of get-credential was changed to "CredUI". "CredUI returns a username with "\" prepended. When passing that to the Process.Start method, it has intermittent difficulty dealing with that form of a username. " --Lee Holmes
To change it back to CLI, run following script:

new-property HKLM:\SOFTWARE\Microsoft\MSH\1\ShellIds `
-property ConsolePrompting -value "True" -force

See newsgroup thread here for details. Also checkout ::: MSH ::: Blog Entry for this issue.
/Added on 27th Jan 2006, 15:46

So if you were a non-privilege user "testac" , you run id.msh:
UserSID= S-1-5-21-xxxxxxxxx-xxxxxxxxxx-xxxxxxxxxx-xxx (Domain\testac)
AuthenticationType= NTLM
ImpersonationLevel= None
Token= xxxx
Groups=
GroupSID= S-1-5-21-xxxxxxxxx-xxxxxxxxxx-xxxxxxxxxx-xxx (Domain\None)
GroupSID= S-1-1-0 (Everyone)
GroupSID= S-1-5-32-545 (BUILTIN\Users)
GroupSID= S-1-5-4 (NT AUTHORITY\INTERACTIVE)
GroupSID= S-1-5-11 (NT AUTHORITY\Authenticated Users)
GroupSID= S-1-2-0 (LOCAL)

After su to an Administrator user "tony". Then you run id.msh again in new msh.exe window:
UserSID= S-1-5-21-xxxxxxxxx-xxxxxxxxxx-xxxxxxxxxx-xxxx (Domain\tony)
AuthenticationType= MICROSOFT_AUTHENTICATION_PACKAGE_V1_0
ImpersonationLevel= None
Token= xxxx
Groups=
GroupSID= S-1-5-21-xxxxxxxxx-xxxxxxxxxx-xxxxxxxxxx-xxx (Domain\None)
GroupSID= S-1-1-0 (Everyone)
GroupSID= S-1-5-32-544 (BUILTIN\Administrators)
GroupSID= S-1-5-32-545 (BUILTIN\Users)
GroupSID= S-1-5-4 (NT AUTHORITY\INTERACTIVE)
GroupSID= S-1-5-11 (NT AUTHORITY\Authenticated Users)
GroupSID= S-1-2-0 (LOCAL)

Have Fun!

[Edit: Monad has now been renamed to Windows PowerShell. This script or discussion may require slight adjustments before it applies directly to newer builds.]

Tags: msh monad PowerShell

A Clone of id/whoami in MSH

2006-01-24T23:41:00.000-05:00

I would like to have something like id/whoami in linux while working with MSH. So I wrote a simple script.

##############################################
# File Name: id.msh
# Return User and Group information of current logon user
# By Tony http://mshforfun.blogspot.com
##############################################
$me=[System.Security.Principal.WindowsIdentity]::GetCurrent()
"UserSID= " + $me.User + " (" + $me.Name + ")"
"AuthenticationType= " + $me.AuthenticationType
"ImpersonationLevel= " + $me.ImpersonationLevel
"Token= " + $me.Token
"Groups= "
foreach($Group in $me.Groups)
{
"`tGroupSID= " + $Group + " (" + $Group.Translate([System.Security.Principal.NTAccount]) + ")"
}

Output looks like:
UserSID= S-1-5-21-xxxxxxxxx-xxxxxxxxxx-xxxxxxxxxx-xxxx (Domain\me)
AuthenticationType= NTLM
ImpersonationLevel= None
Token= xxxx
Groups=
GroupSID= S-1-5-21-xxxxxxxxx-xxxxxxxxxx-xxxxxxxxxx-xxx (Domain\None)
GroupSID= S-1-1-0 (Everyone)
GroupSID= S-1-5-32-545 (BUILTIN\Users)
GroupSID= S-1-5-4 (NT AUTHORITY\INTERACTIVE)
GroupSID= S-1-5-11 (NT AUTHORITY\Authenticated Users)
GroupSID= S-1-2-0 (LOCAL)

Well, write a clone of who would be fun. Any volunteer?

[Edit: Monad has now been renamed to Windows PowerShell. This script or discussion may require slight adjustments before it applies directly to newer builds.]

Tags: msh monad PowerShell

Combination Rights, Inheritance and Propagation Settings of File System Access Rules

2006-01-16T10:44:00.000-05:00

A more complexed example of combination rights, inheritance and propagation settings of access rules for directory.

Yesterday, I happened to find a code example of manipulating ACLs for .NET framework 1.0: ACLs in .NET: A C# library containing wrapper classes for ACL, ACE, Security descriptors, Security Attributes, Access tokens, etc. It is so complicated to working on platform invokation. As you can see from my previous blog entry Play with ACL in MSH, Play with ACL in MSH (continued) and /\/\o\/\/'s blog entry Adding a Simple AccesRule to a file ACL in MSH, with .NET 2.0 things getting much easier. Today I am going to dig a little deeper and talk about more complex example.

1. We have to deal with FileSystemRights enumeration when creating a new FileSystemAccessRule object. We can see from following that FileSystemRights contain 23 names:

[system.enum]::getnames([System.Security.AccessControl.FileSystemRights])

ListDirectory
ReadData
WriteData
CreateFiles
CreateDirectories
AppendData
ReadExtendedAttributes
WriteExtendedAttributes
Traverse
ExecuteFile
DeleteSubdirectoriesAndFiles
ReadAttributes
WriteAttributes
Write
Delete
ReadPermissions
Read
ReadAndExecute
Modify
ChangePermissions
TakeOwnership
Synchronize
FullControl

FileSystemRights enumeration has a FlagsAttribute attribute that allows a bitwise combination of its member values. In MSH, "-bor" is used as "Bitwise OR" Operator. To better understand that, we can find from following example that "Read" actually means ReadData | ReadExtendedAttributes | ReadAttributes |ReadPermissions.

[int]([System.Security.AccessControl.FileSystemRights]::Read)

131209

[System.Security.AccessControl.FileSystemRights]::ReadData `
-bor [System.Security.AccessControl.FileSystemRights]::ReadExtendedAttributes `
-bor [System.Security.AccessControl.FileSystemRights]::ReadAttributes `
-bor [System.Security.AccessControl.FileSystemRights]::ReadPermissions

131209

Added on 2006-01-17 12:27
Marcel Comment:"A comma separated list of values cast can be cast to a flags enum."

[System.Security.AccessControl.FileSystemRights] "ReadData, ReadExtendedAttributes, ReadAttributes, ReadPermissions"

Will do same thing!

So it is easier to use the combination values such as FullControl, Read and Write, rather than specifying each component value separately. But we still can build a custumized value such as "Read OR Write" as below.

$Rights= [System.Security.AccessControl.FileSystemRights]::Read `
-bor [System.Security.AccessControl.FileSystemRights]::Write

2. All of our previous examples were simple because we never deal with inheritance and propagation settings. If a directroy contain both files and child directory, things get more compicated. Every access rule is either explicit or inherited: inherited rules come from a parent container while explicit rules are added by user later on. So you can only manipulate its explicit rules.
There are two inheritance flags: container inherit (CI), object inherit (OI) and None.
There are two propagation flags: inherit only (IO), no-propagate inherit (NP) and None.

$Inherit=[System.Security.AccessControl.InheritanceFlags]::ContainerInherit `
-bor [System.Security.AccessControl.InheritanceFlags]::ObjectInherit
$Prop=[System.Security.AccessControl.PropagationFlags]::InheritOnly

Now we can construct our access rule and set it to a directory:

# set AccessControlType : Allow / Deny
$Access=[System.Security.AccessControl.AccessControlType]::Allow
# create new access rule
$AccessRule = new-object System.Security.AccessControl.FileSystemAccessRule `
("testac",$Rights,$Inherit,$Prop,$Access)
# validate access rule
$Sid = $AccessRule.IdentityReference.Translate`
([System.Security.Principal.securityidentifier])
$ACL=get-acl D:\msh\tmp
$ACL.AddAccessRule($AccessRule)
set-acl -AclObject $ACL -Path D:\msh\tmp

3. What if you do NOT want to inherit access rules from you parents?

$ACL=get-acl D:\msh\tmp
$ACL.SetAccessRuleProtection(
           # changes from parent won't propagate
           $true,
           # do not keep current inheritance settings
           $false );
set-acl -AclObject $ACL -Path D:\msh\tmp

Reference: Manage Access to Windows Objects with ACLs and the .NET Framework

[Edit: Monad has now been renamed to Windows PowerShell. This script or discussion may require slight adjustments before it applies directly to newer builds.]

Tags: msh monad PowerShell

Use Search-Entrez Cmdlet and Format Results

2006-01-13T16:51:00.000-05:00

In my previous blog entry Author a Monad Cmdlet as Web Service Client, I created a cmdlet Search-Entrez. Let's see what we can do with this cmdlet.

get-command search-entrez | format-list

Name : search-entrez
CommandType : Cmdlet
Definition : search-entrez [-Database] String [-Keywords] String [[-MaxRecor
d] String] [[-Email] String] [[-Field] String] [[-RelativeDate]
String] [[-MinimumDate] String] [[-MaximumDate] String] [[-Dat
eType] String] [-Verbose] [-Debug] [-ErrorAction ActionPreferen
ce] [-ErrorVariable String] [-OutVariable String] [-OutBuffer I
nt32] [-WhatIf] [-Confirm]

Path :
AssemblyInfo :
DLL : D:\msh\Entrez-mshsnapin.dll
HelpFile : Entrez-mshsnapin.dll-Help.xml
ParameterSets : {__AllParameterSets}
Type : Entrez.SearchEntrezCmd
Verb : search
Noun : entrez

Database: the Entrez database name (can be pubmed, protein, nucleotide, nuccore, nucgss, nucest, structure, genome, books, cancerchromosomes, cdd, domains, gene, genomeprj, gensat, geo, gds, homologene, journals, mesh, ncbisearch, nlmcatalog, omia, omim, pmc, popset, probe, pcassay, pccompound, pcsubstance, snp, taxonomy, unigene, unists)

Keywords: the search strategy. All words should be URL encoded (that is to say blank should be +). Can be keywords plus "AND/OR/NOT"

Other parameter is not mandatory.
Using MaximumRecord would limit numbers of results returned to you.
Using Email parameter would help NCBI server inform you if something went wrong.

$resultSummary=search-entrez pubmed "cancer+T+cell" -MaxRecord 10 -Email test@test.com -verbose

VERBOSE: Searching Entrez database...
Database: pubmed
Keywords: cancer+T+cell
VERBOSE: Creating Entrez WebService...
VERBOSE: Submit search...
VERBOSE: Number of results item found:51521
VERBOSE: Getting results summary...
VERBOSE: Number of item retrieved:10
VERBOSE: Write results to pipline...

Because I was too lazy to write code for results parsing, I just emitted an eUtils.eSummaryResultType object to monad pipline, which make search-Entrez cmdlet return a non-human-readable output.

$resultSummary | format-list

ERROR :
DocSum : {16408214, 16407851, 16406172, 16404742, 16404738, 16404427, 16403911, 16403282, 16401550, 16399573}

Where ERROR is a string object for error and Docsum is a collection of DocSumType objects.

public class DocSumType {
private string idField;
private ItemType[] itemField;
public string Id {
get {return this.idField;}
set {this.idField = value;}}
public ItemType[] Item {
get {return this.itemField;}
set {this.itemField = value;}}
}

To make things even more complicated, the DocSumType.ItemType is a collection of NESTED ItemType object. As you can see ItemType.Items is also a collection of ItemType objects.

public class ItemType {
private ItemType[] itemsField;
private string[] textField;
private string nameField;
private ItemTypeType typeField;
public ItemType[] Items {
get {return this.itemsField;}
set {this.itemsField = value;}}
public string[] Text {
get {return this.textField;}
set {this.textField = value;}}
public string Name {
get {return this.nameField;}
set {this.nameField = value;}}
public ItemTypeType Type {
get {return this.typeField;}
set {this.typeField = value;}}
}

The good news is that they seem to only nest TWO levels deep for pubmed results. So I write a script to format results.

# Format-eSummary.msh
# Script to format (pubmed) results from Search-Entrez cmdlet
#requires -MshSnapIn EntrezSnapin
param
(
$Summary =$(throw "Please specify an eUtils.eSummaryResultType object to parise.")
)
if ($Summary.DocSum.Length -eq 0)
{
"No hits found!"
return 1
}
foreach ($sum in $Summary.DocSum)
{
"Primary IDs:" + $sum.Id
foreach ($Item in $sum.Item)
{
$Item.Name + ": " + $Item.Text
if ($Item.Items)
{
foreach($ChildItem in $Item.Items)
{
"`t" + $ChildItem.Name + ": "+ $ChildItem.Text
}
}
} "====================================================="
}
return 0

.\Format-eSummary.msh $resultSummary

Primary IDs:16408214
PubDate: 2006 Jan 12
EPubDate: 2006 Jan 12
Source: Cancer Immunol Immunother
AuthorList:
Author: Prell RA
Author: Gearin L
Author: Simmons A
Author: Vanroey M
Author: Jooss K
Title: The anti-tumor efficacy of a GM-CSF-secreting tumor cell vaccine is not
inhibited by docetaxel administration.
Volume:
Issue:
Pages: 1-9
LangList:
Lang: English
NlmUniqueID: 8605732
ISSN: 0340-7004
ESSN: 1432-0851
PubTypeList:
PubType: Journal Article
RecordStatus: PubMed - as supplied by publisher
PubStatus: aheadofprint
ArticleIds:
doi: 10.1007/s00262-005-0116-4
pubmed: 16408214
DOI: 10.1007/s00262-005-0116-4
History:
received: 2005/11/08 00:00
accepted: 2005/12/15 00:00
aheadofprint: 2006/01/12 00:00
pubmed: 2006/01/13 09:00
medline: 2006/01/13 09:00
References:
HasAbstract: 1
PmcRefCount: 0
FullJournalName: Cancer immunology, immunotherapy : CII.
SO: 2006 Jan 12;:1-9
=====================================================
Primary IDs:16407851
PubDate: 2006 Jan 9
EPubDate: 2006 Jan 9
Source: Oncogene
......

Tags: msh monad

Author a Monad Cmdlet as Web Service Client

2006-01-12T22:35:00.000-05:00

Steps to author a cmdlet, Search-Entrez, which works as NCBI eUtils web service client to build your own Entrez data pipeline.

For those who don't use Entrez: Entrez is the integrated, text-based search and retrieval system used at NCBI for the major databases, including PubMed, Nucleotide and Protein Sequences, Protein Structures, Complete Genomes, Taxonomy, and others. If you were a biologist, you probably use Entrez everyday. The graphic below for a breif view of Entrez integration.

Usually general user will access Entrez by its web form interface. For data mining NCBI made a utility called: eUtils, which provide a URL tool as well as a SOAP interface. The URL tool requires to construct a customized search URL and returns results as XML. To save some time, I decided to use SOAP interface (version 1.3a). More help on eUtils utility can be found here. Information about SOAP interface can be find here.

eUtils have contain several tools :

EInfo: Provides field index term counts, last update, and available links for each database.
ESearch: Searches and retrieves primary IDs (for use in EFetch, ELink, and ESummary) and term translations and optionally retains results for future use in the user's environment.
EPost: Posts a file containing a list of primary IDs for future use in the user's environment to use with subsequent search strategies.
ESummary: Retrieves document summaries from a list of primary IDs or from the user's environment.
EFetch: Retrieves records in the requested format from a list of one or more primary IDs or from the user's environment.
ELink: Checks for the existence of an external or Related Articles link from a list of one or more primary IDs. Retrieves primary IDs and relevancy scores for links to Entrez databases or Related Articles; creates a hyperlink to the primary LinkOut provider for a specific ID and database, or lists LinkOut URLs and Attributes for multiple IDs.
EGQuery: Provides Entrez database counts in XML for a single search using Global Query.
ESpell: Retrieves spelling suggestions.

We will use ESearch and ESummary this time. Now let's start:
1. Create a "Class Libaray" in Visual C# Express.
2. Project -> Add Reference -> System.Management.Automation
3. Project -> Add Web Reference -> http://eutils.ncbi.nlm.nih.gov/entrez/eutils/soap/eutils_lite.wsdl (without EFech tool) -> Go -> Change Web Refernece name to "eUtils"
4. Create new class derived from MshSnapin object.
5. Create new class derived from Cmdlet object.
6. Build project to EntrezSnapin.dll.
7. Install mshsnapin using .NET SDK installutils.exe tool.

set-alias installutil $env:windir\Microsoft.NET\Framework\v2.0.50727\installutil
installutil EntrezSnapin.dll

8. Load mshsnapin using add-mshsnapin cmdlet

get-mshsnapin -reg
add-mshsnapin EntrezSnapin

9. Now you can enjoy powerful Entrez search tool under msh prompt.
C# Code:

using System;
using System.ComponentModel;
using System.Management.Automation;
using System.Collections;
namespace Entrez
{
    /// <summary> This class defines the properties of a snapin</summary>
    [RunInstaller(true)]
    public class EntrezSnapin : MshSnapIn
    {
        /// <summary>Creates an instance of DemonSnapin class.</summary>
        public EntrezSnapin() : base()
        {
        }
        ///<summary>The snapin name which is used for registration</summary>
        public override string Name
        { get
            { return "EntrezSnapin";
            }
        }
        /// <summary>Gets vendor of the snapin.</summary>
        public override string Vendor
        { get
            { return "http://mshforfun.blogspot.com";
            }
        }
        /// <summary>Gets description of the snapin. </summary>
        public override string Description
        { get
            { return "Cmdlets to build your own NCBI Entrez Data pipeline";
            }
        }
    }
    /// <summary>
    /// Submit search to entrez database and return results IDs
    /// </summary>
    [Cmdlet("search", "entrez", SupportsShouldProcess = true)]
    public class SearchEntrezCmd : Cmdlet
    {
        #region Parameters
        private string db="pubmed";
        /// <summary>Entrez Database</summary>
        [Parameter(Mandatory=true, Position=0)]
        public string Database
        {
            get { return db; }
            set { db = value; }
        }
        private string term;
        /// <summary>search terms or phrases with or without Boolean operators</summary>
        [Parameter(Mandatory = true, Position = 1)]
        public string Keywords
        {
            get { return term; }
            set { term = value; }
        }
        private string retmax = "50";
        /// <summary>Maximum number of records to return</summary>
        [Parameter(Position = 2)]
        public string MaxRecord
        {
            get { return retmax; }
            set { retmax = value; }
        }
        private string email = "";
        /// <summary>To identify your search results history on server</summary>
        [Parameter(Position = 3)]
        public string Email
        {
            get { return email; }
            set { email = value; }
        }
        private string field = "";
        /// <summary>specific search field for pubmed database</summary>
        [Parameter(Position = 4)]
        public string Field
        {
            get { return field; }
            set { field = value; }
        }
        private string reldate = "";
        /// <summary>Limit items a number of days immediately preceding today's date</summary>
        [Parameter(Position = 5)]
        public string RelativeDate
        {
            get { return reldate; }
            set { reldate = value; }
        }
        private string mindate = "";
        /// <summary>Lower bounded of two specific dates</summary>
        [Parameter(Position = 6)]
        public string MinimumDate
        {
            get { return mindate; }
            set { mindate = value; }
        }
        private string maxdate = "";
        /// <summary>Lower bounded of two specific dates</summary>
        [Parameter(Position = 7)]
        public string MaximumDate
        {
            get { return maxdate; }
            set { maxdate = value; }
        }
        private string datetype = "";
        /// <summary>Limit dates to a specific date field based on database</summary>
        [Parameter(Position = 8)]
        public string DateType
        {
            get { return datetype; }
            set { datetype = value; }
        }
        #endregion

        private string usehistory = "y";
        private string tool = "Monad_Entrez_Client";
        private string retstart = "0";
        private string rettype = "uilist";
        private string sort = "";
        /// <summary>
        /// Communicating purpose only
        /// </summary>
        protected override void BeginProcessing()
        {
            WriteVerbose("Searching Entrez database...\r\nDatabase: " + db + "\r\nKeywords: " + term);
        }
        /// <summary>
        /// Get Entrez search results
        /// </summary>
        protected override void ProcessRecord()
        {
            if (ShouldProcess("Searching Entrez database...\r\nDatabase: " + db + "\r\nKeywords: " + term))
            {
                try
                {
                    WriteVerbose("Creating Entrez WebService...");
                    eUtils.eUtilsService serv = new eUtils.eUtilsService();
                    WriteDebug("WebService:" + serv.ToString());
                    WriteVerbose("Submit search...");
                    // call NCBI ESearch utility
                    // NOTE: search term should be URL encoded
                    eUtils.eSearchResultType Search_Results = serv.run_eSearch_MS(db, term, "", "", usehistory, tool, email, field, reldate, mindate, maxdate, datetype, retstart, retmax, rettype, sort );
                    WriteDebug("Search Results:" + Search_Results.ToString());
                    if (Search_Results.ERROR != null)
                    {
                        ThrowTerminatingError(new ErrorRecord(new Exception("web service method error"), "eUtils.run_eSearch_MS", ErrorCategory.InvalidResult, Search_Results));
                    }
                    WriteVerbose("Number of results item found:" + Search_Results.Count);
                    WriteVerbose("Getting results summary...");
                    // call NCBI ESummary utility
                    // NOTE: search term should be URL encoded
                    eUtils.eSummaryResultType Summary_Results = serv.run_eSummary_MS( db, "", Search_Results.WebEnv, Search_Results.QueryKey, Search_Results.RetStart, Search_Results.RetMax, tool, email );
                    WriteDebug("Results summary:" + Summary_Results.ToString());
                    if (Summary_Results.ERROR != null)
                    {
                        ThrowTerminatingError(new ErrorRecord(new Exception("web service method error"), "eUtils.run_eSummary_MS", ErrorCategory.InvalidResult, Summary_Results));
                    }
                    WriteVerbose("Number of item retrieved:" + Summary_Results.DocSum.Length);
                    WriteVerbose("Write results to pipline...");
                    // results output
                    WriteObject(Summary_Results);
                }
                catch (Exception e)
                {
                    ThrowTerminatingError(new ErrorRecord(e, "eUtils", ErrorCategory.ResourceUnavailable, db+term));
                }
            }
        }
    }
}

[Edit: Monad has now been renamed to Windows PowerShell. This script or discussion may require slight adjustments before it applies directly to newer builds.]

Tags: msh monad PowerShell

Bring Statistical Analysis Power to MSH: Hosting R Environment

2006-01-09T00:39:00.000-05:00

Quote from /\/\o\/\/ blog entry Hosting Iron python in MSH: "Nice thing about Monad that it's easy to host other Languages inline." Well, since /\/\o\/\/ did VB.net, Monad team blog did C# and those japanese guy did python. I would like to something a biologist should do: Hosting R Environment in MSH.

R is an integrated suite of software facilities for data manipulation, calculation and graphical display. It consists of a language plus a run-time environment with graphics, a debugger, access to certain system functions, and the ability to run programs stored in script files. R language is an interpreted computer language derived from S language. Scientist use R mostly as a statistical analysis tool. You can download a copy of R from R-project home page (under GNU license).

My last project involve R is a gene expression profile analysis from microarray experiment two years ago. If you were a computer geek, it will take you less than 30min to learn to use R environment because it is object-oriented and synax similar to C.

Monad can can interact with other program using COM-based technologies. For example:
1. From /\/\o\/\/ blog: Monad Really Does Rock
2. From Monad team blog: Check spelling script

In order to use R language interpreter in MSH, we need another R addon: R-DCOM, which works with Rproxy.dll (in the R distribution) and R.dll to support transfer of data to and from R and remote execution of R commands, as well as embedding of an R graphics window.

After insatll R and r-DCOM, we can enjoy the power of R under MSH prompt:

# Create a new com object for R

$RCom = new-object -com StatConnectorSrv.StatConnector

# Call Init method to load R environment

$RCom.Init("R")

# Check out available methods

$Rcom |gm

TypeName: System.__ComObject#{18c8b660-81a2-11d3-9254-00e09812f727}

Name MemberType Definition
---- ---------- ----------
AddGraphicsDevice Method void AddGraphicsDevice (string, ISGFX)
Close Method void Close ()
Evaluate Method Variant Evaluate (string)
EvaluateNoReturn Method void EvaluateNoReturn (string)
GetConnectorInformation Method string GetConnectorInformation (Informa...
GetErrorId Method int GetErrorId ()
GetErrorText Method string GetErrorText ()
GetInterpreterInformation Method string GetInterpreterInformation (Infor...
GetServerInformation Method string GetServerInformation (Informatio...
GetSupportedTypes Method void GetSupportedTypes (int)
GetSymbol Method Variant GetSymbol (string)
Init Method void Init (string)
RemoveGraphicsDevice Method void RemoveGraphicsDevice (string)
SetCharacterOutputDevice Method void SetCharacterOutputDevice (IStatCon...
SetErrorDevice Method void SetErrorDevice (IStatConnectorChar...
SetSymbol Method void SetSymbol (string, Variant)
SetTracingDevice Method void SetTracingDevice (IStatConnectorCh...
SetUserInterfaceAgent Method void SetUserInterfaceAgent (IStatConnec...

# Send expression to R interpreter
# where c() is a function to assign a "vector" Variant to x
# remember R language is case senitive

$Rcom.EvaluateNoReturn ("x <- c(79.98, 80.04, 80.02, 80.04, 80.03, 80.03, 80.04, 79.97, 80.05, 80.03, 80.02, 80.00, 80.02)")
$Rcom.EvaluateNoReturn ("y <-c(80.02, 79.94, 79.98, 79.97, 79.97, 80.03, 79.95, 79.97)")

# use R graphic device to make boxplot

$Rcom.EvaluateNoReturn ("boxplot(x, y)")

# Student t test: two variant; two sided; unpaired

$Rcom.Evaluate("t.test(x, y, alternative = c(`"two.sided`"))")

3.24986738055522 #t
12.0271084945182 #delta-f
0.00693932661444795 #P-value (if less than 0.05, means statitical significant)
0.0138552637266242 #lower 95 percent confidence interval
80.0207692307692 #average of x
0 #alternative hypothesis: true difference in means is not equal to 0
two.sided
Welch Two Sample t-test
x and y
# Dispose R Environment

$Rcom.Close()

Have fun!

[Edit: Monad has now been renamed to Windows PowerShell. This script or discussion may require slight adjustments before it applies directly to newer builds.]

Tags: msh monad PowerShell

GUI version of NCBI Blastn(nr) MSH script

2006-01-08T01:07:00.000-05:00

An NCBI Blastn (using nr database) client MSH script with winform GUI. See NCBI Blastn under MSH command line for a introduction of NCBI Blastn program.

Although personally I like command line interface, there are always some users prefer GUI. Well, since monad can use System.Windows.Forms.form, it can create comman winform GUI which is similar to what tk/tcl can do.

One of the big advantage of monad is that script block can be used a event handler.

Another thing need to mention is that monad use grave-accent(`) but not "\" for escape sequences. To get more help on escape character using command:
help about_escape_character
Because NCBI Blast server running unix like OS, so the returning results using \n but not \r\n for a new line. We have to replace those \n with \r\n to get a proper view in textbox control.

# Blastn-GUI.msh
# ===========================================================================
#
# This code is for test purposes only. Use it at your own risk.
#
# Please do not submit or retrieve more than one request every two seconds.
#
# Results will be kept at NCBI for 24 hours. For best batch performance,
# It is recommend that you submit requests after 2000 EST (0100 GMT) and
# retrieve results before 0500 EST (1000 GMT).
#
# by tony 2006 http://mshforfun.blogspot.com/
# ===========================================================================
[void] [Reflection.Assembly]::Load( `
"System.Windows.Forms, Version=2.0.0.0, Culture=neutral, `
PublicKeyToken=b77a5c561934e089")
$script:form = new-object System.Windows.Forms.form
$Panel = new-object System.Windows.Forms.Panel
$Label = new-object System.Windows.Forms.Label
$Query = new-object System.Windows.Forms.TextBox
$Submit = new-object System.Windows.Forms.Button
$script:Results = new-object System.Windows.Forms.TextBox
$script:Status = new-object System.Windows.Forms.TextBox

function SubmitQuery([String]$Sequence)
{
if ($Sequence -match "[^ATGCatgcNn]")
{
    $Status.Text += "Error 1: Invalid charactors in query sequence! Only A/T/G/C/N can be used!`r`n`r`n"
    $Status.Refresh()
    $Status.ScrollToCaret()
    return
}

$Status.Text += "`r`n==================================================`r`n"
$Status.Text += (get-date).ToString("MM-dd-yyyy HH:mm") + ": Submiting sequence...`r`n"
$Status.Refresh()
$Status.ScrollToCaret()

$uri="http://www.ncbi.nlm.nih.gov/blast/Blast.cgi?CMD=Put&PROGRAM=blastn&DATABASE=nr&QUERY=" + $Sequence
$BlastClient = new-object System.Net.WebClient
$pagecontent = $BlastClient.DownloadString($uri);
if ($pagecontent -match "   RID = (.*)")
{
   $RID=$Matches[1]
   $Status.Text += "RID =" + $RID + "`r`n"
   $Status.Refresh()
   $Status.ScrollToCaret()
}

if ($pagecontent -match "   RTOE = (.*)")
{
$TimeToComplete= $Matches[1]
$Status.Text += "Estimated Time to finish searching: " + $TimeToComplete + " seconds`r`nWaiting...`r`n`r`n"
$Status.Refresh()
$Status.ScrollToCaret()
}

Start-sleep $TimeToComplete

While ($true)
{

   $uri= "http://www.ncbi.nlm.nih.gov/blast/Blast.cgi?CMD=Get&FORMAT_OBJECT=SearchInfo&RID=" + $RID
   $pagecontent = $BlastClient.DownloadString($uri);
   if ($pagecontent -match "Status=WAITING")
   {
    $Status.Text += (get-date).ToString("MM-dd-yyyy HH:mm") + ": Search not finished yet, waiting... `r`n"
    $Status.Refresh()
    $Status.ScrollToCaret()
    Start-sleep 5
    continue
   }
   if ($pagecontent -match "Status=FAILED")
   {
     $Status.Text += (get-date).ToString("MM-dd-yyyy HH:mm") + ": Error 2 -- Search failed!`r`n`r`n"
     $Status.Refresh()
     return
   }
   if ($pagecontent -match "Status=UNKNOWN")
   {
     $Status.Text += (get-date).ToString("MM-dd-yyyy HH:mm") + ": Error 3-- Search expired!`r`n`r`n"
     $Status.Refresh()
     return
   }
   if ($pagecontent -match "Status=READY")
   {
      if ($pagecontent -match "ThereAreHits=yes")
      {
         $Status.Text += (get-date).ToString("MM-dd-yyyy HH:mm") + ": Search complete, retrieving results...`r`n`r`n"
    $Status.Refresh()
    $Status.ScrollToCaret()
    break
      }
      else
      {
        $Status.Text += (get-date).ToString("MM-dd-yyyy HH:mm") + ": Error 4-- No hits found!`r`n`r`n"
    $Status.Refresh()
        return
      }
   }
   $Status.Text += (get-date).ToString("MM-dd-yyyy HH:mm") + ": Error 5-- Unknown error! `r`n`r`n"
   $Status.Refresh()
   return
}
$uri= "http://www.ncbi.nlm.nih.gov/blast/Blast.cgi?CMD=Get&RID=" + $RID + "&ALIGNMENTS=500&ALIGNMENT_VIEW=QueryAnchored&FORMATOBJECT=Alignment&FORMAT_TYPE=TEXT"
$Results.Text = $BlastClient.DownloadString($uri).Replace("`n","`r`n");
return
}

$form.text = "NCBI Blastn(nr) GUI"
$form.Size = "800,600"

$Panel.Size = "794, 32"
$Panel.Location = "0, 0"

$Label.Text = "Query:"
$Label.Location = "10, 8"
$Label.Size = "80, 32"
$Panel.Controls.Add($Label)

$Query.Location = "100, 8"
$Query.Size = "500, 20"
$Panel.Controls.Add($Query)

$Submit.Location = "710, 8"
$Submit.Text = "Submit"
$Submit.Add_Click({SubmitQuery($Query.Text)})
$Panel.Controls.Add($Submit)

$form.Controls.Add($Panel)

$Results.Location = "0, 40"
$Results.Multiline = $True
$Results.Size = "790, 400"
$Results.ScrollBars = "Vertical"
$Results.ReadOnly = $true
$form.Controls.Add($Results)

$Status.Location = "0, 440"
$Status.Multiline = $True
$Status.Size = "790, 100"
$Status.ScrollBars = "Vertical"
$Status.ReadOnly = $true
$form.Controls.Add($Status)

$form.showdialog()

[Edit: Monad has now been renamed to Windows PowerShell. This script or discussion may require slight adjustments before it applies directly to newer builds.]

Tags: msh monad PowerShell

Play with ACL in MSH (continued)

2006-01-03T16:10:00.000-05:00

/\/\o\/\/ kindly replied my blog entry “Play with ACL in MSH”. In his blog entry “Adding a Simple AccesRule to a file ACL in MSH”, he found a shortcut to create an instance of “System.Security.AccessControl.FileSystemAccessRule”. Great job! But things can get even better!

1. You want to change a accessrule but not overwrite it, so AddAccessRule() method is better than SetAccessRule() method. If you want to grant a FullControl privilege to exact same user, they are the same. What if you want to apply a more complex privilege? For example: Read + Write nut not Delete.

2. get-acl cmdlet return different object:
For a file, it will return “System.Security.AccessControl.FileSecurity”;
For a directory, it will return “System.Security.AccessControl.DirectorySecurity”.
Luckily enough, both object have AddAccessRule() and RemoveAccessRule() methods. And Even better, both methods take a “System.Security.AccessControl.FileSystemAccessRule” as parameter. So we can set Access control of a file or directory in one simple script.

# AddRemove-AccessRule.MSH
# Add or remove simple access rule to a file/directory
# using text parameters
#
# original writen by /\/\o\/\/ 2006
# http://mow001.blogspot.com
#
# modified by tony 2006
# http://mshforfun.blogspot.com
#
#Usage AddRemove-Acl FileOrDirectory (Action) user Rights (Access)
# Action: Add / Remove
# Rights: ListDirectory / ReadData / WriteData / CreateFiles /
# CreateDirectories / AppendData / ReadExtendedAttributes /
# WriteExtendedAttributes / Traverse / ExecuteFile /
# DeleteSubdirectoriesAndFiles / ReadAttributes / WriteAttributes/ Write /
# Delete / ReadPermissions / Read / ReadAndExecute / Modify /
# ChangePermissions / TakeOwnership / Synchronize / FullControl
# Access: Allow / Deny

Param (
$FileDir,
$User,
$Action = "Add",
[System.Security.AccessControl.FileSystemRights] $Rights,
[System.Security.AccessControl.AccessControlType] $Access = "Allow"
)

trap{Break}

$AccessControl = get-acl $FileDir

$AccessRule = new-object System.Security.AccessControl.FileSystemAccessRule($User,$Rights,$Access)

# check if given user is Valid, this will break function if not so.
$Sid = $AccessRule.IdentityReference.Translate([System.Security.Principal.securityidentifier])

resolve-path $FileDir

"=============================================================="
"Before changes"
$AccessControl.AccessToString

if ($Action.ToUpper() -eq "ADD") {$AccessControl.AddAccessRule($AccessRule)}
elseif ($Action.ToUpper() -eq "REMOVE") {$AccessControl.RemoveAccessRule($AccessRule)}

set-acl -aclobject $AccessControl -path $FileDir

"=============================================================="
"After " + $Action + " access rule: " + "User-"+ $User + " Rights-" + $Rights + " Access-" + $Access
(get-acl $FileDir).AccessToString

[Edit: Monad has now been renamed to Windows PowerShell. This script or discussion may require slight adjustments before it applies directly to newer builds.]

Tags: msh monad PowerShell

Working with SQL server in monad using pre-compiled .NET Class Library (3)

2006-01-03T00:23:00.000-05:00

Now the usual routine operation:

Add a plasmid
Edit a plasmid

Why there is no “Delete plasmid” operation?

Because Biologists Never, Never, Ever lose their plasmids!

1. Add following code to our PlasmidView.DLL:

namespace PlasmidView

public class SQLProvider
{
public static PlasmidFull PlasmidFull_ByPlasmidID (int plasmidid)
{… }

public static void Plasmid_Add (PlasmidFull PlasmidToAdd)
{… }
public static void Plasmid_Edit (PlasmidFull PlasmidToEdit)
{…}

public static string Enzyme_ByEnzymeID (int enzymeID)
{…}
public static string BacResistanceType_ByTypeID (int TypeID)
{…}
public static string MamResistanceType_ByTypeID (int TypeID)
{…}
public static string InsertType_ByInsertTypeID (int insertTypeID)
{…}
}
public class PlasmidFull : PlasmidCore
{…}
}

2. Get a Plasmid by PlasmidID

[void][System.Reflection.Assembly]::LoadFile("D:\MSH\PlasmidView.dll")
$Plasmid= [PlasmidView.SQLProvider]::PlasmidFull_ByPlasmidID (10)
$Plasmid

BacResistance : 1
MamResistance : 2
HasGlycerolStock : False
InsertDescription : IL-17R full length receptor
InsertLength : 2451
InsertTypeID : 1
InsertType : Human
LeftEnzymeID : 14
LeftEnzyme : BamHI
RightEnzymeID : 62
RigthEnzyme : NcoI
IsBluntLigation : False
IsSequenced : True
Comments :
FullSeq :
InsertSeq :
PlasmidID : 10
BoxID : B
BoxPosition : 1
PlasmidName : IL-17R-FL-HA
DateCreated : 10/30/2005 9:18:18 PM
UserID : 1
UserCreated : me
PlasmidType : Cytokine Receptor
PlasmidTypeID : 3
ParentPlasmidID : 9
ParentPlasmid : pCMV4 HA Neo

3. Edit this plasmid entry

$Plasmid.PlasmidName = “mouse IL-17R-FL-HA”
$Plasmid.InsertTypeID = 2
[PlasmidView.SQLProvider]::Plasmid_Edit($Plasmid)
$Plasmid= [PlasmidView.SQLProvider]::PlasmidFull_ByPlasmidID (10)
$Plasmid

BacResistance : 1
MamResistance : 2
HasGlycerolStock : False
InsertDescription : IL-17R full length receptor
InsertLength : 2451
InsertTypeID : 2
InsertType : Mouse
LeftEnzymeID : 14
LeftEnzyme : BamHI
RightEnzymeID : 62
RigthEnzyme : NcoI
IsBluntLigation : False
IsSequenced : True
Comments :
FullSeq :
InsertSeq :
PlasmidID : 10
BoxID : B
BoxPosition : 1
PlasmidName : mouse IL-17R-FL-HA
DateCreated : 10/30/2005 9:18:18 PM
UserID : 1
UserCreated : me
PlasmidType : Cytokine Receptor
PlasmidTypeID : 3
ParentPlasmidID : 9
ParentPlasmid : pCMV4 HA Neo
4. Add a plasmid entry

$Plasmid= new-object PlasmidView.PlasmidFull
$Plasmid.BacResistance = 1
$Plasmid.MamResistance = 2
$Plasmid.HasGlycerolStock = $False
$Plasmid.InsertDescription = "truncated IL-17R receptor"
$Plasmid.InsertLength = 1851
$Plasmid.InsertTypeID = 2
$Plasmid.LeftEnzymeID = 14
$Plasmid.RightEnzymeID = 62
$Plasmid.IsBluntLigation = $False
$Plasmid.IsSequenced = $True
$Plasmid.BoxID = "B"
$Plasmid.BoxPosition = 90
$Plasmid.PlasmidName = "truncated mouse IL-17R-FL-HA"
$Plasmid.DateCreated = get-date
$Plasmid.UserID = 1
$Plasmid.PlasmidTypeID = 3
$Plasmid.ParentPlasmidID = 9

$Plasmid
$Plasmid= [PlasmidView.SQLProvider]::Plasmid_Add($Plasmid)
[PlasmidView.SQLProvider]::Plasmid_All() where-object {$_.PlasmidName
-like "*truncated mouse IL-17R-FL-HA*"}

PlasmidID : 22
BoxID : B
BoxPosition : 90
PlasmidName : truncated mouse IL-17R-FL-HA
DateCreated : 1/3/2006 12:09:52 AM
UserID : 1
UserCreated : me
PlasmidType : Cytokine Receptor
PlasmidTypeID : 3
ParentPlasmidID : 9
ParentPlasmid : pCMV4 HA Neo

Conclusion:
1. One of the advantages of using monad is that it is fully integrated with .Net frameworks. Although MSH does not have class definition machinery right now, we can easily construct a customized class in C# and use it in monad.

2. Pre-compiled class library will greatly simplify monad script. Even a new user can be trained to work with fairly complex administration jobs.

3. Security should always be in mind! Expose crucial objects or methods to unauthorized user may cause disastrous effect to system.

Tags: msh monad

Working with SQL server in monad using pre-compiled .NET Class Library (2)

2006-01-01T21:28:00.000-05:00

Now we can do something fun.

1. Add following code to our PlasmidView.DLL

using System.Collections;
namespace PlasmidView
{
public class SQLProvider
{

// return ArrayList of “PlasmidCore” objects of all plasmid information
public static ArrayList Plasmid_All ()
{...}

// return a PlasmidCore object
public static PlasmidCore PlasmidCore_ByPlasmidID (int plasmidid)
{...}

// return User objects
public static User User_ByUserID (int userid)
{...}

// return Plasmid Type (String)
public static string PlasmidType_ByPlasmidTypeID (int plasmidTypeID)
{...}

public class PlasmidCore
{... }

public class User
{... }
}

I omit most of C# code here

2. Try to get all plasmid informtion

[void][System.Reflection.Assembly]::LoadFile("D:\MSH\PlasmidView.dll")
[PlasmidView.SQLProvider]::Plasmid_All()

PlasmidID       : 1
BoxID           : A
BoxPosition     : 1
PlasmidName     : pCMV4 Hygro
DateCreated     : 10/30/2005 8:58:24 PM
UserID          : 1
UserCreated     : me
PlasmidType     : Backbone Vector
PlasmidTypeID   : 1
ParentPlasmidID : 0
ParentPlasmid   : None

PlasmidID       : 2
BoxID           : A
BoxPosition     : 2
PlasmidName     : pCMV4 Neo
DateCreated     : 10/30/2005 9:07:16 PM
UserID          : 1
UserCreated     : me
PlasmidType     : Backbone Vector
PlasmidTypeID   : 1
ParentPlasmidID : 0
ParentPlasmid   : None
…

3. Try to search plasmid with “hygro” in their names:

[PlasmidView.SQLProvider]::Plasmid_All() where-object {$_.PlasmidName -like “*hygro*”} foreach-object {“============”; “ID:”+ $_.PlasmidID; “Name:” + $_.PlasmidName}

============
ID:1
Name:pCMV4 Hygro
============
ID:20
Name:TAg Hygro
…

Conclusion:
1. We can easily return a group of objects to monad using ArrayList. Unlike strong typed language (for example, C#) monad detects the base object of each Item in ArrayList. You do NOT have to cast each ArrayList Item to its original base object (PlasmidView.PlasmidCore) when quoting them in a pipline.

2. When database is small, using where-object cmdlet to do search is a good idea. Actually I even forgot I had a Plasmid_Search( ) method in my original C# code. But when database grew bigger, better use a Search function to return limited number of objects only from search results.

Tags: msh monad

Working with SQL server in monad using pre-compiled .NET Class Library (1)

2005-12-29T20:25:00.000-05:00

As an immunologist, I have to deal with a lot of plasmids. In order to share plasmids with my colleagues, I stored all information of plasmids in an SQL server (express edition) running on a Windows XP desktop. I wrote a small winform program to manage the information stored the SQL server. I have all the data type and program logic (including SQL query) packaged in NET Classes. So if I could reuse those .NET classes in monad, I can do some routine works in MSH scripts and will not waste time to start from scratch.

What is plasmid anyway? Well, plasmid is a circular DNA molecule found in E.coli. Biologists often use them for gene cloning and expression. If you still don’t know what I am talking about, google (http://www.google.com/search?q=plasmid) for details.

First, let’s make a DLL ready for MSH.
I opened my winform project; got rid of GUI components; recompiled the project to Class Library (PlasmidView.DLL). The DLL contained classes below:

using System;
using System.Data;
using System.Data.SqlClient;

namespace PlasmidView
{
     public class SQLProvider
     {
          // retuen a System.Data.SqlClient.SqlConnection
private static SqlConnection GetDBConnection()     {
               string ConnectionStr = "server=localhost\\SQLExpress;database=PlasmidDB;Integrated Security=SSPI;Trusted_Connection=Yes";
               try {
                    return new SqlConnection(ConnectionStr);}
               catch (SqlException e)     {
                    string SqlMsg = ConstructSqlErrorMessage(e);}
               return null;}

          public static string ConstructSqlErrorMessage(SqlException e)
          {
               string SqlMsg="";
               for (int i=0; i < e.Errors.Count; i++)     {
                    SqlMsg += "Index #" + i + ";" +
                         "Message: " + e.Errors[i].Message + ";" +
                         "LineNumber: " + e.Errors[i].LineNumber + ";" +
                         "Source: " + e.Errors[i].Source + ";" +
                         "Procedure: " + e.Errors[i].Procedure;}
               return SqlMsg;}

          // return a “status” object represent database statitcs
          public static Status GetDBStatus () {
               Status StatusToGet= null;
               SqlConnection myConnection= GetDBConnection();
               try {
                    SqlCommand myCommand = new SqlCommand("dbo.DBStatus", myConnection);
// run a stored procedure “dbo.DBStatus”
                    myCommand.CommandType = CommandType.StoredProcedure;

                    myConnection.Open();
                    SqlDataReader reader = myCommand.ExecuteReader();
                    if (reader.Read()){
                         StatusToGet = new Status();
                         StatusToGet.TotalPlasmidNum          =     reader.GetInt32(0);
                         StatusToGet.TotalBackboneNum     =     reader.GetInt32(1);
                         StatusToGet.LastPlasmidID          =     reader.GetInt32(2);
                         StatusToGet.TotalEnzymeNum          =     reader.GetInt32(3);
                         StatusToGet.TotalUserNum          =     reader.GetInt32(4);}
               }
               catch (SqlException e){
                    string SqlMsg = ConstructSqlErrorMessage(e);}
               finally     {
                    myConnection.Close();}
               return StatusToGet;
          }

     }
     public class Status
     {
          int totalPlasmidNum          = 0;
          int totalBackboneNum          = 0;
          int lastPlasmidID          = 0;
          int totalEnzymeNum          = 0;
          int totalUserNum          = 0;

          public int TotalPlasmidNum {
               get {return totalPlasmidNum;}
               set {totalPlasmidNum=value;}
          }

          public int TotalBackboneNum {
               get {return totalBackboneNum;}
               set {totalBackboneNum=value;}
          }

          public int LastPlasmidID {
               get {return lastPlasmidID;}
               set {lastPlasmidID=value;}
          }

          public int TotalEnzymeNum {
               get {return totalEnzymeNum;}
               set {totalEnzymeNum=value;}
          }

          public int TotalUserNum     {
               get {return totalUserNum;}
               set {totalUserNum=value;}
          }
          public Status()     {}
}

Second, let’s load the PlasmidView.DLL to MSH

[void][System.Reflection.Assembly]::LoadFile("D:\MSH\PlasmidView.dll")

Third, Let’s create a instance of “Status” object:

$Status = new-object PlasmidView.Status
$Status |gm

    TypeName: PlasmidView.Status

Name                 MemberType Definition
----                 ---------- ----------
Equals               Method     System.Boolean Equals(Object obj)
get_LastPlasmidID    Method     System.Int32 get_LastPlasmidID()
get_TotalBackboneNum Method     System.Int32 get_TotalBackboneNum()
get_TotalEnzymeNum   Method     System.Int32 get_TotalEnzymeNum()
get_TotalPlasmidNum  Method     System.Int32 get_TotalPlasmidNum()
get_TotalUserNum     Method     System.Int32 get_TotalUserNum()
GetHashCode          Method     System.Int32 GetHashCode()
GetType              Method     System.Type GetType()
set_LastPlasmidID    Method     System.Void set_LastPlasmidID(Int32 value)
set_TotalBackboneNum Method     System.Void set_TotalBackboneNum(Int32 value)
set_TotalEnzymeNum   Method     System.Void set_TotalEnzymeNum(Int32 value)
set_TotalPlasmidNum  Method     System.Void set_TotalPlasmidNum(Int32 value)
set_TotalUserNum     Method     System.Void set_TotalUserNum(Int32 value)
ToString             Method     System.String ToString()
LastPlasmidID        Property   System.Int32 LastPlasmidID {get;set;}
TotalBackboneNum     Property   System.Int32 TotalBackboneNum {get;set;}
TotalEnzymeNum       Property   System.Int32 TotalEnzymeNum {get;set;}
TotalPlasmidNum      Property   System.Int32 TotalPlasmidNum {get;set;}
TotalUserNum         Property   System.Int32 TotalUserNum {get;set;}

Now, let’s try the GetDBStatus () method

[PlasmidView.SQLProvider]::GetDBStatus()

TotalPlasmidNum  : 21
TotalBackboneNum : 8
LastPlasmidID    : 20
TotalEnzymeNum   : 103
TotalUserNum     : 8

There two things I want to emphases:

1. The original design requires a username and password for database access.   I used Windows NT authentication for SQL server connection, and PlasmidView.SQLProvider class expose all SQL server operation as “static” methods, so the security check has been bypassed.  To regain access control over different user, I have to change C# code to check user credential every time before a SQL server operation.

2. There is really no big difference when using a .Net class in C# or monad. What you need to do is to load the DLL and create the object by new-object cmdlet. Because monad automatically convert “public” class properties to String, quoting a object under MSH prompt will get formatted list of object public properties.

Tags: msh monad

NCBI Blastn under MSH command line

2005-12-28T13:13:00.000-05:00

From NCBI web site:
The Basic Local Alignment Search Tool (BLAST) finds regions of local similarity between sequences. The program compares nucleotide or protein sequences to sequence databases and calculates the statistical significance of matches. BLAST can be used to infer functional and evolutionary relationships between sequences as well as help identify members of gene families.

Blastn is one of my most frequently used tools from NCBI. In stead of going to their web site, I can now run blastn under command line. It is convenient and even faster than web interfaces. Although I use "nr" database here, you can change the URL string to use other database or other blast program.

As for scripting, nothing magic here. Just rewrite original perl script using .NET class: System.Net.WebClient . For Regular Expression, we can use "-match" expression.
Thanks for Lee Holmes Blog

# Begin of script
# =============================================================
# This code is for test purposes only. Use it at your own risk.
# Please do not submit or retrieve more than one request every
# two seconds. Results will be kept at NCBI for 24 hours. For
# best batch performance,they recommend that you submit requests
# after 2000 EST (0100 GMT) and retrieve results before 0500 EST
# (1000 GMT).
# reference: http://www.ncbi.nlm.nih.gov/blast/Doc/urlapi.html
# reference: http://www.ncbi.nlm.nih.gov/blast/docs/web_blast.pl
# =============================================================
# return codes:
# 0 – success
# 1 - invalid arguments
# 2 - no hits found
# 3 - rid expired
# 4 - search failed
# 5 - unknown error
# =========================================================
param([string] $query)

if (-not $query)
{
"Please specify query seqence!"
return 1
}

#Submit query sequence
"================================================================="
"Query sequence:"
""
$query
""
"Submit query sequence..."
$uri="http://www.ncbi.nlm.nih.gov/blast/Blast.cgi?CMD=Put&PROGRAM=blastn&DATABASE=nr&QUERY=" + $query
$BlastClient = new-object System.Net.WebClient
$pagecontent = $BlastClient.DownloadString($uri);
"================================================================="
# Get RID
$pagecontent -match " RID = (.*)"
$RID=$Matches[1]
"RID=" + $RID
" "
$pagecontent -match " RTOE = (.*)"
$TimeToComplete= $Matches[1]
"Time to complete search: " + $TimeToComplete + " seconds or sooner. Waiting..."
" "
Start-sleep $TimeToComplete

#Waiting for results
While ($true)
{
Start-sleep 5
$uri= "http://www.ncbi.nlm.nih.gov/blast/Blast.cgi?CMD=Get&FORMAT_OBJECT=SearchInfo&RID=" + $RID

$pagecontent = $BlastClient.DownloadString($uri);

if ($pagecontent -match "Status=WAITING")
{
"Results not ready, waiting..."
"================================================================="
continue
}

if ($pagecontent -match "Status=FAILED")
{
"Search failed, exiting..."
return 2
}

if ($pagecontent -match "Status=UNKNOWN")
{
"Search expired, exiting..."
return 3
}

if ($pagecontent -match "Status=READY")
{
if ($pagecontent -match "ThereAreHits=yes")
{
"Search complete, retrieving results...";
"================================================================="
break
}
else
{
"No hits found.\n";
return 4
}
}
return 5
}

# Get results
$uri= "http://www.ncbi.nlm.nih.gov/blast/Blast.cgi?CMD=Get&RID=" + $RID + "&ALIGNMENTS=500&ALIGNMENT_VIEW=QueryAnchored&FORMATOBJECT=Alignment&FORMAT_TYPE=TEXT"
$pagecontent = $BlastClient.DownloadString($uri);
$pagecontent >.\blast.results
Get-content .\blast.results | more
return 0
#End of script

So if we do this under MSH prompt:
. blastn-nr.msh AAAAAAAAGGGGGGCCCCCTTTT

the output would like:

==============================================================
Query sequence: AAAAAAAAGGGGGGCCCCCTTTT
Submit query sequence...
RID=1135712055-18130-135523741431.BLASTQ1
Time to complete search: 12 seconds or sooner. Waiting...
Results not ready, waiting...
Search complete, retrieving results...
=============================================================

BLASTN 2.2.12 [Aug-07-2005]Reference: Altschul, Stephen F., Thomas L. Madden,
Alejandro A. Schffer,Jinghui Zhang, Zheng Zhang, Webb Miller, and David J. Lipman
(1997), "Gapped BLAST and PSI-BLAST: a new generation ofprotein database search
programs", Nucleic Acids Res. 25:3389-3402.

RID: 1135712055-18130-135523741431.BLASTQ1

Database: All GenBank+EMBL+DDBJ+PDB sequences (but no EST, STS,GSS,environmental
samples or phase 0, 1 or 2 HTGS sequences) 3,659,359 sequences; 16,326,075,880
total letters

Query= (23 letters)

Score ESequences producing significant alignments: (Bits) Valued
AK202294.1 Mus musculus cDNA, clone:Y1G0140P22, strand:m... 40.1 0.067gb
AF467007.1AF467007S1 Homo sapiens histidine N-methyltrans... 38.2 0.27
dbjAK192209.1 Mus musculus cDNA, clone:Y1G0108D05, strand:m... 38.2 0.27
embAJ333598.1HSA333598 Homo sapiens genomic sequence surrou... 38.2 0.27
embAJ343149.1HSA343149 Homo sapiens genomic sequence surrou... 38.2 0.27
embAJ343845.1HSA343845 Homo sapiens genomic sequence surrou... 38.2 0.27
gbAY811874.1 Schistosoma japonicum SJCHGC01957 protein mRNA, p 36.2 1.0
dbjAB232923.1 Oryzias latipes hox gene cluster, complete cds, 36.2 1.0
dbjBS000120.2 Pan troglodytes chromosome 22 clone:RP43-007D... 36.2 1.0
......

>gbAY811874.1 Schistosoma japonicum SJCHGC01957 protein mRNA, partial cds
Length=718
Score = 36.2 bits (18),
Expect = 1.0
Identities = 18/18 (100%),
Gaps = 0/18 (0%)
Strand=Plus/Minus
Query 2 AAAAAAAGGGGGGCCCCC 19
Sbjct 156 AAAAAAAGGGGGGCCCCC 139
......

Database: All GenBank+EMBL+DDBJ+PDB sequences (but no EST, STS,
GSS,environmentalsamples or phase 0, 1 or 2 HTGS sequences)
Posted date: Dec 26, 2005 4:16 AM
Number of letters in database: -853,793,300
Number of sequences in database: 3,659,359
Lambda K H 1.37 0.711 1.31
GappedLambda K H 1.37 0.711 1.31
Matrix: blastn matrix:1 -3
Gap Penalties:
Existence: 5,
Extension: 2
Number of Sequences: 3659359
Number of Hits to DB: 2403072
Number of extensions: 36138
Number of successful extensions: 36138
Number of sequences better than 10: 21
Number of HSP's better than 10 without gapping: 21
Number of HSP's gapped: 36138
Number of HSP's successfully gapped: 21
Number of extra gapped extensions for HSPs above 10: 36070
Length of query: 23
Length of database: 16326075880
Length adjustment: 18
Effective length of query: 5
Effective length of database: 16260207418
Effective search space: 81301037090
Effective search space used: 81301037090
A: 0X1: 11 (21.8 bits)
X2: 15 (29.7 bits)
X3: 25 (49.6 bits)
S1: 11 (22.3 bits)
S2: 17 (34.2 bits)
====================================================
Have fun!

[Edit: Monad has now been renamed to Windows PowerShell. This script or discussion may require slight adjustments before it applies directly to newer builds.]

Tags: msh monad PowerShell

Play with ACL in MSH

2005-12-19T23:26:00.000-05:00

I have a laptop running Windows XP home edition. As you known, to force you pay more money for professional edition, Mico$oft disabled the "Security" tab in file or directory "Properties" dialogue window. To set ACL, I have to use cacls.exe (come with windows). If you want more functionality, use setacl.exe (under GPL, you can download from sourceforge.org)

It is OK, if you just do adjustments for a few files. It becomes a tedious job, if you have lot of files or directories to modify. So let's give MSH a try.

There are two cmd-let designed for this job:
get-acl : Gets the access control list (ACL) associated with a file or object.
get-acl [[-Path] System.String[]] [[-Filter] System.String] [[-Include]
system.String[]] [[-Exclude] System.String[]] [[-Audit] [System.Boolean]]

set-acl: Set the security Access Control List for an item or items.
set-acl [-ACLObject] aclobject [-Path path]
[-Include include] [-Exclude exclude] [-Filter filter] [-Passthru]

But there is a trick here. In order to set group or set owner, you need an instance of [system.security.principal.ntaccount] object in hand.

#######################################
# Let's get acl for file text.txt
$acl=get-acl text.txt
$acl format-list

#You will get something like
#Path : FileSystem::D:\text.txt
#Owner : Computer\me
#Group : Computer\None
#Access : BUILTIN\Administrators Allow FullControl
# Computer\me Allow FullControl
#Audit :
#Sddl : Bla…Bla…Bla…

#So we can manipulate this acl object now. Let's try to change group to
# BUILTIN\Administrators.
#Get a [system.security.principal.ntaccount] object
$Account = new-object system.security.principal.ntaccount("Administrators ")

#To check whether the group is valid
$SID = $Account.translate([system.security.principal.securityidentifier])
$SID

#You will see
#BinaryLength AccountDomainSid Value
#------------ ---------------- -----
# 16 S-1-5-32-544
#If you see some error message here, you $Account is invalid.

#Use setgroup method of acl object
$acl.setgroup($Account)
$acl format-list

#You will get something like
#Path : FileSystem::D:\text.txt
#Owner : Computer\me
#Group : BUILTIN\Administrators (We made change here!!!!!!!!!!!!!!!)
#Access : BUILTIN\Administrators Allow FullControl
# Computer\me Allow FullControl
#Audit :
#Sddl : Bla…Bla…Bla…

#But this ACL object is in memory, we need to apply them to file
set-acl -aclobject $acl -path D:\text.txt

#make sure you have both -aclobject and -path, otherwise you will get some error.
###############################################

This scheme can be easily changed to modify directory acl or grant access to any user.
You can use get-member cmd-let to explore other methods or property of $acl. I will leave those excise to readers.

Reference
http://mow001.blogspot.com/2005/10/getting-and-using-securityprincipal.html

[Edit: Monad has now been renamed to Windows PowerShell. This script or discussion may require slight adjustments before it applies directly to newer builds.]

Tags: msh monad PowerShell

Find Amino Acid mutation

2005-12-16T23:11:00.000-05:00

Yesterday, we had a quiz on Mass Spectrometry. There is one question Find Amino Acid mutation:
There is a single amino acid mutation on SOD protein from ALS patient. After degradation by Lys-C proteinase and MALDI-TOF analysis, there is a molecular ion at 1689Da. But in wild type SOD protein, it should be 1765Da. That is to say the single amino acid mutation cause reduce of peptide molacular weight of 66Da. What exactly is that mutation.

In a another word: there are 20 amino acid,the difference of molecular weight is 66Da, what 2 amino acid is that. This is a simple combination. In the DOS world, I proberbly use QBasic. But now we can try MSH:

#Name of Amino acid
$AAName = @("Ala";"Arg";"Asn";"Asp";"Cys";"Glu";"Gln";"Gly";"His";"Ile";"Leu";
"Lys";"Met";"Phe";"Pro";"Ser";"Thr";"Trp";"Tyr";"Val")

#Molecular weight of amino acid
$AAMw = @(71.08;156.19;114.10;115.09;103.15;129.12;128.13;57.05;137.14;113.16;
113.16;128.17;131.20;147.18;97.12;87.08;101.11;186.21;163.18;99.13)

#Substract one verse another
for ($i=0;$i -lt 20;$i++) {
for ($j=$i; $j -lt 20;$j++) {
if ($AAMw[$i] -ge $AAMw[$j]) {
$change= $AAMw[$i] - $AAMw[$j]
}
else {
$change= $AAMw[$j] - $AAMw[$i]
}
if ($change -gt 66 -and $change -lt 67) {
if ($AAMw[$i] -ge $AAMw[$j]) {
$AAName[$i] + "(" + $AAMw[$i].Tostring() + ")" + "->" + $AAName[$j] + "("
+ $AAMw[$j].ToString() + ")" + ":" + $change.ToString()
}
else {
$AAName[$j] + "(" + $AAMw[$j].Tostring() + ")" + "->" + $AAName[$i] + "("
+ $AAMw[$i].ToString() + ")" + ":" + $change.ToString()
}

}
}
}

Results:
His(137.14)->Ala(71.08):66.06
Tyr(163.18)->Pro(97.12):66.06

Tags: msh monad

My First MSH Script

2005-12-09T23:02:00.000-05:00

#===========================================================
#This script will check colinux service, start a colinux console (NT).
#This script comes with NO warrenty! Use at your own risk!
#===========================================================

"Cheching Colinux Service...."
$CoService = get-service Where-object {$_.ServiceName -like "*colinux*"}
if (!$CoService) { throw "Colinux Service Not Installed ! "}
"Found Colinux Service..."
$CoService
if ( $coservice.status.ToString() -ne "Running"){
"Colinux Service is NOT running ! Try to start Colinux Service..."
$coservice.Start()
}
"Colinux Service Is Running ! Get Colinux-daemon Process..."
$CoProcess = get-process Where-object {$_.ProcessName -like "*colinux-daemon*"}
if (!$CoProcess){ throw "Colinux-daemon Is Not Running ! " }
"Colinux-daemon Is Running ... "
$CoProcess
"Try to start Colinux console..."
$CoConsoleProcess = get-process Where-object {$_.ProcessName -like "*colinux-console*"}
$CoPID = [string] $CoProcess.Id
if (!$CoConsoleProcess){
D:\colinux\colinux-console-nt -a $CoPID
}
else{
"Colinux-console (NT) already running !
"Quiting ..."
}

Tags: msh monad

Ready for MSH

2005-12-02T22:52:00.000-05:00

MSH, code name "Monad" , is next generation of microsoft command line tools.

Get MSH
http://www.microsoft.com/technet/scriptcenter/topics/msh/download.mspx

Information for monad
http://www.reskit.net/monad/

Monad team blog
http://blogs.msdn.com/monad/

Tags: msh monad