# Wednesday, March 10, 2010

This is unashamedly a post for developers, in particular those with an interest in functional languages. With the advent of PowerShell 2.0, some of you may have noticed that ScriptBlocks - which I suppose could also be called anonymous functions or lambdas - gained a new method: GetNewClosure. Closures are one of the essential tools for functional programming., something I’ve been trying to learn more about over the last few years. I don’t really have an opportunity to use it in work other than the hybrid trickery available in C# 3.0, but I have been tinkering a lot with PowerShell 2.0 to see if some of the tricks of the functional trade could be implemented. It’s just a shell language, but there are some nice features in there that enable a wide variety of funky stuff.

Partial Application

In a nutshell, partial application of a function is when you pass in only some of the parameters and get a function back that can accept the remaining parameters:

# define a simple function
function test {
    param($a, $b, $c);
    "a: $a; b: $b; c:$c"
}

# partially apply with -c parameter
$f = merge-parameter (gcm test) -c 5

# partially apply with -c and -a then execute with -b (papp is an alias)
& (papp (papp (gcm test) -c 3) -a 2) -b 7

# partially apply the get-command cmdlet with -commandtype
# and assign the result to a new function
si function:get-function (papp (gcm get-command) -commandtype function)

This is by no means a complete implementation of a partial application framework for powershell. The merge-parameter function (aliased to papp) currently only works with the default parameterset and does not mirror any of the parameteric attributes in the applied function or cmdlet. I'm not saying it couldn't do that, but this is purely a proof of concept. The module is listed below and is also available from PoshCode at http://poshcode.org/1687

# save as functional.psm1 and drop into your module path
Set-StrictMode -Version 2

$commonParameters = @("Verbose",
                      "Debug",
                      "ErrorAction",
                      "WarningAction",
                      "ErrorVariable",
                      "WarningVariable",
                      "OutVariable",
                      "OutBuffer")

<#
.SYNOPSIS
    Support function for partially-applied cmdlets and functions.
#>
function Get-ParameterDictionary {
    [outputtype([Management.Automation.RuntimeDefinedParameterDictionary])]
    [cmdletbinding()]
    param(
        [validatenotnull()]
        [management.automation.commandinfo]$CommandInfo,
        [validatenotnull()]
        [management.automation.pscmdlet]$PSCmdletContext = $PSCmdlet
    )
    
    # dictionary to hold dynamic parameters
    $rdpd = new-object Management.Automation.RuntimeDefinedParameterDictionary

    try {
        # grab parameters from function
        if ($CommandInfo.parametersets.count > 1) {
            $parameters = $CommandInfo.ParameterSets[[string]$CommandInfo.DefaultParameterSet].parameters
        } else {
            $parameters = $CommandInfo.parameters.getenumerator() | % {$CommandInfo.parameters[$_.key]}
        }        
                
        $parameters | % {
            
            write-verbose "testing $($_.name)"
                                    
            # skip common parameters        
            if ($commonParameters -like $_.Name) {                                  
                
                write-verbose "skipping common parameter $($_.name)"
                
            } else {
                
                $rdp = new-object management.automation.runtimedefinedparameter
                $rdp.Name = $_.Name
                $rdp.ParameterType = $_.ParameterType
                
                # tag new parameters to match this function's parameterset
                $pa = new-object system.management.automation.parameterattribute
                $pa.ParameterSetName = $PSCmdletContext.ParameterSetName
                $rdp.Attributes.Add($pa)
                
                $rdpd.add($_.Name, $rdp)
            }
            
        }
    } catch {
    
        Write-Warning "Error getting parameter dictionary: $_"
    }
    
    # return
    $rdpd
}

<#
.SYNOPSIS
    Function that accepts a FunctionInfo or CmdletInfo reference and one or more parameters
    and returns a FunctionInfo bound to those parameter(s) and their value(s.)
.DESCRIPTION
    Function that accepts a FunctionInfo or CmdletInfo reference and one or more parameters
    and returns a FunctionInfo bound to those parameter(s) and their value(s.)
    
    Any parameters "merged" into the function are removed from the available parameters for
    future invocations. Multiple chained merge-parameter calls are permitted.
.EXAMPLE

    First, we define a simple function:
    
    function test {
        param($a, $b, $c, $d);
        "a: $a; b: $b; c:$c; d:$d"
    }
    
    Now we merge -b parameter into functioninfo with the static value of 5, returning a new
    functioninfo:
    
    ps> $x = merge-parameter (gcm test) -b 5
    
    We execute the new functioninfo with the & (call) operator, passing in the remaining 
    arguments:
    
    ps> & $x -a 2 -c 4 -d 9
    a: 2; b: 5; c: 4; d: 9
    
    Now we merge two new parameters in, -c with the value 3 and -d with 5:
    
    ps> $y = merge-parameter $x -c 3 -d 5
    
    Again we call $y with the remaining named parameter -a:
    
    ps> & $y -a 2
    a: 2; b: 5; c: 3; d: 5
.EXAMPLE

    Cmdlets can also be subject to partial application. In this case we create a new
    function with the returned functioninfo:
    
    ps> si function:get-function (merge-parameter (gcm get-command) -commandtype function)
    ps> get-function
                
.PARAMETER _CommandInfo
    The FunctionInfo or CmdletInfo into which to merge (apply) parameter(s.)
    
    The parameter is named with a leading underscore character to prevent parameter
    collisions when exposing the targetted command's parameters and dynamic parameters.
.INPUTS
    FunctionInfo or CmdletInfo
.OUTPUTS
    FunctionInfo
#>
function Merge-Parameter {    
    [OutputType([Management.Automation.FunctionInfo])]
    [CmdletBinding()]
    param(
        [parameter(position=0, mandatory=$true)]
        [validatenotnull()]
        [validatescript({
            ($_ -is [management.automation.functioninfo]) -or `
            ($_ -is [management.automation.cmdletinfo])
        })]
        [management.automation.commandinfo]$_Command
    )
    
    dynamicparam {
        # strict mode compatible check for parameter
        if ((test-path variable:_command)) {
            # attach input functioninfo's parameters to self
            Get-ParameterDictionary $_Command $PSCmdlet
        }
    }

    begin {
        write-verbose "merge-parameter: begin"
        
        # copy our bound parameters, except common ones              
        $mergedParameters = new-object 'collections.generic.dictionary[string,object]' $PSBoundParameters
        
        # remove our parameters, leaving only target function/CommandInfo's args to curry in
        $mergedParameters.remove("_Command") > $null
        
        # remove common parameters
        $commonParameters | % {
            if ($mergedParameters.ContainsKey($_)) {
                $mergedParameters.Remove($_)  > $null
            }
        }
    }
    
    process {
        write-verbose "merge-parameter: process"
        
        # temporary function name
        $temp = [guid]::NewGuid()

        $target = $_Command

        # splat our fixed named parameter(s) and then splat remaining args
        $partial = {
            [cmdletbinding()]
            param()
            
            # begin dynamicparam
            dynamicparam
            {                
                $targetRdpd = Get-ParameterDictionary $target $PSCmdlet
        
                # remove fixed parameters
                $mergedParameters.keys | % {
                    $targetRdpd.remove($_) > $null
                }
                $targetRdpd
            }
            begin {
                write-verbose "i have $($mergedParameters.count) fixed parameter(s)."
                write-verbose "i have $($targetrdpd.count) remaining parameter(s)"
            }
            # end dynamicparam
            process {
                $boundParameters = $PSCmdlet.MyInvocation.BoundParameters
                
                # remove common parameters (verbose, whatif etc)
                $commonParameters | % {
                    if ($boundParameters.ContainsKey($_)) {
                        $boundParameters.Remove($_)  > $null
                    }
                }
                
                # invoke command with fixed parameters and passed parameters (all named)
                . $target @mergedParameters @boundParameters
                
                if ($args) {
                    write-warning "received $($args.count) arg(s) not part of function."
                }
            }
        }
        
        # emit function/CommandInfo
        new-item -Path function:$temp -Value $partial.GetNewClosure()
    }
    
    end {
        # cleanup
        rm function:$temp
    }    
}

new-alias papp Merge-Parameter -force

Export-ModuleMember -Alias papp -Function Merge-Parameter, Get-ParameterDictionary

Have fun[ctional]!

posted on Wednesday, March 10, 2010 8:01:07 PM (Eastern Standard Time, UTC-05:00)  #    Comments [1] Trackback
# Friday, August 14, 2009

update: 2009-09-27 - removed usage of closures - turns out i hadn't tested that modification properly. oops.

Here’s a quick one – in order to sway excessive BASH jealously, I knocked this up to persist the last 100 history items between PowerShell sessions. It hooks the special engine (as opposed to object) event “poweshell.exiting” and runs a script to save history to an XML file using the universally useful Export-CliXML cmdlet. Another trick in there is to use a closure to capture the value of the $historyPath variable. I need to do this because powershell event handlers use their own runspace (and will lose the values of the variables in the current runspace). I also could have passed the value via the –MessageData parameter and done it that way, but I figured I’m already in v2 territory so lets use that feature ;-)

# save last 100 history items on exit
$historyPath = Join-Path (split-path $profile) history.clixml

# hook powershell's exiting event & hide the registration with -supportevent.
Register-EngineEvent -SourceIdentifier powershell.exiting -SupportEvent -Action {
    Get-History -Count 100 | Export-Clixml (Join-Path (split-path $profile) history.clixml) }

# load previous history, if it exists
if ((Test-Path $historyPath)) {
    Import-Clixml $historyPath | ? {$count++;$true} | Add-History
    Write-Host -Fore Green "`nLoaded $count history item(s).`n"
}

Dump this into your profile and have fun!

posted on Friday, August 14, 2009 9:28:58 PM (Eastern Daylight Time, UTC-04:00)  #    Comments [0] Trackback
# Saturday, March 28, 2009

After some hints from Ibrahim on the Microsoft PowerShell team, I realised it was possible to rewrite the dynamic module body generation from the Get-Interface function in my last post without using string literals and nested here-string tricks. I was able to increase the brevity by using a new [to PowerShell] v2 feature called “Closures.” Ibrahim talks a bit about the technique over on the official PowerShell Blog. It’s a common feature in so-called Functional languages and there are plenty of other tidbits to read about it, both academic and practical if you search online.

A closure in PowerShell, in short, lets you take a snapshot of a ScriptBlock by calling its GetNewClosure() method. The snapshot is taken of its variables and will let you pass it around so it is no longer directly tied to the particular scope chain it was declared in. As some are fond of saying though, Thar Be Dragons. Although closing around a ScriptBlock will make copies of the PSVariables that are within, it is still subject to the whims of .NET – particularly ByRef and ByValue semantics. If a ScriptBlock contains a PSVariable pointing to an integer, it’s safe to say that integer will be copied and is frozen as that value as long as you remain within the domain of the current SessionState context (more about this later).

.NET intervenes: ByRef and ByVal

Closing around a ScriptBlock that contains PSVariables that point to reference types, however - like Arrays for example – will only duplicate the reference, not the instance itself. If elements in the array are changed via the PSVariable from the originating scope, the corresponding array element in the closure will also change. After all – you got a copy of the pointer, not the value itself. This is the very essence of ByRef and ByVal.

Let’s take a look at the revised Get-Interface script:

function Get-Interface {

#.Synopsis
#   Allows PowerShell to call specific interface implementations on any .NET object. 
#.Description
#   Allows PowerShell to call specific interface implementations on any .NET object. 
#
#   As of v2.0, PowerShell cannot cast .NET instances to a particular interface. This makes it
#   impossible (shy of reflection) to call methods on explicitly implemented interfaces.   
#.Parameter Object
#   An instance of a .NET class from which you want to get a reference to a particular interface it defines.
#.Parameter InterfaceType
#   An interface type, e.g. [idisposable], implemented by the target object.
#.Example
#   // a class with explicitly implemented interface   
#   public class MyObj : IDisposable {
#      void IDisposable.Dispose()
#   }
#   
#   ps> $o = new-object MyObj
#   ps> $i = get-interface $o ([idisposable])
#   ps> $i.Dispose()      
#.ReturnValue
#   A PSCustomObject with ScriptMethods and ScriptProperties representing methods and properties on the target interface.
#.Notes
# AUTHOR:    Oisin Grehan http://www.nivot.org/
# LASTEDIT:  2009-03-28 18:37:23
# REVISION:  0.2

    [CmdletBinding()]
    param(
        [ValidateNotNull()]
        $Object,
        
        [ValidateScript( { $_.IsInterface } )]
        [type]$InterfaceType
    )

    $script:t  = $Object.GetType()
    
    try {
        
        $script:m  = $t.GetInterfaceMap($InterfaceType)

    } catch [argumentexception] {
        
        throw "Interface $($InterfaceType.Name) not found on ${t}!"
    }

    $script:im = $m.InterfaceMethods
    $script:tm = $m.TargetMethods
    
    # TODO: use param blocks in functions instead of $args
    #       so method signatures are visible via get-member
    
    $body = {
         param($o, $i) 
         
         $script:t  = $o.GetType()
         $script:m  = $t.GetInterfaceMap($i)
         $script:im = $m.InterfaceMethods
         $script:tm = $m.TargetMethods
                  
         for ($ix = 0; $ix -lt $im.Count; $ix++) {
            
            $mb = $im[$ix]

            # for the function body, we close over $ix to capture the index
            # so even on the next iteration of this loop, the $ix value will
            # be frozen within the function's scriptblock body
            set-item -path function:script:$($mb.Name) -value {

                # call corresponding target method
                $tm[$ix].Invoke($o, $args)

            }.GetNewClosure() -verbose -force

            if (!$mb.IsSpecialName) {
                # only export the function if it is not a getter or setter.
                Export-ModuleMember $mb.Name -verbose
            }
         }
    }

    write-verbose $body.tostring()    
    
    # create dynamic module
    $module = new-module -ScriptBlock $body -Args $Object, $InterfaceType -Verbose
    
    # generate method proxies - all exported members become scriptmethods
    # however, we are careful not to export getters and setters.
    $custom = $module.AsCustomObject()
    
    # add property proxies - need to use scriptproperties here.
    # modules cannot expose true properties, only variables and 
    # we cannot intercept variables get/set.
    
    $InterfaceType.GetProperties() | % {

        $propName = $_.Name
        $getter   = $null
        $setter   = $null
       
        if ($_.CanRead) {
            
            # where is the getter methodinfo on the interface map?
            $ix = [array]::indexof($im, $_.GetGetMethod())
            
            # bind the getter scriptblock to our module's scope
            # and generate script to call target method
            #
            # NOTE: we cannot use a closure here because sessionstate
            #       is rebound to the module's, and $ix would be lost
            $getter = $module.NewBoundScriptBlock(
                [scriptblock]::create("`$tm[{0}].Invoke(`$o, @())" -f $ix))
        }
        
        if ($_.CanWrite) {
            
            # where is the setter methodinfo on the interface map?
            $ix = [array]::indexof($im, $_.GetSetMethod())
            
            # bind the setter scriptblock to our module's scope
            # and generate script to call target method
            #
            # NOTE: we cannot use a closure here because sessionstate
            #       is rebound to the module's, and $ix would be lost
            $setter = $module.NewBoundScriptBlock(
                [scriptblock]::create(
                    "param(`$value); `$tm[{0}].Invoke(`$o, `$value)" -f $ix))
        }
        
        # add our property to the pscustomobject
        $prop = new-object management.automation.psscriptproperty $propName, $getter, $setter
        $custom.psobject.properties.add($prop)
    }
    
    # insert the interface name at the head of the typename chain (for get-member info)
    $custom.psobject.TypeNames.Insert(0, $InterfaceType.FullName)
    
    # dump our pscustomobject to pipeline
    $custom
}

The changes are around line 56, where I now declare $body with a script syntax, no longer using [scriptblock]::create. Where previously I used a subexpression inside a here-string to general literal function declarations, I now use a “for” loop and set-item to create the functions (thanks Ibrahim!). You can see that I am calling GetNewClosure() on the function body each time I create a new function. This “captures” the value of the $ix index variable, so that the value will not change on subsequent loops. If I did not use closures, when the for loop ended, all of the functions would be using the same terminating value of the loop for $ix, which would be $im.count + 1. Doh.

Module SessionState Context, Wha?

So if I ripped out that literal string parsing stuff for the module $body, why am I still using it for the property getter/setter generation? (lines 115 / 130)  Well, I have no choice: Creating a new module creates an entirely new SessionState; in simple terms, session state is essentially a giant Hashtable in which PSVariables are stored. If I closed over those get/set ScriptBlocks containing the $ix PSVariable, when it is bound to the module it will use the new module’s SessionState – which is empty! The value of $ix would be undefined. The other variables in those strings are escaped, because they will be evaluated in the module’s context later, when the getters and setters are invoked. Those variables are seeded by the module’s initialization done at the new-module call at line 88.

It’s all very succinct, but rest assured, very powerful.

Have fun!

NOTE: I realise that some more refactoring could probably eliminate this script parsing, but the value is in understanding how modules may affect usage of closures, and the problems that may occur.

posted on Saturday, March 28, 2009 6:23:11 PM (Eastern Standard Time, UTC-05:00)  #    Comments [0] Trackback