What’s new in PowerShell 5.0 Preview for developers?

There’s some great information about the improvements to DSC, the new OneGet package manager and the like over on the Windows Server blog, but I am not going to repeat that here. As with most of these community technical previews (CTPs), they tend to only highlight the things they want the rest of us to poke at and assess, or the things they see as the most impactful. As they don’t really have the cycles to document the less impactful stuff, there are sometimes some hidden gems in there, so, with a decompiler, coffee and some custom tools to compare assemblies, let’s see what we can find. This analysis is based on examining System.Management.Automation only, comparing it to the 4.0 RTM version.

Debugging Jobs

Now, one of the more recent things to be added in v4 was remote debugging but there hasn’t been much said about it. Regardless, debugging remote jobs wasn’t a scenario it handled very well. That’s no longer an issue, it seems:

image

I did notice some quirky behaviour though. My test script writes to the output stream and when I exit the debugger with “exit,” I am still attached to the remote job and the output will continue to stream to the screen until I hit ctrl+c. It’s only then I am returned to the calling scope. Also, when you first attach, you can see that it will consume all of the output stream before dumping you at the interactive debugger.

Debugging Custom Jobs

Of course if you were paying attention, you’ll remember that v3 introduced the notion of custom jobs, by way of the JobSourceAdapter framework. Previously, the only kind of jobs we had were for handling events and for running scripts like the example above. The adapter extension point lets anyone use an external system as a source of “jobs” and optionally allow control of these via the standard job cmdlets, Start/Stop/Suspend/Resume/Remove-Job. Now, we can expose our own custom remote debuggers for our custom jobs:

    public interface IJobDebugger {
        bool IsAsync { get; set; }
        Debugger Debugger { get; }
    }

This is possible because the Debugger class became abstract in v3 also.  Derive from this to facilitate your own remote custom job debugging.

AST Serialization

The other thing I noticed was that the entire AST (abstract syntax tree) graph lost the ability to be serialized: The SerialiableAttribute was removed. IIRC, there are some subtle bugs that can happen when you round-trip an AST via the serializer, bugs that were not noticed in v4 when it was enabled. I can’t for the life of me remember what they were, but if I find my notes, I’ll update this post. This ability was used internally to clone an AST graph quickly, but now that the graph has been rendered non-serializable, all AST instances gained a Copy() method instead.

Updated Type Accelerator functions for PowerShell 3.0 and 4.0

I noticed that my old type accelerator functions don’t work anymore in later versions of powershell due to some internal changes for caching. This sometimes happens when you tinker when innards like that.

Here's the source, but it's also up on poshcode.org.

#requires -version 3

function New-TypeAlias {
<#
    .synopsis
        Add a new type alias (accelerator)
    .description
        Add a new type accelerator to the built-in list of accelerators
        like [int], [runspace], [psobject] etc.
    .parameter Type
        The target type to alias (can be an attribute type.)
    .parameter Name
        The alias to use for the target type.
    .inputs
        None. This command does not accept pipeline input.
    .outputs
        None.

#>
    param(
        [parameter(mandatory, position=0)]
        [validatenotnull()]
        [type]$Type,

        [parameter(mandatory, position=1)]
        [validatenotnullorempty()]
        [string]$Name
    )

    $accel = [psobject].Assembly.GetType("System.Management.Automation.TypeAccelerators")
    $accel::add($Name, $Type)

    # reset cache
    $accel.getfield("allTypeAccelerators", [reflection.bindingflags]"nonpublic,static").setvalue($accel, $null)
}

function Get-TypeAlias {
<#
    .synopsis
        Gets all or a matching subset of type aliases (accelerators)
    .description
        Gets all or a matching subset of user-defined and built-in type
        aliases (accelerators).

        You may use wildcards to match the alias name(s).
    .parameter Name
        The alias name to find (optional, may contain wildcards.)
    .inputs
        None. This command does not accept pipeline input.
    .outputs
        Zero or more dictionary entries representing a type alias & type literal pairing.
#>
    param(
        [parameter(position=0)]
        [validatenotnullorempty()]
        [string]$Name = "*"
    )
    
    $dict = [psobject].Assembly.GetType("System.Management.Automation.TypeAccelerators")::get
    $matches = $dict.Keys | ? { $_ -ilike $Name }

    if ($matches) {
        $dict.GetEnumerator() | where key -in $matches
    } elseif (
        -not [System.Management.Automation.WildcardPattern]::ContainsWildcardCharacters($Name))
    {
        write-error -Message "No exact match found for alias '$Name'"
    }
}

Manning: PowerShell Deep Dives on Deal of the Day

I had the honour to be involved doing a little bit of editing work on the developer section for Jeffrey Hicks’ latest venture on Manning Books: PowerShell Deep Dives. Today it’s their Deal of the day, so use the following code to get a discount. All profits go to Save the Children!

Code: dotd0725tw

Here’s an abstract:

PowerShell Deep Dives is a trove of essential techniques and practical guidance. It is rich with insights from experts who won them through years of experience. The book's 28 chapters, grouped in four parts (Administration, Scripting, Development, and Platforms), were hand-picked by four section editors: Jeffery Hicks, Richard Siddaway, Oisín Grehan, and Aleksandar Nikolić. Whether you're just getting started with PowerShell or you already use it daily, you'll find yourself returning to this book over and over.

PowerShell–Bug Fixing and Assembly Patching for Guts and Glory

I’ve got a little break from the grind at the moment in work, being on what they call “The Bench.” This is a hockey reference of course (obvious to all fellow Canadians, I’d assume); the relative calm before the storm of the next project. Taking a break from my usual activities, I decided to take the time to look around for something painful, pointless and fun to do with PowerShell. Hah, who am I kidding. This is my favourite thing to do. Anyway, there was a bit of discussion on our private MVP mailing list concerning some weird behaviour of one of the new cmdlets in PowerShell 3.0: Invoke-RestMethod. It turns out it has a bug. Well, it appears to have about a dozen right now, but I chose this one to look at since it was the one we were discussing.

The Invoke-RestMethod Bug

After I’d done the work for this blog, I went to log the bug but someone had already done it. The "invoke-restmethod-is-truncating-results" bug was logged by Trevor Sullivan, a well known PowerShell enthusiast within our community. He noticed that when it’s used to grab an RSS or ATOM feed, it seems to give you back only about half of the items you want, and it’s not the first half either. I fired up Reflector (I swear if program icons could be worn down, this one would be a shiny nub at this point) and quickly isolated the method in question. I know this cmdlet name has a generic sounding noun indicating that it is used primarily for REST calls, but it actually gives special treatment to data that appears to be a syndication feed. This is presumed to cater to the most common use cases, according to Microsoft. The reconstituted method that does the work looks like this:

private bool TryProcessFeedStream(
    BufferingStreamReader responseStream)
{
    bool flag = false;
    try
    {
        XmlReaderSettings secureXmlReaderSettings =
            this.GetSecureXmlReaderSettings();

        XmlReader reader = XmlReader.Create(responseStream,
            secureXmlReaderSettings);

        for (int i = 0; (i < 10) && reader.Read(); i++)
        {
            if (string.Equals("rss", reader.Name,
                    StringComparison.OrdinalIgnoreCase) ||
                string.Equals("feed", reader.Name,
                    StringComparison.OrdinalIgnoreCase))
            {
                flag = true;
                break;
            }
        }
        if (flag)
        {
            XmlDocument document = new XmlDocument();
            while (reader.Read())
            {
                if ((reader.NodeType == XmlNodeType.Element) &&
                   (string.Equals("Item", reader.Name,
                       StringComparison.OrdinalIgnoreCase) ||
                    string.Equals("Entry", reader.Name,
                       StringComparison.OrdinalIgnoreCase)))
                {
                    XmlNode sendToPipeline =
                        document.ReadNode(reader);

                    base.WriteObject(sendToPipeline);
                }
            }
        }
        return flag;
    }
    catch (XmlException)
    {
    }
    finally
    {
        responseStream.Seek(0L, SeekOrigin.Begin);
    }
    return flag;
}

Analysis

The method itself looks pretty simple. It scans through the first ten nodes looking for a "feed” or “rss” element. If it finds this, it creates a new empty XML document instance and scans the rest of the input looking for “entry” or “item” elements, the meat of an ATOM or RSS feed respectively. If it finds one, it clones it into the holding document, and loops around the while condition, which advances the reader, one node at a time until it returns false, terminating the loop.

I hadn’t read Trevor’s report at this time, so the talk on the list said that the items coming appeared to be pretty random. Now, faced with the simplicity of the processing code, I figured it was just skipping nodes. I suspected a double-read call going on somewhere and the document.ReadNode line was my primary suspect; a quick look in Reflector at the ReadNode implementation confirmed this. In order to validate the bug and the subsequent fix, I decided to mock up the method in PowerShell script so I could repro the behavior:

function bugbug {
    # returns last TEN items from the powershell team blog
    $req = [System.Net.WebRequest]::Create(
        "http://blogs.msdn.com/powershell/rss.aspx")
    $res = $req.GetResponse()
    $stream = $res.GetResponseStream()

    $flag = $false

    $xmlsettings = [System.Xml.XmlReaderSettings] @{
        CheckCharacters = $false;
        CloseInput = $false;
        IgnoreProcessingInstructions = $true;
        MaxCharactersFromEntities = 0x400L;
        DtdProcessing = "Ignore"
    }

    $reader = [System.Xml.XmlReader]::Create($stream, $xmlsettings) 

    for ($i = 0; ($i -lt 10) -and $reader.Read(); $i++) {
        if (@("rss", "feed") -icontains $reader.Name) {
            $flag = $true
            break;
        }
    }

    if ($flag) {
        $doc = new-object System.Xml.XmlDocument
        while ($reader.Read()) {
            if (($reader.NodeType -eq "Element") -and
                (@("item", "entry") -icontains $reader.Name)) {
                $node = $doc.ReadNode($reader)
            }
        }
    }

    if ($res) { $res.Dispose() }
}

The Fix

So, knowing now that the ReadNode method will advance the reader one node as a side effect (technically, it will not advance the reader if the context node is an attribute, but that’s not our case here,) the problem is clear. We should loop until the reader is at the end, and when processing a node we should either call ReadNode to clone it, or call Read on the reader to try the next node.  With this in mind, the fixed while loop – modeled in script – looks like this:

# loop until EOF
while ($reader.ReadState -ne "EndOfFile") {
    if (($reader.NodeType -eq "Element") -and `
        (@("item", "entry") -icontains $reader.Name)) {
        # copy node
        $node = $doc.ReadNode($reader)
    } else {
        # next
        [void]$reader.Read()
    }
}

Ok, great. We’ve figured out what the code should have been, or at least one apparently working variant of it. So given that I prequalified this adventure with my masochistic intentions, let’s set about fixing the bug.

Reflexil

Many of you will be familiar with Lutz Roeder’s excellent Reflector utility, which allows anyone to glance inside any .NET assemblies to discover how they are implemented. Indeed, this is how I diagnosed the bug as explained above. One  of the cool things about Reflector is that it allows third party extensions that can hook into the engine to provide additional functionality. One of these such extensions is the most excellent Reflexil, which leverages JB Evain’s Mono.Cecil library to allow “editing” of assemblies.

image

Now, when I say “editing,” I am probably making it sound a lot less fiddly than it really is for most cases. While Reflexil will let you compile inline C# to edit or augment an assembly, the inline source must essentially be “stand alone” in that it cannot reference things beyond its scope or visibility. For example, you can’t make reference to private or protected members of the Type you are augmenting or editing. So, what I ended up doing was writing a partial implementation of the method I wanted to patch and then using Reflector to display the IL (intermediate language.) I then dumped the IL of the faulty method, and pasted both streams into an Excel spreadsheet so I could figure out what parts of the method I had to edit in Reflexil. My first attempt had me absent mindedly compile up my replacement method as Debug build, and the IL ended up being so wildly different that it would have taken me much longer than necessary to patch the assembly. Oops!

image

A bit more tinkering with compile options and targeting the Release configuration yielded me with a corrected IL dump that only differed by about fifteen instructions.

Final Steps

So after finally saving the modified assembly, Microsoft.PowerShell.Commands.Utility.Patched.dll, to disk, I am ready to load it into PowerShell. Now, if you’re one of the people who is still awake by this point, you’re probably thinking: “Well, that ain’t gonna work. You broke the strong name signature when you edited the assembly.” Indeed I did, but by using the SDK’s SN.EXE strong name tool, I disabled strong name verification for this assembly. This allows me to load the assembly and the CLR with skip the verification steps for the strong name, ultimately letting me replace the broken Invoke-Restmethod with a working one. Et Voila: Invoke-RestMethod is returning all items from the RSS feed, like it should do:

image

As an aside, another approach would have been to edit the assembly in the GAC in-situ, as the CLR will not verify strong named assemblies that are already in the GAC. Verification only happens at install time; this is an optimization for the loader that was added some time ago.

So, yeah, I could have just written a function with the same name to replace the Cmdlet, letting command precedence do the hard work, but that’s a bit boring, right?

Have fun!

Signing unsigned assemblies in NuGet packages

I wrote a set of PowerShell cmdlets for signing unsigned assemblies with the SNK of your choice. Certain platforms (I'm looking at you, SharePoint) just work better with SN assemblies, but this constrains dependencies to NuGet packages that contain only signed assemblies.

Many people believe using strong names is more of a hindrance than a help (myself included,) but corporate policies often dictate their usage. My strong naming package makes it simple to sign assemblies, even without the source code. No more begging package authors to release signed packages, or fiddling with github repos.

If you sign 3rd party NuGet package assemblies, I suggest signing them in-situ, in their original locations, so you can continue to reference the package and so not risk missing out on updates. When you update a package, simply resign the assemblies and compile away.

To get started, open the NuGet PM console and type:

Install-Package Nivot.StrongNaming

You must have a solution open at this time. This package is not tied to any project; it just adds new commands to the PM console (documented below.)

Nivot.StrongNaming

  • v1.0.0 [2013/04/29]
    • Initial release.
  • v1.0.1 [2013/04/29]
    • Updated metadata.
  • v1.0.2 [2013/04/30]
    • Added license and project URL.
    • Added readme.MD

About

A set of PowerShell Cmdlets to facilitate signing of unsigned 3rd party assemblies with a key of your choice, to allow them to be referenced by strongly named projects.

A NuGet package is available at: https://nuget.org/packages/Nivot.StrongNaming

Syntax

All cmdlets accept pipeline input. The AssemblyFile parameter is aliased to PSPath, so it will bind to piped files.

  • Test-StrongName [-AssemblyFile] <string[]> [<CommonParameters>]

    Returns true if an assembly has a strong name.

  • Import-StrongNameKeyPair [-KeyFile] <string> [<CommonParameters>]

  • Import-StrongNameKeyPair [-KeyFile] <string> -Password <securestring> [<CommonParameters>]

    Imports a simple unprotected SNK or a password-protected PFX, returning a StrongNameKeyPair instance for consumption by Set-StrongName. If your PFX file has a blank password, you must provide a SecureString of the empty string "". SecureString instances are returned from the Read-Host cmdlet with the -AsSecureString parameter.

  • Set-StrongName [-AssemblyFile] <string[]> -KeyPair <StrongNameKeyPair> [-NoBackup] [-Passthru] [-Force] [-DelaySign] [-WhatIf] [-Confirm] [<CommonParameters>]

    Assigns a strong name identity to an assembly.

    The -KeyPair parameter accepts a System.Reflection.StrongNameKeyPair output from the Import-StrongNameKeyPair cmdlet., which accepts either simple unprotected SNK files or password-protected PFX files.

    The -NoBackup switch directs the cmdlet to skip creating a .bak file alongside the newly signed assembly.

    The -Passthru switch will output a FileInfo representing the newly signed assembly to the pipeline.

    The -DelaySign switch will create a delay-signed assembly from a public key only SNK (it can also create one if the SNK contains both private and public keys.) This is useful if you can't get access to the full private key at your company. This will allow you to compile against previously unsigned nuget packages at least.

    The -Force switch will allow you to overwrite an existing strong name on an assembly.

    NOTE: You may supply -WhatIf to see what would be done, without actually doing it.

  • Get-AssemblyName [-AssemblyFile] <string[]> [<CommonParameters>]

    Returns a System.Reflection.AssemblyName instance from any assembly file.

FAQ: How Do I?

Get the default package root folder

PM> $root = join-path (split-path $dte.solution.filename) packages

Load an unprotected snk

PM> $key = Import-StrongNameKeyPair -KeyFile .\folder\key.snk
PM> dir *.dll | Set-StrongName -KeyPair $key -Verbose

Load a password-protected PFX

PM> $key = Import-StrongNameKeyPair -KeyFile .\folder\key.pfx -Password (Read-Host -AsSecureString)
******

Sign some unsigned assemblies

PM> cd (join-path $root unsignedPackage)
PM> dir -rec *.dll | set-strongname -keypair $key -verbose

(Re)sign some assemblies forcefully

PM> dir -rec *.dll | set-strongname -keypair $key -force

Sign only unsigned assemblies

PM> dir -rec *.dll | where { -not (test-strongname $_) } | set-strongname -keypair $key -verbose

PowerShell– A Peek at the Poke Module

I wrote this a good long time ago now, but somehow I never bothered to blog it. I guess I thought it was a bit too specific to be of general interest to people, but my good friend and long time MVP Karl Prosser tells me otherwise. This is the Wiki page from bitbucket repository, the link for which you'll find at the foot of the page. Basically, it's a module that lets you "peek" at objects to view and manipulate their internals. That is to say, you can access non-public methods, fields and properties just like they were public. You can also create instances of non-public types easily. I originally wrote this to help me when debugging things and to explore PowerShell itself from the inside out, interactively. It turns out to be a pretty powerful tool. I hope you enjoy it.


Here you'll find examples of how to peek and poke objects using the Poke module.

Version History

  • 1.0.1 - Compatibility fixes for v3 beta / .net 4.5
  • 1.0 - Initial release
Examples
# peek at a Job instance using pipeline syntax
$job = start-job { 42 } | peek
$job | get-member

which results in the new extended output format for get-member:

   TypeName:
Pokeable.System.Management.Automation.PSRemotingJob#676f9716-c167-47c6-ab0d-4d8cedbbe44d

Name                            Modifier  MemberType Definition
----                            --------  ---------- ----------
Equals                          public    Method     bool Equals(System.Object obj)
GetHashCode                     public    Method     int GetHashCode()
GetType                         public    Method     type GetType()
CheckDisconnectedAndUpdateState private   Method*    void CheckDisconnectedAndUpdateState(System....
CommonInit                      private   Method*    void CommonInit(int throttleLimit, System.Co...
ConnectJob                      internal  Method*    void ConnectJob(guid runspaceInstanceId)
ConnectJobs                     internal  Method*    void ConnectJobs()
ConstructLocation               private   Method*    string ConstructLocation()
Dispose                         protected Method*    void Dispose(bool disposing)
FindDisconnectedChildJob        private   Method*    System.Management.Automation.PSRemotingChild...
GetAssociatedPowerShellObject   internal  Method*    powershell GetAssociatedPowerShellObject(gui...
GetJobsForComputer              internal  Method*    System.Collections.Generic.List[System.Manag...
GetJobsForOperation             internal  Method*    System.Collections.Generic.List[System.Manag...
GetJobsForRunspace              internal  Method*    System.Collections.Generic.List[System.Manag...
GetRunspaces                    internal  Method*    System.Collections.Generic.IEnumerable`1[[Sy...
HandleChildJobStateChanged      private   Method*    void HandleChildJobStateChanged(System.Objec...
HandleJobUnblocked              private   Method*    void HandleJobUnblocked(System.Object sender...
InternalStopJob                 internal  Method*    void InternalStopJob()
SetStatusMessage                private   Method*    void SetStatusMessage()
StopJob                         public    Method*    void StopJob()
SubmitAndWaitForConnect         private   Method*    void SubmitAndWaitForConnect(System.Collecti...
ToString                        public    Method*    string ToString()
__GetBaseObject                 -         Method*    System.Management.Automation.PSRemotingJob, ...
__GetModuleInfo                 -         Method*    psmoduleinfo __GetModuleInfo()
atleastOneChildJobFailed        private   Field*     bool atleastOneChildJobFailed
blockedChildJobsCount           private   Field*     int blockedChildJobsCount
CanDisconnect                   internal  Property*  bool CanDisconnect { get; set; }
disconnectedChildJobsCount      private   Field*     int disconnectedChildJobsCount
finishedChildJobsCount          private   Field*     int finishedChildJobsCount
HasMoreData                     public    Property*  bool HasMoreData { get; set; }
HideComputerName                internal  Property*  bool HideComputerName { get; set; }
isDisposed                      private   Field*     bool isDisposed
Location                        public    Property*  string Location { get; set; }
moreData                        private   Field*     bool moreData
StatusMessage                   public    Property*  string StatusMessage { get; set; }
throttleManager                 private   Field*     System.Management.Automation.Remoting.Thrott...
_stopIsCalled                   private   Field*     bool _stopIsCalled
_syncObject                     private   Field*     System.Object _syncObject

You can call methods, set fields and properties (if they have setters - it doesn't matter if they're private, protected or internal.)

You can proxy/peek Types as well as instances:

# proxy a public type by piping it
$type = [text.stringbuilder] | peek
   TypeName: Pokeable.System.RuntimeType#System.Text.StringBuilder

Name             Modifier MemberType Definition
----             -------- ---------- ----------
Equals           public   Method     bool Equals(System.Object obj)
GetHashCode      public   Method     int GetHashCode()
GetType          public   Method     type GetType()
FormatError      private  Method*    static void FormatError()
ThreadSafeCopy   private  Method*    static void ThreadSafeCopy(System.Char*, mscorlib, Version=4...
ToString         public   Method*    string ToString()
__CreateInstance -        Method*    .ctor (), .ctor (int capacity), .ctor (string value), .ctor ...
__GetBaseObject  -        Method*    type __GetBaseObject()
__GetModuleInfo  -        Method*    psmoduleinfo __GetModuleInfo()
CapacityField    private  Field*     string CapacityField
DefaultCapacity  internal Field*     int DefaultCapacity
MaxCapacityField private  Field*     string MaxCapacityField
MaxChunkSize     internal Field*     int MaxChunkSize
StringValueField private  Field*     string StringValueField
ThreadIDField    private  Field*     string ThreadIDField

Peeking at non-public types:

# nonpublic types can't be specified using type literal
# syntax, so in this case you should use the -name parameter
$type = peek -name MS.Internal.Xml.XPath.XPathParser

# nonpublic objects returned from methods, properties or fields
# are not "peeked" themselves, so you may need to peek the return value:
$manager = peek (start-job { 42 } | peek).throttlemanager
$manager.throttlelimit = 64 # bump throttle limit ;)

Of course, you can peek instances too:

$sb = new-object system.text.stringbuilder
$proxy = peek $sb
$proxy | gm
   TypeName: Pokeable.System.Text.StringBuilder#45f12364-1906-45b3-b48b-a77acd81e3f0

Name                                                     Modifier MemberType Definition
----                                                     -------- ---------- ----------
GetHashCode                                              public   Method     int GetHashCode()
GetType                                                  public   Method     type GetType()
Append                                                   public   Method*    System.Text.StringBu...
AppendFormat                                             public   Method*    System.Text.StringBu...
AppendHelper                                             private  Method*    void AppendHelper(st...
AppendLine                                               public   Method*    System.Text.StringBu...
Clear                                                    public   Method*    System.Text.StringBu...
CopyTo                                                   public   Method*    void CopyTo(int sour...
EnsureCapacity                                           public   Method*    int EnsureCapacity(i...
Equals                                                   public   Method*    bool Equals(System.T...
ExpandByABlock                                           private  Method*    void ExpandByABlock(...
FindChunkForByte                                         private  Method*    System.Text.StringBu...
FindChunkForIndex                                        private  Method*    System.Text.StringBu...
Insert                                                   public   Method*    System.Text.StringBu...
InternalCopy                                             internal Method*    void InternalCopy(Sy...
MakeRoom                                                 private  Method*    void MakeRoom(int in...
Next                                                     private  Method*    System.Text.StringBu...
Remove                                                   private  Method*    System.Text.StringBu...
Replace                                                  public   Method*    System.Text.StringBu...
ReplaceAllInChunk                                        private  Method*    void ReplaceAllInChu...
ReplaceBufferAnsiInternal                                internal Method*    void ReplaceBufferAn...
ReplaceBufferInternal                                    internal Method*    void ReplaceBufferIn...
ReplaceInPlaceAtChunk                                    private  Method*    void ReplaceInPlaceA...
StartsWith                                               private  Method*    bool StartsWith(Syst...
System.Runtime.Serialization.ISerializable.GetObjectData private  Method*    void System.Runtime....
ToString                                                 public   Method*    string ToString()
VerifyClassInvariant                                     private  Method*    void VerifyClassInva...
__GetBaseObject                                          -        Method*    System.Text.StringBu...
__GetModuleInfo                                          -        Method*    psmoduleinfo __GetMo...
Capacity                                                 public   Property*  int Capacity { get; ...
Chars                                                    public   Property*  char Chars { get; se...
Length                                                   public   Property*  int Length { get; se...
MaxCapacity                                              public   Property*  int MaxCapacity { ge...
m_ChunkChars                                             internal Field*     char[] m_ChunkChars
m_ChunkLength                                            internal Field*     int m_ChunkLength
m_ChunkOffset                                            internal Field*     int m_ChunkOffset
m_ChunkPrevious                                          internal Field*     System.Text.StringBu...
m_MaxCapacity                                            internal Field*     int m_MaxCapacity

Have fun!

View or download the Poke module from bitbucket.

The Bourne Again PowerShell (BAPS) AWKWARD Module

Update 2013/4/3: This is an April Fools post. I think I got a bit too subtle here despite best efforts to make this kind of ridiculous. Apparently being tagged “OMGPONIES” is not enough to give it away. Yes, the script actually works, but it’s more of a roundabout way to compare the “awkward” AWK syntax with the more readable, idiomatic PowerShell. Thanks for reading.

Sometimes PowerShell’s object pipeline just gets in the way. Often I find myself needing to pipe a few commands together and while I lay it out in my head, I am already unconsciously preparing the regular expressions I’ll need to convert the text to a structure required by the receiving command. Other times I am idly flicking through a well-thumbed and worn binder of common commands and their output on sheets of grid paper so I can figure out what columns I need to munge. Muscle memory is a hard beast to tame, so with that in mind, I started out designing a module that would let me scratch that regex itch, and also keep things kind of “PowerShelly.” It then dawned on me that I could probably extend this further into a full POSIX* compatibility layer, with the codename for this being WARD (Windows Adaptable Reusable Development). My first implementation for AWK/WARD looked a little like this:

function awk {
    $input
}

While that implementation works well for a surprising number of cases, there are others which are a little trickier, so I figured I'd add a little intelligence to the commands for some of the more common use cases for AWK. Some examples:

# count lines in a file
cat text.txt | awk 'END{print NR}'

# Add all fields in ALL lines and print the sum
#
# sheet is:
#
#    a, b, c, d
#    1, 2, 3, 4
#    1, 2, 3, 4
#    1, 2, 3, 4
cat sheet.txt | awk '{s=0; for (i=1; i<=NF; i++) s=s+$i; print s}'

# Print 25 "A" characters
awk 'BEGIN{while (a++<25) s=s "A"; print s}'

Here's the current source of the AWKWARD BAPS module:

function awk ($expression) {
    
    switch ($expression) {
        
        # count lines
        'END{print NR}' {
            $input | measure
        }

        # print the total number of fields ("words") in all lines
        '{ total = total + NF }; END {print total}' {
            $input | measure -word
        }

        # print the sums of the fields of every line
        '{s=0; for (i=1; i<=NF; i++) s=s+$i; print s}' {
            $input | convertfrom-csv | measure -sum * | select property, sum
        }

         # add all fields in ALL lines and print the sum
        '{for (i=1; i<=NF; i++) s=s+$i}; END{print s}' {
            $input | convertfrom-csv | measure -sum * | measure -sum sum
        }

         # create a string of a specific length (e.g., generate N spaces)
        ([regex]'BEGIN{while \(a\+\+<(\d+)\) s=s "(.+?)"; print s}') {
            $matches[2] * $matches[1]
        }
        
        # print first N lines of file (emulates behavior of "head")
        ([regex]'NR < (\d+)') {
            $input | select -first $matches[1]
        }

        # print the last 2 lines of a file
        '{y=x "\n" $0; x=$0};END{print y}' {
            $input | select -last 2    
        }

         # print the last line of a file
        'END{print}' {
            $input | select -last 1
        }

        # print the last N lines of a file (circular buffer)
        ([regex]'{a\[NR%(\d+)\]=\$0}END{for(i=NR+1;i<=NR+(?:\d+);i++)print a\[i%(?:\d+)\]}') {
            $input | select -last $matches[1]
        }

        # emit matching lines for regex
        { $_ -as [regex] } {
            $input | where { ($expression -as [regex]).ismatch($_) }
        }
        
        default {
            write-warning "unsupported expression"
            $input
        }
    }
}
}

If you want to play around with the module, you can autoinstall it by running the following one-liner:

iex (New-Object Net.WebClient).DownloadString(“http://bit.ly/e0Mw9w”)

Have fun!

Ensuring a PowerShell script will always run in a 64 bit shell

Someone on Jabbr earlier today was struggling a bit with a NuGet init.ps1 script while trying to work with the IIS module. They thought the NuGet Package Manager shell was 64 bit, but in fact it’s a 32 bit shell as Visual Studio is itself 32 bit and will remain that way for the foreseeable future. So, if I have a dependency on some 64 bit binary module, how can I ensure that the script will get run in the right environment? Well, here’s some “stub” script that you can put at the top of your ps1 file that will detect its environment and relaunch itself in a 64 bit shell, passing along any arguments that were given.

I’d suggest keeping your 64 bit scripts in a separate ps1 and calling them from init.ps1 to do the work. Because the script will be relaunched in a 64 bit shell, you won’t have access to the package manager console intrinsics like $DTE, nor can you pass any “live” objects to the essentially external script. Stick to strings and other primitives like [int] and [bool] etc.

# am I running in 32 bit shell?
if ($pshome -like "*syswow64*") {
    write-warning "Restarting script under 64 bit powershell"

    # relaunch this script under 64 bit shell
    # if you want powershell 2.0, add -version 2 *before* -file parameter
    & (join-path ($pshome -replace "syswow64", "sysnative") powershell.exe) -file `
        (join-path $psscriptroot $myinvocation.mycommand) @args

    # exit 32 bit script
    exit
}

# start of script for 64 bit powershell

write-warning "hello from $pshome"
write-warning "My original arguments $args"

Also available on poshcode: http://poshcode.com/3827

Disclaimer: I’ve tested this with PowerShell 3.0 only.

PowerShell - Convert a .NET Type’s static methods into a Module

Updated 2012/9/24: $type.Name -> $type.FullName (otherwise only types directly in System namespace are found... oops!)

Here’s something I just knocked up recently to have a nice and simple way to import groups of functions temporarily from a .NET Type as a module. When you’re done with them, you can unload at any time using Remove-Module. As the inline help mentions, you must pipe the output of this function into Import-Module to make the functions available. You have to be somewhat familiar with the methods you’re converting in order to know what order to pass parameters. Some are obvious in that they only take one argument (like Sin, Cos or Tan) but others you’ll have to double check yourself. It would have been nice to convert .NET parameters and overloads into parameter sets, but the differences between how PowerShell and the .NET compilers resolve ambiguities can be very different and would only work for mostly simple cases. Here’s how I like to quickly check method syntax:

PS C:\projects> [math]::Log

OverloadDefinitions
-------------------
static double Log(double d)
static double Log(double a, double newBase)

Here's the function itself ( also available on poshcode (updated) )

function ConvertTo-Module {
<#
    .SYNOPSIS
    Quickly convert a .NET type's static methods into functions

    .DESCRIPTION
    Quickly convert a .NET type's static methods into functions.
    
    This function returns a PSModuleInfo, so you should pipe its
    output to Import-Module to use the exported functions.

    .PARAMETER Type
    The type from which to import static methods. 

    .INPUTS
    System.String, System.Type

    .OUTPUTS
    PSModuleInfo

    .EXAMPLE
    ConvertTo-Module System.Math | Import-Module -Verbose

    .EXAMPLE
    [math] | ConvertTo-Module | Import-Module -Verbose

#>
    [outputtype([psmoduleinfo])]
    param(
        [parameter(
            position=0,
            valuefrompipeline=$true,
            mandatory=$true)]
        [validatenotnull()]
        [type]$Type
    )

    new-module {
        param($type)
         
        ($exports = $type.getmethods("static,public").Name | sort -uniq) | `
            % {
                $func = $_
                new-item "function:script:$($_)" `
                    -Value {
                        # look mom! no [scriptblock]::create!
                        ($type.FullName -as [type])::$func.invoke($args)

                    }.GetNewClosure() # capture the value of $func
            }
        export-modulemember -function $exports
    } -name $type.Name -ArgumentList $type
}

Have fun!

Emulating Bash / GNU Readline with PowerShell 3.0

Introducing PSReadline 1.0 (Beta)

This is a module that takes advantage of a new hook added to PowerShell 3.0 that allows you to completely take over the readline API. To do so, you must define a function like:

function PSConsoleHostReadline {
    [Console]::Readline()
}

The is an example of a very simple implementation. You may think it's enough, until you realise that there is absolutely zero line editing: The cursor keys don't work, no home/end, no tab completion, nothing! There's a lot more to do than just grabbing a line of text.

This module tries to emulate the Unix Bash/GNU Readline experience. Tab completion works by dumping out a long line of space-separated matches, and will only complete the current line up to the maximum amount of shared leading letters for all matches based on the current token.

Thankfully, you will also get Bash style tab completion for types, cmdlets, parameters and their values as the PSReadline module uses PowerShell 3.0's powerful and fast tab completion APIs.

This is a beta release, so expect a glitch or two. Currently all of the bindings are based on EMACS. The next release will let you define your own bindings. The EMACS bindings are documented below.

Have fun!
readline
Installation
=============

- Download ZIP file, unblock with unblock-file cmdlet.
- Extract to ~\documents\windowspowershell\modules\
  -- This should result in a PSReadline folder 

PS> Import-Module PSReadline

Known issues:
=============

- <esc> does not clear the current line
- does not use powershell history (so get-history returns nothing)
- doesn't support fancy prompt functions with newlines and/or those
  that use write-host; single line prompt function only

Credits
==============
- Miguel de Icaza (getline.cs)
  
  Thanks to his unending masochism & for donating 
  the guts of the Readline emulation, which was torn kicking &
  screaming from an old Mono REPL C# shell. Much massaging and 
  poking was needed for NT & PowerShell, but it works.

Common Bindings
=========================================
Home          Cursor Home
LeftArrow     Cursor Left
RightArrow    Cursor Right
UpArrow       History - Previous
DownArrow     History - Next
Enter         Done
Backspace     Backspace
Delete        Delete Character
Tab           Tab / Tab Complete

EMACS Bindings
=========================================
Ctrl+A        Home
Ctrl+E        End
Ctrl+B        Left
Ctrl+F        Right
Ctrl+P        History - Previous
Ctrl+N        History - Next
Ctrl+K        Kill to EOL
Ctrl+Y        Yank
Ctrl+D        Delete Character
Ctrl+L        Refresh
Ctrl+R        Reverse Search History

Alt+B         Word - Backwards
Alt+F         Word - Forwards
Alt+D         Word - Delete
Alt+BkSpc     Word - Delete Backwards

=========================================
Ctrl+Q        Quote
=========================================

About the author

Irish, PowerShell MVP, .NET/ASP.NET/SharePoint Developer, Budding Architect. Developer. Montrealer. Opinionated. Montreal, Quebec.

Month List

Page List