A couple of months ago I had a brief Q&A with the cdndevs guys over on blogs.msdn.com. It looks like it went live recently because I’ve been getting stopped in the streets and harassed for a photo etc. No, not really. But if you want to read, take a look over at http://blogs.msdn.com/cdndevs/archive/2009/02/12/mvp-insider-q-a-with-oisin-grehan.aspx

Some of you may have heard that CTP3 has support for transferring files from remote servers using BITS (Background Intelligent Transfer Service) and quite possibly had a little dig around for it. It’s not easy to find as using “get-help BITS” will not return any help unless you’ve already loaded the module. Not a great win for discoverability, but hey, sometimes you just have to read the manual, expletive deleted. So, lets have a quick look at how this works.

So, ok, we can see we’ve got a module for FileTransfer (BITS), but where did this come from? Take a peek into $pshome\Modules to find out. This is the location for system-wide (global) modules. It rests under $PSHome, which is the home for PowerShell, in $env:systemroot\windowspowershell\v1.0.

Yes, even PowerShell v2 lives there. This is one of those unfortunate things whereby the team decided early on that they would support side-by-side versions of PowerShell, implying that v2 might not be backwards compatible with v1. This is reflected also in the choice of file extension: ps1. The extension ps2 would have been used for v2. This was probably a poor show of faith in their own abilities to design such a great version one product, because as it turns out, v1.0 was such an excellent release that they were able to build on it for v2 without compromising, or breaking backwards compatibility (apart from a couple of minor edge cases, mainly bugfixes). So, the end result is that future versions of PowerShell will inherit the $pshome, pay a little inheritance tax and hopefully avoid foreclosure :).

The files in the FileTransfer module directory are laid out as follows:

(Hey look! another built-in module – PSDiagnostics - I’ll leave that one for you guys to explore).

So, there are four files there. A ps1xml Format file, which tells PowerShell how to textually render the .NET objects returned by the Cmdlets. A psd1 Module Manifest file, which tells PowerShell what files comprise the Module and a binary DLL which in this case does NOT contain the Cmdlets, but instead is what’s called an “Interop Assembly.”  (sometimes known as Primary Interop Assemblies or PIAs, read about them here: Primary Interop Assemblies – there are subtle differences between IAs and PIAs, but not enough to warrant discussion here). Essentially this DLL lets .NET, and by extension, PowerShell, talk to the native COM APIs that wrap the BITS services on XP,Vista,Win7,Win 2003 and Win 2008. Lets take a peek into the Module Manifest:

If it looks like I just dumped a Hashtable out, you’re right. That’s all a manifest is: a Hashtable. Of course, the key names are important and in this case, lets take a look at two in particular, NestedModules and RequiredAssemblies.

The RequiredAssemblies key is responsible for actively loading .NET assemblies containing standard .NET types. It does not extract Providers and Cmdlets and add them to the runspace; this is what the other key, NestedModules, does. Like I said already, this DLL is the interop assembly. So where is the NestedModules loading that other module from? I don’t see any other DLLs in the directory, and I know there are no Cmdlets in that interop assembly. Lets take a peek at one of the Cmdlets itself and find out:

Aha, it’s reading it from the GAC. When PowerShell was installed, it must have installed this DLL there. This is the DLL that contains our Cmdlets. I know this is true because I used one of the Cmdlets to tell me where it lives. “GCM” is the alias for Get-Command which will get us metadata about any given command in PowerShell (including functions – new to v2).

So, lets use a one-liner to get the names of all the Cmdlets in this module, and the synopsis summary for the help:

This yields the following:

I guess this covers it for the moment. Going into a full discussion about how BITS actually works is worth another post in itself. Suffice it to say, there are plenty of examples in the help itself. Just run:

get-help add-filetransfer –example | more

To see an example on how to use the Cmdlet (or any of the others). The team have done a good job here.

Have fun!

It’s been quite a few years -- November 14, 2006 to be exact -- since the final release of PowerShell 1.0. Let’s see what the lazy buggers have been up to since then. Only kidding, they’re far from lazy; we’ve gained an extra one hundred and six new cmdlets, moving from 131 to 237. We’ve also gained another provider, WSManProvider which lets you explore and manipulate the WS-Man configuration. Finally, the number of public Types (.NET classes usable by 3rd parties to extend PowerShell) has increased from 447 to 749. So how do I know all this? Well, some months ago I wrote a suite of build analysis Cmdlets for examining the assemblies that comprise PowerShell. In its current form it’s of very little use to 3rd parties, but I’ve had some requests to open it up and allow it to analyze any binary modules and snap-ins, which is a good idea I think. Watch this space. Anyway, I’m going to dump out some interest information on the differences between v1.0 and v2.0 CTP3, along with the one-liners I’m using to generate the information.

update feb 5: added breaking changes

Breaking Changes to Windows PowerShell 1.0

The following changes in Windows PowerShell V2.0 CTP3 might prevent features designed for Windows PowerShell 1.0 from working correctly.

• The value of the PowerShellVersion registry entry in HKLM\SOFTWARE\Microsoft\PowerShell\1\PowerShellEngine has been changed to 2.0.
• New cmdlets and variables have been added. These are listed below. These new elements might conflict with variables and functions in profiles and scripts.
• -IEQ operator does a case insensitive comparison on characters.
• Get-Command gets functions along with cmdlets by default.
• Any native command that generates a user interface blocks if it is pipelined to the Out-Host cmdlet.
• Added new language keywords: Begin, Process, and End. Any commands called begin, process or end are interpreted as language keywords and might result in parsing errors.
• Cmdlet name resolution has changed. In Windows PowerShell 1.0, a runtime error was generated when two Windows PowerShell snap-ins exported cmdlets with the same name. In Windows PowerShell V2, the last cmdlet loaded is the one that is executed if the cmdlet name is not qualified by a snap-in name.
• Terminating errors that are thrown in a pipeline do not terminate the pipeline. Instead, they are written to the host.
• A function called with '-?' parameter gets the help topic for the function, if one is included in the function.

There are no changes to Cmdlets other than Get-Command’s –PSSnapin parameter is now an alias to a –Module parameter. This reflects the move away from administrator-installed snap-ins, and towards the friendlier Module system. For all intents and purposes, you can treat v1 snapins as modules as load them as such with the Import-Module Cmdlet.

New Cmdlets

Here are the new Cmdlets, sorted by noun and alongside the textual synopsis is one is present in the CTP3 help, or the syntax if there is no help. The command I ran to generate this was:

Whew, that’s quite a few. Now, onto the API.

.NET API Differences

This time, I’m just dumping out the namespaces and Types per namespace. It would be a bit much to dump out all the Types themselves. The one-liner:

Here are the namspaces and respective public Type count for v1.0:

v2.0 CTP3 API

Here are the namspaces and respective public Type count for v2.0 CTP3:

As you can see, there has been a nice expansion of namespaces and Types to work with, mostly coming from the sterling work being done to generalize the “Jobs” infrastructure. Also, a fair chunk is tied up with the refactoring and reorganizing of Runspaces to allow for jobs (which also covers eventing), remoting and background pipelines. This is something I will cover in more detail in a future post.

Have fun!

You might have noticed these two accelerators in the list I published recently along with the technique how to add your own type accelerators. It’s not immediately clear that they are related, but they very much are. This is going to be a fairly developer-oriented post, so there won’t be any hand-holding here.

[PowerShell]

The [PowerShell] accelerator is aliased to System.Management.Automation.PowerShell. It has one static method, Create(), which returns an instance of the class itself. In v1, if you wanted to set up asynchronous pipelines and do other fancy stuff, you had to get your hands dirty with Runspace and Pipeline instances and know how to wire them all together. The [PowerShell] class makes it a lot easier to do this; think of it a “PowerShell Pipeline Runner” class. At its simplest, you can just “new up” an instance, assign some script using the AddScript method and call Invoke(). You can run this asynchronously using the BeginInvoke/EndInvoke pattern which should be familiar to all .NET developers who’ve done a bit of threading work.

[RunspaceFactory]

This accelerator is aliased to System.Management.Automation.Runspaces.RunspaceFactory. It has two static methods of interest, CreateRunspace and CreateRunspacePool. I’m going to focus on the latter because it has more interesting uses which you will see. This latter method lets you create a collection of Runspace instances that are essentially reusable. The great thing is that you don’t have to worry about any of the details. It just works; this leads me to the next part: queuing local pipeline jobs to be run in the background.

The pool you create can be constrained in many ways by using the various overloads of the CreateRunspacePool. You can even pass it a RunspaceConnectionInfo object so that the queued pipelines are run on remote servers. This is done by using the New-PSSession cmdlet to create a session to a remote machine running PowerShell 2.0 with WinRM configured correctly.

Queueing Pipelines to a Runspace Pool

This is where the magic really happens. Simply new up a PowerShell instance, assign the pool to it and run a command. New up as many PowerShell instnaces as you like, and as long as you assign each of them the same pool, the pool automagically looks after processing them as fast as it can and will never go over its hard limits you give it for the number of simultaneous runspaces. In this next script, I set up a pool of three runspaces. I then queue up six pipelines to be run. I am using the BeginInvoke method to start the command in the background. You’ll see when it runs that each command will make a beep of a certain frequency when it finally starts up. You can hear the first three jobs start up pretty much straight away, as each completes, another starts up. Magic!

Feel free to post questions or requests for clarification, but I did say it was a bit tough and developer-oriented. I have a hint that our good friend and fellow MVP, Karl Prosser (aka Klumsy), will be wrapping up some of this stuff into some nice admin-oriented background  command functions.

IMPORTANT:

Due to the WaitAll call at line 40, this script will not work in an STA thread (i.e. in PowerShell ISE). Use PowerShell.EXE to run this script. Everything else will work in ISE fine.

Have fun!

A module manifest file is a PowerShell data file (.psd1) with the same name as the module directory. For example, the FileTransfer module installed with CTP3 -- which contains Cmdlets for BITS -- is in a directory called $pshome\Modules\FileTransfer and contains a manifest file called $pshome\Modules\FileTransfer\FileTransfer.psd1. Module manifests are optional, but highly recommended.

A module manifest contains a hashtable declared using the @{} hashtable literal syntax. Valid keys are listed in the following table:

Key	Required	Type	Description
ModuleToProcess	Optional	String	Script module or binary module file associated with this manifest which also becomes the root module for nested modules. If no module is specified, the manifest itself becomes the root module for nested modules. A binary module is the new name for a snap-in DLL. v1 style snapins can be loaded this way, without having to register them first with installutil.exe; A binary module does NOT need a PSSnapIn class defined.
ModuleVersion	Required	String convertible to System.Version	Version of the module
GUID	Optional	String	Unique identifier for the module which can be used to verify against the module name
Author	Optional	String	Identifies the author of the module
CompanyName	Optional	String	Identifies the company that created the module
Copyright	Optional	String	Module copyright
Description	Optional	String	Describes the contents of the module
PowerShellVersion	Optional	String convertible to System.Version	Minimum required version of the PowerShell engine
CLRVersion	Optional	String convertible to System.Version	Minimum required version of the CLR
RequiredModules	Optional	List of module names	List of modules that must already be loaded globally (note: required modules are not loaded automatically - this could optionally be done using a script in ScriptsToProcess).
RequiredAssemblies	Optional	String array	List of assemblies that will be loaded using the same algorithm as Add-Type
ScriptsToProcess	Optional	String array	Identifies the list of scripts to process when the module is imported. These scripts are dot sourced into the caller’s environment. Only .ps1 files can be specified.
TypesToProcess	Optional	String array	List of .ps1xml type files to process using Update-TypeData
FormatsToProcess	Optional	String array	List of .ps1xml format files to process using Update-FormatData
NestedModules	Optional	String array	List of .ps1, .psm1, .psd1, and .dll files to process on Import-Module. Files are processed in the order listed. DLLs are scraped for cmdlets/providers and .ps1 script files are dot sourced into the module’s session state.
ExportedFunctions	Optional	String array, wildcards supported	List of functions to export. If not defined or if asterisk is specified, all functions imported from nested modules are re‑exported. To prevent export, use the empty string ‘’.
ExportedCmdlets	Optional	String array, wildcards supported	List of cmdlets to export. If not defined or if asterisk is specified, all cmdlets imported from nested modules are re‑exported. To prevent export, use the empty string ‘’. A big change in CTP3 is that binary Cmdlets can now be scoped – previously they were always global.
ExportedVariables	Optional	String array, wildcards supported	List of variables to export. If not defined or if asterisk is specified, all variables imported from nested modules are re‑exported. To prevent export, use the empty string ‘’.
ExportedAliases	Optional	String array, wildcards supported	List of aliases to export. If not defined or if asterisk is specified, all aliases imported from nested modules are re‑exported. To prevent export, use the empty string ‘’.
PrivateData	Optional	Object	Data to be passed to the module via the manifest file.

I’ve highlighted certain things in the description that are important to note. These are the things that might trip you up.

Have fun!