Interacting with VMware vCO and the Rest API using PowerShell – getting a list of workflows

Recently I needed to get a list of VMware vCO workflows from a remote server using PowerShell. A colleague of mine pointed me in the right direction by providing me with a URL to access the vCO Rest API on, as well as letting me know what I needed to send in order to authenticate.

To connect and retrieve content back in the PowerShell example below, we’ll need to:

  • Access the Rest API URL for Orchestrator using a web client object
  • Send basic authentication in the header of our request

Notes:

One thing I did notice is that when you use your web browser to test the URL, the result is returned to you as XML, however when I used a web client object in PowerShell, I got a result returned to me in JSON. The PowerShell script below is therefore tailored to interpret the result as JSON. This being so, you’ll need to make sure you are using PowerShell 3.0 or above, as the ConvertFrom-Json cmdlet is only available using PowerShell 3.0 and above.

When sending your authentication details with the web client object request, make sure your username/password combo are used in this format:

Authorization: Basic username:password

This means that your header you add to your web client option, should be added with the string as per the above, but with the username:password part encoded using base64. The script below takes this all into account, and all you need to do is provide your username and password for vCenter Orchestrator to the PowerShell function, it will handle the base64 encoding and passing of the values to the web client itself.

Anyway, enough of that, let us get onto the actual script itself. This is presented as a PowerShell function. Load it into your session (copy-paste) or add it to your PS profile for future use. Apologies for the formatting – Syntax Highlighter really messes with the formatting and nice clean indentation I normally have in my scripts!

Here is a direct download of the PowerShell script if the script paste below doesn’t work for you:
[wpdm_file id=31]

 

Function Get-VcoWorkflow() 
{

<#
.SYNOPSIS
Fetches vCO Workflow information and details from a vCenter Orchestrator server

.DESCRIPTION
Fetches vCO Workflow information and details from a vCenter Orchestrator server

.PARAMETER Username
Username for the vCO server

.PARAMETER Password
Password for the vCO server

.PARAMETER Server
The vCO server hostname or IP address

.PARAMETER PortNumber
The port to connect on

.EXAMPLE
PS F:\> Get-VcoWorkflow -Username Sean -Password mypassword -Server 192.168.60.172 -PortNumber 8281

.LINK

http://www.shogan.co.uk

.NOTES
Created by: Sean Duffy
Date: 30/03/2014
#>

[CmdletBinding()]
param(
[Parameter(Position=0,Mandatory=$true,HelpMessage="Specify your vCO username.",
ValueFromPipeline=$true,ValueFromPipelineByPropertyName=$true)]
[String]
$Username,

[Parameter(Position=1,Mandatory=$true,HelpMessage="Specify your vCO password.",
ValueFromPipeline=$false,ValueFromPipelineByPropertyName=$true)]
[String]
$Password,

[Parameter(Position=2,Mandatory=$true,HelpMessage="Specify your vCO Server or hostname.",
ValueFromPipeline=$false,ValueFromPipelineByPropertyName=$true)]
[String]
$Server,

[Parameter(Position=2,Mandatory=$true,HelpMessage="Specify your vCO Port.",
ValueFromPipeline=$false,ValueFromPipelineByPropertyName=$true)]
[ValidateRange(0,65535)] 
[Int]
$PortNumber
)

process 
{
$Report = @() | Out-Null

# Craft our URL and encoded details note we escape the colons with a backtick.
$vCoURL = "https`://$Server`:$PortNumber/vco/api/workflows"
$UserPassCombined = "$Username`:$Password"
$EncodedUsernamePassword = [System.Convert]::ToBase64String([System.Text.Encoding]::UTF8.GetBytes($UserPassCombined))
$Header = "Authorization: Basic $EncodedUsernamePassword"

# Ignore SSL warning
[System.Net.ServicePointManager]::ServerCertificateValidationCallback = {$true}

# Create our web client object, and add header for basic authentication with our encoded details
$wc = New-Object System.Net.WebClient;
$wc.Headers.Add($Header)

# Download the JSON response from the Restful API, and convert it to an object from JSON.
$jsonResult = $wc.downloadString($vCoURL)
$jsonObject = $jsonResult | ConvertFrom-Json

# Create a blank Report object array
$Report = @()

# Iterate over the results and transform the key/value pair like formatting to a proper table.
# Note I use a hashtable to populate a new PSObject with properties, using values found in the original JSON result
foreach ($link in $jsonObject.link)
{
	$kvps = $link.attributes
	$HashTable = @{}
	$kvps | foreach { $HashTable[$_.name] = $_.value } # foreach item in the link object, populate the hashtable
	$NewPSObject = New-Object PSObject -Property $HashTable # Create a new PSObject and populate the properties/values using our hashtable

	# Add our populated object to our Report array
	$Report += $NewPSObject
}

return $Report

}
}

I hope that this script comes in handy, and gives you an idea as to how you can retrieve and convert data from vCO and its RESTful API for use in PowerShell 🙂

Results example after running the Function:
get-vcoworkflow-results

 

Simple Content Delivery Network (CDN) using Amazon AWS (S3 + CloudFront)

 

Content Delivery Networks

Having a content delivery network has many benefits for your users or clients. One of the most obvious reasons of having a CDN, is the ability to serve up content to your users from multiple (often the most optimal) locations.  Users access files that originate from one original source location, but the content is delivered by the closest location(s), often with the lowest latency and highest possible speed.

Using Amazon CloudFront, you can share dynamic, static, or even streamed content to users (including full websites), using Amazon’s global network of edge locations. This means that content can be served to users at the highest possible speeds, with the lowest possible latencies. In this blog post, I will cover the steps you need to take to deploy a basic CDN using Amazon AWS. For this purpose, we will leverage a combination of Amazon S3 + CloudFront.

 

Setting up Amazon S3

Amazon S3 (Amazon Simple Storage Service) is essentially Amazon’s “storage for the Internet”, and as explained above, CloudFront is a content delivery network service. As such, both products sit in Amazon’s “Storage & Content Delivery” stack.

 

  • To get started you will of course need an Amazon AWS account. Go to http://aws.amazon.com/ and register. You will need to provide credit card details, but most products have some sort of free tier that you can utilise for initial testing (usually free for up to 1 year, based on certain utilisation thresholds).
  • Once you are all signed up, you’ll need to navigate to the AWS Web Console. This is the central location you can use to manage all AWS services (among other options such as the AWS SDK and Command Line).
aws-console-example
The central, AWS Web Management Console
  • To start, we’ll need to define an origin location for our content. This is the location our original files are kept. For this purpose, we will use Amazon S3. It allows us easy access to files that we place in something Amazon call a “bucket”. I like to think of it as a folder, or container. You can have as many buckets as you wish, however each one’s name needs to be completely unique across Amazon S3. Click on “S3” under the “Storage & Content Delivery” heading of your AWS Console to get started.
  • From here, you will be greeted with a welcome page and some explanation of what S3 is. Simply click “Create Bucket” to get going.

create-bucket

 

  • Provide a unique bucket name, and specify a region to use. Regions have the benefit of allowing organisations to comply with storage regulation rules – for example, if you were storing client data that you were bound legally to keep within the UK, you would specify the Ireland region.

new-bucket

 

  • Your new bucket will appear in the S3 Management Console after being created. Simply click the name of the bucket to open it. For our simple CDN, we’ll just be serving up one single file – pretend this was a really large file that needed efficient distribution to many people – for example a large media file. At the top left, you’ll see an “Upload” button. Click this, and choose a file to upload as your test file. I will be using a simple image file. (By the way, Amazon have a service called “Amazon Import/Export”, which allows you to send really large amounts of data via post on portable media to Amazon for them to upload directly to your Amazon S3 or Glacier services).
  • Click “Start Upload” once you have chosen a file to test with.
  • After the file is finished uploading, it will appear in the console under your bucket name. (I called mine “image-for-distribution.png”).

example-file-in-bucket

 

  • Right-click the file, and choose the option “Make Public” for this test. This choice would be affected by the nature of the files you would want to deliver to users in your own configuration, but for this simple example, this is what I am choosing.
  • Right-click the file again, and choose “Properties“. Here you can get the direct, public link to your file and test access to it in your web browser. This is simple, direct access, and is not the access we are aiming for, as we will utilise our CDN with CloudFront to serve the file in our final configuration. This is just to test that the direct link is working.

aws-file-properties

 

Setting up CloudFront and your Distribution

  • Now that we know our basic file is being correctly served from Amazon S3, we’ll navigate to “CloudFront” from the main AWS Console (aws.amazon.com). A quick way to get there is by clicking the orange cube icon in the top left of your AWS page – wherever you are in the console, it’ll take you back to the main AWS console. From there just click “CloudFront“.
  • In CloudFront, we’ll want to create something called a “Distribution“. Click the “Create Distribution” button to get started.

create-distribution

 

  • Make sure you select “Download” type for the “delivery method” when asked on the next page, then click “Continue“.

cloudfront-delivery-method

 

  • We’ll now select various options for our CloudFront Distribution.
    • For “Origin Domain Name“, click the text box and you’ll see a populated list of Amazon S3 buckets. Your bucket you created earlier should feature here. Click it to select it.
    • The “Origin ID” should auto populate based on your S3 bucket name you chose.
    • If you wish to restrict users to only access your content via CloudFront URLs, and not direct by S3 URLs, then choose “Yes” for “Restrict Bucket Access“.
    • If you chose “Yes” for restricting bucket access, you’ll also need to create a “Comment” and “Grant Read Permissions” on the bucket for CloudFront’s access to the S3 bucket. Click “Yes, Update Bucket Policy” to have CloudFront get read access automatically to the S3 bucket.
    • Select “HTTP and HTTPS” for “Viewer Protocol Policy“.
    • You can customise the object caching properties if you wish, but for this example, just leave the “Default Cache Behavior Settings” on their defaults.
    • Now you can set your “Distribution Settings“. Choose “Use All Edge Locations (Best Performance)” for “Price Class“. This will ensure that all edge locations around the world are used to distribute your content in the fastest, most efficient way to your users. You could also restrict this to other groups of regions e.g. only the US and Europe for example – this would be a cheaper option, but not as efficient for all users globally.
    • Next, we can add an alternate CNAME for the distribution. This is highly recommended so that you can provide your own domain name formatted URLs to users, instead of a long, ugly default Amazon CloudFront URL. Enter something now, (for example I will use cdn.shogan.co.uk as I own the domain and can create this CNAME record myself in DNS). Once you are complete with this distribution setup, you should get the Distribution URL, and point a new CNAME record to the full URL that CloudFront assigns to your distribution.
    • Leave all other options at their defaults for now, and make sure that the last option “Distribution State” is “Enabled“, then click the “Create Distribution” button at the very bottom.

example-distribution-settings1 example-distribution-settings2

  • Your Distribution should now be created. Use the Navigation menu on the left side of the screen and click “Distribution” to see a list of your CloudFront Distributions.

cloudfront-distributions

 

  • At first the “Status” will show “InProgress“. After a few minutes this should change to “Deployed“.
  • In the mean time, look for your “Domain Name” that this Distribution has been assigned, and go and create a CNAME record pointing the CNAME you specified when creating this distribution, to the domain name. For example, you may have something like dxxxxxxxxxm.cloudfront.net. In my case, I specified a CNAME of cdn.shogan.co.uk, so I will create a CNAME record linking these together.

 

Testing

Once your CNAME record is created, type in your new CNAME record, followed by a forward slash, and then the name of the file you originally uploaded to your S3 bucket that is linked to by this CloudFront distribution. For example, my file was called “file-for-distribution.png” and my CNAME record I made is cdn.shogan.co.uk. So to utilise my CloudFront CDN, I would simply access the file as “cdn.shogan.co.uk/image-for-distribution.png”. If your DNS takes a while to apply/propagate, then you can simply use the CloudFront domain name assigned to your Distribution (for example dxxxxxxxxxxm.cloudfront.net/yourfilename.extension) to test out your distribution. Remember to ensure your distribution is in a deployed state before testing. You should now see your file served up in your web browser via your brand spanking new Amazon AWS powered CDN!

 

Conclusion

That concludes the basic setup of a Amazon S3 + CloudFront powered Content Delivery Network. I hope this was useful for some. In forthcoming blog posts I will delve into setting up custom logging and monitoring / alerting for your CDN. Please remember to like/share/tweet this post out to friends if you thought it was useful.

 

 

Setting up DNS SRV records for an Office 365 migration (on 123-reg)

I needed to setup some DNS records for an Office 365 migration earlier and was initially slightly confused translating the settings Microsoft supplied us to those needed as input on 123-reg’s Advanced DNS configuration. The MX, TXT and CNAME records were simple enough, but it was the SRV records that needed a bit of fiddling to get right.

As an example on the SRV records, MS give you something like this:

Type Service Protocol Port Weight Priority TTL Name Target
SRV _sip _tls 443 1 100 1 Hour thedomain.co.uk sipdir.online.lync.com
SRV _sipfederationtls _tcp 5061 1 100 1 Hour thedomain.co.uk sipfed.online.lync.com

123-reg give you this interface to enter SRV records yourself:

Screen Shot 2013-06-13 at 14.24.42

 

By looking at the examples you can start to understand how to translate the Service, Protocol, and Weight items that MS give you, into the 123-reg input boxes (which do not exist individually for Service, Protocol and Weight).

In the first SRV record example –

Hostname therefore becomes: _sip._tls (the Service + the Protocol with a dot (.) between)

TTL of course becomes: 3600 (1 hr)

Priority is 100

Destination (the most confusing one) becomes: 1 443 sipdir.online.lync.com. (note that it starts with Weight (1), then a space, then the port number (443), then the Target (sipdir.online.lync.com), followed by a dot (.)

That forms your complete SRV record. By entering these along with the other records you require, you should have a fully functional Office 365 setup on your custom domain name.

Getting and Setting Path Selection Policies with PowerCLI

A colleague I work with was looking for a script that would list all disk devices on ESXi hosts that were not set to Round Robin. After this, he wanted to be able to set all of these to the Round Robin Path Selection Policy (PSP). The two tasks can actually be achieved really easily with just a few lines of PowerCLI.

Getting disk devices on all ESX hosts that do not use the Round Robin PSP:

$AllESXHosts = Get-VMHost | Where { ($_.ConnectionState -eq "Connected") -or ($_.ConnectionState -eq "Maintenance")} | Sort Name
Foreach ($esxhost in $AllESXHosts) {
	Get-VMhost $esxhost | Get-ScsiLun -LunType disk | Where { $_.MultipathPolicy -notlike "RoundRobin" } | Select CanonicalName,MultipathPolicy
}

To set the path selection policy to something else, it is a simple modification of the above to add the Set-ScsiLun cmdlet:

 

$AllESXHosts = Get-VMHost | Where { ($_.ConnectionState -eq "Connected") -or ($_.ConnectionState -eq "Maintenance")} | Sort Name
Foreach ($esxhost in $AllESXHosts) {
	Get-VMhost $esxhost | Get-ScsiLun -LunType disk | Where { $_.MultipathPolicy -notlike "RoundRobin" } | Set-ScsiLun -MultipathPolicy "RoundRobin"
}

There are quite a few scripts floating around out there that achieve similiar results, but I thought I would do my own take on this task. Below is my full script that offers a simple menu system with two options, one to list/log all current path selection policies on hosts that are not round robin to a timestamped log file, and one to set all path selection policies not round robin, to round robin and log these.

 

# Description:  Offers to either check and log path selection policies not set to RoundRobin, OR, set and log these disk PSPs to RoundRobin
# Author: Sean Duffy
# Web/blog: http://www.shogan.co.uk
# Date: 25/05/2013
## IMPORTANT: Always test scripts in preproduction or your lab before using them live!!

#region UserDefined
# Setup the location of our log files...
# This should be the only custom bit you need to set (the location you want log files to be)
# This will only be used when changing path selection policies
$SetPSPlogfile = "C:\temp\SetPathSelectionPolicy_UpdateScript_Log.txt"
# This will only be used when seeing what path selection policies are in place that are not round robin
$GetPSPlogfile = "C:\temp\GetPathSelectionPolicy_UpdateScript_Log.txt"
#endregion

# Ask for connection details, then connect using these
$vcenter = Read-Host "Enter vCenter Name or IP"
$username = Read-Host "Enter your username"
$password = Read-Host "Enter your password"
$Connection = Connect-VIServer $vcenter -User $username -Password $password

# Grab all ESX hosts on the connection that are either connected, or connected and in maintenance mode
$AllESXHosts = Get-VMHost | Where { ($_.ConnectionState -eq "Connected") -or ($_.ConnectionState -eq "Maintenance")} | Sort Name

# Prompt user with two options - 1) List all disk devices where no roundrobin PSP is set, or 2) Change all disk devices to RoundRobin PSP
Clear
Write-Host "1) Log all disk devices on all hosts where RoundRobin not set." -ForegroundColor Yellow
Write-Host "2) Set and log all disk devices on all hosts where RoundRobin not set, to RoundRobin." -ForegroundColor Yellow
$MenuChoice = Read-Host "Enter your selection (1/2)"

# We could use a switch statement here too, but an if,elseif,else statement is fine in this case...
# Chose option 1
if ($MenuChoice -like "1") {
	# Log the username to our log file
	$User = $Connection.User
	Write "User logged in for reading of PSPs: $User" | Out-File $GetPSPlogfile -Append
	Write-Host "Paths not set to RoundRobin for each host will be logged to the $GetPSPlogfile"
	Foreach ($esxhost in $AllESXHosts) {
		# Write an entry into our log for the host we busy reading PSPs from
		$Now = Get-Date
		Write "$Now :: Getting paths on $esxhost where not set to RoundRobin" | Out-File $GetPSPlogfile -Append
		Write-Host "ESX Host: $esxhost :: Disk devices not set to RoundRobin" -ForegroundColor Cyan
		Get-VMhost $esxhost | Get-ScsiLun -LunType disk | Where { $_.MultipathPolicy -notlike "RoundRobin" } | Select CanonicalName,MultipathPolicy | Out-File $GetPSPlogfile -Append
	}
	Write "------------- End of this run -------------" | Out-File $GetPSPlogfile -Append
	Disconnect-VIServer * -Confirm:$false
}
# Chose option 2
elseif ($MenuChoice -like "2") {
	# Prompt user with a warning first and ask to confirm to continue as this is a change to policies...
	Write-Host "WARNING: You are about to loop through all ESX hosts found and change disks to a PSP of RoundRobin! 
	Are you sure you want to continue? (Y/N): " -ForegroundColor Yellow -NoNewline
	$Answer = Read-Host

	# If user answered y, or Y, then continue with the change...
	if ($Answer -like "y") {
		# Log the username to our log file
		$User = $Connection.User
		Write "User logged in for change: $User" | Out-File $SetPSPlogfile -Append
		# Loop through all ESX hosts
		Foreach ($esxhost in $AllESXHosts) {
			# Write an entry into our log for the host we are working on along with the current timestamp
			$Now = Get-Date
			Write "$Now :: Setting PSP for the following current devices on $esxhost to RoundRobin" | Out-File $SetPSPlogfile -Append
			Write-Host "ESX Host: $esxhost :: Now setting path selection policy for all disk devices that are not already set to RoundRobin, to RoundRobin" -ForegroundColor Cyan
			# Find all disk devices where they are not already set to Round Robin Path Selection Policy and log these to log file
			Get-VMHost $esxhost | Get-ScsiLun -LunType disk | Where { $_.MultipathPolicy -notlike "RoundRobin" } | Select CanonicalName,MultipathPolicy | Out-File $SetPSPlogfile -Append
			# Find all disk devices where they are not already set to Round Robin Path Selection Policy and set these to RoundRobin
			Get-VMhost $esxhost | Get-ScsiLun -LunType disk | Where { $_.MultipathPolicy -notlike "RoundRobin" } | Set-ScsiLun -MultipathPolicy "RoundRobin"
		}
		Write "------------- End of this run -------------" | Out-File $SetPSPlogfile -Append
		Disconnect-VIServer * -Confirm:$false
	}
	else {
		Disconnect-VIServer * -Confirm:$false
		Write-Host "Script aborted" -ForegroundColor White
	}
}
# Didn't enter a valid choice...
else {
	Disconnect-VIServer * -Confirm:$false
	Write-Host "Not a valid choice! Exiting script..."
}

A function to lookup Host System friendly name by MoRef using PowerCLI

In a recent blog post, I showed a simple method of outputting a list of hosts with their friendly names, as well as their MoRef (Managed Object Reference) names alongside eachother, enabling you to match up which host belongs to which MoRef. I wanted to take that a little further, with a function that is able to return the friendly name of a host’s MoRef that is input into the function. I have used this is a larger reporting script, where I can only get the MoRef of a host via it’s property within a cluster object. Basically, I look for any Failover hosts (admission control policy), which is an array, and the hosts are listed as indexed objects of this array. They are also only displayed as MoRef names, so at this point, instead of inserting the MoRef into my results, I insert the MoRef into this function, return the friendly name, and input this instead. Which allows the person reading the report to easily identify the host! $cluster.ExtensionData.Configuration.DasConfig.AdmissionControlPolicy.FailoverHosts <- In this example, $cluster is a particular cluster using the “specify a failover host” policy, and “FailoverHosts” is the array, with each object within containing a host MoRef. For example FailoverHosts[0].Value would be one instance, and may equate to “HostSystem-host-28” for example.   So here is the function. It takes two mandatory parameters: -MoRef (the MoRef of the host in question of course), and -Cluster (the name of the Cluster to do the lookup in) – the function loops through each host in this cluster to look for a host that matches the input MoRef.

Function Get-VMHostByMoRef() {

<#
.SYNOPSIS
Fetches host name by input MoRef and Cluster to look in

.DESCRIPTION
Fetches host name by input MoRef and Cluster to look in

.PARAMETER MoRef
The MoRef of the host system

.PARAMETER Cluster
The name of the cluster to do the lookup in

.EXAMPLE
PS F:\> Get-VMHostByMoRef -MoRef HostSystem-host-28 -Cluster MyCluster01

.LINK
http://www.shogan.co.uk

.NOTES
Created by: Sean Duffy
Date: 22/02/2013
#>

[CmdletBinding()]
param(
[Parameter(Position=0,Mandatory=$true,HelpMessage="Specify the Host MoRef name you would like to query for it's friendly name.",
ValueFromPipeline=$true,ValueFromPipelineByPropertyName=$true)][String]$MoRef,[Parameter(Position=1,Mandatory=$true,HelpMessage="Specify the Cluster to search hosts in.",
ValueFromPipeline=$true,ValueFromPipelineByPropertyName=$true)][String]$Cluster
)

process {

$AllHosts = Get-Cluster $Cluster | Get-VMHost
$thehost = $AllHosts | Where {$_.ExtensionData.MoRef -match $MoRef} | Select-Object -Property Name

return $thehost

}
}

Here is a quick sample of the output when called directly from the PowerCLI prompt (note the MoRef of “host-22” used to find the real host name of “esxi02.homelab.local”: Hopefully this may be of use to some – add it to your PowerCLI script/function toolkit or throw it into your PowerShell $profile for easy access in the future!