Importing VM Images and Volumes with PowerShell and C#

Version 2.2.0 of the AWS Tools for Windows PowerShell and AWS SDK for .NET contained updates to make it easy to import virtual machine images and disk volumes into Amazon EC2. In the case of PowerShell, there are revised Import-EC2Instance and Import-EC2Volume cmdlets, while in the SDK there is a new helper class, DiskImageImporter, in the Amazon.EC2.Import namespace. This post examines the updates and shows you how to use PowerShell or C# to upload your images.

Importing via PowerShell

You can import virtual machine images into Amazon EC2 by using Import-EC2Instance, or you can use Import-EC2Volume to import disk images as EBS volumes. In both cases, Amazon EC2 must convert the uploaded content before it can be used. To track progress of the conversion, the cmdlets return a ConversionTask object containing the task ID plus other supporting information.

Both cmdlets can also be used to upload the content but defer the conversion to a later time. In this mode, the cmdlets return the Amazon S3 object key of an import manifest that has been created on your behalf based upon the parameters you supply and the image file being uploaded. To start the conversion, you run the same cmdlet that was used to upload the artifacts, but this time you pass the object key to the -ManifestFileKey parameter and remove the -ImageFile parameter. The cmdlets will then request Amazon EC2 begin conversion and return to you a ConversionTask object.

Monitoring Conversion Progress

Starting a conversion is great, but then what? To track the conversion progress, you can use the Get-EC2ConversionTask cmdlet. This cmdlet accepts a ConversionTask object (strictly speaking, it wants the conversion task ID contained within the object) and outputs the current status of the conversion. The ConversionTask object can be from either an instance conversion or a volume conversion.

You can also cancel a conversion using the Stop-EC2ConversionTask cmdlet. Just like Get-EC2ConversionTask, this cmdlet takes a ConversionTask object, or conversion task ID, and it instructs Amazon EC2 to abandon the in-progress conversion.

Now that we know the basics of uploading and converting, let’s take a look at some examples of the cmdlets and the options available.

Upload and Convert

Uploading with immediate conversion is the default operating mode for the cmdlets. For this mode, the cmdlet requires the name of the image file to upload and the destination bucket for the artifacts (the bucket will be created if required). When importing a VM image, you add parameters to specify the instance type, architecture, and platform (the default platform is ‘Windows’ if you don’t specify it). For example:

Import-EC2Instance -ImageFile C:Windows.2012-disk1.vmdk `
                   -InstanceType m3.xlarge `
                   -Architecture x86_64 `
                   -BucketName myvmimages ` 
                   -KeyPrefix windows.2012

The -KeyPrefix parameter is optional and can be used to collect related image files together in Amazon S3. Each image file will be uploaded to an S3 key path of keyprefix/guid, with a new guid used for each upload.

Here’s an example of importing a disk volume:

Import-EC2Volume -ImageFile C:Windows.2012-disk2.vmdk `
                 -BucketName myvmimages `
                 -KeyPrefix windows.2012.volume2 `
                 -AvailabilityZone us-west-2a

For a volume, you must specify the Availability Zone for the region it is being imported to. You can retrieve the set of Availability Zones for a region by using the Get-EC2AvailabilityZone cmdlet:

PS C:> Get-EC2AvailabilityZone -Region us-west-2 | select ZoneName

ZoneName
--------
us-west-2a
us-west-2b
us-west-2c

Upload Only, Defer Conversion

To upload an image file but defer conversion, you add the -UploadOnly switch to the cmdlets. Parameters related to Amazon EC2 instances or Availability Zones are omitted:

Import-EC2Instance -UploadOnly `
                   -ImageFile C:Windows.2012-disk1.vmdk `
                   -BucketName myvmimages `
                   -KeyPrefix windows.2012


Import-EC2Volume -UploadOnly `
                 -ImageFile C:Windows.2012-disk2.vmdk `
                 -BucketName myvmimages `
                 -KeyPrefix windows.2012

Notice how both cmdlets take the same parameters for this mode. When run with the -UploadOnly switch, both cmdlets return to the pipeline the Amazon S3 object key of the import manifest. You’ll need this to start the conversion at a later date, as explained below.

Regarding Volume Sizes

None of the examples in this post use the -VolumeSize parameter. In this case, the import will use the size of the image file, rounded up to the nearest GB, on your behalf with the minimum size of a volume being 8 GB. To use a different volume size, simply add the -VolumeSize parameter. EBS volumes can be a minimum of 1 GB, but EC2 instance boot volumes must be 8 GB or greater.

Starting a Deferred Conversion

To start conversion of a disk image that has been uploaded previously, you need the S3 object key of the import manifest, which you pass to the -ManifestFileKey parameter, in addition to parameters describing the instance or volume (but omitting the -ImageFile parameter, since the image has been uploaded already). The -ManifestFileKey parameter takes an array of manifest key names, so you can upload several images and then request conversion as a batch operation. Note that when dealing with VM images, the EC2 instance parameter values apply to all of the images so you can’t mix Linux and Windows, or 32-bit/64-bit images in a batch:

PS C:> $manifest1 = Import-EC2Instance -ImageFile ...
windows.2012/f21dcaea...
PS C:> $manifest2 = Import-EC2Instance -ImageFile ...
windows.2012/4e1dbaea...
PS C:> $manifest3 = Import-EC2Instance -ImageFile ...
windows.2012/d32dcfed...

PS C:> Import-EC2Instance `
               -ManifestFileKey @($manifest1, $manifest2, $manifest3) `
               -InstanceType m3.xlarge `
               -Architecture ...

Handling Upload Errors

If the upload of the image file fails, the cmdlets will leave what has been successfully uploaded in the Amazon S3 bucket and return to you the S3 object key to the manifest, together with instructions on how to resume the operation or cancel and delete the uploaded content. Here’s an example of the output message when an upload fails:

PS C:> Import-EC2Instance -ImageFile C:Windows.2012-disk1.vmdk ...   

Import-EC2Instance : The import operation failed to upload one or more image file parts.
To resume: re-run the cmdlet and add the -Resume parameter.
To cancel and remove uploaded artifacts: inspect the S3 bucket and delete all objects with key prefix
'windows.2012/f21dcaea-4cff-472d-bb0f-7aed0dcf7cf9'.
.....

To resume the upload, the command to run would therefore be

Import-EC2Instance -ImageFile C:Windows.2012-disk1.vmdk `
                   -Resume `
                   -KeyPrefix windows.2012 `
                   -InstanceType ...

Note that the -Resume parameter currently applies only to the upload phase of the import process. If the subsequent EC2 conversion fails with an error, you need to submit a new import request (with new upload) to correct whatever error EC2 has reported.

Importing via C#

The PowerShell cmdlets perform the upload and conversion work using a new helper class in the AWS SDK for .NET, so the same capabilities are available to SDK-based applications. The new class is in the Amazon.EC2.Import namespace and is called DiskImageImporter. The class handles uploads and conversion of VM images and disk volumes and can also be used to perform upload-only or upload-and-convert usage scenarios, just like the cmdlets.

The following code snippet shows how you can upload and request conversion of an image file from C#:

using Amazon.EC2.Import;

...
const string imagesBucket = "mybucketname";

var importer = new DiskImageImporter(RegionEndpoint.USWest2, imagesBucket);

var launchConfiguration = new ImportLaunchConfiguration
{
    InstanceType = "m3.xlarge",
    Architecture = "x86_64",
    Platform = "Windows",
};

importer.ImportInstance
  (@"C:Windows.2012-disk1.vmdk",
   null,                   // file format -- inferred from image extension
   null,                   // volume size -- infer from image size
   "windows.2012",         // key prefix of artifacts in S3
   launchConfiguration,    // EC2 instance settings
   (message, percentComplete) =>
       Console.WriteLine(message +
           (percentComplete.HasValue
               ? string.Format("{0}% complete", percentComplete.Value)
               : string.Empty))
 );
...

More information on importing images and volumes to Amazon EC2 can be found at Importing a VM into Amazon EC2.

As you can see, importing virtual machine artifacts to Amazon EC2 is now easy and convenient for Windows and .NET users. Be sure to let us know via our comments section what other operations you’d find convenient to have available!

AWS Developer Tools Blog