Velostrata v2.7 Release Notes

New Features

AWS: Encrypted EBS Disks Support

When creating native EBS disks for migrated instances, it is now possible to select EBS disk encryption as well as customer specified KMS keys.

Cloud Extension Availability Enhancements

Velostrata Cloud Extension now provides high availability for workloads running in streaming migration and cached mode by using multipath access to workload storage in an active-passive model. When the primary Cloud Edge node fails, the workloads failover to use the secondary Cloud Edge node; failback to the primary Cloud edge node is done once it becomes active again.

The Cloud Extension is now more robust with the addition of auto-healing mechanisms.

User operations that were previously blocked when Cloud Extension was impaired are now available and depend on granular component health rather than overall Cloud Extension state.

Removal of the Windows Servicing Instance Component

The Windows Servicing Instance component was removed in the Cloud Extension deployment. This enhances our mass scaling of Windows OS adaptation phase of multiple concurrent migrations; this also reduces the cost of a Cloud Extension.

Azure: Large disks supported

Microsoft announced, on June 15th 2017, an increase of maximum disk sizes that extends the maximum size of Azure disks from 1,024 GB to 4,095 GB (see announcement here). Velostrata version 2.7 now supports the migration of disks with an extended size to Azure.  

Ubuntu Support

Streaming migration support for Ubuntu 14.04.x.

Offline Migration Enhancements

Added unmapped blocks optimization for offline migration of Windows workloads.

Tested offline migration for the following OS versions:

  • Windows Server 2008
  • Ubuntu 12
  • Ubuntu 14
  • Ubuntu 16.04

Compatibility and Installation

ESXi, vCenter Server, and vSphere Web Client Compatibility

This Velostrata release is compatible with the following VMware versions:

  • vCenter: 5.5U1, 6.0U1, 6.5.

Note: vCenter is a required component in a deployment.

  • ESXi: 5.5U1, 6.0 U1. Note: 5.0 and 5.1 may be used in certain configurations; contact support for details.

Web Browser Compatibility

The latest versions of the following Web browsers are known to be compatible with the Velostrata vCenter Web Client plugin: Chrome, Internet Explorer, Firefox, Safari.

The vCenter Web Client leverages the Adobe Flash extension in your browser. Velostrata recommends that the Flash plug-in for your browser be updated to the latest version. Use Flash player version 19.0.0.245 or later.

To check your current Flash player version, go to: http://www.adobe.com/software/flash/about/.

Virtual Machine Compatibility

Only virtual machines configured with BIOS virtual firmware are supported. UEFI virtual firmware is not supported. A conversion procedure may be available; contact Velostrata Support.

Velostrata makes use of Virtual Machine snapshots. Virtual Machine disks in Dependent mode (default) and Virtual RDM mode are supported with full functionality.

Virtual Machine disks in Independent mode (persistent and non-persistent) and Physical RDM mode are supported with limited functionality;contact Velostrata Support for details if used.

IDE virtual disks are currentlynot supported. Change the virtual disk type to SCSI to enable move to cloud.

Physical Server Support (Beta)

Migrating a physical server to cloud is done by booting a Velostrata Connector ISO image into the server’s RAM from a virtual or physical DVDROM/CDROM device. The Velostrata connector maps the local storage and creates a Stub VMware VM as a management object for Velostrata cloud migration operations. From this point on, the migration is done in similar way to the migration of other VMs, only in Write Isolation mode.

Note:The Stub VM created in the process is intended for Velostrata management operations only, and not set up for local execution on vSphere. It is set up with no network interface, and a minimal CPU/RAM setting.

System requirements:

  • Disk types supported include SAS, SATA, SSD, virtual disks presented by hardware controller, and SAN volumes mounted on physical HBAs. PATA/IDE disks are not supported.
  • Minimum 4GB RAM.
  • Physical DVDROM/CDROM or virtual CDROM to boot the Velostrata Connector ISO from.

Follow user guide instructions for the physical migration process. For further details, see Migrating Physical Servers.

Guest Operating System Compatibility

Supported Windows Versions:

  • Windows Server 2008 R2 SP1 or higher.
  • Windows Server 2012/Windows Server 2012 R2.
  • Windows 2016. Currently available for AWS only (Beta)

Supported Windows Versions with Offline Migration (see further details in Using Offline Migration):

  • Windows Server 2003, Windows Server 2003R2, Windows Server 2008

Supported Linux Distributions and Versions:

  • RHEL 6.x, RHEL 7.x
  • CentOS 6.x, CentOS 7.x
  • SUSE Linux Enterprise Server 11 SP2 or higher
  • SUSE Linux Enterprise Server 12 SP2 or higher
  • Ubuntu 14.04.x

Note: Ubuntu support for streaming migration was recently introduced in version 2.7. Velostrata welcomes customer feedback on this new capability.

Supported Linux Versions in Offline Migration (see further details in Using Offline Migration):

  • Ubuntu 12.x, Ubuntu 16.x
  • RHEL 5.1 or higher 64-bit RHEL 5.11 and higher supported also with 32-bit

Note: Block device naming considerations differ across target cloud provider and depend on underlying virtualization platforms.Use of either LVM or GUID-based device names in configuration files is recommended. The Velostrata RPM automatically checks for direct device mapping in the typical configuration files (/etc/fstab, and GRUB) and replaces the physical source (e.g. /dev/sda/) with a GUID.

General

Only MBR disks are supported for Guest OS boot disks. Both MBR and GPT disks are supported for data disks. If you have machines with GPT boot disks, contact Velostrata Support for possible conversion options.

#4266: Run-in-cloud and migration operations may fail for Windows Server 2016 workloadwhen Symantec Endpoint Protection (SEP) is installed. This may also happen when SEP appears to be disabled.

Workaround: Modify workload Network interface bindings to remove SEP option 

  1. Download Microsoft Network VSP Bind (nvspbind) - download link
  2. Install Microsoft_Nvspbind_package.EXE into c:\temp
  3. Open Command prompt as an Administrator and run the following: nvspbind.exe /d * symc_teefer2

Known Instance Type Compatibility Issues

AWS

The following AWS instance types may experience missing driver issues. Refer to the AWS driver lists for specific kernel and driver information.

#3681:Workloads running RHEL7.2 fail in AWS instance type g2.8xlarge.

Workaround:Use a different instance type.

Azure

#3779: Workloads runningWindows OS may fail to boot in Azure using A-series VM sizes.

Workaround: Use Av2-series VM sizes.

#4432: Workloads running Windows 2008R2 may fail to boot in Azure using DSv2-series VM sizes.

Workaround: Use alternative VM sizes (e.g. Av2-series and D-series).

Cloud Connectivity

The minimum link bandwidth between on-premises and the Cloud Extension nodes is recommended to be the larger of:

  1. Link bandwidth of 20 Mbit/sec, symmetric
  2. Upload bandwidth to cloud calculated as total number of VMs migrating concurrently, multiplied by 0.5-1 Mbit/sec per VM.

This is required for appropriate access times when moving VMs to the cloud in streaming mode.

For example, Velostrata recommends a minimum of 50-100 Mbit/sec for production use of up to 100 VMs running in the cloud.

Known Limitations

  1. VMware virtual disk consolidation warning - When write-back consistency checkpoints are persisted in the vSphere Datastore on-premises, a snapshot rotation and virtual disk consolidation is performed. During consolidation, the virtual machine will have an event showing that its disks require consolidation. This status is temporary and will automatically get resolved when consolidation triggered by Velostrata has completed. A similar behavior occurs when moving a virtual machine to run back on-premises.
  2. Typical storage throughput per virtual disk - Due to vSphere Storage API limitations, throughput achieved on-premises per VMDK is observed to be limited to about 20-30Mbytes/sec. When workloads are highly concentrated into a single virtual disk, the initial performance seen in the cloud may be limited due to an increase in storage access latency back on-premises. This situation typically resolves itself within minutes, as soon as an active working set is established in the Velostrata cache, in the cloud.
  3. Maximum concurrent read sessions per ESX host - Due to vSphere Storage API limitations, the number of storage access sessions per ESX host is constrained. Velostrata enforces a limit of up to 60 VMDKs or RDMsper ESX host, for virtual machines that run in the cloud (active). Multiple ESX hosts may be used. Given that these virtual machines are not actually executing on the ESX hosts, these hosts can be of minimal specs.
  4. Free space on source datastore - Write back activity for a VM running in the cloud is temporarily put on hold when the backing vSphere Datastore’s free capacity is lower than 10%, and a vCenter Alarm is raised to indicate of the issue. Write back will resume automatically as space is freed up on the datastore

Workaround: Monitor the Uncommitted Writes counter and correlate it to low-disk space events from the vSphere datastore. When write back is put on-hold, the Uncommitted Writes graph indicates a growing count for longer periods.

  1. #3647 - The "collect console log" action button in the Velostrata portlet on the VM Summary page does not work when running the vCenter Web Client on Google Chrome browser, in “Incognito” mode.

Workaround: Work in Chrome normal browsing mode.

Fixed Bugs

Build #14771

#4683: Azure only: Windows server 2012 R2 with OS update KB4025333 fails to boot in azure with Velostrata (boot sequence gets stuck). Issue fixed.

#4679: when no customer keys are defined in the account KMS, migration and prepare-to-detach wizards get stuck. Fixed.

#4684: Tasks cancellation may take few minutes, in which time new VM operations fails. Fixed.

#4677: Velostrata vCenter plug-in may result in connection errors when vCenter is configured in linked mode. Fixed.

#4665: On-prem virtual appliance cannot run on ESXi 5.1 without manual configuration change. Fixed.

#4415:After the system is running for a lengthy period of time, the API authentication may become faulty. This impacts any process that requires use of the API such as Automation Runbook and Powershell. Fixed.

Fixes included in Velostrata prep RPM #1.7-20

#4964: Networking issues after detach for a Linux server with virtual static IP.

Fixed by resetting network configuration for eth0.

#4667: When installing RPM on a physical server, post-detach actions may be executed on-prem. Fixed.

#4705: In migration of physical server with NIC bonding configuration, networking issues encountered after detach.  Fixed by removing the bonding interface.

#4668: SUSE only: In migration of server with a static route configuration, networking issues encountered after detach. Fixed.

#4670: Migration of physical server with Linux OS failed for the following cases:

Hardware specific mount configurations

LVM filters which included /dev/cciss, /dev/mapper but did include /dev/sdX or /dev/xvdX

Fixed.

Build #14427

ID
Description
Comment
4257
Physical server migration is not supported when iSCSI multipath is enabled.
Fixed in a new ISO connector (also works with Velostrata 2.6).
2551
Workload whose disks were shrunk using VMware-Converter is unable to boot in cloud.
Fixed
3412
In some cases, creating a Cloud Extension fails with the following message: "Failed to Create Cloud Extension: Your previous request to create the named bucket succeeded and you already own it…."
Fixed
4012
Storage policy and security group cannot be changed in Spot market bid mode.
Fixed
3265
Too many parallel stop/start VM in cloud requests (API, PowerShell or vCenter WebClient Plugin) may cause operation to time out. 
Fixed
4554
All migration tasks fail when moving back on prem or cancelling the "run-in-cloud" process for a VM that suffers from storage IO issues.  
Fixed
4473
Exporter workers communicate with Velostrata proactive support even when proactive support was set as disabled. 
Fixed
4172
The Migration wizard enables selection of the Cloud Edge node regardless of subnet; this may lead to erroneous user configuration of VM using the Cloud Edge node in the wrong AZ. 
Fixed, now the Cloud Edge node is derived from the selected subnet.
4136
When a large write-back queue was accumulated (e.g. vCenter connection problems), that queue was not managed effectively and created excessive snapshots. 
Fixed by improving batch handling.
4098
Offline migration to AWS is not supported for Windows 2003 with certain instance types that supports enhanced networking by default (C3, C4, D2, I2, R3, and M4 instance types). 
Fixed, enhanced networking option is disabled by Velostrata when workload OS is Windows Server 2003.
4061
3908
In some cases, a test clone operation would fail when the system cannot create snapshot in quiesce mode through vCenter (vmtools disabled). 
Fixed, Velostrata recommends enabling vmtools but will not fail test-clone if it is disabled.
4023
In Create Cloud Extension wizard, not all user roles are displayed properly. 
Fixed.
3960
When opening an instance screenshot window in lower resolution displays, there is no easy way to close it. 
Fixed.
3894
List of storage accounts in prepare to detach wizard is hard to navigate. 
Fixed, list is now sorted alphabetically.
4139
In some cases, the uncommitted writes information was wrong due to sync Issues between management and backend services. 
Fixed
3804
3805
When virtual appliance on prem was configured to use DHCP and later configuration changed to static IP, an error was encountered connecting to the network.
Fixed
4419
2512
After detach for instance with OS RHEL7/CentOS7, active network interface is defined as ibft0 instead of eth0..
Fixed by updating relevant networking config

Known Issues

#1257: When Velostrata Prep RPM is installed on a SUSE Linux Enterprise Server 11, the VM obtains a DHCP IP address in addition to an existing static IP configuration. This issue occurs when the VM is started on-premises in a subnet that is enabled with DHCP services.

Note: The issue does not occur when the subnet has no DHCP services. There is no connectivity impact for communications with the original static IP address.

#1027: In some cases, when a VM is moved to run-in-cloud while a 3rd party VM-level backup solution holds a temporary snapshot, the Velostrata periodic write-back operations will not complete even after the backup solution deletes the temporary snapshot. The uncommitted writes counter on the VM will show an increasing size and no consistency checkpoint will be created on-premises.

Workaround: Select the Run On-Premises action for the VM and wait for the task to complete, which will commit all pending writes. Then select the Run-in-Cloud action again. Note that committing many pending writes may take a while. Do not use the Force option as this will result in the loss of the uncommitted writes.

#1745: After registering the Velostrata plug-in, running the Cloud Extension wizard might generate an error “XXXXXXXXXX” upon “Finish”.

Workaround: Un-register the Velostrata plug-in and restart the vSphere Web client service, then re-register the plug-in. Contact support if issue persists.

#1744: In rare cases, creating a Cloud Extension in Microsoft Azure fails due to the Azure service reporting it is busy.

Workaround: If the Cloud Extension appears in “impaired” state, retry using the “Repair Cloud Extension” operation. If no Cloud Extension entry is shown, retry the create operation.

#1667: vCenter reboot causes Velostrata tasks in vCenter to disappear from UI. This is a vCenter limitation.

Workaround: Use the Velostrata PowerShell module to monitor Velostrata managed VMs or Cloud Extensions tasks that are currently running.

#1951: With an ESXi host in maintenance mode, if a VM is moved to cloud, the operation will fail and get stuck in the rollback phase.

Workaround: Manually cancel the Run-in-Cloud task, migrate the VM to another ESXi host in the cluster and retry the Run-in-Cloud operation.

#1917: A Run-in-Cloud operation fails with the error - "Failed to create virtual machine snapshot. The attempted operation cannot be performed in the current state (Powered off)".

Workaround: The VMware VM snapshot file may be pointing to a non-existent snapshot. Contact support for assistance in correcting the issue.

#1808: In rare cases, after running a Workload back on premises - Workload VMDK's are locked. In certain cases, this is due to network disruptions between the Velostrata management appliance and the ESXi host on which the workload is running.

Note: The issue will resolve itself after 1-2 hours.

#2736: During Detach, the operation might fail with the following error message "Operation was canceled".

Workaround: Retry the Detach operation.

# 3775: When performing a detach after a cancel detach operation, the action may fail.

Workaround: Retry the operation.

#3682: In the event of certain system failures, Velostrata components disconnect from vCenter. In this case, an event may not be sent, resulting in the alarm either not being set properly or not being cleared properly.

Workaround: Clear the alarm manually in vCenter.

#3320: For workloads with Windows machine using a retail license, when returning from the cloud, the license is not present.

Workaround: Reinstall the license.

#3281: vCenter “Export OVA” operation is available when the VM in cloud is running in cache mode, however, this operation results in a corrupted OVA.

Workaround: Only create OVA after the detach.

#3142: In rare cases, when a cloud component instance is created and system fails before it is tagged, the instance will remain untagged. This will not allow full clean-up or repair of the CE.

Workaround: Manually tag the instance, and then run "Repair".

#2596: In Azure, in some cases when canceling the detach operation in the middle of the operation - after the instance is terminated, the VM rolls back on-prem.

Best practice: Instead of cancelling the detach task in process, wait for the task to complete and then run the “Cancel Detach” action.

Workaround: Migrate the VM to the cloud again.

#3988: AWS: When performing detach or reconfigure actions for a VM in dedicated instance tenancy type, and choosing an instance type that is not supported by AWS for dedicated instances, the instance will not be able to run in the cloud.

Workaround: Change the cloud instance type to one that is supported in dedicated instances mode.

#4604: Cloud Extension high availability does not work for workloads running Ubuntu OS with LVM configuration.

#3199: Suse12: Due to a bug in SUSE kernel older than 4.2, configurations that include BTRFS mounts with subvolumes are not supported.
Workaround: upgrade to SUSE version with Kernal >=4.2 (SP2).