• Introduction
• Enabling Live Migration
• Configuring Live Migration
• Performing the Live Migration
  • Cancelling a Live Migration
• What can go wrong?
• Node Eviction
• Conclusion
• References

Introduction

This blog post will be explaining on KubeVirt’s ability to perform live migration of virtual machines.

Live Migration is a process during which a running Virtual Machine Instance moves to another compute node while the guest workload continues to run and remain accessible.

The concept of live migration is already well-known among virtualization platforms and enables administrators to keep user workloads running while the servers can be moved to maintenance for any reason that you might think of like:

• Hardware maintenance (physical, firmware upgrades, etc)
• Power management, by moving workloads to a lower number of hypervisors during off-peak hours
• etc

KubeVirt also includes support for virtual machine migration within Kubernetes when enabled.

Keep reading to learn how!

Enabling Live Migration

To enable live migration we need to enable the feature-gate for it by adding LiveMigration to the key:

apiVersion: v1
kind: ConfigMap
metadata:
  name: kubevirt-config
  namespace: kubevirt
  labels:
  kubevirt.io: ""
data:
  feature-gates: "LiveMigration"

A current kubevirt-config can be edited to append “LiveMigration” to an existing configuration:

kubectl edit configmap kubevirt-config -n kubevirt

data:
  feature-gates: "DataVolumes,LiveMigration"

Configuring Live Migration

If we want to alter the defaults for Live-Migration, we can further edit the kubevirt-config like:

apiVersion: v1
kind: ConfigMap
metadata:
  name: kubevirt-config
  namespace: kubevirt
  labels:
  kubevirt.io: ""
data:
  feature-gates: "LiveMigration"
  migrations: |-
  parallelMigrationsPerCluster: 5
  parallelOutboundMigrationsPerNode: 2
  bandwidthPerMigration: 64Mi
  completionTimeoutPerGiB: 800
  progressTimeout: 150

Parameters are explained in the below table (check the documentation for more details):

Performing the Live Migration

Live migration is initiated by posting an object VirtualMachineInstanceMigration to the cluster, indicating the VM name to migrate, like in the following example:

apiVersion: kubevirt.io/v1alpha3
kind: VirtualMachineInstanceMigration
metadata:
  name: migration-job
spec:
  vmiName: vmi-fedora

This will trigger the process for the VM.

Status:
  Conditions:
    Status: True
    Type: LiveMigratable
  Migration Method: BlockMigration

If the VM is Live-Migratable, the request will submit successfully. The status change will be reported under VMI.status. Once live migration is complete, a status of Completed or Failed will be indicated.

Cancelling a Live Migration

If we want to abort the Live Migration, ‘Kubernetes-Style’, we’ll just delete the object we created for triggering it.

In this case, the VM status for migration will report some additional information:

Migration State:
  Abort Requested: true
  Abort Status: Succeeded
  Completed: true
  End Timestamp: 2019-03-29T04:02:49Z
  Failed: true
  Migration Config:
    Completion Timeout Per GiB: 800
    Progress Timeout: 150
  Migration UID: 57a693d6-51d7-11e9-b370-525500d15501
  Source Node: node02
  Start Timestamp: 2019-03-29T04:02:47Z
  Target Direct Migration Node Ports:
    39445: 0
    43345: 49152
    44222: 49153
  Target Node: node01
  Target Node Address: 10.128.0.46
  Target Node Domain Detected: true
  Target Pod: virt-launcher-testvmimcbjgw6zrzcmp8wpddvztvzm7x2k6cjbdgktwv8tkq

Note that there are some additional fields that indicate that Abort Requested happened and in the above example that it has Succeded, in this case, the original fields for migration will report as Completed (because there’s no running migration) and Failed set to true.

What can go wrong?

Live-migration is a complex process that requires transferring data from one ‘VM’ in one node to another ‘VM’ into another one, this requires that the activity of the VM being live-migrated to be compatible with the network configuration and throughput so that all the data can be migrated faster than the data is changed at the original VM, this is usually referred to as converging.

Some values can be adjusted (check the table for settings that can be tuned), to allow it to succeed but as a trade-off:

• Increasing the number of VMs that can migrate at once, will reduce the available bandwidth.
• Increasing the bandwidth could affect applications running on that node (origin and target).
• Storage migration (check the Info note in the Performing the Live Migration section on the differences) might also consume bandwidth and resources.

Node Eviction

Sometimes, a node requires to be put on maintenance and it includes workloads on it, either containers or, in KubeVirt’s case, VM’s.

It is possible to use selectors, for example, move all the virtual machines to another node via kubectl drain <nodename>, for example, evicting all KubeVirt VM’s from a node can be done via:

kubectl drain <node name> --delete-local-data --ignore-daemonsets=true --force --pod-selector=kubevirt.io=virt-launcher

According to documentation, --delete-local-data, --ignore-daemonsets and --force are required because:

• Pods using emptyDir can be deleted because the data is ephemeral.
• VMI will have DaemonSets via virt-handler so it’s safe to proceed.
• VMIs are not owned by a ReplicaSet or DaemonSet, so kubectl can’t guarantee that those are restarted. KubeVirt has its own controllers for it managing VMI, so kubectl shouldn’t bother about it.

If we omit the --pod-selector, we’ll force eviction of all Pods and VM’s from a node.

spec:
  evictionStrategy: LiveMigrate

Conclusion

As a briefing on the above data:

• LiveMigrate needs to be enabled on KubeVirt as a feature gate.
• LiveMigrate will add status to the VMI object indicating if it’s a candidate or not and if so, which mode to use (Block or Live)
  • Based on the storage backend and other conditions, it will enable LiveMigration or just BlockMigration.

References

• Live Migration
• Node Drain/Eviction
• Rook