Configuration

The inventory may define these hosts:

• observers: Set of hosts to observe the cluster (only the first is considering)
• Hypervisors: Set of machines to hosts VMs.

Node redundancy

All nodes in the cluster have access to a shared storage via Ceph (see Shared storage section). With it, the cluster is in N to N redundancy mode.

• Corosync will provides messaging and membership services.
• Pacemaker will manage the cluster (synchronize resources between each node).

Image Added

More details on Pacemaker here and Corosync here.

pacemaker-remote

pacemaker-remote is a component which can be installed inside a VM to allow Pacemaker to manage and monitor resources inside this VM.

For instance, with pacemaker-remote pacemaker can monitor services and containers directly inside a VM.

For more information about pacemaker-remote refer to https://clusterlabs.org/pacemaker/doc/2.1/Pacemaker_Remote/singlehtml/.

Management tool

The vm_manager project is an high-level interface of pacemaker and ceph Ceph to manage the VM like a resource. He is installed during the installation step and provides the vm-mgr command.

Sub-command

All sub-commands has -n, --name required option to specify which resource should be used.

• add_colocation: Adds a colocation constraints constraint between resources

• clone: Creates a copy of the VM

• create: Generates a new resource from a VM

• create_snapshot: Creates a snapshot of a resource

• disable: Stops and removes the resource on the cluster

• enable: Adds and starts the resource on the cluster

• get_metadata: Gets a metadata of a resource

• list: Lists all resources

• list_metadata: Lists all keys of a resource

• list_snapshots: Lists all created snapshots

• purge: Deletes all snapshots of a resource

• remove: Removes the resource

• remove_snapshot: Removes a snapshot of a resource

• rollback: Rollbacks to a snapshot for a resource

• set_metadata: Sets a metadata of a resource

• start: Start a resource

• status: Gets the status of a resource

• stop: Stops a resource

Resources status

• Undefined:
• Disabled:
• Failed:
• Started:
• Starting:
• Stopping:
• Stopped:

Manage VM

Add VM in the cluster:

vm-mgr create --name NAME --xml /path/to/configuration.xml --image /path/to/disk.qcow2 --disable --force --enable-live-migration --migration-user virtu --migration-to-timeout 180

Check the execution of the resource:

crm status

Get the status of the resource:

vm-mgr status --name NAME

Delete VM in the cluster:

vm-mgr remove --name NAME

...

Replace a dead node

The ansible/playbooks/replace_machine_remove_machine_from_cluster.yaml playbook can remove a node in the cluster. For this, the machine_to_remove should be set to the hostname to remove.
The below command should be launch in the ansible project.

cqfd run ansible-playbook -i /path/to/inventory.yaml -e machine_to_remove=HOSTNAME playbooks/replace_machine_remove_machine_from_cluster.yaml

...