This page handle the scheduling policy of processes between the different CPUs available on a SEAPATH hypervisor
SEAPATH default CPU isolation
SEAPATH aims to host virtual machine with real time needs. To achieve that, process scheduling must be tuned in order to offer the best performance to the VM.
...
| Info |
|---|
| In the Ansible inventory of the hypervisors, these CPUs are defined by the `isolcpus` variables. |
Tuned
The Debian version of SEAPATH uses tuned (https://github.com/redhat-performance/tuned)
...
On Yocto, tuned is not used. Instead, all these configurations are done at compile time.
Scheduling virtual machines
SEAPATH virtual machines are managed by Qemu.
...
By default, all these threads will be managed by the Linux scheduler and thus run on the non isolated cores. But they can also be pinned to specific CPUs, what forced them to run on it.
Standard virtual machines
For a VM without any performance or real time needs, it is no use to handle any of the Qemu threads a particular way :
All threads will inherit a default priority and scheduling type (TS 19)
All threads will be handled by the Linux scheduler on the non isolated cores
Real time virtual machines
For a VM where performance and determinism is needed, here are our recommendations :
...
For more information read page Virtual machines on SEAPATH.
Finer control with cgroup (optional)
Implementation in SEAPATH
The Linux kernel uses cgroups in order to isolate processes. These cgroups work in a hierarchy where each layer restrains the resources a process can access too. Systemd also uses this mechanism by grouping his processes in slices.
...
TODO : put the link to the inventories README once written
Utility of slices CPU isolation
Using these slices is useful to get a preset of CPU isolation for virtual machines. When placing a VM in either machine-rt or machine-nort slice it will be automatically scheduled on the CPUs of this slice.
It seems particularly useful when deploying many VMs at once.
...
| Info |
|---|
| This new isolation layer protects from really advanced attacks. Because it has drawbacks (see below), the question remains open if you should or not activate this feature. |
Drawbacks
By activating CPU isolation on the machine slice, the management threads of the VM will be scheduled on the allowed CPU list of the slice. This new mechanism implies two things :
...
For more information, read page Virtual machines on SEAPATH.
Specific configurations
NUMA
NUMA (Non-Uniform Memory Access) refers to machines that have the ability to contain several CPU sockets. Each of these sockets has its own cache memory, which means that accessing memory from one socket to another is much slower than accessing memory on its own socket.
...
If your system contains more than one NUMA cells, you must be careful to pin all the vCPU threads of one VM on the same NUMA cell. Otherwise, the data transfer between two cells will significantly slow down the VM.
Hyper-threading
Most of the modern CPUs support hyper-threading. This option can be enabled in the BIOS and double the number of CPUs available on the system. However, the newly created CPUs are not as fast and independent as classic ones.
...
| Info |
|---|
| On most systems, logical CPUs are grouped in numerical order (0 with 1, 2 with 3 …) but this is not always the case. Always refer to `virsh capabilities` to check the exact architecture. |
Annex: list of tuned modifications
Below a list of all scheduling modifications done by tuned.
...