Add updated tuning configuration

Uses a much nicer CPU tuning configuration, leveraging systemd's
AllowedCPUs and CPUAffinity options within a set of slices (some
default, some custom).

Configuration is also greatly simplified versus the previous
implementation, simply asking for a number of CPUS for both the system
and OSDs, and calculating everything else that is required.

Also switches (back) to the v2 unified cgroup hierarchy by default as
required by the systemd AllowedCPUs directive.

group_vars/default/pvc.yml

@@ -87,6 +87,9 @@ pvc_nodes:
     ipmi_host: "{{ ipmi['hosts']['pvchv1']['hostname'] }}" # Note the node hostname key in here
     ipmi_user: "{{ ipmi['users']['pvc']['username'] }}"
     ipmi_password: "{{ ipmi['users']['pvc']['password'] }}"
+    cpu_tuning:             # Example of cpu_tuning overrides per-node, only relevant if enabled; see below
+      system_cpus: 2
+      osd_cpus: 2
   - hostname: "pvchv2" # This name MUST match the Ansible inventory_hostname's first portion, i.e. "inventory_hostname.split('.')[0]"
     is_coordinator: yes
     node_id: 2
@@ -146,6 +149,37 @@ pvc_sriov_enable: False
 #            Lowering the stack limit may cause poor performance or crashes in Zookeeper during some tasks.
 #pvc_zookeeper_stack_limit: 256M  # 1/4 of default
 
+# CPU tuning
+# > ADVANCED TUNING: For most users, this is unnecessary and PVC will run fine with the default CPU
+#   allocations. Adjust these options only for clusters where CPU optimization is needed.
+# > Defines CPU tuning/affinity options for various subsystems within PVC. This is useful to
+#   help limit the impact that noisy elements may have on other elements, e.g. busy VMs on
+#   OSDs, or system processes on latency-sensitive VMs.
+# > To enable tuning, set enabled to yes.
+# > Within "nodes", two counts are specified:
+#    * system_cpus: The number of CPUs to assign to the "system" slice, i.e. all non-VM,
+#                   non-OSD processes on the system. Should usually be at least 2, and be
+#                   higher on the coordinators of larger clusters (i.e. >5 nodes).
+#    * osd_cpus: The number of CPUs to assign to the "osd" slice, i.e. all OSD processes.
+#                Should be at least 1 per OSD, and ideally 2 per OSD for best performance.
+#   A third count, for the VM CPUs, is autogenerated based on the total node CPU count and
+#   the above two values (using all remaining CPUs).
+# > Tuning is done based on cores; for systems with SMT (>1 thread-per-core), all SMTs within
+#   a given core are also assigned to the same CPU set. So for example, if the system assigns
+#   2 system_cpus, there are 16 cores, and there are 2 threads per core, the list will be:
+#     0,1,16,17
+#   leveraging the assumption that Linux puts all cores before all threads.
+# > This tuning section under "nodes" is global to the cluster; to override these values on
+#   a per-node basis, use the corresponding "cpu_tuning" section of a given "pvc_nodes" entry
+#   as shown below.
+# > If disabled after being enabled, the tuning configurations on each node will be removed
+#   on the next run. A reboot of all nodes is required to fully disable the tuning.
+cpu_tuning:
+  enabled: no
+  nodes:
+    system_cpus: 2      # Set based on your actual system configuration (min 2, increase on coordinators if many nodes)
+    osd_cpus: 2         # Set based on your actual number of OSDs
+
 # Configuration file networks
 # > Taken from base.yml's configuration; DO NOT MODIFY THIS SECTION.
 pvc_upstream_device: "{{ networks['upstream']['device'] }}"