Fix Synapse cache auto-tuning variables to use bytes, not KB

Fixup for spantaleev#3017

This reverts 1cd82cf and also multiplies results by `1024`
so as to pass bytes to Synapse, not KB (as done before).

1cd82cf was correctly documenting what we were doing (passing KB values),
but that's incorrect.

Synapse's Config Conventions
(https://matrix-org.github.io/synapse/latest/usage/configuration/config_documentation.html#config-conventions)
are supposed to clear it up, but they don't currently state what happens when you pass a plain number (without a unit suffix).

Thankfully, the source code tells us:
https://github.com/element-hq/synapse/blob/bc1db16086d0718c9c0bb61b32b825ba62049bb0/synapse/config/_base.py#L181-L206

> If an integer is provided it is treated as bytes and is unchanged.
>
> String byte sizes can have a suffix of ...
> No suffix is understood as a plain byte count.

We were previously passing strings, but that has been improved in 3d73ec8.

Regardless, non-suffixed values seem to be treated as bytes by Synapse,
so this patch changes the variables to use bytes.

Moreover, we're moving from `matrix_synapse_memtotal_kb` to
`matrix_synapse_cache_size_calculations_memtotal_bytes` as working with
the base unit everywhere is preferrable.

Here, we also introduce 2 new variables to allow for the caps to be
tweaked:

- `matrix_synapse_cache_size_calculations_max_cache_memory_usage_cap_bytes`
- `matrix_synapse_cache_size_calculations_target_cache_memory_usage_cap_bytes`

CHANGELOG.md

@@ -9,8 +9,8 @@ The playbook has always used a very conservative cache factor (`matrix_synapse_c
 
 The playbook now uses **a 20x larger cache factor** (currently `10`), adjusts a few other cache-related variables, and **enables cache auto-tuning** via the following variables:
 
-- `matrix_synapse_cache_autotuning_max_cache_memory_usage` - defaults to 1/8 of total RAM with a cap of 2GB; values are specified in KB
-- `matrix_synapse_cache_autotuning_target_cache_memory_usage` - defaults to 1/16 of total RAM with a cap of 1GB; values are specified in KB
+- `matrix_synapse_cache_autotuning_max_cache_memory_usage` - defaults to 1/8 of total RAM with a cap of 2GB; values are specified in bytes
+- `matrix_synapse_cache_autotuning_target_cache_memory_usage` - defaults to 1/16 of total RAM with a cap of 1GB; values are specified in bytes
 - `matrix_synapse_cache_autotuning_min_cache_ttl` - defaults to `30s`
 
 These values should be good defaults for most servers, but may change over time as we experiment further.

docs/maintenance-synapse.md

@@ -86,8 +86,8 @@ Tuning the cache factor is useful only to a limited degree (as its crude to do i
 
 Cache autotuning is **enabled by default** and controlled via the following variables:
 
-- `matrix_synapse_cache_autotuning_max_cache_memory_usage` - defaults to 1/8 of total RAM with a cap of 2GB; values are specified in KB
-- `matrix_synapse_cache_autotuning_target_cache_memory_usage` - defaults to 1/16 of total RAM with a cap of 1GB; values are specified in KB
+- `matrix_synapse_cache_autotuning_max_cache_memory_usage` - defaults to 1/8 of total RAM with a cap of 2GB; values are specified in bytes
+- `matrix_synapse_cache_autotuning_target_cache_memory_usage` - defaults to 1/16 of total RAM with a cap of 1GB; values are specified in bytes
 - `matrix_synapse_cache_autotuning_min_cache_ttl` - defaults to `30s`
 
 You can **learn more about cache-autotuning and the global cache factor settings** in the [Synapse's documentation on caches and associated values](https://matrix-org.github.io/synapse/latest/usage/configuration/config_documentation.html#caches-and-associated-values).

roles/custom/matrix-synapse/defaults/main.yml

@@ -557,13 +557,35 @@ matrix_synapse_caches_global_factor: 10
 matrix_synapse_caches_expire_caches: true
 matrix_synapse_caches_cache_entry_ttl: "1080m"
 matrix_synapse_caches_sync_response_cache_duration: "2m"
+
+# Controls how much memory this role thinks is available for cache-size-related calculations.
+# By default, all of the server's memory is taken into account, but you can adjust this.
+# You can also go for directly adjusting cache-sizes (matrix_synapse_cache_autotuning_max_cache_memory_usage, matrix_synapse_cache_autotuning_target_cache_memory_usage) instead of adjusting this.
+matrix_synapse_cache_size_calculations_memtotal_bytes: "{{ (ansible_memtotal_mb * 1024 * 1024) | int }}"
+
+# Controls the cap to use for matrix_synapse_cache_autotuning_max_cache_memory_usage.
+matrix_synapse_cache_size_calculations_max_cache_memory_usage_cap_bytes: "{{ (2 * 1024 * 1024 * 1024) }}" # 2GB
+
+# Controls the cap to use for matrix_synapse_cache_autotuning_target_cache_memory_usage.
+matrix_synapse_cache_size_calculations_target_cache_memory_usage_cap_bytes: "{{ (1 * 1024 * 1024 * 1024) }}" # 1GB
+
 matrix_synapse_cache_autotuning_min_cache_ttl: "30s"
-# The Cache tune math used here is a derivative of the same math used to autotune sizes for postgres.
-# The memtotal variable can in theory be overiden to make Synapse think it has less ram to work with.
-# But if your at the point of considering that just override the math or put static values in.
-matrix_synapse_memtotal_kb: "{{ ansible_memtotal_mb*1024|int }}"
-matrix_synapse_cache_autotuning_max_cache_memory_usage: "{{ (2097152 if (matrix_synapse_memtotal_kb|int/8)/1024 >= 2048 else matrix_synapse_memtotal_kb|int/8) | int }}"
-matrix_synapse_cache_autotuning_target_cache_memory_usage: "{{ (1048576 if (matrix_synapse_memtotal_kb|int/16)/1024 >= 1024 else matrix_synapse_memtotal_kb|int/16) | int }}"
+
+matrix_synapse_cache_autotuning_max_cache_memory_usage: |-
+  {{
+    [
+      (((matrix_synapse_cache_size_calculations_memtotal_bytes | int) / 8) | int),
+      (matrix_synapse_cache_size_calculations_max_cache_memory_usage_cap_bytes | int),
+    ] | min
+  }}
+
+matrix_synapse_cache_autotuning_target_cache_memory_usage: |-
+  {{
+    [
+      (((matrix_synapse_cache_size_calculations_memtotal_bytes | int) / 16) | int),
+      (matrix_synapse_cache_size_calculations_target_cache_memory_usage_cap_bytes | int),
+    ] | min
+  }}
 
 # Controls whether Synapse will federate at all.
 # Disable this to completely isolate your server from the rest of the Matrix network.

roles/custom/matrix-synapse/tasks/validate_config.yml

@@ -92,6 +92,7 @@
     - {'old': 'matrix_synapse_caches_autotuning_max_cache_memory_usage', 'new': 'matrix_synapse_cache_autotuning_max_cache_memory_usage'}
     - {'old': 'matrix_synapse_caches_autotuning_target_cache_memory_usage', 'new': 'matrix_synapse_cache_autotuning_target_cache_memory_usage'}
     - {'old': 'matrix_synapse_caches_autotuning_min_cache_ttl', 'new': 'matrix_synapse_cache_autotuning_min_cache_ttl'}
+    - {'old': 'matrix_synapse_memtotal_kb', 'new': '<superseded by matrix_synapse_cache_size_calculations_memtotal_bytes>'}
 
 - name: (Deprecation) Catch and report renamed settings in matrix_synapse_configuration_extension_yaml
   ansible.builtin.fail: