Add support for per-component log file (Issue #8306, PR #8471)

# Description

Add configurable logging.

closes #8306
closes #8307

closes #8356

## includes
1. per-component log files via the config file
```ini
[logging]
server = server_log.yml
scheduler = scheduler.log.tmpl
```
2. tool to dump log file ` inmanta print_default_logging_config [compiler|scheduler|server]`
3. moved resource action log to scheduler

## Questions and results
1. [x] do we want comments in the files outputted by the tool?
           - comments are in the files in misc, not in the tool output
3. [x] do we want to enforce a naming convention on the logger. (it is in the code, but I wouldn't, as we don't have a lot of feeling with this )
         - no: we don't know logging well enogu
5. [x] should `-v` work when using the config file?
          - force in a cli logger at this level

## Next steps

1. [ ] remove all resource action alike functions from the server
2. [ ] add a way to validate config files for specific tools (like scheduler) -> #8482
6. [ ] Some methods in the stable api of the logging are deprecated, do we drop them? -> #8485
7. [ ] Hook on LSM  #8492
8. [ ] TTY forcing? -> #8484
9. [ ] load config from env var -> #8308
10. [ ] document logging config -> #8493

# Self Check:

Strike through any lines that are not applicable (`~~line~~`) then check the box

- [x] Attached issue to pull request
- [x] Changelog entry
- [x] Type annotations are present
- [x] Code is clear and sufficiently documented
- [x] No (preventable) type errors (check using make mypy or make mypy-diff)
- [x] Sufficient test cases (reproduces the bug/tests the requested feature)
- [x] Correct, in line with design
- [ ] End user documentation is included or an issue is created for end-user documentation (add ref to issue here: ) -> #8493
- [ ] If this PR fixes a race condition in the test suite, also push the fix to the relevant stable branche(s) (see [test-fixes](https://internal.inmanta.com/development/core/tasks/build-master.html#test-fixes) for more info)

changelogs/unreleased/8306-per_component_logging.yml

@@ -0,0 +1,8 @@
+description: Add support for per-component log file
+issue-nr: 8306
+change-type: minor
+destination-branches: [master]
+sections:
+  minor-improvement: "{{description}}"
+
+

misc/inmanta.cfg

@@ -1,25 +1,10 @@
 [config]
 # The directory where the server stores its state
 state-dir=/var/lib/inmanta
-
 # The directory where the server stores log file. Currently this is only for the output of
 # embedded agents.
 log-dir=/var/log/inmanta
 
-# Force the hostname of this machine to a specific value
-#node-name =
-
-# The deploy interval of the agent
-agent_deploy_interval = 600
-
-# The splaytime added to the agent_deploy_interval. Set this to 0 to disable splaytime
-agent_deploy_splay_time = 600
-
-# The repair interval of the agent
-agent_repair_interval = 86400
-
-# The splaytime added to the agent_repair_interval. Set this to 0 to disable splaytime
-agent_repair_splay_time = 600
 
 [database]
 # The hostname of the database server
@@ -68,10 +53,6 @@ fact-expire = 3600
 # than fact-expire
 #fact-renew = 1200
 
-# On boot and at regular intervals the server will purge versions that have not been deployed. This
-# is the number of most recent undeployed versions to keep available.
-#available-versions-to-keep=10
-
 # The number of seconds between version purging
 #purge-versions-interval=3600
 
@@ -90,3 +71,4 @@ host = localhost
 port = 8888
 #ssl=false
 #ssl_ca_cert_file=
+

misc/scheduler_log.yml.tmpl

@@ -0,0 +1,57 @@
+# Generated using inmanta -c /etc/inmanta/inmanta.cfg --log-file-level DEBUG --timed-logs print_default_logging_config scheduler
+formatters:
+  core_console_formatter:
+    # Console formatter with coloring
+    (): inmanta.logging.MultiLineFormatter
+    fmt: '%(log_color)s%(name)-25s%(levelname)-8s%(reset)s%(blue)s%(message)s'
+    # Don't shorten logger names by default
+    keep_logger_names: true
+    log_colors:
+      CRITICAL: red
+      DEBUG: cyan
+      ERROR: red
+      INFO: green
+      WARNING: yellow
+    no_color: false
+    reset: true
+  # Normal formatter
+  core_log_formatter:
+    format: '%(asctime)s %(levelname)-8s %(name)-10s %(message)s'
+handlers:
+  core_tornado_debug_log_handler:
+    # Special handler that captures tornado max_clients limit reached messages and turns them into warnings
+    # As these may cause instability
+    class: inmanta.logging.TornadoDebugLogHandler
+    level: DEBUG
+  scheduler_handler:
+    # Root handler for the scheduler
+    class: logging.handlers.WatchedFileHandler
+    filename: /var/log/inmanta/agent-{environment}.log
+    formatter: core_log_formatter
+    level: DEBUG
+    mode: a+
+  scheduler_resource_action_handler:
+    # log only the resource actions as reported by the executors, live
+    class: logging.handlers.WatchedFileHandler
+    filename: /var/log/inmanta/resource-actions-{environment}.log
+    formatter: core_log_formatter
+    level: DEBUG
+loggers:
+  inmanta.resource_action:
+    # parent for all resource action logs
+    # name is inmanta.resource_action.[agentname] for all agents
+    handlers:
+    - scheduler_resource_action_handler
+    level: DEBUG
+    propagate: true
+  tornado.general:
+    handlers:
+    - core_tornado_debug_log_handler
+    level: DEBUG
+    propagate: true
+root:
+  handlers:
+  - scheduler_handler
+  level: DEBUG
+version: 1
+disable_existing_loggers: false

misc/server_log.yml

@@ -0,0 +1,45 @@
+# Generated using inmanta -c /etc/inmanta/inmanta.cfg --log-file /var/log/inmanta/server.log --log-file-level 2 --timed-logs print_default_logging_config server
+formatters:
+  core_console_formatter:
+    # Console formatter with coloring
+    (): inmanta.logging.MultiLineFormatter
+    fmt: '%(log_color)s%(name)-25s%(levelname)-8s%(reset)s%(blue)s%(message)s'
+    # Don't shorten logger names by default
+    keep_logger_names: true
+    log_colors:
+      CRITICAL: red
+      DEBUG: cyan
+      ERROR: red
+      INFO: green
+      WARNING: yellow
+    no_color: false
+    reset: true
+  # Normal formatter
+  core_log_formatter:
+    format: '%(asctime)s %(levelname)-8s %(name)-10s %(message)s'
+handlers:
+  server_handler:
+    # Main handler for the server
+    class: logging.handlers.WatchedFileHandler
+    filename: /var/log/inmanta/server.log
+    formatter: core_log_formatter
+    level: INFO
+    mode: a+
+  core_tornado_debug_log_handler:
+    # Special handler that captures tornado max_clients limit reached messages and turns them into warnings
+    # As these may cause instability
+    class: inmanta.logging.TornadoDebugLogHandler
+    level: DEBUG
+loggers:
+  tornado.general:
+    handlers:
+    - core_tornado_debug_log_handler
+    level: DEBUG
+    propagate: true
+root:
+  handlers:
+  - server_handler
+  level: INFO
+
+version: 1
+disable_existing_loggers: false