Skip to content

Commit

Permalink
Merge pull request openwallet-foundation#2550 from shaangill025/feat_…
Browse files Browse the repository at this point in the history
…per_tenant_logging

Feat: Per Tenant Logging
  • Loading branch information
dbluhm authored Nov 15, 2023
2 parents 31b6e0b + e57aeaf commit b51d988
Show file tree
Hide file tree
Showing 27 changed files with 550 additions and 447 deletions.
76 changes: 68 additions & 8 deletions Logging.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,29 +12,27 @@ Other log levels fall back to `WARNING`.

* `--log-level` - The log level to log on std out.
* `--log-file` - Path to a file to log to.
* `--log-handler-config` - Specifies `when`, `interval`, `backupCount` for the `TimedRotatingFileMultiProcessHandler`. These 3 attributes are passed as a `;` seperated string. For example, `when` of d (days), `interval` of 7 and `backupCount` of 1 will be passed as `D;7;1`. Note: `backupCount` of 0 will mean all backup log files will be retained and not deleted at all. More details about these attributes can be found [here](https://docs.python.org/3/library/logging.handlers.html#timedrotatingfilehandler). `TimedRotatingFileMultiProcessHandler` supports the ability to cleanup logs by time and mantain backup logs and a custom JSON formatter for logs.
* `--log-fmt-pattern` - Specifies logging.Formatter pattern to override default patterns.
* `--log-json-fmt` - Specifies whether to use JSON logging formatter or text logging formatter. Defaults to `False`.
* `--log-config` - Specifies a custom logging configuration file


Example:

```sh
./bin/aca-py start --log-level debug --log-file acapy.log --log-handler-config "d;7;1" --log-fmt-pattern "%(asctime)s [%(did)s] %(filename)s %(lineno)d %(message)s" --log-json-fmt
./bin/aca-py start --log-level debug --log-file acapy.log --log-config aries_cloudagent.config:default_per_tenant_logging_config.ini

./bin/aca-py start --log-level debug --log-file acapy.log --log-config ./aries_cloudagent/config/default_per_tenant_logging_config.yml
```

## Environment Variables

The log level can be configured using the environment variable `ACAPY_LOG_LEVEL`.
The log file can be set by `ACAPY_LOG_FILE`.
The log config can be set by `ACAPY_LOG_CONFIG`.
The log rotating file handler config can be set by `ACAPY_LOG_HANDLER_CONFIG`.
The log formatter pattern can be set by `ACAPY_LOG_FMT_PATTERN`.
The log json formatter flag can be set by `ACAPY_LOG_JSON_FMT`.

Example:

```sh
ACAPY_LOG_LEVEL=info ACAPY_LOG_FILE=./acapy.log ACAPY_LOG_CONFIG=./acapy_log.ini ACAPY_LOG_HANDLER_CONFIG="d;7;1" ./bin/aca-py start
ACAPY_LOG_LEVEL=info ACAPY_LOG_FILE=./acapy.log ACAPY_LOG_CONFIG=./acapy_log.ini ./bin/aca-py start
```

## Acapy Config File
Expand All @@ -53,6 +51,68 @@ Also if log-level is set to WARNING, connections and presentations will be logge
## Log config file
The path to config file is provided via `--log-config`.

Find an example in [default_logging_config.ini](aries_cloudagent/config/default_logging_config.ini).

You can find more detail description in the [logging documentation](https://docs.python.org/3/howto/logging.html#configuring-logging).

For per tenant logging, find an example in [default_per_tenant_logging_config.ini](aries_cloudagent/config/default_per_tenant_logging_config.ini), which sets up `TimedRotatingFileMultiProcessHandler` and `StreamHandler` handlers. Custom `TimedRotatingFileMultiProcessHandler` handler supports the ability to cleanup logs by time and mantain backup logs and a custom JSON formatter for logs. The arguments for it such as `file name`, `when`, `interval` and `backupCount` can be passed as `args=('acapy.log', 'd', 7, 1,)` [also shown below]. Note: `backupCount` of 0 will mean all backup log files will be retained and not deleted at all. More details about these attributes can be found [here](https://docs.python.org/3/library/logging.handlers.html#timedrotatingfilehandler)

```
[loggers]
keys=root
[handlers]
keys=stream_handler, timed_file_handler
[formatters]
keys=formatter
[logger_root]
level=ERROR
handlers=stream_handler, timed_file_handler
[handler_stream_handler]
class=StreamHandler
level=DEBUG
formatter=formatter
args=(sys.stderr,)
[handler_timed_file_handler]
class=logging.handlers.TimedRotatingFileMultiProcessHandler
level=DEBUG
formatter=formatter
args=('acapy.log', 'd', 7, 1,)
[formatter_formatter]
format=%(asctime)s %(wallet_id)s %(levelname)s %(pathname)s:%(lineno)d %(message)s
```

For `DictConfig` [`dict` logging config file], find an example in [default_per_tenant_logging_config.yml](aries_cloudagent/config/default_per_tenant_logging_config.yml) with same attributes as `default_per_tenant_logging_config.ini` file.

```
version: 1
formatters:
default:
format: '%(asctime)s %(wallet_id)s %(levelname)s %(pathname)s:%(lineno)d %(message)s'
handlers:
console:
class: logging.StreamHandler
level: DEBUG
formatter: default
stream: ext://sys.stderr
rotating_file:
class: logging.handlers.TimedRotatingFileMultiProcessHandler
level: DEBUG
filename: 'acapy.log'
when: 'd'
interval: 7
backupCount: 1
formatter: default
root:
level: INFO
handlers:
- console
- rotating_file
```
3 changes: 3 additions & 0 deletions aries_cloudagent/admin/server.py
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@
from marshmallow import fields

from ..config.injection_context import InjectionContext
from ..config.logging import context_wallet_id
from ..core.event_bus import Event, EventBus
from ..core.plugin_registry import PluginRegistry
from ..core.profile import Profile
Expand Down Expand Up @@ -407,6 +408,8 @@ async def setup_context(request: web.Request, handler):
) = self.multitenant_manager.get_wallet_details_from_token(
token=token
)
wallet_id = profile.settings.get("wallet.id")
context_wallet_id.set(wallet_id)
meta_data = {
"wallet_id": walletid,
"wallet_key": walletkey,
Expand Down
63 changes: 0 additions & 63 deletions aries_cloudagent/config/argparse.py
Original file line number Diff line number Diff line change
Expand Up @@ -1013,46 +1013,6 @@ def add_arguments(self, parser: ArgumentParser):
"('debug', 'info', 'warning', 'error', 'critical')"
),
)
parser.add_argument(
"--log-handler-config",
dest="log_handler_config",
type=str,
metavar="<log-handler-config>",
default=None,
env_var="ACAPY_LOG_HANDLER_CONFIG",
help=(
"Specifies when, interval, backupCount for the "
"TimedRotatingFileHandler. These attributes are "
"passed as a ; seperated string. For example, "
"when of D (days), interval of 7 and backupCount "
"of 1 will be passed as 'D;7;1'. Note: "
"backupCount of 0 will mean all backup log files "
"will be retained and not deleted at all."
),
)
parser.add_argument(
"--log-fmt-pattern",
dest="log_fmt_pattern",
type=str,
metavar="<log-fmt-pattern>",
default=None,
env_var="ACAPY_LOG_FMT_PATTERN",
help=(
"Specifies logging formatter pattern as string. Examples are included "
"in 'Logging.md'. For information regarding different attributes "
"supported in the pattern, please look at "
"https://docs.python.org/3/library/logging.html#logrecord-attributes."
),
)
parser.add_argument(
"--log-json-fmt",
action="store_true",
env_var="ACAPY_LOG_JSON_FMT",
help=(
"Specifies whether to use JSON logging formatter or "
"text logging formatter."
),
)

def get_settings(self, args: Namespace) -> dict:
"""Extract logging settings."""
Expand All @@ -1063,29 +1023,6 @@ def get_settings(self, args: Namespace) -> dict:
settings["log.file"] = args.log_file
if args.log_level:
settings["log.level"] = args.log_level
if args.log_handler_config:
try:
handler_config_attribs = (args.log_handler_config).split(";")
settings["log.handler_when"] = handler_config_attribs[0]
settings["log.handler_interval"] = int(handler_config_attribs[1])
settings["log.handler_bakcount"] = int(handler_config_attribs[2])
except IndexError:
raise ArgsParseError(
"With --log-handler-config, the provided argument must be "
"in 'when;interval;backupCount' format. Each of the 3 "
"attributes for TimedRotatingFileHandler must be specified."
)
except ValueError:
raise ArgsParseError(
"With --log-handler-config, 'interval' and 'backupCount' "
"should be a number [int]"
)
if args.log_fmt_pattern:
settings["log.fmt_pattern"] = args.log_fmt_pattern
if args.log_json_fmt:
settings["log.json_fmt"] = True
else:
settings["log.json_fmt"] = False
return settings


Expand Down
27 changes: 27 additions & 0 deletions aries_cloudagent/config/default_per_tenant_logging_config.ini
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
[loggers]
keys=root

[handlers]
keys=stream_handler, timed_file_handler

[formatters]
keys=formatter

[logger_root]
level=ERROR
handlers=stream_handler, timed_file_handler

[handler_stream_handler]
class=StreamHandler
level=DEBUG
formatter=formatter
args=(sys.stderr,)

[handler_timed_file_handler]
class=logging.handlers.TimedRotatingFileMultiProcessHandler
level=DEBUG
formatter=formatter
args=('/home/aries/log/acapy-agent.log', 'd', 7, 1,)

[formatter_formatter]
format=%(asctime)s %(wallet_id)s %(levelname)s %(pathname)s:%(lineno)d %(message)s
23 changes: 23 additions & 0 deletions aries_cloudagent/config/default_per_tenant_logging_config.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
version: 1
formatters:
default:
format: '%(asctime)s %(wallet_id)s %(levelname)s %(pathname)s:%(lineno)d %(message)s'
handlers:
console:
class: logging.StreamHandler
level: DEBUG
formatter: default
stream: ext://sys.stderr
rotating_file:
class: logging.handlers.TimedRotatingFileMultiProcessHandler
level: DEBUG
filename: '/home/aries/log/acapy-agent.log'
when: 'd'
interval: 7
backupCount: 1
formatter: default
root:
level: INFO
handlers:
- console
- rotating_file
Loading

0 comments on commit b51d988

Please sign in to comment.