Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

monitor type builder #3510

Closed
ndr-brt opened this issue Oct 4, 2023 · 7 comments · Fixed by #3621
Closed

monitor type builder #3510

ndr-brt opened this issue Oct 4, 2023 · 7 comments · Fixed by #3621
Assignees
Labels
enhancement New feature or request
Milestone

Comments

@ndr-brt
Copy link
Member

ndr-brt commented Oct 4, 2023

Feature Request

Currently the monitor instance is passed to all the components that needs it, but it doesn't get "specialized", so if from a component an info is logged:

monitor.info("interesting information");

on the console will be printed without an indication of the component, this could make logs hard to read.
A current workaround is to manually add a prefix on the log message, but a more holistic solution could be to have a way to build a Monitor instance that will add a prefix, like

var componentMonitor = monitor.forComponent("Component");
new Component(monitor);

then every Component log will have the [Component] prefix

Which Areas Would Be Affected?

e.g., DPF, CI, build, transfer, etc.

Why Is the Feature Desired?

Are there any requirements?

Solution Proposal

If possible, provide a (brief!) solution proposal.

@ndr-brt ndr-brt added feature_request New feature request, awaiting triage triage all new issues awaiting classification labels Oct 4, 2023
@jimmarino
Copy link
Contributor

jimmarino commented Oct 4, 2023

By "component," could we use something more meaningful than the class name since that is not very informative? For example, it could include the extension name and "service" or "subsystem" that the message is emitted from.

This could potentially be handled in the injector, which could create a wrapper implementation.

@ndr-brt
Copy link
Member Author

ndr-brt commented Oct 4, 2023

By "component," could we use something more meaningful than the class name since that is not very informative? For example, it could include the extension name and "service" or "subsystem" that the message is emitted from.

This could potentially be handled in the injector, which could create a wrapper implementation.

Yes, I didn't specify it, the string will valued by the injector, so likely in the extensions's initialize method

@ndr-brt ndr-brt added enhancement New feature or request and removed feature_request New feature request, awaiting triage triage all new issues awaiting classification labels Oct 11, 2023
@ndr-brt ndr-brt added this to the Backlog milestone Oct 11, 2023
@hamidonos hamidonos self-assigned this Nov 9, 2023
@hamidonos
Copy link
Contributor

hamidonos commented Nov 9, 2023

I think the suggested solution works fine, although in my opinion it shouldn't be the monitors responsibility to create new specialized monitors for (extension) components.

Solution 1

The monitor is a singleton provided by the ServiceExtensionContext. We can build a decorator around the Monitor.
The decorator holds information about the context of the log so that
Registered Web API context alias: control
can become
[JerseyExtension]: Registered Web API context alias: control.

By using a decorator we can continue using the original monitor from the service extension context.

Inside the service extension we create the decorator like this:

var serviceExtensionMonitor = new ServiceExtensionMonitor(monitor);

The decorator could look like:

public class ServiceExtensionMonitor implements Monitor {

    private final Monitor monitor;
    private final String context;

    public ServiceExtensionMonitor(Monitor monitor) {
        this.monitor = monitor;
        this.context = getExtensionName();
    }

    // decorate methods of the actual monitor here

    private String getExtensionName() {
        return StackWalker
                .getInstance(StackWalker.Option.RETAIN_CLASS_REFERENCE)
                .walk(stackFrames -> stackFrames
                        .skip(1)
                        .filter(s -> SystemExtension.class.isAssignableFrom(s.getDeclaringClass()))
                        .findFirst()
                        .map(StackWalker.StackFrame::getDeclaringClass)
                        .map(Class::getSimpleName)
                        .orElse(null));
    }
}

The getExtensionName method will walk through the stacktrace to find the parent extension class. The operation is not very time consuming and since this is happening on application bootstrap it won't impact the running application.

By doing this we can avoid the manual configuration of the context where context = extension name.

Note that the context attribute holds the actual value to be prefixed on the log messages.

In some cases we don't want the extension name to be prefixed to the logs but rather the component name or some different value.

For example: All logs written by the TransferProcessManager should be prefixed by TransferProcessManager and not TransferCoreExtension.

This can be done by passing a string to the constructor of the decorator. In those cases the getExtensionName method will be ignored. Similar to what @ndr-brt suggested previously.

    public ServiceExtensionMonitor(Monitor monitor, String context) {
        this.monitor = monitor;
        this.context = context;
    }

var serviceExtensionMonitor = new ServiceExtensionMonitor(monitor, "TransferProcessManager");

Solution 2

It might also be desirable to prefix the class name of the component so that
[JerseyExtension]: Registered Web API context alias: control
could become
[JerseyExtension] [JerseyRestService]: Registered Web API context alias: control

This can come in handy where the monitor is passed further down the component tree of an extension. That way multiple components (or classes) inside an extension can be differentiated.

With Solution 1 this can only be achieved by stack walking through the stacktrace. This would have to take place inside the monitor implementation(s).
In my opinion we should not do that, because this will impact the application performance since stack walking has to be performed for every log operation. Furthermore it shouldn't be the monitors responsibility to resolve the caller class that way.

A better solution could be to have every component instantiate it's own decorator which also holds information about the components class while the extension name gets resolved automatically by stack walking.

    this.monitor = new ServiceExtensionMonitor(monitor, this.class.getName());

or

    this.monitor = new ServiceExtensionMonitor(monitor, "some-other-random-prefix");

This is again done on application start-up, meaning that there is no impact on application performance.

@ndr-brt
Copy link
Member Author

ndr-brt commented Nov 9, 2023

I don't think we should bind it to the ServiceExtension, the majority of monitor calls are done in the services, not in the extensions, so I would call it differently, like PrefixMonitor, and remove that StackWalker logic.

I think the suggested solution works fine, although in my opinion it shouldn't be the monitors responsibility to create new specialized monitors for (extension) components.

We could leave to the users the discover of the existence of such a class, but a method on the Monitor interface that fluently permits would be more intuitive to use, like:

default Monitor withPrefix(String prefix) {
    return new PrefixMonitor(this, prefix);
} 

that could then be used this way:

var transferProcessManagerMonitor = context.getMonitor().withPrefix("TransferProcessManager");

@jimmarino
Copy link
Contributor

I don't think we should bind it to the ServiceExtension, the majority of monitor calls are done in the services, not in the extensions, so I would call it differently, like PrefixMonitor, and remove that StackWalker logic.

I think the suggested solution works fine, although in my opinion it shouldn't be the monitors responsibility to create new specialized monitors for (extension) components.

We could leave to the users the discover of the existence of such a class, but a method on the Monitor interface that fluently permits would be more intuitive to use, like:

default Monitor withPrefix(String prefix) {
    return new PrefixMonitor(this, prefix);
} 

that could then be used this way:

var transferProcessManagerMonitor = context.getMonitor().withPrefix("TransferProcessManager");

I prefer this way as, in addition to being simpler, it decouples the prefix from the actual code artifact emitting the message. For example, what we shouldn't adopt is the traditional approach to logging where prefixes correspond to class names since that is virtually useless from an operations perspective.

@hamidonos
Copy link
Contributor

Alright, I will go with @ndr-brt's suggested solution then

Keep in mind that with @ndr-brt's solution it is possible to chain prefixes, meaning that something like
DEBUG 2023-11-09T15:28:32.332917 [TransferCoreExtension] [TransferProcessManager] : Process 11fb870a-61d9-43a9-a1f5-bff1f74ff08e is now INITIAL
is possible

I will omit the Classname prefix for now unless you think otherwise

@jimmarino
Copy link
Contributor

Alright, I will go with @ndr-brt's suggested solution then

Keep in mind that with @ndr-brt's solution it is possible to chain prefixes, meaning that something like DEBUG 2023-11-09T15:28:32.332917 [TransferCoreExtension] [TransferProcessManager] : Process 11fb870a-61d9-43a9-a1f5-bff1f74ff08e is now INITIAL is possible

I will omit the Classname prefix for now unless you think otherwise

This issue should only address the change @ndr-brt outlined and not include message refactorings of the codebase. Java class names don't make sense to include in messages (except in a stacktrace).

@ndr-brt we should discuss creating a DR for how prefixes should be defined. I prefer the concept of a "component" or "subsystem" that maps to a discrete unit of code artifacts.

hamidonos added a commit to mercedes-benz/EclipseDataSpaceConnector that referenced this issue Nov 16, 2023
jimmarino pushed a commit that referenced this issue Nov 22, 2023
* feat: add PrefixMonitor (#3510)

* fix: fix checkstyle error for PrefixMonitor

* fix: pr comment updates

* fix: pr comment updates

* fix: pr comment updates

* fix: checkstyle error

* fix: remove mockito reset before each test in PrefixMonitorTest

* fix: checkstyle test

* fix: rollback changes in TransferRequest

* fix: add javadoc to Monitor withPrefix method

* fix: change javadoc highlighting
ndr-brt pushed a commit that referenced this issue Dec 6, 2023
* feat: add PrefixMonitor (#3510)

* refactor: remove connector id (#3396)

* fix: add missing "since"

* fix: add missing table name in drop column statement

* fix: remove connector id from tests

* refactor: add deprecated warning to swagger api

* fix: remove wrong change

* fix: remove connector_id from schema creation in schema.sql
ndr-brt pushed a commit that referenced this issue Dec 11, 2023
* feat: add PrefixMonitor (#3510)

* docs: publish data plane public api into dedicated swagger hub page

* fix: remove extensions/data-plane/data-plane-common/README.md

* fix: add missing :extensions:data-plane:data-plane-public-api dependency to tests

* fix: fix pr comments

* fix: fix pr comments

* fix: revert changes in data-transfer.md

* fix: pr fixes
ndr-brt pushed a commit that referenced this issue Mar 20, 2024
* feat: add PrefixMonitor (#3510)

* feat: add token rotation mechanism

* test: adding tests for hashicorp token rotation

* fix: fix checkstyle errors

* fix: set empty list when policies null

* docs: update DEPENDENCIES

* fix: pr comments

* fix: pr comments

* fix: pr comments

* add custom retry mechanism to token renewal

* merge upstream main

* minor fixes

* extract hashicorp config value validation into separate class

* checkstyle fixes

* remove vault timeout config

* clean up

* added ConfigImplTest test for double config

* remove root token property from token look up data

* use in-built retry mechanism of EdcHttpClient for token operations

* using single thread executor for hashicorp client

* remove retry config from HashicorpVaultConfig

* revert config impl changes

* revert SettingResolver changes

* test call of FallbackFactories.retryWhenStatusIsNotIn(..) with static mocks

* implement custom Fallback factory for hashicorp client

* replace token dto classes by map<string, object>

* merge upstream

* update pr comments

* update pr comments

* update pr comments

* clean up merge

* merge main into feature branch
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants