LIP: 0005
Title: Introduce new flexible, resilient and modular architecture for Lisk Core
Author: Nazar Hussain <[email protected]>
Discussions-To: https://research.lisk.com/t/introduce-new-flexible-resilient-and-modular-architecture-for-lisk-core/
Status: Obsolete
Type: Informational
Created: 2018-09-06
Updated: 2021-09-07
This LIP proposes a new application architecture for Lisk Core, that is of a flexible and modular design. The goals of the new application architecture are:
- Looser coupling between modules through functional isolation.
- Optional elastic scaling for modules over multiple cores, threads or machines.
- Easier extensibility through the use of a plugin pattern and a supporting API.
- Increased resilience to individual module processing failure.
This LIP is licensed under the Creative Commons Zero 1.0 Universal.
Lisk Core is a NodeJS application running on a single process. The limitation of single-process architectures and tightly coupled code logic can have different impacts on the system. For example, consider the following scenarios:
- We cannot utilize all available hardware cores, as long as the application runs in a single process due to the nature of NodeJS.
- Due to tightly coupled code, we cannot easily refactor any particular module without an impact on the whole application.
- We cannot ensure that each individual component of the application remains functional, whilst any other component (or more than one component) faces a problem.
- If we face a heavy load on the HTTP API, which blocks the resources, this will impact the performance of block propagation and other components.
Such problems encouraged us to orchestrate new flexible, easy to manage, scalable and resilient architecture for Lisk Core.
When designing the architecture for a distributed and decentralised system, a few points need to be considered:
- Different modules could be deployed to different machines in the future. Moreover, communication between modules cannot be assumed to be reliable. Therefore, the communication between modules should be fail-safe. In case of any network failure, a module should check and try to recover automatically.
- There is always latency in the communication between modules, so the code should expect it and handle it properly.
- We have no control or direct guidance over how most individuals install the software, so the distribution should be easy to install.
- A corollary of the previous point is that we have no control over the physical machines that run Lisk Core, so we should aim to build software which can work well with a range of physical resources.
- All systems are susceptible to unplanned crashes, so this architecture should be resilient in such cases and support fail-over.
Taking note of the above points, aim for redesigning the architecture of the Lisk Core is to achieve the following:
- Identify components which should stay functionally isolated from each other.
- Design an architecture such that functionally isolated components can form the basis of a multi-process application, in order to utilise the potential of multiple hardware cores of the physical processor if available.
- Design each component in a resilient way to tackle brittleness of the multi-processing. This means that a failure of one component will have minimal impact on other components and that components can recover individually.
- Most of the components should scale elastically depending upon the available physical resources.
- Individual components should be flexible enough to be installed using the plugin pattern.
- Lay a foundation on which instances of the Lisk Core application can be scaled to run different components on different physical machines and still operate in a mutually exclusive manner.
- Provide an elegant API which can be extended easily when creating new components for the Lisk Core application.
- Provide a reliable framework on which new variants of the Lisk Core application can be based upon.
These considerations have led us to the following architecture:
+---------------------------------------------------------------------+
| LISK CORE |
|+-------------------------------------------------------------------+|
|| MODULES ||
|| ||
||+-------------------------------+ +-------------------------------+||
||| | | |||
||| CORE MODULES | | PLUGGABLE MODULES |||
||| | | |||
||+-------------------------------+ +-------------------------------+||
|+-------------------------------------------------------------------+|
| /|\ |
| / | \ |
| | CHANNELS |
| \ | / |
| \|/ |
|+-------------------------------------------------------------------+|
|| COMPONENTS ||
|+-------------------------------------------------------------------+|
|| CONTROLLER ||
|+-------------------------------------------------------------------+|
+---------------------------------------------------------------------+
Here you can find the specification for each component of the architecture in details.
Lisk Core in the above diagram denotes the complete ecosystem of the application as composed by various components. The components should be connected with each other to work and drive the blockchain.
The Controller will be a parent process responsible for managing every user interaction with each component of the ecosystem. E.g. restarting the node, starting a snapshot process, etc. It is derived by an executable file which will be the entry point to interact with Lisk Core.
- The Controller will be responsible for initialization of infrastructure-level components: Database, Cache, Logger, System.
- The Controller will also initialize each module separately. If any module is configured to load as a child process, then it is the Controller's responsibility to do so.
- The Controller will define a set of events, such that each component can subscribe in the same process, or over an IPC channel in case of a child process. Most of the data flow will be handled through the propagation of such events.
- Each module can also define its own custom events or actions and will register that list with the Controller at the time of initialization. Thus the Controller will have a complete list of events which may occur in the modules of Lisk Core at any given time.
Components are shared objects within the Controller layer which any module can utilize. Component can use channels if required for implementation behavior. The following components are proposed currently.
This component will be responsible for all database activity in the system. This component will expose an interface with specific features for getting or setting particular database entities. It will also expose a raw handler to the database object so that any module can extend it for its own use. Insert a reference to child DB entities LIP.
Logger will be responsible for all application-level logging activity and log everything in JSON format. This central logger component can be passed to any module, where it can be extended by adding module-specific fields.
This component will provide basic caching capabilities, generic enough for any module to use if required.
This component will provide a central registry of up-to-date system information. Especially network height, nonce, broadhash, nethash and network specific constants. This component will use channels and events to make all instances of the component stay in sync in different modules.
Modules are a vital part of the proposal. These will contain all of the business logic and operational code for the ecosystem. Each module can reside within the main Controller process or can designate that it should be spawned as a child process. This will enable the Lisk Core instance to distribute the processing and utilize multiple cores.
Modules can be further categorized into two types:
Core Modules should be shipped along with the Lisk Core distribution itself. These modules should constitute the minimum requirements to run a functional Lisk Core instance.
Pluggable Modules should be distributed separately, so that they can be plugged into any Lisk Core instance and can be removed/disabled at any time. These should extend the existing instance with a specific (and circumscribed) set of features. The community is also provided with the opportunity to develop their own modules for Lisk Core and distribute them as npm packages.
The implementation details of a module are ultimately up to the module developer, but by default, a module must export an object from main entry file of package.json adhering to the following structure:
// Exported from the main file of the JavaScript package
export default {
/**
* A unique module name accessed throughout out the system.
* If some module has already been registered with the same alias, an error will be thrown.
*/
alias: "moduleName",
/**
* Package information containing the version of the software and other details.
* The easiest way is to refer to the relevant package.json.
*/
pkg: require("../package.json"),
/**
* Supported configurations for the module with default values.
*/
defaults: {},
/**
* List of valid events to register with the Controller.
* Once the application is running, each event name will be prefixed by the module’s alias, e.g. `moduleName:event1`.
* Any module running on the instance will be able to subscribe or publish these events.
*/
events: [],
/**
* List of valid actions to register with the Controller.
* Once the application is running, each action name will be prefixed by the module’s alias, e.g. `moduleName:action1`.
* Action definition can be provided on module load with the help of the channels.
* Source module can define the action while others can invoke that action.
*/
actions: [],
/**
* The method to be invoked by Controller to load the module.
* Module developers should ensure that all loading logic is completed during the lifecycle of this method.
* The Controller will emit an event `lisk:ready` which a module developer can use to perform some activities
* which should be performed when every other module is loaded. Some activities which you want to perform when
* every other module is loaded.
*
* @param {Channel} channel - An interface to a channel
* @param {Object} options - An object of module options
* @return {Promise<void>}
*/
load: async (channel, options) => {},
/**
* The method to be invoked by the Controller to perform cleanup of the module.
*
* @return {Promise<void>}
*/
unload: async () => {}
};The following events and actions should be implemented in the redesigned Lisk Core and be accessible by all modules.
| Event | Description |
|---|---|
| module:registeredToBus | Triggered when the module has completed registering its events and actions with the controller. So when this event is triggered, the subscriber of event can be sure that the Controller has whitelisted its requested events and actions. |
| module:loading:started | Triggered just before the Controller calls the module’s load method. |
| module:loading:error | Triggered if any error occurred during call of the module's load method. |
| module:loading:finished | Triggered just after the module’s load method has completed execution. |
| module:unloading:started | Triggered just before the Controller calls the module’s unload method. |
| module:unloading:error | Triggered if any error occurred during call of module’s unload method. |
| module:unloading:finished | Triggered just after the module’s unload method has completed execution. |
| lisk:ready | Triggered when the Controller has finished initialising the modules and each module has been successfully loaded. |
| Action | Description |
|---|---|
| lisk:getComponentConfig | A controller action to get the configuration of any component defined in controller space. |
The module life cycle consists of the following events in the order listed below, assuming two modules m1 and m2 are defined to be loaded.
Loading
- m1:registeredToBus
- m1:loading:started
- m1:loading:finished
- m2:registeredToBus
- m2:loading:started
- m2:loading:finished
- lisk:ready
Unloading
- m1:unloading:started
- m1:unloading:finished
- m2:unloading:started
- m2:unloading:finished
For the initial implementation, sequential loading/unloading is recommended as shown above. The feasibility of loading modules in parallel could be researched as a potential future improvement.
Modules will communicate to each other through channels. These channels will be event-based, triggering events across various listeners. Modules running in different processes will communicate with each other over IPC channels.
Every module export a load method, which accepts two parameters: a channel and an options object. The options object is simply the JSON object containing the module specific options provided in the config file.
The channel parameter will be an instance of a channel and its type depends upon the type of module. For now, we propose two types of channels:
| Channel Type | Description |
|---|---|
| EventEmitterChannel | Communicates with modules which reside in the same process as the Controller. |
| ChildProcessChannel | Communicates with modules which do not reside in the same process as the Controller. |
The Controller will be responsible for creating channels of the relevant type depending on how it loads each module.
Whichever channel implementation the module receives when it's load method is called, it must expose a consistent interface defining the at least following four methods.
Used to subscribe to events occurring on the controller.
channel.subscribe("lisk:ready", event => {});This function accepts two arguments. The first is the event name prefixed with the name of the relevant module. The second argument is a callback which accepts one argument, which will be an instance of an event object.
Used to publish events to the controller, which will be delivered to all events subscribers.
channel.publish("chain:newTransaction", transactionObject);This function accepts two arguments. The first one is the event name prefixed with the name of the relevant module. The second argument is the data object to be passed along the event.
Defines an action for the module, which can be invoked later by other modules.
channel.action("verifyTransaction", async action => {});This function accepts two arguments. The first one is the action name without prefixing a module name. As the current module’s name will always be prefixed when the Controller registers the action. An action cannot be defined for an external module. The second argument is a callback which accepts one argument, which will be an instance of an action object.
Used to invoke an action for a module.
result = await channel.invoke('chain:verifyTransaction', transactionObject);This function accepts two arguments. The first one is the event name prefixed with the name of the relevant module. The second argument is the data object to be passed along the action.
Event objects should conform to a unified interface for all event communication between modules. Each event must implement a serialize and deserialize mechanism so that a unified data format can be transported over channels. It should be a simple JavaScript object with the following attributes.
| Property | Type | Description |
|---|---|---|
| name | string | The name of the event which is triggered. |
| module | string | The name of the target module for which event was triggered. |
| source | string | The name of source module which published that event. |
| data | mixed | The data which was sent while publishing the event. |
Action object should be a unified interface for all action based communication between modules. Actions must implement a serialize and deserialize mechanism to get a unified data format to be transported over channels. It should be a simple javascript object with attributes listed below.
| Property | Type | Description |
|---|---|---|
| name | string | Name of the action which is invoked. |
| module | string | The name of the target module for which action was invoked. |
| source | string | The name of source module which invoked that action. |
| params | mixed | The data which was associated with the invocation for the action. |
This proposal is intended to conform to the existing blockchain protocol specification without any amendments. So it will be 100% backwards compatible at the point when this proposal is adopted.
How deep should the segregation of functionality be?
In the first phase of implementation, the suggestion is to open three separate modules, which will be run in child processes. Once this reorganisation is complete, we can investigate how to best improve the architecture by dividing a module into further modules.
How will debugging work with this architecture?
Nothing will change in regard to debugging. User will start the whole ecosystem of modules with one command and will see consolidated logs on a console.
For debugging IPC channels, we could add extensive logging to log any activity on the controller, so we can deeply track inter-process communication. For interactive debugging, all native Node.js debugging features are intended to work with this architecture.
Will the modules be used in other products?
Any module we would create is designed to be used in Lisk Core. As every module is dependent on Lisk Controller to be available, it is not an intended use case to run Lisk Core modules as part of other products like Lisk Commander.
As each module will have a well defined set of events, actions and protocol to communicate. So if used properly, a modules's functionality can be used in other Node.js projects or sidechains.
Which tool will we use for IPC communication?
We are not finalizing any tool at the moment to implement the IPC channel concept. Probable and available options are a custom node implementation, the PM2 implementation or to look for any other tool for this purpose. We will probably experiment with all options to choose the best one for our architecture.
What is the database component?
The database component will be used to perform any kind of RDBMS activity. We call it component because it will be initialized and stay available on the controller layer to be utilized by any other module (the way we are doing right now).
Modules, which are spawned as child processes, create an instance of this component on their own. For creating a new instance, a module can either pass a custom configuration or ask the controller to share only the configuration for the database (json object). So the respective module can use the same configuration, override or pass a custom configuration. In the end, each module will have its own instance of the database component.
How to refactor the current code base?
The above architecture requires a considerable code changes. A viable plan is defined as a milestone and can be found at LiskArchive/lisk-modular#12