Instantiate objects and call functions using dictionary configs in Python using Genos. This package was originally developed to help python developers in making configurable software components.
While Hydra lets you instantiate objects and functions, it doesn't support recursive instantiation. Plus, Hydra is mostly used for config management. So, we decided to build Genos by referring to Hydra and added the functionality of recursive instantiation. E.g.,
Install Genos
pip install genosInstantiate a Python Class using Genos
from genos import recursive_instantiate
ned = {
"cls": "genos.examples.King",
"params":{
"name": "Eddard Stark",
"queen": "Catelyn Stark",
"allegiance": "Robert Baratheon"
}
}
obj = recursive_instantiate(ned)
print(obj)
pip install genospoetry add genosThe following examples will show how this library can make your life easier by letting you instantiate python objects from dictionaries. First, let's consider a basic example where we simply instantiate a single class.
class King:
def __init__(self, name:str, queen:str, allegiance:str):
self.name = name
self.queen = queen
self.allegiance = allegiance
def __repr__(self):
return f"Name:{self.name}\nQueen:{self.queen}\nAllegiance:{self.allegiance}"
def print_name(self):
print(self.name)We need to pass 3 parameters to instantiate this class.
Note that these classes are located in the genos.examples.* subpackage.
So, let's say we wish to instantiate a King object for Eddard Stark because, afterall, Winter is coming.
from genos import recursive_instantiate
ned = {
"cls": "genos.examples.King",
"params":{
"name": "Eddard Stark",
"queen": "Catelyn Stark",
"allegiance": "Robert Baratheon"
}
}
obj = recursive_instantiate(ned)
print(obj)
# Name:Eddard Stark
# Queen:Catelyn Stark
# Allegiance:Robert BaratheonWell, this seemed quite simple.
But rarely are things so simple in life.
Consider another class that takes an instance of King as a parameter.
class House:
def __init__(self, king:King, home:str, sigil:str):
self.king = king
self.home = home
self.sigil = sigil
def __repr__(self):
return f"King:{self.king.name}\nHome:{self.home}\nSigil:{self.sigil}"This is where recursive instantiation comes into action. To initialize an object for this class, we can very easily create a nested dictionary and pass it to our magic method. Of course, we'll be instantiating an object for House Stark.
from genos import recursive_instantiate
stark = {
"cls": "genos.examples.House",
"params": {
"king":{
"cls": "genos.examples.King",
"params":{
"name": "Eddard Stark",
"queen": "Catelyn Stark",
"allegiance": "Robert Baratheon"
}
},
"home":"Winterfell",
"sigil":"Direwolf"
}
}
obj = recursive_instantiate(stark)
print(obj)
# output
# King:Eddard Stark
# Home:Winterfell
# Sigil:DirewolfThe examples shown above always use keyword arguments to instantiate the classes. But we can also choose to simply pass in the positional arguments as shown below.
from genos import recursive_instantiate
stark = {
"cls": "genos.examples.House",
"args": [
{
"cls": "genos.examples.King",
"params":{
"name": "Eddard Stark",
"queen": "Catelyn Stark",
"allegiance": "Robert Baratheon"
}
},
"Winterfell",
"Direwolf"
]
}
obj = recursive_instantiate(stark)
print(obj)
# output
# King:Eddard Stark
# Home:Winterfell
# Sigil:DirewolfThe following example makes use of both positional and keyword arguments together to instantiate the House class. We do not pass the keyword for the king parameter but we do so for the following parameters:home and sigil.
from genos import recursive_instantiate
stark = {
"cls": "genos.examples.House",
"args": [
{
"cls": "genos.examples.King",
"params":{
"name": "Eddard Stark",
"queen": "Catelyn Stark",
"allegiance": "Robert Baratheon"
}
}
],
"params": {
"home":"Winterfell",
"sigil":"Direwolf"
}
}
obj = recursive_instantiate(stark)
print(obj)
# output
# King:Eddard Stark
# Home:Winterfell
# Sigil:DirewolfJust like we classes, we can also instantiate functions by calling recursive_instantiate. The following example shows how we can instantiate and call a simple multiply function using genos.
from genos import recursive_instantiate
function_call = {
"cls": "genos.examples.multiply",
"args": [12, 1.3]
}
result = recursive_instantiate(function_call)
print(result)
# output
# 15.600000000000001For running the following examples you will need to install Pytorch.
pip install torchSuch workflows where we need to instantiate multiple classes recursively is more evident in Deep Learning and related fields.
NeuralSpace has been actively working in this space, building tools for Natural Language Processing (NLP). We have created this tool to make things easier for us.
The following example shows a scenario where you need different components/modules to create your own custom neural network for some specific task. The individual classes are merely wrappers around PyTorch functions. Let's get started.
The following example classes can be found in genos.examples.complex_examples.py.
from torch import nn
class ActivationLayer(nn.Module):
'''
Gives two choices for activation function: ReLU or Tanh.
Introduces non-linearity in neural-networks. Usually applied
after an affine transformation.
'''
def __init__(self, activation:str):
super().__init__()
if activation == 'relu':
self.activation = nn.ReLU()
else:
self.activation = nn.Tanh()
def forward(self, x):
return self.activation(x)
class AffineLayer(nn.Module):
'''
Performs an affine transformation on the input tensor.
For an input tensor "x", the output is W.x + b, where W
is a trainable weight matrix and b is the bias vector.
'''
def __init__(self, in_features:int, out_features:int, activation: ActivationLayer):
super().__init__()
self.transform = nn.Linear(in_features=in_features, out_features=out_features)
self.activation = activation
def forward(self, x):
return self.activation(self.transform(x))
class LSTMLayer(nn.Module):
'''
A wrapper over LSTM layer.
'''
def __init__(self, input_size, hidden_size, batch_first, dropout):
super().__init__()
self.lstm = nn.LSTM(input_size=input_size, hidden_size=hidden_size, batch_first=batch_first)
self.dropout = nn.Dropout(dropout)
def forward(self, x):
output, _ = self.lstm(x)
return self.dropout(output) The three classes above will now be used to create a custom neural network.
Note carefully that in order to instantiate an AffineLayer, we need to pass an object of ActivationLayer.
The CustomModel will comprise of two components: AffineLayer and LSTMLayer.
class CustomModel(nn.Module):
def __init__(self, affine_layer:AffineLayer, lstm_layer:LSTMLayer):
super().__init__()
self.affine_layer = affine_layer
self.lstm_layer = lstm_layer
def forward(self, x):
return self.affine_layer(self.lstm_layer(x))The instantiation of this class using genos will be as follows.
from genos import recursive_instantiate
custom_obj = \
{
"cls": "genos.examples.CustomModel",
"params": {
"affine_layer": {
{
"cls": "examples.example.AffineLayer",
"params": {
"in_features": 256,
"out_features": 256,
"activation": {
"cls": "genos.examples.ActivationLayer",
"params": {
"activation": "relu"
}
}
}
},
{
"cls": "genos.examples.LSTMLayer",
"params":{
"input_size": 256,
"hidden_size":256,
"batch_first":True,
}
}
}
}
}
model = recursive_instantiate(custom_obj)
x = torch.randn(32, 100, 256)
out = model(x)
print(out.shape)
# [32, 100, 256]If you do not wish to use genos for instantiating your functions or classes, you can still use it to find and load different classes within your project structure. The following example shows the usage of get_class() function from genos to locate the King class.
from genos import get_class
class_path = "genos.examples.King"
class_reference = get_class(class_path)
eddard_stark = class_reference(name="Eddard Stark", queen="Catelyn Stark",
allegiance="Robert Baratheon")
print(eddard_stark)
# Name:Eddard Stark
# Queen:Catelyn Stark
# Allegiance:Robert BaratheonSimilar to the get_class() method above, you can also use get_method() function from genos to find functions in your project structure and instantiate them normally.
from genos import get_method
method_path = "genos.examples.multiply"
method_reference = get_method(method_path)
result = method_reference(2, 3.5)
print(result)
# 7.0- Python >=3.7, <4
- Tested on Mac 10.15.6 Catalina, Ubuntu 18.04
# clone the repo
$ git clone https://github.com/Neural-Space/genos.git
# Install system-level dependencies
$ make install-system-deps
# Install environment level dependencies
$ make install-deps# run the tests to make sure everything works
$ make unit-test
# check coverage of the code
$ make test-coverageRead the contribution guideline over here.
Icons made by Skyclick from www.flaticon.com