Suggestion: BiLinear Combination

[milannnnn]

First, I want to thank you all for developing and maintaining such an amazing model. While playing around with it, I noticed that it has some issues with correctly modeling some scaling (future) regressors. For example, if we have a future regressor that represents working hours (accounting for weekends, personal and/or public holidays), which has a scaling effect on the series (e.g. electricity consumption), the model does not appear to utilize that information fully, as all the components are being added together (and e.g. the seasonality components are not being scaled by that regressor):

	def forward():
		...
		trend = ...
		addiditve_features = ...
		multiplicative_features = ...

		additive_components = torch.zeros_like()
		for additive_feature in addiditve_features:
			additive_components += additive_feature 
		  
		multiplicative_components = torch.zeros_like()
		for multiplicative_feature in multiplicative_features:
			multiplicative_components += multiplicative_feature 

		out = trend + additive_components + trend.detach() * multiplicative_components
		return out

One idea I had was to introduce a bi-linear combination that would enable direct scaling of all the components (so the additive components could be scaled by the scaling ones (e.g. by the working hours regressor):

	def __init__(self):
	        ...
		self._bilinear = nn.Bilinear(n_additive_features+1, n_multiplicative_features+1, 1)
	
	def forward():
		...
		trend = ...

		additive_components = torch.stack([trend]+[addiditve_features], dim=2)
		
		# We add a ones_like tensors to ensure the original trend and additive features can be passed 
		# directly to the output tensor
		multiplicative_components = torch.stack([torch.ones_like(...)]+[multiplicative_features], dim=2)

		# We combine them with a bilinear module
		out = self._bilinear(additive_components, multiplicative_components).squeeze(dim=2)
		return out

For code simplicity, I used the multiplicative features for scaling, but, for model consistency, it would probably be best to introduce another set of features (e.g. scaling features).

What are your thoughts on this?