-
Notifications
You must be signed in to change notification settings - Fork 74
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Enhancement] add normalization and trend to Field
#94
Comments
@LSchueler : what do you think about the idea to simply allow a callable function for |
Sorry, I forgot about this issue! I really like the idea! Or we provide some kind of plugin solution which would make it possible to call I also dig the idea of allowing callable functions for Do you think there would be an application where one needs a trend before the transformation and one afterwards? |
I had the following idea: """Normalizer demonstration."""
import numpy as np
import gstools as gs
class Normalizer:
"""Normalizer class."""
def __init__(self, **kwargs):
# only use values, that have a provided default value
for key, value in self.default_para().items():
setattr(self, key, kwargs.get(key, value))
def default_para(self):
"""Get default parameters."""
return {}
def transform(self, values):
"""Transform to target distribution."""
pass
def normalize(self, values):
"""Transform to normal distribution."""
pass
class LogNormal(Normalizer):
"""Log-normal fields."""
def transform(self, values):
"""Transform to log-normal distribution."""
return np.exp(values)
def normalize(self, values):
"""Transform to normal distribution."""
return np.log(values)
class BoxCox(Normalizer):
"""Log-normal fields."""
def default_para(self):
"""Get default parameters."""
return {"shift": 0, "lmbda": 1}
def transform(self, values):
"""Transform to target distribution."""
if np.isclose(self.lmbda, 0):
return np.exp(values) - self.shift
return (1 + self.lmbda * values) ** (1 / self.lmbda) - self.shift
def normalize(self, values):
"""Transform to normal distribution."""
values += self.shift
if np.isclose(self.lmbda, 0):
return np.log(values)
return (values ** self.lmbda - 1) / self.lmbda
norm_ln = LogNormal()
print(norm_ln.transform(-1))
norm_bc = BoxCox(lmbda=0)
a = norm_bc.transform(-1)
b = norm_bc.normalize(a)
print(a, b)
this could than be used in the Krige or SRF class: model = gs.Gaussian()
srf = SRF(model, normalizer=norm_bc) Then, the rest is done internally. This is mostly important for I also had a second thought about the trend. I would always apply the trend to normal data, since residuals in regression are assumed to be normal distributed. If somebody wants it another way, they can always do it by hand. |
What do think about not only providing the classes, but also instances of them. Something like from gstools.transform import log_normal
norm_ln = log_normal.transform(-1) Then the user wouldn't have to instantiate standard tranformation classes. |
Then let's just support trends to the normal data! |
I don't see a big difference to: from gstools.normalize import LogNormal
norm_ln = LogNormal().transform(-1) And it would be only "simple" for parameterless transformations. |
Another good thing would be to provide a norm_bc = BoxCox()
norm_bc.fit(data) or with a shortcut: norm_bc = BoxCox(data=data) |
Reminder: Rewrite precipitation tutorial after this was merged. |
Normalizer
The provided
transform
submodule serves only as a post-processor for fields.It would be nice to provide another way of transforming fields with a
Normalizer
class, that provides a transformation with its inverse transformation (normalization) of the field-values to gain normality.Examples:
The difference to the provided transformations is, that these are not destructive and can be inverted. The advantage is, that it could be used for kriging to inverse-transform the conditional values and then transform the field afterwards (so we could provide log-normal kriging for example)
Trend
Like already implemented in the kriging routines, we could simply allow to provide a functional mean, that is applied to the field. We would need a switch to state if the trend should be added before or after normalization.
The text was updated successfully, but these errors were encountered: