You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Does .validate_statistics() really detect anomalies in distribution only for categorical features? "For now drift detection is only supported for categorical features." and "For now skew detection is only supported for categorical features." - doesn't it seem weird?
For example, I have a N(0,1) distributed numerical feature in my train data. Now in serving data this numerical feature has N(10,1) distribution. Any solutions from TFDV for this case?
The text was updated successfully, but these errors were encountered:
wrapper228
changed the title
sa
Detection of skew/drift in distribution of numerical feature
Jan 15, 2020
That's correct -- as of now, TFDV supports drift and skew detection only for categorical features. So, unfortunately, we don't currently have a solution for finding such a distribution shift in numeric features. However, we are planning to add support for skew and drift detection for numeric features in the future.
Yes -- support for detecting drift and skew for numeric features has been added to TFDV, as of Version 0.25.0.
To detect drift or distribution skew in numeric features, specify a jensen_shannon_divergence threshold in the drift_comparator or skew_comparator in your schema.
Does .validate_statistics() really detect anomalies in distribution only for categorical features? "For now drift detection is only supported for categorical features." and "For now skew detection is only supported for categorical features." - doesn't it seem weird?
For example, I have a N(0,1) distributed numerical feature in my train data. Now in serving data this numerical feature has N(10,1) distribution. Any solutions from TFDV for this case?
The text was updated successfully, but these errors were encountered: