fix: dropping NaT values should happen after converting to datetime (#…

…152) * feature: allow splitting the datetime column by some string, when reading from csv or excel Signed-off-by: F.N. Claessen <[email protected]> * fix: dropping NaT values should happen after converting to datetime objects, which in case of dayfirst=True happens inside interpret_special_read_cases Signed-off-by: F.N. Claessen <[email protected]> --------- Signed-off-by: F.N. Claessen <[email protected]>
SeitaBV · Nov 3, 2023 · aca114c · aca114c
1 parent bb56075
commit aca114c
Showing 1 changed file with 4 additions and 4 deletions.
diff --git a/timely_beliefs/beliefs/utils.py b/timely_beliefs/beliefs/utils.py
@@ -639,10 +639,6 @@ def read_csv(
             ]
         )
 
-    # Exclude rows with NaN or NaT values
-    if not kwargs.get("keep_default_na", True):
-        df = df.dropna()
-
     # Preserve order of usecols
     if "usecols" in kwargs:
         df = df[[col for col in kwargs["usecols"] if col in df.columns]]
@@ -652,6 +648,10 @@ def read_csv(
         df, sensor, resample, timezone, dayfirst, split=datetime_column_split
     )
 
+    # Exclude rows with NaN or NaT values
+    if not kwargs.get("keep_default_na", True):
+        df = df.dropna()
+
     # Apply optionally set belief timing
     if belief_horizon is not None and belief_time is not None:
         raise ValueError("Cannot set both a belief horizon and a belief time.")