You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
TrainerState.save_to_json fails due to np.float32 values not being JSON serializable, causing crashes when saving training state.
Problem:
The TrainerState.save_to_json method in the transformers library fails when trying to save training state because np.float32 values are not natively serializable in JSON. This issue causes the training process to crash when saving checkpoints.
Steps to Reproduce:
Run the model distillation script using SentenceTransformerTrainer.
The training proceeds normally but fails when saving state due to an np.float32 serialization error.
The error message typically looks like:
TypeError: Object of type float32 is not JSON serializable
Expected Behavior:
The training state should save without errors.
The script should complete training and store checkpoints correctly.
Proposed Fix:
Convert np.float32 values to Python float before saving JSON.
Patch TrainerState.save_to_json to handle this conversion.
The text was updated successfully, but these errors were encountered:
TrainerState.save_to_json fails due to np.float32 values not being JSON serializable, causing crashes when saving training state.
Problem:
The TrainerState.save_to_json method in the transformers library fails when trying to save training state because np.float32 values are not natively serializable in JSON. This issue causes the training process to crash when saving checkpoints.
Steps to Reproduce:
Run the model distillation script using SentenceTransformerTrainer.
The training proceeds normally but fails when saving state due to an np.float32 serialization error.
The error message typically looks like:
TypeError: Object of type float32 is not JSON serializable
Expected Behavior:
The training state should save without errors.
The script should complete training and store checkpoints correctly.
Proposed Fix:
Convert np.float32 values to Python float before saving JSON.
Patch TrainerState.save_to_json to handle this conversion.
The text was updated successfully, but these errors were encountered: