Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Numpy errors #143

Closed
treitpeter opened this issue Mar 6, 2023 · 3 comments
Closed

Numpy errors #143

treitpeter opened this issue Mar 6, 2023 · 3 comments
Labels
bug Something isn't working

Comments

@treitpeter
Copy link

treitpeter commented Mar 6, 2023 •
edited by bittremieux
Loading

windows 10 Pro
12 core, 24 logical processor

Installed via instructions and everything went fine until I actually tried to run the denovo mode on a metaproteomics file. I then generated a pytorch deprecation warning, and two numpy type errors related to runtime (runtimewarning: mean of empty slice and runtimewarning: invalid value encountered in double_scalars). I'm unfortunately unable to decipher more.
Any help would be much appreciated.

Here is my log:

C:\Users\Alphapept>conda activate casanovo_env

(casanovo_env) C:\Users\Alphapept>casanovo --mode=denovo --peak_path=D:\Casanovo\MGF\20191024_QX7_PeTr_LC02_SA_MicrobPred_19.mgf
Global seed set to 454
2023-03-06 16:12:53,274 INFO [casanovo/MainProcess] casanovo._get_model_weights : Model weights file C:\Users\Alphapept\AppData\Local\casanovo\casanovo_massivekb_v3_0_0.ckpt retrieved from local cache
2023-03-06 16:12:53,274 INFO [casanovo/MainProcess] casanovo.main : Casanovo version 3.2.0
2023-03-06 16:12:53,274 DEBUG [casanovo/MainProcess] casanovo.main : mode = denovo
2023-03-06 16:12:53,274 DEBUG [casanovo/MainProcess] casanovo.main : model = C:\Users\Alphapept\AppData\Local\casanovo\casanovo_massivekb_v3_0_0.ckpt
2023-03-06 16:12:53,274 DEBUG [casanovo/MainProcess] casanovo.main : peak_path = D:\Casanovo\MGF\20191024_QX7_PeTr_LC02_SA_MicrobPred_19.mgf
2023-03-06 16:12:53,290 DEBUG [casanovo/MainProcess] casanovo.main : peak_path_val = None
2023-03-06 16:12:53,290 DEBUG [casanovo/MainProcess] casanovo.main : config = C:\Users\Alphapept\miniconda3\envs\casanovo_env\Lib\site-packages\casanovo\config.yaml
2023-03-06 16:12:53,290 DEBUG [casanovo/MainProcess] casanovo.main : output = C:\Users\Alphapept\casanovo_202303061612532023-03-06 16:12:53,290 DEBUG [casanovo/MainProcess] casanovo.main : random_seed = 454
2023-03-06 16:12:53,290 DEBUG [casanovo/MainProcess] casanovo.main : n_peaks = 150
2023-03-06 16:12:53,290 DEBUG [casanovo/MainProcess] casanovo.main : min_mz = 50.0
2023-03-06 16:12:53,290 DEBUG [casanovo/MainProcess] casanovo.main : max_mz = 2500.0
2023-03-06 16:12:53,290 DEBUG [casanovo/MainProcess] casanovo.main : min_intensity = 0.01
2023-03-06 16:12:53,290 DEBUG [casanovo/MainProcess] casanovo.main : remove_precursor_tol = 2.0
2023-03-06 16:12:53,293 DEBUG [casanovo/MainProcess] casanovo.main : max_charge = 10
2023-03-06 16:12:53,293 DEBUG [casanovo/MainProcess] casanovo.main : precursor_mass_tol = 50.0
2023-03-06 16:12:53,293 DEBUG [casanovo/MainProcess] casanovo.main : isotope_error_range = (0, 1)
2023-03-06 16:12:53,293 DEBUG [casanovo/MainProcess] casanovo.main : dim_model = 512
2023-03-06 16:12:53,293 DEBUG [casanovo/MainProcess] casanovo.main : n_head = 8
2023-03-06 16:12:53,293 DEBUG [casanovo/MainProcess] casanovo.main : dim_feedforward = 1024
2023-03-06 16:12:53,293 DEBUG [casanovo/MainProcess] casanovo.main : n_layers = 9
2023-03-06 16:12:53,293 DEBUG [casanovo/MainProcess] casanovo.main : dropout = 0.0
2023-03-06 16:12:53,293 DEBUG [casanovo/MainProcess] casanovo.main : dim_intensity = None
2023-03-06 16:12:53,293 DEBUG [casanovo/MainProcess] casanovo.main : custom_encoder = None
2023-03-06 16:12:53,293 DEBUG [casanovo/MainProcess] casanovo.main : max_length = 100
2023-03-06 16:12:53,293 DEBUG [casanovo/MainProcess] casanovo.main : residues = {'G': 57.021464, 'A': 71.037114, 'S': 87.032028, 'P': 97.052764, 'V': 99.068414, 'T': 101.04767, 'C+57.021': 160.030649, 'L': 113.084064, 'I': 113.084064, 'N': 114.042927, 'D': 115.026943, 'Q': 128.058578, 'K': 128.094963, 'E': 129.042593, 'M': 131.040485, 'H': 137.058912, 'F': 147.068414, 'R': 156.101111, 'Y': 163.063329, 'W': 186.079313, 'M+15.995': 147.0354, 'N+0.984': 115.026943, 'Q+0.984': 129.042594, '+42.011': 42.010565, '+43.006': 43.005814, '-17.027': -17.026549, '+43.006-17.027': 25.980265}
2023-03-06 16:12:53,293 DEBUG [casanovo/MainProcess] casanovo.main : n_log = 1
2023-03-06 16:12:53,293 DEBUG [casanovo/MainProcess] casanovo.main : tb_summarywriter = None
2023-03-06 16:12:53,293 DEBUG [casanovo/MainProcess] casanovo.main : warmup_iters = 100000
2023-03-06 16:12:53,293 DEBUG [casanovo/MainProcess] casanovo.main : max_iters = 600000
2023-03-06 16:12:53,293 DEBUG [casanovo/MainProcess] casanovo.main : learning_rate = 0.0005
2023-03-06 16:12:53,293 DEBUG [casanovo/MainProcess] casanovo.main : weight_decay = 1e-05
2023-03-06 16:12:53,293 DEBUG [casanovo/MainProcess] casanovo.main : train_batch_size = 32
2023-03-06 16:12:53,293 DEBUG [casanovo/MainProcess] casanovo.main : predict_batch_size = 1024
2023-03-06 16:12:53,293 DEBUG [casanovo/MainProcess] casanovo.main : n_beams = 5
2023-03-06 16:12:53,293 DEBUG [casanovo/MainProcess] casanovo.main : logger = None
2023-03-06 16:12:53,293 DEBUG [casanovo/MainProcess] casanovo.main : max_epochs = 30
2023-03-06 16:12:53,293 DEBUG [casanovo/MainProcess] casanovo.main : num_sanity_val_steps = 0
2023-03-06 16:12:53,293 DEBUG [casanovo/MainProcess] casanovo.main : train_from_scratch = True
2023-03-06 16:12:53,293 DEBUG [casanovo/MainProcess] casanovo.main : save_model = True
2023-03-06 16:12:53,293 DEBUG [casanovo/MainProcess] casanovo.main : model_save_folder_path =
2023-03-06 16:12:53,293 DEBUG [casanovo/MainProcess] casanovo.main : save_weights_only = True
2023-03-06 16:12:53,293 DEBUG [casanovo/MainProcess] casanovo.main : every_n_train_steps = 50000
2023-03-06 16:12:53,293 DEBUG [casanovo/MainProcess] casanovo.main : n_workers = 0
2023-03-06 16:12:53,293 INFO [casanovo/MainProcess] casanovo.main : Predict peptide sequences with Casanovo.
2023-03-06 16:12:53,402 DEBUG [fsspec.local/MainProcess] local.__init__ : open file: C:/Users/Alphapept/AppData/Local/casanovo/casanovo_massivekb_v3_0_0.ckpt
2023-03-06 16:12:53,621 INFO [depthcharge.data.hdf5/MainProcess] hdf5.__init__ : Reading 1 files...
D:\Casanovo\MGF\20191024_QX7_PeTr_LC02_SA_MicrobPred_19.mgf: 180109spectra [01:20, 2230.87spectra/s]
2023-03-06 16:14:16,357 WARNING [py.warnings/MainProcess] warnings._showwarnmsg : C:\Users\Alphapept\miniconda3\envs\casanovo_env\lib\site-packages\pytorch_lightning\trainer\connectors\accelerator_connector.py:589: LightningDeprecationWarning: The Trainer argument `auto_select_gpus` has been deprecated in v1.9.0 and will be removed in v2.0.0. Please use the function `pytorch_lightning.accelerators.find_usable_cuda_devices` instead.
  rank_zero_deprecation(

2023-03-06 16:14:16,373 WARNING [py.warnings/MainProcess] warnings._showwarnmsg : C:\Users\Alphapept\miniconda3\envs\casanovo_env\lib\site-packages\pytorch_lightning\trainer\connectors\data_connector.py:224: PossibleUserWarning: The dataloader, predict_dataloader 0, does not have many workers which may be a bottleneck. Consider increasing the value of the `num_workers` argument` (try 24 which is the number of cpus on this machine) in the `DataLoader` init to improve performance.
  rank_zero_warn(

Predicting DataLoader 0:   0%|                                                                 | 0/176 [00:00<?, ?it/s]2023-03-06 16:14:29,527 WARNING [py.warnings/MainProcess] warnings._showwarnmsg : C:\Users\Alphapept\miniconda3\envs\casanovo_env\lib\site-packages\torch\nn\modules\transformer.py:276: UserWarning: The PyTorch API of nested tensors is in prototype stage and will change in the near future. (Triggered internally at ..\aten\src\ATen\NestedTensorImpl.cpp:177.)
  output = torch._nested_tensor_from_mask(output, src_key_padding_mask.logical_not(), mask_check=False)

2023-03-06 16:17:41,319 WARNING [py.warnings/MainProcess] warnings._showwarnmsg : C:\Users\Alphapept\miniconda3\envs\casanovo_env\lib\site-packages\numpy\core\fromnumeric.py:3432: RuntimeWarning: Mean of empty slice.
  return _methods._mean(a, axis=axis, dtype=dtype,

2023-03-06 16:17:41,319 ****WARNING [py.warnings/MainProcess] warnings._showwarnmsg : C:\Users\Alphapept\miniconda3\envs\casanovo_env\lib\site-packages\numpy\core\_methods.py:190: RuntimeWarning: invalid value encountered in double_scalars
  ret = ret.dtype.type(ret / rcount)****

2023-03-06 16:21:59,982 WARNING [py.warnings/MainProcess] warnings._showwarnmsg : C:\Users\Alphapept\miniconda3\envs\casanovo_env\lib\site-packages\pytorch_lightning\trainer\call.py:48: UserWarning: Detected KeyboardInterrupt, attempting graceful shutdown...
  rank_zero_warn("Detected KeyboardInterrupt, attempting graceful shutdown...")

Predicting DataLoader 0:   0%|                                                                                    | 0/176 [07:31<?, ?it/s]
@bittremieux
Copy link
Collaborator

These are just warnings and happen from time to time. This is not unexpected and normally doesn't interfere with running Casanovo.

@bittremieux bittremieux added the bug Something isn't working label Mar 6, 2023
@treitpeter
Copy link
Author

treitpeter commented Mar 7, 2023 •
edited by bittremieux
Loading

I understand. Since then there has been no progress in the predicting dataloader bar since yesterday and I saw that python wasn't using any CPU or memory resources, I restarted with the same specs. Unfortunately I'm returning the same error code

C:\Users\Alphapept>activate casanovo_env

(casanovo_env) C:\Users\Alphapept>casanovo --mode=denovo --peak_path=D:\Casanovo\MGF\20191024_QX7_PeTr_LC02_SA_MicrobPred_19.mgf
Global seed set to 454
2023-03-07 09:25:02,075 INFO [casanovo/MainProcess] casanovo._get_model_weights : Model weights file C:\Users\Alphapept\AppData\Local\casanovo\casanovo_massivekb_v3_0_0.ckpt retrieved from local cache
2023-03-07 09:25:02,075 INFO [casanovo/MainProcess] casanovo.main : Casanovo version 3.2.0
2023-03-07 09:25:02,075 DEBUG [casanovo/MainProcess] casanovo.main : mode = denovo
2023-03-07 09:25:02,075 DEBUG [casanovo/MainProcess] casanovo.main : model = C:\Users\Alphapept\AppData\Local\casanovo\casanovo_massivekb_v3_0_0.ckpt
2023-03-07 09:25:02,075 DEBUG [casanovo/MainProcess] casanovo.main : peak_path = D:\Casanovo\MGF\20191024_QX7_PeTr_LC02_SA_MicrobPred_19.mgf
2023-03-07 09:25:02,075 DEBUG [casanovo/MainProcess] casanovo.main : peak_path_val = None
2023-03-07 09:25:02,075 DEBUG [casanovo/MainProcess] casanovo.main : config = C:\Users\Alphapept\miniconda3\envs\casanovo_env\Lib\site-packages\casanovo\config.yaml
2023-03-07 09:25:02,075 DEBUG [casanovo/MainProcess] casanovo.main : output = C:\Users\Alphapept\casanovo_20230307092502
2023-03-07 09:25:02,075 DEBUG [casanovo/MainProcess] casanovo.main : random_seed = 454
2023-03-07 09:25:02,075 DEBUG [casanovo/MainProcess] casanovo.main : n_peaks = 150
2023-03-07 09:25:02,075 DEBUG [casanovo/MainProcess] casanovo.main : min_mz = 50.0
2023-03-07 09:25:02,075 DEBUG [casanovo/MainProcess] casanovo.main : max_mz = 2500.0
2023-03-07 09:25:02,075 DEBUG [casanovo/MainProcess] casanovo.main : min_intensity = 0.01
2023-03-07 09:25:02,075 DEBUG [casanovo/MainProcess] casanovo.main : remove_precursor_tol = 2.0
2023-03-07 09:25:02,075 DEBUG [casanovo/MainProcess] casanovo.main : max_charge = 10
2023-03-07 09:25:02,075 DEBUG [casanovo/MainProcess] casanovo.main : precursor_mass_tol = 50.0
2023-03-07 09:25:02,075 DEBUG [casanovo/MainProcess] casanovo.main : isotope_error_range = (0, 1)
2023-03-07 09:25:02,075 DEBUG [casanovo/MainProcess] casanovo.main : dim_model = 512
2023-03-07 09:25:02,075 DEBUG [casanovo/MainProcess] casanovo.main : n_head = 8
2023-03-07 09:25:02,091 DEBUG [casanovo/MainProcess] casanovo.main : dim_feedforward = 1024
2023-03-07 09:25:02,091 DEBUG [casanovo/MainProcess] casanovo.main : n_layers = 9
2023-03-07 09:25:02,091 DEBUG [casanovo/MainProcess] casanovo.main : dropout = 0.0
2023-03-07 09:25:02,091 DEBUG [casanovo/MainProcess] casanovo.main : dim_intensity = None
2023-03-07 09:25:02,091 DEBUG [casanovo/MainProcess] casanovo.main : custom_encoder = None
2023-03-07 09:25:02,091 DEBUG [casanovo/MainProcess] casanovo.main : max_length = 100
2023-03-07 09:25:02,091 DEBUG [casanovo/MainProcess] casanovo.main : residues = {'G': 57.021464, 'A': 71.037114, 'S': 87.032028, 'P': 97.052764, 'V': 99.068414, 'T': 101.04767, 'C+57.021': 160.030649, 'L': 113.084064, 'I': 113.084064, 'N': 114.042927, 'D': 115.026943, 'Q': 128.058578, 'K': 128.094963, 'E': 129.042593, 'M': 131.040485, 'H': 137.058912, 'F': 147.068414, 'R': 156.101111, 'Y': 163.063329, 'W': 186.079313, 'M+15.995': 147.0354, 'N+0.984': 115.026943, 'Q+0.984': 129.042594, '+42.011': 42.010565, '+43.006': 43.005814, '-17.027': -17.026549, '+43.006-17.027': 25.980265}
2023-03-07 09:25:02,091 DEBUG [casanovo/MainProcess] casanovo.main : n_log = 1
2023-03-07 09:25:02,091 DEBUG [casanovo/MainProcess] casanovo.main : tb_summarywriter = None
2023-03-07 09:25:02,091 DEBUG [casanovo/MainProcess] casanovo.main : warmup_iters = 100000
2023-03-07 09:25:02,091 DEBUG [casanovo/MainProcess] casanovo.main : max_iters = 600000
2023-03-07 09:25:02,091 DEBUG [casanovo/MainProcess] casanovo.main : learning_rate = 0.0005
2023-03-07 09:25:02,091 DEBUG [casanovo/MainProcess] casanovo.main : weight_decay = 1e-05
2023-03-07 09:25:02,091 DEBUG [casanovo/MainProcess] casanovo.main : train_batch_size = 32
2023-03-07 09:25:02,091 DEBUG [casanovo/MainProcess] casanovo.main : predict_batch_size = 1024
2023-03-07 09:25:02,091 DEBUG [casanovo/MainProcess] casanovo.main : n_beams = 5
2023-03-07 09:25:02,091 DEBUG [casanovo/MainProcess] casanovo.main : logger = None
2023-03-07 09:25:02,091 DEBUG [casanovo/MainProcess] casanovo.main : max_epochs = 30
2023-03-07 09:25:02,091 DEBUG [casanovo/MainProcess] casanovo.main : num_sanity_val_steps = 0
2023-03-07 09:25:02,091 DEBUG [casanovo/MainProcess] casanovo.main : train_from_scratch = True
2023-03-07 09:25:02,091 DEBUG [casanovo/MainProcess] casanovo.main : save_model = True
2023-03-07 09:25:02,091 DEBUG [casanovo/MainProcess] casanovo.main : model_save_folder_path =
2023-03-07 09:25:02,091 DEBUG [casanovo/MainProcess] casanovo.main : save_weights_only = True
2023-03-07 09:25:02,091 DEBUG [casanovo/MainProcess] casanovo.main : every_n_train_steps = 50000
2023-03-07 09:25:02,091 DEBUG [casanovo/MainProcess] casanovo.main : n_workers = 0
2023-03-07 09:25:02,091 INFO [casanovo/MainProcess] casanovo.main : Predict peptide sequences with Casanovo.
2023-03-07 09:25:02,199 DEBUG [fsspec.local/MainProcess] local.__init__ : open file: C:/Users/Alphapept/AppData/Local/casanovo/casanovo_massivekb_v3_0_0.ckpt
2023-03-07 09:25:02,418 INFO [depthcharge.data.hdf5/MainProcess] hdf5.__init__ : Reading 1 files...
D:\Casanovo\MGF\20191024_QX7_PeTr_LC02_SA_MicrobPred_19.mgf: 180109spectra [01:21, 2202.72spectra/s]
2023-03-07 09:26:26,198 WARNING [py.warnings/MainProcess] warnings._showwarnmsg : C:\Users\Alphapept\miniconda3\envs\casanovo_env\lib\site-packages\pytorch_lightning\trainer\connectors\accelerator_connector.py:589: LightningDeprecationWarning: The Trainer argument `auto_select_gpus` has been deprecated in v1.9.0 and will be removed in v2.0.0. Please use the function `pytorch_lightning.accelerators.find_usable_cuda_devices` instead.
  rank_zero_deprecation(

2023-03-07 09:26:26,231 WARNING [py.warnings/MainProcess] warnings._showwarnmsg : C:\Users\Alphapept\miniconda3\envs\casanovo_env\lib\site-packages\pytorch_lightning\trainer\connectors\data_connector.py:224: PossibleUserWarning: The dataloader, predict_dataloader 0, does not have many workers which may be a bottleneck. Consider increasing the value of the `num_workers` argument` (try 24 which is the number of cpus on this machine) in the `DataLoader` init to improve performance.
  rank_zero_warn(

Predicting DataLoader 0:   0%|                                                                 | 0/176 [00:00<?, ?it/s]2023-03-07 09:26:39,419 WARNING [py.warnings/MainProcess] warnings._showwarnmsg : C:\Users\Alphapept\miniconda3\envs\casanovo_env\lib\site-packages\torch\nn\modules\transformer.py:276: UserWarning: The PyTorch API of nested tensors is in prototype stage and will change in the near future. (Triggered internally at ..\aten\src\ATen\NestedTensorImpl.cpp:177.)
  output = torch._nested_tensor_from_mask(output, src_key_padding_mask.logical_not(), mask_check=False)

2023-03-07 09:29:52,090 WARNING [py.warnings/MainProcess] warnings._showwarnmsg : C:\Users\Alphapept\miniconda3\envs\casanovo_env\lib\site-packages\numpy\core\fromnumeric.py:3432: RuntimeWarning: Mean of empty slice.
  return _methods._mean(a, axis=axis, dtype=dtype,

2023-03-07 09:49:34,481 WARNING [py.warnings/MainProcess] warnings._showwarnmsg : C:\Users\Alphapept\miniconda3\envs\casanovo_env\lib\site-packages\numpy\core\_methods.py:190: RuntimeWarning: invalid value encountered in double_scalars
  ret = ret.dtype.type(ret / rcount)

Would you have any suggestions to some things I could try?

@bittremieux
Copy link
Collaborator

Is this a system without GPUs (only CPU) by any chance? There are currently some issues which prevent Casanovo from running if there's no GPU available (#106).

Also note that running on Windows will not be able to use multiprocessing, and for optimal performance Linux is recommended.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@bittremieux @treitpeter