Python Outputs, Logs and Errors

At the moment this is just the old page from the confluence site.

Output Files

Python produces a number of files. The most important ones are as follows:

.wind_save - a binary file that contains essentially all information about the wind as calculate in the ionization phase of the program. This file can be read with the routine py_wind, where you can see and print out parameters such as the fraction ionization of CIV in ascii or fits-formats.

.spec_tot - an ascii file that contains various spectra from the ionization-calculation phase of the program. The first few lins of the file are as follows:

    # Python Version 58e
    # Freq.        Lambda  Emitted  Star+BL  Disk     Wind     HitSurf  Scattered
    3.023938e+14 9913.975 1.07e+33 2.03e+31 1.05e+33 1.05e+30 4.11e+31        0
    3.049952e+14 9829.418  1.1e+33 2.24e+31 1.07e+33 3.97e+30 4.42e+31        0
     3.075965e+14 9746.292 1.09e+33  2.1e+31 1.07e+33 1.22e+30 3.63e+31        0
    3.101978e+14 9664.559 1.11e+33 1.97e+31 1.09e+33 1.33e+30 4.34e+31        0
     3.127991e+14 9584.186 1.08e+33 2.03e+31 1.06e+33 1.27e+30 4.75e+31        0

The first two columns are fairly obvious. Lambda is in Angstroms. The remainder indicate the luminosity of the system in specific bands. Emitted is the total emergent spectrum, Star+BL is the emergent spectrum from photons bundles originating on the Star or BL, Disk and Wind are the same for photons originating in the disk and wind respectively. HitSurf represents photons that did not escape the system but ran into a boundary, and Scattered are photons that somewhere along their path out of the system were actually scattered.

.log_spec_tot - an ascii file which contains the same information as .spec_tot, but with alogarithmically space frequency intervals. This gives better sampling of the SED in a lot of cases and is much better for plotting things such as the input spectrum.

.spec - an ascii file that contains the final detailed spectra for the wavelengths of interest. Photons bundles are generated in cycles in python and the .spec file is actually written out at the end of each cycle as the program is running in the spectrum-generation phase of the program. So one can inspect the spectrum as it is building up. The beginning of the file is as follows:

    # Python Version 58e
    # Freq.        Lambda  Emitted  Star+BL  Disk     Wind     HitSurf  Scattered  Ang:45.0  Ang:60.0  Ang:75.0
    3.000623e+14 9991.008 1.04e-11 1.66e-13 1.02e-11 4.15e-15 3.78e-13        0 1.55e-11 9.07e-12 3.56e-12
    3.003321e+14 9982.032 1.05e-11 1.96e-13 1.03e-11    2e-14 3.49e-13        0 1.51e-11 8.89e-12 3.47e-12
    3.006019e+14 9973.072 1.03e-11 1.88e-13 1.01e-11 1.51e-14 3.19e-13        0 1.43e-11 8.53e-12 3.32e-12

where the first line indicates the version of python used to generate the spectrum, the second gives a brief description of each column, and the remainder of the file is the spectrum. The most important columns are 1 and 2, which are respectively the frequency and wavelength and the columns marked Ang, which give the spectrum that would be observed from the object at various inclination angles. (The number of columns is therefore variable depending on how many inclinations one asked for in the input files.) The other columns Emitted, Star+BL, Disk, Wind, HitSurf and Scattered have approximately the same meaning as in the .spec_tot file.

.diag - an ascii file that records the progress of the program and gives a log of the errors and warnings that occurred as the program was executing. As of Python76b_dev, this is contained in a subfolder entitled diag_root, where root.pf is your input file. In parallel mode, one diag folder for each thread is produced.

Questions about the outputs How does one evaluate whether the the calculation is converging? At the end of each ionization cycle, and tests are done to indicate whether cells are "converging" or are "converged". A cell is said to be converged if all of the following are true:

The ratio abs(t_r_old-t_r)/(t_r_old+t_r) < epsilon
The ratio abs(t_e_old-t_e)/(t_e_old+t_e) < epsilon
The ratio abs(heating-cooling)/(heating+cooling) < epsilon

At present (58f) epsilon is set to 5%, so basically this means nothing is changing by more than 10%. A cell is said to be "converging" if both of the following are true: The change in electron temperature in this and the previous cycle have the opposite sign, and The change in electron temperature for this cycle is less than the previous one. The two tests are independent of one another, and so in principle a cell can be converged, but not converging, or vice versus. Furthermore a cell can fall into and out of convergence. This information is summarized in the .diag file. The relevant information at present has the following format:

  !!python: Number of ionizing photons 0 lum of ionizing photons 0
  !!wind_update:  Absorbed flux   4.98e+29  (photo 4.91e+29 ff 7.30e+27 lines 0.00e+00)
  !!wind_update: Wind luminosity  2.89e+34 (recomb 1.68e+34 ff 1.20e+34 lines 4.27e+30) after update
  !!wind_update: Max change in t_r  -1352 at cell   27 (0,27)
  !!wind_update: Ave change in t_r    -18 from   9967 to   9949
  !!wind_update: Max change in t_e  -1507 at cell   72 (2,12)
  !!wind_update: Ave change in t_e   -360 from   2469 to   2109
  !!Check_converging: 65 (0.132) converged and 482 (0.976) converging of 494 cells

The first couple of lines give crude information about how energy is flowing in the system. The middle lines tell one how the temperatures are changing on average and in the worst case throughout the grid. The last line indicates which cells are regarded as converged and which are still on the path to conversion. It's would be possible, and might well be a good idea to expand the information about what was changing. We just need to define what we would like to see The easiest way to access this information out of the .diag file is to use grep, e.g. grep Check_converging whatever.diag or grep wind_update whatever.diag A typical, that is "good" result is as follows

 !!Check_converging:  89 (0.164) converged and 543 (1.000) converging of 543 cells
 !!Check_converging: 179 (0.330) converged and 543 (1.000) converging of 543 cells
 !!Check_converging: 227 (0.418) converged and 515 (0.948) converging of 543 cells
 !!Check_converging: 428 (0.788) converged and 453 (0.834) converging of 543 cells
 !!Check_converging: 514 (0.947) converged and 528 (0.972) converging of 543 cells
 !!Check_converging: 529 (0.974) converged and 510 (0.939) converging of 543 cells
 !!Check_converging: 531 (0.978) converged and 428 (0.788) converging of 543 cells
 !!Check_converging: 533 (0.982) converged and 373 (0.687) converging of 543 cells
 !!Check_converging: 539 (0.993) converged and 381 (0.702) converging of 543 cells
 !!Check_converging: 532 (0.980) converged and 383 (0.705) converging of 543 cells

This was a model with a 30x30 grid. Of the 900 cells, 543 cells contained plasma and are therefore checked out for convergence. If you are concerned about convergence issues and want to learn more about what is happening there are ways to restart a calculation that are described under Using a pre-existing model and ways to print out the windsave file for every cycle, see Python Keyword Definitions , under Python Inputs.

Reporting to Screen

Verbosity

Python has a verbosity setting, which can be used to suppress what is printed to screen. By default, Python is run with verbosity 4. The verbosity can be specified with the v flag, e.g.

py -v 2 root.pf

The verbosity settings are as follows- note that all Logs and Errors are only printed by thread 0 when in parallel mode, and the verbosity does not affect the diag file output:

• 0: print nothing to screen except stderr and stdout (which are used very infrequently in code). This will also print out any error messages MPI throws up in parallel mode.
• 1: print only log parallel statements which report on progress of various threads
• 2-4: print Log and error statements for thread 0, and print the keywords and values of the parameter file as it is read in
• 5: or more, print log_silent statements and error_silent statements

This is clearly slightly illogical at the moment and will be improved. It is dealt with in the routines rdpar.c and log.c.

Errors

In serial mode, errors are reporting correctly for the thread you are on, but in parallel mode the errors on screen are only those of the 0th thread. Thus, two scripts can be used to track the errors thrown up

watchdog.py root.pf    #run while run is ongoing
py_error.py root.pf    #run after run has finished

These scripts provide the user with on screen error summaries and can be piped to a file.

Currently, python has a few behaviours to control errors. Firstly, if an error occurs more than 100 times then it stops being repoted (note however, that it is still counted). Secondly, if an error occurs more than 1000000/n_threads times then the run is stopped. This may need to be changed.