Adopt Astropy file formats for ASCII outputs

[jhmatthews]

@kslong has suggested, and I think I agree, that adopting file formats consistent with Astropy would be sensible. The astropy.io.ascii module allows one to read in files of the following format

# comments
Freq.        Lambda  Created  Emitted   CenSrc  Disk     Wind     HitSurf Scattered A10P0.50
1.873716e+15 1599.989  1.07e-31 1.07e-31 1.07e-31        0              0 1.55e-31 2.47e-31        0
1.873728e+15 1599.979   8.4e-32  8.4e-32  8.4e-32        0              0 2.37e-31 1.55e-31 1.23e-31        0
1.873741e+15 1599.968  9.16e-32 9.16e-32 9.16e-32        0            0        0        0 1.23e-31        0

i.e. the first line has no comments.

With commands like

from astropy.io import ascii
data = ascii.read("output.spec")

@Higginbottom , any objection? We could have try/except commands in our scripts to catch users who didn't have astropy installed.

[Higginbottom]

I don't fully understand - it looked like the first line did have a comment in the example! Is this just for the .spec files, in which case what is the proposed new format. If this is a really important feature to implement, then naturally I have no objections - but if the .spec file format is significantly changed, it will mean I'll need to make changes to scripts I currently use, so IMHO it should only be made if there is a demonstrable benefit to be had....

[kslong]

Nick,

The generalized reason for doing this is that the community is adopting this kind of format for reading and writing ascii files. They have developed routines in astropy which allow you to read such files, without counting columns.

Had we and you been using this type of file format and the astropy routines, the change I made in the file formats of one of the files a few months ago would NOT have affected your routines. Furthermore, using the astropy routines would also give us more flexible ways to create files in the future, e.g including the density and T in a single file, and or choosing to write files as fits tables instead of ascii files. The astropy routines determine the format of the file and read it without your having to specify what format is.

On the other hand, if you don’t like astropy then the only thing you would need to do is to ignore the row that contains the column names, and keep counting.

Knox

[Higginbottom]

Oh, I understand - so the line that James was referring to was the line with the names of the columns in - which is a comment at the moment, but would not be a comment in future. It looked like the suggestion was to remove all of the comment lines, which would be a problem since I have some quite elaborate scripts that use the parameters of the simulation, normally stored in comments in the preamble of a .spec file, to do lots of things. If they will all still be there than thats totally cool!

Nick

[kslong]

Correct. It is just one line, actually just one character, that would change.

[jhmatthews]

Yes, sorry, I wasn't overly clear there.

This is implemented by pull request #123

Nick, this should make everything easier anyway. I've tried to describe the procedures on this wiki page - Feel free to test my python functions on any model run with the latest dev version and let me know if anything messes up (@kslong , @smangham , @Higginbottom )

Once I've finished testing the 'advanced mode' stuff I will also make other diagnostic outputs follow the same convention. This should mean that dealing with python outputs is in general a lot easier.