-
Notifications
You must be signed in to change notification settings - Fork 0
Installation
The persistence S/W is largely self contained and should be retrievable directly from git. As described below one needs to retrieve or link to the calibration files separately.
The current pipeline version of the pipeline S/W is on the master branch, but other branches can be retrieved. In particular, The primary branch where changes are collected prior to deployment is the dev branch.
The persistence software should run in the anaconda/astroconda environment. The current (17/03) version of the master branch is can only be run using the Python 2.7 version of astroconda, whereas the dev branch runs either under Python 2.7 or Python 3.4.
To retrieve the S/W, simply clone the archive to the place one wants to keep the code, e.g a directory you have created named "persistence "that is where you place python projects
The calibration files needed to run the S/W, which contain some fits files, are not part of the git distribution and are in a tar.gz file, named PerCal....gz that can be retrieved by from here.
There are three places the calibration files can be located.
- They can be placed in the working directory
- They can be placed in a sub-directory PerCal of the working directory
- They can be placed in a directory defined by the environment variable PERCAL
To run the persistence software, you must establish certain environment variables
-
Add the (git) directory persistence to your PATH, e.g.
-
if using csh or equivalents: setenv PATH ${PATH}:${HOME}/py_progs/persistence
-
if using bash or equivalents: export PATH=$PATH:$HOME/py_progs/persistence
-
Add the directory persistence and to your PYTHONPATH, e. g.
-
if using csh or equivalents: setenv PYTHONPATH ${PYTHONPATH}:${HOME}/py_progs/persistence
-
if using bash or equivalents: export PYTHONPATH=$PYTHONPATH:$HOME/persistence
Once you have sourced the environment variabls, one should carray out a basic check of the installation:
- To check that you have the PATH fixed correctly, simply type: run_persist.py -h
(in a shell where astroconda has been enabled). This will print out some help information for this routine
- To check that your PYTHONPATH has (some of) the correct links, simple start python from the command line, and issue the follow commands as follows: python: import run_persist import numpy import pylab import astropy
If none of these commands turn up errors, your environment for python is probably OK, and you can exit python with Ctrl-D. If you have problems with the first command, then there is a problem in setting up the persistence S/W specifically. If you have problems with the last 3, it means your general environment is not set up properly.
If one wishes to calculate persistence pipeline for an IR dataset, one needs access to that dataset and all of the IR datasets that were taken for some period of time prior to the dataset. See Retrieving Data
Although there are a number of executable scripts in the persistence package, and although various routines can be run separately, the two primary routines are
- per_list.py - which (by default) sets up the files observations.ls and observations.sum for managing the pipeline. More specifically, per_list.py looks at all of the flt files in all subdirectories of the working directory in order to create a history of the observations.
- run_persist.py - runs the persistence pipeline, producing the persistence products, and updating the the observations.sum
These programs have lots of options which can be displayed with per_list.py -h or run_persist.py -h
To run the persistence pipeline the first time on a specific dataset, execute the following commands
- per_list.py -np 4, where -np -4 means per_list will be run in parallel on 4 processors. This step can take a while if one is running per_list on the entire QuickLook archive because it opens every flt file in the entire archive to extract keywords. Later updates are quicker because the program only has to open new or changed datasets.
- run_persist.py -local iaam01jcq, where iaam01jcq is a dataset name, and -local implies that the output products will be placed in a subdirectory Persist of the current working directory. If -local is omitted, then the output products will be placed in a subdirectory Persist located just below the location of the flt file (which is not what you want if you are using the Quicklook archive.)