Merge branch 'main' into separate-data

.github/workflows/main.yml

@@ -1,5 +1,5 @@
 name: "Build and Test"
-on: [push, pull_request]
+on: [push, pull_request, release]
 jobs:
   pre_build:
     runs-on: ubuntu-latest
@@ -18,15 +18,11 @@ jobs:
     runs-on: ${{ matrix.os }}
     strategy:
       matrix:
-        os: [ubuntu-latest, macos-latest]
-        python-version: ['3.8', '3.10']
+        os: [ubuntu-20.04, macos-latest]
+        python-version: ['3.8', '3.10', '3.12']
         include:
-          - os: ubuntu-20.04
-            python-version: '2.7'
           - os: ubuntu-20.04
             python-version: '3.6'
-          - os: macos-latest
-            python-version: '2.7'
           - os: macos-latest
             python-version: '3.6'
       fail-fast: false
@@ -56,6 +52,7 @@ jobs:
         if: ${{ matrix.os == 'macos-latest' }}
         run: |
           brew install \
+            gdbm \
             fftw \
             pkg-config
       - uses: actions/setup-python@v4
@@ -69,7 +66,7 @@ jobs:
             numpy \
             matplotlib \
             scipy \
-            "ephem<4.1.1" \
+            ephem \
             jupyterlab \
             jupyter_client \
             nbformat \

.gitignore

@@ -72,6 +72,7 @@ instance/
 
 # Sphinx documentation
 docs/_build/
+docs/source/rnotes.rst
 
 # PyBuilder
 .pybuilder/

CHANGELOG

@@ -1,23 +1,44 @@
+2.1.8
+ * Added polarization basis conversions functions to lsl.imaging.utils
+ * Improved TBF frame count reliability
+ * Fixed several bugs in lsl.sim.dp.point_source() that lead to data that would not image properly
+ * Dropped unlzw in favor of using the gzip utility
+ * Made antenna inclusion/exclusion more robust in lsl.imaging.data.VisibilityDataSet.get_antenna_subset()
+ * Added support for "long (ionospheric) product names"
+ * Updated the Hamming window coefficients
+ * Switched the PFB in lsl.correlator.fx to a Hamming window
+ * Updated the documentation in lsl.correlator.fx to clarify that 'pfb' enables a 4-tap Hamming windowed PFB
+ * Updated the documentation in lsl.correlator.fx to clarify that 'pfb' overrides the 'window' keyword
+ * Added better documentation on the LWA data formats
+ * Fixed a bug in lsl.reader.ldp.TBFFile that caused it to die on files with gaps in the frequency coverage
+ * Updated lsl.imaging.util.build_gridded_image to allow for the image size and pixel size to be used instead of the (u,v) grid size and resolution
+ * Added image size and pixel scale options to imageIDI.py
+ * Added a -d/--decimation option to correlateTBF.py to allow on-the-fly channel decimation
+ * Fixed a bug in lsl.sim.vis.build_sim_data when computing the hour angle and declination for non-zenith phase centers
+ * Added support for Python 3.12
+
 2.1.7
  * Updated the LWA-SV SSMIF to 2022/11/15
  * Updated the LWA1 SSMIF to 2022/11/15
  * Fixed a build problem on Python 3.8+
+ * Added support to the lsl.readers for returning DRX, TBN, and TBF data as "ci8" data
+ * Added support to the lsl.readers for returning VDIF data as "i8" data
 
 2.1.6
- * Updated for Python 3.10
- * Cleaned up the naming of the tutorial notebooks
- * Added support for observing with TRK_LUN
+  * Updated for Python 3.10
+  * Cleaned up the naming of the tutorial notebooks
+  * Added support for observing with TRK_LUN
 
 2.1.5
- * Fixed a few more problems with the new cache management
- * Fixed a problem in lsl.sim.vis.add_baseline_noise() that caused the SEFD values to be ignored in some cases
- * Various clean ups
+  * Fixed a few more problems with the new cache management
+  * Fixed a problem in lsl.sim.vis.add_baseline_noise() that caused the SEFD values to be ignored in some cases
+  * Various clean ups
 
 2.1.4
- * Fixed a couple of problems with the new cache management
- * Fixed a couple of problems with the new file locking
- * Fixed a couple of problems with the lsl.testing.SilentVerbose class
- * More cleanup in the test suite
+  * Fixed a couple of problems with the new cache management
+  * Fixed a couple of problems with the new file locking
+  * Fixed a couple of problems with the lsl.testing.SilentVerbose class
+  * More cleanup in the test suite
 
 2.1.3
   * Added support for adding/subtracting timedelta instances from FrameTimestamp instances

README.md

@@ -27,7 +27,6 @@ REQUIREMENTS
  * pyephem >= 3.7.5.3
  * aipy >= 3.0.1
  * pytz >= 2012c
- * unlzw >= 0.1.1
  * matplotlib >= 1.1 (required for some of the scripts)
  * BeautifulSoup (required for some of the scripts)
  * casacore (required for measurement set support)
@@ -40,13 +39,13 @@ Install LSL by running:
 
 	pip install . [--root=<prefix>|--user]
 
-If the '--root' option is not provided, then the installation tree root directory is the same as for the Python interpreter used to run `setup.py`.  For instance, if the Python interpreter is in '/usr/local/bin/python', then '<prefix>' will be set to '/usr/local'.  Otherwise, the explicit <prefix> value is taken from the command line option.  The package will install files in the following locations:
+If the '--root' option is not provided, then the installation tree root directory is the same as for the Python interpreter used to run `setup.py`.  For instance, if the Python interpreter is in '/usr/local/bin/python', then '<prefix>' will be set to '/usr/local'.  Otherwise, the explicit \<prefix> value is taken from the command line option.  The package will install files in the following locations:
  * <prefix>/bin
  * <prefix>/lib/python3.6/site-packages
  * <prefix>/share/doc
  * <prefix>/share/install
 
-If an alternate '<prefix>' value is provided, you should set the PATH environment to include directory '<prefix>/bin' and the PYTHONPATH environment to include directory '<prefix>/lib/python3.6/site-packages'.
+If an alternate '\<prefix>' value is provided, you should set the PATH environment to include directory '\<prefix>/bin' and the PYTHONPATH environment to include directory '\<prefix>/lib/python3.6/site-packages'.
 
 If the '--user' option is provided, then then installation tree root directory will be in the current user's home directory.
 

VERSION

@@ -1 +1 @@
-2.1.7
+2.1.8

docs/.readthedocs.yaml

@@ -0,0 +1,17 @@
+version: 2
+
+build:
+  os: ubuntu-22.04
+  tools:
+    python: "mambaforge-22.9"
+
+conda:
+  environment: docs/.rtd_environment.yaml
+
+sphinx:
+  configuration: docs/source/conf.py
+
+python:
+  install:
+    - method: pip
+      path: .

docs/notebooks/DR Spectrometer Tutorial.ipynb

@@ -382,6 +382,15 @@
     "ax.legend(loc=0)\n",
     "plt.show()"
    ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# Additional Info",
+    "\n",
+    "In addition to the modules provided in LSL there are also a variety of scripts in the \"DRX/HDF5\" directory of the [commissioning repo](https://github.com/lwa-project/commissioning) to analyze data.\n"
+   ]
   }
  ],
  "metadata": {

docs/notebooks/DRX Tutorial.ipynb

@@ -420,6 +420,15 @@
    "source": [
     "Note that the shape of the output spectra is 3-D in the case of StokesMaster as opposed to 2-D for SpecMaster.  The extra dimension stores the four Stokes parameters in the other of I, Q, U, and V."
    ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# Additional Info",
+    "\n",
+    "In addition to the modules provided in LSL there are also a variety of scripts in the [commissioning repo](https://github.com/lwa-project/commissioning) to analyze data.\n"
+   ]
   }
  ],
  "metadata": {

docs/notebooks/LDP Tutorial.ipynb

@@ -607,6 +607,15 @@
     "except Exception as e:\n",
     "    print(\"ERROR: %s\" % str(e))"
    ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# Additional Info",
+    "\n",
+    "In addition to the modules provided in LSL there are also a variety of scripts in the [commissioning repo](https://github.com/lwa-project/commissioning) to analyze data.\n"
+   ]
   }
  ],
  "metadata": {

docs/notebooks/SDF Generation.ipynb

@@ -315,6 +315,15 @@
    "source": [
     "The biggest difference between making IDFs and making SDFs is the terminology.  Instead of Sessions there are Runs and instead of Observations there are scans."
    ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# Additional Info",
+    "\n",
+    "In addition to the modules provided in LSL there are also a variety of scripts in the [session schedules repo](https://github.com/lwa-project/session_schedules) to help build schedules.\n"
+   ]
   }
  ],
  "metadata": {

docs/notebooks/TBW & TBN Tutorial.ipynb

@@ -552,6 +552,15 @@
     "beamdata = beamformer.phase_and_sum(antennas, iq_data, sample_rate=1e5, azimuth=10.9, elevation=83.2)\n",
     "print(beamdata.shape)"
    ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# Additional Info",
+    "\n",
+    "In addition to the modules provided in LSL there are also a variety of scripts in the [commissioning repo](https://github.com/lwa-project/commissioning) to analyze data.\n"
+   ]
   }
  ],
  "metadata": {