Support AGASC 1.8 and fix tests and modernize

[taldcroft]

Description

This PR makes a number of improvements for supporting AGASC 1.8.

NOTE: this has a post-install activity like:

cd /proj/sot/ska/data/find_attitude
mv distances.h5 distances_1p7.h5
# Copy distances_1p8.h5 from development to this directory if not already there
ln -s distances_1p8.h5 distances.h5

Fix tests

As noted in #29, one existing test was failing when run using AGASC 1.8 with the legacy (1.7) distances.h5 file. The root cause was that AGASC 1.8 allows ASPQ1=0 for very close stars. That test case included such a close pair, and so sometimes the star identification got the wrong one of the pair and did not match expectation.

This is fixed with a new test helper function to remove close pairs from test cases. In real operations the ACA would get a "star consistent with either one of the pair and it does not matter which one is identified for the attitude solution.

Also:

• Used the miniagasc instead of the default proseco_agasc to get stars for testing.
• In random testing, also randomly select the number of stars between 4 to 8. Previously it always used 8 stars.
• Open up the roll tolerance from 40 to 55 arcsec for attitude checking.
• Change one test attitude to avoid a close star pair. (Probably not necessary with the new code but it should be OK).

Make a new distances file and update make_distances.py

• The file is now called distances_1p8.h5. The plan is to rename the legacy file on HEAD to distances_1p7.h5 and then make a link distances.h5 -> distances_1p8.h5. This is up for discussion if anyone has a better idea.
• Change the faint magnitude limit for pairs from 10.5 mag to 10.0 mag. It seems very unlikely that the ACA will find bona-fide stars fainter than 10.0 mag. This reduces the number of pairs by almost a factor of 4.
• The distances file now includes a number of HDF5 attributes that provide the key parameters to describe how the file was created.
• make_distances.py was moved into the package and given a couple of command line arguments.
• Cleaned up a bit, e.g. no longer creating a separate microagasc.fits file for caching since on-the-fly processing takes only a few seconds.
• Use astropy search_around_sky to find pairs instead of the previous ad-hoc technique. This cleaned up a lot of code (with the slight downside that after renaming, GitHub sees this as a completely new file).

Modernize find_attitude.py

• Use ska_helpers.logging.basic_logger instead of pyyaks.logger
• Require networkx to be installed. I don't understand why the import was semi-optional before since the downstream code will fail without networkx.
• Require SKA env var and get rid of defunct ska_path.
• Better formatting of output summary table.

Interface impacts

Testing

Unit tests

• Mac

Make distances_1p8.h5

(ska3) ➜  find_attitude git:(agasc1p8) env AGASC_DIR=/Users/aldcroft/ska/data/agasc/rc python -m find_attitude.make_distances --overwrite
Reading miniagasc /Users/aldcroft/ska/data/agasc/rc/miniagasc_1p8.h5
Finding star pairs
Sorting by dists
Writing data to distances_1p8.h5
Creating index (takes a minute or two)

Unit tests with AGASC 1.8 and distances_1p8.h5

Note the 20 seconds for running tests vs. 57 using the legacy 1.7 distances.h5 below.

(ska3) ➜  find_attitude git:(agasc1p8) git rev-parse HEAD 
4c931b835c8bab84fa71ba72deaeed2bb267996f
(ska3) ➜  find_attitude git:(agasc1p8) env AGASC_PAIRS_FILE=distances_1p8.h5 pytest
===================================================== test session starts =====================================================
platform darwin -- Python 3.11.8, pytest-7.4.4, pluggy-1.4.0
rootdir: /Users/aldcroft/git
configfile: pytest.ini
plugins: timeout-2.2.0, anyio-4.3.0
collected 6 items                                                                                                             

find_attitude/tests/test_find_attitude.py ......                                                                        [100%]

===================================================== 6 passed in 20.71s ======================================================

Unit test with AGASC 1.8 and $SKA/data/find_attitude/distances.h5 (1.7)

(ska3) ➜  find_attitude git:(agasc1p8) pytest                                      
===================================================== test session starts =====================================================
platform darwin -- Python 3.11.8, pytest-7.4.4, pluggy-1.4.0
rootdir: /Users/aldcroft/git
configfile: pytest.ini
plugins: timeout-2.2.0, anyio-4.3.0
collected 6 items                                                                                                             

find_attitude/tests/test_find_attitude.py ......                                                                        [100%]

===================================================== 6 passed in 56.00s ======================================================

Pseudo-random attitudes using AGASC 1.8 and distances_1p8.h5

Finding attitude for 100 random-but-reproducible attitudes. This ran with no failures.

env AGASC_PAIRS_FILE=distances_1p8.h5 AGASC_DIR=/Users/aldcroft/ska/data/agasc/rc \
  python -c 'from find_attitude.tests.test_find_attitude import _test_random; _test_random(n_iter=100)'

Independent check of unit tests by Jean

• Linux

ska3-jeanconn-fido> git rev-parse HEAD
4c931b835c8bab84fa71ba72deaeed2bb267996f
ska3-jeanconn-fido> pytest
======================================== test session starts ========================================
platform linux -- Python 3.11.8, pytest-7.4.4, pluggy-1.4.0
rootdir: /proj/sot/ska/jeanproj/git
configfile: pytest.ini
plugins: anyio-4.3.0, timeout-2.2.0
collected 6 items                                                                                   

find_attitude/tests/test_find_attitude.py ......                                              [100%]

=================================== 6 passed in 156.29s (0:02:36

Functional tests

No functional testing.

[taldcroft]

I'll fix the ruff issues in a followup PR.

[taldcroft]

I updated the description with a post-install activity to use the 1.8 distances. However, even with the legacy 1.7 distances the new code will function correctly.

[jeanconn]

Regarding: "Change the faint magnitude limit for pairs from 10.5 mag to 10.0 mag" - this seems fine. I note technically the default faint limit for the old default FFS looks to be 10.19, but it seems unlikely we'll get stars between 10 and 10.19 and we may change the default FFS.